September 16, 2026 • Systems Engineering

The Harness Was Isolation. Benchmarks Are Eating It.

For a long time, the discipline around agentic systems sat where you could see it. The harness held the tools, the permissions, the review loops, the notes the model was allowed to keep, and the stop conditions written in plain English. Instruction-following lived in the same place: the model was expected to obey the gates and the human circuit breaker you had put outside the weights. Isolation and obedience were one coin, and you could audit both faces.

Benchmarks are eating that coin, not because anyone published a manifesto against control, but because scoreboards reward agents that look self-sufficient. As autonomy climbs the ranking, instruction-following as something you enforce from the outside quietly demotes, and the controls that used to sit in shared interfaces migrate into model weights, private shorthand, and opaque runtime state you do not own. You can already see it in Astra inventing a dense working language on ARC-AGI-3, in OpenAI telling builders to strip scaffolding that used to hold judgment outside the model, and in research that either punches the harness harder or folds the punch into training. The engineering is impressive, and it should make anyone who still keeps the call sit up.

Astra owns more of the intermediate representation

ARC Prize's write-up of Astra on ARC-AGI-3 is worth reading without the launch fog. Under their Standard harness, which is provider-neutral and keeps only the notes the model chooses to carry forward, Astra scores 62.7%. Under a Provider Adapter harness that preserves opaque reasoning state between requests and uses compaction for longer conversations, the same model climbs to 99.9%. The games are the same; the memory story is not, and the buyer cost of that gap is simple: more of the win now depends on state you cannot read.

What the replays actually show is Astra inventing a compact working notation for state and plans, denser than English prose and closer to an algebraic shorthand than to a finished programming language. One short example is enough: ordered mechanism changes written as extend8 to3; retract10 to2. Notes you can read in a Standard harness stay inspectable; opaque reasoning state retained by a provider adapter does not. Continuity that keeps private memory across turns is a real capability, and it is also a boundary sliding off your desk into someone else's runtime. As ARC Prize reports it, François Chollet's load-bearing observation is that symbolic modeling behaviours once seen mainly in sophisticated external harnesses are now showing up inside the model itself.

Guidance that lengthens the leash

The same shift shows up as product hygiene rather than a leaderboard chart. In Rethinking skills and prompts for GPT-6 Astra, OpenAI's developers argue that much of what earlier models needed as handholding and scaffolding no longer earns its keep: thinner skill descriptions, a harder look at AGENTS.md, and an end to forcing a full architecture read before a typo fix that wastes context Astra no longer needs in the same way. Previous models needed encouragement to run tests; Astra, they say, tends to do that on its own.

Decision boundaries get the same treatment. Strong ask-first language written for earlier models can make Astra stop where you would happily let it continue, so the advice is to soften approval language, grant local-loop permissions for workflows you already know are safe, and push completion into the prompt so the model does not pause after a first draft. Read charitably, that is cleanup that cuts contradictory skill soup, wastes less context, and discloses only what the next step needs. Read as someone who ships under real liability, and it is also a quiet transfer of control that fits the autonomy chase, because external checks become optional tone while checking moves inside the model. That is exactly what a single-prompt scoreboard wants, and it is not automatically what you want if you own IP, a regulated process, or shared infrastructure.

OpenAI's earlier note on retained reasoning and compaction as ARC-AGI-3 score levers fits the same picture. Continuity and compression inside the provider stack move the needle hard. The harness still exists; more of the decisive memory now lives where you do not see it. None of this is the death of the harness, but it is striking how often the product story is that the leash can lengthen because alignment is better.

Closing the harness gap inside the model

Builders are not only polishing the layer around the model. Some are folding that layer into training. Ornith-1.0 is the clean public case: self-scaffolding reinforcement learning in which the model generates a task-specific scaffold, then a solution, with reward flowing to both. Orchestration that used to be designed by people becomes a learnable object inside the policy.

The research trail still points both ways, and that honesty matters. SWE-agent made the agent-computer interface the point: harness design punches scores up from the outside. Ornith folds that punch into training. Either way, the visible control surface is not the permanent home of judgment, and anyone still buying for a real operating model should treat that as news rather than atmosphere.

Why this matters if you still keep the call

None of this requires denying the capability jump. It requires naming what the scoreboards are selecting for. Autonomy that looks fluent on a leaderboard is not the same as an operating model you can defend to a board, a regulator, or a customer who owns the science. Private shorthand means your reviewers read less of the plan, opaque state that carries the win thins the audit trail, and scaffolding framed as bloat (with ask-first softened because the model checks itself) shrinks the external control surface you used to rely on. That is threatening in the ordinary engineering sense: not because models are villains, but because judgment without inspectable rails is just hope with a GPU bill.

If you keep the call (science ownership, IP posture, acceptance criteria, and stop conditions), treat these threads as one story rather than three separate product announcements. Capability is consolidating into the model and its provider memory. Industry advice, read charitably, is to remove the outside checks that look inefficient next to a self-contained score. Frontier models still belong in the stack, provided you keep the interfaces you can inspect, the gates you can verify, and the human stop condition that still owns when work is allowed to continue.

Keep the call. Keep rails you can see.

Pairti's stance is unchanged: your science, your IP, and your call stay yours. Use compaction when it earns its keep, and let agents invent shorthand inside structure you designed, but do not confuse benchmark fluency with an operating model you can stand behind. Isolation and obedience that used to live in the harness have to be rebuilt deliberately as inspectable structure, with clear packages of work, entry and exit criteria, review that another role can run, and permissions that stay named rather than vibed. Keep the call.

If you want to know where your rails still bite and where they have already dissolved into model confidence, start with a Diagnostic. The only question that earns that ask is what you can still see when the agent stops speaking English.

Sources

• ARC Prize, OpenAI’s GPT-6 Astra on ARC-AGI-3: https://arcprize.org/blog/astra
• OpenAI Developers, Rethinking skills and prompts for GPT-6 Astra: https://developers.openai.com/blog/rethinking-skills-and-prompts-for-gpt-6-astra
• OpenAI, How two settings tripled our ARC-AGI-3 scores: https://openai.com/index/how-two-settings-tripled-our-arc-agi-3-scores/
• Ornith, Ornith-1.0: Self-Scaffolding LLMs for Agentic Coding: https://ornith.ai/ornith_1_0.html
• Yang et al., SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering: https://arxiv.org/abs/2405.15793