GPT-5.6 ARC-AGI-3 Score Tripled: API Cost Impact

OpenAI has shown that an agent’s harness can matter almost as much as its model. In a July 29 experiment, the company changed two Responses API settings for GPT-5.6 Sol: it retained private reasoning between game actions and replaced rolling context truncation with compaction.

On the public ARC-AGI-3 task set, the score rose from 13.3% to 38.3% Relative Human Action Efficiency (RHAE), about a 2.9-fold improvement. Output-token use fell by roughly 6x at the same time. OpenAI estimates that the average human tester scored 48%.

This is not an API price cut. The live table above reads current rates from our pricing dataset. The economic change is potentially more important: a better-configured agent can finish more work while generating far fewer billed output tokens.

What OpenAI changed

ARC-AGI-3 asks agents to discover the rules of unfamiliar 2D games without instructions. The official generic harness discarded the model’s private reasoning after every action. Once the interaction exceeded its rolling limit, it also removed the oldest observations and actions.

That forced GPT-5.6 Sol to repeatedly reconstruct its plan while gradually losing evidence about what it had already tried.

OpenAI rebuilt the harness with the Responses API. Passing the previous response ID preserved reasoning across turns, while compaction summarized long histories instead of simply deleting the oldest entries.

The thresholds were similar but not identical: the generic harness started dropping old history after 175,000 characters, while the Responses API harness compacted at 175,000 tokens. OpenAI says the ARC grids tokenize at roughly one token per character, making the limits broadly comparable.

ARC-AGI-3 setup	Official harness	Retained reasoning + compaction
Public-set RHAE score	13.3%	38.3%
Relative output-token use	1x baseline	About 0.17x
Long-run memory	Oldest history removed	Important history compacted
Private reasoning between actions	Discarded	Retained

The result does not prove that every workload will triple its score or cut output by the same amount. It does show that a generic evaluation harness can badly understate a model designed to maintain reasoning state.

Pricing impact

OpenAI did not announce new token rates with this experiment. Sol, Terra, and Luna remain at the rates rendered from pricing.json above.

The relevant buying metric is now cost per successful task, not price per million tokens or benchmark score alone. If an agent reaches the answer with one-sixth of the output while succeeding more often, fewer retries and shorter reasoning traces can outweigh the nominal rate.

There is an important billing boundary. OpenAI bills reasoning tokens as output tokens, so using one-sixth as many output tokens implies that the output-token component was about 83% lower at the same rate. It does not prove that the whole run was 83% cheaper. When responses are chained with previous_response_id, earlier inputs in the chain are still billed as input, and OpenAI did not publish the full input, cached-input, cache-write, tool, compaction, or dollar ledger for both harnesses.

The experiment’s 175,000-token compaction point also stayed below GPT-5.6’s long-context pricing threshold. OpenAI says prompts above 272,000 input tokens are charged at 2x the standard input rate and 1.5x the standard output rate for the full request.

This also changes cross-provider evaluations. A test that preserves one model’s native state but strips another model’s reasoning is not measuring comparable production setups. Teams comparing the OpenAI pricing page with Anthropic pricing or Google AI pricing should give each model the context-management features its provider recommends, then measure the entire run.

Who wins—and who loses

Developers running long-horizon coding, browser, research, and operations agents benefit most. These workflows contain many turns, repeated tool calls, and histories long enough for compaction to matter.

OpenAI also benefits because the result supports its argument that benchmarks should use provider-optimized harnesses. ARC-AGI-3 still benefits by exposing a real weakness: without persistent state, even a frontier model can appear unable to learn.

Generic leaderboards lose some authority. A single score may reflect context policy, prompt design, tool plumbing, and reasoning retention as much as raw model capability. Buyers who route expensive production traffic from headline scores alone carry the largest risk.

What API teams should do

For new agent workflows, start with the Responses API rather than legacy Chat Completions. Preserve the previous response across tool calls and enable compaction before histories approach the context limit.

Then rerun a production-shaped evaluation and record:

• accepted-task rate;
• total input, cached input, and output tokens;
• retries and fallback calls;
• wall-clock time; and
• human review or correction time.

Compare the final cost per accepted result in the token calculator. If Sol passes but a lower-cost route also clears the quality bar, use Sol as an escalation model instead of the default.

For more context on the same model family, read our GPT-5.6 efficiency and pricing analysis.

Labs availability and blocker

AI Pricing Guru Labs already includes GPT-5.6 Sol, Terra, and Luna in its 49-task cost-per-correct-answer suite. That confirms public routed availability for all three models, but it does not reproduce OpenAI’s ARC-AGI-3 result.

Our current suite is deterministic and single-turn through OpenRouter chat routes. It cannot test first-party Responses API state, previous_response_id, retained private reasoning, server-side compaction, or the ARC game environment. A valid follow-up needs the public ARC games, fixed settings for both harnesses, first-party Responses API access, and complete token and spend logs. Until those conditions are met, the 13.3%-to-38.3% result remains an official OpenAI claim rather than an independent Labs finding.

Bottom line

OpenAI’s ARC-AGI-3 result is a harness-efficiency story, not a rate-card change. Retained reasoning and compaction raised GPT-5.6 Sol’s public-set score from 13.3% to 38.3% while cutting output tokens roughly sixfold.

The practical lesson is straightforward: benchmark the model and its production configuration together. A cheaper-looking generic run can be more expensive if forgotten reasoning, truncated history, and retries waste most of the tokens.

Sources: OpenAI, How enabling two settings tripled our scores on the ARC-AGI-3 benchmark, Compaction, Reasoning models, Conversation state, GPT-5.6 Sol model page, and API pricing. Published and verified July 29, 2026.