GPT-5.6 Sol Vision Benchmark: Cost Impact

Roboflow’s GPT-5.6 vision benchmark reached Hacker News on August 17, but this is not a new OpenAI model launch. Roboflow published the test on July 16 after evaluating Sol, Terra, Luna, and GPT-5.5 on object detection, counting, OCR, and targeted text extraction.

The result is narrower than the viral headline: Sol was the best OpenAI model Roboflow tested, while Gemini 3.5 Flash remained the practical leader for high-volume detection and counting. OpenAI’s official model documentation confirms that Sol accepts image input. Our chart, calculator, and pricing table use the current August 17 dataset.

What Roboflow measured

Sol’s biggest improvement was object detection. Its score rose by 32.4 mAP points over GPT-5.5, while counting accuracy improved by 8.1 percentage points. OCR and targeted extraction did not improve.

Metric	GPT-5.5	GPT-5.6 Sol	Sol change
Object detection, mAP@50	13.8	46.2	+32.4
Object counting accuracy	64.9%	73.0%	+8.1 points
OCR mean similarity	91.2%	90.7%	-0.5 points
Text extraction similarity	87.6%	82.5%	-5.1 points

The cheaper GPT-5.6 tiers captured much of Sol’s detection gain:

Metric	Luna	Terra	Sol
Object detection, mAP@50	43.3	44.7	46.2
Object counting accuracy	66.2%	67.6%	73.0%
OCR mean similarity	88.4%	88.8%	90.7%
Text extraction similarity	81.4%	79.4%	82.5%

The “best vision model” headline needs a qualifier

Roboflow calls Sol OpenAI’s best vision model to date. It does not call Sol the overall benchmark winner. The report says Gemini 3.5 Flash still led detection and counting and offered a better cost profile for large image batches.

That competitor is now marked legacy in our dataset, so a new production evaluation should include its active successor, shown in the live comparison above. Compare the current OpenAI pricing page and Google AI pricing page rather than treating a July benchmark as today’s complete buying guide.

Pricing impact after OpenAI’s July cuts

Roboflow’s per-image cost observations predate OpenAI’s July 30 reductions for Terra and Luna. They are useful benchmark traces, not current list-price guarantees.

Model	Reported cost vs Sol	Average latency	Current context
GPT-5.6 Sol	1.00x	About 10 sec	Standard rate unchanged
GPT-5.6 Terra	About 0.40x	About 6 sec	Token rates later fell 20%
GPT-5.6 Luna	Under 0.20x	Just over 5 sec	Token rates later fell 80%
Gemini 3.5 Flash	About 0.32x	Close to Luna	Legacy model in current data

Roboflow did not publish the input, image, reasoning, and output token ledger for each row, so the old per-image figures cannot be repriced exactly. The direction is clear: Sol’s benchmark economics are broadly unchanged, while Terra and especially Luna should now cost less for the same token trace. See our GPT-5.6 price-cut analysis for the current tier changes.

Why production cost can differ sharply

Prompt format changed detection quality by roughly 15 mAP points in Roboflow’s tests. GPT-5.6 worked best with absolute XYXY pixel coordinates, while the Gemini configuration used normalized YXYX coordinates. A generic prompt can therefore make the same model look worse and trigger expensive retries.

Roboflow also observed unstable Sol boxes on images around 2,000 × 2,000 pixels or larger at lower reasoning effort. Higher effort improved stability but raised token use, latency, and cost. Resizing or cropping before inference is the practical first fix.

The report previews an upcoming benchmark rather than publishing a complete reproduction package. It does not provide the full image set, sample count by task, prompts, model settings, raw outputs, or token ledger. Treat the numbers as a strong pilot signal, not a universal ranking.

What vision teams should do

1. Test Sol, Terra, Luna, and an active Gemini model on the same private images.
2. Lock coordinate format, reasoning effort, resolution, and output schema before comparing results.
3. Measure cost per accepted image after retries—not only cost per first response.
4. Route difficult scenes to Sol, then test whether Terra or Luna can handle the easier majority.
5. Use the token cost calculator for budget ranges, but record actual multimodal usage from production traces.

Bottom line

GPT-5.6 Sol is a real step forward for OpenAI vision, especially for detection and counting. The benchmark does not show a clean sweep: GPT-5.5 remained slightly better on OCR and materially better on targeted extraction, while Gemini 3.5 Flash led the report’s key detection and counting comparisons.

For buyers, Luna may be the most interesting result. It approached Terra and Sol on detection, ran fastest within the GPT-5.6 family, and received the largest subsequent price cut. Reserve Sol for images where its extra counting accuracy or reasoning quality changes the accepted result.

Sources: Roboflow’s GPT-5.6 vision benchmark, the Hacker News discussion, OpenAI’s GPT-5.6 model documentation, and OpenAI’s price-performance update. Verified August 17, 2026.