Cross-harness memory replay blocker · Beacon
AI Pricing Guru Labs
Benchmark coverage notes
Not every model announcement, vendor benchmark, or product claim belongs in one deterministic text leaderboard. These decisions document what is measured, what stays separate, and what evidence would make a future comparison valid.
The inclusion rule
A result enters the main leaderboard only when the exact model or route can run the same published tasks with fixed settings, machine-graded outputs, complete token records, and a defensible current price basis. The linked analyses contain the evidence and current pricing context; the benchmark itself remains vendor-neutral.
Private-preview route and billing blocker · OpenAI on Google Cloud
No authorized Google Cloud private-preview project, pinned endpoint/model version, enabled region/quota, or verified Google Cloud rate and usage ledger. Existing exact-route Sol/Luna text scores are not Google Cloud latency, reliability, security or cost results; Astra remains unavailable on our maintained Labs route.
Read the evidence and buyer impact →Included Sol and Luna · Terminal X private investment-evaluation replay blocker
The exact Sol and Luna routes retain text-suite results. Terminal X reports 88.6% senior-reviewer agreement over 500 judgments, about 30% lower query cost and 25% more mandate-drift detections; none is a Labs result. No authorized private data room, full 54-criterion rubric, named comparator, fixed point-in-time documents, pinned retrieval/prompts/model versions, raw verdicts or complete billed token/tool/reviewer ledger is available. Require rights-cleared matched cases, independent qualified reviewers and repeated trials before an investment-workflow score.
Read the evidence and buyer impact →Signals-only model · authorized trace-analysis replay blocker · Laminar flow-1
No authorized Laminar Signals test project or pinned Signals harness, trace dataset and grading rubric. No public standalone inference endpoint or downloadable flow-1 weights verified. Require matched traces, prompts/schema, pinned model/harness, raw findings, repeat trials, independent grading and complete cloud/analysis cost evidence. The vendor 523-trace result and 23x claim are not Labs results.
Read the evidence and buyer impact →Included managed model · specialized vLLM GB300 hardware-replay blocker
Kimi K3 retains its managed-route text result. The merged opt-in vLLM backend has not been independently replayed: no authorized 8/16-GB300 allocation or pinned hardware harness is available. Require exact vLLM/weight commits, GPU and NVLink topology, precision, identical real prompts, same-hardware stock/optimized controls, prefill and decode timing, actual speculative acceptance, repeated correctness tests, and complete GPU-hour and operating-cost ledgers. Synthetic 842 tokens/s and 16-versus-8-GPU AgentX results are not Labs scores or measured API savings.
Read the evidence and buyer impact →Unreleased-model and pricing blocker · Microsoft MAI-Base-2
Still undergoing post-training; no public exact-model route, weights, token rate or authorized Labs access verified. Vendor-held-out BPB comparisons are not Labs results. A valid result requires authorized exact-model access, pinned weights or API snapshot, fixed tasks and settings, raw outputs, retries, token counts, latency, and complete serving or API costs.
Read the evidence and buyer impact →Astra-route, Amp/Puck voice and Ultrafast workflow replay blockers
No authorized pinned Amp/Puck/Ultrafast account, matched design/marketing/voice tasks, raw trajectories, quota/credit/token/orb ledger or accepted-output grader. The 49-task text suite cannot score full-duplex voice or Amp orchestration. Astra is unavailable on our maintained route; the existing exact GPT-6.1 Sol text result stays separate. Customer design/marketing examples and up-to-6× inference-speed claims are not Labs scores.
Read the evidence and buyer impact →Included Sol and Luna · Astra-route and multi-day research replay blockers · Third Moment Research
GPT-6.1 Sol and GPT-6 Luna have exact-route results in our 49-task text suite. Astra is unavailable on our maintained route. The suite cannot replay 10000-agent 36–48-hour finance research or Daybreak environment testing. Require authorized model/cyber access, pinned data and Docker/Codex/outer harness, fixed tasks and baselines, complete trajectories and billing-category/compute ledgers, repeated trials and independent correctness/accepted-investigation grading. No customer runtime or token-efficiency claim is imported as a Labs score.
Read the evidence and buyer impact →Beta access and robotics-harness blocker · Physical Intelligence Pi API
Batch-onboarded beta; AI Pricing Guru has no approved Pi API account, credentials, pinned hosted model ID or published billing unit/rate. The text suite cannot evaluate robot actions. A valid test needs an authorized robot or simulator, fixed task and policy versions, demonstrations, repeated trials, action/latency/failure traces, success grading and a complete training/inference cost ledger.
Read the evidence and buyer impact →Music-client scope and authorized-workflow replay blocker · Spotatui
No benchmarkable Spotatui model route or authorized listening-history/playback replay with pinned backend, token/quota/compute ledger and accepted-recommendation grader; text leaderboard cannot score music playback
Read the evidence and buyer impact →Long-video harness and authorized-route blocker · Higgsfield with GPT-6 Astra
The text suite cannot measure over-10-minute video continuity or future voice-to-movie, and the current Labs route lacks Astra access. Require authorized exact workflow access, pinned planning and render models, fixed scripts/assets and export quality, complete outputs/traces, token/tool and credit bills, failures, latency, editing time, blinded continuity/accuracy grading, and released voice access before a comparative score. No accepted-minute cost is disclosed.
Read the evidence and buyer impact →Policy behavior and product-route blocker · Claude abuse rule
Anthropic's November 12 Usage Policy rule is not a new model. Claude.ai and Claude Code can end rare persistently abusive chats, but the 49-task deterministic text suite does not test that behavior or API enforcement. A valid replay needs authorized scenarios, pinned product and policy versions, repeat traces, a narrow rubric, interruption counts, and full cost records; no policy-behavior score is inferred from the headline.
Read the evidence and buyer impact →Workflow-security replay blocker · Grok Bot Slack disclosure
A first-person incident report describes a Grok Bot finance report reaching the wrong Slack group. Our fixed text suite cannot replay a private cloud computer, shared credentials, bank and Slack connectors, lookalike destinations, or approval state. A valid study needs authorized synthetic accounts, pinned Bot and connector versions, action traces, repeated trials, leak grading, and full usage plus review-cost records. No incident frequency or benchmark score is inferred.
Read the evidence and buyer impact →Included exact Sol route · Asana browser-agent replay blocker
Our exact openai/gpt-6.1-sol route has a 49-task deterministic text result, but Asana’s 144-run browser study and 76× estimated model-cost claim are external. A fair replay needs the pinned catalog/browser build, anonymized Model B identity or an equivalent disclosed control, identical prompts and tools, cache markers, screenshot/history policies, raw traces, token and service-tier ledger, browser costs, timing, capped-run accounting, and independently graded completed tasks. Astra in Codex is the experimenter, not a scored browser-run result.
Read the evidence and buyer impact →Decision-model game-harness blocker · Jevman Pac-Man
Opper's 100-game-per-model leaderboard is a vendor-run structured-decision benchmark, not our Labs score. Jev's TypeSafe System One route and Opper's gateway routing need separate invoices. Our 49-task text suite cannot accept typed game actions; a fair replay requires a pinned game build, exact model routes and versions, identical state and prompts, latency and backup-move rules, raw responses, token/credit ledgers, and an accepted-action grader.
Read the evidence and buyer impact →Finance-harness and route blocker · Samaya earnings prediction
Samaya reports a 456-company, point-in-time finance result, not a Labs score. Our 49-task text suite lacks its licensed financial data, cutoff gate, retrieval, prompts, consensus baseline, scoring code, full traces, and usage ledger; Astra is unavailable on our exact benchmark route. No cost per accepted forecast can be inferred.
Read the evidence and buyer impact →Creative-campaign workflow and route blocker · Pollo AI with OpenAI
Pollo reports less time selecting models but publishes no pinned GPT-5.6 variant or GPT-Image-2.5 API route, creative task set, prompts, image and video outputs, accepted-asset rubric, baseline, traces, render bill, or complete cost ledger. Astra is unavailable on the maintained text route; that 49-task suite cannot measure campaign production or video quality. No customer claim is imported as a Labs score.
Read the evidence and buyer impact →ChatGPT-only UI and reproducibility blocker · GPT-6 Intelligent UI
OpenAI's October 7 Chat-tab rollout uses GPT-6 Sol on eligible paid tiers and Luna on Free and Go, but no public Intelligent UI API, pinned renderer, interface-task dataset, controls, accessibility grader, or usage ledger is announced. Existing exact-route Sol and Luna text scores are not UI scores; a fair replay needs matched accounts, fixed prompts, raw interactive outputs, timing, usability and correctness grading, and complete seat or API cost records.
Read the evidence and buyer impact →Unreleased-model and math-validation blocker · OpenAI frontier research
OpenAI's internal mathematics model has no public API route, ID, access date, or price. Its published mathematical claims are not our Labs scores. Reproduction needs authorized exact-model access, fixed problem statements and attempts, complete traces and compute bills, Lean artifacts and checker versions, and independent expert validation; existing Astra, Sol, and Luna text results remain separate.
Read the evidence and buyer impact →Enterprise-workflow replay and route blocker · Atlassian Rovo with OpenAI
The partnership is a product integration, not a new model or benchmark score. GPT-6.1 Sol and Luna retain their exact-route text results; the current Labs route lacks Astra access. A Rovo-versus-direct-agent study needs authorized Jira and Confluence data, matching permissions and tools, pinned model routes, Rovo credits or complete API bills, raw traces, retries, latency, human approvals, and accepted-action grading.
Read the evidence and buyer impact →Governance-scope blocker · Claude constitution and morality feature
Anthropic's constitution is a training-policy document, not a callable new model. Exact Claude routes retain their existing results, but our 49-task deterministic text suite cannot measure moral judgment or safety. A fair study needs a versioned policy, rights-cleared scenarios, blinded review, refusal and harm criteria, repeat traces, token bills, and reviewer-time cost.
Read the evidence and buyer impact →Included model · Claude Code workflow replay blocker · Opus 5.5
The exact Opus 5.5 model has a 49-task deterministic text result, but the new Claude Code guidance on long autonomous runs, steering, effort, fast mode, and fallback has no paired accepted-code cost result. A fair replay needs pinned Claude Code and model versions, identical starting commits and tasks, tools and permissions, cache condition, complete traces and model-switch notices, token bills, latency, tests, retries, and blinded review time.
Read the evidence and buyer impact →Open-weight serving and cost-route blocker · Aleph Alpha Kolibri-1
Kolibri is not in our maintained Labs roster: no funded exact callable route or comparable infrastructure-cost ledger is available. A fair 49-task replay requires a pinned checkpoint and plugin, GPU and precision, identical tasks, raw outputs, latency, failures, token counts, and allocated serving spend. Vendor benchmarks are not imported as our Labs results.
Read the evidence and buyer impact →Mixed inclusion and route blocker · GPT-6 family model guide
The exact GPT-6.1 Sol and GPT-6 Luna routes have maintained deterministic results, while the current Labs route is not authorized for GPT-6 Astra. OpenAI's coding, research, computer-use, and long-running-agent recommendations are not imported as Labs scores; a comparable family replay needs identical tasks, snapshots, prompts, reasoning settings, tools, retries, latency, complete token and billing records, and one acceptance rubric
Read the evidence and buyer impact →Physical-hardware and local-inference harness blocker · Raspberry Pi 4/5 2GB
The board price announcement is historical; current per-SKU price verification is separate, and a retail computer is not a callable model route. A comparable local-AI result needs a pinned board revision, memory, storage, cooling, power limits, operating system, model artifact and quantization, fixed tasks, warm-up policy, latency, correctness, energy, failure, and complete system-cost records
Read the evidence and buyer impact →Authorized-run and reproducibility blocker · Gemini 4 Argon / Google RSI
Public paid endpoint is documented, but no authorized funded Argon evaluation with this exact route, fixed reasoning level and full response/token/retry/latency/billing ledger has been completed by AI Pricing Guru. No paid calls initiated for this release. A valid 49-task result needs pinned reasoning/settings, accepted-task scores and the full evidence ledger. Google documents a stable paid endpoint; no missing-route claim remains. A recursive-self-improvement experiment additionally needs multi-generation controls, human-intervention logs and compute costs.
Read the evidence and buyer impact →Included exact route · launch-day 49-task result · GPT-6.1 Sol
The exact openai/gpt-6.1-sol route scored 49/49 with zero errors for $0.016892, or $0.00034473 per correct answer, on the current full-roster deterministic suite; OpenAI's coding, computer-use, science, and professional-work benchmarks remain outside that score
Read the evidence and buyer impact →Vision-harness and authorized-route blocker · GPT-6 Astra
Roboflow's image benchmark is independent workload evidence; our maintained suite is text-only and the current route lacks Astra access. A comparable replay needs the frozen image set and labels, exact resizing and coordinate formats, pinned model snapshots, prompts and reasoning settings, raw responses, complete token and billing records, latency, repeat trials, and detection, counting, reasoning, segmentation, and re-identification graders
Read the evidence and buyer impact →Unreleased-model and safety-replay blocker · GPT-6.1 Astra
There is no official public model ID, endpoint, pinned snapshot, system card, or reproducible evaluation package; a valid replay needs official access, exact tool and sandbox policy, deception and scope-authorization tasks, raw traces, tool-call and egress logs, complete token, tool, correction, and reviewer cost records, repeated runs, and independent safety review
Read the evidence and buyer impact →Accounting-workbook replay and route blocker · Basis with GPT-6 Astra
Astra is unavailable on the maintained route, and a fair replay needs authorized access, rights-cleared workbooks and sources, pinned model, reasoning, cache, and tool settings, raw traces, complete token and tool cost records, repeated runs, output diffs, reviewer-time measurement, and blinded qualified-accountant grading
Read the evidence and buyer impact →Internal-research scope · safety-control replay blocker · OpenAI tool-use pause
The paused models are unnamed internal research systems, while the public OpenAI status API reports normal operation; a defensible replay would require the exact model and sandbox, authorized DNS and egress tests, monitor traces, stop-path evidence, complete cost records, and independent security review
Read the evidence and buyer impact →Included models · sales-demo agent replay blocker · Proaction with Codex
Sol and Luna have model-level results, but the private Codex workflow lacks anonymized prospect briefs, pinned plans and models, usage traces, invoices, acceptance grading, sales cohorts, and security-review evidence
Read the evidence and buyer impact →Scope, provenance, and financial-validation blocker · Claude-built FIRE calculator
The calculator is deterministic local Python rather than a model route; a Claude development replay needs the original requirements, full transcript, exact model and tool versions, starting repository, complete token and retry ledger, human edits, tests, and independent software and financial review
Read the evidence and buyer impact →Enterprise-access and replay blocker · Airbnb with GPT-6 Astra
Astra is unavailable on the maintained route, and the single-user 3–4-pass anecdote lacks named controls, fixed documents, route settings, raw traces, complete bills, repeat trials, and blinded acceptance grading
Read the evidence and buyer impact →Video-editing agent replay and route blocker · invideo with GPT-6 Astra
Astra is unavailable on the maintained route, the text suite cannot reproduce tracking, grading, timeline tools, rendering, or visual review, and the 3× success claim lacks a baseline rate, clip set, raw passes, traces, bills, and blinded grading
Read the evidence and buyer impact →Legal-drafting replay and route blocker · Harvey with GPT-6 Astra
Astra is unavailable on the maintained route, and a fair legal replay needs rights-cleared matters, pinned retrieval, baseline models, complete traces and bills, privilege controls, and blinded qualified-lawyer grading
Read the evidence and buyer impact →Included models · voice-agent replay blocker · Ringg with OpenAI
GPT-5.6 Luna, Terra, and Sol have exact-route text results, but Ringg's 65% resolution and 90% model-cost claims need consented calls, pinned routing and speech stacks, business tools, transfer rules, raw traces, complete bills, independent grading, and repeat-contact follow-up
Read the evidence and buyer impact →Research-agent replay and route blocker · Parallel with GPT-6 Astra
Astra is unavailable on the maintained route, the 49-task text suite cannot reproduce multi-site research, and the 50% claims lack baseline models, fixed questions, traces, billing records, citation checks, and blinded report grading
Read the evidence and buyer impact →Included exact route · fresh 49-task result · Claude Opus 5.5
The exact anthropic/claude-opus-5.5 route scored 49/49 with zero errors for $0.05422; GPT-6 Sol also scored 49/49 for $0.015922, so Opus cost 3.4x more on this narrow deterministic suite
Read the evidence and buyer impact →Included exact routes · fresh 49-task results · GPT-6 Sol and Luna
Both exact OpenRouter routes scored 49/49 with zero errors; Luna cost $0.0013686 and Sol $0.015702 for the fixed suite, while OpenAI's broader vendor benchmarks remain outside that score
Read the evidence and buyer impact →Policy scope · no benchmarkable artifact · OpenAI third-party assessments
OpenAI proposes long-term independent review of safety cases, safeguards, capability evaluations, and misalignment incidents, but publishes no new model claim, callable route, fixed task set, traces, or grading artifact for the 49-task text leaderboard
Read the evidence and buyer impact →Validated answer · sealed-replay and route blocker · GPT-6 Astra Enigma break
An expert validated the MVUEH key and plaintext, but its now-public answer cannot be replayed cleanly; a fair Labs result needs exact Astra access, sealed unsolved messages, pinned tools, full traces and bills, cheaper controls, and independent grading
Read the evidence and buyer impact →Included models · incident-period availability boundary · Claude
Fable 5 and Opus 5 retain their fixed 49-task quality scores; the September 22 outage has no synchronized route, region, retry, status-code, latency, request-ID, or billed-usage dataset for a defensible uptime score
Read the evidence and buyer impact →Live first-party route · Labs transport blocker · Grok 4.7
The xAI API route is live, but the maintained OpenRouter benchmark transport did not list Grok 4.7 at the September 21 check; a comparable result needs the exact route, fixed 49-task run, complete responses, and billed spend evidence
Read the evidence and buyer impact →Task-contract and cost-measurement blocker · CUA-S1-FORMS
The open checkpoint scores structured form choices rather than text answers; a valid study needs pinned forms, documents, UI snapshots, driver and hardware plus action, outcome, latency, escalation, and total-cost grading
Read the evidence and buyer impact →Route and analytics-harness blocker · Hex with GPT-6 Astra
Astra is live, but our route lacks access and the story has no replayable dataset, fixed questions, traces, visualization grader, reviewer-time baseline, or usage ledger
Read the evidence and buyer impact →Platform-scope and availability blocker · Android 17 QPR1 API 37.1
The new framework surface is not an AI model route; a valid study needs pinned Pixel and AOSP-derived builds, cross-device app tasks, runtime checks, traces, failure grading, engineering time, and device coverage
Read the evidence and buyer impact →Speech-to-text harness blocker · Grok Voice Transcribe 2.0
The priced REST and streaming routes are live, but the deterministic text suite cannot measure transcription errors, diarization, endpointing, latency, correction effort, or cost per accepted transcript
Read the evidence and buyer impact →Agent-harness and cost-ledger blocker · AutoBot
The free harness is public, but its desktop benchmark needs pinned ChatGPT, model, Mac, task, trace, intervention, and usage evidence before cost-per-task ranking
Read the evidence and buyer impact →Included model + cyber-harness blocker · DeepSeek V4.1 Flash
V4.1 Flash is ranked 49/49 for deterministic text; Enclave's 11/11 hacking result needs repaired challenge versions, pinned containers, raw transcripts, path graders, and a complete cache/retry cost ledger
Read the evidence and buyer impact →Included Luna + controlled-route/replay blocker · Astra code review
Luna is ranked 49/49; Astra is unavailable on our route, and Entelligence's vendor-run PR result needs an independent, fully pinned code-review replay
Read the evidence and buyer impact →Route and availability blocker · Siri model replacement
The underlying Claude and GPT routes are ranked, but Apple's private Siri path has no public entitlement, reproducible contract, usage ledger, or bill
Read the evidence and buyer impact →Governance and availability blocker · frontier pacing
A voluntary slowdown debate and evaluator commitment provide no new callable model, route, task bundle, or billed run
Read the evidence and buyer impact →Policy and availability blocker · US model distillation
Garry Tan proposed a regime, not a callable student model, artifact, route, recipe, or cost ledger
Read the evidence and buyer impact →Preference-evidence blocker · default coding model
Ask HN names many defaults, but provides no controlled tasks, routes, token ledger, or acceptance grader
Read the evidence and buyer impact →Included model · local-hardware replay blocker
V4.1 Flash is ranked 49/49; the 16 GB M1 SSD-streaming result needs a separate controlled hardware harness
Read the evidence and buyer impact →Route and replay blocker · GPT-6 Astra self-testing agents
The API is live, but our route lacks access and OpenAI publishes no reproducible Perplexity or Devin task bundle
Read the evidence and buyer impact →Included exact route · DeepSeek V4.1 Flash
Fresh 49-task run scored 49/49; the old V4 Flash result was not relabeled
Read the evidence and buyer impact →Governance-scope blocker · Cami Clark/Anthropic profile
A recycled informal-influence story has no callable model artifact or reproducible performance claim
Read the evidence and buyer impact →Product and access blocker · ChatGPT for Financial Services
A custom-priced managed workspace cannot be replaced by an Astra API call
Read the evidence and buyer impact →Speech-harness blocker · GPT-Live-1
Full-duplex voice and backend delegation need an end-to-end audio benchmark
Read the evidence and buyer impact →Agent-harness blocker · Spotify Portal shunt
Claude and Gemini are ranked; the 90% context claim needs a paired accepted-task replay
Read the evidence and buyer impact →Included model · local-throughput replay blocker
Kimi K3 is ranked; Deltafin's four-SSD Mac result needs a hardware harness
Read the evidence and buyer impact →Included model · quantum-hardware replay blocker
Sol is ranked for text; the MIT qubit workflow is not reproducible from the published evidence
Read the evidence and buyer impact →Image-harness blocker · Falconsai NSFW classifier
The artifact is open; the current leaderboard measures the wrong task
Read the evidence and buyer impact →Scope blocker · Mistral €3B funding
Capital raised is not a callable model route
Read the evidence and buyer impact →Included model · migration replay boundary
Fable 5 is ranked; the case-study rewrite is not a Labs score
Read the evidence and buyer impact →Policy scope blocker · Mistral data training
A model benchmark cannot verify provider-side training use
Read the evidence and buyer impact →Included · GPT-5.6 Sol direct price cut
Fresh 49-task run uses OpenAI's new promotional rate
Read the evidence and buyer impact →Image-harness blocker · Grok Imagine retirement
The November 2 redirect cannot be scored in a text suite
Read the evidence and buyer impact →Router and training blocker · Experiential
A traffic-fitted gateway is not one model route
Read the evidence and buyer impact →Availability and physical-harness blocker · Model Hardware Standard
MHS is a hardware specification, not a model route
Read the evidence and buyer impact →Included models with creative-writing replay blocker · Gemini, Claude, OpenAI
Student preference is outside the deterministic 49-task score
Read the evidence and buyer impact →Included with Kiro-harness blocker · GPT-5.6 family
The models are ranked; Kiro's 82% claim is not reproduced
Read the evidence and buyer impact →Local-harness blocker · Junie Local Qwen3.6-27B
The managed base model is available; JetBrains' local package is not comparable here
Read the evidence and buyer impact →Included with route-price boundary · GPT-5.6 Sol
Sol stays ranked at the canonical direct rate
Read the evidence and buyer impact →Included with visual-harness blocker · GPT-5.6 Sol
Sol is ranked for text; Roboflow's image scores stay separate
Read the evidence and buyer impact →Included · DeepSeek V4 peak/off-peak pricing
Existing results stay valid; cost recalculation waits for the effective rate
Read the evidence and buyer impact →Included with evidence boundary · GPT-5.6 builder guidance
The models stay ranked; provider-run harness claims stay separate
Read the evidence and buyer impact →Security-harness blocker · Claude Code Auto mode
Opus 5 stays ranked; a targeted bypass cannot become a text-benchmark score
Read the evidence and buyer impact →Speech-harness blocker · Nari Qwen3-TTS
The artifacts are open; the current leaderboard measures the wrong task
Read the evidence and buyer impact →Harness blocker · Vomit for Claude Code
Claude is ranked; the local rewrite layer needs paired fidelity testing
Read the evidence and buyer impact →Harness blocker · Graft for Claude Code
The model is ranked; the context layer needs a paired repository study
Read the evidence and buyer impact →Availability blocker · Z.ai Ox Alpha
The preview route is live; its final identity and durable price are not
Read the evidence and buyer impact →Availability blocker · Qwen3.8-Flash
The price is official; the endpoint is not live yet
Read the evidence and buyer impact →Route blocker · Qwen3.8-27B on Cerebras
The API price is live; the 1,500 tokens/s claim awaits a comparable Labs run
Read the evidence and buyer impact →Service-tier blocker · GPT-5.6 Sol Ultrafast
The model is already ranked; the preview tier is not reproducible yet
Read the evidence and buyer impact →Controlled-route blocker · GPT-6 Astra
The API is live; AI Pricing Guru's current Labs route is not authorized for Astra
Read the evidence and buyer impact →Availability blocker · GPT-5.6 Cyber
Daybreak Red is controlled access, not a public benchmark route
Read the evidence and buyer impact →Availability blocker · GPT-Realtime-2.1
Realtime voice needs a different benchmark harness
Read the evidence and buyer impact →Availability blocker · Meta's next open-source model
There is no model artifact to benchmark yet
Read the evidence and buyer impact →Evaluation blocker · Claude content marking
The announced detection API is not available for a reproducible test
Read the evidence and buyer impact →Scope blocker · Claude output-training policy
A terms boundary is not a model-performance result
Read the evidence and buyer impact →Scope blocker · Claude Code plan economics
Labs can price the models, but not reproduce a private seat allowance
Read the evidence and buyer impact →Rostered · Grok 4.6 result pending
The route is available; the funded refresh gate is not
Read the evidence and buyer impact →Availability blocker · Grok Bot
A managed cloud teammate cannot be substituted with a Grok API call
Read the evidence and buyer impact →
Coverage notes are not benchmark scores. When a model becomes callable or a reproducible artifact appears, we rerun or reprice the relevant experiment and update the main Labs leaderboard.