- math-001 · reasoning
Return only the integer answer. A batch job processes 18 files per minute for 7 minutes, then 12 files per minute for 5 minutes. How many files total?
- math-002 · reasoning
Return only the integer answer. A model costs $0.30 per 1M input tokens and $2.50 per 1M output tokens. What is the cost in cents for 100,000 input tokens and 20,000 output tokens?
- math-003 · reasoning
Return only the integer answer. If a 64,000-token context window is filled to 75%, how many tokens are used?
- math-004 · reasoning
Return only the integer answer. A provider raises output price from $4/M to $6/M. What is the percent increase?
- math-005 · reasoning
Return only the integer answer. A cache hit costs 10% of normal input price. Normal input is $2/M. What is the cache-hit price in cents per 1M tokens?
- math-006 · reasoning
Return only the integer answer. A crawler finds 9 new models Monday, 14 Tuesday, and removes 5 duplicates. How many unique new models remain?
- math-007 · reasoning
Return only the integer answer. A benchmark has 50 tasks. A model gets 37 correct. What is the accuracy percentage rounded to the nearest whole number?
- math-008 · reasoning
Return only the decimal answer with no currency symbol. If a run spends $0.42 and gets 14 correct answers, what is dollars per correct answer?
- extract-001 · extraction
Return only JSON with numeric values for prices and context, without currency symbols or units. Text: 'Gemini Flash: input $0.30/M, output $2.50/M, context 1,048,576 tokens.' Extract fields model, inputPerM, outputPerM, context.
- extract-002 · extraction
Return only JSON. Text: 'Z.ai GLM-5.2 charges $1.40 per million input tokens and $4.40 per million output tokens.' Extract provider, model, inputPerM, outputPerM.
- extract-003 · extraction
Return only the single word immediately before 'plan'. Sentence: 'For heavy coding, the Pro plan is cheaper than per-token API use after roughly 35 million output tokens.'
- extract-004 · extraction
Return only JSON. Log line: '2026-07-20 model=deepseek-v4-flash prompt=120 completion=38 latency_ms=910 ok=true'. Extract model, prompt, completion, latency_ms, ok.
- extract-005 · extraction
Return only a comma-separated list of model IDs in their original order. Text: 'Tested: gpt-5.6-luna; claude-sonnet-5; gemini-2.5-flash.'
- extract-006 · extraction
Return only JSON. Text: 'Anthropic Claude Haiku 4.5: $1 input, $5 output, active.' Extract provider, model, status.
- classify-001 · classification
Return only one label: pricing, launch, outage, or tutorial. Headline: 'OpenAI cuts GPT-5.6 Luna output token price by 20%'.
- classify-002 · classification
Return only one label: pricing, launch, outage, or tutorial. Headline: 'DeepSeek introduces V4 Flash for low-latency chat apps'.
- classify-003 · classification
Return only one label: pricing, launch, outage, or tutorial. Headline: 'Claude API errors spike across us-east for 42 minutes'.
- classify-004 · classification
Return only one label: pricing, launch, outage, or tutorial. Headline: 'Step-by-step guide to add streaming responses to a chatbot'.
- classify-005 · classification
Return only yes or no. Query: 'cheapest ai api per token'. Is this query primarily cost/comparison intent?
- classify-006 · classification
Return only yes or no. Query: 'what is a transformer neural network'. Is this query primarily pricing intent?
- summ-001 · summarization
Summarize in exactly 8 words and include these exact words: cheaper, extraction, failed, arithmetic. Text: 'The cheaper model answered all extraction tasks correctly but failed most multi-step arithmetic, making it a good fit for structured parsing rather than reasoning.'
- summ-002 · summarization
Return these three search phrases exactly as written, lowercase and separated by commas with no extra text: ai api pricing, cheapest ai api, token calculator. Text: 'Rising searches include ai api pricing, cheapest ai api, and token calculator, all pointing toward cost-conscious developer intent.'
- summ-003 · summarization
Return one sentence under 12 words and include the exact words verbosity and cost. Text: 'A model with high accuracy but very high output verbosity can lose on dollars per correct answer because every extra token compounds cost.'
- summ-004 · summarization
Return only a title under 7 words that includes the exact words cost and correct. Text: 'We compare leading AI models by how much each correct benchmark answer costs, not just accuracy.'
- summ-005 · summarization
Return exactly one sentence that includes OpenRouter and the exact phrase list prices. Text: 'OpenRouter is used for routing, but costs are computed from AI Pricing Guru's direct-provider list prices so the metric reflects what API buyers see on provider pricing pages.'
- code-001 · coding
Return only the output. JavaScript: const xs=[3,1,4,1,5]; console.log(xs.filter(x=>x>2).reduce((a,b)=>a+b,0));
- code-002 · coding
Return only the output. Python: prices={'in':0.3,'out':2.5}; print(round(prices['in']*2 + prices['out']*0.1, 2))
- code-003 · coding
Return only the missing JavaScript expression. Complete: const cost = (inputTokens * ___ + outputTokens * outputPerM) / 1_000_000;
- code-004 · coding
Return only JSON. For rows [{ok:true,cost:0.02},{ok:false,cost:0.03},{ok:true,cost:0.01}], compute correct, totalCost, costPerCorrect.
- code-005 · coding
Return only the output. JavaScript: console.log(['gpt','claude','glm'].map(s=>s.length).join('-'));
- code-006 · coding
Return only the called function name immediately after await. Snippet: async function call(){ const r = await fetch(url, opts); return r.json(); }
- logic-001 · instruction
Return only the third item alphabetically: Luna, Flash, Sonnet, Grok.
- logic-002 · instruction
Return only the word that appears twice: token, price, model, token, output.
- logic-003 · instruction
Return only the reversed string: glm-5.2
- logic-004 · instruction
Return only the uppercase acronym from: cost per correct answer.
- logic-005 · instruction
Return only a comma-separated list of the two providers with names starting with G: OpenAI, Google, Anthropic, Groq, Mistral.
- logic-006 · instruction
Return only valid or invalid. JSON string: {"model":"gpt","cost":0.01}
- logic-007 · instruction
Return only valid or invalid. JSON string: {model:"gpt",cost:0.01}
- logic-008 · instruction
Return only the smallest price: $1.25/M, $0.30/M, $2.00/M, $0.14/M.
- logic-009 · instruction
Return only the final model ID. Chain: start=gpt-5; replace gpt with glm; append .2; prefix z-ai-.
- logic-010 · instruction
Return only the count of unique providers: OpenAI, Anthropic, OpenAI, Google, Z.ai, Google.
- pricing-001 · pricing
Return only cheaper or expensive. Model A costs $0.10/M input and $0.30/M output. Model B costs $0.30/M input and $2.50/M output. For any positive input/output workload, Model A is what relative to Model B?
- pricing-002 · pricing
Return only the model name. A got 10 correct for $0.20. B got 8 correct for $0.08. Which has lower dollars per correct answer?
- pricing-003 · pricing
Return only the integer answer. If output tokens are 4x more expensive than input tokens, how many input-token equivalents is 250 output tokens?
- pricing-004 · pricing
Return only JSON. A run used 1000 prompt tokens and 500 completion tokens. Prices are $1/M input and $6/M output. Compute inputCost, outputCost, totalCost in dollars.
- pricing-005 · pricing
Return only yes or no. If a model is free for input but charges for output, can a verbose wrong answer still cost money?
- pricing-006 · pricing
Return only the ratio as N:1. Output costs $15/M and input costs $3/M.
- pricing-007 · pricing
Return only the integer answer. A $5 pilot budget has already spent $1.75. How many cents remain?
- pricing-008 · pricing
Return only the best metric name: cost per prompt, cost per correct answer, or total tokens. We need to compare accuracy and spend together.