Manifest Drops Its LLM Router: Pricing Impact

Manifest says it no longer believes prompt-based model routing saves money for most of the workloads it observed. The inference gateway company will shut down its router on September 1, 2026, after operating it for four months across 7,000 cloud users.

That is a useful warning, not a universal verdict. Manifest did not publish a controlled benchmark, invoice reduction, quality score, or distribution of routed requests. Buyers should treat the post as production evidence that a routing layer has costs beyond its model bill—not as proof that all routing is uneconomic.

The same-day release of DeepSeek V4 Flash 0731 sharpens the question: when capable default models are already inexpensive and heavily cached, a router has less room to earn back its own complexity. Compare current options on the DeepSeek pricing page, OpenAI pricing page, and token cost calculator.

What Manifest deprecated

Manifest launched its LLM router in March and deprecated it in June. The system classified each incoming prompt as simple, standard, complex, or reasoning, then selected a corresponding model tier.

The company identified three economic problems:

Problem	Why it matters for cost	Better control
Prompt-only classification	The real difficulty may appear only after repository inspection, search, or tool calls	Route by known workflow or task type
Model switching	Repeated prefixes can lose their cache affinity when traffic moves between providers	Keep sessions sticky to one model
Behavioral variation	Evals, prompts, tracing, and failure handling multiply across models	Standardize a default and explicit fallbacks

Manifest’s example is a request to improve tests in a repository. The prompt looks identical whether the target is a small static site or the Linux kernel; the work discovered after the first tool call is radically different. A router that sees only the opening request cannot reliably price that hidden complexity.

Why caching can beat routing

Manifest argues that prefix caching often removes more cost than switching models. System prompts and conversation history are repeated prefixes, so keeping a working session on one provider can preserve discounted cache reads.

This is especially relevant to DeepSeek V4 Flash 0731. Its stable API route upgraded in place on July 31, and the live table above shows both regular and cached-input pricing. Existing integrations can keep one model identity, regression-test the new snapshot, and retain cache-friendly traffic patterns without introducing a prompt classifier.

The buyer metric should be cost per accepted task. Include router calls, cache misses, retries, latency, human corrections, evaluation maintenance, and failures—not only the token rate of the model chosen for the first call.

Does this mean LLM routers are dead?

No. Manifest’s finding applies most directly to generic routers that infer complexity from an isolated prompt. Explicit routing can still work when the application already knows the job: extraction, moderation, code review, long-context analysis, or premium final verification.

That distinction also explains why the result does not invalidate vLLM’s micro-agent routing approach. A bounded workflow with known roles, budgets, and escalation rules is different from guessing task complexity before tools expose the work.

Routers are easiest to justify when requests are repetitive, outcomes are machine-scoreable, and cheap and premium models show a stable quality gap. They are hardest to justify inside long agent sessions where context, cache state, and required tools evolve over time.

There is measured counterevidence to Manifest’s experience. Experiential Labs’ World Model Optimizer reported 40.8% lower inference cost with a 0.87-point quality gain at its cost-saver setting. Its maximum-savings setting cut measured cost by 46.2% but lost 0.54 quality points. That test used one held-out cohort of 1,199 scenarios and nine relatively close models; the project warns that the cohort is favorable to routing. Our World Model Optimizer cost analysis covers the full methodology and limitations.

The two reports can both be true. Routing a known, repeatable extraction job is different from guessing the complexity of an open-ended agent session before it begins.

Labs status: router benchmark blocked

Manifest’s deprecated router is not included in the Cost-per-Task Labs leaderboard. It is not a separately priced model, has no stable public model ID for this routing policy, and is scheduled to shut down on September 1.

The current 49-task suite compares callable models under one deterministic harness. It cannot answer whether a classifier chooses the right model across a private mix of easy and hard tasks while preserving cache state and accounting for fallbacks.

A defensible router benchmark needs production-like traces, at least two candidate models, a fixed routing policy, cache telemetry, escalation accounting, and a single-model control. Until that harness exists, using a model leaderboard as evidence for or against routing would be misleading.

What developers should do now

1. Pick one battle-tested default using a replay set of real production tasks.
2. Preserve model affinity within sessions so cached prefixes remain useful.
3. Add routes only for task classes the application can identify before inference.
4. Use explicit premium escalation for failed checks or high-value review—not a vague complexity guess.
5. Compare the full system against the best single-model baseline every month.

Teams that want a managed open-model control group can benchmark Novita’s listed DeepSeek and open-model routes. Confirm the exact snapshot served, keep the same eval set, and include gateway overhead in the result.

Affiliate disclosure: AI Pricing Guru may earn a commission from the sponsored Novita link at no extra cost to you. It does not affect this analysis.

Bottom line

Manifest’s deprecation is a strong case against automatic prompt-complexity routing as a default cost feature. Its experience says the apparent token savings can return as cache loss, inconsistent behavior, and engineering overhead.

The practical answer is narrower than “never route”: prove one default model first, then introduce task-aware or failure-driven routes only where measured savings survive the full cost of the system.

Sources: Manifest’s router postmortem, its earlier September 1 deprecation notice, the Hacker News discussion, Experiential Labs’ routing methodology, DeepSeek’s official V4 Flash 0731 model card, and AI Pricing Guru’s live pricing dataset.