Blog · 2026-07-25 · Vynaris Team
Upfront routing misprices complexity: the re-route math and the 20% reuse rule
Request-level routers pick a model before they know task complexity. The re-route math, the 20% reuse rule, and why over-provisioning is the bigger leak.
A request-level router picks a model before it knows how hard the task is. When it guesses wrong, the bill splits two ways: over-provisioning an easy task to a frontier model, or under-provisioning a hard one to a cheap model that stalls and forces a re-route. On our coding-task shape at live prices verified 2026-07-25, over-provisioning wastes $0.32 per task and under-provisioning wastes $0.06. The expensive mistake is the one everyone ignores.
TL;DR
- On a 40k-input / 8k-output coding task, sending an easy job to Opus 5 instead of Haiku 4.5 costs $0.40 vs $0.08 — a $0.32 over-provision tax, 5.0x the cheap price. Under-provisioning a hard job (Haiku stalls halfway, then re-route to Opus 5) wastes $0.06. The over-provision leak is 5.3x larger per incident.
- A mid-task re-route can either restart from scratch or carry the stalled work forward. Carry-over beats a restart only when the premium model can reuse more than 20% of the abandoned output. That threshold equals the model's input/output price ratio (Opus 5: $5/$25 = 0.20), and it is the same 20% for every current Claude model.
- Across a realistic task population, a request-level router runs 21.9% over an oracle ($253.60 vs $208.00 per 1,000 tasks). 84% of that waste is over-provisioning easy tasks; only 16% is the re-route double-spend. Prices verified 2026-07-25.
The claim that started this
Kun Chen (@kunchenguid, 1,587 likes / 171 replies / 196k views, read 2026-07-25) argued that "any attempt to do model routing at request level ... is fundamentally flawed," because "the complexity of a task only reveals itself when you start working on it." His fix: let a smart model plan and delegate implementation, then monitor execution, so routing happens "at the boundary between agents and subagents, not per each LLM request."
The engineering argument is a cost argument in disguise, and no routing post we have published has priced it. This one does. We treat the quote as the hook, not evidence; every dollar below comes from a provider's live pricing page (captured 2026-07-25) or the arithmetic shown.
What a request-level router commits to
An upfront LLM router reads the request text, picks one model, and lives with that choice for the whole task. Two ways it can be wrong, each with its own cost shape:
- Over-provision. The task was easy, but the router sent it to a frontier model. You pay the premium rate on work a cheap model would have finished. Pure flat waste, paid every time.
- Under-provision. The task was hard, but the router sent it to a cheap model. The cheap model burns tokens, stalls, and something re-routes the job to a premium model. Now you pay twice: the wasted cheap attempt plus the premium finish.
The mispricing is the gap between what the router spends and what a perfect (oracle) router would spend. Both error types feed it, but they are not the same size, and the intuition that under-provisioning is the scary one is backwards.
The workload (edit these)
Assumption Value Note
---------------------- ----------------------------------------------------------------- ------------------------------------------------------------
Task input context `X` 40,000 [input tokens](https://vynaris.com/glossary/input-tokens) repo files, instructions, tool output at the handoff point
Task output `O` 8,000 [output tokens](https://vynaris.com/glossary/output-tokens) code plus reasoning to finish
Cheap model Haiku 4.5 at $1 / $5 per 1M the under-provision pick
Premium model Opus 5 at $5 / $25 per 1M the correct pick for hard tasks
Stall point `f` 0.5 fraction of output the cheap model emits before it re-routes
Reuse fraction the lever share of the stalled output the premium model keepsOne caveat we state up front: X is the effective input at the moment of handoff, not the total an agent loop processes. Real agent loops re-send context every turn and multiply input several times over; we modelled that separately in where an agent's tokens actually go. Folding the loop multiplier in raises every absolute number here but does not move the break-even, which is a ratio.
Base per-task costs on this shape, using cost-per-token times token counts:
- Haiku 4.5, whole task:
40,000 x $1 + 8,000 x $5per 1M =$0.04 + $0.04 = $0.08. - Opus 5, whole task:
40,000 x $5 + 8,000 x $25per 1M =$0.20 + $0.20 = $0.40. - Haiku 4.5 stalled at
f = 0.5:40,000 x $1 + 4,000 x $5per 1M =$0.04 + $0.02 = $0.06.
Failure mode 1: over-provisioning is the expensive one
An easy task routed to Opus 5 costs $0.40 where Haiku 4.5 would have cost $0.08. That is $0.32 of pure waste, 5.0x the correct price, and you pay it in full on every misrouted easy task. There is no cheap-first attempt to partially recover; the premium bill is the whole bill.
Compare the under-provision waste: $0.06, the cost of the abandoned Haiku attempt. Over-provisioning throws away 5.3x more money per incident than under-provisioning. The reason is structural. Output tokens cost 5x input on every Claude model, and over-provisioning pays the premium output rate on the entire task, while under-provisioning only wastes a cheap partial. Right-sizing the easy tasks down is worth far more than salvaging the hard ones.
Failure mode 2: the mid-task re-route, restart vs carry-over
When a cheap model stalls, the premium model can start two ways.
Full restart. The premium re-reads the context and redoes the task from scratch. It ignores the stalled work entirely. Cost: $0.40 (the whole Opus 5 task), on top of the $0.06 Haiku waste.
Carry-over. The premium reads the context plus the cheap model's partial output as fresh input, then writes only the part it cannot reuse. Because the handoff crosses models, the carried tokens are not a cache hit on the premium; they are billed at full input price. The premium pays extra input to read the stalled work, and saves output by not rewriting the reusable share.
Carry-over cost on Opus 5, X = 40k, O = 8k, f = 0.5, as the reusable fraction moves:
Reuse of stalled output Opus 5 carry-over Opus 5 restart Cheaper
----------------------- ----------------- -------------- ----------
0% $0.42 $0.40 restart
20% $0.40 $0.40 tie
40% $0.38 $0.40 carry-over
60% $0.36 $0.40 carry-over
80% $0.34 $0.40 carry-over
100% $0.32 $0.40 carry-overAt 0% reuse, carry-over is worse than a restart: you paid to re-read work you then rewrote anyway. The crossover is exact. Carry-over beats a restart only when reuse exceeds the premium model's input/output price ratio:
carry-over wins <=> reuse > input_price / output_price
Opus 5: $5 / $25 = 0.20 -> reuse must clear 20%Every current Claude model prices output at 5x input, so the threshold is 20% across Opus 5, Sonnet 5, and Haiku 4.5. A model with a narrower gap (output only twice input) would need 50% reuse before carry-over paid off. The rule is model-specific, and it is the single number to check before wiring up any "resume the stalled task" path.
Put your own task shape and stall point into the calculator to find where your re-route path breaks even.
The population mispricing, and where it hides
One task tells you the incident costs; a task population tells you the bill. Take a mix where 60% of tasks are easy (Haiku suffices) and 40% are hard (need Opus 5). Give the router a realistic error profile: it over-provisions 20% of easy tasks and under-provisions 30% of hard ones, with a full-restart re-route.
Router Cost per 1,000 tasks Vs oracle
------------------------------------ -------------------- ----------------
Oracle (always right) $208.00 baseline
Upfront router, restart re-route $253.60 +$45.60 (+21.9%)
Upfront router, 60%-reuse carry-over $248.80 +$40.80 (+19.6%)The request-level router runs 21.9% over the oracle. Now split that $45.60 by cause:
Leak Cost per 1,000 Share
----------------------------------------------------- -------------- -----
Over-provisioning easy tasks to Opus 5 $38.40 84%
Under-provisioning hard tasks (re-route double-spend) $7.20 16%The whole "complexity reveals itself mid-task" worry is about under-provisioning, and it is real. It is also the small half of the bill. Over-provisioning easy work to a frontier model is 84% of the leak here, and carry-over cannot touch it. Switching to a 60%-reuse carry-over trims the mispricing from 21.9% to 19.6%, because it only heals the 16% slice. Fixing the classifier's easy-task false alarms is where the money is.
How this differs from our other routing posts
This is not confidence-gated cascade routing. A cascade runs the cheap model first on purpose, on every call, and escalates by a confidence gate; its savings formula S = (1 - r) - e assumes the premium reprocesses from scratch, the restart case with zero carry-over. Here the cheap pass is a mistake, not a strategy, and the new lever is the reuse fraction that a cascade never models.
It is not offline downroute detection, which replays logs to find calls a cheap model could have handled with no double-spend, the recipe in which LLM calls a cheaper model could handle. It is not the static per-step tier split in route the plumbing, keep the planner, where each step runs once at a fixed tier. And it is a different question from whether vendor router savings claims survive audit, which we checked in the router savings claims audit. The distinct model here is the cost of committing before you know complexity, plus the carry-over economics of undoing that commitment mid-task.
When upfront routing is fine, and when NOT to re-route
- Homogeneous tasks. If your workload is nearly all one difficulty, an upfront pick is almost always right and the mispricing shrinks to near zero. Route once and move on.
- Reuse below the threshold. If the stalled work is likely garbage (the cheap model went down a wrong path), reuse is under 20% and carry-over loses. Restart clean, or better, stop under-provisioning that task class.
- Cheap waste that dwarfs the premium bill. Our numbers assume the cheap model stalls at
f = 0.5. If it grinds tof = 0.9before anyone notices, the wasted cheap attempt grows and the re-route penalty with it. A monitor that catches the stall early is worth more than the carry-over trick. - Wide easy/hard split with a weak classifier. If you cannot tell easy from hard cheaply, the over-provision leak dominates and no re-route logic fixes it. Spend the effort on the classifier, or adopt the plan-then-delegate structure Kun Chen describes, where a planner sizes the work before committing a model.
The honest tradeoff: mid-task carry-over is a real saving, but a narrow one. It repairs the smaller of the two leaks and only above a hard 20% reuse floor. If you build one thing, build the easy-task detector, not the resume-the-stall path.
Doing this on Vynaris
The math above is all you need to size this yourself; this section is the paid shortcut, labeled as such. Vynaris is an OpenAI-compatible gateway that sizes each request against your own difficulty signal before it commits a model, and reports the realized over-provision and under-provision rates so you can see the mispricing instead of guessing it. Where a re-route fires, it prices restart against carry-over on your live token shape and picks the cheaper path. See the docs and pricing.
FAQ
Why is over-provisioning worse than under-provisioning? Over-provisioning pays the premium output rate (5x the input rate on Claude models) across the entire task. Under-provisioning only wastes a partial cheap attempt. On our shape that is $0.32 vs $0.06 per incident, a 5.3x gap.
What is the 20% reuse rule? On a mid-task re-route, carry-over beats a full restart only when the premium model can reuse more than the input/output price ratio of the stalled output. Opus 5 is $5/$25, so the threshold is 20%; below it, restart is cheaper because you pay input to re-read work you rewrite anyway.
Does carry-over get a cache discount? No. The handoff crosses models, so the carried tokens are new input to the premium model, billed at full input price, not the 0.1x cache-hit rate. Carry-over inside the same model (an effort bump, not a model switch) can use prompt caching; a model switch cannot.
How much does request-level routing cost versus a perfect router? On a 60/40 easy/hard mix with a 20% over-provision and 30% under-provision error rate, 21.9% over the oracle ($253.60 vs $208.00 per 1,000 tasks). 84% of that is over-provisioning. Prices verified 2026-07-25.
Should I build mid-task re-routing? Only if under-provisioning is a large share of your tasks and stalled work is usually salvageable (reuse above 20%). Otherwise the classifier that stops over-provisioning saves more.
Sources
- X, Kun Chen on request-level routing (1,587 likes / 171 replies / 196,407 views, read 2026-07-25): https://x.com/kunchenguid/status/2080386212162941055
- Anthropic pricing (Opus 5 $5/$25, Sonnet 5 introductory $2/$10, Haiku 4.5 $1/$5, cache read 0.1x input), captured 2026-07-25: https://platform.claude.com/docs/en/about-claude/pricing
- Cost model script and all per-task arithmetic: derived from the assumptions table above (published in artifacts).
Prices change. We re-verify every figure in this post monthly and stamp updates. Numbers here are current as of 2026-07-25.
The lesson: a request-level router is not wrong because it re-routes; it is wrong because it commits before it can price the task. Fix the over-provisioning first, then re-route the stalls only when reuse clears 20%.