VynarisEarly betaGet your API key

Blog · 2026-09-14 · Vynaris Team

TrueFoundry's router cut cost per correct answer 68.6%: the $0.185 failure threshold

TrueFoundry's self-reported router cut cost per correct answer 68.6%, but one extra miss erases savings above $0.185.

TrueFoundry's self-reported 550-prompt benchmark cut spend from $3.23 to $0.99 while pass rate slipped from 96.4% to 94.2%. Recomputed, cost per correct answer fell from $0.00609 to $0.00191, a 68.6% reduction. But the all-Opus baseline is economically safer whenever an extra wrong answer costs more than $0.185.

Prices verified 2026-09-14.

TL;DR

Verdict table

Metric                                        All Opus 5                         Routed ladder                      Change
--------------------------------------------  ---------------------------------  ---------------------------------  ---------------------------
Published bill, 550 graded prompts            $3.23                              $0.99                              -69.3%
Pass rate                                     96.4%                              94.2%                              -2.2 points
Expected correct answers from aggregate rate  530.2                              518.1                              -12.1
Cost per prompt                               $0.005873                          $0.001800                          -69.3%
Cost per correct answer                       $0.006092                          $0.001911                          -68.6%
Mean latency                                  7.6 seconds                        4.0 seconds                        -47.4%
Decision                                      Use when misses cost under $0.185  Use when misses cost under $0.185  Measure on your own prompts

The cost result survives the quality adjustment. The operational verdict depends on the value of an answer, not the percentage saved.

What TrueFoundry measured

The TrueFoundry benchmark compares two setups. Its baseline sends every prompt to Claude Opus 5. Its routed arm classifies each request and chooses Claude Haiku 4.5, Claude Sonnet 4.5, or Opus.

The company reports 16 public datasets and about 4,700 calls. That headline contains three different evidence classes:

That separation matters. Only the 550-prompt group supports a cost-per-correct calculation. The four production-shaped sets can support a cost claim, not a quality-adjusted claim.

This is a vendor-published benchmark. TrueFoundry does not provide a downloadable per-request trace in the cited article. We can audit its arithmetic and framing, but we cannot reconstruct token counts or rerun the exact calls.

Editable assumptions

Input                                    Value used                       Receipt or treatment
---------------------------------------  -------------------------------  --------------------------------------------------------------------------------------------------
Graded prompts                           550                              11 datasets x 50 prompts, reported by TrueFoundry
Baseline cost                            $3.23                            Reported aggregate
Router cost                              $0.99                            Reported aggregate
Baseline pass rate                       96.4%                            Reported aggregate
Router pass rate                         94.2%                            Reported aggregate
Baseline model                           Claude Opus 5                    Reported by TrueFoundry
Router ladder                            Haiku 4.5 / Sonnet 4.5 / Opus 5  Reported by TrueFoundry
Standard input/output rates              $1/$5, $3/$15, $5/$25 per MTok   [Anthropic pricing](https://platform.claude.com/docs/en/about-claude/pricing), verified 2026-09-14
Cache, tools, retries, regional uplifts  Not added                        Source does not break them out

Anthropic's models overview verifies Opus 5 and Haiku 4.5 as current model names. Its live pricing table also lists Sonnet 4.5. The benchmark article does not disclose each tier's token volume. Current list prices therefore validate the ladder, not the published $3.23 and $0.99 totals.

Recomputing the 550-prompt bill

Start with cost per prompt:

All Opus: $3.23 / 550 = $0.0058727 per prompt
Router:    $0.99 / 550 = $0.0018000 per prompt
Savings:   1 - ($0.99 / $3.23) = 69.35%

Now apply each pass rate to the same 550 prompts. These are aggregate expectations, so decimals are normal:

All Opus expected correct = 550 x 0.964 = 530.2
Router expected correct   = 550 x 0.942 = 518.1

All Opus cost/correct = $3.23 / 530.2 = $0.006091
Router cost/correct   = $0.99 / 518.1 = $0.001911

The routed arm is 3.19x cheaper per expected correct answer. Put another way, the quality-adjusted cost falls 68.62%. TrueFoundry rounds this to 3.2x cheaper.

The less flattering denominator is error. Baseline errors are 3.6% of prompts. Router errors are 5.8%. The absolute change is only 2.2 points, but the relative error increase is 61.1%.

Both statements are true. Cost per correct answer improves sharply. Errors also become materially more frequent.

Use the LLM cost calculator for your own input and output shape. It cannot reproduce this benchmark exactly because TrueFoundry does not disclose tier-level token counts.

The $0.185 failure threshold

The useful buyer question is not whether 69% sounds large. It is what the extra misses cost.

At 1,000 prompts, the published aggregates imply:

Quantity                     Derived result
---------------------------  --------------
All-Opus bill                $5.8727
Routed bill                  $1.8000
Cash saved                   $4.0727
Additional expected misses   22
Savings per additional miss  $0.1851

The threshold follows directly:

($0.0058727 - $0.0018000) / (0.964 - 0.942) = $0.1851

If one additional wrong answer costs less than $0.185, the routed arm wins on expected spend. If it costs more, the all-Opus arm wins. The threshold includes only model spend and the value assigned to a miss. It excludes engineering labor and downstream retries because the public benchmark does not quantify them.

This turns a generic price-performance ratio into a decision. A low-stakes chat reply may sit below the threshold. A wrong production change, financial answer, or customer action probably does not.

Savings vary from 44% to 91%

The aggregate hides a wide workload spread. Academic-set savings range from 44% on HumanEval to 79% on GSM8K. The four production-shaped sets range from 69% on code-heavy chat to 91% on real-user chat.

Reported routing savings across 15 public datasets
Reported cost savings versus all-Claude Opus 5 across 15 datasets. Production sets report cost only, not answer-key quality. Source: TrueFoundry. Verified 2026-09-14.

That spread is evidence for workload-specific model routing. It is not evidence that production quality held. TrueFoundry says the four open-ended production sets have no answer key. Their reported overall saving is 80%, but their cost per correct answer is unknowable from the published data.

The coding results also resist a neat headline. HumanEval saved 44%, MBPP saved 55%, and code-heavy chat saved 69%. Real-user chat reached 91%. Prompt mix decides whether the router sees enough easy work to matter.

The hard-question failure mode

TrueFoundry's 570-question difficulty set exposes the tradeoff. Its free heuristic retained 90% of baseline quality while saving 76%. It routed none of those questions to Opus. On hard questions, it retained only 35% of baseline quality.

The small-model classifier changed the operating point. It sent 31% of questions to Opus, retained 95% of baseline quality, and saved 66%. Ten percentage points of savings bought five points of quality retention.

The source does not publish the baseline dollar bill for this set. We cannot convert that trade into dollars per recovered correct answer. Still, it identifies the mechanism. A free classifier reads prompt shape. Short hard questions can look cheap even when their answer risk is high.

This is where a LLM gateway can create false confidence. The request succeeds technically. The response arrives. The wrong tier simply answers it badly.

What this means for routing

First, segment by consequence before complexity. Requests below the $0.185 miss threshold are candidates for aggressive routing. Requests above it need a stricter gate or a premium default.

Second, measure cost per outcome, not only cost per task. TrueFoundry's raw bill falls 69.35%. Its quality-adjusted bill falls 68.62%. The result remains strong here, but that will not always happen.

Third, keep production quality separate from production savings. A router can report 91% lower spend on open-ended chat without proving that the answers stayed useful. Attach deterministic checks, sampled human review, or business outcomes before promoting that number.

Fourth, include latency only after quality. The routed graded set fell from 7.6 seconds to 4.0 seconds, a 47.4% reduction. Faster wrong answers are still wrong.

Our earlier audit of router savings claims found the same denominator problem. Percent saved is not enough. The router break-even guide shows why low call volume can also make routing work uneconomic.

Honest tradeoff

The heuristic is attractive because classification adds no model call. That same simplicity is its weakness. It routed 0% of the difficulty set to Opus and retained 35% of quality on hard questions.

The small-model classifier recovers quality, but it gives back ten points of savings and routes 31% to Opus. There is no universal best setting. The right point depends on the cost of a miss and the share of deceptively short, hard prompts.

For a low-stakes workload, the heuristic result is compelling. For tasks where one wrong answer costs dollars, the published $0.185 threshold says to protect quality first.

FAQ

Did TrueFoundry prove a 3.2x lower cost per correct answer?

Its published aggregates support that arithmetic. We recompute $0.006091 for all Opus and $0.001911 for the router, a 3.19x ratio. The benchmark remains self-reported and lacks a per-request trace.

Does the 80% production saving include quality?

No. The four production-shaped datasets report cost and routing. TrueFoundry says open-ended chat has no answer key. The 80% figure is a spend result, not a cost-per-correct result.

Why does a 2.2-point quality drop matter?

Because errors rose from 3.6% to 5.8%. That is a 61.1% relative increase. Whether it matters economically depends on the cost of each additional miss.

When is the heuristic router worth using?

Use it when extra misses cost less than about $0.185 under this benchmark's economics. Re-estimate that threshold with your own bill and measured pass-rate delta.

Should hard prompts always go to Opus 5?

Not automatically. The source shows the heuristic struggled on hard questions. A better classifier or deterministic verification may be cheaper than sending every request to Opus.

Sources