VynarisEarly betaGet your API key

Blog · 2026-08-20 · Vynaris Team

When Not to Use an LLM Router: The 12,570-Task Break-Even

An LLM router loses below 12,570 tasks/month in this worked case, or at any volume once added failures exceed 0.126%. Prices verified 2026-08-20.

An LLM router is a net loss below 12,570 tasks per month in our worked case. Add a 0.10% failure penalty and break-even rises to 61,479 tasks. At 0.126%, more volume cannot save it. Prices verified 2026-08-20; every workload value is editable.

TL;DR

Verdict table

Workload condition                                   Worked result                                               Verdict
---------------------------------------------------  ----------------------------------------------------------  -----------------------------
One model, no safely routable tasks                  $21.10 per 1,000 tasks after a 5.5% fee, versus $20 direct  Stay direct
$0.02 default, 50% routable, no reliability penalty  Break-even at 12,570 tasks/month with a $79 fixed cost      Route above the line
Same workload, 0.10% added failures at $5 each       Break-even at 61,479 tasks/month                            Measure failures first
Already-cheap $0.002 default                         Break-even at 382,567 tasks/month before reliability cost   Usually stay direct
Router hop exceeds remaining p95 latency headroom    Cost saving does not repair the SLO                         Stay direct
Multiple providers are required for failover         Reliability benefit may exceed token savings                Price uptime, not only tokens

This is a boundary test, not another router-savings claim. Our earlier post-Luna router re-rate asks how a price cut shrinks arbitrage. This guide asks whether the whole layer pays for itself.

Price the whole layer

An LLM router chooses a model or provider for each request. That model routing can reduce the model bill. The surrounding inference gateway can also add fees, fixed plans, another network hop, and another failure surface.

The correct unit is cost per task, including failed outcomes. Token savings alone leave four bills out:

  1. The variable platform fee.
  2. Any fixed observability or team plan.
  3. Cheap attempts that fail and run again on the default model.
  4. Router-caused failures, latency breaches, and their business cost.

Write the monthly decision as:

net monthly value
= volume × (direct task cost - routed task cost after variable fees)
- fixed monthly cost
- volume × incremental failure rate × cost per failed outcome

If the result is negative, do not add the router. If the latency gate fails, do not add it even when the dollar result is positive.

Editable assumptions

We use blended token rates so the worked example shows both per-million pricing and per-request cost. These rates are scenario inputs, not a claim about any named model.

Parameter                          Value                            Source or derivation
---------------------------------  -------------------------------  -------------------------------------
Task shape                         3,000 input + 500 output tokens  Editable representative request
Default blended rate               $5/1M input + $10/1M output      Editable scenario
Default task cost                  $0.020                           3,000/1M × $5 + 500/1M × $10
Cheap blended rate                 $1/1M input + $2/1M output       Editable scenario
Cheap attempt cost                 $0.004                           3,000/1M × $1 + 500/1M × $2
Routable share                     50%                              Editable assumption
Misroute rate inside routed share  10%                              Editable assumption; redo on default
Variable fee                       5.5%                             OpenRouter pay-as-you-go public price
Fixed monthly cost                 $79                              Helicone Pro public price
Failed-outcome cost                $5                               Editable support or rework assumption

The 5.5% and $79 figures are two public fee shapes stacked for a stress test. They are not a bundle, and nobody must buy both. Remove the fixed term if your stack has none. Replace the percentage with your actual contract.

The 12,570-task break-even

Let D be the $0.020 default cost. Let C be the $0.004 cheap attempt. Half the tasks route cheaply, and 10% of that half must be repeated on the default.

routed compute per task
= (1 - 0.50) × $0.020
+ 0.50 × ($0.004 + 0.10 × $0.020)
= $0.013000

after 5.5% variable fee
= $0.013000 × 1.055
= $0.013715

saving before fixed cost
= $0.020000 - $0.013715
= $0.006285 per task

fixed-cost break-even
= $79 / $0.006285
= 12,569.61 tasks per month

Round to operational units and the line is 12,570 tasks per month. At 10,000 tasks, direct inference costs $200. The routed stack costs $137.15 plus $79, or $216.15. Routing loses $16.15.

At 100,000 tasks, direct costs $2,000. Routing costs $1,371.50 plus $79, or $1,450.50. It saves $549.50. Volume finally spreads the fixed cost thin enough.

Replace the blended rates with your measured input, cache, and output meters in the LLM cost calculator, then put the resulting direct and cheap task costs into the saved script.

Reliability creates a cliff

The 10% misroute rate above is already billed. Each bad cheap attempt runs again on the default. It does not represent failures introduced by gateway configuration, policy mistakes, an unavailable gateway, or a route that returns an accepted-looking wrong answer.

Price that second category separately:

reliability penalty per task
= incremental router failure rate × cost per failed outcome

At a $5 failed-outcome cost, 0.10% added failures cost 0.001 × $5 = $0.005 per task. The original $0.006285 margin falls to $0.001285. The $79 fixed term now needs 61,478.60 tasks per month.

At 0.125% added failures, the penalty is $0.006250 per task. Only $0.000035 remains. Break-even jumps to 2,257,143 tasks per month. That is 180x the no-penalty line.

At 0.126%, the penalty becomes $0.006300. It exceeds the $0.006285 variable saving. The router loses $0.000015 per task before fixed cost. More volume makes the loss larger.

Log-scale bar chart showing router break-even rising from 12,570 to 2,257,143 monthly tasks as incremental failure rises from 0% to 0.125%
Illustrative $0.02 task, 50% routed, 10% misroutes and $5 failed-outcome cost. Break-even rises 180x as incremental router failures approach 0.125%. Public fee inputs verified 2026-08-20.

The general reliability ceiling is:

maximum added failure rate
= saving before reliability / failed-outcome cost
= $0.006285 / $5
= 0.1257%

Your failed-outcome cost may be $0.20 of automated retry work or $500 of human escalation. We cannot choose it for you. Hiding it at zero is still a choice, and usually the flattering one.

Cheap defaults leave almost no room

Right-sizing has the most value when the default is expensive and many tasks can move down safely. An already-cheap default leaves a narrow spread.

Use the same 3,000-input and 500-output task. Set the default blended rates to $0.50/1M input and $1/1M output. That costs $0.002 per task. Set the cheaper route to $0.30/1M and $0.60/1M, or $0.0012 per attempt.

With the same 50% routable share and 10% misroute rate, routed compute is $0.0017 per task. The 5.5% fee raises it to $0.0017935. The saving is only $0.0002065 per task. Recovering $79 requires 382,567 tasks per month.

Reliability tolerance also collapses. Divide $0.0002065 by the same $5 failed-outcome cost. Just 0.00413% incremental failures erase the variable saving. A cheap model need not be free to make routing uneconomic. It only needs to be close to the floor.

One model means there is nothing to route

Set the routable share to zero. The router sends every task to the same default. Compute stays at $0.020, while the 5.5% fee raises it to $0.0211. That is $21.10 per 1,000 tasks instead of $20, before any fixed plan.

An extra layer may still provide logs, keys, budgets, or policy enforcement. Call that an observability purchase. Do not call it routing savings.

The same distinction matters for bring your own key. OpenRouter's live pricing lists a $25,000 monthly list-price inference allowance for pay-as-you-go BYOK, then a 5% fee above it. At $30,000, the fee is ($30,000 - $25,000) × 5% = $250. At $100,000, it is $3,750. Enterprise lists a $200,000 allowance. Price the overage, not the headline percentage against the whole bill.

Latency is a hard gate

Do not invent a universal gateway-latency number. Measure your path. Latency depends on region, provider choice, cache state, routing logic, and fallback behavior.

Run three tests from the same client region:

  1. Call the provider directly and record successful-request p50 and p95 time to first token.
  2. Call the same model and provider through the gateway with routing disabled. This isolates the hop.
  3. Enable the route policy. Track the chosen lane, retries, accepted outcomes, and p95 again.

Reject the layer if its p95 delta exceeds the headroom left in your service-level objective. A positive monthly saving does not compensate for a broken interactive experience.

LiteLLM's routing documentation warns that usage-based routing can add significant latency because it performs Redis operations. OpenRouter documents that a failed initial completion adds latency before fallback. Neither gives you permission to copy a generic number into your own SLO.

A minimal decision function

The DIY path is one function and one shadow test. You do not need a routing platform to decide whether a routing platform is justified.

def router_margin(
    direct_cost,
    cheap_cost,
    routable_share,
    misroute_rate,
    variable_fee_rate,
    fixed_monthly_cost,
    monthly_volume,
    added_failure_rate,
    failed_outcome_cost,
):
    routed_compute = (
        (1 - routable_share) * direct_cost
        + routable_share * (cheap_cost + misroute_rate * direct_cost)
    )
    after_fee = routed_compute * (1 + variable_fee_rate)
    reliability = added_failure_rate * failed_outcome_cost
    return monthly_volume * (direct_cost - after_fee - reliability) - fixed_monthly_cost

Shadow the route without changing production behavior. Log which lane would have been selected. Replay those tasks on the candidate model. Grade accepted outcomes with deterministic checks where possible. Then replace the assumptions with observed costs, route shares, failures, and latency deltas.

The saved math script computes every table here. An independent Fraction-based checker reproduces the 12,569.61, 61,478.60, and 382,566.59 break-evens.

The honest tradeoff

A router can earn its place without lowering the model bill. Multi-provider fallback can improve availability. Central keys and spend controls can reduce operational risk. OpenRouter states that it monitors provider response times, errors, and availability, then routes using that feedback.

That benefit can make the incremental failure term negative. If routing prevents more failures than it introduces, the reliability credit belongs in the same equation. Our fallback-chain outage cost model shows how to price that side.

The opposite tradeoff also holds. Self-hosted software can avoid a percentage fee while creating an engineering bill. Our LiteLLM versus managed gateways TCO prices that labor floor. Free software and a free operation are not the same thing.

The paid shortcut: Vynaris

The method above is sufficient to stay direct, build a static switch, or evaluate any gateway. Vynaris provides OpenAI-compatible routing, cost visibility, and fallbacks as a managed layer. Review the documentation and public pricing, then run the same break-even test. If the measured result is negative, keep the direct provider call.

FAQ

When should I not use an LLM router?

Stay direct when one model handles every task, monthly volume sits below your fixed-cost break-even, the default is already near the cheap floor, added failures erase the model spread, or the gateway hop consumes your latency headroom.

How do I calculate router break-even?

Subtract routed task cost, variable fees, and expected failure cost from direct task cost. Divide fixed monthly cost by the remaining per-task saving. If the remaining saving is zero or negative, no finite volume breaks even.

Does a 5.5% fee mean routing must save more than 5.5%?

Not exactly. The fee applies to the routed compute bill in this model, not the original direct bill. Here routing cuts compute from $0.020 to $0.013, then the fee adds $0.000715. Include the fee in dollars before comparing.

Is a free or self-hosted router automatically cheaper?

No. A zero software fee can still carry infrastructure, maintenance, incident response, and on-call work. Put those costs into the fixed monthly term.

Can routing improve reliability?

Yes. Provider-aware fallback can avoid outages and rate limits. Measure the net change in accepted outcomes. A reliability improvement is a credit; an added failure surface is a penalty.

Sources