---
title: "Confidence-gated cascade routing: the double-spend behind the 65-85% savings claim"
description: "A confidence-gated cascade pays the cheap model on every call plus the premium on escalations."
canonical: https://vynaris.com/blog/confidence-gated-cascade-routing-cost-math
author: "Dhrumil Barot"
author_bio: "Dhrumil is an AI inference and LLM infrastructure expert. Dhrumil built Vynaris to make model access, routing and pricing simple and transparent for developers."
published: 2026-07-23
updated: 2026-08-22
---

# Confidence-gated cascade routing: the double-spend behind the 65-85% savings claim

Published 2026-07-23 · Updated 2026-08-22 · By [Dhrumil Barot](https://vynaris.com/about#founder), Founder

A confidence-gated cascade pays the cheap model on every call plus the premium on escalations. The real savings formula and break-even escalation rate, on live 2026 prices.

A confidence-gated cascade runs a cheap model on every call, scores how sure it is, and escalates only the low-confidence answers to a premium model. On the escalated fraction you pay twice: the cheap attempt plus the premium re-run. That double-spend means your savings are not `1 minus escalation rate`; they are `(1 minus the cheap/premium cost ratio) minus the escalation rate`. On live prices verified 2026-07-23, the popular 65-85% cost-cut claim only holds when the cheap tier costs near zero. Swap in a cloud cheap model and the same escalation rate saves far less.

## TL;DR

- Cascade savings vs always-premium follow one formula: `S = (1 - r) - e`, where `r` is the cheap model's per-call cost divided by the premium's, and `e` is the escalation rate. The ceiling is `1 - r`; every point of escalation subtracts a point.
- The Hacker News claim making the rounds (route 15-35% of queries up, cut API cost 65-85%) checks out **only for an on-device cheap tier** where `r` is ~0. On our task shape it computes to 85% at 15% escalation and 65% at 35% escalation. With a cloud [Haiku 4.5](https://vynaris.com/models#claude-haiku-4-5) cheap tier (`r = 0.20`), the same 15-35% escalation saves 65% down to 45%, not 65-85%.
- A separate judge gate is paid on every call and can push the cascade underwater. Cheap Haiku 4.5 into premium [Sonnet 5](https://vynaris.com/models#claude-sonnet-5) with a Haiku judge loses 3.1% versus just always running Sonnet 5 at a 25% escalation rate. Prices verified 2026-07-23.

## The finding, before the mechanics

A hybrid on-device project on Hacker News this week (75 points, 12 comments, read 2026-07-23) claims it "taught a small model to know when it's wrong," routes only 15-35% of queries to a premium cloud model, and cuts API cost 65-85%. The number is real. What the headline hides is *why* it is real, and the reason does not transfer to a pure-cloud cascade.

A cascade is not picking a cheaper model per request. It is sequential: the cheap model answers **first, on every call**. A confidence gate then accepts that answer or escalates it to a premium model, which reprocesses the input from scratch. Every escalated call carries two bills, not one. This is the [model-routing](https://vynaris.com/glossary/category/routing-and-cost#model-routing) pattern with a hidden tax, and the tax is the whole story below. Every dollar figure comes from a provider's live pricing page (captured 2026-07-23) or the arithmetic shown.

## What a cascade actually pays

Define the per-call cost of the cheap model as `c_cheap` and the premium as `c_prem`. With escalation rate `e` (the fraction of calls the gate sends up) and a per-call gate overhead `g`:

```
cascade cost per call   = c_cheap + g + e * c_prem
always-premium per call = c_prem
```

The cheap model and the gate are paid on **every** call. The premium is paid on the escalated fraction `e`. Divide through by `c_prem` and the savings versus always-premium collapse to a two-term formula:

```
S = 1 - (c_cheap + g + e*c_prem) / c_prem
  = (1 - r - g/c_prem) - e
```

where `r = c_cheap / c_prem`. With a self-scored gate (the cheap model reports its own confidence, so `g` is ~0), this simplifies to `S = (1 - r) - e`. Two levers, and both cut the same direction:

- `1 - r` is the **ceiling**: the most you can save, reached only if you escalate nothing.
- `e` is the **haircut**: each point of escalation is one point off the ceiling, because you double-pay those calls.

The break-even escalation rate, where the cascade stops beating always-premium, is `e* = 1 - r - g/c_prem`. Above it, you are paying two bills to do one job.

## The workload (edit these)

| Assumption | Value | Note |
| --- | --- | --- |
| Representative task | 1,500 [input tokens](https://vynaris.com/glossary/input-tokens) / 500 [output tokens](https://vynaris.com/glossary/output-tokens) | one request in and out |
| Premium model | [Opus 4.8](https://vynaris.com/models#claude-opus-4-8) at $5 / $25 per 1M | the model you would run on every call |
| Cheap candidates | Haiku 4.5, [Gemini 3.5 Flash-Lite](https://vynaris.com/models#gemini-3-5-flash-lite), [DeepSeek v4-flash](https://vynaris.com/models#deepseek-v4-flash), on-device | the first-pass model |
| Escalation rate `e` | the lever | fraction the gate sends to premium |
| Gate overhead `g` | $0 self-scored, else a judge call | paid on every call |

At $5 per 1M input and $25 per 1M output, Opus 4.8 costs `(1,500 x $5 + 500 x $25) / 1,000,000 = $0.02` per call. That is the always-premium baseline every cascade below is measured against. Prices verified 2026-07-23.

## The cheap-to-premium ratio decides the ceiling

`r` is the only thing that sets how much a cascade *can* save. Here it is for four cheap tiers against an Opus 4.8 premium, on the task shape above.

| Cheap tier | Cost per call | `r` vs Opus 4.8 | Savings ceiling `1 - r` | Break-even escalation `e*` |
| --- | --- | --- | --- | --- |
| On-device (local small model) | ~$0.00000 | 0.000 | 100.0% | 100.0% |
| DeepSeek v4-flash | $0.00035 | 0.018 | 98.2% | 98.2% |
| Gemini 3.5 Flash-Lite | $0.00170 | 0.085 | 91.5% | 91.5% |
| Haiku 4.5 | $0.00400 | 0.200 | 80.0% | 80.0% |

An on-device cheap tier has no marginal API cost, so its ceiling is 100% and it never loses to always-premium until you escalate every call. A cloud Haiku 4.5 tier costs a fifth of Opus 4.8 per call, so its ceiling is 80%: even escalating nothing, you still pay the Haiku bill on the whole volume. That gap between a 100% ceiling and an 80% ceiling is exactly what separates the on-device demo from your cloud rebuild.

## Stress-testing the 65-85% claim

Now put the escalation rate back in and check the headline directly. The claim is 15-35% escalation yields a 65-85% cut. Using `S = (1 - r) - e`:

| Cheap tier | Savings at 15% escalation | Savings at 35% escalation |
| --- | --- | --- |
| On-device (`r` ~ 0) | 85.0% | 65.0% |
| DeepSeek v4-flash (`r` = 0.018) | 83.2% | 63.3% |
| Haiku 4.5 (`r` = 0.20) | 65.0% | 45.0% |

The on-device row reproduces the claim to the point: 85% at 15%, 65% at 35%. It works because `r` is ~0, so savings are just `1 - e`. The moment the cheap tier is a billed cloud API, the ceiling drops by `r` and the whole band slides down. A Haiku 4.5 cascade at the same 15-35% escalation saves 65% to 45% — the top of its range equals the *bottom* of the on-device range. The double-spend on the cheap tier ate 20 points.

DeepSeek v4-flash is the interesting middle: at `r = 0.018` it is cheap enough that its cascade behaves almost like on-device (83.2% down to 63.3%). If your cheap tier has to be a cloud model and quality allows it, the cheapest cloud model preserves the most of the ceiling.

![Horizontal bar chart, single blue hue, of cascade savings versus always-premium at a 25% escalation rate across five cheap-to-premium pairings: on-device to Opus 4.8 75%, DeepSeek v4-flash to Opus 4.8 73%, Gemini 3.5 Flash-Lite to Opus 4.8 66%, Haiku 4.5 to Opus 4.8 55%, Haiku 4.5 to Sonnet 5 25%](https://cdn.sanity.io/images/u1qdn8dx/production/9e6e7c937f8142366f6ba599895ae46acafde0b3-940x430.png?w=1440&fit=max&auto=format)

*Cascade savings vs always-premium at a 25% escalation rate. The 65-85% claim needs the near-zero-cost on-device cheap tier (top bar); a cloud cheap tier and a narrow premium gap erode it. Prices verified 2026-07-23.*

## The escalation-rate sensitivity

Savings decay linearly in the escalation rate, so the gate's calibration is the entire game. Here is a Haiku 4.5 to Opus 4.8 cascade across the range, self-scored gate.

| Escalation rate `e` | Cascade $/1,000 calls | Savings vs always-Opus |
| --- | --- | --- |
| 0% | $4.00 | 80.0% |
| 10% | $6.00 | 70.0% |
| 25% | $9.00 | 55.0% |
| 50% | $14.00 | 30.0% |
| 80% | $20.00 | 0.0% |
| 100% | $24.00 | -20.0% |

At 100% escalation you pay Haiku on all 1,000 calls plus Opus on all 1,000, so the cascade costs 20% *more* than just calling Opus directly. The break-even is 80%: below it the cascade wins, above it you would have been cheaper going straight to premium. A gate that escalates aggressively to protect quality can quietly walk you past that line. Drop your own token shape and escalation rate into [the calculator](https://vynaris.com/calculator?in=1500&out=500) to find your version of this table.

## When the gate costs more than it saves

The self-scored gate assumption (`g` = 0) is the best case. Many teams run a **separate judge**: a small model reads the cheap answer and decides whether to escalate. That judge is a call on every request. Price it as a Haiku 4.5 [LLM-as-judge](https://vynaris.com/glossary/category/evals-and-quality#llm-as-judge) reading the 1,500-token prompt plus the 500-token draft and emitting a 50-token verdict: `(2,000 x $1 + 50 x $5) / 1,000,000 = $0.00225` per call. Against an Opus 4.8 premium that is `g/c_prem = 0.1125`, so it shaves 11.3 points off every break-even.

The judge is survivable when the premium-to-cheap gap is wide. It is fatal when the gap is narrow. Run a Haiku 4.5 cheap tier into a **Sonnet 5** premium (introductory $2/$10, so $0.008 per call) instead of Opus 4.8:

| Cascade (Haiku 4.5 cheap -> Sonnet 5 premium) | Break-even escalation `e*` | Savings at 25% escalation |
| --- | --- | --- |
| Self-scored gate (`g` = 0) | 50.0% | 25.0% |
| Separate Haiku judge (`g` = $0.00225) | 21.9% | -3.1% |

With `r = 0.5` the ceiling is only 50% before you escalate anything. Add the judge and the break-even falls to 21.9%: escalate more than about one call in five and the cascade loses money. At a 25% escalation rate it is already 3.1% underwater versus just always running Sonnet 5. The rule: a cascade needs both a wide premium-to-cheap cost gap **and** a near-free gate. Kill either and the double-spend wins.

## The accuracy floor you cannot escape

The escalation rate is not a free dial. Lowering `e` to save more ships more low-confidence answers from the cheap model. The gate's job is to escalate exactly the calls the cheap model would get wrong. A gate with poor recall either escalates too much (you pay the double-spend and lose savings) or too little (you ship worse answers and call it a saving). The on-device pitch "know when it's wrong" is a bet on high gate recall, and that calibration, not the routing structure, earns the savings. Measure the gate's escalate-when-wrong rate on your own traffic before trusting any headline percentage.

## How this differs from the other routing posts

The double-spend is what makes runtime cascade math distinct. It is **not** offline downroute detection: replaying logs to find which share a cheap model could have handled has no double-spend, because each call goes to exactly one model — the recipe in [which LLM calls a cheaper model could handle](https://vynaris.com/blog/which-calls-a-cheaper-model-could-handle-detection-howto). It is not upfront task-type routing, where a classifier sends each request to one model by type, as in the [coding-agent model-routers savings math](https://vynaris.com/blog/coding-agent-model-routers-savings-math-2026). And it is a different axis from the labor break-even in [when token optimization is not worth it](https://vynaris.com/blog/token-optimization-break-even-when-not-worth-it): that asks whether the engineering pays back at your volume; this asks whether the routing structure saves once the double-spend is counted. We audited inflated vendor router claims the same way in the [router savings claims audit](https://vynaris.com/blog/llm-router-savings-claims-audit-2026).

## When NOT to run a cascade

- **Narrow premium-to-cheap gap.** If `r` is above ~0.4, the ceiling is already thin and a modest escalation rate erases it. Just run the premium model, or pick a cheaper cheap tier to widen the gap.
- **You need a separate judge and the gap is not huge.** A billed judge on every call is a flat tax. If it pushes the break-even below your real escalation rate, the cascade is a net loss. Prefer a self-scored gate (logprobs, the model's own confidence) whenever the quality bar allows.
- **Escalation you cannot hold down.** If protecting quality forces you past `e* = 1 - r - g/c_prem`, the cascade costs more than always-premium. Confirm your gate stays under it.
- **Low volume.** The gate and escalation logic are engineering. On a few thousand calls a month the build cost swamps any per-call saving; pay the premium bill and skip the cascade.

## Doing this on Vynaris

The formula above is all you need to model a cascade yourself; this section is the paid shortcut, clearly labeled. Vynaris is an [OpenAI-compatible](https://vynaris.com/glossary/category/ecosystem-and-products#openai-compatible-api) gateway that runs the cascade for you: cheap-first attempt, a self-scored confidence gate (no separate judge call), escalation only above your confidence threshold, and per-call reporting of the realized escalation rate and savings against the always-premium baseline. You see the measured `e` and `S`, not a demo number. See the [docs](https://vynaris.com/docs) and [pricing](https://vynaris.com/pricing).

## FAQ

**What is confidence-gated cascade routing?** A pattern where a cheap model answers every request first, a confidence gate scores that answer, and only low-confidence ones escalate to a premium model, which reprocesses the input. It trades a double-spend on escalated calls for a cheap first pass on everything else.

**Why don't cascade savings equal \`1 minus the escalation rate\`?** The cheap model is paid on every call, not just the non-escalated ones. Savings are `(1 - r) - e`, where `r` is the cheap-to-premium per-call cost ratio. Only when the cheap tier costs ~0 (on-device) does `r` vanish and savings approach `1 - e`.

**Does the 65-85% cost-cut claim hold?** Only for an on-device cheap tier where `r` is ~0. On our 1,500-in/500-out task that is 85% at 15% escalation and 65% at 35%. A cloud Haiku 4.5 cheap tier (`r` = 0.20) saves 65% down to 45% over the same band. Prices verified 2026-07-23.

**When does a cascade lose money?** Above the break-even escalation rate `e* = 1 - r - g/c_prem`. A Haiku 4.5 to Opus 4.8 cascade breaks even at 80% escalation; a Haiku 4.5 to Sonnet 5 cascade with a separate Haiku judge breaks even at 21.9% and is already 3.1% underwater at 25%.

**Self-scored gate or separate judge?** A self-scored gate (the model's own confidence or logprobs) adds no per-call cost. A separate judge is a billed call on every request and lowers every break-even by `g/c_prem`, 11.3 points against an Opus 4.8 premium here. Use a judge only when the premium-to-cheap gap is wide enough to absorb it.

## Sources

- Hacker News, on-device hybrid cascade routing thread (75 points / 12 comments, read 2026-07-23): https://news.ycombinator.com/item?id=49010782
- Hacker News, "Smart model routing in Claude, Codex, Cursor" (216 points / 113 comments, read 2026-07-23): https://news.ycombinator.com/item?id=48688700
- Anthropic pricing (Opus 4.8 $5/$25, Haiku 4.5 $1/$5, Sonnet 5 $2/$10 standard), re-verified 2026-08-14: https://platform.claude.com/docs/en/about-claude/pricing
- DeepSeek API pricing (v4-flash $0.14/$0.28), captured 2026-07-23: https://api-docs.deepseek.com/quick_start/pricing
- Google Gemini API pricing (3.5 Flash-Lite $0.30/$2.50), captured 2026-07-23: https://ai.google.dev/gemini-api/docs/pricing
- Cost model script and all per-call arithmetic: derived from the assumptions table above.

Prices change. We re-verify every figure in this post monthly and stamp updates. Numbers here are current as of 2026-07-23.

The lesson: a cascade is not free money because it is cheap-first. Savings are `(1 - r) - e`. Widen the premium-to-cheap gap, keep the gate self-scored, and hold the escalation rate under the break-even, or skip the cascade and pay the premium bill.

## About the author

[Dhrumil Barot](https://vynaris.com/about#founder), Founder. Dhrumil is an AI inference and LLM infrastructure expert. Dhrumil built Vynaris to make model access, routing and pricing simple and transparent for developers. [LinkedIn](https://www.linkedin.com/in/bdhrumil) · [X](https://x.com/fyidhrumil)