---
title: "The real price of structured output: JSON mode and tool-call token overhead, measured"
description: "Tools add a fixed token tax before your prompt: 67 to 77% of a small structured call. Per-model math plus the caching fix. Verified 2026-07-19."
canonical: https://vynaris.com/blog/structured-output-json-tool-call-token-overhead-cost
author: "Dhrumil Barot"
author_bio: "Dhrumil is an AI inference and LLM infrastructure expert. Dhrumil built Vynaris to make model access, routing and pricing simple and transparent for developers."
published: 2026-07-19
updated: 2026-10-14
---

# The real price of structured output: JSON mode and tool-call token overhead, measured

Published 2026-07-19 · Updated 2026-10-14 · By [Dhrumil Barot](https://vynaris.com/about#founder), Founder

Tools add a fixed token tax before your prompt: 67 to 77% of a small structured call. Per-model math plus the caching fix. Verified 2026-07-19.

Turn on tools and Claude adds a fixed block of input tokens before it reads a word of your prompt. On [Claude Opus 4.8](https://vynaris.com/models#claude-opus-4-8), forcing a tool call adds 410 tokens of tool-use system prompt; a 200-token schema pushes the fixed overhead to 610. On a small extraction call of 300 input and 15 output tokens, that overhead is *67% of the input bill*. Prices and token counts verified 2026-07-19. Here is when the tax dominates a cheap-model bill and when it is noise.

## TL;DR

- Anthropic publishes the exact fixed [token](https://vynaris.com/glossary/category/cost-and-finops#cost-per-token) cost of turning on tools. Forcing a tool call adds *410 tokens* on Opus 4.8, *474* on [Sonnet 5](https://vynaris.com/models#claude-sonnet-5), *588* on [Haiku 4.5](https://vynaris.com/models#claude-haiku-4-5), before your prompt or your schema.
- On a high-volume structured call (300 in, 15 out, 200-token schema), the fixed tool tax is *62% to 73% of the total bill*. On Haiku 4.5 it is *$788 of every $1,163* per million calls.
- The cheap workhorse is punished hardest. Haiku 4.5's tool-use system prompt is *588 tokens, 43% more* than Opus 4.8's 410. [Caching](https://vynaris.com/glossary/category/inference-and-serving#prompt-caching) the tool prefix cuts the bill *56% to 61%*.

## What we computed, and why

[Structured output](https://vynaris.com/glossary/category/agents-and-harnesses#structured-outputs) feels free. You add a `tools` array or a JSON schema, the model returns clean fields, and nothing on the invoice says "structured output." The cost is real; it is just filed under input tokens. Every tool-enabled request carries a [tool-use](https://vynaris.com/glossary/category/agents-and-harnesses#tool-use) system prompt the API injects for you, plus the tokens of your schema, plus per-request `tool_use` and `tool_result` blocks. None of it is your content, and all of it bills at the input rate on every call.

Anthropic is the one major provider that prints the fixed part as an exact per-model number, so we can quantify it from public docs instead of guessing. We take those published counts, apply live 2026-07-19 prices, and find the workload shape where the tax stops being a rounding error and becomes the bill.

Every figure below comes from a script; nothing was computed by hand.

## The published receipts (verified 2026-07-19)

These are Anthropic's stated token counts for enabling tools. The tool-use system prompt is added whenever at least one tool is present; the count depends on `tool_choice`.

| Model | Tool system prompt (auto) | Tool system prompt (forced) |
| --- | --- | --- |
| [Claude Opus 4.8](https://vynaris.com/models#claude-opus-4-8) | 290 | 410 |
| [Claude Sonnet 5](https://vynaris.com/models#claude-sonnet-5) | 354 | 474 |
| [Claude Haiku 4.5](https://vynaris.com/models#claude-haiku-4-5) | 496 | 588 |
| [Claude Opus 4.7](https://vynaris.com/models#claude-opus-4-7) | 675 | 804 |

Named tools cost extra on top, also published: the bash tool adds *325 tokens* on Opus 4.7 and 4.8 (244 on Sonnet 4.6 and earlier), and the `text_editor_20250429` tool adds *700 tokens* on Claude 4.x. These are fixed per request, independent of what the tool does. Your own custom tool adds its schema tokens, which you must measure with the count-tokens endpoint; we use 200 tokens as an editable placeholder.

The load-bearing detail: the tool-use system prompt does not scale with a bigger cheaper model discount. Haiku 4.5, the model you reach for precisely because it is cheap at high volume, carries a *588-token* forced-tool system prompt against Opus 4.8's *410*. The cheap model pays *43% more* fixed tokens per call than the flagship.

## The assumptions (edit these)

A high-volume structured-extraction call: classify or extract a small record, return a compact JSON object, forced tool call so the model must emit the structure.

| Assumption | Value | Note |
| --- | --- | --- |
| Content input | 300 tokens | the actual text to classify or extract |
| Output tokens | 15 | a small structured JSON result |
| Custom tool schema | 200 tokens | workload-specific; measure with count-tokens |
| Tool choice | forced (any/tool) | the model must call the tool |
| Volume basis | 1,000,000 calls | high-throughput pipeline |

## Results: the tax before your prompt

For each model, the fixed tool tax is the tool-use system prompt plus the 200-token schema. Content input is 300 tokens.

| Model | Fixed tax | Total input | Tax as % of input |
| --- | --- | --- | --- |
| Claude Opus 4.8 | 610 | 910 | 67% |
| Claude Sonnet 5 | 674 | 974 | 69% |
| Claude Haiku 4.5 | 788 | 1,088 | 72% |
| Claude Opus 4.7 | 1,004 | 1,304 | 77% |

![Horizontal stacked bars per model showing the fixed tool tax as the majority of a small structured call's input, 67 to 77 percent, with the 300-token content as the minority slice](https://cdn.sanity.io/images/u1qdn8dx/production/a349dd8acc6d7f2b535d28eaa4ed627a1e2b7035-1288x672.png?w=1440&fit=max&auto=format)

*Fixed tool tax (tool-use system prompt + 200-token schema) vs 300-token content, forced tool call. Published token counts, prices verified 2026-07-19.*

Now the money, per million calls, uncached, at live 2026-07-19 prices.

| Model | Input $ | Output $ | Total $ | Tax $ | Tax as % of bill |
| --- | --- | --- | --- | --- | --- |
| Claude Opus 4.8 | $4,550 | $375 | $4,925 | $3,050 | 62% |
| Claude Sonnet 5 | $1,948 | $150 | $2,098 | $1,348 | 64% |
| Claude Haiku 4.5 | $1,088 | $75 | $1,163 | $788 | 68% |
| Claude Opus 4.7 | $6,520 | $375 | $6,895 | $5,020 | 73% |

On Haiku 4.5, *$788 of every $1,163* per million calls buys no content. It is the toll for structure. Switch to a cheaper model to save money and you carry the toll with you: the fixed tax is a larger share of the smaller bill, not a smaller one.

If you run structured calls at this volume, [put your own schema size and call count through the calculator](https://vynaris.com/calculator) before you assume the model price is what you are paying for.

## When the tax is noise

The tax bites small-output, high-volume calls. It disappears on reasoning-heavy ones. Take the same Opus 4.8 forced-tool call and grow it to 4,000 input and 2,000 output tokens.

| Call shape | Fixed tax | Tax as % of bill |
| --- | --- | --- |
| 300 in / 15 out | 610 tok | 67% of input, 62% of bill |
| 4,000 in / 2,000 out | 610 tok | 4.2% of bill |

Same 610-token tax, two verdicts. On a classification call it is the bill. On a long generation it rounds to nothing. The rule: the tool tax matters in inverse proportion to your output. High call volume with tiny outputs is the danger zone, which is exactly where teams deploy structured output to squeeze cost. The [reasoning-token bill](https://vynaris.com/blog/reasoning-token-tax-worked-bills) is the opposite failure mode, where invisible output tokens dominate; structured output is invisible input.

## The fix: cache the tool prefix

The tool-use system prompt, the named-tool definitions, and your schema are identical on every call. That is a stable prefix, and stable prefixes belong in the cache. Anthropic reads cache hits at 0.1x the input rate (verified 2026-07-19), so a 610-token prefix served from cache bills like 61 tokens.

| Model | Uncached $/1M calls | Tool prefix cached $/1M calls | Saved |
| --- | --- | --- | --- |
| Claude Opus 4.8 | $4,925 | $2,180 | 56% |
| Claude Sonnet 5 | $2,098 | $885 | 58% |
| Claude Haiku 4.5 | $1,163 | $454 | 61% |
| Claude Opus 4.7 | $6,895 | $2,377 | 66% |

Caching does not shrink the tool-use system prompt; it re-prices it. The [break-even hit rate for caching](https://vynaris.com/blog/prompt-caching-break-even-2026) is low here because the prefix is served on almost every call in a high-volume pipeline, so the 1.25x write premium is paid once and amortized across millions of reads. This is the same prefix-caching lever that dominates [what a coding agent costs per task](https://vynaris.com/blog/coding-agent-cost-per-task-6-models); a classification pipeline is just the extreme version, where the prefix is most of the request.

## What it means for routing

- *Measure the tax before you pick the model.* On tiny-output workloads the fixed tool overhead can outweigh the input-price difference between two models. Haiku 4.5 at $1/M input still pays a 588-token forced-tool tax; Opus 4.8 at $5/M pays only 410. Count the fixed tokens, not just the sticker rate.
- *Cache the tool block, always, at volume.* A 56% to 61% cut for one `cache_control` field is the highest-return change on a structured pipeline. If your calls are sporadic and fall outside the 5-minute window, the write premium repeats and the saving shrinks; batch them.
- *Prefer \`auto\` to forced when correctness allows.* The auto tool-use system prompt is smaller than the forced one on every model (290 versus 410 on Opus 4.8). If your prompt reliably elicits the tool call without forcing, you save the difference on every request.

## Where this model is wrong

Honesty first, because a cost model that hides its failure modes is marketing.

- *We only priced the published fixed part.* The `tool_use` and `tool_result` content blocks add per-request tokens that scale with your arguments and returned data. We held those out because they are workload-specific. Your real overhead is the fixed tax plus those blocks, so treat these numbers as the floor.
- *The 200-token schema is a placeholder.* A schema with many fields and long descriptions can run several hundred tokens more. Measure yours with the count-tokens endpoint; the tax scales with it directly.
- [*Model routing*](https://vynaris.com/glossary/model-routing) *across providers is not apples-to-apples here.* OpenAI and Google bill the same category of tokens for [function calling](https://vynaris.com/glossary/category/agents-and-harnesses#function-calling) and JSON schemas, and those tokens are real input on every call. Neither publishes a single fixed per-model system-prompt count the way Anthropic does, so we did not put a fabricated number in the table. The mechanism is identical; only Anthropic's receipt is exact.
- *Anthropic's newer tokenizer bills more tokens for the same text.* Opus 4.8 and Sonnet 5 produce roughly 30% more tokens than earlier models on identical content (provider note, verified 2026-07-19). The fixed tool counts above are already in that tokenizer; your content tokens carry the same [surcharge](https://vynaris.com/blog/anthropic-tokenizer-30-percent-more-tokens-cost).

When is the tax worth paying without a second thought? When your output is large, when you need guaranteed-valid structure a prompt cannot promise, or when call volume is low. The tax is a fixed cost; amortize it over enough output or few enough calls and it stops mattering. It only hurts the high-volume, tiny-output shape, and there the fix is caching, not a cheaper model.

## FAQ

*How many tokens does turning on tools add?* On Claude Opus 4.8, a forced tool call adds a 410-token tool-use system prompt (290 with `auto`), plus your schema, plus any named-tool definitions such as bash (325) or text_editor (700). Verified 2026-07-19 from Anthropic's pricing page.

*Does JSON mode or structured output cost extra?* The output itself bills at the normal output rate, but the tool or schema that enforces the structure adds fixed input tokens on every call. On a 300-in, 15-out call with a 200-token schema, that overhead is 62% to 73% of the bill depending on the model.

*Why is the overhead worse on a cheaper model?* The tool-use system prompt does not shrink with price. Haiku 4.5's forced-tool system prompt is 588 tokens, 43% more than Opus 4.8's 410. On a small call the fixed tax is a larger share of the cheaper model's smaller bill.

*How do I reduce structured-output cost?* Cache the tool prefix; it is identical on every call and reads at 0.1x, cutting the bill 56% to 61%. Prefer `auto` tool choice over forced when correctness allows, and keep schemas tight.

## Sources

- Anthropic pricing, tool-use system-prompt token counts, bash and text_editor tool tokens, cache multipliers, captured 2026-07-19: https://platform.claude.com/docs/en/about-claude/pricing
- OpenAI API pricing (structured output / function calling bills as input tokens), captured 2026-07-19: https://developers.openai.com/api/docs/pricing
- Cost model script and arithmetic: from the assumptions table above.

Prices change. We re-verify every figure in this post monthly and stamp updates. Numbers here are current as of 2026-07-19.

Want the same math on your own workload? Vynaris is an OpenAI-compatible gateway that routes each request to the cheapest right-sized model and shows the per-request cost, including the fixed tool overhead you pay before your prompt is read. One base URL swap. Get an API key at vynaris.com.

## About the author

[Dhrumil Barot](https://vynaris.com/about#founder), Founder. Dhrumil is an AI inference and LLM infrastructure expert. Dhrumil built Vynaris to make model access, routing and pricing simple and transparent for developers. [LinkedIn](https://www.linkedin.com/in/bdhrumil) · [X](https://x.com/fyidhrumil)