Blog · 2026-07-23 · Vynaris Team

OpenAI-compatible APIs: what one base_url swap gets you, and the 3.5x caching tax

Pointing the OpenAI SDK at Claude, DeepSeek, or Gemini takes three lines. Chat and streaming map cleanly; prompt caching, response_format, and usage detail objects silently break. On a cache-heavy agent workload, losing caching alone re-bills the same tokens 3.5x. Prices verified 2026-07-23.

Pointing the OpenAI SDK at Anthropic, DeepSeek, or Gemini takes three lines: a new base URL, a new key, a new model name. The chat and streaming paths map cleanly. What the swap does not carry over is where the money hides. Anthropic's compatibility layer does not support prompt caching, silently ignores response_format and tool strict, and returns empty usage detail objects. On a cache-heavy agent workload, losing caching alone re-bills the same tokens 3.5x. Prices and provider docs verified 2026-07-23.

TL;DR

The swap is real: client = OpenAI(base_url=..., api_key=...), then a provider model name. Chat completions, streaming, and tool calls map cleanly across Anthropic, DeepSeek, and Gemini OpenAI-compatible endpoints.
What silently breaks is the cost machinery. On the Anthropic compat layer, prompt caching is unsupported, so a 20,000-token cached system prompt reused across a 10-turn session costs $0.22 instead of $0.063 on Haiku 4.5 — 3.5x, and 4.3x at a 100,000-token prompt. Same model, same tokens, one base URL.
The usage.prompt_tokens_details and usage.completion_tokens_details objects are always empty on the Anthropic compat endpoint, so any cost meter that reads cached or reasoning-token counts from them reports zero and undercounts the breakdown. Verified against provider docs 2026-07-23.

The swap, in three lines

The OpenAI-compatible endpoint is a real convenience. You keep the OpenAI SDK, its retries, and its streaming parser, and change only the connection:

from openai import OpenAI

client = OpenAI(
    api_key="ANTHROPIC_API_KEY",
    base_url="https://api.anthropic.com/v1/",   # DeepSeek: https://api.deepseek.com
)                                                # Gemini: .../v1beta/openai/
resp = client.chat.completions.create(
    model="claude-opus-4-8",                     # provider's own model names
    messages=[{"role": "user", "content": "Who are you?"}],
)

DeepSeek is natively OpenAI-compatible; its endpoint is https://api.deepseek.com with deepseek-v4-flash or deepseek-v4-pro. Gemini exposes https://generativelanguage.googleapis.com/v1beta/openai/ with names like gemini-3.6-flash. Anthropic uses https://api.anthropic.com/v1/ with Claude model names. The request bodies you already send mostly work. The failures are in what you stop getting back.

What maps cleanly

For the common request shape, the mapping is honest. From the Anthropic compatibility reference (captured 2026-07-23), these are fully supported:

OpenAI field                                                  Status on Anthropic compat
------------------------------------------------------------  --------------------------
`messages` (system/user/assistant/tool)                       Supported
`model`, `max_tokens`, `max_completion_tokens`                Supported
`stream`, `stream_options`                                    Supported
`tools[].function` name/description/parameters                Supported
`tool_calls` in the response                                  Supported
`top_p`, `stop`, `parallel_tool_calls`                        Supported
`usage.prompt_tokens` / `completion_tokens` / `total_tokens`  Supported
`x-ratelimit-*` and `retry-after` headers                     Supported

If your app sends plain chat requests, streams tokens, and calls tools, a base URL swap will run. This is why the swap feels safe: the happy path is genuinely covered, and the total-token counts you bill against are correct. The problem is the fields you relied on for cost control, not the ones you relied on to get an answer.

One field is easy to forget when you stream. OpenAI's SDK omits the usage object entirely on streamed responses unless you pass stream_options={"include_usage": true}, which asks for a final chunk carrying the token counts. That flag is supported on the Anthropic compat layer, so set it or a streaming migration silently loses even the top-line total_tokens it was billing against. The catch: the counts you get back are the correct totals, but the prompt_tokens_details and completion_tokens_details sub-objects inside that final chunk are still empty, which is the leak covered below.

What silently breaks

The Anthropic docs state it plainly: "most unsupported fields are silently ignored rather than producing errors." Gemini's compatibility layer uses the same wording: unsupported OpenAI parameters are "silently ignored." Silent is the operative word. Your code runs, returns a 200, and quietly drops the feature you were paying attention to.

What you sent                      What happens on Anthropic compat              Why it costs you
---------------------------------  --------------------------------------------  --------------------------------------
Prompt caching (`cache_control`)   Unsupported; no caching                       Full input re-billed every turn
`response_format` (JSON mode)      Ignored                                       Free-form output; repair retries
tool `strict`                      Ignored; schema not guaranteed                Malformed tool JSON; repair retries
`usage.prompt_tokens_details`      Always empty                                  Cost meter can't see cached tokens
`usage.completion_tokens_details`  Always empty                                  Cost meter can't see reasoning tokens
`seed`, `logprobs`, `logit_bias`   Ignored / empty                               Reproducibility and eval tooling break
`audio` input, `n` > 1             Stripped; `n` must be 1                       Silent behavior change
system/developer messages          Concatenated into one leading system message  Prompt structure changes

None of these throw. That is the trap: a "drop-in" swap that returns valid responses while dropping the exact mechanisms that keep your bill down. The two that move real money are caching and the empty usage detail objects.

The caching tax, computed

Prompt caching is the single biggest lever on an agent bill, because a large system-and-tools prompt is re-sent on every turn. Native, that prompt is written to cache once (1.25x input) and read back at 0.1x input on every later turn. Through the OpenAI-compat endpoint, Anthropic does not cache at all, so that prompt is billed at full input rate every turn.

Take an agent session: a 20,000-token cached system/tools prompt, 500 fresh input tokens and 300 output tokens per turn, 10 turns. Prices verified 2026-07-23.

Model      Native (cached) per session  Compat (no cache) per session  Ratio  At 10,000 sessions/mo
---------  ---------------------------  -----------------------------  -----  ---------------------
Haiku 4.5  $0.0630                      $0.2200                        3.49x  +$1,570/mo
Opus 4.8   $0.3150                      $1.1000                        3.49x  +$7,850/mo

The multiple is identical because caching scales with the input rate, not the model. It also grows with the size of the reused prompt: the more you would have cached, the more the swap costs you.

Cached prompt tokens  Native per session (Haiku 4.5)  Compat per session  Caching tax
--------------------  ------------------------------  ------------------  -----------
2,000                 $0.0243                         $0.0400             1.65x
10,000                $0.0415                         $0.1200             2.89x
20,000                $0.0630                         $0.2200             3.49x
50,000                $0.1275                         $0.5200             4.08x
100,000               $0.2350                         $1.0200             4.34x

A 100,000-token tool-and-context prompt, the kind a real agent carries, is re-billed 4.3x once caching silently drops. Run your own prompt size and turn count through the calculator before you assume a compat swap is free.

Horizontal bar chart, single blue hue, of the caching tax as a multiple of the native cached bill by cached-prompt size for Haiku 4.5 over a 10-turn session: 2,000 tokens 1.65x, 5,000 2.28x, 10,000 2.89x, 20,000 3.49x, 50,000 4.08x, 100,000 4.34x — The caching tax of a base_url swap that drops prompt caching, by cached-prompt size. Same model, same tokens, one base_url. Prices verified 2026-07-23.

The gap widens further for long-lived agents. Anthropic's native API offers a 1-hour cache tier (2x input to write, still 0.1x to read) alongside the default 5-minute tier used above. A support bot or a coding agent that keeps the same tools-and-context prompt warm across a whole shift reads it back at 0.1x for an hour on the native SDK. Through the compat layer there is no cache tier at all, so the longer the prompt stays reusable, the larger the bill the swap leaves on the table.

The fix on Anthropic is not the compat layer: caching works on the native Anthropic SDK. On Gemini, caching is reachable through the compat layer only via the extra_body field, not the standard OpenAI parameter, so a portable OpenAI-only request skips it there too.

The metering blindness

The second money leak is quieter. On the Anthropic compat endpoint, usage.prompt_tokens_details and usage.completion_tokens_details are always empty. Those are exactly the objects a cost meter reads to break out cached input tokens and reasoning output tokens. Your total_tokens is correct, so your top-line bill is right, but any per-call attribution that trusts the detail objects reports zero cached and zero reasoning tokens.

This matters most for teams migrating off OpenAI, where those detail objects are populated and their meter learned to trust them. Point the same meter at Claude via compat and it silently loses the cache and reasoning lines. We built the honest version of this meter in metering dollars per call; the short version is to price from the provider's own cost-per-token rates and turn structure, not from optional detail fields a compat layer may leave empty.

The structured-output retry tax

response_format and tool strict are both ignored on the Anthropic compat layer, so JSON mode and guaranteed schema conformance are gone. Your parser now meets free-form text or loosely-shaped tool arguments. Each malformed response that needs a repair retry re-bills the whole call.

Malformed rate on tool/JSON calls  Effective cost on that path
---------------------------------  ---------------------------
5%                                 +5%
10%                                +10%
20%                                +20%

The percentages are illustrative, not measured, but the mechanism is exact: a dropped strict turns a guaranteed-parse call into a probabilistic one, and the failures are re-billed. For guaranteed schema conformance, Anthropic points you to Structured Outputs on the native API, which the compat layer cannot reach.

How this differs from routing and gateway posts

This is OpenAI-compatible SDK mechanics, not routing economics. It is not about a markup a gateway charges, which we audited in what six AI gateways actually charge, and not about model-routing which model to pick per call. It is the layer underneath both: what a single base URL swap silently changes about your bill before any routing or gateway sits on top. The per-model list prices behind every number here are the same ones tracked in the July 2026 cost-per-task pricing table.

When the swap is fine

Testing and evaluation. For comparing model quality with code you already have, the compat layer is exactly what Anthropic recommends it for. Nothing below production-cost scale is at risk.
Cache-light workloads. If you have no large reusable prompt (short one-shot calls, no shared system-and-tools block), the caching tax is near zero and the swap costs you nothing on that axis.
You bill from total tokens, not detail objects. If your meter prices from total_tokens and your own knowledge of the request shape, the empty detail objects do not hurt you.
DeepSeek as the target. DeepSeek is natively OpenAI-compatible, so the compat path is its normal path, not a lossy bridge. Watch the 2026-07-24 deprecation of the deepseek-chat and deepseek-reasoner names in favor of deepseek-v4-flash and deepseek-v4-pro.

Doing this on Vynaris

The DIY path is clear: for anything cost-sensitive on Anthropic, use the native SDK so caching and Structured Outputs work; on Gemini, reach caching through extra_body; and meter from total tokens plus request structure, not optional detail fields. This section is the paid shortcut, clearly labeled. Vynaris is an OpenAI-compatible gateway that keeps one base URL and one SDK while preserving prompt caching and the per-call cost breakdown across providers, so the swap does not silently drop the mechanisms that control your bill. See the docs and pricing.

FAQ

Does the OpenAI SDK work with Claude, DeepSeek, and Gemini? Yes. Each exposes an OpenAI-compatible endpoint: change the base URL, the API key, and the model name. Chat completions, streaming, and tool calls map cleanly. The gaps are in provider-specific features and, on Anthropic, in prompt caching and the usage detail objects.

Does prompt caching work through the OpenAI-compatible endpoint? Not on Anthropic — its docs list prompt caching as unsupported on the compat layer; use the native Anthropic SDK. On Gemini, caching is reachable only via the extra_body field, not a standard OpenAI parameter. Dropping caching re-bills a 20,000-token cached prompt 3.5x on a 10-turn session. Prices verified 2026-07-23.

Why did my bill go up after a "drop-in" base_url swap? The most likely cause is lost prompt caching: the same tokens are billed at full input rate every turn instead of 0.1x on cache reads. Repair retries from an ignored response_format or strict add more. None of these throw errors, so the only signal is the invoice.

Can I still meter cost per call after the swap? Yes, but not from usage.prompt_tokens_details or usage.completion_tokens_details on the Anthropic compat endpoint — they are always empty. Price from total_tokens split by your known input/output shape and the provider's live rates instead.

Which fields are silently ignored on the Anthropic compat layer? Among others: response_format, tool strict, seed, logprobs, logit_bias, presence_penalty, frequency_penalty, reasoning_effort, service_tier, and metadata. n must be 1, temperature above 1 is capped, and system/developer messages are concatenated into one leading system message.

Sources

Anthropic OpenAI SDK compatibility (supported/ignored fields, caching unsupported, empty usage details), captured 2026-07-23: https://platform.claude.com/docs/en/api/openai-sdk
Anthropic pricing (Haiku 4.5 $1/$5, Opus 4.8 $5/$25, cache read 0.1x, 5m write 1.25x), captured 2026-07-23: https://platform.claude.com/docs/en/about-claude/pricing
Google Gemini OpenAI compatibility (base URL, "unsupported parameters silently ignored", caching via extra_body), captured 2026-07-23: https://ai.google.dev/gemini-api/docs/openai
DeepSeek API docs (native OpenAI compatibility, model names, 2026-07-24 name deprecation), captured 2026-07-23: https://api-docs.deepseek.com/
Cost model script and per-session arithmetic: derived from the assumptions above.

Prices change. We re-verify every figure in this post monthly and stamp updates. Numbers here are current as of 2026-07-23.

The lesson: a base URL swap is a convenience, not a free lunch. It carries the answer and drops the accounting. Before you ship one to production, confirm caching still works, that your meter does not depend on the detail objects, and that guaranteed JSON is not something you quietly gave up.