Blog · 2026-08-07 · Updated 2026-08-29 · By Dhrumil Barot, Founder

Zero-token agent memory: pricing the 570–18,552 LLM tokens generative systems still spend

Zero-Mem removes 570–18,552 LLM memory-operation tokens per query, but final-QA and encoder costs remain. Reproducible model-by-model math.

Generative agent-memory pipelines spend 569.54 to 18,552.39 LLM tokens on memory operations before the final answer call. On gpt-4o-mini that is $0.162 to $5.287 per 1,000 queries for memory alone. Zero-Mem reports $0 for those operations. The final-QA bill stays. Encoder compute stays. The 57.6% latency cut is not a dollar figure.

Prices verified 2026-08-07.

TL;DR

• Zero-Mem (arXiv 2607.29377) reports zero LLM calls and zero LLM tokens for every memory step outside final QA. Table 2 lists LightMem at 569.54 tokens/query, SimpleMem at 9,153.41, GAM at 18,552.39, and Zero-Mem at 0.
• Pricing those totals at a 70% input / 30% output split on gpt-4o-mini ($0.15 / $0.60 per 1M) yields $0.162, $2.609, $5.287, and $0.000 per 1,000 queries for memory operations only.
• The same GAM token bill on Sonnet 5 introductory pricing ($2 / $10) is $81.631 per 1,000 queries. On deepseek-v4-flash it is $3.377.
• A shared final-QA shape of 4,200 input and 150 output tokens still costs $0.720 per 1,000 queries on gpt-4o-mini for every method. Zero-token memory does not delete the reader.
• Code is not public yet (promised after peer review). Treat the token counts as paper-reported, not independently re-run.

The verdict table

Token counts come from Zero-Mem Table 2 under a unified GPT-4o-mini backbone. Dollar columns apply live OpenAI gpt-4o-mini rates with an editable 70/30 input/output split because the paper publishes total tokens, not the split.

What we computed and why

Most agent-memory cost talk collapses three bills into one slogan: “memory tokens.” They are not the same line item.

1. Memory-operation LLM tokens — summarize, extract, reflect, link, rewrite, or plan retrieval with a generative model. This is the bill Zero-Mem sets to zero.
2. Final-QA LLM tokens — the reader that answers from retrieved evidence. Identical reader and context budget across methods in the paper.
3. Encoder / index compute — NER, BM25, dense embeddings such as BGE-M3, graph updates. Not LLM tokens. Not free compute. Separate from (1).

We price (1) from Table 2, keep (2) as a shared editable shape, and bound (3) with a public embedding rate only as a ceiling illustration. We do not turn the paper’s 57.6% memory-operation time reduction into USD. Time is time.

Assumptions table (edit these)

If your agent writes memory on every turn and answers less often, multiply the memory-op line by writes-per-answer. The paper’s efficiency table is query-centric.

Results: memory-operation dollars on gpt-4o-mini

gpt-4o-mini is the paper’s closed-source backbone: $0.15 per 1M input token cost and $0.60 per 1M output token cost.

At 70% input / 30% output:

Sensitivity on GAM alone (same 18,552.39 tokens):

The split moves the bill. It does not restore generative memory to Zero-Mem’s $0 memory-op line.

Same tokens, different stickers

Keep GAM’s 18,552.39 memory tokens fixed. Change only the model rate card.

That is a 24.2× spread ($3.377 → $81.631) on an unchanged token count. Model routing for memory writes is a real lever. Deleting the writes is a larger one when the write path is generative.

Put the shared final-QA shape into the calculator before you compare stacks. Memory-op savings only matter relative to the reader you still pay.

Final QA stays on the bill

Under our editable reader shape (4,200 in / 150 out):

Stack on gpt-4o-mini (memory ops + final QA; encoder excluded from the total):

For LightMem-class pipelines, the reader dominates. For GAM-class generative memory, memory ops dominate. Zero-Mem’s cost claim is specifically about that middle column.

Encoder compute is not “zero cost”

Zero-Mem builds an entity–context graph (non-generative NER such as spaCy), a temporal hierarchy, BM25, and dense access with BGE-M3. The paper is explicit: encoder computation is accounted for separately from LLM tokens.

As a public API bound, 800 embedding tokens on text-embedding-3-small ($0.02/1M) is $0.016 per 1,000 queries. That bound is tiny next to GAM’s $5.287 memory-op line on gpt-4o-mini. It is also not Zero-Mem’s production bill if BGE-M3 runs on your GPUs. Self-hosted encoders trade GPU/CPU time and index maintenance for LLM stickers. Budget them. Do not call them free because they are not chat completions.

How to decide: generative memory vs deterministic retrieval

Use this as an implementation checklist, not a brand preference.

1. Meter memory ops separately. Log LLM tokens (and dollars) for write/update/reflect/retrieve-planning calls apart from final QA. If you only see one meter, you cannot apply this table. Our dollars-per-call how-to covers the field layout.
2. Compare against Table 2’s shape. If your generative memory path is near LightMem’s ~570 tokens/query, the dollar win from deleting it on gpt-4o-mini is $0.162/1,000 queries. If it looks like GAM’s ~18.6k, the win is $5.287 on gpt-4o-mini and $81.631 on Sonnet 5.
3. Keep provenance. Zero-Mem’s design argument is that LLM-written summaries can omit or merge evidence. Cost and auditability move together when the source of record stays the original trace.
4. Preserve a [context window](https://vynaris.com/glossary/category/inference-and-serving#context-window) budget for the reader. Deterministic retrieval still injects evidence tokens into final QA. Shrinking memory-op tokens while bloating retrieved context can erase the win.
5. Watch write amplification. Agents that call a memory LLM on every tool observation pay the memory line far more often than the QA line. Measure writes per answer.
6. Do not budget the 57.6% latency figure as cash. Table 2 reports LightMem at 0.51 s/query and Zero-Mem at 0.22 s/query for memory operations ((0.51 − 0.22) / 0.51 ≈ 0.569; the paper states 57.6%). Latency can justify engineering time. It is not a receipt for $/query.

Honest tradeoff: when generative memory still wins

Do not rip out LLM memory writes if your product needs generated abstractions as the product surface — for example, user-visible “memories,” coach-style reflections, or compressed narratives you show in a UI. Deterministic retrieval returns traces; it does not invent a clean paragraph unless the reader does it at answer time (which then lands on the final-QA bill).

Also skip a Zero-Mem-style rebuild when memory-op tokens are already near the LightMem floor and your encoder/index ops cost more engineering hours than the $0.162/1,000 you would save on gpt-4o-mini. Cheap generative memory plus prompt caching on stable prefixes can beat a half-finished graph project.

Reproducibility caveat: the authors say code and implementation details land after peer review in the paper. Until then, treat Table 2 as a primary-source measurement under their harness, not a drop-in library you can re-bill tonight.

FAQ

Does “zero-token memory” mean the agent is free?

No. Final QA still bills input/output token pricing. Encoders, NER, and indexes still burn compute.

Why assume 70/30 for the token split?

The paper reports total memory-operation tokens. 70/30 is a disclosed assumption. The sensitivity table shows the range.

Can I convert the 57.6% time cut into dollars?

No. We refuse that conversion. GPU utilization, batching, and concurrency decide whether wall-clock maps to rent. The paper does not publish that mapping.

How does this relate to forecasting agent workflows?

Call-graph growth and re-sent history often dominate long agents. Memory-op deletion is one lever; session shape is another. See our pre-build agent cost forecast.

Sources

• Zero-Mem: Zero-Token Memory Operations for LLM Agents (arXiv 2607.29377), captured 2026-08-07: Table 2 token and time figures; zero LLM tokens outside final QA; code after peer review.
• OpenAI API pricing, captured 2026-08-07: gpt-4o-mini $0.15/$0.60; gpt-5.6-luna $0.20/$1.20; text-embedding-3-small $0.02/1M.
• DeepSeek pricing, captured 2026-08-07: deepseek-v4-flash $0.14/$0.28 (cache miss).
• Gemini API pricing, captured 2026-08-07: gemini-3.5-flash-lite $0.30/$2.50 standard.
• Anthropic pricing, re-verified 2026-08-14: Haiku 4.5 $1/$5; Sonnet 5 $2/$10 standard after Anthropic canceled the scheduled increase.

All arithmetic is in the accompanying script. No Vynaris traffic, customer, or routing data is used.

Further reading: Lemmalog’s extraction-cost break-even prices the repeated reader-stage saving across two public memory benchmarks without inventing the one-time extraction bill.