Blog · 2026-08-07 · Vynaris Team
Zero-token agent memory: pricing the 570–18,552 LLM tokens generative systems still spend
Zero-Mem removes 570–18,552 LLM memory-operation tokens per query, but final-QA and encoder costs remain. Reproducible model-by-model math.
Generative agent-memory pipelines spend 569.54 to 18,552.39 LLM tokens on memory operations before the final answer call. On gpt-4o-mini that is $0.162 to $5.287 per 1,000 queries for memory alone. Zero-Mem reports $0 for those operations. The final-QA bill stays. Encoder compute stays. The 57.6% latency cut is not a dollar figure.
Prices verified 2026-08-07.
TL;DR
- Zero-Mem (arXiv 2607.29377) reports zero LLM calls and zero LLM tokens for every memory step outside final QA. Table 2 lists LightMem at 569.54 tokens/query, SimpleMem at 9,153.41, GAM at 18,552.39, and Zero-Mem at 0.
- Pricing those totals at a 70% input / 30% output split on gpt-4o-mini ($0.15 / $0.60 per 1M) yields $0.162, $2.609, $5.287, and $0.000 per 1,000 queries for memory operations only.
- The same GAM token bill on Sonnet 5 introductory pricing ($2 / $10) is $81.631 per 1,000 queries. On deepseek-v4-flash it is $3.377.
- A shared final-QA shape of 4,200 input and 150 output tokens still costs $0.720 per 1,000 queries on gpt-4o-mini for every method. Zero-token memory does not delete the reader.
- Code is not public yet (promised after peer review). Treat the token counts as paper-reported, not independently re-run.
The verdict table
Memory system LLM tokens / query (memory ops) Memory-op $/1,000 queries on gpt-4o-mini What is still billed
------------- ------------------------------- ---------------------------------------- --------------------------------
LightMem 569.54 $0.162 Final QA + encoder/index compute
SimpleMem 9,153.41 $2.609 Final QA + encoder/index compute
GAM 18,552.39 $5.287 Final QA + encoder/index compute
Zero-Mem 0 $0.000 Final QA + encoder/index computeToken counts come from Zero-Mem Table 2 under a unified GPT-4o-mini backbone. Dollar columns apply live OpenAI gpt-4o-mini rates with an editable 70/30 input/output split because the paper publishes total tokens, not the split.
What we computed and why
Most agent-memory cost talk collapses three bills into one slogan: “memory tokens.” They are not the same line item.
- Memory-operation LLM tokens — summarize, extract, reflect, link, rewrite, or plan retrieval with a generative model. This is the bill Zero-Mem sets to zero.
- Final-QA LLM tokens — the reader that answers from retrieved evidence. Identical reader and context budget across methods in the paper.
- Encoder / index compute — NER, BM25, dense embeddings such as BGE-M3, graph updates. Not LLM tokens. Not free compute. Separate from (1).
We price (1) from Table 2, keep (2) as a shared editable shape, and bound (3) with a public embedding rate only as a ceiling illustration. We do not turn the paper’s 57.6% memory-operation time reduction into USD. Time is time.
Assumptions table (edit these)
Assumption Value used Why it is editable
---------------------------- ---------------------------------------------------------------------------------------------------------- ------------------------------------------------------------------
Memory tokens/query LightMem 569.54; SimpleMem 9,153.41; GAM 18,552.39; Zero-Mem 0 Paper Table 2 totals
Input share of memory tokens 70% (sensitivity 90% / 50%) Paper does not publish in/out split
Memory model prices Live provider $/1M, verified 2026-08-07 Sticker rates change
Final-QA shape 4,200 input + 150 output Same reader/budget across methods; not a measured production trace
Encoder API bound 800 embedding tokens × $0.02/1M ([text-embedding-3-small](https://developers.openai.com/api/docs/pricing)) Paper uses BGE-M3; this is a public price bound, not their bill
Queries priced Per 1,000 memory-evaluated queries Matches Table 2’s per-query framingIf your agent writes memory on every turn and answers less often, multiply the memory-op line by writes-per-answer. The paper’s efficiency table is query-centric.
Results: memory-operation dollars on gpt-4o-mini
gpt-4o-mini is the paper’s closed-source backbone: $0.15 per 1M input token cost and $0.60 per 1M output token cost.
At 70% input / 30% output:
Method Input tokens Output tokens $/query $/1,000 queries
--------- ------------ ------------- --------- ---------------
LightMem 398.678 170.862 $0.000162 $0.162
SimpleMem 6,407.387 2,746.023 $0.002609 $2.609
GAM 12,986.673 5,565.717 $0.005287 $5.287
Zero-Mem 0 0 $0.000000 $0.000
Sensitivity on GAM alone (same 18,552.39 tokens):
Input share GAM $/1,000 on gpt-4o-mini
----------- --------------------------
90% input $3.618
70% input $5.287
50% input $6.957The split moves the bill. It does not restore generative memory to Zero-Mem’s $0 memory-op line.
Same tokens, different stickers
Keep GAM’s 18,552.39 memory tokens fixed. Change only the model rate card.
Model Input / output $/1M GAM memory-op $/1,000 queries
------------------------------------------------------------------------- ------------------- -----------------------------
deepseek-v4-flash $0.14 / $0.28 $3.377
gpt-4o-mini $0.15 / $0.60 $5.287
[gpt-5.6-luna](https://vynaris.com/models#gpt-5-6-luna) $0.20 / $1.20 $9.276
[gemini-3.5-flash-lite](https://vynaris.com/models#gemini-3-5-flash-lite) $0.30 / $2.50 $17.810
[Haiku 4.5](https://vynaris.com/models#claude-haiku-4-5) $1.00 / $5.00 $40.815
Sonnet 5 $2.00 / $10.00 $81.631
That is a 24.2× spread ($3.377 → $81.631) on an unchanged token count. Model routing for memory writes is a real lever. Deleting the writes is a larger one when the write path is generative.
Put the shared final-QA shape into the calculator before you compare stacks. Memory-op savings only matter relative to the reader you still pay.
Final QA stays on the bill
Under our editable reader shape (4,200 in / 150 out):
Model Final-QA $/1,000 queries
--------------------- ------------------------
deepseek-v4-flash $0.630
gpt-4o-mini $0.720
gpt-5.6-luna $1.020
gemini-3.5-flash-lite $1.635
Haiku 4.5 $4.950
Sonnet 5 $9.900Stack on gpt-4o-mini (memory ops + final QA; encoder excluded from the total):
Method Memory ops Final QA LLM total / 1,000 queries Memory share of LLM bill
--------- ---------- -------- ------------------------- ------------------------
LightMem $0.162 $0.720 $0.882 18.4%
SimpleMem $2.609 $0.720 $3.329 78.4%
GAM $5.287 $0.720 $6.007 88.0%
Zero-Mem $0.000 $0.720 $0.720 0.0%For LightMem-class pipelines, the reader dominates. For GAM-class generative memory, memory ops dominate. Zero-Mem’s cost claim is specifically about that middle column.
Encoder compute is not “zero cost”
Zero-Mem builds an entity–context graph (non-generative NER such as spaCy), a temporal hierarchy, BM25, and dense access with BGE-M3. The paper is explicit: encoder computation is accounted for separately from LLM tokens.
As a public API bound, 800 embedding tokens on text-embedding-3-small ($0.02/1M) is $0.016 per 1,000 queries. That bound is tiny next to GAM’s $5.287 memory-op line on gpt-4o-mini. It is also not Zero-Mem’s production bill if BGE-M3 runs on your GPUs. Self-hosted encoders trade GPU/CPU time and index maintenance for LLM stickers. Budget them. Do not call them free because they are not chat completions.
How to decide: generative memory vs deterministic retrieval
Use this as an implementation checklist, not a brand preference.
- Meter memory ops separately. Log LLM tokens (and dollars) for write/update/reflect/retrieve-planning calls apart from final QA. If you only see one meter, you cannot apply this table. Our dollars-per-call how-to covers the field layout.
- Compare against Table 2’s shape. If your generative memory path is near LightMem’s ~570 tokens/query, the dollar win from deleting it on gpt-4o-mini is $0.162/1,000 queries. If it looks like GAM’s ~18.6k, the win is $5.287 on gpt-4o-mini and $81.631 on Sonnet 5.
- Keep provenance. Zero-Mem’s design argument is that LLM-written summaries can omit or merge evidence. Cost and auditability move together when the source of record stays the original trace.
- Preserve a [context window](https://vynaris.com/glossary/category/inference-and-serving#context-window) budget for the reader. Deterministic retrieval still injects evidence tokens into final QA. Shrinking memory-op tokens while bloating retrieved context can erase the win.
- Watch write amplification. Agents that call a memory LLM on every tool observation pay the memory line far more often than the QA line. Measure writes per answer.
- Do not budget the 57.6% latency figure as cash. Table 2 reports LightMem at 0.51 s/query and Zero-Mem at 0.22 s/query for memory operations ((0.51 − 0.22) / 0.51 ≈ 0.569; the paper states 57.6%). Latency can justify engineering time. It is not a receipt for $/query.
Honest tradeoff: when generative memory still wins
Do not rip out LLM memory writes if your product needs generated abstractions as the product surface — for example, user-visible “memories,” coach-style reflections, or compressed narratives you show in a UI. Deterministic retrieval returns traces; it does not invent a clean paragraph unless the reader does it at answer time (which then lands on the final-QA bill).
Also skip a Zero-Mem-style rebuild when memory-op tokens are already near the LightMem floor and your encoder/index ops cost more engineering hours than the $0.162/1,000 you would save on gpt-4o-mini. Cheap generative memory plus prompt caching on stable prefixes can beat a half-finished graph project.
Reproducibility caveat: the authors say code and implementation details land after peer review in the paper. Until then, treat Table 2 as a primary-source measurement under their harness, not a drop-in library you can re-bill tonight.
FAQ
Does “zero-token memory” mean the agent is free?
No. Final QA still bills input/output token pricing. Encoders, NER, and indexes still burn compute.
Why assume 70/30 for the token split?
The paper reports total memory-operation tokens. 70/30 is a disclosed assumption. The sensitivity table shows the range.
Can I convert the 57.6% time cut into dollars?
No. We refuse that conversion. GPU utilization, batching, and concurrency decide whether wall-clock maps to rent. The paper does not publish that mapping.
How does this relate to forecasting agent workflows?
Call-graph growth and re-sent history often dominate long agents. Memory-op deletion is one lever; session shape is another. See our pre-build agent cost forecast.
Sources
- Zero-Mem: Zero-Token Memory Operations for LLM Agents (arXiv 2607.29377), captured 2026-08-07: Table 2 token and time figures; zero LLM tokens outside final QA; code after peer review.
- OpenAI API pricing, captured 2026-08-07: gpt-4o-mini $0.15/$0.60; gpt-5.6-luna $0.20/$1.20; text-embedding-3-small $0.02/1M.
- DeepSeek pricing, captured 2026-08-07: deepseek-v4-flash $0.14/$0.28 (cache miss).
- Gemini API pricing, captured 2026-08-07: gemini-3.5-flash-lite $0.30/$2.50 standard.
- Anthropic pricing, re-verified 2026-08-14: Haiku 4.5 $1/$5; Sonnet 5 $2/$10 standard after Anthropic canceled the scheduled increase.
All arithmetic is in the accompanying script. No Vynaris traffic, customer, or routing data is used.
Further reading: Lemmalog’s extraction-cost break-even prices the repeated reader-stage saving across two public memory benchmarks without inventing the one-time extraction bill.