---
title: "Zero-token agent memory: pricing the 570–18,552 LLM tokens generative systems still spend"
description: "Zero-Mem removes 570–18,552 LLM memory-operation tokens per query, but final-QA and encoder costs remain. Reproducible model-by-model math."
canonical: https://vynaris.com/blog/zero-token-agent-memory-cost-vs-llm-written-memory
author: "Dhrumil Barot"
author_bio: "Dhrumil is an AI inference and LLM infrastructure expert. Dhrumil built Vynaris to make model access, routing and pricing simple and transparent for developers."
published: 2026-08-07
updated: 2026-08-29
---

# Zero-token agent memory: pricing the 570–18,552 LLM tokens generative systems still spend

Published 2026-08-07 · Updated 2026-08-29 · By [Dhrumil Barot](https://vynaris.com/about#founder), Founder

Zero-Mem removes 570–18,552 LLM memory-operation tokens per query, but final-QA and encoder costs remain. Reproducible model-by-model math.

Generative agent-memory pipelines spend 569.54 to 18,552.39 [LLM](https://vynaris.com/glossary/category/inference-and-serving#large-language-model) tokens on memory operations before the final answer call. On [gpt-4o-mini](https://vynaris.com/models#gpt-4o-mini) that is $0.162 to $5.287 per 1,000 queries for memory alone. Zero-Mem reports $0 for those operations. The final-QA bill stays. Encoder compute stays. The 57.6% latency cut is not a dollar figure.

Prices verified 2026-08-07.

## TL;DR

- Zero-Mem (arXiv 2607.29377) reports zero LLM calls and zero LLM tokens for every memory step outside final QA. Table 2 lists LightMem at 569.54 tokens/query, SimpleMem at 9,153.41, GAM at 18,552.39, and Zero-Mem at 0.
- Pricing those totals at a 70% input / 30% output split on gpt-4o-mini ($0.15 / $0.60 per 1M) yields $0.162, $2.609, $5.287, and $0.000 per 1,000 queries for memory operations only.
- The same GAM token bill on [Sonnet 5](https://vynaris.com/models#claude-sonnet-5) introductory pricing ($2 / $10) is $81.631 per 1,000 queries. On [deepseek-v4-flash](https://vynaris.com/models#deepseek-v4-flash) it is $3.377.
- A shared final-QA shape of 4,200 input and 150 output tokens still costs $0.720 per 1,000 queries on gpt-4o-mini for every method. Zero-token memory does not delete the reader.
- Code is not public yet (promised after peer review). Treat the token counts as paper-reported, not independently re-run.

## The verdict table

| Memory system | LLM tokens / query (memory ops) | Memory-op $/1,000 queries on gpt-4o-mini | What is still billed |
| --- | --- | --- | --- |
| LightMem | 569.54 | $0.162 | Final QA + encoder/index compute |
| SimpleMem | 9,153.41 | $2.609 | Final QA + encoder/index compute |
| GAM | 18,552.39 | $5.287 | Final QA + encoder/index compute |
| Zero-Mem | 0 | $0.000 | Final QA + encoder/index compute |

Token counts come from Zero-Mem Table 2 under a unified GPT-4o-mini backbone. Dollar columns apply live OpenAI gpt-4o-mini rates with an editable 70/30 input/output split because the paper publishes total tokens, not the split.

## What we computed and why

Most agent-memory cost talk collapses three bills into one slogan: “memory tokens.” They are not the same line item.

1. **Memory-operation LLM tokens** — summarize, extract, reflect, link, rewrite, or plan retrieval with a generative model. This is the bill Zero-Mem sets to zero.
2. **Final-QA LLM tokens** — the reader that answers from retrieved evidence. Identical reader and context budget across methods in the paper.
3. **Encoder / index compute** — NER, BM25, dense embeddings such as BGE-M3, graph updates. Not LLM tokens. Not free compute. Separate from (1).

We price (1) from Table 2, keep (2) as a shared editable shape, and bound (3) with a public embedding rate only as a ceiling illustration. We do **not** turn the paper’s 57.6% memory-operation *time* reduction into USD. Time is time.

## Assumptions table (edit these)

| Assumption | Value used | Why it is editable |
| --- | --- | --- |
| Memory tokens/query | LightMem 569.54; SimpleMem 9,153.41; GAM 18,552.39; Zero-Mem 0 | Paper Table 2 totals |
| Input share of memory tokens | 70% (sensitivity 90% / 50%) | Paper does not publish in/out split |
| Memory model prices | Live provider $/1M, verified 2026-08-07 | Sticker rates change |
| Final-QA shape | 4,200 input + 150 output | Same reader/budget across methods; not a measured production trace |
| Encoder API bound | 800 embedding tokens × $0.02/1M ([text-embedding-3-small](https://developers.openai.com/api/docs/pricing)) | Paper uses BGE-M3; this is a public price bound, not their bill |
| Queries priced | Per 1,000 memory-evaluated queries | Matches Table 2’s per-query framing |

If your agent writes memory on every turn and answers less often, multiply the memory-op line by writes-per-answer. The paper’s efficiency table is query-centric.

## Results: memory-operation dollars on gpt-4o-mini

gpt-4o-mini is the paper’s closed-source backbone: $0.15 per 1M [input token cost](https://vynaris.com/glossary/category/cost-and-finops#input-token-cost) and $0.60 per 1M [output token cost](https://vynaris.com/glossary/category/cost-and-finops#output-token-cost).

At 70% input / 30% output:

| Method | Input tokens | Output tokens | $/query | $/1,000 queries |
| --- | --- | --- | --- | --- |
| LightMem | 398.678 | 170.862 | $0.000162 | $0.162 |
| SimpleMem | 6,407.387 | 2,746.023 | $0.002609 | $2.609 |
| GAM | 12,986.673 | 5,565.717 | $0.005287 | $5.287 |
| Zero-Mem | 0 | 0 | $0.000000 | $0.000 |

![Bar chart of memory-operation LLM cost per 1,000 queries on gpt-4o-mini for LightMem, SimpleMem, GAM, and Zero-Mem.](https://cdn.sanity.io/images/u1qdn8dx/production/2bd3726332db89c48b9bc34e21a8c4c6277b5901-1260x700.png?w=1440&fit=max&auto=format)

*Memory-op LLM $/1,000 queries on gpt-4o-mini. Log scale for the 32.6× positive-value spread; zero is labeled separately. Source: Zero-Mem Table 2 × OpenAI prices, verified 2026-08-07. 70/30 in/out assumption.*

Sensitivity on GAM alone (same 18,552.39 tokens):

| Input share | GAM $/1,000 on gpt-4o-mini |
| --- | --- |
| 90% input | $3.618 |
| 70% input | $5.287 |
| 50% input | $6.957 |

The split moves the bill. It does not restore generative memory to Zero-Mem’s $0 memory-op line.

## Same tokens, different stickers

Keep GAM’s 18,552.39 memory tokens fixed. Change only the model rate card.

| Model | Input / output $/1M | GAM memory-op $/1,000 queries |
| --- | --- | --- |
| deepseek-v4-flash | $0.14 / $0.28 | $3.377 |
| gpt-4o-mini | $0.15 / $0.60 | $5.287 |
| [gpt-5.6-luna](https://vynaris.com/models#gpt-5-6-luna) | $0.20 / $1.20 | $9.276 |
| [gemini-3.5-flash-lite](https://vynaris.com/models#gemini-3-5-flash-lite) | $0.30 / $2.50 | $17.810 |
| [Haiku 4.5](https://vynaris.com/models#claude-haiku-4-5) | $1.00 / $5.00 | $40.815 |
| Sonnet 5 | $2.00 / $10.00 | $81.631 |

![Log-scale bar chart of GAM memory-operation cost per 1,000 queries across six models.](https://cdn.sanity.io/images/u1qdn8dx/production/fd7d30130be15a4f3bcf1c578d0b76200a835eda-1330x700.png?w=1440&fit=max&auto=format)

*GAM memory-op bill by model. Log scale for the 24.2× spread; labels are absolute USD. Prices verified 2026-08-07.*

That is a 24.2× spread ($3.377 → $81.631) on an unchanged token count. [Model routing](https://vynaris.com/glossary/category/routing-and-cost#model-routing) for *memory writes* is a real lever. Deleting the writes is a larger one when the write path is generative.

Put the shared final-QA shape into [the calculator](https://vynaris.com/calculator?model=gpt-4o-mini&in=4200&out=150) before you compare stacks. Memory-op savings only matter relative to the reader you still pay.

## Final QA stays on the bill

Under our editable reader shape (4,200 in / 150 out):

| Model | Final-QA $/1,000 queries |
| --- | --- |
| deepseek-v4-flash | $0.630 |
| gpt-4o-mini | $0.720 |
| gpt-5.6-luna | $1.020 |
| gemini-3.5-flash-lite | $1.635 |
| Haiku 4.5 | $4.950 |
| Sonnet 5 | $9.900 |

Stack on gpt-4o-mini (memory ops + final QA; encoder excluded from the total):

| Method | Memory ops | Final QA | LLM total / 1,000 queries | Memory share of LLM bill |
| --- | --- | --- | --- | --- |
| LightMem | $0.162 | $0.720 | $0.882 | 18.4% |
| SimpleMem | $2.609 | $0.720 | $3.329 | 78.4% |
| GAM | $5.287 | $0.720 | $6.007 | 88.0% |
| Zero-Mem | $0.000 | $0.720 | $0.720 | 0.0% |

For LightMem-class pipelines, the reader dominates. For GAM-class generative memory, memory ops dominate. Zero-Mem’s cost claim is specifically about that middle column.

## Encoder compute is not “zero cost”

Zero-Mem builds an entity–context graph (non-generative NER such as spaCy), a temporal hierarchy, BM25, and dense access with BGE-M3. The paper is explicit: encoder computation is accounted for separately from LLM tokens.

As a public API *bound*, 800 [embedding](https://vynaris.com/glossary/category/embeddings-and-representations#embedding) tokens on text-embedding-3-small ($0.02/1M) is $0.016 per 1,000 queries. That bound is tiny next to GAM’s $5.287 memory-op line on gpt-4o-mini. It is also not Zero-Mem’s production bill if BGE-M3 runs on your GPUs. Self-hosted encoders trade GPU/CPU time and index maintenance for LLM stickers. Budget them. Do not call them free because they are not chat completions.

## How to decide: generative memory vs deterministic retrieval

Use this as an implementation checklist, not a brand preference.

1. **Meter memory ops separately.** Log LLM tokens (and dollars) for write/update/reflect/retrieve-planning calls apart from final QA. If you only see one meter, you cannot apply this table. Our [dollars-per-call how-to](https://vynaris.com/blog/agents-cost-aware-dollars-per-call-howto) covers the field layout.
2. **Compare against Table 2’s shape.** If your generative memory path is near LightMem’s ~570 tokens/query, the dollar win from deleting it on gpt-4o-mini is $0.162/1,000 queries. If it looks like GAM’s ~18.6k, the win is $5.287 on gpt-4o-mini and $81.631 on Sonnet 5.
3. **Keep provenance.** Zero-Mem’s design argument is that LLM-written summaries can omit or merge evidence. Cost and auditability move together when the source of record stays the original trace.
4. **Preserve a \[context window\](https://vynaris.com/glossary/category/inference-and-serving#context-window) budget for the reader.** Deterministic retrieval still injects evidence tokens into final QA. Shrinking memory-op tokens while bloating retrieved context can erase the win.
5. **Watch write amplification.** Agents that call a memory LLM on every tool observation pay the memory line far more often than the QA line. Measure writes per answer.
6. **Do not budget the 57.6% latency figure as cash.** Table 2 reports LightMem at 0.51 s/query and Zero-Mem at 0.22 s/query for memory operations ((0.51 − 0.22) / 0.51 ≈ 0.569; the paper states 57.6%). Latency can justify engineering time. It is not a receipt for $/query.

## Honest tradeoff: when generative memory still wins

Do not rip out LLM memory writes if your product *needs* generated abstractions as the product surface — for example, user-visible “memories,” coach-style reflections, or compressed narratives you show in a UI. Deterministic retrieval returns traces; it does not invent a clean paragraph unless the reader does it at answer time (which then lands on the final-QA bill).

Also skip a Zero-Mem-style rebuild when memory-op tokens are already near the LightMem floor *and* your encoder/index ops cost more engineering hours than the $0.162/1,000 you would save on gpt-4o-mini. Cheap generative memory plus [prompt caching](https://vynaris.com/glossary/category/inference-and-serving#prompt-caching) on stable prefixes can beat a half-finished graph project.

Reproducibility caveat: the authors say code and implementation details land after peer review in the [paper](https://arxiv.org/abs/2607.29377). Until then, treat Table 2 as a primary-source measurement under their harness, not a drop-in library you can re-bill tonight.

## FAQ

**Does “zero-token memory” mean the agent is free?**

No. Final QA still bills [input/output token pricing](https://vynaris.com/glossary/category/routing-and-cost#input-output-token-pricing). Encoders, NER, and indexes still burn compute.

**Why assume 70/30 for the token split?**

The paper reports total memory-operation tokens. 70/30 is a disclosed assumption. The sensitivity table shows the range.

**Can I convert the 57.6% time cut into dollars?**

No. We refuse that conversion. GPU utilization, batching, and concurrency decide whether wall-clock maps to rent. The paper does not publish that mapping.

**How does this relate to forecasting agent workflows?**

Call-graph growth and re-sent history often dominate long agents. Memory-op deletion is one lever; session shape is another. See our [pre-build agent cost forecast](https://vynaris.com/blog/forecast-agent-workflow-api-cost-before-building).

## Sources

- [Zero-Mem: Zero-Token Memory Operations for LLM Agents](https://arxiv.org/abs/2607.29377) (arXiv 2607.29377), captured 2026-08-07: Table 2 token and time figures; zero LLM tokens outside final QA; code after peer review.
- [OpenAI API pricing](https://developers.openai.com/api/docs/pricing), captured 2026-08-07: gpt-4o-mini $0.15/$0.60; gpt-5.6-luna $0.20/$1.20; text-embedding-3-small $0.02/1M.
- [DeepSeek pricing](https://api-docs.deepseek.com/quick_start/pricing), captured 2026-08-07: deepseek-v4-flash $0.14/$0.28 (cache miss).
- [Gemini API pricing](https://ai.google.dev/gemini-api/docs/pricing), captured 2026-08-07: gemini-3.5-flash-lite $0.30/$2.50 standard.
- [Anthropic pricing](https://www.anthropic.com/pricing), re-verified 2026-08-14: Haiku 4.5 $1/$5; Sonnet 5 $2/$10 standard after Anthropic canceled the scheduled increase.

All arithmetic is in the accompanying script. No Vynaris traffic, customer, or routing data is used.

Further reading: [Lemmalog’s extraction-cost break-even](https://vynaris.com/blog/lemmalog-38x-context-cut-extraction-cost-break-even) prices the repeated reader-stage saving across two public memory benchmarks without inventing the one-time extraction bill.

## About the author

[Dhrumil Barot](https://vynaris.com/about#founder), Founder. Dhrumil is an AI inference and LLM infrastructure expert. Dhrumil built Vynaris to make model access, routing and pricing simple and transparent for developers. [LinkedIn](https://www.linkedin.com/in/bdhrumil) · [X](https://x.com/fyidhrumil)