Glossary · Category

Additional Long-Context & Retrieval Techniques

8 plain-English definitions.

Infini-attention

A long-context technique that compresses older context into a fixed-size recurrent memory alongside standard local attention, aiming for bounded memory regardless of sequence length.

Landmark attention

A long-context technique that inserts special "landmark" tokens summarizing chunks of context, letting the model selectively retrieve relevant chunks instead of attending to everything.

LongRoPE

A context-extension method that searches for non-uniform RoPE rescaling factors to extend context length (e.g., to 2M+ tokens) with minimal fine-tuning and quality loss.

Maximal marginal relevance (MMR)

A retrieval re-ranking technique that balances relevance to the query against diversity among selected results, avoiding redundant near-duplicate chunks.

Memory-augmented transformer

An architecture that gives a model access to an external read/write memory store beyond its fixed context window, used for long-horizon or lifelong-learning tasks.

Retrieval-augmented long context

A hybrid strategy that combines RAG-style retrieval with a large context window, retrieving only the most relevant chunks rather than stuffing an entire corpus into the prompt.

Sliding window KV cache eviction

A memory-management policy that discards older KV-cache entries outside a fixed recent window, bounding memory use for very long or streaming generations.

StreamingLLM

A long-context inference technique that keeps a small set of initial "attention sink" tokens plus a sliding window of recent tokens in the KV cache, enabling stable generation over effectively unbounded streams without retraining.