Glossary · Inference & serving

PagedAttention

A memory management algorithm (from vLLM) that stores the KV cache in non-contiguous fixed-size blocks like OS virtual memory pages, cutting GPU memory waste.

Related terms

• FlashAttention
• KV cache
• Ollama
• SGLang
• autoscaling (inference)
• batch API

Vynaris is an inference gateway that routes each request to the cheapest right-sized model and shows the receipt. Get an API key or read the docs.