Glossary · Inference & serving

SGLang

An open-source LLM serving framework known for RadixAttention-based prefix caching and a structured generation language for complex prompting.

Related terms

• FlashAttention
• KV cache
• Ollama
• PagedAttention
• autoscaling (inference)
• batch API

Vynaris is an inference gateway that routes each request to the cheapest right-sized model and shows the receipt. Get an API key or read the docs.