Glossary · Inference & serving

batch API

A provider endpoint (e.g. OpenAI, Anthropic) for submitting large volumes of non-urgent requests that are processed asynchronously at a lower cost.

Related terms

• FlashAttention
• KV cache
• Ollama
• PagedAttention
• SGLang
• autoscaling (inference)

Vynaris is an inference gateway that routes each request to the cheapest right-sized model and shows the receipt. Get an API key or read the docs.