Glossary · Category

Additional Terms Rounding Out Coverage

10 plain-English definitions.

Cascade routing

A routing strategy that first tries a cheap/fast model and escalates to a stronger, more expensive model only if the initial response is judged insufficient.

Compositional generalization

A model's ability to correctly handle novel combinations of concepts it has seen individually but not together during training.

Confidence-based routing

A routing strategy that uses a model's own output confidence (e.g., logprobs or a self-reported certainty) to decide whether to accept its answer or escalate to a stronger model.

Downshift routing

A gateway pattern that automatically sends a request to a cheaper, smaller "workhorse" model instead of a larger default model when the task is judged simple enough, without sacrificing acceptable quality.

Ensemble inference

Combining outputs from multiple models or multiple sampled generations to produce a more reliable final answer than any single generation.

Escalation tier (routing)

A designation for a stronger, more expensive model reserved for requests that a cheaper model in a routing cascade failed or was judged unlikely to handle well.

Function/tool call schema validation

Automated checking that a model's generated tool call arguments conform to the declared parameter types and required fields before execution.

Multi-hop reasoning

A reasoning task or evaluation category requiring a model to chain together facts or inferences from multiple sources to reach a final answer.

Router model (capability classifier)

A small, fast model used by a gateway to classify an incoming request's difficulty or required capability before deciding which downstream model should handle it.

Zero-shot cross-lingual transfer

A model's ability to perform a task in a language it received little or no task-specific fine-tuning data for, by leveraging shared multilingual representations.