Glossary · Category

Additional Sampling, Decoding & Reasoning Techniques

9 plain-English definitions.

Acceptance rate (speculative decoding)

The fraction of draft-model-proposed tokens that the target model accepts during speculative decoding; higher acceptance rates yield greater speedups.

Diverse beam search

A beam search variant that explicitly penalizes similarity between beams to produce more varied candidate outputs.

Draft tree / tree-based speculative decoding

A speculative decoding refinement where the draft model proposes a tree of multiple candidate continuations rather than a single linear sequence, increasing the chance of a long accepted match.

Length penalty

A decoding adjustment (common in beam search) that biases toward longer or shorter outputs by scaling the sequence score by a function of its length.

Multi-token prediction

A training objective or architecture modification where the model predicts several future tokens at once per forward pass, used both to speed training and enable Medusa-style speculative decoding.

Nucleus sampling temperature interaction

The compounding effect of applying both temperature scaling and top-p truncation together, since temperature reshapes the distribution top-p then truncates.

Repetition/n-gram blocking

A decoding constraint that forbids repeating any n-gram that has already appeared in the generated output, a hard alternative to the softer repetition penalty.

Self-verification / verifier model

A separate model or prompting technique used to check whether a generated answer or reasoning chain is correct before it's finalized or returned.

Speculative sampling verification

The step in speculative decoding where the target model checks draft tokens against its own distribution and accepts/rejects them via a rejection-sampling rule that guarantees the same output distribution as normal decoding.