Glossary · Category
Evals & quality
56 plain-English definitions.
- adversarial testing (LLM)
- Deliberately crafting difficult or malicious inputs to probe a model's weaknesses or safety boundaries.
- AgentBench
- A benchmark suite that evaluates LLM agents across diverse interactive environments like web browsing, coding, and games.
- answer relevance
- A RAG/eval metric measuring whether a generated response actually addresses the question asked.
- ARC-AGI
- A benchmark designed to test general fluid reasoning and abstraction in a way that resists memorization, used as a proxy for progress toward AGI.
- benchmark contamination
- When a model's training data accidentally includes benchmark test questions, inflating its measured performance.
- canary testing (model updates)
- Running a small subset of traffic against a new model version to catch quality regressions before a full rollout.
- Chatbot Arena
- A crowdsourced platform where users compare anonymous model outputs head-to-head to generate Elo-style leaderboard rankings.
- confidence calibration
- How well a model's stated confidence in an answer matches its actual likelihood of being correct.
- content moderation (LLM)
- Automated filtering of model inputs and outputs for harmful, unsafe, or policy-violating content.
- context precision
- A RAG evaluation metric measuring how much of the retrieved context was actually relevant to answering the query.
- context recall
- A RAG evaluation metric measuring whether all necessary information needed to answer the query was present in the retrieved context.
- Elo rating (LLM)
- A ranking system (borrowed from chess) used to rank models based on head-to-head human or model preference votes.
- eval drift
- When an evaluation benchmark's relevance or discriminating power degrades over time as models improve or overfit to it.
- eval harness
- The tooling infrastructure that runs a model against a benchmark or test suite and scores its outputs automatically.
- eval-driven development
- Building and iterating on LLM applications by writing evaluation tests first, similar to test-driven development in traditional software.
- evals
- Structured tests used to measure and compare how well LLMs perform on specific tasks, capabilities, or safety criteria.
- factuality
- The extent to which a model's claims are objectively true and verifiable.
- faithfulness (RAG)
- A metric assessing whether a RAG system's generated answer accurately reflects the content of its retrieved source documents.
- false refusal rate
- The measured frequency at which a model incorrectly refuses to answer safe, legitimate requests.
- golden dataset
- A curated, high-quality set of input-output examples used as ground truth for evaluating model or pipeline performance.
- GPQA
- A graduate-level benchmark (Google-Proof Q&A) of hard science questions designed to resist simple web lookup.
- groundedness
- The degree to which a model's output is actually supported by the retrieved or provided source context, rather than invented.
- guardrails (output validation)
- Automated checks that validate or filter an LLM's output before it reaches the end user, catching unsafe or malformed responses.
- hallucination
- When a model generates plausible-sounding but factually incorrect or fabricated information.
- hallucination rate
- The measured frequency at which a model produces factually incorrect or unsupported claims on a given evaluation set.
- HumanEval
- A benchmark measuring a model's ability to generate correct code from natural-language programming problem descriptions.
- inter-rater reliability
- A statistical measure of how consistently different human evaluators agree when scoring the same model outputs.
- LLM benchmark
- A standardized dataset and scoring methodology used to compare model capabilities across providers.
- LLM observability
- Tooling that logs, traces, and monitors LLM application behavior in production, including prompts, responses, latency, and cost.
- LLM-as-judge
- Using a strong LLM to automatically score or compare the quality of other models' outputs instead of relying solely on human raters.
- MMLU
- A widely cited benchmark (Massive Multitask Language Understanding) testing model knowledge across dozens of academic subjects.
- model drift
- Gradual degradation or change in a model's output quality or behavior over time, often due to upstream provider updates.
- offline evals
- Evaluations run against a fixed, pre-collected dataset rather than on live production traffic.
- online evals
- Evaluations run continuously against live production traffic to monitor real-world model quality.
- over-refusal
- When a model unnecessarily declines to answer a benign request due to overly cautious safety training.
- pairwise comparison (evals)
- An evaluation method where a judge (human or model) picks the better of two candidate responses rather than scoring each in isolation.
- pass@k
- A code-generation evaluation metric measuring the probability that at least one of k sampled model outputs passes a correctness test.
- PII detection
- Automatically identifying personally identifiable information in text, used to redact sensitive data before or after LLM processing.
- prompt regression
- A drop in output quality caused by an unintended side effect of editing a prompt.
- prompt versioning
- Tracking and managing changes to prompts over time like code, so teams can test, compare, and roll back prompt versions.
- refusal rate
- The percentage of requests a model declines to answer, tracked as both a safety and usability metric.
- regression testing (LLM)
- Re-running an evaluation suite after a prompt, model, or pipeline change to catch quality drops before shipping.
- response quality scoring
- Automated or human-assigned scores rating how good an LLM's output is against defined criteria.
- saturation (benchmark)
- When most top models score near-perfectly on a benchmark, making it no longer useful for distinguishing capability differences.
- self-consistency check
- Verifying an answer's reliability by sampling it multiple times and checking whether the model converges on the same result.
- silent model degradation
- Unannounced changes to a hosted model's weights or serving stack that cause quality shifts users don't expect.
- snapshot testing (LLM)
- Comparing a model or prompt's current output against a previously saved "known good" output to catch unintended changes.
- SWE-bench
- A benchmark that tests whether an AI coding agent can resolve real-world GitHub issues in actual software repositories.
- sycophancy
- A model's tendency to agree with or flatter the user's stated opinion rather than giving an independently accurate answer.
- sycophancy benchmark
- An evaluation set specifically designed to measure how often a model changes a correct answer to agree with user pushback.
- task success rate
- The percentage of times an agent or model completes an assigned task correctly, a core agent evaluation metric.
- toxicity classifier
- A model or tool that scores text for harmful, offensive, or abusive content, often used to filter LLM inputs or outputs.
- tracing (LLM)
- Recording the full sequence of calls, tool invocations, and intermediate steps in an LLM application run for debugging and analysis.
- trajectory evaluation
- Assessing the full sequence of steps an agent took to complete a task, not just its final output.
- uncertainty estimation
- Techniques for measuring how confident a model is in its own output, used to trigger fallback or human review.
- win rate (model comparison)
- The percentage of head-to-head comparisons in which one model's output is preferred over another's.