Glossary · Category

Safety & Alignment Terminology

10 plain-English definitions.

Alignment tax

The performance cost (on capability benchmarks) sometimes incurred when a model is heavily safety- or preference-aligned.

Backdoor attack (model)

A trained-in vulnerability where a model behaves normally except when a specific trigger input activates hidden malicious behavior.

Differential privacy

A mathematical framework for training models with formal guarantees limiting how much any single training example can influence the final model, protecting individual privacy.

Guardrails

Input/output filtering or policy-enforcement layers wrapped around a model to block disallowed content, PII leakage, or off-policy behavior.

Jailbreak

A prompt or technique specifically designed to bypass a model's safety training and elicit disallowed content.

Membership inference attack

An attack that attempts to determine whether a specific data point was part of a model's training set, a privacy concern for sensitive training data.

Red teaming

The practice of adversarially probing a model to discover harmful, unsafe, or policy-violating behaviors before deployment.

Refusal

A model's trained behavior of declining to answer or complete a request it judges disallowed, unsafe, or outside policy.

System card

A more comprehensive safety/capability disclosure document (broader than a model card) often published alongside frontier model releases, covering red-teaming and risk assessments.

Watermarking (LLM output)

Techniques that embed statistically detectable but imperceptible patterns into generated text to later identify it as AI-generated.