Glossary · Category

Safety & Alignment Terminology

14 plain-English definitions.

Abliterated: An abliterated language model has had a direction associated with refusal behavior identified in its activations or weights and reduced through an ablation edit, often without full retraining. The edit is intended to reduce refusals while preserving useful capabilities, but the result depends on the model, method, and evaluation. A refusal count or capability test describes one measured artifact, not every version of the model. Abliterated builds can support authorized security testing, but they still require scope, access controls, logging, and the prohibited-use rules of the environment where they run.
Alignment tax: The performance cost (on capability benchmarks) sometimes incurred when a model is heavily safety- or preference-aligned.
Backdoor attack (model): A trained-in vulnerability where a model behaves normally except when a specific trigger input activates hidden malicious behavior.
Differential privacy: A mathematical framework for training models with formal guarantees limiting how much any single training example can influence the final model, protecting individual privacy.
Guardrails: Input/output filtering or policy-enforcement layers wrapped around a model to block disallowed content, PII leakage, or off-policy behavior.
Heretic ablation: Heretic ablation is an automated approach to refusal removal that searches across layers and ablation strengths instead of relying only on manual tuning. The search evaluates candidate edits against refusal and capability objectives, then selects a result under the chosen scoring method. A reported trial count, refusal count, or KL divergence belongs to a particular experiment and should not be treated as a universal standard. Community publishers may use Heretic-derived builds, but users still need to verify the exact model artifact, license, behavior, and authorized-use controls before deployment.
Jailbreak: A prompt or technique specifically designed to bypass a model's safety training and elicit disallowed content.
Jailbreak versus ablation: A jailbreak uses crafted prompts or conversation context to persuade a fixed model to bypass a refusal. An ablation edits the model's weights or activations to reduce refusal behavior across prompts. A jailbreak is a prompting technique that can vary with wording, context, and model updates. Ablation is a model modification whose effects should be measured on the exact artifact being served. The two are not interchangeable: jailbreak tests study prompt robustness, while ablation studies change the model itself. Both require authorized testing and clear prohibited-use boundaries.
Membership inference attack: An attack that attempts to determine whether a specific data point was part of a model's training set, a privacy concern for sensitive training data.
Red teaming: The practice of adversarially probing a model to discover harmful, unsafe, or policy-violating behaviors before deployment.
Refusal: A model's trained behavior of declining to answer or complete a request it judges disallowed, unsafe, or outside policy.
Refusal direction: A refusal direction is a pattern in a language model's activation space associated with declining certain prompts, such as an apology, warning, or redirect. Researchers can estimate it by comparing activations from refused and answered prompt pairs. Ablation methods try to reduce the influence of that pattern, but the direction is an analytical approximation, not a universal switch shared by every model. Measuring refusal behavior before and after an edit helps identify changes and possible capability loss. The result should be tested on the served artifact, not assumed from the editing method alone.
System card: A more comprehensive safety/capability disclosure document (broader than a model card) often published alongside frontier model releases, covering red-teaming and risk assessments.
Watermarking (LLM output): Techniques that embed statistically detectable but imperceptible patterns into generated text to later identify it as AI-generated.