Glossary · Category

Additional Architecture & Training Detail

12 plain-English definitions.

Activation steering

An interpretability/control technique that adds a learned or identified direction vector to a model's internal activations at inference time to steer its behavior toward or away from a concept.

Attention head dimension

The size of the vector space each individual attention head operates in, equal to the model's hidden dimension divided by the number of heads (before any grouping like GQA/MQA).

Hidden size / model dimension

The width of the residual stream vector each token is represented by throughout a transformer's layers, a primary driver of model capacity and compute cost.

Induction head

A specific attention-head circuit (identified via interpretability research) that detects repeated token patterns and predicts their continuation, thought to underlie in-context learning.

Intermediate/FFN dimension

The (typically larger) internal width of a transformer's feed-forward sublayer, often several times the model's hidden size.

Logit lens

An interpretability technique that projects intermediate transformer layer activations through the final unembedding matrix to inspect what token predictions a model is "thinking" at each layer.

Mechanistic interpretability

A research field focused on reverse-engineering the internal computations and learned circuits inside neural networks to understand how they produce their outputs.

Number of layers (depth)

The count of stacked transformer blocks in a model, a key architectural dimension alongside width (hidden size) that determines total parameter count and representational depth.

Probing classifier

A small classifier trained on a frozen model's internal representations to test whether a specific piece of information is linearly recoverable from them.

Rotary base frequency (theta)

The hyperparameter controlling the base frequency used in RoPE's rotation angles; increasing it is a common technique to extend a model's usable context length.

Sparse autoencoder (interpretability)

A technique trained on a model's internal activations to decompose them into a larger set of sparse, more human-interpretable features, used to study what concepts a model represents internally.

Weight tying

Sharing the same parameter matrix between a model's input token embedding and its output (unembedding) projection layer, reducing parameter count.