Glossary · Category

Embeddings & Representations

14 plain-English definitions.

Approximate nearest neighbor (ANN) search: An algorithmic approach to finding vectors close to a query vector without exhaustively comparing against every stored vector, trading exact accuracy for speed.
BM25: A classic sparse lexical ranking function based on term frequency and inverse document frequency, still widely used alongside or against dense retrieval.
Chunking: The process of splitting long documents into smaller segments for embedding and retrieval, balancing context completeness against retrieval precision.
Contextualized embedding: A token representation that depends on surrounding context (as produced by transformer layers), as opposed to a static word vector.
Cosine similarity: A metric measuring the angle between two vectors, commonly used to compare embeddings for semantic similarity independent of magnitude.
Dot-product similarity: A similarity metric computed as the raw inner product of two vectors, sensitive to both direction and magnitude, often used in retrieval systems.
Embedding: A dense vector representation of a token, word, or piece of content that captures semantic meaning in continuous space.
Embedding dimension: The length of the vector used to represent each token or input; higher dimensions can capture more nuance at greater memory/compute cost.
HNSW (Hierarchical Navigable Small World): A popular graph-based ANN algorithm that builds a layered navigable graph over vectors for fast approximate similarity search.
Matryoshka embeddings: Embeddings trained so that truncated prefixes of the full vector remain useful, letting applications trade off dimensionality for speed without retraining.
Positional embedding: A learned or fixed vector added to (or combined with) token embeddings to encode sequence position.
Reranker: A model that re-scores an initial set of retrieved candidates using deeper (often cross-attention) relevance modeling to improve final ranking quality.
Sentence/text embedding model: A model trained to map an entire sentence or document to a single dense vector for semantic similarity, search, or clustering tasks.
Token embedding matrix: The learned lookup table mapping each vocabulary token to its initial dense vector representation before any transformer layers process it.