Glossary · Category

Classic Architectures (for contrast)

5 plain-English definitions.

Convolutional neural network (CNN)

A neural architecture using learned local filters, dominant in computer vision and occasionally applied to sequence modeling before transformers took over.

Gated recurrent unit (GRU)

A simplified gated recurrent architecture similar to LSTM with fewer parameters, also predating widespread transformer adoption.

Long short-term memory (LSTM)

A gated recurrent architecture designed to better retain information over long sequences than plain RNNs, dominant in NLP before transformers.

Recurrent neural network (RNN)

A pre-transformer sequence architecture that processes tokens one at a time while maintaining a hidden state, largely superseded by transformers for language modeling due to poor parallelization.

Seq2seq model

An encoder-decoder architecture originally built on RNNs for mapping an input sequence to an output sequence, the conceptual precursor to transformer encoder-decoder models.