Glossary · Category

Multi-Model & Merging Techniques

4 plain-English definitions.

DARE (Drop And REscale)

A model-merging technique that randomly drops a fraction of fine-tuned weight deltas and rescales the rest before merging, reducing interference between merged task vectors.

Frankenmerge / passthrough merge

A model-merging technique that stacks or interleaves layers from different models to build a larger composite model, rather than averaging weights of the same architecture.

Mixture of agents

An inference-time ensembling technique where outputs from several different LLMs are aggregated or refined by another LLM to produce a stronger final answer than any single model alone.

SLERP merging

A model-merging technique that interpolates between two models' weights along a spherical path in parameter space, rather than simple linear averaging.