Glossary · Training & tuning

direct preference optimization (DPO)

A simpler alternative to RLHF that fine-tunes a model directly on preference pairs without training a separate reward model.

Related terms

• Hugging Face Transformers
• LoRA (low-rank adaptation)
• QLoRA
• alignment (LLM)
• extended thinking
• fine-tuning

Vynaris is an inference gateway that routes each request to the cheapest right-sized model and shows the receipt. Get an API key or read the docs.