Glossary · Quantization
Imatrix quantization
Imatrix quantization uses an importance matrix built from representative text to guide low-bit rounding. The matrix helps preserve more important weights or activations at higher precision than a purely uniform approach. Formats such as IQ4_XS are intended to improve the quality-size tradeoff, but the result depends on the model, calibration data, runtime, and context. A quantized refusal-reduced model can behave differently from its full-precision source, so evaluate the exact served artifact. Memory fit is also model-specific and should be measured rather than inferred from the format name alone.
Related terms
Vynaris is an inference gateway that routes each request to the cheapest right-sized model and shows the receipt. Get an API key or read the docs.