Tarık Şenkal Tags
Efficiency
2 posts
Efficiency7 minQuantization in Production: FP8, NVFP4, and Where Quality Actually Drops
How to choose between FP8, NVFP4, AWQ and GPTQ: the hardware support matrix, serving with vLLM, and why low-bit quantization degrades non-English output first.
Efficiency7 minKnowledge Distillation and Small Models: Transferring Knowledge from a Large Teacher to a Small Student
Knowledge distillation transfers a large teacher model’s knowledge into a small, fast student model. We explain soft labels, temperature, and the distillation loss, and why, how, and when it works, with intuitive examples.