Quantization, Pruning and Distillation
Shrink models: lower-precision weights, removed connections, or a small student model trained to mimic a large teacher.
Advanced
8-bit or 4-bit quantization cuts memory several times with small quality loss.
Spotted a mistake or want to add detail? Suggest an improvement — Content Contributors' edits are reviewed by editors before they go live.
Discussion (0)
Sign in or join free to ask questions and discuss this topic.