← Skill graph
Quantisation and compression
Part of LLM inference cost and latency optimisation
Needs first
Helps first
Nothing recommended beyond what is required.
Unlocks
Nothing depends on this one.
Where it is assessed
2 assessed items, 0 done.
Verified sources
- MIT 6.5940 EfficientML.ai · MIT HAN Lab
- AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
- GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
- LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale
- SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
- ggml-org/llama.cpp