← Skill graph
Model serving
Part of LLM inference cost and latency optimisation
Needs first
Helps first
Nothing recommended beyond what is required.
Where it is assessed
3 assessed items, 0 done.
Verified sources
- Designing Data-Intensive Applications
- Transformer Inference Arithmetic
- MIT 6.5940 EfficientML.ai · MIT HAN Lab
- Efficiently Scaling Transformer Inference
- Fast Inference from Transformers via Speculative Decoding
- Efficient Memory Management for Large Language Model Serving with PagedAttention
- stanford-cs336/spring2025-lectures
- ggml-org/llama.cpp
- vllm-project/vllm