← Skill graph
Transformer architecture
Part of Transformer architecture
Needs first
Helps first
Nothing recommended beyond what is required.
Where it is assessed
9 assessed items, 0 done.
- AIE-204 M3.3 Build the model · tier 1
- AIE-204 M4.3 Three modern components, measured one at a time · tier 2
- AIE-204 M4.7 Retention: modern architectural components · tier 1
- AIE-204 M5.4 The reproduction: account for the gap · tier 2
- AIE-204 M6.1 The annotated implementation, published · tier 3
- AIE-204 M6.2 Every choice, and what it buys · tier 3
- AIE-204 M6.4 Read a new architecture cold · tier 2
- AIE-204 M6.5 Oral defense · tier 4
- AIE-204 M6.6 Retention: the whole architecture, cold · tier 1
Verified sources
- CMU Advanced NLP Fall 2024 · Graham Neubig
- LLM Foundations (LLM Bootcamp) · The Full Stack
- Introduction - Hugging Face NLP Course
- Neural Networks: Zero to Hero · Andrej Karpathy
- Attention Is All You Need
- Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
- huggingface/transformers
- Stanford CS25 - Transformers United · Stanford Online