← Skill graph
Training dynamics
Part of Deep learning
Needs first
Helps first
Nothing recommended beyond what is required.
Where it is assessed
10 assessed items, 0 done.
- AIE-203 M2.3 Instrument a training run · tier 1
- AIE-203 M2.4 Three broken runs, and the tell for each · tier 2
- AIE-203 M2.6 Retention: training pathologies and their signatures · tier 1
- AIE-203 M3.3 Initialisation decides whether it learns · tier 1
- AIE-203 M3.4 Does normalisation help here, or is it cargo · tier 2
- AIE-203 M3.6 Retention: initialisation and normalisation · tier 1
- AIE-203 M5.3 A scaling curve you measured yourself · tier 2
- AIE-203 M5.4 The network you built, and what its curves showed · tier 3
- AIE-203 M5.5 Oral defense · tier 4
- AIE-203 M5.6 Retention: scale, compute and what a curve supports · tier 1
Verified sources
- Deep Learning
- Dive into Deep Learning
- Practical Deep Learning for Coders
- Let's reproduce GPT-2 (124M) · Andrej Karpathy
- Neural Networks: Zero to Hero · Andrej Karpathy
- MIT Introduction to Deep Learning | 6.S191 · Alexander Amini
- Training Compute-Optimal Large Language Models
- Scaling Laws for Neural Language Models
- karpathy/nanoGPT
- CS231n Winter 2016 · Andrej Karpathy
- Stanford CS336 Language Modeling from Scratch I 2025 · Stanford Online