Deep Learning

AIE-203

3 credits · 65h required · after AIE-202

Backpropagation built by hand, then the training dynamics that decide whether a network learns anything at all. The gap this closes is specific: you can already call a model and measure it, and you cannot yet look at a loss curve that has gone flat and say why. Everything here is aimed at that — an autodiff engine you wrote, activation statistics you have plotted, and three broken training runs you produced on purpose so the fourth one, which breaks by accident, is recognisable.

The lab. A laptop, deliberately. Every network in this course trains on CPU in minutes: a scalar autodiff engine, an MLP on a few thousand examples, a small convolutional net. Nothing here needs a rented GPU, because a course gated on hardware is a course that gets abandoned in week three. The constraint is also the lesson — with no compute to hide behind, a bad initialisation shows up as a dead network rather than as a slightly worse number.

By the end you can

Implement reverse-mode autodiff from scratch and use it to train a network
Debug a training run from its loss curve
Apply and justify normalisation, initialisation and regularisation choices
Read an architecture diagram and implement it

What this builds

Deep learning · supporting0/17 demonstrated
Engineering foundations · supporting0/3 demonstrated

Next assessment. Your own autodiff engine — 180 min, tier 1. Machine-checkable, so you can attempt it before reading anything and find out whether you need to.

0/17 assessed items demonstrated · 0/14 read or watched. The second number is how you close a gap; only the first is evidence that it closed.

0 of 17 demonstrated · 0/14 read

0m of 65h

M1 · Autodiff, written from scratch

0/7 · 15h

lecture · 2h · tier 0 self-marked
Neural networks — 3Blue1Brown
lecture · 3h · tier 0 self-marked
Neural Networks: Zero to Hero — Andrej Karpathy
assignment · 3h · tier 1 machine-verified
assignment · 3h · tier 1 machine-verified
reading · 2h · tier 0 self-marked
Welcome to PyTorch Tutorials
assignment · 1h 45m · tier 1 machine-verified
retention · 15m · tier 1 machine-verified

M2 · Training dynamics, and reading a run

0/6 · 14h

lecture · 3h · tier 0 self-marked
Neural Networks: Zero to Hero — Andrej Karpathy
reading · 2h 30m · tier 0 self-marked
Deep Learning
assignment · 3h · tier 1 machine-verified
assignment · 3h · tier 2 panel-assessed
lecture · 2h 15m · tier 0 self-marked
MIT Introduction to Deep Learning | 6.S191 — Alexander Amini
retention · 15m · tier 1 machine-verified

M3 · Initialisation, normalisation, regularisation

0/6 · 13h

lecture · 2h 30m · tier 0 self-marked
CS231n Winter 2016 — Andrej Karpathy
reading · 2h 30m · tier 0 self-marked
Deep Learning
assignment · 3h · tier 1 machine-verified
assignment · 2h 45m · tier 2 panel-assessed
reading · 2h · tier 0 self-marked
Dive into Deep Learning
retention · 15m · tier 1 machine-verified

M4 · Architectures, and reading a diagram into code

0/6 · 12h

reading · 2h · tier 0 self-marked
Understanding LSTM Networks
lecture · 2h 30m · tier 0 self-marked
CS231n Winter 2016 — Andrej Karpathy
assignment · 3h · tier 1 machine-verified
reading · 2h 15m · tier 0 self-marked
Dive into Deep Learning
assignment · 2h · tier 2 panel-assessed
retention · 15m · tier 1 machine-verified

M5 · Scale, and a network of your own

0/6 · 11h

reading · 2h · tier 0 self-marked
Scaling Laws for Neural Language Models
reading · 1h 45m · tier 0 self-marked
Training Compute-Optimal Large Language Models
assignment · 3h · tier 2 panel-assessed
project · 3h · tier 3 artifact
defense · 1h · tier 4 defended
retention · 15m · tier 1 machine-verified