start here

Before you start

What to know before lecture 01, how long the whole thing takes, and a plan for the first week.

1What to know first

The lectures assume some math and some Python: calculus (derivatives, the chain rule, gradients), probability (expectations, distributions, KL divergence), and basic linear algebra; Python at an intermediate level, PyTorch basics, and NumPy.

For the exercises you need Python 3.10+, PyTorch, NumPy, Gymnasium, and pytest. A laptop is enough for everything through lecture 08. The language-model lectures describe full-scale training; you can follow the code without running it at scale.

before lecture 01

I can take a gradient of a function of several variables, using the chain rule.
I know what an expectation is and can compute one for a small discrete distribution.
I can read the definition of KL divergence and say what it measures.
I can multiply a matrix by a vector without looking it up.
I can write a PyTorch training loop: model, loss, backward, optimizer step.
I can install a Python package and run pytest from a terminal.

0 of 6 ticked. This browser remembers.

2How long it takes

The curriculum gives a time estimate for every lecture. The planner adds about 2 hours per exercise and puts the tracks end to end. Every track sits on Foundations; after that you can take the tracks in the order you need them.

Hours a week: 8

1. Foundations

2. Classical RL

3. RLHF

4. RLAIF

5. RLVR and reasoning

6. Agentic RL

7. Offline RL

8. Systems

Figure 1. The tracks laid end to end at your pace. Estimates are the curriculum's time per lecture plus about 2 hours per exercise; some lectures give their own, longer estimates.

3Week one

Set up Python 3.10+ and run pip install -r exercises/requirements.txt (PyTorch, NumPy, Gymnasium, pytest).
Read lecture 01 and watch David Silver's lectures 2 and 3 (MDPs, dynamic programming).
Do 01-mdps: value iteration on a gridworld. Get the tests green.
Read lecture 02 and do 02-policy-gradients: REINFORCE on CartPole.
At the end of the week, close everything and do the checkpoints for 01 and 02 from memory: write both Bellman equations, and derive ∇J = E[∇ log π · R].

4Stuck?

Every exercise has graduated hints and a reference solution. Take the next hint only when you are stuck, and look at the solution only when you decide to.

Working with an AI tutor such as Claude Code or Codex? Point it at AGENTS.md. It should have you edit starter.py, run pytest exercises/NN-topic/, and give you the next hint, not the solution.

If a lecture says something that looks wrong, check the cited paper. Fixing an error in an unreviewed lecture, and noting what was wrong, is the most useful kind of contribution here.