Study Reinforcement Learning

← home

who made this

About

Who made this, how it started, and whose teaching it builds on.

I’m Yad Konrad. I started this repo on 31 December 2016 as a list of the talks and courses I was learning reinforcement learning from: Joelle Pineau’s and Pieter Abbeel’s 2016 talks, Sutton and Barto’s book, David Silver’s course, and Berkeley’s CS 294. It has grown into notes, a lecture series, tested exercises, and this site.

How I got here

  1. Self-driving in GTA San Andreas. My way into RL was getting a car to drive itself in GTA San Andreas. That became Nvidia-Autopilot-Keras (January 2017): a Keras version of NVIDIA’s end-to-end self-driving model (Bojarski et al. 2016, arXiv:1604.07316).
  2. Berkeley. CS 294 Deep Reinforcement Learning, spring 2017, taught by Sergey Levine, John Schulman, and Chelsea Finn. My notes from it are kept as I wrote them.
  3. Then program synthesis, then NLP with RL, then LLMs, and then other things.

Along the way:

  • David Silver’s course. DeepMind’s David Silver taught the UCL course (2015) that was on the first version of this list. All ten lectures are on the Watch page.
  • OpenAI Gym. I worked on and contributed to Gym environments, under a GitHub account I don’t have anymore.

The lecture series runs in a similar order: classical RL first, then RL for language models.

How this was made

  • The 2016 list of talks, books, and courses and the 2017 CS 294 notes are mine, written while I was learning. They stay as written.
  • The 34 lectures, the exercises, the cheat sheets, and this site were drafted with AI coding assistants and then edited. Every lecture is marked unreviewed until a person has read it end to end and checked it. The exercises are tested against reference solutions.

Credit

This leans on other people’s teaching:

  • Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction.
  • David Silver’s Reinforcement Learning course (UCL, 2015).
  • Sergey Levine, John Schulman, and Chelsea Finn: CS 294 at Berkeley, now CS 285.
  • Joelle Pineau’s and Pieter Abbeel’s 2016 talks, which this repo’s first list was built around.
  • Arthur Juliani’s Simple Reinforcement Learning with TensorFlow series.
  • OpenAI’s Spinning Up in Deep RL, and Hugging Face’s Deep RL Course.
  • The authors of every paper the lectures cite.

The site runs on Mycelium, my digital garden theme, built with Astro. The source for everything is on GitHub.