Study Reinforcement Learning

← home

track six of eight

Agentic RL

About 9 hours, 8% of the whole. 3 lectures.

LLMtools, code,a browsertool callresult, and later a reward
Figure.Many turns go by before the reward arrives.

Multi-step tasks with tools, browsers, and code. The reward usually arrives at the end, so credit assignment gets hard.

Start with lecture 16. Each lecture lists what to read first, and ends with a checkpoint: something to do with the page closed.

  1. 16Agentic RL: tool use and multi-turn interaction~3 h

    checkpointExplain why long-horizon agentic tasks make credit assignment hard, and why "reward = tests pass" is gameable.

    ← start here

  2. 24Computer use and browser agents2–3 h

    checkpointExplain why pixel-coordinate action spaces are hard to ground from screenshots, and one failure mode specific to prompt injection from web content.

  3. 25Long-horizon credit assignment~3 h

    checkpointExplain why GAE breaks at 1000+ steps and what hindsight relabeling / tree-search-then-train each buy you.