unreviewed draftNo one has checked this end to end. Treat the math, code, and citations as unverified. Source on GitHub.
Topic: Actor-critic and PPO. Goes with: lecture L04, L06. Status: planned. To be built next.
The plan: relate actor-critic to REINFORCE-with-a-learned-baseline; derive the PPO clipped surrogate objective; do the actor-critic exercise (links to exercises/04-actor-critic/) and a small PPO sketch.