RLAIF

An LLM judge or a written constitution stands in for the human labeler.

Start with lecture 14. Each lecture lists what to read first, and ends with a checkpoint: something to do with the page closed.

1. 14Constitutional AI, RLAIF, and Self-Improvement2–3 h

   checkpointExplain the two phases of Constitutional AI; name one bias an LLM judge has and how you'd fight it.

   ← start here

2. 27RLAIF and synthetic preferences at scale2–3 h

   checkpointName three biases of an LLM-as-judge and a mitigation for each; explain why a weak judge caps the policy's ceiling.

3. 34Self-distillation and self-improvement loops2–3 h

   checkpointExplain when a self-improvement loop converges vs. collapses; describe the role of the filter / verifier in keeping the loop honest.