Working with an AI tutor

Instructions for AI coding agents (Codex, Claude Code, etc.) working in this repo. Codex reads this file; Claude Code reads CLAUDE.md, which points here. Both apply.

What this repo is

A personal study repository for reinforcement learning and its use in training LLMs. It holds:

• the original course notes from 2017 (notes/cs294-2017/, notes/sutton-barto-digest/)
• a self-study lecture series taking RL from MDPs to RLHF (notes/lectures/)
• worked, tested coding exercises (exercises/)
• a curated reading list of recent papers (reference/papers/)
• small tools for collecting papers and drafting write-ups (tools/)

It is a learning environment, not a library or a product. A person is working through this material. An agent’s job is to help them learn it (explaining, drafting, checking work, building exercises), not to “complete” the repo. There is no done state to race toward.

Layout

Path	What it is	Editable by an agent?
notes/cs294-2017/, notes/sutton-barto-digest/	Original 2017 hand-written notes (CS294 student notes; Sutton & Barto digest); idiosyncratic voice, kept as written	No. Reference only. Never reword.
notes/lectures/	The 34-lecture series, NN-topic.md	Yes, under the rules below
notes/cheat-sheets/, notes/diagrams/	Quick reference	Yes
exercises/NN-topic/	A task, a starter file, tests, a reference solution, hints	Yes
reference/papers/	Reading lists. The PAPERS.md files are generated by the collector; the per-topic READMEs are hand notes.	READMEs yes; don’t hand-edit PAPERS.md; re-run the collector.
tools/arxiv-collector/	Script that fetches papers from arXiv	Yes
tools/content-pipeline/	Scripts that draft blog posts / threads from papers (auxiliary, not part of the study path)	Yes
drafts/	Scratch space for new material before a person reviews it	Yes: new drafts land here first
CURRICULUM.md	The ordered syllabus	Yes
readme.md, CHANGELOG.md	Repo front matter	Yes

Review status

The repo mixes notes a person wrote and notes nobody has reviewed yet. So nobody has to guess what’s trustworthy, every doc under notes/ and reference/ carries a status comment near the top:

<!-- status: <state> | last-reviewed: <YYYY-MM-DD or "never"> -->

<state> is one of:

• hand-written: a person wrote it. Trusted. Don’t rewrite it; propose edits to the person.
• reviewed: a person has read it end to end, fixed the errors, and stands behind it. Trusted.
• unreviewed: nobody has checked it yet. May contain wrong math, code that doesn’t run, broken links, made-up citations. Fine as a starting point. Treat every claim in it as unverified.

Rules:

• New material starts unreviewed: either in drafts/, or in place with status: unreviewed.
• An agent may not set a file to reviewed. Only a person does that, after actually reading it. An agent can do the legwork (fix the dead links, verify the citations, run the code, flag what’s wrong) and then say “this is ready for you to review.” The status stays unreviewed until a person signs off.
• Editing a reviewed or hand-written file may un-review it. Say so in your summary.
• Exercises don’t carry a status comment; they’re verified by their tests passing.

Voice

The repo has a voice: plain and direct, a little informal, written by someone learning the material. Match it.

• No first-person claims the person didn’t make. Unreviewed material doesn’t get to claim a person’s learning journey it didn’t have: no “I spent months on this,” no “these are my notes to future me.” Write in second person (“you’ll hit this bug around update 50”) or plain exposition.
• No marketing voice. Not a product launch. Don’t call things “comprehensive,” “powerful,” “cutting-edge,” “robust.” Don’t open a section with “Why this matters.” Don’t close with “the future is bright.”
• No AI-slop tells. No emoji as bullets or in headings. No rule-of-three padding (“fast, simple, and elegant”). No “it’s not just X, it’s Y.” No “let’s dive in.” Sentence-case headings. If ~/.claude/skills/anti-slop-guide is available, follow it.
• Be specific. “The loss explodes around update 50 if you don’t normalize the advantage” beats “this can be unstable.”
• Keep the old notes’ quirks. The 2017 hand-written notes (notes/cs294-2017/) say “quadratize (it could be a word)” and reference the author being from Iraq. That stays. Don’t sand it down.

Citations

Some of the existing material contains invented papers and wrong attributions. Going forward:

• Every paper reference needs an identifier that resolves: an arXiv ID (arXiv:2305.18290), a DOI, or a venue + year someone can check.
• Verify a citation before adding it. If you can’t, don’t fake one: write “there’s probably relevant work on X; I haven’t confirmed a paper.”
• arXiv IDs encode the date: YYMM.NNNNN. 2509.16679 is September 2025, not 2024. The ID and the year you write must agree.
• Affiliations drift. If you’re not sure where someone works now, leave it out.

How lectures are structured

A lecture in notes/lectures/ is NN-topic.md. The series runs foundations → modern RL → LLM alignment → applications; CURRICULUM.md has the order, notes/README.md is the index. A lecture covers, loosely: the problem it solves, the intuition, the math with commentary, code that runs, where it breaks, the relevant papers, when to reach for it, and exercises. Keep that shape, but don’t staple the eight headers on by rote; let the material set the structure. If a lecture has a matching exercise in exercises/, link to it.

How exercises work

exercises/NN-topic/ contains:

• README.md: the task and the acceptance criteria (which tests must pass)
• starter.py: a skeleton with TODOs for the student to fill in
• test_*.py: pytest tests that pass only when the implementation is correct
• solution/: a reference implementation
• HINTS.md: graduated hints; hint 1 is a nudge, the last is nearly the answer

An agent acting as tutor: have the student edit starter.py, run pytest exercises/NN-topic/, and when they’re stuck hand over the next hint from HINTS.md, not the solution, not two hints ahead. Show solution/ only if they ask for it directly. An exercise is done when its tests pass and the student can explain why the implementation works.

Working in the repo

1. New material → draft in drafts/ (or in place, marked unreviewed), with the status comment.
2. Broken things (dead links, wrong citations, code that won’t run) → just fix them, and note what changed.
3. Don’t hand-grow reference/papers/*/PAPERS.md; those come from tools/arxiv-collector/. Run the collector if the lists are stale.
4. Touched anything under exercises/? Run pytest.
5. Don’t commit unless asked. When you summarize, say plainly what’s still unreviewed versus fixed-and-checked.

Don’t

• Don’t reword the 2017 hand-written notes (notes/cs294-2017/, notes/sutton-barto-digest/).
• Don’t mark your own output reviewed.
• Don’t add a citation you haven’t verified.
• Don’t touch files outside the repo, shell config, or git history without being asked.
• Don’t delete the author’s data (e.g. tools/arxiv-collector/papers_database.json) on your own initiative.
• Don’t turn this into a framework: no setup.py, no CI, no package layout. It’s notes and exercises.

Source: AGENTS.md on GitHub.