DPO as implicit reward optimization under KL constraints
A deep review of when Direct Preference Optimization is equivalent to KL-regularized reward optimization and where that equivalence breaks.
Research notes
Public thinking from the edge of the research process.
These memos stay intentionally compact. Each one isolates a mechanism, paper, or question so the work remains visible before it becomes polished.
12 published
Public memos so far
Feb to Mar 2026
Current public note window
One question each
A note format built for clarity over coverage
Format
How these notes are written.
- 01 Keep it narrow Each memo stays centered on one claim, one paper, or one mechanism.
- 02 Prefer signal over summary The aim is a strong takeaway, not exhaustive coverage.
- 03 Leave open questions visible The most useful part is often what still feels unresolved.
Recent focus
A few themes keep resurfacing.
Preference optimization
Layer norm placement
Hard attention
Length extrapolation
Feature geometry
KV memories
Latest
The newest memo goes first.
Everything else stays browseable below, but the freshest research question should be immediate.
Archive
The full memo archive.
Ordered newest first so the cadence and evolving interests are easy to scan.
DPO as implicit reward optimization under KL constraints
When DPO matches KL-regularized reward optimization, and where the approximation starts to leak.
Read memoPre-LN vs Post-LN
How layer norm placement shapes gradient flow and training stability in deep Transformers.
Read memoHard attention and Turing completeness
Architectural ingredients, formal expressivity, and what they say about real-world attention.
Read memoThe softmax bottleneck in language modeling
Why a low-rank output layer limits expressivity and how mixture-of-softmaxes expands it.
Read memoALiBi vs RoPE
Why ALiBi extrapolates to longer contexts more reliably than RoPE.
Read memoFFN layers as key-value memories
Evidence that mid-layer MLPs store and retrieve factual associations.
Read memoAttention as modern Hopfield memory
A compact review of the Hopfield-network view of attention, storage capacity, and retrieval.
Read memoIn-context learning as implicit gradient descent
Evidence that Transformers can implement gradient-descent-like procedures in context.
Read memoGrokking as a phase transition in transformer training
How weight decay can trigger a delayed shift from memorization to algorithmic generalization.
Read memoSparsity as a control knob for superposition
How sparsity penalties can flip a model between dense superposition and cleaner feature geometry.
Read memoEvaluating IOI circuits
Faithfulness, completeness, and minimality as standards for mechanistic explanations.
Read memoBootstrapping a weekly research memo
A lightweight public writing habit for turning weekly notes into usable research output.
Read memo