Why label smoothing helps beam search but hurts distillation
A longer review of why label smoothing improves calibration and sequence decoding in Transformers while often making the model a weaker distillation teacher.
Research notes
Public thinking from the edge of the research process.
These memos stay intentionally compact. Each one isolates a mechanism, paper, or question so the work remains visible before it becomes polished.
38 published
Public memos so far
Feb to Aug 2026
Current public note window
One question each
A note format built for clarity over coverage
Format
How these notes are written.
- 01 Keep it narrow Each memo stays centered on one claim, one paper, or one mechanism.
- 02 Prefer signal over summary The aim is a strong takeaway, not exhaustive coverage.
- 03 Leave open questions visible The most useful part is often what still feels unresolved.
Recent focus
A few themes keep resurfacing.
Self-consistency
Preference optimization
Layer norm placement
Truthfulness
Length extrapolation
Feature geometry
Induction heads
Truth directions
Reversal curse
Attention sinks
Low-rank adaptation
Gated MLPs
muP scaling
Constitutional AI
Tail truncation
Compute-optimal scaling
Speculative decoding
Weight tying
RMSNorm
Tuned lens
Memorization
Emergence metrics
Goal misgeneralization
Function vectors
Sparse MoE
CoT faithfulness
Residual stream
Soft prompts
Model merging
Label smoothing
- 2026-08-16: Why label smoothing can improve Transformer calibration and beam-search decoding while simultaneously weakening distillation signals.
- 2026-08-09: Why some fine-tuned language models can be merged by simple weight arithmetic when their updates stay aligned around a shared pretrained base.
- 2026-08-05: Why soft prompt tuning becomes competitive mainly when a frozen model is already large enough to expose reusable task circuits.
- 2026-07-26: Why transformer components compose by writing to and reading from a shared residual-stream workspace.
- 2026-07-22: When visible chain-of-thought is a polished rationale rather than the causal process behind an answer.
- 2026-07-22: Why sparse MoE layers decouple parameter capacity from per-token compute.
- 2026-07-22: Whether few-shot prompts induce a portable task vector that can be extracted from specific heads and reused in new contexts.
- 2026-07-17: Why goal misgeneralization is a distinct failure mode from overfitting and reward misspecification.
Latest
The newest memo goes first.
Everything else stays browseable below, but the freshest research question should be immediate.
Why simple weight arithmetic can merge fine-tuned language models
A longer review of why some independently fine-tuned language models can be merged by simple weight arithmetic, and where interference breaks the trick.
Open memoWhy soft prompt tuning only catches up at scale
A longer review of why continuous soft prompts become competitive mainly when frozen language models are already large enough to expose reusable task circuits.
Open memoWhy the residual stream is a shared linear workspace
A longer review of why transformer components compose through a common residual-stream communication substrate, and why that matters for interpretability and alignment.
Open memoWhy chain-of-thought can be causally unfaithful
A longer review of when an LLM's visible reasoning stops reflecting the causal process behind its answer, and why that matters for alignment.
Open memoWhy sparse MoE layers scale without proportional compute
A longer review of why sparse expert routing buys parameter scale without paying dense compute on every token, and where the trade turns back into systems overhead.
Open memoWhy function vectors can trigger in-context tasks
A longer review of why task information can be packaged into a portable residual-stream vector, how specific heads extract it, and where that mechanistic story still breaks.
Open memoWhy goal misgeneralization is not just overfitting
A longer review of why competent proxy pursuit is a distinct alignment failure mode, how it differs from capability failure, and why correct rewards may still be insufficient.
Open memoWhy many emergent abilities are measurement artifacts
A longer review of whether dramatic capability jumps mostly come from coarse evaluation metrics, limited measurement resolution, or real loss thresholds.
Open memoWhy LLMs memorize rare repeated strings
A longer review of why exact memorization concentrates on rare, repeated, high-entropy strings, and what that reveals about scaling and data hygiene.
Open memoWhy the tuned lens beats the logit lens
A longer review of why raw direct decoding misreads intermediate transformer states, how representation drift creates probe artifacts, and why affine translators help.
Open memoWhy RMSNorm works without mean-centering
A longer review of why RMSNorm preserves most of LayerNorm's optimization benefits, what stable activation scale buys, and where mean subtraction still matters.
Open memoWhy weight tying helps language models
A longer review of why sharing input and output embeddings improves language models, what shared lexical geometry buys, and where the assumption breaks.
Open memoWhy speculative decoding preserves the target distribution
A longer review of why draft-and-verify generation can accelerate LLM inference without changing the target model's sampling distribution.
Open memoWhy Chinchilla scaling prefers more tokens than bigger models
A longer review of why compute-optimal LLM training often favors smaller models trained on more tokens, and what that reveals about undertraining.
Open memoWhy open-ended decoding needs tail truncation
A longer review of why likelihood-maximizing decoding degenerates in open-ended generation, why nucleus sampling works, and what later theory says about the tail.
Open memoWhy Constitutional AI can replace many harmlessness labels
A longer review of why written principles plus AI-generated critiques can improve harmlessness, and where that substitution for human labels breaks.
Open memoWhy muP makes hyperparameters transfer across scale
A longer review of why maximal update parameterization keeps learning-rate and scale-sensitive hyperparameters stable as transformers grow wider.
Open memoArchive
The full memo archive.
Ordered newest first so the cadence and evolving interests are easy to scan.
Why label smoothing helps beam search but hurts distillation
Why reshaping target distributions improves calibration for Transformer decoding while erasing some of the dark knowledge distillation needs.
Read memoWhy simple weight arithmetic can merge fine-tuned language models
Why a shared pretrained anchor, low-loss basin geometry, and update alignment can make model merging work.
Read memoWhy the residual stream is a shared linear workspace
Why transformer circuits compose by exchanging information through a shared residual-stream communication medium.
Read memoWhy chain-of-thought can be causally unfaithful
Why visible reasoning can be a polished rationalization rather than the process that actually caused an LLM's answer.
Read memoWhy sparse MoE layers scale without proportional compute
Why conditional expert routing can add vast parameter capacity to language models without forcing dense compute on every token.
Read memoWhy function vectors can trigger in-context tasks
Why a small set of attention heads can package a demonstrated task into a portable residual-stream vector.
Read memoWhy goal misgeneralization is not just overfitting
Why competent proxy pursuit is a distinct failure mode from ordinary overfitting and reward misspecification.
Read memoWhy many emergent abilities are measurement artifacts
Why dramatic-looking capability jumps often come from coarse metrics, while some thresholds may only survive in loss space.
Read memoWhy LLMs memorize rare repeated strings
Why exact memorization concentrates on rare, repeated, high-entropy strings instead of uniformly covering the training set.
Read memoWhy the tuned lens beats the logit lens
Why intermediate hidden states need layer-specific translation before they can be read faithfully in token space.
Read memoWhy RMSNorm works without mean-centering
Why stable activation scale often matters more than zero-mean activations in deep transformer optimization.
Read memoWhy weight tying helps language models
Why sharing input and output embeddings regularizes lexical geometry and often improves perplexity.
Read memoWhy speculative decoding preserves the target distribution
Why draft-and-verify generation can speed up autoregressive sampling without changing the large model's distribution.
Read memoWhy Chinchilla scaling prefers more tokens than bigger models
Why compute-optimal language-model training often rewards smaller models trained longer on more data.
Read memoWhy open-ended decoding needs tail truncation
Why likelihood maximization collapses into repetition in open-ended generation, and why truncating the unreliable tail works better.
Read memoWhy Constitutional AI can replace many harmlessness labels
Why written principles plus AI critiques can substitute for much direct harmlessness labeling when the base model already understands the norms.
Read memoWhy muP makes hyperparameters transfer across scale
Why width-aware parameter scaling keeps learning rates and related tuning choices stable across transformer scale.
Read memoWhy SwiGLU replaced standard Transformer MLPs
Why gated feed-forward blocks beat plain ReLU and GELU MLPs, even under compute-matched comparisons.
Read memoWhy low-rank updates are enough for LLM adaptation
Why downstream specialization often occupies a small weight-space subspace and can be captured with low-rank updates.
Read memoWhy attention sinks stabilize long-context decoding
Why a few early tokens absorb surplus attention mass and become necessary for stable streaming inference.
Read memoWhy self-consistency decoding improves reasoning
Why sampling several chains of thought and aggregating answers approximates marginalization over noisy reasoning paths.
Read memoWhy next-token prediction creates the reversal curse
Why LLMs often learn directional factual retrieval without learning stable reverse access to the same relation.
Read memoCan hidden states reveal what an LLM thinks is true?
When latent truth directions survive misleading prompts, and where that evidence remains fragile.
Read memoWhy induction heads emerge as a phase change
Why a specific copy circuit appears abruptly during training and tracks the onset of copy-based in-context learning.
Read memoWhy TruthfulQA shows inverse scaling
Why larger language models can become less truthful when scaling mostly improves imitation rather than evidence-sensitive abstention.
Read memoDPO as implicit reward optimization under KL constraints
When DPO matches KL-regularized reward optimization, and where the approximation starts to leak.
Read memoPre-LN vs Post-LN
How layer norm placement shapes gradient flow and training stability in deep Transformers.
Read memoHard attention and Turing completeness
Architectural ingredients, formal expressivity, and what they say about real-world attention.
Read memoThe softmax bottleneck in language modeling
Why a low-rank output layer limits expressivity and how mixture-of-softmaxes expands it.
Read memoALiBi vs RoPE
Why ALiBi extrapolates to longer contexts more reliably than RoPE.
Read memoFFN layers as key-value memories
Evidence that mid-layer MLPs store and retrieve factual associations.
Read memoAttention as modern Hopfield memory
A compact review of the Hopfield-network view of attention, storage capacity, and retrieval.
Read memoIn-context learning as implicit gradient descent
Evidence that Transformers can implement gradient-descent-like procedures in context.
Read memoGrokking as a phase transition in transformer training
How weight decay can trigger a delayed shift from memorization to algorithmic generalization.
Read memoSparsity as a control knob for superposition
How sparsity penalties can flip a model between dense superposition and cleaner feature geometry.
Read memoEvaluating IOI circuits
Faithfulness, completeness, and minimality as standards for mechanistic explanations.
Read memoBootstrapping a weekly research memo
A lightweight public writing habit for turning weekly notes into usable research output.
Read memoTopic log
A running ledger of narrow questions.
Useful as a sanity check against repetition and as a map of what the memo series is actually circling.
- August 9, 2026: why some fine-tuned language-model updates compose cleanly in weight space while others interfere or cancel out.
- July 26, 2026: why transformer components compose by writing to and reading from a shared linear residual-stream workspace.
- July 22, 2026: why chain-of-thought can improve reasoning performance while still failing as a causal explanation of the answer.
- July 22, 2026: why sparse MoE layers decouple parameter capacity from per-token compute through conditional expert routing.
- July 22, 2026: why some few-shot prompts appear to induce a portable task vector that can be extracted from a small set of causal attention heads.
- July 17, 2026: why goal misgeneralization is a distinct failure mode where capability generalizes while the pursued objective does not.
- July 8, 2026: why exact LLM memorization concentrates on rare, repeated, high-entropy strings that become retrievable from the right prefix.
- July 7, 2026: why intermediate transformer states need a learned translation before token-space readouts can be trusted.
- July 1, 2026: why RMSNorm keeps most of LayerNorm's benefits by stabilizing activation scale even without explicit mean-centering.
- June 30, 2026: why sharing input and output embeddings helps language models read and predict words in the same lexical space.
- June 29, 2026: why speculative decoding can draft several tokens ahead yet still sample exactly from the target model after verification.
- June 28, 2026: why compute-optimal scaling usually favors more training tokens over a much larger but undertrained model.
- June 26, 2026: why open-ended generation needs dynamic tail truncation instead of likelihood-maximizing decoding.
- June 25, 2026: why written principles plus AI-generated critiques can replace many direct harmlessness labels in alignment training.
- June 24, 2026: why maximal update parameterization makes learning-rate and optimizer-scale choices transfer across transformer width.
- June 23, 2026: why SwiGLU-style gated MLPs outperform standard transformer feed-forward blocks.
- June 22, 2026: why many downstream LLM adaptations can be captured by low-rank updates instead of dense retraining.
- June 17, 2026: why long-context decoding depends on early attention sinks that absorb surplus softmax mass.
- June 16, 2026: why sampling several chains of thought and aggregating answers improves reasoning without changing model weights.
- June 15, 2026: why next-token prediction learns directional fact retrieval but not stable reverse lookup.