Skip to content

Papers, by module

The papers behind each module, with links (arXiv where one exists). Each module starts with a short Read first list, in order: read those in full, and reproduce one figure from at least one. Skim the rest when a module or lab needs them.

For every paper, write four lines in your journal (paper-notes template): claim · evidence · compute · what I would test next.

Contents: 03 Optimization and training · 04 Transformers · 05 Pretraining and scaling · 06 Post-training · 07 Inference · 08 Evaluation and research · 09 Interpretability and safety · 10 Applied LLM systems · Staying current

Tip

Read a paper in three passes: (1) abstract, figures and conclusion, 10 minutes, decide whether to continue; (2) the method and the main experiment, noting every number you would need to reproduce it; (3) the appendix, which is where the hyperparameters and the caveats live.

03 Optimization and training

Read first 1. Loshchilov and Hutter 2019, Decoupled Weight Decay Regularization (AdamW): why weight decay is not L2 regularization under Adam. Lab 02 matches PyTorch's AdamW to 1e-10 over 25 steps. 2. McCandlish et al. 2018, An Empirical Model of Large-Batch Training: the critical batch size and the gradient noise scale. 3. Wortsman et al. 2023, Small-scale proxies for large-scale Transformer training instabilities: reproduce big-model instabilities on a small GPU.

Then - Kingma and Ba 2014, Adam - Yang et al. 2022, Tensor Programs V (μP: transfer hyperparameters across widths) - Micikevicius et al. 2017, Mixed Precision Training - Jordan 2024, Muon (blog) and Liu et al. 2025, Muon is Scalable for LLM Training - Hägele et al. 2024, Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations (WSD schedules and cooldowns)

04 Transformers

Read first 1. Vaswani et al. 2017, Attention Is All You Need 2. Su et al. 2021, RoFormer (RoPE): derive the rotation yourself. 3. Llama Team 2024, The Llama 3 Herd of Models: the most complete public account of a modern dense recipe. 4. DeepSeek-AI 2024, DeepSeek-V3 Technical Report: MLA, fine-grained MoE, FP8 training.

Then - Radford et al. 2019, Language Models are Unsupervised Multitask Learners (GPT-2) - Brown et al. 2020, Language Models are Few-Shot Learners (GPT-3) - Shazeer 2019, Fast Transformer Decoding: One Write-Head is All You Need (MQA) - Ainslie et al. 2023, GQA - Shazeer 2020, GLU Variants Improve Transformer - Zhang and Sennrich 2019, Root Mean Square Layer Normalization - Xiong et al. 2020, On Layer Normalization in the Transformer Architecture (pre-LN) - Touvron et al. 2023, LLaMA and Llama 2 - DeepSeek-AI 2024, DeepSeek-V2 (multi-head latent attention) - Fedus et al. 2021, Switch Transformers - Gu and Dao 2023, Mamba - Chen et al. 2023, Extending Context Window of Large Language Models via Position Interpolation - Peng et al. 2023, YaRN - Dosovitskiy et al. 2020, An Image is Worth 16x16 Words (ViT) - Radford et al. 2021, Learning Transferable Visual Models From Natural Language Supervision (CLIP) - Liu et al. 2023, Visual Instruction Tuning (LLaVA)

05 Pretraining and scaling

Read first 1. Hoffmann et al. 2022, Training Compute-Optimal Large Language Models (Chinchilla); then Besiroglu et al. 2024, Chinchilla Scaling: A replication attempt, for how fragile the fit is. Lab 08. 2. Penedo et al. 2024, The FineWeb Datasets: data quality as the main lever. 3. Rajbhandari et al. 2020, ZeRO: the memory accounting behind every large run. Lab 09.

Then - Kaplan et al. 2020, Scaling Laws for Neural Language Models - Muennighoff et al. 2023, Scaling Data-Constrained Language Models - Sardana et al. 2023, Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws - Li et al. 2024, DataComp-LM (DCLM) - Lee et al. 2021, Deduplicating Training Data Makes Language Models Better - Xie et al. 2023, DoReMi - Groeneveld et al. 2024, OLMo and OLMo Team 2024, 2 OLMo 2 Furious - Shoeybi et al. 2019, Megatron-LM - Narayanan et al. 2021, Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM (PTD-P) - Korthikanti et al. 2022, Reducing Activation Recomputation in Large Transformer Models (sequence parallelism) - Liu et al. 2023, Ring Attention - Zhao et al. 2023, PyTorch FSDP

06 Post-training

Read first 1. Ouyang et al. 2022, Training language models to follow instructions with human feedback (InstructGPT) 2. Rafailov et al. 2023, Direct Preference Optimization: derive the closed form. Lab 11. 3. Shao et al. 2024, DeepSeekMath (GRPO) and DeepSeek-AI 2025, DeepSeek-R1. Lab 12. 4. Hu et al. 2021, LoRA. Lab 10.

Then - Christiano et al. 2017, Deep Reinforcement Learning from Human Preferences - Schulman et al. 2017, PPO and 2015, GAE - Azar et al. 2023, A General Theoretical Paradigm to Understand Learning from Human Preferences (IPO) - Ethayarajh et al. 2024, KTO - Meng et al. 2024, SimPO - Gao et al. 2022, Scaling Laws for Reward Model Overoptimization - Zhou et al. 2023, LIMA - Bai et al. 2022, Training a Helpful and Harmless Assistant with RLHF and Constitutional AI - Yu et al. 2025, DAPO and Liu et al. 2025, Understanding R1-Zero-Like Training (Dr. GRPO) - Lambert et al. 2024, Tülu 3 - Lightman et al. 2023, Let's Verify Step by Step - Wei et al. 2022, Chain-of-Thought Prompting and Wang et al. 2022, Self-Consistency - Snell et al. 2024, Scaling LLM Test-Time Compute Optimally and Muennighoff et al. 2025, s1 - Hinton et al. 2015, Distilling the Knowledge in a Neural Network and Agarwal et al. 2024, On-Policy Distillation of Language Models (GKD) - Dettmers et al. 2023, QLoRA and Liu et al. 2024, DoRA

07 Inference

Read first 1. Pope et al. 2022, Efficiently Scaling Transformer Inference: the napkin math of serving. Lab 03. 2. Dao et al. 2022, FlashAttention. Lab 06. 3. Kwon et al. 2023, Efficient Memory Management for LLM Serving with PagedAttention (vLLM) 4. Leviathan et al. 2023, Fast Inference from Transformers via Speculative Decoding. Lab 14. 5. Frantar et al. 2023, GPTQ. Lab 13.

Then - Milakov and Gimelshein 2018, Online normalizer calculation for softmax - Dao 2023, FlashAttention-2 and Shah et al. 2024, FlashAttention-3 - Yu et al. 2022, Orca: A Distributed Serving System for Transformer-Based Generative Models (OSDI 2022; continuous batching) - Zheng et al. 2024, SGLang: Efficient Execution of Structured Language Model Programs - Agrawal et al. 2024, Sarathi-Serve (chunked prefill) - Zhong et al. 2024, DistServe (prefill–decode disaggregation) - Chen et al. 2023, Accelerating Large Language Model Decoding with Speculative Sampling - Cai et al. 2024, Medusa and Li et al. 2024, EAGLE - Dettmers et al. 2022, LLM.int8() - Xiao et al. 2023, SmoothQuant - Lin et al. 2024, AWQ - Rouhani et al. 2023, Microscaling Data Formats for Deep Learning (MX formats) - Tillet et al. 2019, Triton: An Intermediate Language and Compiler for Tiled Neural Network Computations (MAPL 2019)

08 Evaluation and research

Read first 1. Miller 2024, Adding Error Bars to Evals: the statistics of module 08 §2. Lab 15. 2. Chen et al. 2021, Evaluating Large Language Models Trained on Code: HumanEval and the unbiased pass@k estimator. 3. Zheng et al. 2023, Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena: judge biases and agreement. 4. Biderman et al. 2024, Lessons from the Trenches on Reproducible Evaluation of Language Models

Benchmarks - Hendrycks et al. 2020, MMLU and Wang et al. 2024, MMLU-Pro - Rein et al. 2023, GPQA and Phan et al. 2025, Humanity's Last Exam - Cobbe et al. 2021, GSM8K, Hendrycks et al. 2021, MATH, Glazer et al. 2024, FrontierMath - Jimenez et al. 2023, SWE-bench and Jain et al. 2024, LiveCodeBench - Zhou et al. 2023, IFEval - Yao et al. 2024, τ-bench, Mialon et al. 2023, GAIA, Xie et al. 2024, OSWorld, Chan et al. 2024, MLE-bench, METR 2024, RE-Bench - Kwa et al. 2025, Measuring AI Ability to Complete Long Tasks (METR time horizons) - Liang et al. 2022, Holistic Evaluation of Language Models (HELM) - Chiang et al. 2024, Chatbot Arena and Singh et al. 2025, The Leaderboard Illusion

Contamination, judges and harness sensitivity - Oren et al. 2023, Proving Test Set Contamination in Black Box Language Models - Shi et al. 2023, Detecting Pretraining Data from Large Language Models (Min-K% probability) - Yang et al. 2023, Rethinking Benchmark and Contamination for Language Models with Rephrased Samples - Zhang et al. 2024, A Careful Examination of Large Language Model Performance on Grade School Arithmetic (GSM1k) - Wang et al. 2023, Large Language Models are not Fair Evaluators (position bias) - Panickssery et al. 2024, LLM Evaluators Recognize and Favor Their Own Generations - Dubois et al. 2024, Length-Controlled AlpacaEval - Sclar et al. 2023, Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design - Alzahrani et al. 2024, When Benchmarks are Targets

Research method and statistics - Card et al. 2020, With Little Power Comes Great Responsibility - Bouthillier et al. 2021, Accounting for Variance in Machine Learning Benchmarks - Madaan et al. 2024, Quantifying Variance in Evaluation Benchmarks - Dodge et al. 2019, Show Your Work: Improved Reporting of Experimental Results (EMNLP 2019)

09 Interpretability and safety

Read first 1. Elhage et al. 2021, A Mathematical Framework for Transformer Circuits: QK/OV circuits and composition. 2. Olsson et al. 2022, In-context Learning and Induction Heads. Lab 16. 3. Elhage et al. 2022, Toy Models of Superposition 4. Bricken et al. 2023, Towards Monosemanticity 5. Greshake et al. 2023, Not what you've signed up for: indirect prompt injection

Features and sparse autoencoders - Cunningham et al. 2023, Sparse Autoencoders Find Highly Interpretable Features in Language Models - Templeton et al. 2024, Scaling Monosemanticity - Gao et al. 2024, Scaling and evaluating sparse autoencoders (TopK) - Rajamanoharan et al. 2024, Gated SAEs and JumpReLU SAEs - Bussmann et al. 2024, BatchTopK Sparse Autoencoders - Lieberum et al. 2024, Gemma Scope - Dunefsky et al. 2024, Transcoders Find Interpretable LLM Feature Circuits - Chanin et al. 2024, A is for Absorption - Karvonen et al. 2025, SAEBench - Kantamneni et al. 2025, Are Sparse Autoencoders Useful? A Case Study in Sparse Probing

Circuits, patching, probing and steering - Wang et al. 2022, Interpretability in the Wild: a Circuit for Indirect Object Identification (IOI) - Meng et al. 2022, Locating and Editing Factual Associations in GPT (ROME) - Geva et al. 2020, Transformer Feed-Forward Layers Are Key-Value Memories - Nanda et al. 2023, Progress measures for grokking via mechanistic interpretability - Conmy et al. 2023, Towards Automated Circuit Discovery (ACDC) - Zhang and Nanda 2023, Towards Best Practices of Activation Patching and Heimersheim and Nanda 2024, How to use and interpret activation patching - Kramár et al. 2024, AtP*: An efficient and scalable method for localizing LLM behaviour to components - Makelov et al. 2023, Is This the Subspace You Are Looking For? - Marks et al. 2024, Sparse Feature Circuits - Ameisen et al. 2025, Circuit Tracing and Lindsey et al. 2025, On the Biology of a Large Language Model - Alain and Bengio 2016, Understanding intermediate layers using linear classifier probes; Hewitt and Liang 2019, Designing and Interpreting Probes with Control Tasks (EMNLP 2019); Belinkov 2021, Probing Classifiers - Burns et al. 2022, Discovering Latent Knowledge Without Supervision and Marks and Tegmark 2023, The Geometry of Truth - Belrose et al. 2023, Eliciting Latent Predictions with the Tuned Lens - Turner et al. 2023, Steering Language Models With Activation Engineering; Panickssery et al. 2023, Steering Llama 2 via Contrastive Activation Addition; Zou et al. 2023, Representation Engineering - Arditi et al. 2024, Refusal in Language Models Is Mediated by a Single Direction - Sharkey et al. 2025, Open Problems in Mechanistic Interpretability

Alignment failure modes - Amodei et al. 2016, Concrete Problems in AI Safety - Skalse et al. 2022, Defining and Characterizing Reward Hacking - Sharma et al. 2023, Towards Understanding Sycophancy in Language Models - Turpin et al. 2023, Language Models Don't Always Say What They Think; Lanham et al. 2023, Measuring Faithfulness in Chain-of-Thought Reasoning; Chen et al. 2025, Reasoning Models Don't Always Say What They Think - Baker et al. 2025, Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation and Korbak et al. 2025, Chain of Thought Monitorability - Hubinger et al. 2024, Sleeper Agents - Greenblatt et al. 2024, Alignment faking in large language models - van der Weij et al. 2024, AI Sandbagging: Language Models can Strategically Underperform on Evaluations - Meinke et al. 2024, Frontier Models are Capable of In-context Scheming - Denison et al. 2024, Sycophancy to Subterfuge - Betley et al. 2025, Emergent Misalignment - Greenblatt et al. 2023, AI Control: Improving Safety Despite Intentional Subversion

Misuse, security and governance - Wei et al. 2023, Jailbroken: How Does LLM Safety Training Fail? - Zou et al. 2023, Universal and Transferable Adversarial Attacks on Aligned Language Models (GCG) - Anil et al. 2024, Many-shot jailbreaking - Perez et al. 2022, Red Teaming Language Models with Language Models - Sharma et al. 2025, Constitutional Classifiers - Wallace et al. 2024, The Instruction Hierarchy; Chen et al. 2024, StruQ - Debenedetti et al. 2024, AgentDojo and 2025, Defeating Prompt Injections by Design (CaMeL) - Beurer-Kellner et al. 2025, Design Patterns for Securing LLM Agents against Prompt Injections - Shevlane et al. 2023, Model evaluation for extreme risks and Phuong et al. 2024, Evaluating Frontier Models for Dangerous Capabilities - Mitchell et al. 2019, Model Cards for Model Reporting - Anthropic, Responsible Scaling Policy (current version)

10 Applied LLM systems

Read first 1. Lewis et al. 2020, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks 2. Anthropic 2024, Building Effective Agents 3. Liu et al. 2023, Lost in the Middle 4. Yao et al. 2022, ReAct

Retrieval - Karpukhin et al. 2020, Dense Passage Retrieval - Reimers and Gurevych 2019, Sentence-BERT and Wang et al. 2022, Text Embeddings by Weakly-Supervised Contrastive Pre-training (E5) - Khattab and Zaharia 2020, ColBERT - Nogueira and Cho 2019, Passage Re-ranking with BERT - Cormack et al. 2009, Reciprocal Rank Fusion Outperforms Condorcet and Individual Rank Learning Methods (SIGIR 2009) - Kusupati et al. 2022, Matryoshka Representation Learning - Malkov and Yashunin 2016, HNSW and Jégou et al. 2011, Product Quantization for Nearest Neighbor Search (IEEE TPAMI) - Gao et al. 2022, Precise Zero-Shot Dense Retrieval without Relevance Labels (HyDE) - Anthropic 2024, Contextual Retrieval - Thakur et al. 2021, BEIR and Muennighoff et al. 2022, MTEB

RAG evaluation - Es et al. 2023, Ragas and Saad-Falcon et al. 2023, ARES - Asai et al. 2023, Self-RAG

Agents, tools and structured output - Schick et al. 2023, Toolformer and Patil et al. 2023, Gorilla - Yang et al. 2024, SWE-agent: Agent-Computer Interfaces - Anthropic 2025, Effective context engineering for AI agents and Writing effective tools for agents - Willard and Louf 2023, Efficient Guided Generation for Large Language Models (Outlines) and Dong et al. 2024, XGrammar - Tam et al. 2024, Let Me Speak Freely?

Cost and routing - Chen et al. 2023, FrugalGPT - Ong et al. 2024, RouteLLM: Learning to Route LLMs with Preference Data

Staying current

  • Daily: skim Hugging Face daily papers (arXiv cs.CL and cs.LG, filtered by the community). Read titles; open at most two.
  • Weekly: lab research blogs (Anthropic, OpenAI, Google DeepMind, Meta, DeepSeek, Qwen, AI2), the Transformer Circuits Thread, GPU MODE lectures.
  • Quarterly: re-check the frontier sections of modules 05–07 and the benchmark table in module 08.
  • A paper is not "read" until you can state its claim, the evidence and one experiment you would run next. Write those four lines, or it didn't happen.