Resources (curated: fewer, better)¶
The courses, books and long-form writing worth your time alongside the labs, most of them free. Each entry says what it is for and when in the curriculum to use it. Paper lists live in papers.md; visual guides and PDFs per topic live in the library.
Contents: Courses · Free books · Paid books worth buying · Long-form writing · Tools to learn · Communities · If you have no GPU · Anti-list
Tip
Pick one course per phase and finish it, doing the exercises. Starting five courses and finishing none is the most common way self-study fails. Where a course and a lab cover the same thing, do the lab first, then use the course to fill gaps.
Courses, in the order to take them¶
| # | Course | Cost | Use it for | Pairs with |
|---|---|---|---|---|
| 1 | Karpathy, Neural Networks: Zero to Hero, including Let's reproduce GPT-2 (124M) | Free | The build-from-scratch spine: backprop, language models, tokenizers, GPT | Labs 01, 04, 05 |
| 2 | Stanford CS336, Language Modeling from Scratch | Free (lectures and assignments online) | University-level version of this curriculum; do its assignments alongside labs 04–09 | Modules 04–07 |
| 3 | GPU MODE lectures and Discord | Free | Kernels, Triton, CUDA, profiling | Lab 06, module 02 |
| 4 | OpenAI Spinning Up in Deep RL plus Sutton and Barto (chapters 1–6 and 13) | Free | Policy gradients before RLHF, DPO and GRPO | Module 06, labs 11–12 |
| 5 | MIT 6.5940, TinyML and Efficient Deep Learning Computing (Song Han) | Free | Quantization, pruning, distillation, efficient inference | Module 07, lab 13 |
| 6 | ARENA | Free | Hands-on mechanistic interpretability (TransformerLens, induction heads, SAEs) and RL for LLMs | Module 09, lab 16 |
| 7 | Hugging Face Learn (LLM, deep RL and agents courses) | Free | Practical library skills: transformers, datasets, TRL, agents |
Module 10 |
Fill-in courses, when you need them
- Math: MIT 18.06 Linear Algebra (Gilbert Strang) and 3Blue1Brown's visual series on neural networks for intuition (module 01).
- Deep learning breadth: fast.ai Practical Deep Learning (top-down, code first); Stanford CS231n (vision, excellent backprop notes); Stanford CS224N (NLP).
- In India: NPTEL hosts free IIT courses, including Mitesh Khapra's Deep Learning (IIT Madras), with an optional low-cost proctored certificate that some Indian employers recognize.
- AI safety and governance: BlueDot Impact's free AI safety and governance courses (cohort-based, with reading groups), and Neel Nanda's guide to getting started in mechanistic interpretability.
Free books¶
| Book | What it is for |
|---|---|
| Deisenroth, Faisal and Ong, Mathematics for Machine Learning | Linear algebra, calculus and probability for module 01 |
| Simon Prince, Understanding Deep Learning (2023) | The clearest modern deep learning textbook, with figures and notebooks |
| François Fleuret, The Little Book of Deep Learning | A short, phone-sized summary for revision before interviews |
| Zhang et al., Dive into Deep Learning | Runnable notebooks for every concept |
| Goodfellow, Bengio and Courville, Deep Learning (2016) | Classic reference for optimization and regularization theory |
| Kevin Murphy, Probabilistic Machine Learning | Rigorous probabilistic reference |
| James et al., An Introduction to Statistical Learning | Statistics foundations behind module 08 (resampling, testing) |
| Jurafsky and Martin, Speech and Language Processing (3rd edition draft) | NLP foundations, including information retrieval and evaluation |
| Manning, Raghavan and Schütze, Introduction to Information Retrieval | BM25, evaluation metrics, indexing: the theory behind lab 17 |
| Sutton and Barto, Reinforcement Learning: An Introduction | RL foundations |
| Nathan Lambert, The RLHF Book | Post-training in depth: reward models, PPO, DPO, RL with verifiable rewards |
| Google DeepMind, How to Scale Your Model | TPU/GPU scaling math: rooflines, sharding, inference |
| Hugging Face, The Ultra-Scale Playbook | Data, tensor, pipeline and context parallelism in practice |
Paid books worth buying¶
- Sebastian Raschka, Build a Large Language Model (From Scratch) (2024): gentle companion to labs 04–05.
- Christopher Bishop and Hugh Bishop, Deep Learning: Foundations and Concepts (2023): rigorous reference (also readable free online from the authors' site).
- Chip Huyen, AI Engineering (2025): applied systems and the founder track.
- Chip Huyen, Designing Machine Learning Systems (2022): ML system design interviews.
- Hwu, Kirk and El Hajj, Programming Massively Parallel Processors (4th edition): CUDA fundamentals.
Note
Indian editions of many technical books are sold at a fraction of the international price; check before importing.
Long-form writing worth rereading¶
Performance and scale - Horace He, Making Deep Learning Go Brrrr From First Principles: compute-, memory- and overhead-bound, in one essay. - kipply, Transformer Inference Arithmetic - Simon Boehm, How to Optimize a CUDA Matmul Kernel
Training and research practice - Andrej Karpathy, A Recipe for Training Neural Networks - Lilian Weng's blog: careful surveys of RL, agents, hallucination, reward hacking.
Transformers, visually - Jay Alammar, The Illustrated Transformer - Harvard NLP, The Annotated Transformer - Distill and the Transformer Circuits Thread: interactive interpretability articles.
Applied systems and evals - Anthropic, Building Effective Agents, Contextual Retrieval, Effective context engineering for AI agents - Hamel Husain, Your AI Product Needs Evals - Eugene Yan, Patterns for Building LLM-based Systems and Products
Tools to learn¶
| Tool | For | Module |
|---|---|---|
| lm-evaluation-harness | Standard benchmark evals of open models; --log_samples saves per-item outputs for statistics |
08 |
| Inspect (UK AI Security Institute) | Writing your own evals, including agentic and safety evals | 08, 09 |
| TransformerLens | Hooks, caching and patching on open models | 09 |
| Neuronpedia | Browse SAE features and attribution graphs in the browser | 09 |
| FAISS, or a vector database with hybrid search | Approximate nearest neighbor search at scale | 10 |
Communities (contribute, don't just lurk)¶
GPU MODE Discord · EleutherAI Discord · Hugging Face forums · r/LocalLLaMA · Latent Space · Bengaluru meetups and AI Tinkerers.
Answer questions there after each lab, share your measured numbers (with error bars), and review others' write-ups; that is where collaborators and referrals come from.
If you have no GPU¶
- Every lab's tests run on CPU, and the Triton tests use the interpreter.
- Free hosted notebooks (Google Colab, Kaggle) provide time-limited GPUs that are enough for the scale-up runs in most labs; check their current quotas.
- Interpretability: GPT-2 small runs on a laptop CPU; Neuronpedia needs only a browser.
- Evals and statistics (lab 15, module 08) need no GPU at all.
Anti-list¶
Skip "master AI in 30 days" content, framework-of-the-month tutorials, and prompt-engineering certificates. If a resource doesn't make you build, derive or measure something, it's entertainment.