James, Witten, Hastie, Tibshirani
Notes Best practical stats-based ML introduction ever written.
Recommended Media
A curated collection of books, papers, lectures, and courses that have shaped my thinking about computation, mathematics, intelligence, and the nature of mind.
218
Items
178
Books
5
Videos
192
Completed
AI & Machine Learning
56 itemsRussell & Norvig
Notes The standard AI textbook. Search, logic, planning, learning, language.
Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser, Polosukhin
Notes Introduced the Transformer architecture. The paper that started everything.
Devlin, Chang, Lee, Toutanova
Notes Bidirectional pre-training via masked language modeling. Defined the pre-train/fine-tune paradigm.
Wei, Wang, Schuurmans, Bosma, Xia, Chi, Le, Zhou
Notes Step-by-step reasoning via prompting. Unlocked a new capability class.
Bai, Kadavath, Kundu, Askell, Kernion, Jones, Chen, et al.
Notes Self-critique and revision using principles instead of human labels.
Christiano, Leike, Brown, Martic, Legg, Amodei
Notes Foundational RLHF paper. Learning reward models from human comparisons.
Rafailov, Sharma, Mitchell, Ermon, Manning, Finn
Notes Bypasses reward modeling entirely. Simpler alignment, same results.
Dao, Fu, Ermon, Rudra, Ré
Notes IO-aware attention that is both faster and uses less memory. Essential infrastructure.
Child, Gray, Radford, Sutskever
Notes Sparse attention patterns for long-range dependencies. O(n√n) attention.
Douglas Hofstadter
Notes Consciousness, self-reference, formal systems. Mind-altering.
Brown, Mann, Ryder, Subbiah, Kaplan, Dhariwal, Neelakantan, et al.
Notes 175B parameters. In-context learning emerges at scale. Changed the field.
Radford, Wu, Child, Luan, Amodei, Sutskever
Notes Showed large LMs can perform tasks zero-shot. Introduced the scaling intuition.
Touvron, Lavril, Izacard, Martinet, Lachaux, Lacroix, et al.
Notes Open-weight models competitive with GPT-3. Catalyzed the open-source LLM ecosystem.
Michael Nielsen
Notes Visual, intuitive intro to neural nets. Build understanding from scratch.
Shazeer, Mirhoseini, Maziarz, Davis, Le, Hinton, Dean
Notes Mixture of Experts with learned gating. Conditional computation at scale.
Kevin P. Murphy
Notes VAEs, flows, diffusion, GPs, causal inference — the advanced sequel.
Kevin P. Murphy
Notes Comprehensive modern ML textbook with solid probabilistic foundations.
Yao, Zhao, Yu, Du, Shafran, Narasimhan, Cao
Notes Interleaving reasoning traces and actions. The prompting pattern behind most LLM agents.
Sutton & Barto
Notes The RL bible. Bandits to policy gradients to planning.
Kaplan, McCandlish, Henighan, Brown, Chess, Child, Gray, Radford, Wu, Amodei
Notes Power-law relationships between compute, data, parameters, and loss. Empirical scaling science.
Jurafsky & Martin
Notes The canonical NLP book, updated for the LLM era.
Richard McElreath
Notes The modern Bayesian workflow explained with clarity and care.
Andrej Karpathy
Notes Seminal blog post demonstrating char-level RNN power. Shakespeare, LaTeX, kernel code generation.
Allen B. Downey
Notes Bayesian statistics by writing code. Ideal computational on-ramp.
Allen B. Downey
Notes Probability and statistics for programmers, learn by coding.
Schick, Dwivedi-Yu, Dessì, Raber, Lomeli, Zettlemoyer, Cancedda, Scialom
Notes LMs learning when and how to call external tools. Key step toward agentic LMs.
Hoffmann, Borgeaud, Mensch, Buchatskaya, Cai, Rutherford, et al.
Notes Showed most LLMs were undertrained. Optimal ratio of data to parameters.
Ouyang, Wu, Jiang, Almeida, Wainwright, Mishkin, Zhang, et al.
Notes RLHF applied to GPT-3. The bridge from raw LM to useful assistant.
Simon J. D. Prince
Notes Modern DL explained cleanly without hype.
Ray Solomonoff
Notes Foundational paper on algorithmic probability and universal induction. Basis for AIXI.
Csaba Szepesvári
Notes Free condensed RL theory book; rigorous and compact. Alternative formal RL resource.
Marcus Hutter, Elliot Catt, David Quarel
Notes Newer textbook presenting AIXI and related universal agent theory with updated structure; suitable after the core UAI book.
Stuart J. Russell, Peter Norvig
Notes Standard AI textbook covering search, logic, probabilistic reasoning, RL, multiagent systems, and more. Canonical comprehensive AI text.
David Barber
Judea Pearl
Ray Solomonoff
Notes Later work on convergence properties of Solomonoff induction.
David Silver
Notes Comprehensive lecture series covering RL foundations.
Ian Goodfellow, Yoshua Bengio, Aaron Courville
François Chollet
Catherine F. Higham, Desmond J. Higham
Notes Perspective of deep learning from applied math. Bridges math with neural nets.
Aston Zhang, Zachary C. Lipton, Mu Li, Alexander J. Smola
Notes Free open-source deep learning book with code and math integrated. Interactive deep learning resource with runnable code.
Douglas R. Hofstadter
Notes Explores analogy and cognition via computational models. Classic work on analogy and cognition modeling.
Kevin P. Murphy
Marcus Hutter
Notes Technical talk touching on universal induction and AI foundations.
Marcus Hutter
Notes Lecture on AIXI and universal intelligence theory with deep insights.
Christopher M. Bishop
Moritz Hardt, Benjamin Recht
Notes Modern graduate ML text with causal inference, decision making, and ML foundations. Accessible free textbook with strong conceptual framing.
Daphne Koller, Nir Friedman
Richard S. Sutton, Andrew G. Barto
Notes Mathematical RL fundamentals (MDPs, value functions, dynamic programming, approximate methods). RL foundational text that bridges theory and practice.
Judea Pearl, Dana Mackenzie
Trevor Hastie, Robert Tibshirani, Jerome Friedman
Pedro Domingos
Andrej Karpathy
Review Seminal blog post demonstrating the power of character-level RNNs. Shows Shakespeare generation, Wikipedia generation, LaTeX generation, and Linux kernel code generation. The visualizations of …
Marcus Hutter
Notes Formal theory of universal agents combining Solomonoff induction and sequential decision theory; foundational for AGI theory.
Douglas Hofstadter
Notes Classic exploration of self-reference, formal systems, and the nature of mind.
Mathematics
33 itemsRichard Hammack
Notes Bridge from computation to proof. Sets, logic, induction, functions.
Silvanus P. Thompson
Notes Still the gentlest and most irreverent intro to calculus ever written.
William F. Trench
Notes A full ODE treatment including BVPs and series methods.
Blitzstein & Hwang
Notes Best modern probability textbook. Stat 110 at Harvard.
Jim Hefferon
Notes Clear, concept-first linear algebra without mysticism.
Deisenroth, Faisal, Ong
Notes Unified LA + multivariate calc + probability for ML-minded people.
Paul Halmos
Notes Bite-sized set theory for working mathematicians. Deceptively deep.
Timothy Gowers (ed.)
Notes 1000+ pages surveying all of modern mathematics. The map of math.
Charles C. Pinter
J.H. van Lint, R.M. Wilson
Arieh Iserles
Notes Rigorous introduction to the numerical analysis of ODEs, including stability and stiffness.
David S. Dummit, Richard M. Foote
Larry Wasserman
William Feller
Andrew Gelman, John B. Carlin, Hal S. Stern, David B. Dunson, Aki Vehtari, Donald B. Rubin
Notes Applied Bayesian inference with computing methods. The standard Bayesian statistics reference.
Bartosz Milewski
Stephen Boyd, Lieven Vandenberghe
Walter Rudin
Reinhard Diestel
George Pólya
David J.C. MacKay
Robert V. Hogg, Joseph W. McKean, Allen T. Craig
Notes Rigorous graduate-level probability + statistics; useful for inference and ML foundations.
Sheldon Axler
Notes Abstract, conceptual focus on vector spaces and linear maps. Avoids determinants until the end.
Roger A. Horn, Charles R. Johnson
Notes Comprehensive advanced linear algebra reference.
Steven H. Strogatz
Notes Intuition-rich introduction to qualitative behavior of dynamical systems.
Lloyd N. Trefethen, David Bau
Notes Practical matrix computations and numerical realities.
William H. Press, Saul A. Teukolsky, William T. Vetterling, Brian P. Flannery
Notes Broad coverage of numerical algorithms in science and engineering. Classic cross-topic numerical algorithms.
Walter Rudin
E.T. Jaynes
H.L. Royden, Patrick Fitzpatrick
E. Hairer, S.P. Nørsett, G. Wanner
Notes Definitive reference on modern ODE solvers and stiff systems.
George Casella, Roger L. Berger
Notes Standard rigorous text on estimation, hypothesis testing, and asymptotic theory.
James R. Munkres
Computer Science Theory
23 itemsRay Solomonoff
Notes Foundational paper on algorithmic probability and universal induction. Basis for AIXI.
Cover & Thomas
Notes The standard information theory textbook. Entropy, channels, coding.
Michael Sipser
Notes Automata, computability, complexity. The clearest treatment.
Daphne Koller, Nir Friedman
Notes The comprehensive reference on graphical models, inference, and learning.
Donald Knuth
Notes The Mount Everest of CS. Definitive but demanding.
Benjamin C. Pierce
Alexander Stepanov
Notes Collected notes on programming philosophy. Free PDF.
Robert Sedgewick, Kevin Wayne
Ming Li, Paul Vitányi
Notes The standard comprehensive reference on Kolmogorov complexity. Essential for understanding Solomonoff induction.
Ronald L. Graham, Donald E. Knuth, Oren Patashnik
Notes Blends discrete mathematics with analysis and algorithm design.
Alexander Stepanov, Paul McJones
Notes Rigorous foundations of generic programming. Connects algebra and algorithms. Stepanov’s magnum opus.
Oded Goldreich
Alexander Stepanov, Daniel Rose
Notes History of mathematical ideas underlying generic programming. More accessible than EoP.
Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, Clifford Stein
John E. Hopcroft, Rajeev Motwani, Jeffrey D. Ullman
Wade Trappe, Lawrence C. Washington
Michael Sipser
Notes Core computability and complexity theory.
Richard Bird
Robert Harper
Chris Okasaki
Jean-Philippe Aumasson
Steven S. Skiena
Benjamin C. Pierce
Programming
38 itemsJohn Ousterhout
Notes Deep modules, complexity management. Best modern design book.
JJ Geewax
Notes Practical patterns for designing internal and public APIs.
Gamma, Helm, Johnson, Vlissides
Notes The Gang of Four book. Know these patterns even to argue against them.
Richardson & Amundsen
Notes Hypermedia, representations, profiles. REST done properly.
Don Norman
Notes Affordances, signifiers, mental models. Applies to APIs and UIs alike.
Alexander Stepanov
Notes 18-part lecture series on efficient programming. Covers the intellectual foundations behind STL.
David Vandevoorde, Nicolai M. Josuttis, Douglas Gregor
Robert C. Martin
Steve McConnell
Alfred V. Aho, Monica S. Lam, Ravi Sethi, Jeffrey D. Ullman
Robert Nystrom
Erich Gamma, Richard Helm, Ralph Johnson, John Vlissides
Eric Evans
Scott Meyers
Daniel P. Friedman, Mitchell Wand
Christopher Allen, Julie Moronuki
Miran Lipovača
Harold Abelson, Gerald Jay Sussman
Notes Classic deep dive into computation, abstraction, interpreters, and languages.
Andrew W. Appel
Shriram Krishnamurthi
Jon Bentley
Bryan O'Sullivan, Don Stewart, John Goerzen
Martin Fowler
Rich Hickey
Notes Engineer-philosophical talk about the nature of system and language design.
Sean Parent
Notes Classic talk on recognizing algorithmic patterns. ‘No raw loops’ - shows how rotate solves many problems elegantly.
Harold Abelson, Gerald Jay Sussman
Kent Beck
Brian W. Kernighan, Dennis M. Ritchie
Matthias Felleisen, Daniel P. Friedman
Daniel P. Friedman, Matthias Felleisen
Daniel P. Friedman, David Thrane Christiansen
Frederick P. Brooks Jr.
Brian W. Kernighan, Rob Pike
David Thomas, Andrew Hunt
Daniel P. Friedman, William E. Byrd, Oleg Kiselyov
Daniel P. Friedman, Matthias Felleisen
Sandy Maguire
Michael C. Feathers