March 13, 2026
femtograd
Automatic Differentiation for Statistical Computing
Automatic-Differentiation
Browse posts by tag
March 13, 2026
nabla
Exact Derivatives via Automatic Differentiation
January 19, 2026
Duality: The Hidden Structure of Opposites
Many structures come in pairs: forward/reverse AD, push/pull iteration, encode/decode. Recognizing duality lets you transfer theorems and insights between domains.
January 15, 2025
Differentiation: Three Ways
Three approaches to computing derivatives, forward-mode AD, reverse-mode AD, and finite differences, each with different trade-offs for numerical computing and machine learning.
January 17, 2023
Reverse-Mode Automatic Differentiation
Reverse-mode automatic differentiation is just the chain rule applied systematically. I built one in C++20 to understand what PyTorch and JAX are actually doing.
September 20, 2021
Forward-Mode Automatic Differentiation
Dual numbers extend our number system with an infinitesimal epsilon where epsilon^2 = 0. Evaluating f(x + epsilon) yields f(x) + epsilon * f'(x)—the derivative emerges automatically from the algebra.