June 25, 2024
Reverse-Process Synthetic Data Generation for Math Reasoning
Training LLMs on mathematical reasoning by inverting easy-to-solve problems: generate derivatives, reverse them into integration exercises with full step-by-step solutions.
Calculus
Browse posts by tag
January 15, 2025
Differentiation: Three Ways
Three approaches to computing derivatives, forward-mode AD, reverse-mode AD, and finite differences, each with different trade-offs for numerical computing and machine learning.
October 15, 2022
How to Teach a Computer Calculus with Pattern Matching
Define patterns, define replacements, repeat until done. Watch a 90-line rewrite engine learn to differentiate.
April 12, 2022
Numerical Differentiation
Choosing step size h for finite differences: small enough for a good approximation, not so small that floating-point errors eat your lunch.
September 20, 2021
Forward-Mode Automatic Differentiation
Dual numbers extend our number system with an infinitesimal epsilon where epsilon^2 = 0. Evaluating f(x + epsilon) yields f(x) + epsilon * f'(x)—the derivative emerges automatically from the algebra.