June 25, 2024
Reverse-Process Synthetic Data Generation for Math Reasoning
Training LLMs on mathematical reasoning by inverting easy-to-solve problems: generate derivatives, reverse them into integration exercises with full step-by-step solutions.
Calculus
Browse posts by tag
January 15, 2025
Differentiation: Three Ways
Three approaches to computing derivatives, forward-mode AD, reverse-mode AD, and finite differences, each with different trade-offs for numerical computing and machine learning.
April 12, 2022
Numerical Differentiation
Choosing step size h for finite differences: small enough for a good approximation, not so small that floating-point errors eat your lunch.
September 20, 2021
Forward-Mode Automatic Differentiation
Dual numbers extend the reals with an infinitesimal epsilon where epsilon^2 = 0. Evaluate f(x + epsilon) and you get f(x) + f'(x)*epsilon. The derivative falls out of the algebra.