Optimization

Compilers: Principles, Techniques, and Tools (Dragon Book)

compositional.mle: SICP-Inspired Optimization

An R package where optimization solvers are first-class functions that compose through chaining, racing, and restarts.

Convex Optimization

The Policy: S-Risk Scenarios, Worse Than Extinction

Most AI risk discussions focus on extinction. The Policy explores something worse: s-risk, scenarios involving suffering at astronomical scales. We survive, but wish we hadn't.

Fisher Flow: Optimization on the Statistical Manifold

Gradient descent in Euclidean space ignores the geometry of probability distributions. Natural gradient descent uses the Fisher information metric instead. Fisher Flow makes this continuous.

Your Optimizer Is (Approximately) Propagating Fisher Information

Adam, K-FAC, EWC, and natural gradient are all approximating the same thing at different fidelity levels. The math and the caveats.

Numerical Methods for Maximum Likelihood Estimation

Numerical approaches to maximum likelihood estimation, covering the optimization methods and computational issues that come up in practice.