September 12, 2023
Your Optimizer Is (Approximately) Propagating Fisher Information
Adam, K-FAC, EWC, and natural gradient are all approximating the same thing at different fidelity levels. The math and the caveats.
Natural-Gradient
Browse posts by tag