Maximum likelihood estimation sounds clean on paper: write down the likelihood, take derivatives, set them to zero, solve. In practice, the “solve” step is where things get interesting. Most likelihoods don’t have closed-form solutions, so you need numerical methods, and the choice of method matters more than most textbooks let on.
This write-up covers the numerical side of MLE: the optimization algorithms, convergence issues, and computational tricks that make the difference between getting an answer and getting the right answer. The full treatment is in the PDF.
Related work
For more on the statistical and mathematical context, see my research page and publications.
Discussion