StATS: EM Algorithm (March 15, 2004).

I received an email question about the EM Algorithm. This is a computational approach that works well for missing data problems and data models with latent (unobserved) variabels. The basic approach is to estimate the missing or latent data (E-step), compute maximum likelihood estimates that incorporates the missing/latent estimates (M-step), then update the missing or latent data (E-step) and so forth. There's a book by McLachlan and Krishnan, The EM Algorithm and Extensions, that I have not seen, but which sounds pretty good. There are also a few good web sites about this algorithm.

The Expectation Maximization Algorithm [pdf]. Dellaert F, Georgia Institute of Technology. Accessed on 2004-03-15.

The EM Algorithm and its Extensions. Bell Laboratories. Accessed on 2004-03-15.

A Gentle Tutorial of the EM Algorithm and its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models [pdf]. Bilmes JA, U.C. Berkeley. Accessed on 2004-03-15.

If I get some time, I will show a simple example on my web pages.

This page was written by Steve Simon while working at Children's Mercy Hospital. Although I do not hold the copyright for this material, I am reproducing it here as a service, as it is no longer available on the Children's Mercy Hospital website. Need more information? I have a page with general help resources. You can also browse for pages similar to this one at Category: Statistical computing.