The EM algorithm, Deep Learning and the Softmax Layer
The EM Algorithm as described by Dempster et. al. provides excellent approaches to very difficult inference problems. Recent approaches in deep learning, such as variational autoencoders, give a gradient based approach to similar problems. While the above are unsupervised algorithms, it is the case that the softmax layer commonly used in supervised classification shares many features with the EM algorithm. We will discuss the connections between these algorithms.