Expectation propagation

Expectation propagation (EP) is a technique in Bayesian machine learning. 

EP finds approximations to a probability distribution.  It uses an iterative approach that leverages the factorization structure of the target distribution.   It differs from other Bayesian approximation approaches such as variational Bayesian methods. 

More specifically, suppose we wish to approximate an intractable probability distribution  with a tractable distribution .  Expectation propagation achieves this approximation by minimizing the Kullback-Leibler divergence .  Variational Bayesian methods minimize  instead. 

If  is a Gaussian , then  is minimized with  and  being equal to the mean of  and the covariance of , respectively; this is called moment matching. 

Applications
Expectation propagation via moment matching plays a vital role in approximation for indicator functions that appear when deriving the message passing equations for TrueSkill. 

References

External links
 Minka's EP papers
 List of papers using EP. 

Category:Machine learning
Category:Bayesian statistics