Expected Information Maximization: Using the I-Projection for Mixture Density Estimation

1 INTRODUCTION . Learning the density of highly multi-modal distributions is a challenging machine learning problem relevant to many fields such as modelling human behavior ( Pentland & Liu , 1999 ) . Most common methods rely on maximizing the likelihood of the data . It is well known that the maximum likelihood solution corresponds to computing the M ( oment ) -projection of the data distribution to the parametric model distribution ( Bishop , 2006 ) . Yet , the M-projection averages over multiple modes in case the model distribution is not rich enough to fully represent the data ( Bishop , 2006 ) . This averaging effect can result in poor models , that put most of the probability mass in areas that are not covered by the data . The counterpart of the M-projection is the I ( nformation ) -projection . The I-projection concentrates on the modes the model is able to represent and ignores the remaining ones . Hence , it does not suffer from the averaging effect ( Bishop , 2006 ) . In this paper , we explore the I-projection for mixture models which are typically trained by maximizing the likelihood via expectation maximization ( EM ) ( Dempster et al. , 1977 ) . Despite the richness of mixture models , the averaging problem remains as we typically do not know the correct number of modes and it is hard to identify all modes of the data correctly . By the use of the I-projection , our mixture models do not suffer from this averaging effect and can generate more realistic samples that are less distinguishable from the data . In this paper we concentrate on learning Gaussian mixture models and conditional Gaussian mixtures of experts ( Jacobs et al. , 1991 ) where the mean and covariance matrix are generated by deep neural networks . We propose Expected Information Maximization ( EIM ) 1 , a novel approach capable of computing the I-projection between the model and the data . By exploiting the structure of the I-projection , we can derive a variational upper bound objective , which was previously used in the context of variational inference ( Maaløe et al. , 2016 ; Ranganath et al. , 2016 ; Arenz et al. , 2018 ) . In order to work with this upper bound objective based on samples , we use a discriminator to approximate the required density ratio , relating our approach to GANs ( Goodfellow et al. , 2014 ; Nowozin et al. , 2016 ; Uehara et al. , 2016 ) . The discriminator also allows us to use additional discriminative features to improve model quality . In our experiments , we demonstrate that EIM is much more effective in computing the I-projection than recent GAN approaches . We apply EIM to a synthetic obstacle avoidance task , an inverse kinematic task of a redundant robot arm as well as a pedestrian and car prediction task using the Stanford Drone Dataset ( Robicquet et al. , 2016 ) and a traffic dataset from the Next Generation Simulation program . 2 PRELIMINARIES . Our approach heavily builds on minimizing Kullback-Leibler divergences as well as the estimation of density ratios . We will therefore briefly review both concepts . Density Ratio Estimation . Our approach relies on estimating density ratios r ( x ) = q ( x ) /p ( x ) based on samples of q ( x ) and p ( x ) . Sugiyama et al . ( 2012 ) introduced a framework to estimate such density ratios based on the minimization of Bregman divergences ( Bregman , 1967 ) . For our work we employ one approach from this framework , namely density ratio estimation by binary logistic regression . Assume a logistic regressor C ( x ) = σ ( φ ( x ) ) with logits φ ( x ) and sigmoid activation function σ . Further , we train C ( x ) on predicting the probability that a given sample x was sampled from q ( x ) . It can be shown that such a logistic regressor using a cross-entropy loss is optimal for C ( x ) = q ( x ) / ( q ( x ) + p ( x ) ) . Using this relation , we can compute the log density ratio estimator by log q ( x ) p ( x ) = log q ( x ) / ( q ( x ) + p ( x ) ) p ( x ) / ( q ( x ) + p ( x ) ) = log C ( x ) 1− C ( x ) = σ −1 ( C ( x ) ) = φ ( x ) . The logistic regressor is trained by minimizing the binary cross-entropy argminφ ( x ) BCE ( φ ( x ) , p ( x ) , q ( x ) ) = −Eq ( x ) [ log ( σ ( φ ( x ) ) ) ] − Ep ( x ) [ log ( 1− σ ( φ ( x ) ) ) ] , where different regularization techniques such as ` 2 regularization or dropout ( Srivastava et al. , 2014 ) can be used to avoid overfitting . 1Code available at https : //github.com/pbecker93/ExpectedInformationMaximization Moment and Information Projection . The Kullback-Leibler divergence ( Kullback & Leibler , 1951 ) is a standard similarity measure for distributions . It is defined as KL ( p ( x ) ||q ( x ) ) =∫ p ( x ) log p ( x ) /q ( x ) dx . Due to its asymmetry , the Kullback-Leibler Divergence provides two different optimization problems ( Bishop , 2006 ) to fit a model distribution q ( x ) to a target distribution p ( x ) , namely argminq ( x ) KL ( p ( x ) ||q ( x ) ) ︸ ︷︷ ︸ Moment-projection and argminq ( x ) KL ( q ( x ) ||p ( x ) ) ︸ ︷︷ ︸ Information-projection . Here , we will assume that p ( x ) is the data distribution , i.e. , p ( x ) is unknown but we have access to samples from p ( x ) . It can easily be seen that computing the M-projection to the data distribution is equivalent to maximizing the likelihood ( ML ) of the model ( Bishop , 2006 ) . ML solutions match the moments of the model with the moments of the target distribution , which results in averaging over modes that can not be represented by the model . In contrast , the I-projection forces the learned generator q ( x ) to have low probability whenever p ( x ) has low probability , which is also called zero forcing . 3 RELATED WORK . We will now discuss competing methods for computing the I-projection that are based on GANs . Those are , to the best of our knowledge , the only other approaches capable of computing the Iprojection solely based on samples of the target distribution . Furthermore , we will distinguish our approach from approaches based on variational inference that also use the I-projection . Variational Inference . The I-projection is a common objective in Variational Inference ( Opper & Saad , 2001 ; Bishop , 2006 ; Kingma & Welling , 2013 ) . Those methods aim to fit tractable approximations to intractable distributions of which the unnormalized density is available . EIM , on the other hand , does not assume access to the unnormalized density of the target distributions but only to samples . Hence , it is not a variational inference approach , but a density estimation approach . However , our approach uses an upper bound that has been previously applied to variational inference ( Maaløe et al. , 2016 ; Ranganath et al. , 2016 ; Arenz et al. , 2018 ) . EIM is especially related to the VIPS algorithm ( Arenz et al. , 2018 ) , which we extend from the variational inference case to the density estimation case . Additionally , we introduce conditional latent variable models into the approach . Generative Adversarial Networks . While the original GAN approach minimizes the JensenShannon Divergence ( Goodfellow et al. , 2014 ) , GANs have since been adapted to a variety of other distance measures between distributions , such as the Wasserstein distance ( Arjovsky et al. , 2017 ) , symmetric KL ( Chen et al. , 2018 ) and arbitrary f -divergences ( Ali & Silvey , 1966 ; Nowozin et al. , 2016 ; Uehara et al. , 2016 ; Poole et al. , 2016 ) . Since the I-projection is a special case of an f - divergence , those approaches are of particular relevance to our work . Nowozin et al . ( 2016 ) use a variational bound for f -divergences ( Nguyen et al. , 2010 ) to derive their approach , the f -GAN . Uehara et al . ( 2016 ) use a bound that directly follows from the density ratio estimation under Bregman divergences framework introduced by Sugiyama et al . ( 2012 ) to obtain their b-GAN . While the b-GAN ’ s discriminator directly estimates the density ratio , the f -GAN ’ s discriminator estimates an invertible mapping of the density ratio . Yet , in the case of the I-projection , both the f -GAN and the b-GAN yield the same objective , as we show in Appendix C.2 . For both the f -GAN and b-GAN the desired f -divergence determines the discriminator objective . Uehara et al . ( 2016 ) note that the discriminator objective , implied by the I-projection , is unstable . As both approaches are formulated in a general way to minimize any f -divergence , they do not exploit the special structure of the Iprojection . Exploiting this structure permits us to apply a tight upper bound of the I-projection for latent variable models , which results in a higher quality of the estimated models . Li et al . ( 2019 ) introduce an adversarial approach to compute the I-projection based on density ratios , estimated by logistic regression . Yet , their approach assumes access to the unnormalized target density , i.e. , they are working in a variational inference setting . The most important difference to GANs is that we do not base EIM on an adversarial formulation and no adversarial game has to be solved . This removes a major source of instability in the training process , which we discuss in more detail in Section 4.3 . 4 EXPECTED INFORMATION MAXIMIZATION . Expected Information Maximization ( EIM ) is a general algorithm for minimizing the I-projection for any latent variable model . We first derive EIM for general marginal latent variable models , i.e. , q ( x ) = ∫ q ( x|z ) q ( z ) dz and subsequently extend our derivations to conditional latent variable mod- els , i.e. , q ( x|y ) = ∫ q ( x|z , y ) q ( z|y ) dz . EIM uses an upper bound for the objective of the marginal distribution . Similar to Expectation-Maximization ( EM ) , our algorithm iterates between an M-step and an E-step . In the corresponding M-step , we minimize the upper bound and in the E-step we tighten it using a variational distribution . 4.1 EIM FOR LATENT VARIABLE MODELS . The I-projection can be simplified using a ( tight ) variational upper bound ( Arenz et al. , 2018 ) which can be obtained by introducing an auxiliary distribution q̃ ( z|x ) and using Bayes rule KL ( q ( x ) ||p ( x ) ) = Uq̃ , p ( q ) ︸ ︷︷ ︸ upper bound −Eq ( x ) [ KL ( q ( z|x ) ||q̃ ( z|x ) ) ︸ ︷︷ ︸ ≥0 ] , where Uq̃ , p ( q ) = ∫∫ q ( x|z ) q ( z ) ( log q ( x|z ) q ( z ) p ( x ) − log q̃ ( z|x ) ) dzdx . ( 1 ) The derivation of the bound is given in Appendix B . It is easy to see that Uq̃ , p ( q ) is an upper bound as the expected KL term is always non-negative . In the corresponding E-step , the model from the previous iteration , which we denote as qt ( x ) , is used to tighten the bound by setting q̃ ( z|x ) = qt ( x|z ) qt ( z ) /qt ( x ) . In the M-step , we update the model distribution by minimizing the upper bound Uq̃ , p ( q ) . Yet , opposed to Arenz et al . ( 2018 ) , we can not work directly with the upper bound since it still depends on log p ( x ) , which we can not evaluate . However , we can reformulate the upper bound by setting the given relation for q̃ ( z|x ) of the E-step into Eq . 1 , Uqt , p ( q ) = ∫ q ( z ) ( ∫ q ( x|z ) log qt ( x ) p ( x ) dx+ KL ( q ( x|z ) ||qt ( x|z ) ) ) dz + KL ( q ( z ) ||qt ( z ) ) . ( 2 ) The upper bound now contains a density ratio between the old model distribution and the data . This density ratio can be estimated using samples of qt and p , for example , by using logistic regression as shown in Section 2 . We can use the logits φ ( x ) of such a logistic regressor to estimate the log density ratio log ( qt ( x ) /p ( x ) ) in Equation 2 . This yields an upper bound Uqt , φ ( q ) that depends on φ ( x ) instead of p ( x ) . Optimizing this bound corresponds to the M-step of our approach . In the E-step , we set qt to the newly obtained q and retrain the density ratio estimator φ ( x ) . Both steps formally result in the following bilevel optimization problem qt+1 ∈ argminq ( x ) Uqt , φ∗ ( q ) s.t . φ∗ ( x ) ∈ argminφ ( x ) BCE ( φ ( x ) , p ( x ) , qt ( x ) ) . Using a discriminator also comes with the advantage that we can use additional discriminative features g ( x ) as input to our discriminator that are not directly available for the generator . For example , if x models trajectories of pedestrians , g ( x ) could indicate whether the trajectory reaches any positions that are not plausible such as rooftops or trees . These features simplify the discrimination task and can therefore improve our model accuracy which is not possible with M-projection based algorithms such as EM .

Summary: The paper presents an algorithm to match two distributions with latent variables, named expected information maximization (EIM). Specifically, EIM is based on the I-Projection, which basically is equivalent to minimizing the reverse KL divergence (i.e. min KL[p_model || p_data]); to handle latent variables, an upper-bound is derived, which is the corresponding reverse KL divergence in the joint space. To minimize that joint reverse KL, a specific procedure is developed, leading to the presented EIM. EIM variants for different applications are discussed. Fancy robot-related experiments are used to evaluate the presented algorithm.