Expected Information Maximization: Using the I-Projection for Mixture Density Estimation
Authors: Philipp Becker
Oleg Arenz
Gerhard Neumann
Venue:  (2020)
Keywords: density estimation
information projection
mixture models
generative learning
multimodal modeling

Abstract: Modelling highly multi-modal data is a challenging problem in machine learning. Most algorithms are based on maximizing the likelihood, which corresponds to the M(oment)-projection of the data distribution to the model distribution.
The M-projection forces the model to average over modes it cannot represent. In contrast, the I(nformation)-projection ignores such modes in the data and concentrates on the modes the model can represent. Such behavior is appealing whenever we deal with highly multi-modal data where modelling single modes correctly is more important than covering all the modes. Despite this advantage, the I-projection is rarely used in practice due to the lack of algorithms that can efficiently optimize it based on data. In this work, we present a new algorithm called Expected Information Maximization (EIM) for computing the I-projection solely based on samples for general latent variable models, where we focus on Gaussian mixtures models and Gaussian mixtures of experts. Our approach applies a variational upper bound to the I-projection objective which decomposes the original objective into single objectives for each mixture component as well as for the coefficients, allowing an efficient optimization. Similar to GANs, our approach employs discriminators but uses a more stable optimization procedure, using a tight upper bound. We show that our algorithm is much more effective in computing the I-projection than recent GAN approaches and we illustrate the effectiveness of our approach for modelling multi-modal behavior on two pedestrian and traffic prediction datasets.  

Published as a conference paper at ICLR 2020
EXPECTED INFORMATION MAXIMIZATION USING THE I-PROJECTION FOR MIXTURE DENSITY ESTIMATION
Philipp Becker
Oleg Arenz
Autonomous Learning Robots, KIT Bosch Center for Artificial Intelligence Correspondence to: philipp.becke becker @ kit.edu
Intelligent Autonomous Systems, TU Darmstadt
Gerhard Neumann
Autonomous Learning Robots, KIT Bosch Center for Artificial Intelligence University of Tübingen
ABSTRACT
Modelling highly multi-modal data is challenging problem in machine learning. Most algorithms are based on maximizing the likelihood, which corresponds to the M(oment)--rojection of the data distribution to the model distribution. The M-projection forces the model to average over modes it cannot represent. In con- trast, the naaformationn--rooccctiooo ignores such modes in the data and concentrates on the modes the model can represent. Such behavior is appealing whenever we deal with highly multi-modal data where modelling single modes correctly more important than covering all the modes. Despite this advantage, the I-projection is rarely used in practice due to the lack of algorithms that can efficiently optimize it based on data. In this work, we present new algorithm called Expected Informa- tion Maximization (EIM) for computing the 1-projection solely based on samples for general latent variable models, where focus Gaussian mixtures models and Gaussian mixtures of experts. Our approach applies a variational upper bound the I-projection objective which decomposes the original objective into single objectives for each mixture component as well as for the coefficients, allowing efficient optimization. Similar to GANs, our approach employs discriminators but uses a more stable optimization procedure, using a tight upper bound. We show that our algorithm is much more effective in computing the I-projection than recent GAN approaches and we illustrate the effectiveness of our approach for modelling multi-modal behavior on two pedestrian and traffic prediction datasets.
INTRODUCTION
Learning the density of highly multi-modal distributions is a challenging machine learning problem relevant to many fields such as modelling human behavior (Pentland Liu 1999). Most common methods rely on maximizing the likelihood of the data. It is well known that the maximum likelihood solution corresponds to computing the (itoment)-projeettee model distribution (Bishopl 2006). Yet, the M-projection averages over multiple modes in case the model distribution not rich enough to fully represent the data (Bishop) 2006). This averaging effect can result in poor models, that put most of the probability mass in that not covered by the data. The counterpart of the M-projection is the Infornaation)-projcciionn projection. The I-projection concentrates the modes the model is able represent and ignores the remaining ones. Hence, it does not suffer from the averaging effect (Bishop 2006).
of the data distribution to the parametric
In this paper, explore the proojection for mixture models which are typically trained by maximiz- ing the likelihood via expectation maximization (EM) (Dempster et al. 1977). Despite the richness of mixture models, the averaging problem remains as typically do not know the correct number

---

Published as a conference paper at ICLR 2020
Expert
peojection
(c) proojeccioo
Figure 1: Illustration of the I-projection vs. the M-projection for modelling behavior. (a): A robot reaches target point while avoiding obstacle. There are two different types of solutions, above and below the obstacle. A single Gaussian is fitted to the expert data in joint space. (b): The M- projection fails to reach the target and collides with the obstacle. (c): The I-projection ignores the second mode and reaches the target while avoiding the obstacle.
of modes and it is hard to identify all modes of the data correctly. By the use of the I-projection, our mixture models do not suffer from this averaging effect and can generate more realistic samples that are less distinguishable from the data. In this paper concentrate on learning Gaussian mix- ture models and conditional Gaussian mixtures experts (Jacobs et al. 199T) where the mean and covariance matrix generated by deep neural networks.
We propose Expected Information Maximization (EIM) I a novel approach capable of computing the I-projection between the model and the data. By exploiting the structure of the I-projection, we can derive a variational upper bound objective, which was previously used the context of variational inference (Maaløe al., 2016 Ranganath al., 2016) Arenz et al.) 2018 In order to work with this upper bound objective based on samples, we use a discriminator to approximate the required density ratio, relating our approach GANs (Goodfellow al. 2014 Nowozin al., 2016 [ehara et al. 2016)) The discriminator also allows us to use additional discriminative features improve model quality. our experiments, we demonstrate that EIM is much more effective in computing the I-projection than recent GAN approaches. We apply EIM to a synthetic obstacle avoidance task, an inverse kinematic task of redundant robot arm as well as a pedestrian and car prediction task using the Stanford Drone Dataset (Robicquet et al. 2016) and traffic dataset from the Next Generation Simulation program.
2
PRELIMINARIES
Our approach heavily builds minimizing Kullback-Leibler divergences as well as the estimation of density ratios. We will therefore briefly review both concepts.
Density Ratio Estimation. Our approach relies on estimating density ratios r(z) (f(r) based on samples of q(x) and p(x). Sugiyama al. (2012) introduced framework to estimate such density ratios based on the minimization of Bregman divergences (Bregman) 1967). For our work employ one approach from this framework, namely density ratio estimation by binary logistic regression. Assume a logistic regressor C(x) a (ox)) with logits ((x) and sigmoid activation function a. Further, we train C(x) on predicting the probability that a given sample æ was sampled from (xx) It can be shown that such a logistic regressor using cross-entropy loss is optimal for C(x) = =(xx)/ + p(x)). Using this relation, we can compute the log density ratio estimator by
q(x) log p(z) p(x)/(g(x) p(x))
log q<))/((<)) + p(z)) log I C(x) = - - =(x).
C(x)
The logistic regressor is trained by minimizing the binary cross-entropy
argmin
(z) BCE(d())) F [log (a()))))) Ep(x) [log (1 (
where different regularization techniques such & regularization dropout SSrivastaaa et al. 2014] can used to avoid overfitting.
Code available httba//www..aaaeees.//..o/.//oes infommtt onMaxin Pation
2

---

Published as a conference paper at ICLR 2020
Moment and Information Projection. The Kullback-Leibler divergence (Kullback & Leibler 1951 is a standard similarity measure for distributions. It is defined as (p(z) (z)) 1 p(z) log p(zz//(<)dd Due to its asymmetry, the Kullback-Leibler Divergence provides two dif- ferent optimization problems (Bishop, 2006) to fit a model distribution ((x)) to a target distribution
p(x),, namely
argmin ((x) KL (p(2))la(a))
and argmin ((2))
KL (g(x))lo(x))
Monenn-projection
Information propiitcoooooooionnn
Here, we will assume that p(x)) is the data distribution, í.e., p(x) is unknown but we have access to samples from p(z). It can easily be seen that computing the M-projection to the data distribution is equivalent to maximizing the likelihood (ML) of the model (Bishop) 2006). ML solutions match the moments of the model with the moments of the target distribution, which results in averaging over modes that can not be represented by the model. In contrast, the I-projection forces the learned generator ((x) to have low probability whenever p(r) has low probability, which also called zero forcing.
3 RELATED WORK
We will now discuss competing methods for computing the I-projection that are based on GANs. Those are, to the best of our knowledge, the only other approaches capable of computing the I- projection solely based on samples of the target distribution. Furthermore, we will distinguish our approach from approaches based on variational inference that also use the I-projection.
Variational Inference. The I-projection is a common objective in Variational Inference (Opper & Saad 2001 Bishop 2006 Kingma Welling 2013). Those methods aim to fit tractable approx- imations to intractable distributions of which the unnormalized density is available. EIM, on the other hand, does not assume access to the unnormalized density of the target distributions but only to samples. Hence, it is not variational inference approach, but a density estimation approach. However, our approach uses an upper bound that has been previously applied to variational infer- ence (Maaløe al., 2016 |Ranganthh et al.) 2016 Arenz et al.. 2018). EIM is especially related to the VIPS algorithm (Arenz et al.) 2018), which we extend from the variational inference case to the density estimation case. Additionally, introduce conditional latent variable models into the approach.
Generative Adversarial Networks. While the original GAN approach minimizes the Jensen- Shannon Divergence (Goodfellow et al., 2014), GANs have since been adapted to a variety of other distance measures between distributions, such as the Wasserstein distance (Arjovsky et al., 2017). symmetric KL (Chen et al. 2018) and arbitrary f-divergences (Ali & Silvey 1966] Nowozin al.) 2016 Uehara et al. 2016 Poole et al. 2016). Since the I-projection
special case of an f- divergence, those approaches are of particular relevance to work. Nowozin et al. (2016) use a variational bound for f-divergences (Nguyen et al., 2010) to derive their approach, the f-GAN. Uehara et al. 2016) use a bound that directly follows from the density ratio estimation under Breg- man divergences framework introduced by Sugiyama et al. (2012) to obtain their b-GAN. While the 6-GAN's discriminator directly estimates the density ratio, the f-GAN's discriminator estimates an invertible mapping of the density ratio. Yet, in the case of the I-projection, both the f-GAN and the b-GAN yield the same objective, as we show in Appendix C.2 For both the f-GAN and b-GAN the desired f-divergence determines the discriminator objective. Uehara et al. (016)) note that the discriminator objective, implied by the I-projection, is unstable. As both approaches are formulated in general way to minimize any f-divergence, they do not exploit the special structure of the I- projection. Exploiting this structure permits us apply tight upper bound of the I-projection for latent variable models, which results in higher quality of the estimated models.
Li et al. (2019) introduce an adversarial approach to compute the I-projection based on density ratios, estimated by logistic regression. Yet, their approach assumes access to the unnormalized target density, i.e., they are working in a variational inference setting. The most important difference adversarial formulation and adversarial game has to
to GANs is that do not base EIM be solved. This removes a major source of instability in the training process, which we discuss in more detail in Section 43
3

---

Published as a conference paper at ICLR 2020
4 EXPECTED INFORMATION MAXIMIZATION
Expected Information Maximization (EIM) is a general algorithm for minimizing the I-projection for any latent variable model. We first derive EIM for general marginal latent variable models, i.e., ((x) = ((())=){(())))) and subsequently extend our derivations to conditional latent variable mod- els, i.e. ((<l))
q(z]2, y) EIM uses an upper bound for the objective of the marginal distribution. Similar to Expectation- Maniiiinanaiiiiiaiiionn (EM), our algorithm iterates between an M-step and an E-step. the corresponding M-step, we minimize the upper bound and in the E-step we tighten it using variational distribution.
4.1 EIM FOR LATENT VARIABLE MODELS
The I-projection can be simplified using a (tight) variational upper bound (Arenz et al. 2018 which can be obtained by introducing auxiliary distribution @(=|ar) and using Bayes rule
KL (g(x))[(<z)) Ugpla
Bg(x)) [a(=)]KK [KL (q(2 (1())=)))((==
bound
20
where
Ua - /f ((x)))a(())
(log g(<l=)g(2)
log ((2|z) dzdx.
(I)
p(x)
The derivation of the bound is given in Appendix B It is easy to see that (g) is upper bound
as the expected KL term always non-negative. In the corresponding E-step, the model from the previous iteration, which In the M-step, we update the model distribution by minimizing the upper bound (q). Yet, opposed to Arenz al. (2018), we cannot work directly with the upper bound since it still depends on log p(r), which we cannot evaluate. However, we can afoteelate the upper bound by setting the given relation for q(=|<)) of the E-step into Eq. !
denote as (z),, is used to tighten the bound by setting {(2[z)
Un. (g)r(1)= (0 ((=) (/((rlz)lon
(1(x) do KL (x(a)))l(((x))) dz + KL (q(=)]lu:(z)) (2) p(x)
The upper bound now contains density ratio between the old model distribution and the data. This density ratio can be estimated using samples of q and p. for example, by using logistic regression
as shown in Section 2 We can use the logits Φ(x) of such a logistic regressor to estimate the log
density ratio log(u (T) (p))) in Equation 2 This yields an upper bound U4))(4) that depends on
Φ(x) instead of p(r). Optimizing this bound corresponds to the M-step of our approach. In the E-step, we set (t to the newly obtained q and retrain the density ratio estimator ((x). Both steps formally result in the following bilevel optimization problem
@++1 E ar) (q) s.t. 6'(x) E argmin ( BCE(((Z). q.))).
Using discriminator also comes with the advantage that we can additional discriminative fea- tures g(x) as input to our discriminator that are not directly available for the generator. For example, if a models trajectories of pedestrians, g(x) could indicate whether the trajectory reaches any po- sitions that are not plausible such as rooftops or trees. These features simplify the discrimination task and can therefore improve our model accuracy which is not possible with M-projection based algorithms such as EMM
4.2 EIM FOR CONDITIONAL LATENT VARIABLE MODELS
For conditional distributions, aim at finding the conditional I-projection
argmin Ep(x) KL ([<<<l))))(((()))))
The derivations for the conditional upper bound follow the same steps the derivations in the marginal case, where all distributions are extended by the context variable y. We refer to the supple- ment for details. The log density ratio estimator 6(2, y) now discriminates between samples of the joint distribution x and y. For training Φ(æ, y) we generate a new sample x for each context y, using the distribution (xly)) Hence, the context distribution is the same for the true data and the generated data, the log density ratio of the conditional distributions equal to the log density ratio of the joint distributions.
4

---

Published as a conference paper at ICLR 2020
Log Density
Formulation
(c)
kgp(o) kepde)
<xp
0.0
112
0.2
-3
-2 01 2
Figure 2: An illustrative example of the benefits of EIM versus an adversarial formulation. (a): The true log density ratio of the model and the target distribution (both are Gaussians). The location of the optimum is unbounded. (b): In the adversarial formulation, the generator minimizes the expected log density ratio. If we neglect that the adversarial discriminator changes with every update step of the generator, the generator updates yield an unbounded solution. Hence, too aggressive updates of the generator yield unstable behavior. (c): The upper bound of EIM introduces an additional KL- term as objective. Optimizing this objective directly yields the optimal solution without the need to recompute the density ratio estimate.
4.3 RELATION TO GANS AND EM
There is a close relation of EIM to GANs due to the use of a logistic discriminator for the density ratio estimation. It is therefore informative investigate the differences in the case without latent variables. In an adversarial formulation, the density ratio estimator would directly replace the density ratio in the original I-projection equation, i.e.,
argming(a)) ((x)) / q()))) (x)dæ s.t.
/0 (x) E arggmin ((x) p())) (x))-
However, such adversarial games often hard optimize. In contrast, EIM offers bilevel opti- mization problem where the discriminator explicitly learned on the old data distribution qi (x).
argmin (()) / q(z))" (x)dz + KL (1(x)]lq(()) s.t. (x) E argmin BCE(((x), BC
p(x), (x)).
Thus, there is no cireular dependency between the optimal generator and the optimal discriminator. Figure 2 illustrates that the proposed non-adversarial formulation does not suffer from too large
model updates. Choosing the number and step-size of the updates is thus far less critical.
EIM can also be seen the counter-part of the M-projection with latent variable models, EIM uses the I-projection. However, both approaches decompose the corresponding projections into upper bound (or lower bound for EM) and KL- term that depends on the conditional distribution ((z(z) to tighten this bound. The exact relationship is discussed in Appendix C.I
(EM). While EM optimizes
4.4 EIM FOR GAUSSIAN MIXTURES MODELS
We consider Gaussian mixture models with d components, i.e., multivariate Gaussian distributions (xx[2;) = Mp, Σj) and categorical distribution q(2) Cat(rr) for the coefficients. As the latent distribution ((2) is discrete, the upper bound in EIM (Equation 2) simplifies, as the integral over can be written for the coefficients and the components. For both updates, we will use similar update rules as defined in the VIPS algorithm (Arenz et al., 2018). VIPS uses a trust region optimization for the components and the coefficients, where both updates can be solved in closed form as the components are Gaussian. The trust regions prevent the new model from going too far away from qe where the density ratio estimator is inaccurate, and hence, further stabilize the learning process. We will now sketch both updates, where we refer to Appendix B.2 the full details.
sum. Similar to the EM-algorithm, this objective can be updated individually
For updating the coefficients, assume that the components have not yet been updated, and there- fore KL (q(x) (x)(2))llg (x) = for all z₁. The objective for the coefficients thus simplifies to
argmin (2)
(()))(=)) + KL (o(=)]la(<)) with Φ(zz) Egix| [0(x)],
(3)
5

---

Published as a conference paper at ICLR 2020
where (2(2) can be approximated using samples from the corresponding component, This objective can easily be optimized in closed form, as shown in the VIPS algorithm (Arenz et al. 2018). We also KL trust-region specify the step size of the update. For updating the individual components, the objective simplifies to
argmin (x 2.) (x(()))) [x(x) + KL (g(x
As in VIPS, this optimization problem can be solved in closed form using the MORE algorithm (Abdolmaleki et al. 2015). The MORE algorithm uses quadratic surrogate function that locally approximates (xx). The resulting solution optimizes Equation under KL trust-region. The pseudo-code of EIM for GMMs can be found in Appendix A
4.5 EIM FOR GAUSSIAN MIXTURES OF EXPERTS
In the conditional case, we consider mixtures of experts consisting of d multivariate Gaus- sians, whose parameters depend on input in nonlinear fashion, i.e., g(x[2;,)) = N ( and the gating is given by a neural network with softmax output. We again decompose the resulting upper bound into individual update steps for the components and the gating. Yet, closed-form solutions are no longer available and we need to resort to gradient-bar updates. The objective for updating the gating is given by
Trgming(=ly (pp())(())a.))) (0(=l1))(x y)l) + Epty) [KL (((x)))]la<<([)))]
(5)
We minimize this equation w.r.t. the parameters of the gating by gradient descent using the Adam (Kingma& & Ba 2014) algorithm. The objective for updating single component is given by
argming(xlz..g) ((xla. Bp(yles) [Eg())e))) 16(x, y)] + KL (g(x| (l: (x)x()
(6)
where =(y
(ly)g(2l))/(<z)
Note that we normalized the objective by g(z))
p(y (44 dy to ensure that also components with a low prior g(2,) get large enough gradi- ents for the updates. As we have access to the derivatives of the density ratio estimator w.r.t. x, we can optimize 6 gradient descent using the reparametriiztioon trick (Kingma Welling 2013) and Adam.
5 EVALUATION
We compare our approach to GANs and perform an ablation study on toy task, with data sampled from known mixture models. We further apply our approach two synthetic datasets, learning the joint configurations of planar robot as well as non-linear obstacle avoidance task, and two real datasets, namely the Stanford Drone Dataset (Robiequet et al., (2166) and traffic dataset from the Next Generation Simulation program. A full overview of all hyperparameters and network architectures can found in Appendix D
5.1 COMPARISON TO GENERATIVE ADVERSARIAL APPROACHES AND ABLATION STUDY
We compare to the f-GAN which is the only other method capable of minimizing the I-projection solely based on samples. We use data sampled from randomly generated GMMs with different numbers of components and dimensionalities. To study the influence of the previously mentioned differences of EIM to generative adversarial approaches, we also perform an ablation study. We compare to a version of EIM where we neglect the additional KL-term (EIM, no KL), a version were we trained all components and the coefficients jointly using gradient descent (Joint EIM), and a version where we do both (Joint EIM, no KI.). The average I-projection achieved by the various approaches can be found in Figure 3
5.2 LINE REACHING WITH PLANAR ROBOT
We extended the introductory example the planar reaching task and collected expert data from 10-link planar robot tasked with reaching point line. We fitted GMMs with increasing

---

Published as a conference paper at ICLR 2020
Single Gaussian
(B) Components
(c) Components
101
10ogectio
10-11
E.Pojecttoo
10P
10'3
10-1
EIM, KL.
10-1
Joist EIM
10
Joint HM.. no KL. / /ANS GAN
10-4
30
20
10
16
20
Dinerioonaityy
Dincroloxality
Dinensionalie
Figure 3: Average I-projection achieved for EIM, the f-GAN, and the modified EIM versions. The task is to fit model to samples from randomly generated GMM of different dimensions. Both the model and the target GMM have the same number of components. EIM clearly outperforms the generative adversarial approaches, especially for larger dimensions. The ablation study shows that the separated, closed-form updates clearly yield better results. Neglecting the KL has big influence for lower dimensions, but is out- weighted by the error of the discriminator at higher dimensions.
(2) Average distance
(b) Samples for varying numbers components. Upper EIM. Lower
15
30
90
100
10/
Log,
20 Number Comporents
Figure 4: Average distance to line and samples for robot line reaching. While EIM for small numbers of components ignores modes, not considering the whole line, it learns models that achieve the underlying task, i.e., reach the line. Providing additional information to the density ratio estimator further decreases the average distance the line. EM, on the other hand, averages over the modes, and thus, fails to reach the line even for large numbers of components.
number of components using HIM, EIM with additional features, where the end-effector coordinates for given joint configuration were provided, and EMM Even for large number of components, we see effects similar to the introductory example, i.e., the M-projection solution provided by EM fails reach the line while EIM manages do so. For small numbers, EIM ignores parts of the line, while more and more parts of it get covered as we increase the number of components. With the additional features, the imitation of the line reaching was even more accurate. See Begure 4 for the average distance between the end-effector and the line as well as samples from both EM and EIM.
5.3 PEDESTRIAN AND TRAFFIC PREDICTION
We evaluated approach data from the Stanford Drone Dataset (SDD) (Robicquet et al. 2016) and a traffic dataset from the Next Generation Simulation (NGS) program The SDD data consists of trajectories of pedestrians, bikes, and cars and only used the data corresponding to a single video of a single scene (Video 1, deathCircle). The NGS data consists of trajectories of cars where considered the data recorded on Lankershim Boulevard. In both cases we extracted trajectories of length 5, yielding highly multimodal data due to pedestrians, bikes, and cars moving at different speeds and in different directions. We evaluated on the achieved log-likelihood of EIM and EM, see Figure 5] EM achieves the highest likelihood as it directly optimizes this measure. However, we can already see that EM massively overfits when we increase the number of components as the test-set likelihood degrades. EIM, on the other hand, produced better models with an increasing number of components. Additionally, we generated a mask indicating whether a given point is on the road or
'utps: MMlls transportattinn www.://awwt.ooogtttttteoooommmm www.eeewwwwweeeeeeeeeeeeeeeeeee Sinn SimmlationnNNi Niiiiaatiioo-nS-M----- Vehicle- AcCSsNoeeaiiddoTTigeeeeeeaaaaa

---

Published as a conference paper at ICLR 2020
(a) Log Likelihood SDD
(b) Evaltaation with Mask SDD
(c) Log Likelihood
(d) Evalusaion with Mask NGS
1.4
0.00
1234
1.22
0-
100
30 100 120 200
220
300
300 400 400 200
Figure 5: Results on the traffic prediction tasks. Naturally, EM achieves the highest training log- likelihood. Yet, for large numbers of components, severe amount of overfitting is observed. EIM, on the other hand, has no problems working with high numbers of components and achieves a higher test log-liken rogllike,,oodd, despite optimizing different objective. We also provided 'road mask' as additional features for the discriminator. We used this road mask to evaluate how realistic the generated samples are. While EIM without features produced more realistic samples on the Lanker- shim dataset, we needed the feature input to outperform EM on this evaluation on the SDD dataset.
(a) Success Probability
(b) Samples EIM
(c) Samples
Bald
EIN
c-w Thars
Figure 6: Results on the obstacle avoidance task. (a): Even for a small number of components, EIM has a rather high probability of success, i.e., placing trajectory that does not hit any obstacle. Even with a sufficient number of components, i.e., eight, EM fails to achieve good results. (b) and (c): Samples of a mixture with 4 components, learned by EIM and EM respectively. EM clearly averages over multiple modes in the data distribution.
not and evaluated how realistic the learned models are by measuring the amount of samples violating the mask, i.e., predicting road users outside of the road. We also evaluate version EIM where we provide additional features indicating if the mask is violated. EIM achieves much better value on this mask for the NGS dataset, while we needed the additional mask features for the discriminator on the SDD dataset outperform EMM Both experiments show that EIM can learn highly density estimates that produce more realistic samples than EMM They further show that the models learned by EIM can be refined by additional prior knowledge provided feature vectors.
5.4 OBSTACLE AVOIDANCE
We evaluate the conditional version of EIM on an artificial obstacle avoidance task. The context contains the location of three obstacles within image. The gating, as well the components, are given by deep neural networks. Details about the network architectures can be found in the Appendix. The data consists of trajectories going from the left to the right of the image. The trajec- tories are defined by setting 3 via-points such that no obstacle is hit. To generate the data we sample via- via-points over and under the obstacles with probability proportional to the distance between the obstacle and the image border. Hence, for three obstacles, there are 23 = 8 different modes in the data. Note that, like most real-world scenarios, the expert data is not perfect, and about 13% the trajectories in the dataset hit an obstacle. We fit models with various numbers of components to this data using EIM and EM and compare their performance on generating trajectories that achieve the goal. Results are shown in Figure 6 together with visualization of the task and samples produced by EIM and EM. EIM was able to identify most modes for the different given inputs and did not suffer from any averaging effect. In contrast, EM does not find all modes. As consequence, some
8

---

Published as a conference paper at ICLR 2020
components of the mixture model had to average over multiple modes, resulting in poor quality trajectories.
CONCLUSION
We introduced Expected Information Maximization (EIM), a novel approach for computing the I- projection between general latent variable models and target distribution, solely based samples of the latter. General upper bound objectives for marginal and conditional distributions were derived, resulting an algorithm similar EM, but tailored for the I-projection instead of the M-projection. We introduced efficient methods to optimize these upper bound objectives for mixture models. In our experiments, we demonstrated the benefits of the I-projection for different behavior modelling tasks. The introduced approach opens various pathways for future research. While we focused on mixture models, the derived upper bounds are exclusive to those and can be used for arbitrary latent variable models. Another possibility is an online adaptation of the number of used components. Arenz al. (2018) propose heuristics for such an adaptation their VIPS approach. Those could easily be adapted to our approach.
REFERENCES
Abbas Abdolmaleki, Rudolf Lioutikov, Jan Peters, Nuno Lau, Luis Pualo Reis, and Gerhard Neumann. Model-based relative entropy stochastic search. In Advances in Neural Information Processing Systems, pp. 3537-3545, 2015.
Syed Mumtaz Ali and Samuel D Silvey. A general class of coefficients of divergence of one dis- tribution from another. Journal of the Royal Statistical Society. Series B (Methodological) pp- 131-142, 1966.
O. Arenz, M. Zhong, and G. Neumann. Efficient gradient-free variational inference using policy search. In Proceedings of the 35th International Conference Machine Learning, pp. 234-243. pmlr, 2018.
Martin Arjovsky, Soumith Chintala, and Léon Bottou. arXiv:1701.0787 00777777 2017.
Wasserstein gan. arXiv preprint
Christopher M. Bishop. Pattern Recognition and Machine Learning (Information Science and Statis- tics). Springer- Verlla,,-Verlag, Berlin, Heidelberg, 2006. ISBN 0387310738.
Lev M Bregman. The relaxation method of finding the common point of convex sets and its applica- tion to the solution of problems in convex programming. USSR computational mathematics and mathematical physics, 7(3):200-217 1967.
Liqun Chen, Shuyang Dai, Yunchen Pu, Erjin Zhou, Chunyuan Li, Qinliang Su, Changyou Chen, and Lawrence Carin. Symmetric variational autoencoder and connections to adversarial learning. In International Conference on Artificial Intelligence and Statistics, pp. 661-669, 2018.
Arthur P Dempster, Nan M Laird, and Donald B Rubin. Maximum likelihood from incomplete data via the em algorithm. Journal of the royal statistical society. Series B (methodological), pp. 1-38, 1977.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural infor- mation processing systems, pp. 2672-26800 2014.
Jean-Baptist Hiriart-Urruty and Claude Lemaréchal. Fundamentals of convex analysis. Springer Science & Business Media, 2012.
Robert A Jacobs, Michael Jordan, Steven J Nowlan, and Geoffrey E. Hinton. Adaptive mixtures of local experts. Neural computation, 3(1):79-87, 1991.
Sham M Kakade. A natural policy gradient. Advances in neural information processing systems, pp. 1531- 2002.

---

Published as a conference paper at ICLR 2020
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. International Conference on Learning Representations, 12 2014.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv: 14, 2013.
arXiv preprint
Solomon Kullback and Richard Leibler. On information and sufficiency. The annals of mathe- matical statistics, 22(1):79-88) 86, 1951.
Chunyuan Li, Ke Bai, Jianqiao Li, Guoyin Wang, Changyou Chen, and Lawrence Carin. Adver- sarial learning of a sampler based on an unnormalized distribution. The 22nd International Conference Artificial Intelligence and Statistics, pp. 3302-3311, 2019.
Lars Maalde, Casper Kaae Sønderby, Søren Kaae Sønderby, and Ole Winther. Auxiliary deep gen- erative models. In International Conference on Machine Learning, 1445-1453, 2016.
XuanLong Nguyen, Martin J Wainwright, and Michael Jordan. Estimating divergence functionals and the likelihood ratio by convex risk minimization. IEEE Transactions on Information Theory, 36(11):5847-5861 2010.
Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-gan: plers using variational divergence minimization. In Advances in Neural Information Processing Systems, pp. 271-2791 2016.
Training generative neural sam-
Manfred Opper and David Saad. Advanced mean field methods: Theory and practice. MIT press, 2001.
Alex Pentland and Andrew Liu. Modeling and prediction of human behavior. Neural Comput., 11(1)2299-24,, January 1999. ISSN 0899-7667. doi: 10. 10.16602997699990006666999 URL
http: doi.
Ben Poole, Alexander A Alemi, Jascha Sohl-Dickstein, and Anelia Angelova. Improved generator objectives for gans. arXiv preprint arXiv: 2016.
Rajesh Ranganath, Dustin Tran, and David Blei. Hierarchical variational models. In International Conference on Machine Learning. pp. 324-333, 2016.
Alexandre Robicquet, Amir Sadeghian, Alexandre Alahi, and Silvio Savarese. Learning social eti- quette: Human trajectory understanding in crowded scenes. volume 9912, pp. 549-565, 10 2016. doi: 10. 10.1007777--333119444843.3
ISBN 978-3-319-46483-11
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1):12291198, 2014.
Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori. Density-ratio matching under the breg- man divergence: a unified framework of density-ratio estimation. Annals of the Institute of Sta- ristical Mathematics, 64(5):1009-1000 2012.
Masatoshi Uehara, Issei Sato, Masahiro Suzuki, Kotaro Nakayama, and Yutaka Matsuo. Generative adversarial nets from a density ratio estimation perspective. arXiv preprint arXiv:1610.0290 2016.
10

---

Published as a conference paper at ICLR 2020
A PSEUDO CODE
HIM-for-GMMs () (x=
ii=I..N, x(x)))
Input: Data {xg bj-1..N. Initial Model ((x) = -
(x(2)))(()) = Ei= T,N(x E.)
for i number of iterations do
E-Step:
qe(z) (2), (x(2.) ((x[2x) for all components
Update Density Ratio Estimator:
sample data from model {xg²) Jy=1-.N qx(x)
retrain density ratio estimator (xx on (xp (x)
Jj-i...N and (xg Jj-1...N
M-Step Coefficients:
for in number of components do
compute loss 4 N Σ}= ④ ) with samples {xg Jj=1...N (x(2)2))
end
update ((2) using losses I and MORE equations
M-Step Components:
for in number of components do
fit o(x) surrogate to pairs X Φ Xg )
with samples {xg (") J==.... (x(x)))
update (xx(z;) using surrogate o(x) and MORE equations
end end
Algorithm 1: Expected Information Maximization for Gaussian Mixture Models.
Pseudo-code for EIM for GMMs can be found in algorithm
B DERIVATIONS
Derivations of the upper bound stated in Equation 1 [(())2)(2)dd= and use the identities g(>lz)q(2) (())=)q(x) and log q(x) log (()22)((2)
We assume latent variable models q(x) =
log ((=(xr)
KL (c(x))l((z)) ((x) log g(z) dr /I ((al=)a(2) log g(z) dzdæ
p(x)
p(x)
-fʃ (<))))((=)
log ((<(2)/(2) log (2/o())z)) dzdæ
p(ar)
/// (<)))){(=)
log (<<12)q(2) log (ogg(>/=) + log ((2|>) log log ((z|z) dzdx
p(x)
// (<)))){==) log g(<12)g(2)) logā(z/z) dzdx /ʃ ((<)=))(=) (log (=][x) log ((= dzdr
p(ar)
// (<))=){(=)
log (<<12)((2) logā(z/z) dzdx /(0) / (((2))) log (<(xx) dzdr
p(x)
((2|z)
U(g p. p) Ex(
KL (g(<l=))l((<))))

---

Published as a conference paper at ICLR 2020
After plugging the E-Step, i.e., ((2[=) ()) into the objective it simplifies to
= =ʃʃ (())))(()) log g(<]2)a(2) log (1(222))((2)
=
dzdr
p(x)
9c(xr)
// ((c)=))(=))))))))))))){))))))() (log ((rl))
+ logg(z) -logllp)))) - log (o(()))) - log + log q(x)) dzdr
/f ((<))))(=)) log (x(x) log g(alz)
log g(2)
dzdæ
p(x)
(x(2)))
(t(z)
f ((=) (/ (<x|2) log (1(2))
p(x) + log g(0|2) dr log g(=)
dz
(x(x)2)
qe(z)
f (22) / ((z022 log
2t (z) drdz + / a(2)) q(<]22 log
(<x|2) dxdz f ((())) log ((z)
q((2) de
p(z)
90 (x(222)
/f (<<l|))(() log
(x) dadæ + By(s) [KL (g(x]=)]l] + KL (u(=)lu((())) ((x)
which concludes the derivation of upper bound of latent variable models.
B.1 DERIVATIONS CONDITIONAL UPPER BOUND
By introducing an auxiliary distribution @(2[x, y) the upper bound to the expected KL. for conditional latent variable models g(>ly)) If g(x|2, y) ())(())))))))))))) (2) can be derived by
Pp()) [KL (1(c)1))) // p(u)g(aly) log g(aly) drdy
p(zly)
p(v) // q((]), (19()))))))))) (log g(<aal))
logg()/z,)) dzdrdy
p(rl))
f ply) // q(xl)2 v)9(2l))
(Log g( (2/2 v) - log q(2|z, y) log q(z|Ø, 3) dzdxdy p(zlv)
p(u) //
log (<122, ()(((l)) log (<>|01,)))) dzdxdy p(xlg)
J p(v) /f ((0l),.09(2 8) (log q(> Ir y) log q(2|2, v) dadxdy
f ply) /I ((al2, 1og q(>12, v)((=lv) logg(z/2,)) dzdrdy
p(rly)
- Iʃ p(u)a(atw) / a(=|2, v) log (211, y) dzdxdy
(2] y)
-U.g...p) Ep(n).(aal)) |KL (g(2)0, v) ( v))l
12

---

Published as a conference paper at ICLR 2020
During the E-step the bound is tightened by setting ((z|,x,)) v) (o))
Ulg. q.P)
= =s plv) /ʃ ((1)2,8)9(=1z))
log ((0/2, 2(0)2, v)(()))())))
(6(2/22 v).. dzdzdy
log
p(ayy)
(x(xl))
= ply) /f [((()2 ()9(()))
(log (xx) 22 + log (<(v) logp(z - log (x)2,)) log (z(v) + log qu (alu)) dedrdy
f p(v) /f ((0=),)))(=1v)) log 4r (x(1)) log g(a|2,)) log g(=ly)
dzdrdy
p(xly)
ge (<]
g((2ly)
/ p(u) / ((zlv) (1 ((c2)..))
log (x(1))
q dr log (21ly) dzdy qi (<(1))
log
p(xly)
qe (x((2,y))
plu) / ((>lw)) / g(2022, v) y) log
4e (xll)))) drdzdy
p(zly)
+ p(u) / ((2lw)) ((<2222)) v) log
( v) drdzdy + / plu)| ((xlv) log ((>ly) (<(p)))
dady
(e 2,
(x(l))) drdzdy
plu)a(alll( log p(zly)
+ ((tz))ot=l) KL (g(x)2, (x|2, v))l + Ep()) [KL (g<
which concludes the derivation of the upper bound for conditional latent variable models.
B.2 USING MORE FOR CLOSED FORM UPDATES FOR GMMs
The MORE algorithm, introduced by Abdolmaleki et al. (2015). can be used to solve optimization problems the following form
argmax, ((r)) (ett) s.t. KL (g(x)]lqo)<<))) S
for an exponential family distribution (zx), some function f(z), and an upper bound on the allowed change, e. Abbdlmaleki et al. (2013) show that the optimal solution is given by
f(x)
17 log (2)a()) + f(z)
((x) yord (x) exp
exp
7)
where 17 denotes the Lagrangian multiplier corresponding to the KL constraint. In order to obtain this Lagrangian multiplier, the following, convex, dual function has to be minimized
17 log (x)( f(x) dx.
(1)) = nE + log
exp
For discrete distributions, such as the categorical distribution used to represent the coefficients of GMM, we can directly work with those equations. For continuous distributions, Abdolmaleki al. (2015) propose approximating fx) with a local surrogate. The features to this surrogate are chosen such that they are compatible (Kakade) 2002). i.e., of the same form the distributions sufficient statistics. For multivariate Gaussians, the sufficient statistics are squared features and thus the surrogate compatible to such Gaussian distribution is given by
f(x) =
2 x Fx + f x x fo-
The parameters of this surrogate can now be used to update the natural parameters of the Gaussian,
ie, the precision matrix Q = E-1 and q = Σ-1 44 by
Q Q + 1p and q q: "
-r. 17
In order to apply the MORE algorithm to solve the optimization problems stated in Equation and [Equation 4 we make two trivial modifications. First, we invert the signs in Equation 3 and Equation 4| as we are now maximizing. Second, to account for the additional KL term our objectives, add 1 to n, everywhere except the first term of the sum in Equation 7
13

---

Published as a conference paper at ICLR 2020
C ELABORATION ON RELATED WORK
C.I RELATION BETWEEN EIM AND EM
Recall that the Expectation- Maximization (EM) algorithm (Dempster al. 1977) maximizes the log-likelihood of the data by iteratively maximizing and tightening the following lower bound
/ ((1|)) log ((x22) dz 7(=/x)
+ px()) [ ((=|<) log
((=(2) dz
p(x)) [log [(x)] Ep(x))
(21(2)
() + Bpa)) KL (((2|=)]la(222)))
lower bound
20
is instructive to compare our upper bound (Equation 1) to this lower bound. As mentioned, maxi- mizing the likelihood is equivalent minimizing the M-projection, i.e., argmin KL (p(x))lq<x)) where, in relation to objective, the model and true distribution have switched places in the non- symmetric KI. objectíve. Like our approach, EM introduces an auxiliary distribution 7(2[z) and bounds the objective from below by subtracting the KI between auxiliary distribution and model, i.e., KI. (ğ(= (=)()))))))))) In contrast, we obtain our upper bound by adding KI. (g([[=)]](((])) to the objective. Again, the distributions have exchanged places within the KL.
C.2 EQUALITY OF f-GAN AND --GAN
As pointed out in section 3 both the f-GAN (Nowozin et al. 2016) and the b-GAN (Uehara et al., 2016) yield the same objective for the I-projection.
We start with the f-GAN. Nowozin et al. on a variatonal bound for /divergences Nguyen et al. 2010)
2016) propose the following adversarial objective, based
argmin ((a) argmax (z)) F(g(x), V())) = Bp(a) [e(V(z))) - By(s)) If (z()))))
Here V(x) denotes neural network with linear output, g(v) the output activation and f* the Fenchel conjugate (Hiriart-Urruty Lemaréchal 2012) of f(u), i... the generator function of the f-divergence. For the 1-projection f(u) = log u and (t) = -1 log(t). In theory, the only restriction posed on the choice of g(v) is that outputs only values within the domain of f*( Nowozin et al. (2016) suggest y's for various f- divergences and chose exclusively monotony increasing functions which output large values for samples that are believed to be from the data distribution. For the I-projection they suggest g(v) -ezp(-vv). Thus the f-GAN objective for the I-projection is given by
argmin, (a) argmax F(g(x), Vxx) = - Ea(a) (exp(-V(()) Bx(a)) [1 V(x)].
The b-GAN objective follows from the density ratio estimation framework given by Sugiyama et al. 2012) and is given by
argmin ((x) (r) (f())) - Egt -
Here (u(u) denotes derivative [u) and r(x) denotes an density ratio estimator. We need to enforce r(x) > 0 for all to obtain valid density ratio estimate. In practice this is usually done by
learning ny = logr(z) (xx) instead. Plugging (z).. f()) and f'(u) 1/u into the general 6-GAN
objective yields
argmin ((x) argmax (1)))) F(g(x), -Ep(a)
Egt,,) [1 r()))
Which is the same objective as the -GAN uses. Yet, S-GAN and b-GAN objectives not identical for arbitrary fdiivergences.
D VISUALIZATION OF SAMPLES
D.1 PEDESTRIAN AND TRAFFIC PREDICTION
Samples from the Stanford Drone Dataset be found in Figure 7
14

---

Published as a conference paper at ICLR 2020
(a) Data
(b) EIM
(c) EIM + Features
(d) EM
Figure 7: Samples from the Dataset, EIM, EIM with features and EM, plotted over reference image from the Stanford Drone Dataset and the generated mask. In the mask green corresponds to valid regions and red invalid regions. EIM with the additional feature input generates samples that stay within the 'road mask' and therefore considered be more realistic.
E HYPERPARAMETERS
In all experiments, we realize the density ratio estimator as fully connected neural networks which we train using Adam (Kingma Ba] 2014) and early stopping using validation
Comparison to Generative Adversarial Approaches and Ablation Study
Data: 10, 000 Train Samples, 5, 000 Test Samples, 5, 000 Validation samples (for early stopping the density ratio estimator)
Density Ratio Estimator (EIM) Variational function (ax) (f-GAN): 3 fully connected layers, 50 neurons each, trained with 1.2 regularization with factor 0.001, early stopping and batch size 1, 000
Updates EIM: MORE-like updates with e 0.05 for components and coefficients, 1, 000 samples per component and update
Updates FGAN: Iterate single update steps for generator and discriminator using learning rates of 1e 3 and batch size of 1, 000.
Line Reaching with Planar Robot
Data: 10, 000 train samples, 000 test samples, 000 validation samples (for early stop- ping the density ratio estimator)
Density Ratio Estimation: fully connected layers of width 100, early stopping and batch size 1, 000
Updates: MORE-like updates with e = 0.005 for components and coefficients, 000 samples per component and update
Pedestrian and Traffic Prediction
Data SDD: 7, 500 train samples, 3, 801 test samples, 3, 801 validation samples (for early stopping the density ratio estimator)
Data NGS: 10, 000 train samples, 5, 000 test samples, 5, 000 validation samples (for early stopping the density ratio estimator)
Density Ratio Estimation: fully connected layers of width 256, trained with L2 regular- ization with factor 0.0005 early stopping and batch size 1, 0000
Updates: MORE-like updates with 0.01 for components and coefficients, 1, 000 sam- ples per component and update
15

---

Published as a conference paper at ICLR 2020
Obstacle Avoidance
Data: 1, 000 train contexts with 10 samples each, 500 test contexts with 10 samples each, 500 validation contexts with 10 samples each (for early stopping the density ratio estima- tor).
Density Ratio Estimation: fully connected layers of width 256, trained with L2 regular- ization with factor 0.0005 early stopping and batch size 0000
Component and Gating networks: fully connected layer of width 64 for each component and the gating. Trained with Adam (a le 3, So 0.5) for 10 epochs in each iteration.
16


Reviews:
[{"review_id": "rkxrfrIfcB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper propose EIM an analog to EM but to perform the I-projection (i.e. reverse-KL) instead of the usual M-projection for EM. The motivation is that the reverse-KL is mode-seeking in contrast to the forward-KL which is mode-covering. The authors argue that in the case that the model is mis-specified, I-projection is sometimes desired as to avoid putting mass on very unlikely regions of the space under the target p.\n\nThe authors propose an iterative procedure that alternates between estimating likelihood ratios and proposal distribution by minimizing an upper bound on the reverse-KL. The derivations seem correct. There are some experiments, majoritarily in the robotics domain. As the author point out, likelihood shouldn't be the right metric since you are now minimizing the reverse-KL---I would have liked the authors to spend some more time on the right way to evaluate---and actually use that new metric. Finally, there has been plethora of work on different objectives and distance between distributions as well as a zoo of lower/upper bounds on how to evaluate them---it would be interesting to have more connections to prior work.\n\n[Pros]\n- clearly written\n- clear motivation\n- correct derivations\n- interesting algorithm\n\n[Cons]\n- experiments are a little weak (and focus on a single domain)\n- would have liked to see an explicit algorithm for the optimization procedure\n- small lack of clarity in the presentation of Section 4.1---notation q_t is not introduced for example\n- more discussion about the evaluation metric\n- linking it more to prior work\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "SyxDIDyp5S", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, the authors proposed a new algorithm -- expected information maximization (EIM) -- for computing the I-projection of the data distribution to the model distribution, solely based on samples for general latent variable models, where the paper only focus on Gaussian mixtures models and experts. The proposed method applies a variational upper bound to the I-projection objective which is decomposable for each mixture components and the coefficients. Overall, I think the proposed technique quite sound and results are convincing. However, I do have some questions:\n\nQuestions:\n-- The proposed EIM algorithm in Sec 4.1 seems to require “re-training” the discriminator every time the q function is updated. How this can be applicable to more realistic and complex models where training requires millions of gradient steps? Will the same algorithm can be applied on more general latent variable models or even implicit models like GAN does? As the paper has pointed out, the vanilla f-GAN itself can be seen as optimizing some forms of the I-Projection (reverse Kullback-Leibler divergence) objective.\n-- I am a little confused about Sec 4.3. It seems that the latent variable z is not necessary for the proposed EIM? \n-- Also, the typical practise of training GAN is also iterative between the generator and the discriminator, we sometimes need to update the discriminator with more steps than the generator? Shouldn’t it be the exactly same as the proposed EIM except we have an additional regularization term of KL(q(x) || q_t(x)) which might be the true reason why training gets more stable than standard GAN?\n-- Similar to the previous two questions, how to compute the regularization term KL(q(x) || q_t(x)) in EIM for normal generator which is typically implicit?\n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "B1er3Km0KH", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper presents an algorithm to match two distributions with latent variables, named expected information maximization (EIM). Specifically, EIM is based on the I-Projection, which basically is equivalent to minimizing the reverse KL divergence (i.e. min KL[p_model || p_data]); to handle latent variables, an upper-bound is derived, which is the corresponding reverse KL divergence in the joint space. To minimize that joint reverse KL, a specific procedure is developed, leading to the presented EIM. EIM variants for different applications are discussed. Fancy robot-related experiments are used to evaluate the presented algorithm.\n\nOverall, the paper is in good shape wrt the logic and the writing. My main concerns focus on the novelty (compared to existing methods that are similar but not discussed) and the experiments. For the former, reverse KL has been exploited before, both in the marginal space [1] and the joint one [2]. Other detailed comments are listed below.\n\nAs Eq 4 is for matching two joint distributions, discussions/comparisons should be made to reveal the novelty of the presented EIM over existing methods such as [2], etc.\n\nIn Figure 2 (b), the experimental settings for adversarial learning are not fair, as the discriminator is not fixed there. \n \nIn Sec 4.4, it seems EIM is highly overlapped with VIPS. So what're the advantages of EIM here?\n\nIn Figure 3, how many steps for Generator and Discriminator are used for f-GAN? Does f-GAN finally converge? It would be helpful if some results are given to demonstrate the final state of each method.\n\nIn Eq. 9, adding the denominator q(z_i) will change the optimal solution. Why only add it to the first term?\n\nIn Section 5.3 and Figure 5, “SSD” might be a typo. \n\n[1] Adversarial Learning of a Sampler Based on an Unnormalized Distribution. AISTATS 2018.\n[2] Symmetric Variational Autoencoder and Connections to Adversarial Learning. AISTATS 2019.\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a new algorithm called Expected Information Maximization (EIM) for learning latent variable models while computing the I-projection solely based on samples. The reviewers had several questions, which the authors sufficiently answered. The reviewers agree that the paper should be accepted. The authors should carefully read the reviewer questions and comments and use them to improve their final manuscript. 
Decision: Accept (Poster)