E XPECTED I NFORMATION M AXIMIZATION U SING THE I-P ROJECTION FOR M IXTURE D ENSITY E STIMATION
Anonymous authors
Paper under double-blind review
A BSTRACT
Modelling highly multi-modal data is a challenging problem in machine learning.
Most algorithms are based on maximizing the likelihood, which corresponds to the M(oment)-projection of the data distribution to the model distribution.
The
M-Projection forces the model to average over modes that can not be represented by the model.
In contrast, the I(information)-Projection ignores such modes in the data and concentrates on the modes the model can represent.
Such behavior is appealing whenever we deal with highly multi-modal data where it is more important to model single modes correctly instead of covering all the modes.
Despite this advantage, the I-projection is rarely used in practise due to the lack of algorithms that can efﬁciently optimize it based on data.
In this work, we present a new algorithm called Expected Information Maximization (EIM) for computing the I-projection solely based on samples for general latent variable models, where we focus on
Gaussian mixtures models and Gaussian mixture of experts.
Our approach applies a variational upper bound to the I-projection objective which decomposes the original objective into single objectives for each mixture component as well as for the coefﬁcients, allowing an efﬁcient optimization.
Similar to GANs, our approach also employs discriminators but uses a more stable optimization procedures optimizing a tight upper bound.
We show that our algorithm is much more effective in computing the I-projection than recent GAN approaches and we illustrate the effectiveness of our approach for modelling multi-modal behavior on two pedestrian and trafﬁc prediction datasets.
I NTRODUCTION
Learning the density of highly multi-modal distributions is a challenging machine learning problem relevant to many ﬁelds such as modelling human behavior (Pentland & Liu, 1999).
Most common methods rely on maximizing the likelihood of the data.
It is well known that the maximum likelihood solution corresponds to computing the M(oment)-projection of the data distribution to the parametric model distribution (Bishop, 2006).
Yet, the M-projection averages over multiple modes in case the model distribution is not rich enough to fully represent the data (Bishop, 2006).
This averaging effect can result in poor models, that put most of the probability mass in areas that are not covered by the data.
The counterpart of the M-projection is the I(nformation)-projection.
The I-projection concentrates on the modes the model is able to represent and ignores the remaining ones.
Hence, it does not suffer from the averaging effect (Bishop, 2006).
In this paper, we explore the I-projection for mixture models which are typically trained by maximizing the likelihood via expectation maximization (EM) (Dempster et al., 1977).
Despite the richness of mixture models, the averaging problem still remains as we typically do not know the correct number of modes and it is hard to identify every mode in the data correctly.
By the use of the I-projection, our mixture models do not suffer from this averaging effect and can generate more realistic samples that are less distinguishable from the data.
We concentrate in this paper on learning
Gaussian mixture models and conditional Gaussian mixtures of experts (Jacobs et al., 1991) where the mean and covariance matrix are generated by deep neural networks.
[Figure]
We propose Expected Information Maximization (EIM) 1 , a novel approach capable of computing the I-projection between the model and the data.
By exploiting the structure of the I-projection, we can derive a variational upper bound objective, which was previously used in the context of variational inference (Maaløe et al., 2016; Ranganath et al., 2016; Arenz et al., 2018).
In order to work with this upper bound objective based on samples, we use a discriminator to approximate the required density ratio, relating our approach to GANs (Goodfellow et al., 2014; Nowozin et al., 2016; Uehara et al., 2016).
The discriminator also allows us to use additional discriminative features to improve the model quality.
In our experiments we demonstrate that EIM is much more effective in minimizing the I-projection than recent GAN approaches.
We apply EIM to a synthetic obstacle avoidance task, an inverse kinematic task of a redundant robot arm as well as a pedestrian and car prediction task using the Stanford drone dataset (Robicquet et al., 2016) and a trafﬁc dataset from the Next Generation Simulation program.
P RELIMINARIES
Our approach heavily builds on minimizing Kullback-Leibler divergences as well as density ratio estimates.
We will therefore brieﬂy review both concepts.
Density Ratio Estimation.
Our approach relies on estimating density ratios r ( x ) =
q ( x ) /p ( x ) based on samples of q ( x ) and p ( x ) .
Sugiyama et al.
(2012) introduced a framework to estimate such density ratios based on the minimization of Bregman divergences (Bregman, 1967).
For our work we employ one approach from this framework, namely density ratio estimation by binary logistic regression.
Assume a logistic regressor C ( x ) = σ ( φ ( x )) with logits φ ( x ) and sigmoid activation function σ .
Further, C ( x ) is trained on predicting the probability that a given sample x was sampled from q ( x ) .
It can be shown that such a logistic regressor using a cross entropy loss is optimal for
C ( x ) =
q ( x ) / (cid:0) q ( x ) + p ( x ) (cid:1) .
Using this relation, we can compute the log density ratio estimator by
[Equation]
The logistic regressor is trained by minimizing the binary cross entropy argmin
[Equation]
where different regularization techniques such as (cid:96) 2 regularization or dropout (Srivastava et al., 2014) can be used in addition to avoid overﬁtting.
Moment and Information Projection.
The Kullback-Leibler divergence (Kullback & Leibler, 1951) is a standard similarity measure for distributions.
It is deﬁned as KL ( p ( x ) || q ( x ))
= (cid:82) p ( x ) log p ( x ) /q
( x ) dx.
Due to its asymmetry, the Kullback-Leibler Divergence provides two different optimization problems (Bishop, 2006) to ﬁt a model distribution q ( x ) to a target distribution p ( x ) , namely argmin [Equation] KL
[Equation]
p
[Equation]
x )
[Equation]
x )) (cid:124) (cid:123)(cid:122) (cid:125)
Moment-projection
and argmin q ( x ) KL ( q ( x ) || p ( x )) (cid:124) (cid:123)(cid:122) (cid:125) Information-projection Here, we will assume that p ( x ) is the data distribution, i.e., p ( x ) is unknown but we have access to samples from p ( x ) .
It can easily be seen that computing the M-projection to the data distribution is equivalent to maximizing the likelihood (ML) of the model (Bishop, 2006).
ML solutions match the moments of the model with the moments of the target distribution, which results in averaging over modes that can not be represented by the model.
In contrast, the I-projection forces the learned generator q ( x ) to have low probability whenever p ( x ) has low probability, which is also called zero forcing .
R ELATED W ORK
We will now discuss competing methods for computing the I-projection that are based on GANs.
Those are, to the best of our knowledge, the only other approaches capable of computing the Iprojection solely based on samples of the target distribution.
Furthermore, we will distinguish our approach from approaches based on variational inference that also use the I-projection. Variational Inference.
The I-Projection is a common objective in Variational Inference (Opper &
Saad, 2001; Bishop, 2006; Kingma & Welling, 2013). Those methods aim to ﬁt tractable approximations to intractable distributions of which the unnormalized density is available. On the other hand, EIM does not assume access to the unnormalized density of the target distributions but only to samples, thus it is not a variational inference approach, but a density estimation approach. However, our approach uses an upper bound that has been previously applied to variational inference (Maaløe et al., 2016; Ranganath et al., 2016; Arenz et al., 2018).
EIM is especially related to the VIPS algorithm (Arenz et al., 2018), which we extend from the variational inference case to the density estimation case as well as introducing conditional latent variable models into the approach.
Generative Adversarial Networks.
While the original GAN approach has been shown to minimize the Jensen-Shannon Divergence (Goodfellow et al., 2014), GANs have since been adopted to a variety of other distance measures between distributions, such as the Wasserstein distance (Arjovsky et al., 2017), symmetric KL (Chen et al., 2018) and arbitrary f-divergences (Ali & Silvey, 1966; Nowozin et al., 2016; Uehara et al., 2016; Poole et al., 2016). Since the I-projection is a special case of an f-divergence, those approaches are of particular relevance to our work.
Nowozin et al.
(2016) use a variational bound for f -divergences (Nguyen et al., 2010) to derive their approach, the f -GAN.
Uehara et al.
(2016) use a bound that directly follows from the density ratio estimation under Bregman divergences framework introduced by Sugiyama et al.
(2012) to obtain their b -GAN.
While the b -GAN’s discriminator directly estimates the density ratio, the f -GAN’s discriminator estimates an invertable mapping of the density ratio.
Yet, in the case of the I-projection, both the f -GAN and the b -GAN yield the same objective, as we show in Appendix C.2.
For both the f -GAN and b -GAN the discriminator objective is determined by the desired f -divergence.
Uehara et al.
(2016) note that the discriminator objective which is implied by the I-projection is unstable.
As both approaches are formulated in a general way to minimize any f-divergence, they do not exploit the special structure of the I-projection which permits us to apply a tight upper bound of the I-projection for latent variable models, resulting in a higher quality of the estimated models.
Li et al.
(2019) introduce an adversarial approach to compute the I-projection based on density ratios, estimated by logistic regression. Yet, their approach assumes access to the unnormalized target density, i.e., they are working in a variational inference setting.
Most importantly, opposed to GANs, EIM is not based on an adversarial formulation and no adversarial game has to solved. This removes a major source of instability in the training process which is discussed in more detail in Section 4.3.
E XPECTED I NFORMATION M AXIMIZATION
Expected Information Maximization (EIM), is a general algorithm for minimizing the I-projection for any latent variable model.
We ﬁrst derive EIM for general marginal latent variable models, i.e., q ( x ) = (cid:82) q ( x | z ) q ( z ) dz and subsequently extend our derivations to conditional latent variable models, i.e, q ( x | y )
= (cid:82) q ( x | z, y )
q ( z | y )
dz.
EIM uses an upper bound for the objective of the marginal distribution.
Similar to Expectation Maximization (EM), our algorithm is iterative, where in the corresponding M-step, we minimize the upper bound and in the E-step, we tighten the bound using a variational distribution.
4. EIM FOR L ATENT V ARIABLE M ODELS
The I-projection can be simpliﬁed using a (tight) variational upper bound (Arenz et al., 2018) which can be obtained by introducing an auxiliary distribution ˜ q ( z | x ) and using Bayes rule
[Equation]
The derivation of the bound is given in Appendix B.
It is easy to see that U ˜ q,p ( q ) is an upper bound as the expected KL term is always non-negative.
In the corresponding E-Step, the model from the previous iteration, which we denote as q t ( x ) , is used to tighten the bound by setting ˜ q ( z | x ) =
q t ( x | z ) q t ( z ) /q
t ( x ) .
In the M-Step the model distribution is updated by minimizing the upper bound U ˜ q,p ( q ) Yet, opposed to Arenz et al.
(2018), we can not work directly with the upper bound since it still depends on log p ( x ) which we can not evaluate.
However, we can reformulate the upper bound by setting the given relation for ˜ q
( z | x ) of the E-step into Eq. 4,
U q
[Equation]
The upper bound contains now a density ratio between the old model distribution and the data.
This density ratio can be estimated using samples of q t and p , for example, by using logistic regression, see Section 2.
We can use the logits φ ( x ) of such a logistic regressor to estimate the log density ratio log( q t ( x ) /p
( x )) in Equation 5, yielding an upper bound U q t ,φ ( q ) that depends on φ ( x ) instead of p ( x ) .
Optimizing this bound corresponds to the M-step of our approach.
In the E-step, we simply set q t to the newly obtained q and retrain the density ratio estimator φ ( x ) .
Both steps formally result in the following bilevel optimization problem
[Equation]
Using a discriminator also comes with the advantage that we can use additional discriminative features g ( x ) as input to our discriminator that are not directly available for the generator.
For example, if x models trajectories of pedestrians, g ( x ) could indicate whether the trajectory reaches any positions that are not plausible, for example, rooftops or trees.
These features simplify the discrimination task and can therefore improve our model accuracy which is not possible with M-projection based algorithms such as EM.
4. EIM FOR C ONDITIONAL L ATENT V ARIABLE M ODELS
For conditional distributions, we aim at ﬁnding the conditional I-projection
[Equation]
The derivations for the conditional upper bound follow the same steps as the derivations in the marginal case, where all distributions are now extended by the context variable y . We refer to the supplement for details.
The log density ratio estimator φ ( x, y ) is now trained on the joint space of x and y ..
For each input in our training data set, we generate a new sample x using distribution q old ( x | y ) for training φ ( x, y ) .
Hence, as the distribution for y is the same for the true data and the generated data, φ ( x, y ) estimates the log density ratio of the conditional distributions instead of the joint distributions.
4. R ELATION TO GAN S AND EM
There is a close relation of EIM to GANs due to the use of a logistic discriminator for the density ratio estimation.
It is therefore informative to investigate the differences in the case without latent variables.
In an adversarial formulation, the density ratio estimator would directly replace the density ratio in the original I-projection equation, i.e.,
[Equation]
− 4 − 3 − 2 − 1      (a)LogDensityRatio log q t ( x ) − log p ( x ) − 4 − 3 − 2 − 1  30 .0 .0 .0 .(b)AdversarialFormulation
step 1 step 2 stepstep 4 step 5 stepstep− 4 − 3 − 2 − 1  30 .0 .0 .0 .(c)EIM
p ( x ) q t ( x )
However, such adversarial games are often hard to optimize in a stable manner.
In contrast, EIM offers a bilevel optimization problem where the discriminator is explicitly learned on the old data distribution q t ( x ) ,
[Equation]
i e., there is no circular dependency between the optimal generator and the optimal discriminator.
Figure 2 illustrates that the proposed non-adversarial formulation does not suffer from too large model updates. Choosing the number and step-size of the updates is thus far less critical.
The EIM algorithm can be also seen as the counter-part of Expectation-Maximization (EM).
While
EM is used for the M-projection with latent variable models, EIM uses the I-projection.
However, both approaches decompose the corresponding projections into a upper bound (or lower bound for
EM) and a KL term that depends on the conditional distribution
q ( z | x ) to tighten this bound.
The exact relationship is discussed in Appendix C.1.
4. EIM FOR G AUSSIAN M IXTURES M ODELS
We consider Gaussian mixture models with d components, i.e. multivariate Gaussian distributions q ( x | z i ) =
N ( µ i , Σ i ) and a categorical distribution q ( z ) =
Cat ( π ) for the coefﬁcients.
As the latent distribution q ( z ) is discrete, the upper bound in EIM Equation 5 simpliﬁes, as the integral over z can be written as a sum.
Similar to the EM-algorithm, this objective can be updated individually for the coefﬁcients and the components.
For both updates, we will use similar update rules as deﬁned in the
VIPS algorithm (Arenz et al., 2018).
VIPS uses a trust region optimization for the components and the coefﬁcients, where both can be solved in closed form as the components are Gaussian.
The trust regions prevent that the new model will go too far away from q t where the density ratio estimator is inaccurate, and hence, further stabilizes the learning process.
We will now sketch both updates, where we refer to Appendix B.2 for the full details.
For updating the coefﬁcients, we assume that the components have not yet been updated, and therefore KL ( q ( x | z i ) ||
q t ( x | z i ))
=
for all z i .
The objective for the coefﬁcients thus simpliﬁes to
[Equation]
where φ ( z i ) can be approximated using samples from the corresponding component.
This objective can easily be optimized in closed form, as also shown in the VIPS algorithm
(Arenz et al., 2018).
We also use a KL trust-region to specify the step size of the update.
For updating each individual component, the objective simpliﬁes to
[Equation]
As in VIPS, this optimization problem can be solved in closed form using the MORE algorithm (Abdolmaleki et al., 2015).
The MORE algorithm uses a quadratic surrogate function that locally approximates φ ( x ) .
The resulting solution optimizes Equation 7 under a KL trust region.
The pseudo code of EIM for GMMs can be found in Appendix A.
[Figure]
I
[Figure]
EIMEIM
[Figure]
JointEIMJointEIM
[Figure]
[Figure]
Dimensionality I- P r o j ec ti on (c
[Figure]
8 Components
4. EIM FOR G AUSSIAN M IXTURE OF E XPERTS
In the conditional case we consider mixture of expert models consisting of d multivariate
Gaussians, whose parameters depend on an input y in a nonlinear fashion, i e. q ( x | z i , y ) =
N ( ψ µ,i ( y ) , ψ Σ ,i ( y )) and the gating is given by a neural network with softmax output.
The resulting upper bound can again be decomposed into individual update steps for the components and the gating.
Yet, closed form solutions are no longer available and we need to resort to gradient based updates.
The objective for updating the gating is given by
[Equation]
We minimize this equation w.r.t the parameters of the gating by gradient descent using the Adam (Kingma & Ba, 2014) algorithm.
The objective for updating a single component i is given by
[Equation]
where ˜ p ( y | z )
= p ( y ) q ( z i | y ) /q ( z i ) .
Note that we normalized the objective by q ( z i ) =
(cid:82) p ( y ) q
( z i | y )
d y to insure that also components with a low prior q ( z i ) get large enough gradients for the updates.
As we have access to the derivatives of the density ratio estimator w.r.t.
x , we can optimize Equation 9 with gradient descent using the reparametrization trick (Kingma & Welling, 2013) and Adam.
E VALUATION
We compare our approach to GANs and perform an ablation study on a toy task, where the data is generated from a known mixture model.
We further apply our approach to two synthetic data sets, learning the joint conﬁgurations of a planar robot as well as a non-linear obstacle avoidance task, and two real datasets, namely the Stanford drone dataset (Robicquet et al., 2016) and a trafﬁc dataset from the Next Generation Simulation program.
A full overview of all hyperparameters and network architectures can be found in Appendix E.
5. C OMPARISON TO G ENERATIVE A DVERSARIAL A PPROACHES AND A BLATION S TUDY
We compare to the f -GAN which is the only other method capable of minimizing the I-projection solely based on samples.
We use data sampled from randomly generated GMMs with different numbers of components and dimensionalities.
In order to study the inﬂuence to generative adversarial approaches we also compare to a version of EIM where we neglect the additional KL-term (EIM, no KL), a version which were we trained all components and the coefﬁcients jointly using gradient descent (Joint EIM) and a version where we do both (Joint EIM, no KL).
The average I-projection achieved by the various approaches can be found in Figure 3.
[Figure] [Figure]
5. L INE R EACHING WITH P LANAR R OBOT
We extended the introductory example of the planar reaching task and collected expert data from a 10 -link planar robot tasked with reaching a point on a line.
We ﬁtted GMMs with an increasing number of components using EIM, EIM with additional features, where the end-effector coordinates for a given joint conﬁguration were provided, and EM.
Even for a large number of components, we see effects similar to the introductory example, i.e., the M-projection solution provided by EM fails to reach the line while EIM manages to do so.
For small numbers of components, parts of the line are ignored by EIM, while more and more parts of it get covered as we increase the number of components.
For the additional features, the line reaching was even more accurately imitated.
See
Figure 4, for average distance between hand and the line as well as samples from both EM and EIM.
5. P EDESTRIAN AND T RAFFIC P REDICTION
We evaluated our approach on data from the Stanford Drone Dataset (SDD) (Robicquet et al., 2016) and a trafﬁc dataset from the Next Generation Simulation (NGS) program 2 .
The SDD data consists of trajectories of pedestrians, bikes and cars and we only used the data corresponding to a single video of a single scene (Video 1, deathCircle).
The NGS data consists of trajectories of cars where we considered the data recorded on Lankershim Boulevard.
In both cases we extracted trajectories of length 5 , yielding highly multimodal data due to pedestrians, bikes and cars moving at different speeds and in different directions.
We evaluated on the achieved log-likelihood of EIM and EM, see Figure 5.
EM was able to achieve the highest likelihood as this measure is directly optimized
[Figure]
by EM.
However, we can already see that EM massively overﬁts in case we increase the number of components as the test-set likelihood degrades.
Yet, EIM could produce better models with an increasing number of components.
Additionally, we generated a mask indicating whether a given point is on the road or not and evaluated how realistic the learned models are by measuring the amount of samples violating the mask, i.e., predicting road users outside of the road.
We also evaluate a version of EIM where we provide additional features indicating if the mask is violated.
EIM achieves a much better value on this mask for the NGS dataset, while we needed the additional mask features for the discriminator on the SDD dataset to outperform EM.
Both experiments show that EIM can produce highly multi-modal density estimates that produce more realistic samples than
EM and can be further reﬁned by additional prior knowledge provided as feature vectors.
5. O BSTACLE A VOIDANCE
We evaluate the conditional version of EIM on an artiﬁcial obstacle avoidance task.
The conditional input is given by the location of three obstacles within an image.
The gating as well as the components are given by deep neural networks.
More details are given in the Appendix.
The data consists of trajectories going from the left to the right of the image.
The trajectories are deﬁned by settingvia points such that no obstacle is hit.
To generate the data we sample via points over and under the obstacles with a probability proportional to the distance between the obstacle and the image border.
Hence, for three obstacles, there are 2 3 = 8 different modes in the data.
Note that, like in most real world scenarios, the expert data is not perfect and about 13% of the trajectories in the dataset hit an obstacle.
We ﬁt models with various numbers of components to this data using EIM and EM and compare their performance on generating trajectories that achieve the goal.
Results are shown in Figure 6 together with a visualization of the task and samples produced by EIM and EM.
EIM was able to identify most modes for the different given inputs and did not suffer from any averaging effect.
In contrast, EM was not able to ﬁnd all modes.
As a consequence, some components of the mixture model had to average over multiple modes, resulting in poor quality trajectories.
C ONCLUSION
We introduced Expected Information Maximization (EIM), a novel approach for computing the Iprojection between general latent variable models and a target distribution, solely based on samples of the latter.
General upper bound objectives for marginal and conditional distributions were derived, resulting in an algorithm similar to EM, but tailed for the I-projection instead of the M-projection.
We introduced efﬁcient methods to optimize these upper bound objectives for mixture models.
In our experiments we demonstrated the beneﬁts of the I-projection for different behavior modelling tasks.
The introduced approach opens various pathways for future research.
While we focused on mixture models, the derived upper bounds are not exclusive to those, but can be used for arbitrary latent variable models.
Another possibility is an online adaptation of the number of used components.
Arenz et al.
(2018) propose heuristics for such an adaptation in their VIPS approach.
Those can easily be adapted to our approach.
P SEUDO C
ODE
EIM-for p =
q Input: Data { x ( j ) p } j =1 ··· N , Initial Model q ( x ) =
(cid:80) di
=1 q ( x | z i ) q ( z i ) = (cid:80) di
=
π i N ( x | µ i , Σ i ) for i in number of iterations do
E-Step: q t ( z ) =
q ( z ) , q t ( x |
z i ) =
q ( x | z i ) for all components i Update Density Ratio Estimator: sample data from model { x ( j ) q } j =1 ··· N ∼ q t ( x ) retrain density ratio estimator φ ( x ) on { x ( j ) p } j =1 ··· N and { x ( j ) q } j =1 ··· N M-Step Coefﬁcients: for i in number of components do compute loss l i = 1 N (cid:80) N j =1 φ (cid:16) x ( j ) q (cid:17) with samples { x ( j ) q } j =1 ··· N ∼ q t ( x | z i ) end update q ( z ) using losses l i and MORE equations
M-Step Components: for i in number of components do ﬁt ˆ φ ( x ) surrogate to pairs (cid:16) x ( j ) q , φ (cid:16) x ( j ) q (cid:17)(cid:17) with samples { x ( j ) q } j =1 ··· N ∼ q t ( x | z i ) update q ( x | z i ) using surrogate ˆ φ ( x ) and MORE equations end end
Pseudo Code for EIM for GMMS can be found in algorithm
B D ERIVATIONS
Derivations of the upper bound stated in Equation 3.
We assume latent variable model q ( x )
=
(cid:82) q ( x | z ) q ( z ) dz and use the identities q ( x | z ) q ( z ) =
q ( z | x ) q ( x ) and log q ( x ) = log q ( x | z ) q ( z ) − log q ( z | x ) .
KL ( q ( x ) || p ( x )) =
(cid:90) q
( x ) log q ( x ) p ( x ) dx =
(cid:90)(cid:90) q
( x | z ) q ( z ) log q ( x ) p ( x ) dzdx = (cid:90)(cid:90) q
( x | z ) q ( z ) (cid:18) log q ( x | z ) q ( z ) p ( x ) − log q ( z | x ) (cid:19) dzdx
=
(cid:90)(cid:90) q
( x | z ) q ( z ) (cid:18) log q ( x | z ) q ( z ) p ( x ) − log q ( z | x ) + log ˜ q ( z | x ) − log ˜
q ( z | x ) (cid:19) dzdx =
(cid:90)(cid:90) q
( x | z ) q ( z ) (cid:18) log q ( x | z ) q ( z ) p ( x ) − log ˜
q ( z | x ) (cid:19) dzdx − (cid:90)(cid:90) q
( x | z ) q ( z ) (log q ( z | x ) − log ˜
q
( z | x )) dzdx =
(cid:90)(cid:90) q
( x | z ) q ( z ) (cid:18) log q ( x | z ) q ( z ) p ( x ) − log ˜
q ( z | x ) (cid:19) dzdx − (cid:90) q ( x ) (cid:90) q
( z | x ) log q ( z | x ) ˜ [Equation] ( z | x ) dzdx =
U ( q, ˜ q, p ) − E
After plugging the E-Step, i
[Equation]
e., ˜ q
( z | x ) =
q t ( x | z ) q t ( z ) /q
t ( x ) , into the objective it simpliﬁes to
U
[Equation]
which concludes the derivation of upper bound of latent variable models
[Equation]
B. D ERIVATIONS C ONDITIONAL U PPER B OUND
By introducing an auxiliary distribution ˜ q ( z | x
[Equation]
y )
the upper bound to the expected KL for conditional latent variable models q ( x | y )
= (cid:82) q ( x | z, y )
q ( z | y ) dz can be derived by
[Equation]
) − log
[Equation]
( z | x
[Equation]
y ) (cid:19) dzdxdy − (cid:90) p ( y )
[Equation]
q ( x | z
[Equation]
y )
[Equation]
( z
[Equation]
y ) (log q ( z | x, y ) − log ˜ q ( z | x, y ))
dzdxdy =
(cid:90) p ( y ) (cid:90)(cid:90)
[Equation]
( x | z, y )
[Equation]
( z | y ) (cid:18)
[Equation]
=
[Equation] [Equation]
p
[Equation]
p ( y )
[Equation]
( x | y )
[ KL ( q ( z | x, y ) ||
˜ q ( z | x, y ))] .
During the E-step the bound is tightened by setting ˜ q
[Equation]
z
[Equation]
x, y
[Equation]
which concludes the derivation of the upper bound for conditional latent variable models
[Equation]
B. U SING MORE FOR CLOSED FORM UPDATES FOR GMM S
The MORE algorithm, as introduced by Abdolmaleki et al.
[Equation] can be used to solve optimization problems of the following form
[Equation]
for an exponential family distribution q ( x ) , some function f and an upper bound on the allowed change, (cid:15) .
Abdolmaleki et al.
(2015) show that the optimal solution is given by
[Equation]
where η denotes the Lagrangian multiplier corresponding to the KL constraint.
In order to obtain this Lagrangian multiplier, the following, convex, dual function has to be minimized
[Equation]
For discrete distributions, such as the categorical distribution used to represent the coefﬁcients of a GMM, we can directly work with those equations.
For continuous distributions, Abdolmaleki et al.
(2015) propose approximating f ( x ) with a local surrogate.
The features to ﬁt this surrogate are chosen such that they are compatible (Kakade, 2002), i.e., of the same form as the distributions sufﬁcient statistics.
For multivariate Gaussians the sufﬁcient statistics are squared features and thus the surrogate compatible to such a Gaussian distribution is given by
[Equation]
The parameters of this surrogate can now be used to update the natural parameters of the Gaussian, i.e, the precision matrix Q = Σ − 1 and q = Σ − 1 µ by
[Equation]
In order to apply the MORE algorithm to solve the optimization problems stated in Equationand Equation 7 we make two trivial modiﬁcations.
First, we invert the signs in Equation 6 and
Equation 7, as we are now maximizing.
Second, in order to account for the additional KL term in our objectives, we add 1 to η , everywhere except the ﬁrst term of the sum in Equation
C E LABORATION ON R ELATED W ORK C. R ELATION BETWEEN EIM AND EM
Recall that the Expectation Maximization (EM) algorithm
Dempster et al.
(1977) maximizes the log-likelihood of the data by iterativley maximizing and tightening the following lower bound
[Equation]
[Figure]
[Equation]
It is instructive to compare our upper bound (Equation 3) to this lower bound.
As mentioned, maximizing the likelihood is equivalent to minimizing the M-projection, i.e., KL ( p ( x ) || q ( x )) , where, in relation to our objective, the model and true distribution have switched places in the nonsymmetric KL objective.
Like our approach, EM introduces an auxiliary distribution ˜ q ( z | x ) and bounds the objective from below by subtracting the KL between auxiliary distribution and model, i.e., KL (˜ q ( z | x ) ||
q ( z | x )) .
In contrast, we obtain our upper bound by adding KL ( q ( z | x ) ||
˜ q ( z | x )) to the objective.
Again, the distributions have exchanged places within the KL.
C. E QUALITY OF f AND b -GAN
As pointed out in section 3 both the f -GAN (Nowozin et al., 2016) and the b -GAN (Uehara et al., 2016) yield the same objective for the I-projection.
We start with the f -GAN.
Nowozin et al.
(2016) propose the following adversarial objective, based on a variatonal bound for f -Divergences Nguyen et al.
[Equation]
Here V ( x ) denotes a neural network with linear output, g ( v ) the output activation and f ∗ ( t )
the
Fenchel conjugate (Hiriart-Urruty & Lemar´echal, 2012) of f ( u ) , i.e the generator function of the f-divergence.
For the I-projection f ( u ) = − log u and f ∗ ( t ) = − 1 − log( t ) .
In theory, the only restriction posed on the choice of g ( v ) is that it outputs only values within the domain of f ∗ ( t )
Nowozin et al.
(2016) suggest g ’s for various f − divergences and chose exclusively monotony increasing functions which output large values for samples that are believed to be true.
For the Iprojection they suggest g ( v ) =
− exp ( − v ) .
Thus the f -GAN objective for the I-projection is given by
[Equation]
The b -GAN objective follows from the density ratio estimation framework given by Sugiyama et al.
and is given by
[Equation]
Here f (cid:48) ( u ) denotes the derivative of f ( u ) and r ( x ) denotes an density ratio estimator.
We need to enforce r ( x ) > 0 for all x to obtain a valid density ratio estimate.
In practice this is usually done by learning r l ( x ) = log r ( x ) instead.
Plugging r l ( x ) , f ( u ) and f (cid:48) ( u ) = 1 /u into the general b-GAN objective yields
[Equation]
Which is the same objective as the f-GAN uses.
Yet, f-GAN and b-GAN objectives are not identical for arbitrary f-divergences.
D V ISUALIZATION OF S AMPLES D. P EDESTRIAN AND T RAFFIC P REDICTION
Samples fro the Stanford Drone Dataset can be found in Figure
[Figure]
E H YPERPARAMETERS
In all experiments, we realize the density ratio estimator as fully connected neural networks which we train using Adam (Kingma & Ba, 2014) and early stopping using a validation set.
Comparison to Generative Adversarial Approaches and Ablation Study
• Data: 10 , 000 Train Samples, 5 , 000 Test Samples, 5 ,
Validation samples (early stopping the Density Ratio Estimator) • Density Ratio Estimator (EIM) /
Variational function V ( x ) ( f -GAN): 3 fully connected layers, 50 neurons each, trained with L2 regularization with factor 0 .
001 , early stopping and batch size 1 ,• Updates EIM: MORE-like updates with (cid:15) = 0 .
for components and coefﬁcients, 1 ,samples per component and update • Updates FGAN:
Iterate single update steps for generator and discriminator using learning rates of 1 e − 3 and batch size of 1 , 000 .
Line Reaching with Planar Robot
• Data: 10 , 000 train samples, 5 , 000 test samples, 5 , 000 validation samples (early stopping the density ratio estimator) • Density Ratio Estimation: 2 fully connected layers of width 100 , early stopping and batch size 1 ,• Updates: MORE-like updates with (cid:15) = 0 .
005 for components and coefﬁcients, 1 ,samples per component and update
Pedestrian and Trafﬁc Prediction
• Data SDD: 7 , 500 train samples, 3 , 801 test samples, 3 , 801 validation samples (early stopping the density ratio estimator) • Data NGS: 10 , 000 train samples, 5 , 000 test samples, 5 , 000 validation samples (early stopping the density ratio estimator) • Density Ratio Estimation: 3 fully connected layers of width 256 , trained with L2 regularization with factor 0 .
0005 early stopping and batch size 1 , 000 . • Updates: MORE-like updates with (cid:15) = 0 .
01 for components and coefﬁcients, 1 , 000 samples per component and update
[Table]
• Data: 1 , 000 train contexts with 10 samples each, 500 test contexts with 10 samples each, 500 validation contexts with 10 samples each (for early stopping the density ratio estimator).
• Density Ratio Estimation: 3 fully connected layers of width 256 , trained with L2 regularization with factor 0 .
0005 early stopping and batch size 1 , 000 . • Component and Gating networks: 2 fully connected layer of width 64 for each component and the gating.
Trained with Adam ( α = 1 e − 3 , β 0 = 0 . 5 ) for 10 epochs in each iteration.