Mind the Gap when Conditioning Amortised Inference in Sequential Latent-Variable Models
Authors: Justin Bayer
Maximilian Soelch
Atanas Mirchev
Baris Kayalibay
Patrick van der Smagt
Venue: ICLR 2021 Poster (2021)
Keywords: variational inference
state-space models
amortized inference
recurrent networks

Abstract: Amortised inference enables scalable learning of sequential latent-variable models (LVMs) with the evidence lower bound (ELBO). In this setting, variational posteriors are often only partially conditioned. While the true posteriors depend, e.g., on the entire sequence of observations, approximate posteriors are only informed by past observations. This mimics the Bayesian filter---a mixture of smoothing posteriors. Yet, we show that the ELBO objective forces partially-conditioned amortised posteriors to approximate products of smoothing posteriors instead. Consequently, the learned generative model is compromised. We demonstrate these theoretical findings in three scenarios: traffic flow, handwritten digits, and aerial vehicle dynamics. Using fully-conditioned approximate posteriors, performance improves in terms of generative modelling and multi-step prediction.

Published as a conference paper at ICLR 2021
MIND THE GAP WHEN CONDITIONING AMORTISED INFERENCE IN SEQUENTIAL LATENT-VARIABLE MODELS
Justin Bayer*
Maximilian Soelch
Atanas Mirchev
Baris Kayalibay
Patrick van der Smagt
Machine Learning Research Lab, Volkswagen Group, Munich, Germany [bayerj, m. moseechh..soechh. Bargmx.. ai
ABSTRACT
Amortised inference enables scalable learning of sequential latent-variate models (LVMs) with the evidence lower bound (ELBO). In this setting, variational poste- riors are often only partially conditioned. While the true posteriors depend, e. g.. on the entire sequence of observations, approximate posteriors are only informed by past observations. This mimics the Bayesian filter-a mixture of smoothing posteriors. Yet, we show that the ELBO objective forces partially-conditioned amortised posteriors to approximate products of smoothing posteriors instead. Consequently, the learned generative model is compromised. We demonstrate these theoretical findings in three scenarios: traffic flow, handwritten digits, and aerial vehicle dynamics. Using fully-conditioned approximate posteriors, perfor- mance improves in terms of generative modelling and multi-step prediction.
INTRODUCTION
Variational inference has paved the way towards learning deep latent-variable models (LVMs): max- imising the evidence lower bound (ELBO) approximates maximum likelihood learning (Jordan et al., 1999; Beal, 2003; MacKay, 2003). An efficient variant is amortised variational inference represented by deep neural network, the inference network
where the approximate posteriors (Hinton et al., 1995). produces the parameters of the variational distribution for each observation by single forward pass in contrast to classical variational inference with full optimisation pro- cess per sample. The framework of variational auto-encoders (Kingma & Welling, 2014; Rezende al., 2014) adds the reparameterisation trick for low-variance gradient estimates of the inference network. Learning of deep generative models is hence both tractable and flexible, as all its parts can be represented by neural networks and fit by stochastic gradient descent.
The quality of the solution is largely determined by how well the true posterior is approximated: the gap between the ELBO and the log marginal likelihood is the KL divergence from the approximate to the true posterior. Recent works have proposed ways of closing this gap, suggesting tighter alternatives to the ELBO (Burda al., 2016; Mnih & Rezende, 2016) or more expressive variational posteriors (Rezende & Mohamed, 2015; Mescheder et al., 2017). Cremer et al. (2018) provide an analysis of this gap, splitting it in two. The approximation gap is caused by restricting the approximate posterior to specific parametric form, the variational family. The amortisation gap comes from the inference network failing to produce the optimal parameters within the family.
In this work, address previously unstudied aspect of amortised posterior approximations: suc- cessful approximate posterior design is not merely about picking a parametric form of a probability density. It is also about carefully deciding which conditions to feed into the inference network. This a particular problem in sequential LVMs, where it common practice to feed only a restricted set of inputs into the inference network: not conditioning on latent variables from other time steps can vastly improve efficiency (Bayer & Osendorfer, 2014; Lai et al., 2019). Further, leaving out
"Equal contribution.

---

Published as a conference paper at ICLR 2021
p==|C,C=0)
p(=10,0=1)
p(2
pp=
=(2|
Figure 1: Illustration of the effect of partial conditioning. Consider a latent-variable model p(C, C 2) x p(z), with binary C and arbitrary C. We omit C from the amortised approximate posterior q(z C). Shown are two cases of separated (left) and overlapping (right) Gaussian true posteriors. Top row: the full posteriors p(z C, = 0) and p(z C,C 1) as well as their av- erage, the marginal posterior p(= C). Middle row: Variational Gaussian approximation v/z C) the marginal posterior, which was obtained by stochastic gradient descent on the reverse, mode- seeking KL-divergence (Hoffman et al., 2013). Bottom row: The optimal w(z C) obtained by optimising the ELBO with partially-conitiions located far away from the modes, sharply peaked and shares little mass with the true full posteriors, the marginal posterior as well as the approximate marginal posterior.
amortised approximate posterior W. t. q. It is
observations from the future structurally mimics the Bayesian filter, useful for state estimation in the control of partially observable systems (Karl al., 2017b; Hafner et al., 2019; Lee al., 2020).
We analyse the emerging conditioning gap and the resulting suboptimality for sequential amortised inference models. If the variational posterior is only partially conditioned, all those posteriors equal w. included conditions but different on the excluded ones will need to share an approximate posterior. The result is suboptimal compromise be mitigated by improving variational family or inference network capacity. We empirically show its effects in extensive study on three real-world data the common use case of variational state-space models.
many different true posteriors which cannot
AMORTISED VARIATIONAL INFERENCE
2.1 VARIATIONAL AUTO-ENCODERS
Consider the task of fitting the parameters of a latent-variable model po (x z) = po(x 2)) x po(z) to target distribution p(x) by maximum likelihood:
arg max Ex-((x) [lo (x
In the case of variational auto-encoders (VAEs), the prior distribution (z) is often simple, such as a standard Gaussian. The likelihood pe x z) on the other hand is mostly represented by a deep neural network producing the likelihood parameters
function of z.
The log marginal likelihood log Po (x) = log f po (x, z) dz contains a challenging integral within a logarithm. The maximisation of eq. (1) is thus generally intractable. Yet, for single sample the log marginal likelihood can be bounded from below by the evidence lower bound (ELBO):
log Po (x) 2 log (x) (Z( Po x))
= Ezzgl [log Moggo(( x)] (L(((z)) po(z)) E(00. x),
where a surrogate distribution q, the variational posterior. is introduced. The bound is tighter the better q(z) approximates po x). The gap vanishes with the posterior KI. divergence in eq. (2).
2

---

Published as a conference paper at ICLR 2021
Typically, the target distribution is given by finite data set D {x inference, equally many variational posteriors (z) are determined by direct optimisation.
In=1. In classical variational
For instance, each Gaussian (()) (z) is represented by its mean n and standard deviation o (()), The set all possible distributions of that parametric form called the variational family Q 9 ((z).
Learning the generative model parameters & can then be performed by an empirical risk minimisation approach, where the average negative ELBO over a data set is minimised:
N
Σ ( g (2), x(m)
arg
min 0
The situation is different for VAEs. The parameters of the approximate posterior for specific sample are produced by another deep neural network, the inference network go(z x). Instead of directly optimising for sample-wise parameters, global set of weights e for the inference network is found by stochastic gradient-descent of the expected negative ELBO:
arg min [x-0(x)(((0 (z x).
8,0
This approach tends scale more gracefully large data
The Sequential Case
The posterior of a sequential latent-variable model
T
p(x1TT Zi:r)
II p(x: X1a- Zza)p(Zz
of observations (xi....,xp) = X1:7 and latents (Zi....,ZT) ZT) ZI.T, with Z1.0 0. factorises as
p(Z XI.T
II
p(z: 11
(3)
t=1
The posterior KL divergence is a sum of step-wise divergences (Cover & Thomas, 2006):
KL(g(ZIT p(ZET Xx-T)) = Σ
E EZ 1 [LL( 1-) p(z; Zia x.x))
(4)
t=1
where we have left out the conditions of q(Z lows to focus the non-sequential case our theoretical analysis, even though the phenomenon most commonly occurs with sequential LVMs.
1I. q(z, 1.) for now. This decomposition al-
2.2 THE APPROXIMATION AND AMORTISATION GAPS
The ELBO and the log marginal likelihood are separated by the posterior Kullback-Leibler diver- gence KL(g(z) || pz x)), cf. eq. (2). If the optimal member of the variational family q" E Q is not equal to the true posterior, or p(z x) & Q, then
KL (z) || Po x)) >0
so that the ELBO is strictly lower than the log marginal likelihood the approximation gap.
Using an inference network we must expect that even the optimal variational posterior within the variational family is not found in general. The gap widens:
(L (z x) (z x)) 2 KL(q" (z) || po x)).
This phenomenon was first discussed by Cremer et al. (2018). The additional gap
(z x) (o x)) (T. (z) || po x)) 2 0.
called the amortisation gap.
The two gaps represent two distinct suboptimality cases when training VAEs. The former is caused by the choice of variational family, the latter by the success the amortised search for a good candidate E Q.

---

Published as a conference paper at ICLR 2021
Table Overview of partial conditions for sequential inference networks g(z, Ce) in the literature. T denotes missing conditions vs. the true posterior according to the respective graphical model. DKF acknowledges the true factorization, but does not use 2 in any experiments.
Model
C
Ct
STORN (Bayer Osendorfer, 2014)
X1:7
21:411
VRNN (Chung et al., 2015)
X1:6, 21:1-1
X(+1:7
DKF (Krishnan et al., 2015)
X1:7
2: (exper.)
DKS (Krishnan al., 2017)
x
DVBF (Karl et al., 2017a)
X2, Mt-1
X++1:T
Planet (Hafner al., 2019)
X1, 7t-1
X411:T
SLAC (Lee et al., 2020)
X1, 2:-1
X441:T
3 PARTIAL CONDITIONING OF INFERENCE NETWORKS
VAEs add an additional consideration to approximate posterior design compared to classical varia- tional inference: the choice of inputs to the inference networks. Instead of feeding the entire obser- vation one can choose to feed strict subset C C x. This is common practice in many sequential variants of the VAE, cf. table typically motivated by efficiency or downstream applications. The discrepancy between inference models inputs and the true posterior conditions has been acknowl- edged before, e. g. by Krishnan et al. (2017), where they investigate different variants. Their analysis does not go beyond quantitative empirical comparisons. In the following. we show that such design choices lead to a distinct third cause of inference suboptimality, the conditioning gap.
3.1 MINIMISING EXPECTED KI. DIVERGENCES YIELDS PRODUCTS OF POSTERIORS
The root cause is that all observations x that share the same included conditions C' but differ in the excluded conditions must now share the same approximate posterior by design. This shared approximate posterior optimizes the expected KI. divergence
arg min E (KL( () p(z C,C))].
(5)
w. t. the missing conditions C C. By rearranging (ef. appendix A.1 for details)
E (ss C) p(z C,T))]
KL (5(z C* exp [Lae [log[(z (,,))) log Z,
where Z denotes normalising constant, that the shared optimal approximate posterior is
w(z) exp [Bej [log((z C,C).
Interestingly, this expression bears superficial similarity with the true partially-oonditione posterior
p(z C) = [p(z C, C,C)]
= exp (log [ [p(z C.C)))).
To understand the difference, consider uniform discrete C C' for the sake of the argument. In this case, the expectation is an average over all missing conditions. The true posterior p(z C') is then a mixture distribution of all plausible full posteriors p(z C,C). The optimal approximate posterior, because of the logarithm inside the sum, a product of plausible full posteriors. Such distributions occur in products of experts (Hinton, 2002) Gaussian sensor fusion (Murphy, 2012). Products behave differently from mixtures: an intuition due to Welling (2007) is that factor in product can magle-handedly "veto" sample, while each term in mixture only "pass" it.
This intuition is highlighted in fig. We can see that w is located between the modes and sharply peaked. It shares almost mass with both the posteriors and the marginal posterior. The approxi- mate marginal posterior // on the other hand either covers one or two modes, depending on the width of the two full posteriors-a much more reasonable approximation.
4

---

Published as a conference paper at ICLR 2021
In fact, (z) will only coincide with either p(z C) p(z C,C) in the the extreme when p(z () p(z C,C) p(z C)p(C C) p(z, C C). cf. appendix A.2, which implies
statistical independence of and z given C.
Notice how the suboptimality of w assumed neither particular variational family, nor imperfect amortisation-44 w is the analytically optimal shared posterior. Both approximation and amortisation gap would only additionally affect the KL divergence inside the expectation in eq. (5). As such, the inference suboptimality described here can occur even if those vanish. We call
Ec E (w C) p(z C,C)]]]
the conditioning gap, which in contrast approximation and amortisation gap can only be defined as an expectation W. r. p(x) = p(C, C).
Notice further that the effects described here assumed the KL divergence, a natural choice due its duality with the ELBO (ef. eq. (2)). To what extent alternative divergences (Ranganath et al., 2014; Li & Turner, 2016) exhibit similar or more favourable behaviour left future research.
Posteriors We find that the opti- mal partially-condttioned posterior will not correspond to desirable posteriors such as p(z C*) or p(z C, C) even assuming that the variational family could represent them. This also affects learn- ing the generative model. A simple variational calculus argument (appendix A.3) reveals that, when trained with partially-conditions approximate posteriors, maximum -ikelliiooo models and ELBO- optimal models are generally not the same. fact, they coincide only in the restricted cases where p(z C, q(z C). As seen before this is the case if and only if C 1 z C.
Learning Generative Models with
3.2 LEARNING A UNIVARIATE GAUSSIAN
is worth highlighting the results of this section in minimal scalar linear Gaussian example. The target distribution is p(x) ~ (00, 1). We assume the latent variable model
Pa (z. 2) NNxx: a.2 A(2 10,1)
with free parameter 2 This implies
Pa = (x 0.0.1 + +a²),
(=(21))=N( a(0.1 -22 (1+10a²) -
With a* = /0.9, we recover the target distribution with posterior Pa- (z (x) = [x v0.9r, 0.1). Next, we introduce the variational approximation q(2) = N( M2.o?). With the only condition I missing, this is a deliberately extreme case of partial conditioning where = and 7 = {z) Notice that the true posterior member of the variational family, i.e., approximation gap. We maximise the expected ELBO
Pa (I,
(2)2) = arg max EnnE
Ezwg log
((2)
i..., all observations from share the same approximation g. One can show that
Wa (z) = N/ 0, (100a² + +1)=t).
We immediately see that wa( (2) is neither equal to Pa- Pa-(2 xx) = p(> C,C) nor p(2) = p(= C). Less obvious is that the variance of the optimal solution is much smaller than that of both p(= C,C) and p( | C). This a consequence of the product of the expert compromise discussed earlier: the (renormalised) product of Gaussian densities will always have lower variance than either of the factors. This simple example highlights how poor shared posterior approximations can become.
Further, inserting Wa back into the expected ELBO and optimising for reveals that the maximum likelihood model parameter a" is not optimal. In other words, Pa- (zz, 2) does not optimise the expected EL ELBO in desppitte being the maximum likelihood model.
5

---

Published as a conference paper at ICLR 2021
3.3 EXTENSION TO THE SEQUENTIAL CASE
The conditioning gap may arise in each of the terms of eq. (4) if is partially conditioned. Yet, it is common to leave out future observations, c. q(z Zi.t 1, 1, X++17). To the best of our knowledge, in the literature only sequential applications of VAEs are potentially affected by the con- ditioning gap. Still, sequential latenl-variable models with amortised under-conditioned posteriors have been applied success fully to, e. g., density estimation, anomaly detection, and sequential deci- sion making (Bayer & Osendorfer, 2014; Soelch et al., 2016; Hafner et al., 2019). This may seem at odds with the previous results. The gap is not severe in all cases. Let us emphasize two "safe" cases where the gap vanishes because C, 1 Ze C is approximately true. First, where the partially- and the fully- conditione posterior correspond to the proor transition, e.
p Ue
1 ~ p(z, C.) Rs p( | Ce, C.).
This is for example the case for deterministic dynamics where the transition is single point mass. Second, the case where the observations are sufficient to explain the latent state, i e.
p(zz
X
= p Z Ci) = p(zz C. C.).
A common case are systems with perfect state information.
Several studies have shown that performance gains of fully-conditioned over partially-conditionel approaches are negligible (Fraccaro et al., 2016; Maddison al., 2017; Buesing al., 2018). We conjecture that the mentioned "safe" cases are overrepresented in the studied data sets. For example, environments for reinforcement learning such the gym or MuJoCo environments (Todorov al., 2012; Brockman et al., 2016) feature deterministic dynamics. We will address broader variety of cases in section 5.
RELATED WORK
The bias of the ELBO has been studied numerous times (Nowozin, 2018; Huang & Courville, 2019). A remarkable study by Turner & Sahani (2011) contains a series of carefully designed experiments showing how different forms of assumptions on the approximate posterior let different qualities of the solutions emerge. This study predates the introduction of amortised inference (Kingma & Welling, 2014; Rezende et al., 2014) however. Cremer et al. (2018) identify two gaps arising in the context of VAEs: the approximation gap; e.e. inference error resulting from the true posterior not being member of variational family; and the amortisation gap, e. imperfect inference due to an amortised inference procedure, c. g. a single neural network call. Both gaps occur per sample and are independent of the problems in section 3.
Applying stochastic gradient variational Bayes (Kingma & Welling, 2014; Rezende et al., 2014) to sequential models was pioneered by Bayer & Osendorfer (2014); Fabius et al. (2015); Chung et al. (2015). The inclusion state-space assumptions was henceforth demonstrated by Archer al. (2015); Krishnan al. (2015; 2017); Karl al. (2017a); Fraccaro et al. (2016; 2017); Becker- Ehmek et al. (2019). Specific applications to various domains followed, such as video prediction (Denton & Fergus, 2018), tracking (Kosiorek al., 2018; Hsich et al., 2018; Akhundov et al., 2019) or simultaneous localisation and mapping (Mirchev et al., 2019). Notable performance in model-based sequential decision making has also been achieved by Gregor et al. (2019); Hafner et al. (2019); Lee et al. (2020); Karl et al. (2017b); Bocker-Ehmcl et al. (2020). The overall benefit of latent sequential variables was however questioned by Lai et al. (2019). The performance of sequential latent-variabl
models in comparison to auto-regressive models was studied, arriving at the conclusion that the added stochasticity does not increase, but even decreases performance. We want to point out that their empirical study is restricted to the setting where 2, 1 Z1:t-1 1 X1:7, the appendix of their work. We conjecture that such assumptions lead to a collapse of the model where the latent variables merely help explain the data local in time, i. e. intra-step correlations.
related field is that of missing-value imputation. Here, tasks have increasingly been solved with probabilistic models (Ledig et al., 2017; Ivanov et al., 2019). The difference explicit nature of the missing conditions. Missing conditions are typically directly considered by learning CC C) instead of p(x) and appropriate changes to the loss functions.
work is the

---

Published as a conference paper at ICLR 2021
LA
A MAW
(a) X-axis
(b) y-axis
Figure 2: Posterior-predictive check of prefix-samplin on the Blackbird data set. Possible futures are sampled from the model after having observed a prefix X1:- The state the end of the pre- fix is inferred with bootstrap particle filter. Each plot shows kernel density estimate of the distribution over the final location XT, once for the semi-conditioned model in green and for the fully-conditioned in blue. The true value is marked as vertical, black line. The fully-conditioned model assigns higher likelihood almost all cases and is more concentrated around the truth.
5 STUDY: VARIATIONAL STATE- SPACE MODELS
We study the implications of section 3 for the case of variational state-space models (VSSMs). We deliberately avoid the cases of deterministic dynamics and systems with perfect state information: unmanned aerial vehicle (UAV) trajectories with imperfect state information in section 5.2, a se- quential version of the MNIST data set section 5.3 and traffic flow in section 5.4. The reason is that the gap is absent in these cases, see section 3.3. In all cases, we not only report the ELBO, but conduct qualitative evaluations well. We start out with describing the employed model in section 5.1.
5.1 VARIATIONAL STATE-SPACE MODELS
The two Markov assumptions that every observation only depends on the current state and every state only depends the previous state and condition lead to the following generative model:
T
p(X1.T Zi:T (p:r)
II p(x, zz)p(z; Zz-1; u,11), 1=1
where u1.7 are additional conditions from the data set (such as control signals) and 2o = un := 0. We consider three different conditionings of the inference model
q X1:7, uprr)
q(z) X1:k u1sk) II g(z 1, U1m)):
partial with 1, m = I,, semi with l: > 1, m t, full with m We call the sneak peek, inspired by Karl et al. (2017a). See appendix B for more details.
5.2 UAV TRAIECTORIES FROM THE BLACKBIRD DATA SET
We apply semi- and fully- colyyioondi VSSMs to UAV modelling (Antonini et al., 2018). By discard- ing the rotational information, Each observation x, E R3 is the location of an unmanned aerial vehicle (UAV) in a fixed global frame. The conditions u, E 14 consist of IMU readings, rotor speeds, and pulse-width modula- tion. The emission model was implemented as a Gaussian with fixed, hand-picked standard devia- tions, where the mean corresponds the first three state dimensions (cf. (Akhundov et al., 2019)): p(x; z,) N(u Z1,1:3, ² [0.15, 0.15, 0.075)). We leave out the srrrially-condtiioned case, as it cannot infer the higher-ords order derivatives necessary for rigid-body dynamics. sneak peek of k = 7 for the semi-conditioned model is theoretically sufficient to infer those moments. See appendix C for details.
create system with imperfect state information, ef. section 3.3.
Fully-conditioned models outperform semi-conditioned ones on the test set ELBO, as can be seen table 2a. We evaluated the models on prefix-sampling, i.e. the predictive performance of 1(X X1a,, u. To restrict the analysis to the found parameters of the generative model only, we inferred the filter distribution p(z, X1:t, U1:s) using a bootstrap particle filter (Sarkka, 2013). By not using the respective approximate posteriors, els. Samples from the predictive distribution were obtained via ancestral sampling of the generative model. Representative samples shown in fig. 4. We performed costerior-predictiv check for
ensure faimeess between the different mod-

---

partial Published as a conference paper at ICLR 2021
Table Results on UAV and row-wise MNIST modelling.
(a) ELBO values for models with differently con- ditioned variational posteriors for various data Presented values are averages over ten samples from the inference model. The standard deviations were negligible. Higher is better.
(b) Results for row-wise MNIST. We report the ELBO lower bound the log log-likelihood and KL-Divergence
digit distribution induced by the model from a uniform distribution. Other re- sults by Klushyn et al. (2019).
UAV
Traffic Flow
Distribution
Log-Likelihood t
KL J
val
test
val
test
data
0.002
partial
-299
-2.97
98.99 + 0.06
0.098
semi
1.47
2.13
-.73
-2.75
full -8.45 0.05
0.015
full
2.03
2.41
-2699
-2.78
vhp
-82.74
iwae (L=2)
-82.3
3
8 X
6
9 8 8 15
6
7
4
7
0 6 8 4
2 2 9
4
10l Freeiency
3
4
8
8 8 9 6
8 * A
0
6 7
0.1
Oye
5
2
3
4
4
6
7 6 0
data
parcally
8
1
5
5
B
3
5
0.0
9
4
6
2 4
7 4
0
5
dight
data
partially-cond.
fully-cond.
Figure 3: Left: Class distributions of the respective image distributions induced by a state-of-the-art classifier. The data distribution is close uniform, except for 5. The fully-conditioned model yields too few 5's and is close to uniform for the other digits. The partially-oonditioned model only captures the right frequencies of and 3. Right: Comparison of generative sampling on row-wise MNIST. Samples from the data distribution are shown on the left. The middle and right show samples from models with an partially- and fully-conditioned approximate posterior, respectively.
both models, where we compare the densities of the final observations obtained from prefix sam- pling in fig. Both evaluations qualitatively illustrate that the predictions of the fully-conditioned approach concentrate more around the true values. In particular the partially-conditionn model struggles more with long-term prediction.
5.3 Row- WISE MNIST
We transformed the MNIST data set into a sequential data set by considering one row in the image plane per time step, from top to bottom. This results in stochastic dynamics: similiar initial rows can result in 3, 8, 9 or 0, future rows are very informative (cf. section 3.3). Before all experi- ments, each pixel was binarised by sampling from a Bernoulli with rate in [0. proportional to the corresponding pixel intensity.
The setup was identical that of section 5.2, except that Bernoulli likelihood parameterised by a feed-forward neural network was used: p(x; 2,) B(FNN,, (z.)). No conditions 11rr and short sneak-peek (k 1) were used. The fully-conditioned model outperforms the partially- conditioned by a large margin, placing it significantly closer to state-of-the-art performance, table 2b. This is supported by samples from the model, see fig. 3. Cf. appendix D for details.
For qualitative evaluation, we used state-of-the-ard classifier! to categorise 10,000 samples from each of the models. the data distribution is learned well, we expect the classifier to behave sim- ilarly both data and generative samples, i.c. yield uniformly distributed class predictions. We report KI KL-divergences from the uniform distribution of the class prediction distributions in table 2b. A bar plot of the induced class distributions can be found in fig. 3. Only the fully-conditioned model able to nearly capture a uniform distribution.
httpa//aaa///////oocoo//////er/e ob/2. .4.0 2.2p0/exxaapp. es /mn st_cnn.pt
8

---

Published as a conference paper at ICLR 2021
UAV
Traffic Flow
partial
V
V
V
semi
V
full
Figure 4: Comparison of prefix-sampling. Possible futures 8 (coloured lines) are sampled from the model after having observed prefix X1:k (solid black line) and then compared to the true suffix x;+1:T7 (dashed line). The state the end of the prefix is inferred with a bootstrap particle filter. Left: UAV data, top-down view. Right: Traffic flow data, time-series view.
5.4 TRAFFIC FLOW
We consider the Seattle loop data set (Cui al., 2019; 2020) of average speed measurements of cars at 323 locations on motorways near Seattle, from which we selected a single one (42), The dynamics of this system are highly stochastic, which is of special interest for our study, ef. sec- tion 3.3. Even though all days start out very similar, traffic jams can emerge suddenly. In this scenario the emission model was feed-forward network conditioned on the whole latent state, i.e. p(x; 22) = .V (F o2) We compare partially-, semi- (A: 7) and fully-conditioned models. See appendix E for details. The results are shown in table 2a. While the fully-conditioned posterior emerges the best choice conditioned one are on par on the test set. We suspect that the sneak peek is sufficient to fully capture sensible initial state approximation.
the validation set, the semi-conditioned and the fully-
We performed a qualitative evaluation this scenario as well the form of prefix sampling. Given the first & 12 observations, the task is predict the remaining ones for the day, compare sec- tion 5.2. We show the results in fig. 4. The culltcconditioned model clearly shows more concentrated yet multi-modal predictive distributions. The partially-condition model concentrates too much, and the semi-conditioned one too little.
DISCUSSION & CONCLUSION
We studied the effect of leaving out conditions of amortised posteriors, presenting strong theoretical findings and empirical evidence that can impair maximum likelihood solutions and inference. Our work helps with approximate posterior design and provides intuitions for when full conditioning necessary. We advocate that condially-ondditooed approximate inference models should be used with caution in downstream tasks as they may not be adequate for the task, g., replacing Bayesian filter. general, we recommend conditioning inference models conforming to the true posterior.
REFERENCES
Adnan Akhundov, Maximilian Soelch, Justin Bayer, and Patrick van der Smagt. Variational Tracking and Prediction with Generative Disentangled State-Space Models. arXiv:1910.0622 06200 [cs, stat), October 2019. URL http: /arx/v..vvv/../////a www.riii..rr./oeeeooooeoe
910 06205.
Amado Antonini, Winter Guerra, Varun Murali, Thomas MoCre, and Sertac Karaman. The blackbird dataset: A large-scale dataset for UAV perception in aggressive flight. In Jing Xiao, Torsten Kröger, and Oussama Khatib (eds.), Proceedings of the 2018 International Symposium Experimental Robotics, ISER 2018. Buenos Aires, Argentina, November 5-8, 2018, volume 11 of Springer Proceedings in Advanced Robotics, pp. 130-139, Springer, 2018. doi: 10.10977978 3-030-33950-0) 12. URL https /toi org 10001110000000000---0003---00000
12.

---

Published as a conference paper at ICLR 2021
Evan Archer, Il Memming Park, Lars Buesing, John Cunningham, and Liam Paninski. Black box variational inference for state space models. arXiv:1511.07367 [stat], November 2015. URL http://arxix orroor///////// abs /tt./1111.000 073677.
Justin Bayer and Christian Osendorfer: Learning Stochastic Recurrent Networks. arXiv:1411.7611 [cs, stat], November 2014. URL htt.::ttt///aereeeee
wwwp/ogg///////////////////oe..e ttbs/1411. 761..
Matthew J. Beal. Variational Algorithms for Approximate Bayesian Inference. PhD thesis, Univer- sity College London, UK, 2003. URL http:////atttssaa../////orrrraaa do?uin= uk. bl. ethos. 404387.
Philip Becker-Fhmck, Jan Peters, and Patrick van der Smagt. Switching Linear Dynamics for Variational Bayes Filtering. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceed- ings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 553-562, Long Beach, California, USA, 2019. PMLR. URL http:: http:://pcceeedeeg.eeeee www.mns...........mm.........wwe mlr. press / v http//eccerehhcke html.
Philip Becker-Ehmck, Maximilian Karl, Jan Peters, and Patrick van der Smagt. Learning to Fly via Deep Model-Based Reinforcement Learning. arXiv:2003.0888 [cs, stat], August 2020. URL
/ttp:///aaavv..// org/ abs 08876.
Léon Bottou. Large-scale machine learning with stochastic gradient descent. In Yves Lechevallier and Gilbert Saporta (eds.). 19th International Conference on Computational Statistics, COMP- Keynote. Invited and Contributed Papers, 2010. doi: 10. 10.1097/97- 2600-3)) 16. URL https: pp. 177-186. 1110010000------01 //dei.org//1/
STAT 2010, Paris, France, August 22-27, 2010
Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojeiech Zaremba. OpenAl Gym. arXiv:1606.01540 [cs], June 2016. URL http: /arxiv. org/abs/ /1666. 01540.
Lars Buesing, Theophane Weber, Sebastien Racaniere, S. M. Ali Eslami, Danilo Rezende, David P. Reichert, Fabio Viola, Frederic Besse, Karol Gregor, Demis Hassabis, and Daan Wierstra. Learn- ing and Querying Fast Generative Models for Reinforcement Learning. arXiv:1802.0300 [cs], February 2018. URL http::ttt//. /ttp///axvvvv/vaa//ea//eeeeaaaaa http//b////000 03006.
Yuri Burda, Roger B. Grosse, and Ruslan Salakhutdinov. Importance weighted autoencoders. In Yoshua Bengio and Yann LeCun (eds.). 4th International Conference on Learning Representa- tions, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings, 2016.
htlLtt/p//ee /ttp///axvvvvv/a org/ abs /000s/159..000 00519.
Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron C. Courville, and Yoshua Bengio. A Recurrent Latent Variable Model for Sequential Data. In Corinna Cortes, Neil D. Lawrence, Daniel D. Lee, Masashi Sugiyama, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 28: Annual Conference Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada, pp. 2980- 2988, 2015. URL http: /twppapp.ss... nips cc variable maiiiableemdelll-- for sequential daata
Thomas M. Cover and Joy A. Thomas. Elements of Information Theory. Wiley Series in Telecom- munications and Signal Processing. Wiley- Intersentrse,e,ee, USA, 2006. ISBN 978-0-471-24195-9,
Chris Cremer, Xuechen Li, and David Duvenaud. Inference suboptimality in variational autoen- coders. In Jennifer G. Dy and Andreas Krause (eds.), Proceedings of the 35th International Con- ference on Machine Learning, ICML 2018, Stockholmsmüssan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pp. 1086-1094 PMLR, 2018. URL
mlr.. pprss//v0) 1110//reence cremer remerrlaa... .tmml.
Zhiyong Cui, Ruimin Ke, Ziyuan Pu, and Yinhai Wang.
Deep Bidirectional and Unidirectional LSTM Recurrent Neural Network for Network-wide Traffic Speed Prediction. arXiv:1801.02143 fes), November 2019. URL http: //txx//axi////////////aa///e/aaa org wtwg/bs/////oo 1801 801 02143.
10

---

Published as a conference paper at ICLR 2021
Zhiyong Cui, Kristian Henrickson, Ruimin Ke, and Yinhai Wang. Traffic graph convolutional recur- rent neural network: A deep learning framework for setwork-scale traffic learning and forecast- ing. IEEE Transactions Intelligent Transportation Systems, 21 1:4483-48944 2020. doi: 10. 1109/TTTS 2019, URL https: / doi. org/ 111111T/TSS./ 201 9. 295041
Emily Denton and Rob Fergus. Stochastic video generation with learned prior. In Jennifer G.
Dy and Andreas Krause (eds.). Proceedings of the 35th International Conference Machine Learning. ICML 2018, Stockholmsmåssan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pp. 1182-1191. PMIR, 2018. URL http: www.presss/eee //80///80) dentonn 8a. html.
/ //proc/eee/ igg.eedddiiidd....oo ngg. w.w...rr.r..rr.....m.mmm..m.m...
Tim van Erven and Peter Harremoës. Rényi divergence and kullback-leibler divergence. IEEE Transactions Information Theory, (0(7):3777-382) 2014. doi: 10.109/TTT.31422225000 URL https: / /ooi.ii..//// wrg/1o..gooooooooooooooooo...../ 110...119/TTT 2014. .2200500.
Otto Fabius, Joost R. van Amersfoort, and Diederik P. Kingma. Variational recurrent auto-encoders. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference Learning Represen- tations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Workshop Track Proceedings, 2015. URL http: / arxiv. org/ abs /hbs/111... 6581.
Marco Fraccaro, Søren Kaae Sønderby, UIrich Paquet, and Ole Winther. Sequential neural mod- cls with stochastic layers. In Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 29; Anmual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pp. 2199 2207, 2016. URL https: http:////ccceeee........... www.ipuu.p.... ec/ hat.e//eeee.eeeccccccccaaaaaaaaa Abstaaataaae.etteettattaa.a..
Abstract. .tmll
Marco Fraccaro, Simon Kamronn, Ulrich Paquet, and Ole Winther. A disentangled recognition and nonlinear dynamics model for unsupervised learning. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. N. Vishwanathan, and Roman Gamett (eds.), Advances in Neural Information Processing Systems 30: Annual Conference on Neu- ral Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp- 3601-3610, 2017. URL https:ttp//oomc http::/:/pppcceednenneee nwur.n.s...eeeeeeeeeeeneeeeeeene neurips ips pccp/aaper//11//haeeeeeeeee http//2227/hhh///// 7b7a53e23 9 a0 3wwe33eeeeeeeeeeeeeeeee 0377 aat.eee--e...cceee--bteetta html.
I. M. Gelfand and S. V. Fomin. Calculus of Variations. Dover Publications Inc., Mineola, N.Y, March 2003. ISBN 978-0-486-41448-5.
Karol Gregor, George Papamakarios, Frederic Besse, Lars Buesing, and Theophane Weber. Tem- poral difference variational auto-encoder. In 7th International Conference on Learning Repre- sentations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019. URL htttss://pperree .nett net./ forum?
i 1edid=1xx4000 1==11x4gh9t
Danijar Hafner, Timothy P. Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, and James Davidson. Learning latent dynamics for planning from pixels. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learming, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pp. 2555-2565. PMLR, 2019. URL http: ppwpeeddng..... mlr. A.l.ppress/vl v hafnerllfleeeeeeceeamraannnaeeln 9a. html.
G. E. Hinton, P. Dayan, B. J. Frey, and R. M. Neal. The "wake-sleep" algorithm for unsupervised neural networks. Science, 268(5214):1158-111 May 1995. ISSN 0036-8075, 1095-9203. doi: 10. 10.1126//ciencc..7668888 5214/1158.
htRpst::: htt///sccencceeee wcwencemag org/ httpr/////ootet////////
Geoffrey E. Hinton.
Training Products of Experts by Minimizing Contrastive Divergence.
Neural Computation, 14(8): August 2002. ISSN 0899-76677. doi: 16.1
08997660276011888188 URL httsttps: http:////wwwwwmttteeeetee wttp::/itpresssorraaaaaa journal oow.://ooooooo/0////o//////////.
Matthew D. Hoffman, David M. Blei, Chong Wang, and John Paisley. Stochastic Variational Infer- ence. Journal of Machine Learning Research, 13444773,,,3 2013. ISSN 1533-7928, URL http://jml www.:oogrrrrr.e.......e.. ra /of fman1 3a. html.

---

Published as a conference paper at ICLR 2021
Jun-Ting Hsich, Bingbin Liu, De-An Huang, Fei-Fei Li, and Juan Carlos Niebles. Learning to decompose and disentangle representations for video prediction. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Fesa-Bianchi, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 31: Annual Conference on Neural In- formation Processing Systems 2018, NeurlPS 2018, December 3-8, 2018, Montréal, Canada, pp. 515-524, 2018. URL https: wwtppppeeiig...nnee......eeeeeee wwtp:nnwwss...pppeeeeeeeeeeeee cwtpaper/2c1888 0/a0//h///h////// 4 196605elaea0a9c4 654800558000000 7b9ea28 Abstract html.
Chin-Wei Huang and Aaron Courville. Note on the bias and variance of variational inference. (aoo::906600000 [cs, stat], June 2019. URL htlttt/////xiii orgoorrggaaoooo /atg//ab///aa 906. 03708.
Oleg Ivanov, Michael Figunovv and Dmitry P. Vetrov. Variational autoencoder with arbitrary ditioning. In 7th International Conference on Learning Representations, ICLR 2019, New Or- leans. LA, USA, May 6-9, 2019. OpenReview.net, 2019. URL forum? i d=Syxt.JhQqmm
Michael I. Jordan, Zoubin Ghahramani, Tommi S. Jaakkola, and Lawrence K. Saul, An introduction to variational methods for graphical models. Machine Learning, 37(2):183-233, 1999. doi: 10. 10.1003/A4100066590777788 URL https: http:://ooo..../.../////o 1023/22100776597777777
Maximilian Karl, Maximilian Soelch, Justin Bayer, and Patrick van der Smagt. Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data. In 5th Interna- fional Conference on Learning Representations, ICLR 2017, Toulon, Frunce, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017a. URL httttt:://pperreiiieeeee Forum?id-Hy's HygL5....
Maximilian Karl, Maximilian Soelch, Philip Becker-Ehmek, Djalel Benbouzid, Patrick der Smagt, and Justin Bayer. Unsupervised Real-Time Control through Variational Empowerment. 110:01110005101 [stat], October 2017b. URL /attp://arxaae org// abs aww/aaa///// httpg//bb//11..... 01100.0100.......
Diederik P. Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL http: //arxiv.orge abs / 41 412.44224444 6980.
Diederik P. Kingma and Max Welling. Auto-Encoding Variational Bayes In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, 2014. URL.http: / /aaxivvvv.. wrg. abggabs/////eoo 1312. 6114.
Diederik P. Kingma, Tim Salimans, Rafal Józefowicz, Xi Chen, Ilya Sutskever, and Max Welling, Improving variational autoencoders with inverse autoregressive flow. In Daniel D. Lee. Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 29: Annual Conference on Neu- ral Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pp- 4736 4744, 2016. URI. https: http:::sss/pocoeeeddee mew.eeeedddnnn..eee..eeeeeeeeeee neurips ps. cc wwtpccpp/////e/eeee paperr201e 20011/haass /hash/ Aat.sseeeeeeeeeet--stttaaateette .tmll
Alexej Klushyn, Nutan Chen, Richard Kurle, Botond Cseke, and Patrick van der Smagt. Learn- ing Hierarchical Priors in VAEs. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelz- imer, Florence d' A'Ahh-BBB,, Emily B. Fox, and Roman Garnett (eds.), Advances in Newral In- formation Processing Systems 32: Annual Conference on Neural Information Processing Sys- tems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 2866-2875, 2019. URL http:/ htap:://paprrs...ee wips. cc waasscccppppeee wwcpppper//555 httpe///SSS---eeamnngggeeeeeee hien-hhhirracccla priors-in-vaes.
Adam R. Kosiorek, Hyunjik Kim, Yee Whye Teh, and Ingmar Posner. Sequential attend, in- fer, repeat: Generative modelling of moving objects. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett (eds.), Ad- vances in Neural Information Processing Systems 31: Annual Conference on Neural Informa- tion Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pp- 8615-8625, 2018 URL htttps://proceeedingg hew.reeeeeddinnnnnnin..neeses..s nww.ips.ccccc.oo. wwtparipss.cpppppeeaeaeee httppappr//211/ 8/hash/ 7417744 744177442bce77ffa555002222222222 Abst abst html.
12

---

Published as a conference paper at ICLR 2021
Rahul G. Krishnan, Uri Shalit, and David Sontag. Deep Kalman Filters. arXiv:/511.0512) [es, stat], November 2015. URL http: pettp://azxee wttpaawwww.......aaaaoooo.ooooot abs 1111.15112..1
Rahul G. Krishnan, Uri Shalit, and David A. Sontag. Structured inference networks for nonlin- state space models. In Satinder P. Singh and Shaul Markovitch (eds.), Proceedings of the Thirty-Firss AAAI Conference on Artificial Intelligence, February 4-9, 2017, San Francisco, Cal- isornia, USA, pp. 2101-2109, AAAI Press, 2017. URL http:/tt//// /aaaii//aaaa/.e org/ocs////ooooooooo..ooooo..ooo iinexx.. php/AAAT/ CAAAT/AAATIO AAATIIAAI 7 http///ppppe/e////epeeee
Guokun Lai, Zihang Dai, Yiming Yang. and Shinjae Yoo. Re-examination of the role of la- tent variables in sequence modeling. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelz- imer, Florence d' Alché-Buc, Emily B. Fox, and Roman Garnett (eds.), Advances in Neu- ral Information Processing Systems 32: Annual Conference on Neural Information Pro- cessing Systems 2019, NewrIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp- 7812-7822, 2019. URL https: http:///poceedee/eeeeeeeee neuri iss. wttpppuwpps..ppppppeeeeepep htc/papprr/219/haatt d0ac
Abst html.
Christian Ledig, Lucas Theis, Ferenc Huszar, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, and Wenzhe Shi. Photo- realistic single image super-resolution using generative adversarial network. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 20/7, pp- 105-114. IEEE Computer Society, 2017. https:: htt//di..o/// www.dd........oee 1109 1ttp:///...////PP2222000 CCV...201..1.....
doi: D..109/CVPR.2017.1199 URL
Alex X. Lee, Anusha Nagabandi, Pieter Abbeel, and Sergey Levine. Stochastic la- tent actor-critic: Deep reinforcement learning with latent variable model. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, Virtual, 2020. URL https: http:::://ppocceedggg.ssee wttpeuriissccppppppeeeeeeeeeeeee htcp/paper/222///eeee html.
Yingzhen Li and Richard E. Turner. Rényi divergence variational inference. In Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg. Isabelle Guyon, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 29; Annual Conference Neural In- formation Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pp. 1073- 1081, 2016. URL http://papers cttips.cc/aapr//550 6208 1es88-renii--iee renyi-divergence variat connalinnfeennc
David J. C. MacKay. Information Theory, Inference, and Learning Algorithms. Cambridge Univer- sity Press, 2003. ISBN 978-0-521-64298.9
Chris J. Maddison, Dieterich Lawson, George Tucker, Nicolas Heess, Mohammad Norouzi, Andriy Mnih, Arnaud Doucet, and Yee Whye Teh. Filtering variational objectives. Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 30: Annual Confer- ence on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp. 6573 6583, 2017. URL htwpt:tts//ccc http:::ee/////ceeeieeeecee (a//://22222222222222 htt.://44.....221ccccccceae3333 wtw.:aatae....taatttttt......... wbw.taatt...................aa.. html.
.ww.riinnnnnnnnnnnn.oeeeom newrips...... cc/ htcpppppe/201/
Lars M. Mescheder, Sebastian Nowozin, and Andreas Geiger. Adversarial variational bayes: Unify- ing variational autoencoders and generative adversarial networks. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning. ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Re- search, pp. 2391-2400. PMLR, 2017. URL http: / wpwppeeedngga.... ml htt.ggg..........s....ee press/v70/ mescheder17 html.
Atanas Mirchev, Baris Kayalibay, Maximilian Soelch, Patrick van der Smagt, and Justin Bayer. Approximate bayesian inference in spatial environments. In Antonio Bicchi, Hadas Kress- Gazit, and Seth Hutchinson (eds.), Robotics: Science and Systems XV, University of Freiburg, Freiburg Im Breisgau. Germany, June 22-26, 2019, 2019. doi: N0.15607/RSS.2019.X. XV. 083.
URL https:
/ //doi. org/10. 15607 RSS 201
13

---

Published as a conference paper at ICLR 2021
Andriy Mnih and Danilo Jimenez Rezende. Variational inference for monte carlo objectives. In Maria-Florina Florina Balcan and Kilian Q. Weinberger (eds.), Proceedings of the 33nd International Conference Machine Learning. ICML 2016. New York City, NY, USA, June 19-24, 2016, vol- ume 48 of JMLR Workshop and Conference Proceedings, pp. 2188-2196. JMI.R.rrg 2016. URL pett::/prrceest ngs .m pressss / v48/ hb1 6 html.
Kevin P. Murphy. Machine Learning: A Probabilistic Perspective. Adaptive Computation and Machine Learning Series. The MIT Press, 2012. ISBN 978-0-262-01802-9.
Sebastian Nowozin. Debiasing evidence approximations: On importance-weighted weighted autoencoders and jackknife variational inference. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018. URL https:ttps.. http///pppeeeeeeeeeeeeeeeeeeeeet net/ foreet//ooomm//eee feeerrmmmi-Hyeoeoo Sechyyoi--Bbbbb WRb.
Rajesh Ranganath, Sean Gerrish, and David M. Blei. Black box variational inference. Proceed- ings of the Seventeenth International Conference on Artificial Intelligence and Statistics, AISTATS 2014, Reykjavik, Iceland, April 22-25, 2014, volume of JMLR Workshop and Conference Pro- ceedings, pp. 814-822. JMLR. R.org 2014. URL http: http///ooccedieec ngs. ml ranganathle html.
wtt..........s////.... press//333/
Danilo Jimenez Rezende and Shakir Mohamed. Variational inference with normalizing flows. In Francis R. Bach and David M. Blei (eds.), Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015, volume 37 of JMLR Workshop and Conference Proceedings, 1530-1538. JMLR.org, 2015. URL http: http:///pocceeeeeeeeeeeee
mmr. S..L.ppress/v33rreeeedee html.
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and approximate inference in deep generative models. In Proceedings of the 31th International Conference Machine Learning, ICML 2014. Beijing, China. 21-26 June 2014, volume 32 of JMLR Workshop and Conference Proceedings, pp. 1278-1286. JMLR.org, 2014. URL hict::/proceed:: ngs mm wea.gs.m...pres. wal.r.pres//v330 http://33//eeee///
html.
Simo Sarkka. Bayesian Filtering and Smoothing. Cambridge University Press, Cambridge, URL http: //books..
2013. ISBN 978-1-139-34420-3. doi:
cambri ccmmbrrr.gg...orreeee////////o11
Mike Schuster and Kuldip K. Paliwal. Bidirectional recurrent neural networks. IEEE Transactions Signal Processsing, 45(11):2673 2662226611,,,,, 1997. doi: 10.1 10.1109/78.65099 URLhttps://dol.
htg//00.119// 650093.
Maximilian Soelch, Justin Bayer, Marvin Ludersdorfer, and Patrick van der Smagt. Variational Inference for On-line Anomaly Detection in High- Digen-Dimaiional Time Series. arXiv:1602.07109 [cs, stat), February 2016. URL http: /tap:iv////vvv..//aa wwwaaaiiwrggaeeeeeeeeeeeeeoooeee harg//bbs. /oobs/////..... 07109.
Emanuel Todorov, Tom Erez, and Yuval Tassa. MuJoCo: A physics engine for model-based con- trol. 2012 JEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2012, Vilamoura, Algarve, Portugal, October 7-12, 2012, pp. 5026-5033. IEEE, 2012. doi: 10.1 109/ IROS.2013 2012.63861099 URL https::tp///......t.. doii .rg 0. 09 www.0////RRO... 2012. 6386109.
Richard Eric Turner and Maneesh Sahani. Two problems with variational expectation max- imisation for time series models. In A. Taylan Cemgil, David Barber, and Silvia Chi- appa (eds.), Bayesian Time Series Models, pp. 104-124. Cambridge University Press, Cambridge, 2011. ISBN 978-0-521--1676-55 doi: 10110770B8097805111944790000
URL http://///////////o///ooo.or/oon simese---esseeeseeeeeeeeeeeeeee series mageaaace.p..acee.eeeaaaaaaaaaae wtt.://iiieeiieeeeee//////eeeee.
Max Welling. Product of experts.
Scholarpedia, 2(10):3879, October 2007.
2007. ISSN 1941-6016.
doi: 1044299/cholarpecii..777
URL http: http:://ww..cbbllarraaaaaaorrrr
Product of srrcuutt_oo_epperss....
14

---

Published as a conference paper at ICLR 2021
A DETAILED DERIVATIONS
A.1 DERIVATION OF THE SOLUTION TO AN EXPECTED KL- DIVERGEEE
We want show that
E B (L
C)) || C,T))]
KL (0 0)
exp (Boje plogp(z (.T)) - -log 2.
This can be seen from
E
[L
C*) p(z C,C))
Oz C*)
Enolii)) log p(z C,
(eog go
C*] - tele) log (xp (Ecicclogp(( C, C,C)))
KLL
KL
(go(z)
C)
exp c [log( C. log 2,
where the normalizing constant of
exp [ C.T))).
A.2 ANALYZING THE OPTIMAL PARTTALLY-CONDITIONNE POSTERIOR
In this section, show that assuming either w(z) p(z C) w(z) p(z C,C) implies
P(C z, Z,C) p(C C).
We begin by observing
w(z) x exp Bao [og c,C)]
p(z C) p(z C))
exp
Bajo log
p(z (,
p(z C,T)
= =p(al exp T
log
p(z C)
Now, firstly,
w(z) p(z |C)
log p(z C, C,C)
)
exp E Bi(c(o)
p(z C)
p(z
C,O)
(C 2, 2.C)
p(z [C)
p(C
C)
p(C 2, C) pC C).
Secondly,
((2) = p(z C2C)
exp (200) log p(z (.01
p(z C,O)
p(z
p(z C)
p(z C/z|C.C) p(C 2, C)
is constant W. r. C
p(z C)
c(C c)
p(C (, p(C
[c).
15

---

Published as a conference paper at ICLR 2021
A.3 PROOF OF SUBOPTIMAL GENERATIVE MODEL
We investigate whether a maximum likelihood solution p' = arg min, -xp--=ll log is minimum of the expected negative ELBO. From calculus of variations (Gelfand & Fomin, 2003), we derive necessary optimality conditions for maximum likelihood and the expected negative ELBO as
_ d m log (I.T))
dg
a
(max. likelihood)
(7)
dp
dp
0 _ d pp - log p( (1:T)]
dE xx r
[KT]
dg dp"
(ELBO)
dp
dp
respectively. G is a constraint ensuring that a valid density, À and a are Lagrange multipliers. KL refers to the posterior divergence in eq. (2). Equating (7) and (8) and rearranging gives
T
[KL] + (a A) dp
dg
dp - 0.
Equation (9) is necessary and sufficient condition (van Erven & Harremoës, 2014) that the Kullback-Leible divergence is minimised as a function of p, which happens when p(z, Ce C2) = q(z, Ct) for all t.
B MODEL DETAILS
We restrict our study to a class of current of-t-offthhearr art variational sequence models: that of resid- ual state-space models. The fact that inference is not performed on the full latent state, but merely the residual, allows more efficient learning through better gradient flow (Karl et al., 2017b; Fraccaro et al., 2016).
Generative Model The Markov assumptions that every observation only depends on the cur- rent state and every state only depends on the previous state and condition lead to the following model:
T II p(x: z: z:)p(zz 22-1, u-11,
p(X1:7 Zur u1:r)
where u1.r are additional conditions from the data set (such as control signals) and Zn up == 0. Such models can be represented efficiently by choosing convenient parametric form (e. g. neural networks) which map the conditions to the parameters an appropriate distribution D. this work, we let the emission model Dxx be a feed-forward neural network parameterised by 0g:
p(x: 2z) = Dxx (2:)).
For example, we might use Gaussian distribution that is parameterised by its mean and variance, which are outputs feed-forward neural network (FNN).
Further, we represent the transition model as deterministic step plus a component-wise scaled residual:
FNN (z--1, u,1) + FNNe. (z,-1) @
E; De.
Here FNN og (2.-1 produces a gain with which the residual is multiplied ddement-wise. The resid- ual distribution D. is assumed to be zero-centred and independent across time steps. Gaussian noise is hence a convenient choice.
The initial state z needs to have a separate implementation, as it has no predecessor and is hence structurally different. In this work, we consider base distribution that transformed into the distribution of interest by bijective map, ensuring tractability. We inverse rego-regressive flow (Kingma et al., 2016) and write Pa,, (zi) to indicate the dependence on trainable parameters, but do not specify the base distribution further for clarity.
16

---

Published as a conference paper at ICLR 2021
Amortised Inference Models This formulation allows for efficient implementation of infer- ence models. Given the residual, the state is a deterministic function the preceding state and hence it is only necessary infer the residual. Consequently we implement the inference model as such:
( 11 X1,7, ui.r) = (F,NN,N,,, (Ze, f.).
Note that that the state-space assumptions imply p(e; Zi:t 1, X1:T) p(E; Zt-1, Xt+1-T) (Sarkka, 2013). We reuse the deterministic part of the transition ž, = FNNN, (Z,-1, uz-1). Fea- tures f1r firr (fi, , define whether the inference partially-oonditonne and explained below.
Due the absence of a predecessor, the initial latent state 21 needs special treatment as in the generative case. We employ a separate feed-forward neural network to yield the parameters of the variational posterior the first time step:
< XI:T, U Dr, (FNN Auy (fi)).
Under-, Semi-, and Fully-Conditioned Posteriors We control whether a variational posterior is partially-, semi- or fully conditioned by the design of the features fLr. In the case of fully- conditioned approximate posterior, we employ a bidirectional RNN (Schuster Paliwal, 1997):
frr BiRNN de, (XI:T, uI:)),
For the case of an partially- conditiooe model, unidirectional RNN is option. Since it is com- mon practice (Karl et al., 2017b) to condition the initial inference model on parts of the future as well, let the feature at the first time step sneak peek into chunk of length k of the future:
fi FNNo, (Xi:k U1sk)
frr RNN (Xir, up:r),
where the recurrent network is unidirectional. We call this approach semi-conditioned model.
17

---

Published as a conference paper at ICLR 2021
C DETAILS ON THE EXPERIMENTAL SETUP OF UAV MODELLING
We go with a typical split into training, validation and test data. Each consists 10,240 sequences 25 time steps starting out at the origin. The distribution of the residuals D. and approximate pos- teriors D2, De is Gaussian. hyper-parameter search of 128 experiments was conducted. Models were trained by stochastic gradient descent using Adam (Bottou. 2010: Kingma & Ba, 2015) on the negative ELBBO. We selected the best model for semi- and fully-conditioned models separately according to the respective ELBOs on the validation set after 30,000 updates, which was sufficient for all models converge.
C. PAEER-PRRAAETER SEARCH SPACE
fiekd
batch size
Choice([, 16, 32, 64, 128))
ojeimiber..anninngg.uo
Cho 0.01, 0,003, 0.0001, 36-15))
optimizer.btaal
Choicer[0,5 a9p
latcet
Rangetlo-116, high-33)
initial..ftoos
Choice[22 8, 12p
inmallayer sizes.0
Chocce(8,
laye sbees.
Choice| 52, 64))
transtion.kinn
aochastic
transition.fur fune kind
Chosced residaal", 'highway', "plain')
trancition Dractivation
Choiee(16, 64, 1921)
nancitionllyers..e
Chosced sodtsign", 'sotppuus" 'rela,, "unh())
transtien.layers. Luniss
Chocce[16, 32, 128, 1921)
trancition loyers cettivation
Choice| Soltsign" softplus" 'relo,, "aahh))
Chocce(16, 32,
layers Quctivation
Choice[[ "othaig"" 'wotppos", 'rela,, "Dnh())
loyei units
Choiee([16, 64, 1280
im jnitalloyers 1 lactivation
Chosced sottsign" softplus", 'relo,, "anh"D
disturaacce layers Dannis
Choice(166 32,
in www.disturanceeee layers Baccivation
Choicet[ssotsippe 'sotipuus" 'rela", "anh())
in disturbance layers. Lunits
Chocce(116 32, 1280
in disturhance liyers, Laciivation
Choicel Soooeirs""iigg"... "oltplos", 'rol',, "anh))
Choiee[[32 64, 128B)
feature ran.n layers
Choioe([1 2))
featur: bidrecciaiall
True
feanure ran rype
iniiiitial mip-profix. length
festurajasiinitt Jale.
Choies(116 64, 1280
feanira janinitil pilpllyeers Ducctivation
Choseet softsign". softplus', 'reta", 'tabh))
feature inirial mip loyers uniis
Choioe[[16, 32, 64, 1280
Jeanua Ioyer
softiign, 'sopllos', 'sela,, "unh")
emisson. scale kind
fixed ansy((0115 0.15,, 0.075)
emission scale
18

---

Published as a conference paper at ICLR 2021
C.2 HYPER-PARAMETIE OF SEMI- CONDITONDTDO MODEL
field
value
batchasize
opemiize..aaminng rale
0,001
optimizerhetal
(1.9
lanest
initil.n
lntial.lterr.iiee
iniiaalayer sizes.1
trancition kind
stochastic
maaniiion.fuue
transtion.layess.o D.niis
trancition loyers Dnsetivation
Tanth
layers. Lunits
lansstion.laeess
softplus
in layers Quniss
16
in Jninaaloyers Dactivation
Tanth
layers Lactivation
softsiggs
inv disturbance layers, Danntts
in distrrance layers Disctvatioon
Ianh
in._listubancee.l layers.
disturbance. layer.. Lactivation
zelu
feature rinun stses
128
feanure_aatn, Joyers
bidirectioaa
feature
type
pm
feanir intinitil
prefix, pnlppprefxx_essgee
initial mip layers amics
32
inittal layers
softllus
smlp layers.
colure mm initaa mip layers, Lactivation
softsign
emission kind
Ficd
arn1y(0015, 0.0.5
C.3 HYPER-PARAMMTERS OF FULLY-CONDITIONED MODEL
field
value
batch_size size
32
opsimizer.lamien
hptimizerhetal
latent
20
initialn Bloss
12
imtial.tyer__szss..
initial hyer sixes.
transition kind
stoclustic
transition Jlin Jind
hig highway
transitinn
units
tnaneition leyers Disctivatinn
solpplx
Itansition.an layens Lnits
16
transiiion legers Lnctvvation
solppus
laye Oactivation
tanh
inviiiialllyers layers. units
128
in juitial loyers I Lactivation
relu
inw.listuruunee layers.
Jisturbunce loyers. Daccivattion
tanh
inv www.disturunceeec lay ers, I. Lunits
128
www.liturbunce..og lar..ayyer Lactivation
feature-ma.n.Jean mny...lyyerr
featureumnn bidirectional
True
feature Jype
fexture ze inisial laye.
layers O uniis
128
feature zan initia mip layers alattivation
softsign
feature initial mlp. layers. units
32
mwppppayee layens
solplus
amisssoa.scale scale
fixed
emission.scal scale
19

---

Published as a conference paper at ICLR 2021
D DETAILS ON THE EXPERIMENTAL SETUP OF MNIST MODELLING
We conducted a hyper parameter search of 64 experiments for 13,000 iterations. The best experi- ments (according the ELBO the last iteration) were continued for 85,000 further iterations. Test ELBOs are reported in table 2b.
D.1 HYPER-PARAMETERS OF PARTIALLY- PARTIALLY-CONDITTONNE CONDITIONED MODEL
range
batclusize
ogaiiizer.eamingg.uom rate
optimiberhetall
0.8
Jatenn
25
iniial.n
leyer siees.
inttial lnyial.leyer.sie lizessisss
32
emission scale kind
lesenable
emission layers
emission layers Olactivttion
relu
emission layers, Lanits
52
layers.
solplus
Inaneitonn
stochustic
trancition traniiiio.ffueee. fune
highht highway
transition. leyers Ounits
32
sonssign
maaniiionn layers. units
transifion leyers oetivation
solpplus
in jw-jiitilllog loyers Ounits
32
soflsign
initial layers. units
inv innjnitall loyee Lactivation
tanh
in disturbuncee.ae. layers,
64
iinuuuuuruncee layers. Dactivation
solpllu
www.uuurhneeeeeee layers,
inv Jisturbuncee layers Laciisation
sollsign
feature nustales
128
bidirectional
feature mnn enn.eellyppe
gru
featureumnn initial Jnlp lengh
feature intiiiitiall uni.
16
mil. layers
festureunn initial mml layers Dustivation
solpplu
feature.ran. inntiiitaalllnnnm.. layers activation solisign
D.2 HYPER-PARIE PARAMETERS OF FULLY- CULLY-CONDITIONED MODEL
Peemiser.eemming.rrte
0.003
optimizer.betal
0.9
25
Initial...fows
laye sizes.0
64
Initialltyer. sizes.
32
Jearnable
amission
emission emissionlayerss Ounirs.
64
emiseson layers dlactivation
softsign
layers, Lunits
layers. Lacivittinn
sottplus
transition.ind
stoc hastic
mtansition flne kind
hi ghwway
transition. Dennss
192
Inaneition beyers Disctvvatinn
transition.e Inyers. units
192
transitonn layers. occtivation
tanh
lnyes Dunits.
inv_inital.laeee Oucttvvation
sofisin
in layes. units
16
in initial aetivation
rella
mn._fistrbanne..l Jayers...nittt
Jisturbunce loyers. Dactivation
rele
in distrrhnnce Iayes. I Lumils
in wis.turannceeeeeee layers Luctivation
softsign
feature bidiractioaa
True
gra
feature iniital prefpp length
feanura_em.inittall haw.rraallmlppggggs.nno. Dunits
banurarasinitae wwwppaagesssttt..mmmmom Droctivation
festure inniniraa mip layers. uniss
128
fesmira_mminiriall mulp lnyers octivation
solplus
20

---

Published as a conference paper at ICLR 2021
E DETAILS ON THE EXPERIMENTAL SETUP OF TRAFFIC FLOW MODELLING
The data is down-sampled to contain average speeds over 30 minute windows from 6:30 to 19:00, resulting in 26 time steps per day. The data was split into training, validation and testing data by months, January up to July for training, July to September for validation and the remainder for testing. The standard deviation o² was determined during hyper-parameter optimisation along with the architectural and optimisation parameters. A hyper parameter search of 128 configurations was conducted. After 150,000 weight updates, the model for each partially,, semi- and fully-conditioned with the lowest negative ELBO on the validation set was selected.
E.I HYPER-PARAMETER SEARCH SPACE
Geld
range
batchasize
Choscet 64, 128))
optimizer. leamnng rale
(0.00x 0.0001, 05) Choice 0.8, 0.1
optimizer.heaal
n.latest
initial.n
sizzs
Choscet(8, 16, 52, 641)
lithal.yyer.ssisee
Choscer18, 16, 32, 64
transition kind
stocasticc
tranciion.
'resdaal", 'highway", "plain"))
transition.lay laye Dunits
64
trancition loyers octivation
softsign
transition layers kncnat..iti....
orthogonal
laiaallayyes Ouuniss
www.iniial..yeer.... Ouctivation
so0tpius
in in.jnitallayerr Klkmmll jnit...
gloror.normal
disturbance
layers activatiion
sottllu
in disturbance layers, fl..l.li..... kernel I...
glorot normal
feanure rainn sunes
64
feature row.lloors
bidirectional
feanire ren.cell type
grm
feanuresmm.in ianitra wwtppppcfix.eeggeeeeeee
feature rmminitial mip Dunits
Chosce(116 32, 64, 128D
feature
mip layers Dusctivation
Choicer "othsign" 'sofppus", 'rela", "unn(])
feanira jasinitil, milp loyers. units
Chosee[16, 32, 64, 1280
feature ran initial mip leyers octivation
Choicel Soohiigg" solpplos', 'rolu", "unnh])
kind
fixed
emission seale
Choice[ 0.2089651122411755 04449.
emission layers/tunin Dunnts
64
emission layers Alactivation
so0ppus
glomor. normal
21

---

Published as a conference paper at ICLR 2021
E.2
HYPER- PAPEE-PARAMETERS OF PARTIALLY-CONDITION CONDITIONED MODEL
field
value
batchasize
opemiize..aaminng rale
0.0001
optimizerhetal
0.55
latent
64
inital.n
lnteal.leyrr.iiae Sizes.
32
initaa.ayerr sizes.1
64
trancition kind
stochastic
tranciiin.fuue
plain
transition.lyees. D.nnis
64
trancition loyers O netivaation
softsign
transition Kekeneel
orthogonal
layerl in initial loyers autiivatinn
so0tlls
in loyee Oikemel juiti..
glorot sormml
disturancce layers
disturhance layers. acctivation
sottplu
inv disturbance layers, Kernel J...
glorot, normal
feature ren.n stanes
64
leaureJee.nnlgees
bidiractional
feature ren.coll type
canure_amiiiite putp-prefix_ leigth
initial mlp layers Dcstivation
soflsign
feature rnsiiiital layers
pasirraai Jnle-e octivation
reha
feature rnn initial mip leyers uniis
128
siale_Kind
fixed
amission
emission layers Dunits
64
emission layers A Rlattiittinn n n.hidden
solplix
layens kernel initillierr
E.3
OF SEMI-CONDITIONED MODEL
batchasize
128
opeinizer.eaminggmmmm
0.0003
optimizer.httal
0.5
Ialete
64
intiaaln Blowss
inttal..yyersizes..ll
initial Intiaal.yyer.sieess
32
transition.ke kind
stchastic
transitio. kind
trancition.k layen.
trunsition.k leyen activation
sollsign
transition. lay D. ininna. inieall units
orthogonal
64
layers lnv. activaation in layis o Okkemel aniti..
sottplus
glorot.normal
inv disturbance layers, il, uunits
6-1
disturbance Jay.rs. Dlactvation
sottllu
disturbance l...eys...keree......
normall
feature mann stases
6-1
feanire raw.n.lyeee.
feature hidirectional
True
feature rype
gra
camura-raniiittal mnlp-prefix.
feature raninitial mip layens
128
feanure imsinirill miyelayers. units
teanure_ras initiattallaa..mmm.mmmm wwwp.aayer.......o Dlcttvvtto
softsign
feature raninitial mip.layers. activvation
tanh
emission seale kind
fined
0,775802
layers Gunits
emission aactivatnao
soctplus
emission layers Keneel jactzlizzer
glorot, soomal
hidden
22

---

Published as a conference paper at ICLR 2021
E.4 HYPER- PAPAM-PAR OF FULLY-CONDITIONED MODEL
Bield
value
batchasize
opeiiize..aaminng rale
0.0003
optimizerhetal
0.8
latent
64
intial. dizess
initaa.ayer sizes.1
trancition kind
stochastic
tranction.fi
plain
transitionllyers. D.nnits
64
trancition loyers O netivaation
sofhsign
orthogonal
layers in initial loyers aiviiiaionn
so0tlls
in loyee Oikemel juiti..
glorot sormal
disturance layers
disturhance layers. acctivation
sottplus
inv disturbance layers, Kernel I...
glorot, normal
feature ren.n stanes
64
bidiractional
feature ren.coll type
gra
canure_mniiiite pulp-prefix_ leigth
initial layess Duniss
feature insiiiital layers Disctivation
pasirra mile-lyeers units
32
feature ren initial mip leyers octivation
sollsign
Kind
fixed
emission
emission Ouners/uunts Dunits
64
emission layers Rlactiittion
solplix
layens kernel inidillierr
glorol mormall
n.hidden
23


Reviews:
[{"review_id": "fd6aa0I-ns", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "I enjoyed this paper, and think it provides a valuable contribution to sequental latent variable modeling of time series data.\n\nSpecifically, this paper addresses the issue of conditioning in using variational inference to fit sequential latent variable models to data. In addition to potential errors from an amortisation gap or approximation gap, a conditioning gap is identified, where a variational distribution that is not conditioned on all possible information (previous timesteps' observations and latent variables) underperforms. \n\nThis seems like an 'obvious' insight, but I think that is a strength of this paper. It clearly shows why previous work falls short of using all available information to get good performance, through a simple theoretical analysis. Further, empirically the work demonstrates how to correct for the conditioning gap.\n\nI anticipate that through the publication of this paper at ICLR, the authors of future papers in this area will need to be careful in conditioning. This will benefit the research community as a whole, and lead to higher-quality variational approximations and papers.\n\nOne nit:\n\n- although the optimal variational approximation may not be ideal in the theoretical study here, in Section 3.1, I think a discussion of why the KL divergence was used in this study is warranted. For example, there are other divergence measures that do not suffer from the issues presented here  (c.f. https://dataspace.princeton.edu/handle/88435/dsp01pr76f608w).  It would be helpful to point practitioners to this related work, as one option to consider given that the KL divergence learns products of posteriors and this may not be a desirable feature of a divergence for an application.", "extra_scores": {}, "extra_text": {}}, {"review_id": "YkO3GmSn5Q3", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper reviews the issue of partial conditioning of the amortized posterior in sequential latent variable models, typically state-space models trained with a VAE-style loss, but where the posterior used is the filtering rather than smoothing posterior. The author show that training a model with posterior with missing information can lead to a gap in estimating both the posterior and the corresponding model. They show the benefits of using the correct posteriors in simple examples.\n\nOverall, the paper is well written, but its originality is on the low end; a large number of papers describing state-space VAE like models make explicit that the filtering posterior is technically suboptimal compared to the smoothing one; few see benefits in actually using the smoothing posterior (note that [1] derives a valid ELBO using only a one-step smoothing update). The derivation of the shared approximate posterior is standard variational inference derivations, but it is nice to see it explicitly written, and contrasted with the optimal posterior. The paper frequently gives nice intuitions behind various facts (mixture vs product of expert and the gating effect, when is an imperfectly conditioned expert enough, etc.). \n\nThe univariate Gaussian example is a good toy problem to understand the issue at hand. The numerical examples are selected to highlight the benefits of smoothing; they are interesting but perhaps not particularly challenging or surprising, and in relatively short sequences it makes sense that peeking or smoothing would benefit over filtering.  \n \n\nOverall, I am still inclined to accept the paper, as it investigates more clearly and makes more explicit knowledge that is usually treated as folklore and footnotes in other papers. ˆIt does not really offer methods for making smoothing posteriors actually learn more powerful models than filtering posteriors on complex datasets, nor does it technically demonstrate that SOTA state-space models have their performance limited through the information of the posterior (note that state-space models still typically underperform autoregressive ones). \n\n\nMinor:\n-Table 1: The Deep Kalman Filter, in some of its instantiations, has an empty \\bar{C_t}; they explicitly condition z_t on z_tm1 and x_\\geq t (see equation 3). See also [2] for another state space model that considers both the smoothing and filtering posteriors (as others, they note no benefit to the smoothing posterior).\n\n-Equation 5: Technically the left hand side should be the function w, not the particular value w(z) (note z is a bound variable on one side of the equation and bound on the other side). I understand what the authors mean, but it looks strange as it is.\n\n\n[1] Gregor et. al, \"Temporal Difference VAE\"\n[2] Buesing et al., \"Learning and Querying Fast Generative Models for Reinforcement Learning\"", "extra_scores": {}, "extra_text": {}}, {"review_id": "QO_JhhZfI9W", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThe paper considers the problem of Bayesian inference with partially conditioned variational posterior. Namely, this work describes the phenomena of ill-behaved variational posterior for the case of partially observed data. The paper's main theoretical finding is that the partially conditioned variational posterior behaves like a product of experts, resulting in a degenerate solution. Speaking intuitively, the true posterior can be seen as a mixture of distributions: the sum over the unobservable variable. At the same time, the optimal variational posterior mixes as a product of distributions. Clearly, the product of densities hardly depicts features of the mixture since a near-zero value of a single member is enough for zeroing out the product's density.\nNevertheless, such models are successfully applied in some cases, and the authors explain why this theoretically perspectiveless construction could work in practice. The answer is quite straightforward: the partially conditioned variational posterior works only when the unobserved variables do not affect the true posterior. Finally, the authors strengthen their theoretical studies with neat experimental studies.\n\nReview:\nIt is hard to write a useful review for this paper since the authors clearly have thought through many aspects of their work. I find this paper to be a useful piece, both theoretically and practically.\n\nMy only suggestion would be to include several works into consideration. The problem of partial observability is also important for generative image models [1,2]. I don't propose to perform a model comparison, but I think the reader would benefit if you could relate your result with similar works from CV. For instance, why other models avoid this degenerate solution while still conditioning partially. Or how one can possibly escape difficulties when full conditioning is not possible on the test stage.\n\nMinor comments:\n1. page 1, section 2.1. I think by \"minimization of eq. 1\" the authors mean maximization of the marginal likelihood.\n\nReferences:\n1. Ivanov, Oleg, Michael Figurnov, and Dmitry Vetrov. \"Variational autoencoder with arbitrary conditioning.\" arXiv preprint arXiv:1806.02382 (2018).\n2. Ledig, Christian, Lucas Theis, Ferenc Huszár, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken et al. \"Photo-realistic single image super-resolution using a generative adversarial network.\" In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4681-4690. 2017.", "extra_scores": {}, "extra_text": {}}, {"review_id": "4SiYSFzX3r0", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**Summary**\nThis paper investigates the effect of partial conditioning on amortized inference in variational auto-encoders, focusing specifically on sequential data sources where it is common practice to have a posterior that is factorized in such a way that conditioning is partial (usually only conditioning on past signals in the sequence). Given a true posterior that is conditioned on the entire observed datapoint, the authors discuss the effect of having an approximate posterior that is only conditioned on part of the input. As the approximate posterior cannot adapt to the part of the input that is left out of the conditioning, the evidence lower bound becomes less tight, due to the larger KL divergence between the approximate posterior and the true posterior. The authors compare this to the work by Cramer et al. [1], where the distinction was made between having a restricted family of possible distributions for the approximate posterior (approximation gap) and the gap between an amortized approximate posterior with an inference network shared for all datapoints and a non-amortized approximate posterior that is optimized for each datapoint separately (amortisation gap). They argue that partial conditioning leads to a third type of gap which is distinct of the aforementioned inference gaps. Through an example with discrete observations the authors derive that when the true posterior is conditioned on the full data, and the approximate posterior is only partially conditioned, the optimal approximate posterior is something akin to a product of true posteriors over the unconditioned information, and not a mixture where the left out information is marginalized out. Through a 1D example they show that this could lead to overly sharp posteriors that have high densities in regions where the true posterior has very low density. \nAs the authors also state, several studies have shown that full conditioning on future observations results in negligible performance gains. However, the authors conjecture that this is because those results were mainly found on problems where the conditioning issue was not (or less) relevant. \nThe authors demonstrate potential performance gains on 3 datasets where conditioning on future information could be helpful: unmanned aerial vehicle trajectories from the Blackbird dataset, a sequential version of MNIST where the rows of a picture correspond to sequential observations, and a selection of a traffic flow dataset. They perform log likelihood estimates and prefix-sampling to determine the effect of conditioning (partially or fully) on future observations.\n\n\n**Pros**\n- The idea behind the effect of conditioning on the amortization procedure is clearly explained.\n- The exposition that explains that the optimal approximate posterior could correspond to something akin to a product of distributions instead of a mixture is interesting.\n- On the datasets that were selected by the authors, the benefit of full conditioning versus partial conditioning is visible in the quantitative results (log likelihood estimates).\n\n**Cons**\n- The authors argue that the conditioning gap is a distinct gap from the amortization gap that was discussed by Cramer et al. [1]. It is not clear to me why these two gaps are distinct/independent, I would say the conditioning gap is part of the amortization gap introduced by Cramer et al. since the way that conditioning is handled in amortized inference is essential to the gap between the amortized and non-amortized approximate posterior. For instance, in the example of the univariate gaussian in section 3.2, where would the amortization gap from Cramer et al. fit in as a separate gap? The amortization gap can be large because the conditioning is incomplete, or because of the limited flexibility of the neural network mapping from conditioning to parameters. Such a limited flexibility could reach the same type of error as partial conditioning. \n- The effect of the narrow posteriors for partially conditioned approximate posteriors due to it being a product of distributions and not a mixture is not clear in the experiments, even though the authors do hint that this is observed in the qualitative prefix sampling experiments. The sample prefix experiments are furthermore very hard to judge, especially for the traffic flow dataset. The authors draw conclusions from these plots that I can’t confirm by looking at the plots. For instance, with respect to the traffic flow samples the authors state that “the partially conditioned model concentrates too much…”. It seems to me like it concentrates about the same amount as the full model, and I don't see how it’s “too much” as the dashed line is usually among the predictions. I think the authors are incentivized to find this conclusion because they try to argue in figure 1 that products of distributions concentrate too much argument.\n- As the authors state, previous work showed that on popular datasets conditioning on future information has little gains. Even though the authors find datasets where gains can be made, these datasets are not incredibly convincing that this is actually a widespread problem for sequential VAEs. The lack of overlap between datasets that related work is evaluated on  (such as the gym or mujoco datasets or natural speech waveform datasets and polyphonic music datasets) and datasets that this work is evaluated on and the fact that the datasets of this paper are particularly small or artificial makes me a little concerned that the problems need to be cherry picked for the proposed conditioning to have an actual effect. For instance, although the MNIST example is an obvious example where conditioning on future information could help, it is fairly artificial. The authors argue that the gym and mujoco datasets have deterministic dynamics (and therefore shouldn’t suffer from partial conditioning), but do not explain why waveform datasets or polyphonic music datasets are not suitable to study this problem. Together with the fact that related work is discussed but not compared against empirically, this makes it hard to place this work in context with related work and to judge its relevance. \n- I would expect more results on the influence of the sneak-peak parameter k. On the traffic flow dataset the authors suggest that the full model (with largest possible k) can perform worse on the test set than a model with intermediate k because the intermediate-k model already contains sufficient future information. This could be investigated if results were compared for models with more values of k, and a leveling off of performance gains for increasing k could confirm this conjecture.\n\n\n**Minor comments**\n- In section 5 there is a lot of referring to section 5 itself in the middle of sentences, which breaks the flow and seems unnecessary. See for instance the first paragraph of section 5.\n- Are you using statically binarized mnist or dynamically binarized mnist?\n\n\n[1] Cramer et al. inference suboptimality in variational autoencoders. https://arxiv.org/abs/1801.03558\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper studies how suboptimal conditioning sets create
suboptimal variational approximations in variational inference with amortization in state space models. 
While the point made about the role of the conditioning set is not a new one, the point was carried out further and 
more clearly in this paper than previous works. Addressing a couple of issues would 
make the paper stronger:

- Really boiling down in the experiments to know for what models/data
  the "full" approach would add value would provide concrete guidance
  to the community.


- Notation choices in the paper are rough. For example, Appendix A.2
  reads like a type mismatch since the w on the left is a function of
  z but is also equal to a function of z and C.


- Adding a more detailed description of the complement of C in the
  main text
Decision: Accept (Poster)