Self-Supervised Inference in State-Space Models
Authors: David Ruhe
Patrick Forré
Venue: ICLR 2022 Poster (2022)
Keywords: self-supervision
inference
state-space model
Kalman filter
recurrent neural network

Abstract: We perform approximate inference in state-space models with nonlinear state transitions. Without parameterizing a generative model, we apply Bayesian update formulas using a local linearity approximation parameterized by neural networks. It comes accompanied by a maximum likelihood objective that requires no supervision via uncorrupt observations or ground truth latent states. The optimization backpropagates through a recursion similar to the classical Kalman filter and smoother. Additionally, using an approximate conditional independence, we can perform smoothing without having to parameterize a separate model. In scientific applications, domain knowledge can give a linear approximation of the latent transition maps, which we can easily incorporate into our model. Usage of such domain knowledge is reflected in excellent results (despite our model's simplicity) on the chaotic Lorenz system compared to fully supervised and variational inference methods. Finally, we show competitive results on an audio denoising experiment.


Published as a conference paper at ICLR 2022
SELF-SUPERVISEE INFERENCE IN STATE-SPACE MOD- ELS
David Ruhe
Patrick Forré Al4Science, AMLab University of Amsterdam, The Netherlands p.d. d. forreeuva...
AI4Science, AMIab, Anton Pannekoek Institute University of Amsterdam, The Netherlands d. ruhe@uva..... nnl
ABSTRACT
We perform approximate inference in state-space models with nonlinear state tran- sitions. Without parameterizing a generative model, we apply Bayesian update formulas using local linearity approximation parameterized by neural networks. This comes accompanied by a maximum likelihood objective that requires no su- pervision via uncorrupt observations ground truth latent states. The optimiza- tion backpropagates through a recursion similar to the classical Kalman filter and smoother. Additionally, using approximate conditional independence, perform smoothing without having to parameterize a separate model. In scien- tific applications, domain knowledge can give linear approximation of the latent transition maps, which we can easily incorporate into our model. Usage of such domain knowledge is reflected in excellent results (despite our model's simplic- ity) on the chaotic Lorenz system compared to fully supervised and variational inference methods. Finally, we show competitive results on an audio denoising experiment.
INTRODUCTION
Many sequential processes in industry and research involve noisy measurements that describe la- tent dynamics. state-space model is type of graphical model that effectively represents such noise-afflicted data (Bishop, 2006). The joint distribution is assumed to factorize according to di- rected graph that encodes the dependency between variables using conditional probahilities. One usually interested in performing inference, meaning to obtain reasonable estimates of the posterior distribution of the latent states or uncorrupt measurements. Approaches involving sampling (Neal et al., 2),1), variational inference (Kingma & Welling, 2013), belief propagation (Koller & Fried- man, 2009) have been proposed before. Assuming a hidden Markov process (Koller & Friedman, 2009), the celebrated Kalman filter and smoother (Kalman, 1960; Rauch et al., 1965) classical approaches to solving the posterior inference problem. However, the Markov assumption, together with linear Gaussian transition and emission probabilities, limit their flexibility. We present filtering and smoothing methods that are related to the classical Kalman filter updates but are augmented with flexible function estimators without using a constrained graphical model. By noting that the filtering and smoothing recursions can be back-propagated through, these estimators can be trained with a principled maximum- likelihoi objective reminiscent of the noise seiiiiisis objective (Lehtinen et al., 2018; Laine et al., 2019). By using a locally linear transition distribution, the posterior distri- bution remains tractable despite the use of non-linear function estimators. Further, we show how a linearized smoothing procedure can be applied directly to the filtering distributions, discarding the need to train a separate model for smoothing.
To verify what is claimed, we perform three experiments. (1) A linear dynamics filtering experi- ment, where we show how our models approximate the optimal solution with sufficient data. We also report that including expert knowledge yield better estimates of latent states. (2) A more challenging chaotic Lorenz smoothing experiment that shows how our models perform on par with recently proposed supervised models. (3) An audio denoising experiment that uses real-world noise showing practical applicability of the methods.
Our contributions can be summarized as follows.

---

Published as a conference paper at ICLR 2022
Estended Kalmun Sucuthau
Smoothar
Smoothar
Figure/ 1: Best viewed on screen. Qualitative results of our work. To the noisy measurements (1st from left), we apply an extended Kalman smoother (2nd). From the noisy measurements, we learn a recurrent model that does slightly better (3rd). Our recursive model combines expert knowledge with inference (4th), yielding the best result. Ground truth provided for comparison (5th).
We show that the posterior inference distribution of state-space model is tractable while parameter estimation is performed by neural networks. This means that we can apply the classical recursive Bayesian updates, akin to the Kalman filter and smoother, with mild assumptions the generative process.
2. Our proposed method is optimized using maximum likelihood in self-supervised manner. That is, ground truth values of states and measurements are not assumed to be available for training. Still, despite our model's simplicity, our experiments show that it performs better or on par with several baselines.
3. We show that the model can be combined with prior knowledge about the transition and emission probabilities, allowing for better applicability in low data regimes and incentiviz- ing the model to provide more interpretable estimates of the latent states.
4. A linearized smoothing approach is presented that does not require explicit additional pa- rameterization and learning of the smoothing distribution.
RELATED WORK
Becker al. (2019) provide detailed discussion of recent related work, which we build here and in table 1. An early method that extends the earlier introduced Kalman filter by allowing nonlinear transitions and emissions is the Extended Kalman filter (Ljung, 1979). It is limited due the naive approach to locally linearize the transition and emission distributions. Furthermore, the transition and emission mechanisms are usually assumed to be known, or estimated with Expectation Maxi- mization (Moon, 1996). More flexible methods that combine deep learning with variational infer- ence include Black Box Variational Inference (Archer al., 2015). Structured Inference Networks (Krishnan et al., 2017), Kalman Variational Autoencoder (Fraccaro et al., 2017), Deep Variational Bayes Filters (Karl ct al., 2017), Variational Sequential Monte Carlo (Naesseth et al., 2018) and Dis- entangled Sequential Autoencoder (Yingzhen & Mandt, 2018). However, the lower-bound objective makes the approach less scalable and accurate (see also Becker et al. (2019). Furthermore, all of the above methods explicitly assume graphical model, imposing a strong but potentially harmful inductive bias. The BackpropKF (Haarnoja et al., 2016) and Recurrent Kalman Network (Becker al., 2019) move away from variational inference and borrow Bayesian filtering techniques from the Kalman filter. We follow this direction but do not require supervision through ground truth latent states or uncorrupt emissions. Satorras al. (2019) combine Kalman filters through message pass- ing with graph neural networks to perform hybrid inference. We perform some of their experiments by also incorporating expert knowledge. However, contrary to their approach, we do not need su- pervision. Finally, concurrently to this work, Revach et al. (2021) develop KalmanNet. It proposes similar techniques but evaluates them in a supervised manner. The authors, however, do suggest that unsupervised approach can also be feasible. Additionally, we more explicitly state what gen- erative assumptions are required, then target the posterior distribution of interest, and develop the model and objective function from there. Moreover, the current paper includes linearized smoothing (section 6), parameterized smoothing (appendix A), and the recurrent model (appendix C). We also denote theoretical guarantees under the noi se2noi se objective.
2

---

Published as a conference paper at ICLR 2022
scalable state os, uncertiaity noise dir. opt. self-sup.
Ljung (1979)
fix
x
Hochreiter al. (1997)
41x
v
v
Cho (2014)
7
Wallienien at al (2015)
x
x
4
A
e
Watter et al. (2015)
4
x
fix
Krishmun et al. (2017)
Fracearo et (2017)
Kirl et (2017)
Naessethel el al. (2018)
Yingzhen et (2018)
1--1
mb+1
Rangspuram (2018)
x/x < (
3
4
Deer ctal
Saterns et al. (2019)
4
<<
Hauroaa et al. (2016)
Beckerei (2019)
Ours
x/x
V
Wi
Ve++
Table I: We compare whether algorithms are scal- able, state estimation can be performed, models pro- vide uncertainty estimates, noisy or missing data can be handled, optimization is performed directly and if supervision is required. "V xx means that it depends the parameterization.
Figure 2: space model with deeper la- tent structure.
GENERATIVE MODEL A ASSUMPTIONS
In this section, we explicitly state the model's generative process assumptions. First, we assume that we can measure (at least) one run of (noise-afflicted) sequential data Yook == (yo.... 4K), where each yı E Rdd. k O...., K. We abbreviate: Visk := (yI...., ykk) and V< yook 1 and ys := yock and y-k = (Bo:--1, 1. 3k+1.)). We then assume that yo:k is the result of some possibly non-linear probabilistic latent dynamics, i.e., of a distribution p(zo:k). whose variables are given by æo.K == (Zo..... xx) with Ø E IRN, Each ye is assumed to be drawn from some shared noisy emission probability p(ys axx). The joint probability is then assumed to factorize as:
K
P(yo: To:k) = pl ((()
II P( xR).
(1)
k=
Further implicit assumptions about the generative model are imposed via inference model choices (see section 7). Note that this factorization encodes several conditional independences like
11 (y-2,
Typical models that follow these assumptions are linear dynamical systems, hidden Markov models, but also nonlinear state-space models with higher-order Markov chains in latent space like presented in fig. 2.
In contrast to other approaches (e.g.. Krishnan et al. (2015); Johnson et al. (2016); Krishnan et al. (2017)) where one tries model the latent dynamics with transition probabilities p(æ, xx-1) and possibly non-linear emission probabilities p(y xxx), we go the other way around. We assume that all the non-linear dynamics are captured inside the latent distribution p(æo::) ). where at this point we make no further assumption about its factorization, and the emission probabilities are (well- approximated with) linear Gaussian noise model:
ply kx) = N ( HHH,
(3)
where the matrix H represents the measurement device and R is the covariance matrix of the in- dependent additive noise. We make brief argument why this assumption is not too restrictive. First, if one interested in denoising corrupted measurements, any nonlinear emission can be cap- tured directly inside the latent states Fg.- To see this, let z₁ E N-M denote non-emitted state
variables. We then put Iy := [y: 2x]" where yk is computed by applying the nonlinear emis- sion to Zk- We thus include the measurements in the modeled latent state Ø3. Then can put H := [ (M) Second, techniques proposed by Laine et al. (2019) allow for non- Gaussian noise models, relaxing the need for assumption eq. (3). Third, we can locally linearize the
3

---

Published as a conference paper at ICLR 2022
emission (Ljung, 1979). Finally, industrial academic applications include cases where emissions are (sparse) Gaussian measurements and the challenging nonlinear dynamics occur in latent space. Examples can be found in MRI imaging (Lustig et al., 2007) and radio astronomy (Thompson al., 2017).
4 PARAMETERIZATION
In this section, we show how we parameterize the inference model. lot of the paper's work relies on established Bayesian filtering machinery. However, for completeness, like prove how all the update steps remain valid while using neural networks for function estimation.
find good estimates for the latent states Zo.K = (Zo.... (To..... ZKK, which generated Yo:K- For this, we want to infer the marginal condi- tional distributions p(æx yg{a) p(æ, y**)) (for forecasting), for online inference approach (fittering; and p(æ* yo:k) or p(æs y-x), for a full inference approach (smoothing). In the main body of the paper, only consider filtering. Smoothing be performed similarly, which is detailed in the supplementary material (appendix A).
Given noisy measurements Yo:k (yoo.... ... YK)
We start with the following advantageous parameterization:
p( 0k-1- ycck) = N <' FRIek th-1 + Ex|ck Qmlek),
(4)
where FaIcc (g) ēxtck @x|=ch (H+ck) and QMER with neural networks. Next, we have available
parameterized
p(a=-1 Vs(k-1)) =N ( -N
@ | P(K-))SA-))))))))
(5)
i... the previous time-step's conditional marginal distribution of interest. For = 1, this some initialization. Otherwise, it is the result of the procedure we are currently describing. We use this distribution to evaluate the marginalization
p(ak ycn) = / p(ax xk-11 (atcck)p(aa---- p((x-1 ycck dx
(6)
= -N (*x dx|ckk PacckPPaict Palce)
with (y<<)(E<)) Palcc 0 + ēxjcke PMtc((Ek))
PARe Pk- FT + Qujck-
(7)
Note that the distributions under the integral eq. (6) are jointly Gaussian only because of the param- eterization eq. (4). Hence, we can evaluate the integral analytically.
Finally, to obtain the conditional p(
p(
ykk Beck) we use Bayes' rule:
p(æk yk y<ck plyk Lk. yek) p(æx y<ck)
eq.(2) (2)
ply xx)
p(#x yee).
(8)
plux Veck
p Deck
Equation (3) and the result eq. (6) allow us also get an analytic expression for
p(æ y = N (x @Kick PHISE)
(9)
with the following abbreviations:
( dzjck K& (y H dk(ck), (aIS)(HE)) PAicA K2 H Paice (10)
We introduce the Kalman gain matrix similar to the classical formulas:
K = Ps. <k H (( PAik H + +R
(11)
Note that taking the matrix inverse at this place in eq. (11) is more efficient than in the standard Gaussian formulas (for reference presented appendix B) M < N, which holds for our experi- ments.
4

---

Published as a conference paper at ICLR 2022
GRU
GRU
GRU
Yea
F
F
20-1)
g0+3
Figure 3: Our recursive model visualized. Data-point yk is fed, together with hidden state h into GRU unit. The new hidden state h is decoded into multiplicative component F641 and additive component ek+1- Using these, the previous posterior mean TM|Sı is transformed into the prior estimate for time-step lo + #x+15k- yk++1 is used to obtain posterior mean 11+1)5((+1)-
This completes the recursion: we can use eq. (9) for a new time-step + by plugging it back into eq. (6). We have shown how estimating local linear transition using neural networks in eq. (4) ensures that all the recursive update steps from the Kalman filter analytically hold without specifying and estimating generative model.
We note that we could also have parameterized
p(a* ycc) N (Ix [ (a) Pa)CE(MER))
(12)
with Pxjjck(U<c and PRICC (Yec) directly estimated by neural network. This has the advantage that do not rely on local linear transition model. However, it also means that we are esti- mating 2s without any form of temporal regularization. Additionally, it is harder to incorporate prior knowledge about the transitions maps into such model. Nonetheless, we detail this param- eterization further in the supplementary material (appendix C) and include its performance in our experiments in section
To conclude the section, we like to discuss some of the limitations of the approach. 1. The Gaus- sianity assumption of eq. (4) ensures but also restricts eq. (8) and eq. (9) these forms. That is, make a direct assumption about the form of the posterior p( E yεk).. Defending our case, we like to point out that methods such as variational inference (Krishnan et al., 2017) or posterior regularization (Ganchev et al., 2010) also make assumptions (e.g., mean-field Gaussian) about the posterior. 2. Since we did not explicitly specify a factorization of p(zg: ), we cannot ensure that the distributions we obtain from the above procedure form a posterior the ground truth generative process. This does not mean, however, that we cannot perform accurate inference. Arguably, not making explicit assumptions about the generative process is preferred to making wrong assumptions and using those for modeling, which can be the case for variational auto-encoders. 3. The local linearity assumption is justifiable if the length between time-steps is sufficiently small. However, note that the model is more flexible than directly parameterizing eq. (12) (see appendix C) since it reduces that case by putting F₁ = for all k.
FITTING
We have shown in the previous section how parameterization of local linear transition model leads to recursive estimation of p(æz yck) and p(æ; ySk) for all & using classical Bayesian filtering formulas. The inference is only efffective if the estimates FRIck εxjcc and QRick from eq. (4) are accurate. We can use the parameterization p(æ; ycck) of eq. (6), the emission model plys 2x) N | æx R) from eq, (3), and the factorization from eq. (1) to see that an analytic form of the log-likelihood of the data emerges:
/ p(yx (=)p((z kck) dxy
p vck)
(13)
=N (vxl H Tineee H (yek) H R).
5

---

Published as a conference paper at ICLR 2022
K
log (o..)))) =
Σ k=0
log N (ys)
yv ( H PuICH(yen) H + R
(14)
If we put Dujck == H @x) and MAKK = H PAIK H + R. then the maximum- likelihood objective leads to the following loss function, which we can minimize using gradient descent methods w.r.t all model parameters:
K
C :: Σ (ÜNEC yk)) Maje R=0
klckk (UNick - yk) + log det MajER
(15)
Note that each term in the above represents one-step-ahead self-supervised emror term. We thus minimize the prediction residuals Ue|<a (B<+) yk in a norm that is inversely scaled with the above covariance matrix, plus regularizing determinant term, which prevents the covariance matrix from diverging. The arisen loss function is similar to the noise2nois (Lehtinen et al., 2018; Krull al., 2019; Batson & Royer, 2019; Laine al., 2019) objective from computer vision literature, combined with locally linear transition model like Becker et al. (2019). We show in appendix D that this objective will yield correct results (meaning estimating the ground-truth æ&) the noise is independent with Elys Hææ] H A similar procedure in the causality literature given by Schölkopf al. (2016). An algorithmic presentation of performing inference and fitting presented in appendix E.
Note that after fitting the parameters to the data, eq. (6) can directly be used to do one-step ahead forecasting, Forecasting an arbitrary number of time steps is possible by plugging the new value á KK+iK via yk+1 == & K+ij back into the recurrent model, and so on. This is not generative model but merely convenience that deemed worth mentioning.
6 LINEARIZED SMOOTHING
So far, we have only discussed how to perform filtering. Recall that for smoothing, we are instead interested in the quantity p(ar 90:k). A smoothing strategy highly similar to the methods de- scribed earlier can be obtained by explicitly parameterizing such model, which we detail in the supplementary material. Here, we introduce linearized smoothing procedure. The essential advan- tage is that no additional model has to be trained, which can be costly. Several algorithms stemming from the Kalman filter literature can applied, such as the RTS smoother (Rauch et al., 1965) and the two-filter smoother (Kitagawa, 1994). To enable this, we need to assume that the conditional mutual information I (xx-1i ys:K Tk: 3ook-1) is small for all & = 1,..., K. In other words, we assume that approximately have the following conditional independences:
xk-1
yk:K (#+ 90:
(16)
To explain the motivation for this requirement, consider the model in fig. 2. If the states of yook and Iy are known, then the additional information that yk:K has about the latent variable 2 would need to be passed along the unblocked deeper paths like yk+11 < 24+1 + Ex+1 + eş + -k-1. Then the assumption of small [ yk:k 3ey 30:k-1) can be interpreted as that the deeper paths transport less information than the lower direct paths. If we consider all edges to the oy as linear and the edges to the eg's 's as non-lineat maps, the above could be interpreted as an information-theoreti version of expressing that the non-linear correction terms are small compared to the linear parts in the functional relations between the variables.
We will now show that under the earlier assumptions and eq. (16) we get Gaussian approximation: p(æx yo:k) x N Dt 2g, Git). We will do backward induction with 2ς = kK|şk and Gx PKISK To propagate this to previous time steps
the chain rule:
p( yo:k) = - Øs, Yo:k) p(ack
yo:k) days,
(17)
where the second term is known by backward induction and for the first term we make use of the approximate conditional independence from eq. (16) to get
eg. eg. (16)
p 1 Dc, yo:k) p(xx-1
yk:KK , Lk. 1ook-1)
p( Iv. v0.
(18)

---

Published as a conference paper at ICLR 2022
The latter was shown to be Gaussian in section 4:
([
[k-1 0tk
3 k- 1|s--1
PR-1ISK11 FRICK Pr PR-1ISK-1
PR-IISK-1 FT
F, 10:k-1) N
Cklck
PRIcE
(19)
By use of the usual formulas for Gaussians and the reverse Kalman gain matrix J1-1/8 we arrive the following update formulas, k: = K....,1. 1, with Ex= == BKISK K and GK = PKISK:
fx-1jk = Pk-115k-1 F kkcck kkckk
P-1
(20)
Gk-1 Pk-1|Sk-1 Jn-1jk 1/1 (PEIS PAICE) Jk-Ilk
(21)
kk-1 == 1 + J 1/k (&x - @x(ss).
(22)
As such, we can perform inference for all k o,..., K with p(æ* yo.k) = N N(xx) axs Σ Gh Algorithmically, the above presented in appendix E.
RECURRENT NEURAL NETWORK
Before going into the experiments section, we briefly explain how specifically estimate the functions (g)c)()))) GbjeR (g(ck) and QRick(B<C)) that parameterize the transition probability p(arg Y<kk 1x-1) (eq. (4)). The choice of the model here implicitly makes further assumptions about the generative model. If we consider neural networks, the temporal nature of the data suggests recurrent neural networks (Graves al., 2013), convolutional neural networks (Kalchbrenner al., 2014), or transformer architectures (Vaswani et al., 2017). Additionally, if the data is image-based, one might further make use of convolutions. For our experiments, we use Gated Recurrent Unit (GRU) network (Cho et al., 2014), that recursively encodes hidden representations. Therefore, we put
Quick:= L,L
f(h)))
h GRU h,
(23)
ix
where La is Cholesky factor and f is multi-layer perceptron decoder.
8 EXPERIMENTS
We perform three experiments, as motivated in section 1. Technical details on the setup of the ex- periments can be found in the supplementary material (appendix F). We refer the model detailed in section 4 as the recursive filter, as uses the Bayesian update recursion. For smoothing exper- iments, we recursive smoother. The model obtained by parameterizing p(ar; | y<*) directly (eq. (12)) is referred to as the recurrent filter or recurrent smoother, as it only employs recurrent neural networks (and no Bayesian recursion) to estimate said density directly.
8.1 LINEAR DYNAMICS
In the linear Gaussian case, it is known that the Kalman filter will give the optimal solution. Thus, we can get lower bound on the test loss. In this toy experiment, we simulate particle tracking under linear dynamics and noisy measurements of the location. We use Newtonian physics equations as prior knowledge, We generate trajectories T (wo:k Yo:k) J with æg E RR and yı E IR² according
to the differential equations:
0
0
A.c
0
(24)
0
-
Hæ + r with ~ NO, R). H is selection matrix that returns two-dimensional position vector. We run this experiment in filtering setting, i.e.,
We obtain sparse, noisy measurements y:

---

Published as a conference paper at ICLR 2022
Moxuromett:
Edtnded Kalma (Sperttod)
(Spunkisd)
35-
Racurent Smusher 10us)
kdvves (207)
Rauu in Smocher 10us)
Recumive ther Livesized Smoothrr t0um
0.30
MSE
0.28
Ialoon Riw optmal
Recumnn (iberbor))
22
6.22
0.25
10
Sumples
Figure 4: Considers the linear dynam- ies experiment (section 8.1). The mean squared error on the test set (lower is better) as function of the number of examples.
Figure Considers the Lorenz experiment (sec- tion 8.2). The mean squared error the test (lower is better) as a function of the number of examples used for training.
we only use past observations. We compare against (1) the raw, noisy measurements which inher- ently deviate from the clean measurements, (2) the Kalman filter solution where we optimized the transition covariance matrix using clean data (hence supervised), (3) the optimal solution, which Kalman filter with ground truth parameters performing exact inference. To estimate the tran- sition maps for the Kalman filter, we use the standard Taylor series of A-Ar up to the first order. Additionally, we use this expert knowledge as an inductive bias for the recursive filter's transition maps.
In fig. 4 we depict the test mean squared error (MSE, lower is better) as function of the number of training samples. Given enough data, the self-supervised models approximate the optimal solu- tion arbitrarily well. Our recursive model significantly outperforms both the Kalman filter and inference model in the low-data regime by using incorporated expert knowledge. Additionally, report that the recursive model's distance to the ground truth latent states is much closer to the opti- mal solution than both the inference model and Kalman filter. Specifically, squared errors of 0.685 for the inference model, 0.241 for the Kalman filter, 0.161 for the recursive model compared to 0.135 for the optimal Kalman filter. Finally, it is worth noting that the recursive model has much less variance as a function of its initialization.
report average mean
8.2 LORENZ EQUATIONS
We simulate a Lorenz system according to
0
0
x
ã Ax
-1
0
(25)
82
0
-3
[33
We have H - I, R and E R The Lorenz equations model atmospheric convection and form classic example of chaos. Therefore, performing inference is much more complex than in the linear case. This time, we perform smoothing (see appendix A) and compare against (1) the raw measurements, (2) a supervised Extended Kalman smoother (L.jung, 1979), (3) the variational inference approach of Krishnan et al. (2017), (4) the supervised recursive model of Satorras et al. (2019). Our models include a recurrent smoother, a recursive smoother, and the recursive filter with linearized smoothing (section 6). Transition maps for the Extended Kalman smoother and the recursive models are obtained by taking a second-ord -orond- Taylor series of A-tt For the supervised extended Kalman filter, again optimize its covariance estimate using ground truth data.
In fig. 5 we plot the test mean squared error (MSE, lower is better) as a function of the number of examples available for training. It is clear that methods approach the ground truth states with
8

---

Published as a conference paper at ICLR 2022
Whitenonse
Doing the dishes
Dude mianwing
Exercise Bike
notse
Running
Combined
Kalmun Filier (Supervisod)
0.225
0.230
0.232
0.237
0.235
0.230
0.227
Noise/Noise (1 chiines al., 2018)
0:527
0.309
0.448
0.430
0.440
0.383
0.526
SIN (Krishaan et 2017))
0.297
0.345
0.342
Recurrent Filter (Ours)
0.102
0.207
0.213
0.200
0.234
0.175
0.181
Recursive Filter (Our)
0.107
0.206
0.213
0.196
0.232
0.166
0.175
Revursive Filter RTS Smoocher (Ours)
0.100
0.204
0.215
0.197
0.231
0.166
0.176
(Becker
Supervised)
Table 2: Considers the audio denoising experiment (section 8.3). Test mean squared error (MSE,
lower is better) per model and noise subset. Blue numbers indicate second-best performing models.
more data. This is in contrast to the Extended Kalman smoother, which barely outperforms the noisy measurements. We also see that the recursive models significantly outperform the recurrent model in the low-data regime. The recursive filter with linearized smoothing performs comparably to the other models and even better in low-data regimes. We hypothesize that this is because the required assumption for the linearized smoother holds (section 6) and regularizes the model. The variational method of Krishnan et al. (2017) performs poorly in low-data regimes. Most notably, the supervised method of Satorras et al. (2019) outperforms our models only slightly.
8.3 AUDIO DENOISING
Next, test the model on non-fabricated data with less ideal noise characteristics. Specifically, we use the SpeechCommands spoken audio dataset (Warden, 2018). Performing inference on spoken audio is challenging, as it arguably requires understanding natural language. To this end, recent progress on synthesizing raw audio has been made (Lakhooia et al., 2021). However, this requires scaling to much larger and more sophisticated neural networks than presented here, which we deem out the current work's 's scope. Therefore, we take subset of the entire dataset, using audio from the classes "tree", "six", "eight", "yes", and "cat". We overlay these clean audio sequences (m1 (... (N) } with various noise classes that the dataset provides. That is, for every noise class C' we obtain a set of noisy sequences ( yo Jc. We also consider a "combined" class in which we sample from the union of the noise sets. The task is to denoise the audio without having access to clean data. We evaluate the models on non-silent parts of the audio, as performance on those sections is the most interesting. Notably, none of these noise classes is Gaussian distributed.
(1)
We show the mean squared error on the test set of all models per noise class in table 2. Our models outperform the Kalman filter, Noise2Noise (Lehtinen et al., 2018), and SIN (Krishnan et al., 2017) unsupervised baselines. We suspect that the relatively poor performance of SIN is due to its genera- tive Markov assumption, regularizin the model too strongly. The poor performance of Noise2Noise is due to the fact that it does not use the current measurement y/ to infer Tg. Like before, note that the Kalman filter is "supervised" vised RKN (Becker et al., 2019) outperforms our models on most noise classes, but notably not on white noise. Most of these noise classes have temporal structure, making them predictable from past data. This is confirmed by observing these mean squared error values over the course of training. Initially, the values were better than reported in table 2, but the model increasingly fits the noise over time. Thus, although two of the main assumptions about the model (independent Gaussian noise) are violated, we still can denoise effectively. Since the RKN's targets are denoised (hence "supervised"", ), it does not have this problem. In practice, obtaining clean data can be challenging.
optimized its covariance matrix using clean data. The super-
CONCLUSION
We presented an advantageous parameterization of an inference procedure for nonlinear state-space models with potentially higher-order latent Markov chains. The inference distribution is split into linear and nonlinear parts, allowing for a recursion akin to the Kalman filter and smoother algo- rithms. Optimization is performed directly using maximum-likelibee gates through these recursions. Smoothing can be performed similarly, but we additionally proposed linearized smoothing that can directly be applied to the filtering distributions. Our model is simple and builds on established methods from signal processing. Despite this, results showed that it can perform better or on par with fully supervised variational inference methods.
objective that backpropa-

---

Published as a conference paper at ICLR 2022
10 ETHICS STATEMENT
The paper presents simple method to perform inference using noisy sequential data. Applications can be found throughout society, e.g., tracking particles, denoising images or audio, estimating system states. While many such examples are for good, there are applications with ethically debat- able motivations. Among these could be tracking humans or denoising purposefully corrupted data (e.g., to hide one's identity).
11 REPRODUCIBILITY STATEMENT
We are in the process of releasing code for the current work. For clarity and reproducibility, the presented methods are available as algorithms in the supplementary material. Furthermore, we made explicit wherever needed to make approximation or assumption.
REFERENCES
Evan Archer, Memming Park, Lars Buesing, John Cunningham, and Liam Paninski. Black box variational inference for state space models. arXiv preprint arXivv1511.07360 2015.
Philipp Bader, Sergio Blanes, and Fernando Casas. Computing the matrix exponential with an optimized taylor polynomial approximation. Mathematics, 7(12):1174, 2019.
Joshua Batson and Loic Royer. Noise2self: Blind denoising by self-supervision. In ICML, 2019.
Philipp Becker, Harit Pandya, Gregor H. W. Gebhardt, Cheng Zhao, C. James Taylor, and Gerhard Neumann. Recurrent kalman networks: Factorized inference in high-dimensional deep feature spaces. In ICMI., 2019.
Christopher M Bishop. Pattern recognition. Machine learning, 128(9), 2006.
Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Hol- ger Schwenk, and Yoshua Bengio. Learning phrase representations using encoder-decoder for statistical machine translation. arXiv preprint arXiv: 1078, 2014.
Andreas Doerr, Christian Daniel, Martin Schiegg. Nguyen-Tuong Duy, Stefan Schaal, Marc Tous- saint, and Trimpe Sebastian. Probabilistic recurrent state-space models. International Confer- Machine Learning, pp. 1280-1289. PMLR, 2018.
Marco Fraccaro, Simon Kamronn, Ulrich Paquet, and Ole Winther. A disentangled recognition and nonlinear dynamics model for unsupervised learning. arXiv preprint arXiv: 2017.
Kuzman Ganchev, Joao Graça, Jennifer Gillenwater, and Ben Taskar. Posterior regularization for structured latent variable models. The Journal of Machine Learning Research, 1 11:2001-2049, 2010.
Christian Gourieroux, Alain Monfort, and Alain Trognon. Pseudo maximum likelihood methods:
Theory. Econometrica: journal of the Econometrie Society, pp- 681-700, 1984.
Alex Graves, Abdel-rahman Mohamed, and Geoffrey Hinton. Speech recognition with deep recur- rent neural networks. In 2013 IEEE international conference on acoustics, speech and signal processing, pp. 6645-6649, lece, 2013.
Tuomas Haarnoja, Anurag Ajay, Sergey Levine, and Picter Abbeel. Backprop KF; learning discrim- inative deterministic state estimators. In NeurIPS, 2016.
Matthew J. Johnson, David Duvenaud, Alexander B. Wiltschko, Ryan Adams, and Sandeep R. Datta. Composing graphical models with neural networks for structured representations and fast inference. In NeurlPS, 2016.
Nal Kalchbrenner, Edward Grefenstette, and Phil Blunsom. A convolutional neural network for modelling sentences. arXiv preprint 1404. 2014.
10

---

Published as a conference paper at ICLR 2022
Rudolph Emil Kalman. A new approach to linear filtering and prediction problems. 1960.
Maximilian Karl, Maximilian Soelch, Justin Bayer, and Patrick van der Smagt. Deep variational bayes filters: Unsupervised learning of state space models from raw data. In ICLR, 2017.
Diederik Kingma and Max Welling.
Auto-encoding variational bayes.
arXiv preprint
arXiv:1312.6114, 2013.
Genshiro Kitagawa. The two-filter formula for smoothing and an implementation of the gaussian- sum smoother. Anmals of the Institute of Statistical Mathematics, 46(4):605-623, 1994.
Daphne Koller and Nir Friedman. Probabilistic graphical models: principles and techniques. MIT press, 2009.
Rahul G Krishnan, Uri Shalit, and David Sontag. Deep kalman filters. 15.001151 2015.
arXiv preprint
Rahul G. Krishnan, Uni Shalit, and David A. Sontag. Structured inference networks for nonlinear state space models. In AAAI, pp. 2101-2109. AAAI Press, 2017.
Alexander Krull, Tim-Oliver Buchholz, and Florian Jug. Noise2void learning denoising from
single noisy images. In CVPR, 2019.
Samuli Laine, Tero Karras, Jaakko Lehtinen, and Timo Aila. High-quality self-supervised deep image denoising. In NeurlPS, 2019.
Kushal Lakhotia, Evgeny Kharitonov, Wei-Ning Hsu, Yossi Adi, Adam Polyak, Benjamin Bolte, Tu-Anh Nguyen, Jade Copet, Alexei Baevski, Adelrahman Mohamed, al. Generative spoken language modeling from raw audio. arXiv preprint arXiv:2102.01199 2021.
Jaakko Lehtinen, Jacob Munkherg, Jon Hasselgren, Samuli Laine, Tero Karras, Miika Aittala, and Timo Aila. Noise2noise: Learning image restoration without clean data. In ICML, 2018.
Lennart Ljung. Asymptotic behavior of the extended kalman filter a parameter estimator for linear systems. JEEE Transactions on Automatic Control, 24(1):36-50, 1979.
Michael Lustig, David Donoho, and John M Pauly. Sparse mri: The application of compressed sensing for rapid mr imaging. Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine, 58(6): 182-1195. 2007.
Todd K Moon. The 47-60, 1996.
maximization algorithm. IEEE Signal processing magazine, 13(6):
Christian Naesseth, Scott Linderman, Rajesh Ranganath, and David Blei. Variational sequential monte carlo. In International conference on artificial intelligence and statistics, pp. 968-977.
PMLR, 2018.
Radford M Neal et al. Meme using hamiltonian dynamics. Handbook of markov chain monte carlo, 2(11); 2011.
Syama Sundar Rangapuram, Matthias W Seeger, Jan Gasthaus, Lorenzo Stella, Yuyang Wang, and Tim Januschowski. Deep state space models for time series forecasting. Advances neural information processing systems, 31:7785-7794, 2018.
Herbert E Rauch, F and Charlotte T Striebel. Maximum likelihood estimates of linear dynamic systems. AIAA journal, 3(8):1445-145) 1965.
Guy Revach, Nir Shlezinger, Xiaoyong Ni, Adria Lopez Escoriza, Ruud JG van Sloun, and Yonina C Eldar. Kalmannet: Neural network aided kalman filtering for partially known dynamics. arXiv preprint arXiv:2107.1000 10043, 2021.
Victor Garcia Satorras, Max Welling, and Zeynep Akata. Combining generative and discriminative models for hybrid inference. In NewrIPS, 2019.

---

Published as a conference paper at ICLR 2022
Bernhard Schölkopf, David W Hogg, Dun Wang, Daniel Foreman- Mackey, Dominik Janzing, Carl- Johann Cabonl, and Jonas Peters. Modeling confounding by half-sibling regression. Pro- ceedings of the National Academy of Sciences, 113(27):731-7399) 2016.
Richard A Thompson, James M Moran, and George W Swenson Jr. Interferometry and synthesis in radio astronomy. Springer Nature, 2017.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Eukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pp. 5998-6008, 2017.
Niklas Wahlström, Thomas B Schön, and Marc Peter Deisenroth. From pixels torques: Policy learning with deep dynamical models. arXiv preprint arXiv:1502.0220 2015.
Pete Warden. Speech commands: dataset for limited-vocabulary speech recognition. arXiv preprint wwXiv:1804.032000 2018.
Manuel Watter, Jost Tobias Springenberg, Joschka Boedecker, and Martin Riedmiller. Embed to control: A locally linear latent dynamics model for control from raw images. arXiv preprint 150xv 2015.
Li Yingzhen and Stephan Mandt. Disentangled sequential autoencoder. In International Conference on Machine Learning, pp. 5670-5679. PMLR, 2018.
A PARAMETERIZED SMOOTHING
In the main body of the paper, we showed how to parameterize the model for recursive estimation of p(a; ysk)), k 0, ......A K. Additionally, we provided linearized smoothing procedure that yields p(ax V1.K). The disadvantage clearly is the lincarization. Here, we show how we can recursively estimate p(æx yl.k) in similar sense to the filtering setting.
First, we put
p(ak
x 1, 1-) N a% FRI -K- + ēuj-k Qm)
(26)
where FRI- k) and (g and (u)- are two-sided recurrent neural network out- puts similar to section 7. We compute the distribution of interest as follows:
p xx xx)p((F
3-*)
p(æx Book)
(27)
p(yk
k-k)
x p
xx)
p( plak y- y=
(28)
= -p(y
xx))
p(zc oo
y x=
(29)
-=- 1h l
plyx
p(ax 1 y ())( yo:k )or
(30)
where we make the approximation to compute eq. (27) efficiently and recursively. It is justified I (#==1; Ws y= < e for small e. That is, the additional information that conveys about #;-1 marginal if we have all other data. Let
p( yo:k) == N (I 1 P=-1)o-k)
(31)
be previous time-step's posterior. Then put
p(ak Ft- y-*) = (a P + ex Qu)
(32)
where we left out the arguments for the following quantities estimated by an RNN.
y k (y-x)
h h
= GRU (:
é (V)+)
= f(hz 1
(33)
1k+))
Q k (v
12

---

Published as a conference paper at ICLR 2022
Applying the integral in eq. (30) get
J p(æx xx-11, y y/=))///pppp. (xp
yo.k ) = =N (Zxa) [Kk| PR)-a)
(34)
where we put
/ = - + @xI-k
Pk|-k:= == P Fx/-x + Qu1
(35)
A data likelihood can be computed follows
p y-
p(yx xx)p(< 1ol y=x)dx
(36)
p(yx aw) / p(ox-11.02 1 y
(37)
plyk mxn / plara x 1 v=))da
(38)
1ullv- 1y-k
{ pleu x&)) / p(ark 0k-1 9
yo:k dd
(39)
where we made the same approximation eq. (30) as before. evaluates to
/ p(yx wx)p(arx y= = N ~N Øk1-k: MA(-K)
(40)
with
- == -
= + R
(41)
For fitting to the data we now would use a maximum-pseudo-llkee (Gourieroux et al., 1984) approach by maximizing:
K log.N yk कx| -x)=((y)))) (g)- H + R).
poyp y-k) = > O
(42)
1mm0
leading to minimizing the following self-supervised loss function:
(D yx) MMLa (DK) - 4kk + log det M
(43)
where DkI-k = H Φx x and M H ( H +R.
B GAUSSIAN CONDITIONING FORMULAS
Since many of the calculations used in this work based on the Gaussian conditioning formulas. we provide them here. If
p(z) = Nax P). p x)) Nyy Hæ + R),
(44)
(45)
then
p(y) = N(y R R ), p( y) = N (x Σ H R (y b) + P P-1 Σ),
(46)
(47)
with
E Σ=
H R H
(48)
13

---

Published as a conference paper at ICLR 2022
C DIRECT PARAMETERIZATION OF p( yeck)
We show here how to directly parameterize p(a: youk and p(æ y y-*)- This parameterization is referred to the recurrent model in experiments. The procedure is rather straightforward. For filtering, we put p(æx yek) = N (a2 ( (o)ea(<**)) We model:
E Li
HR <k == Ck.
PRICA: == L
= f f(hx"),
(49)
where Lg is a cholesky factor and f a multi-layee perceptron. The argument h is recursively given by
h
GR
(50)
where we employ a Gated Recurrent Unit (GRU) network (Cho et al., 2014). Note that this param- eterization is equivalent to the model described in the main paper with F2 == 0.
For smoothing, p(æk
y y=u) = N
( x), (x)- x))
2 -k Ck:
PaI-K = L Lk
ex LR
= f(hE,ht)
(51)
where La is cholesky factor and f is multi-layer perceptron.
h GRU(B--1, 1x=11)
h == GRU(U hb41).
(52)
Once p(æ) yc*) (filtering) or p(æ: y-k) (smoothing) is obtained, all the procedures for inference and optimization described in the main paper and appendix remain the same.
D BIAS- VARIANCE-NNOSS DECOMPOSITION OF THE SELF-SUPERVISED GENERALIZATION ERROR
Any estimate Zx zx (y=k) for æ& that is not dependent yk will give us -ias-variance- noise decomposition of the generalization error. Note that this setting covers both the filtering and smoothing case. Define "optimal model" y E [yk y-k], then under the specified generative model (section 3) we have
E
|
yk 12 y y-k] =E 110 DE
0%
yx y-k
(53)
EE
|1
12 y-k
Var (y
y- n]
y
21E
(ixx 91))
(0% yv)
(54)
= 1 - 2ズل2 + Var [yrk
y x- + (g) 9it)
E [00 yu) y-
(55)
Var [3
|1 H r2xl12 リー
(56)
E
|
H
v-
E || yーk
2E (Ha 4i) n2
リーk
(57)
22.
E |1
H y-k + tr(RR
(58)
Thus,
E
y-
y
= 11
Uāll IE |1 Hævll 1l リー
tr(RR)
(59)
Note that g
E H
y-
HE
[x
y= Then, define our model ÿk = H The
reducible part of the error becomes
H E Bl [--a] - -
(60)
14

---

Published as a conference paper at ICLR 2022
For this reason, the model output 2y approaches the optimal model under minimization of the self- supervised error (perturbed by H).
Additionally, the model ÿv approaches Hæ (the uncorrupted measurement) under this criterion.
E
1 yxll
E
|10:
H + Hæ
yy:ll"
(61)
E || ii Hæ 11
+ Ha ykll
2 (üs Hxx) _ (Hærk yı))
(62)
E | Hææl
122
+
Ha yx|l2
2 [( H
E Hax yk
(63)
= E
0 H
+ tr((R) + E
H yv III
(64)
If we then take DR: == H then the reducible part of the error approximates the true 2y. (perturbed by H).
E 11 H 11 = =E || (ix xz)1|]
(65)
15

---

Published as a conference paper at ICLR 2022
E ALGORITHMS
Algorithm 1: Recursive Filter (Inference)
input Data (time-series) Yo:k = (yo...... yk emission matrices H and R, parameters ø. output:
For training: Loss value Kk and its gradient VoLk.
2. For inference: BRisk and PAISH for & in O,...,, K. Inference is done via:
p(ax
Ugkk) N z SK Ps sa).
3. For linearized smoothing (section 6): Fejck, PRICAK Eajsa and Paiga, for le in O,..., K.
4. For forecasting: के (K+]]<(++) and <(++))(+))) Forecasting is done via:
p(øK+1 yo:k) / xK+1 (K+]]<((+)) Pk-1))(kt))))
ho 0
C-1 = 0
Poico = Qo
Tp <u éo
for k in O,
K do
Buick = ( PH H + R2) -1
Uu(c H TRtck
LE L + (y) - (m((c)) BeteR (y (u((e) log det B₁ <k
K2 PUICH H BRiek
PRISA == PAIEA K2 H PRicc
SREs SMICK + Ke (yk - ŪK(ck)
h G yu)
((+11)(()1)
((41)(())))
= fo((+)1)
[
Qx+1|<(k+1) ((11)(()))) D
PRtII<(H+)) = P PAISK AT (+ + Qx+1)<(4)))
(141K(())) FetHI((+))) BRick + ((41)(()))
end
For the training case we use backpropagation through the above loop to compute V VoCk.
Algorithm 2: Recursive Filter (Training)
input Data ttime-series) Yo:k = (yo..... yK ). emission matrices and R, initialized parameters dv, number of training rounds J.
output: Model parameters o² for inference at test-time. for i in O...., do
Obtain C and V (() from algorithm 1.
Run preferred optimizer step update parameters & with VoCk (and cQ)
end
16

---

Published as a conference paper at ICLR 2022
Algorithm 3: Parameterized Recursive Smoother (Inference)
input Data (time-series) yo:k (yo..... yk). emission matrices H and R, initialized parameters do
output:
1. Loss value LK and its gradient w.r.t. all model parameters Vo for training.
2. For all & in .,..., UKJo:K. Pajo.K. These can be used for inference through K: p( yo:k) = N (o.aaao ( Pkjo:K).
ho 0
h K++1 =
( 0
Po -0-0- = Qo
To -0 = Co
for k in .... do
B B =
H PH H + R
Uk1 == H 2k|-
c c (y)s [k)
Bay- (yk (u)-a) log det Bg -k
K₁ P H B
P - Pa|-K k K2 H PkI
k - + K2 (yI kk)
h == GRU, (hF², , 3kk
h = GRU (h y
(x+11-(6+1))
(k+1)-(6+1)
= fo , h
(x-11-(++1)
Qx+11-(R+1) bx411-(0+1) ET -0+11-041)
Px Ph+1/-(k+1) (x41)-(4+1) Pajo:K Fy F F++11-(H1)) Qx+1)-(4+1)
2 (x+1)-(b+1) F8411-(0+1) k.ookK + (x+1)-(k+1)
end
For training case we also use backpropagation through the above loop to compute VoLk
Algorithm 4: Parameterized Recursive Smoother (Training)
input : Data (time-series) Yo.k = ..o. УK). emission matrices H and R, initialized parameters do, number of training rounds output: Model parameters o* for inference at test-time.
for I.....I
Obtain Vock from algorithm 3.
Run preferred optimizer step.
end
17

---

Published as a conference paper at ICLR 2022
Algorithm 5: Linearized Smoother
input
Values of: FRI-Ak PaIck- PRISA SMISH for all 0,..., y K, obtained from the recursive filter algorithm.
output: Linearly smoothed distributions p(ats yo:k) = N (ass) ERs Gv.) for
== O....,,. K.
k == SKISK
Gx PKSK
for k K,..., .1 do
J 111 Pk-1|Sk-1 FRlck PMIck
GE-1:- = -P Jn-1 (PaSS PAICE) kk-1kk
-- tb 1|sk1 + J 1/6 (&t @(((k)
end
Algorithm 6: Recurrent Smoother (Training)
input :
Training data (time-series) yo:k = (yo..... уK), emission function ((H (Hæ R). initialized parameters de, number of training iterations n.
output: Optimized parameters o*
for in / to n do
hg : 0
h :
(() =
for k in to K do
h == GRU, (1 11-
b = GRU 9k+1)
ig x
i [ = fo h,r))
P = Lıj- x
Ukl-K = H
BRI-K = (HPs + R
C - + (y) x))=)) B (3 ())-)) log det B
(1) (1)
end
Compute V and apply SGD step with respect to all model parameters, which amounts to backpropagation through the above calculations.
end
For the filter variant, the steps that involve the backward direction (+-) are left out.
18

---

Published as a conference paper at ICLR 2022
Algorithm 7: Recurrent Smoother (Inference)
input : Test data Yo:K (Mo...., 9k). trained parameters o", emission function A (Hæ, R). output: Inferred posteriors p(z Yo:k) = N(xx @xjo:K: Pajo.K) for all k:
ho 0
h = 0
for k in 0 K do
he == GRU (h 9k-1)
h = GRU (h (+
fo ha²)
L [
PRI_a = L -
K. = P (HP + R
&.jo.K = t2 -k Kuly H
PHO.K= = P H.HPP. PRI-k
end
For the filter variant, the steps that involve the backward direction (+- are left out.
19

---

Published as a conference paper at ICLR 2022
F EXPERIMENTS: DETAILS
F.1 LINEAR DYNAMICS
As specified in the main paper, the dynamics according
0
0
Aa
-c
V
(66)
-r
0_ La
Since these are linear transitions, we can calculate any transition directly using æ(t At) xt).
c eq
Q
0 Q
F ==
Q
(67)
0
We used e = 0.06, r = 0.17, At = 1 and covariance
Q = 0.12.
(68)
3
The matrix exponential is computed using Bader et al. (2019). The parameters for the emission distribution:
H
0
R = 0.52
0 111
(69)
We simulate K = 131, 072 trajectory for training, == 16, 384 trajectory for validation and K := 32, 768 for testing. The F that used in the (non-optimal) Kalman filter and recursive model computed as follows:
ASt 2 F = Σ (AIA")/n!
(70)
F.2 LORENZ EQUATIONS
We simulate a Lorenz system according to
0
0
31
Ax
P 31
-1
32
(71)
32
0
8
.33
We integrate the system using di = 0.00001 and sample it uniformly at A/ = 0.05. We use p = 28, 10, 8/3. The transition in At arbitrary time-steps is linearly approximated by Taylor expansion and used in the Kalman smoother and recursive models.
Σ AIA l |A) )" 7L
(72)
We simulate K == 131, 072 steps for training, K = 32, 768 for testing and K = 16, 384 for validation. We have : and thus x E R and y E R use R :== 0.5º
G PARAMETERIZED SMOOTHING: ALTERNATIVE POSTERIOR EVALUATION
We would like point out that the distribution p(æ; Yo:k) be obtained without making assumption eq. (30), which we stretch out here. Initial experiments showed that using these caleu- lations the model did not converge smoothly when using the stated before. However, could be of interest to further investigate. Returning to the posterior of interest
p( Yo:k) = / p(æx Tk-1 Ho::) p(x Yo:K) dx
(73)
/
p(ya ax))
p(ark [ -- (-l-,, ())( Vo:k) da
(74)
p(ui Ik-1, y-
20

---

Published as a conference paper at ICLR 2022
We have
p( yo:k) =N ( 1x-1 Pa-10xx))
(75)
the previous time-step's posterior.
-V (arx F ExI Quil-))
(#t 0k-1, 1, y
k)) g-k)
(76)
where (V éx ( and 2x are estimated by neural network. Combining this with noise model eq. (3) we get:
p(ak (o.Kk
= a - IK-1 + CxI + K (yk H (Fuj- x6-1 + ex)= K₁ H
QuI-R
(77)
= (K (I Kn H) (Fil- xK-1 + ēx)- + K2 yk (I K, H) Q -b):
(78)
where we directly applied the Woodbury matrix identity obtain Kalman gain matrix
K₁ = QuI H (HQ H + R
(79)
Then, applying the integral we get:
p (æ yo:k) = =
(2z NN(2x @xjo:k Pa(O.K)
(80)
with:
Pkju:K - Kn, H) FRI Pk-1j0:K - (I Kn H) + + K,, H) QRI
(81)
and
kbjo:K k + éx/= + K2 (yj H - ēxl
(82)
= (I K H) (F ut 0:K + ex) + K yk.
(83)
21


Reviews:
[{"review_id": "oD7YzSeWOy3", "reviewer": "Reviewer_SVPv", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Major:\n1) The authors keep calling the Kalman filter (and other variants such as the RKN) a 'supervised' method, but these methods do not need a 'ground truth' latent variables. They do, however, need a set of observations and a set of model parameters. It would be good for the authors to clarify what they mean and exactly what the differences are in their approach. \n2) Needs more validation on real datasets, and more visualization for how it performs on the one real dataset that the authors did use. The RKN is seen to perform better than the proposed approach- is this also true for forecasting? What is the comparison between the estimated states x from different models?\n3) The authors need to clarify the utility of their 'prior knowledge' - how much of the recursive filter results are due to this prior knowledge as opposed to the Bayesian filtering? Please separate out the contribution of the prior knowledge from the recursive filter.\n4) The figures could be made much better. Figure 2 could apply to any state space model; the novel ideas take a while to get to.\n\nMinor:\n1) Please detail the difference between 'recurrent' and 'recursive' earlier, when you first introduce the terms in Figure 1. I would also recommend changing the terms to terms that are not as similar.\n2) Please bold the values that perform the best in Table 2.", "summary_of_the_paper": "The authors describe a parametrized inference approach for nonlinear dynamical models with linear observations. They borrow ideas from the well-known Kalman filter, and use Bayesian filtering and smoothing approaches to recursively estimate the states of the nonlinear model. The parameterization makes use of a locally linear transition between subsequent states using neural networks; thus the authors are still able to perform the Kalman-like filtering and smoothing.", "summary_of_the_review": "The ideas in the manuscript are novel, but clarification is needed as stated in the Main Review. The figures need work. Moreover, additional empirical novelty, either further evaluation on the datasets already examined as a part of this manuscript, or on other datasets, is recommended.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "bo2zEEN643Z", "reviewer": "Reviewer_7PDK", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "strengths:\n- The authors proposed a new inference model, and demonstrated the effectiveness of their method through solid quantitative comparisons in the experiment section. The paper is clear written.\n\nweakness:\n- Is it possible to show plots on the estimated latent trajectory for the real audio denoising data? (e.g. plots like fig.1 but for real data) I'm curious about what are the latent states look like for different noise groups? Are there any qualitative differences among the latent trajectory estimated for different methods besides the quantitative differences on MSE?\n\n", "summary_of_the_paper": "The authors proposed a new inference method for state-space model with nonlinear latent states. They showed that their inference model can be incorporated with domain knowledge and optimized in a self-supervised manner. They also demonstrated that their model can achieve competitive results through both simulated and real audio denoising tasks.", "summary_of_the_review": "The authors clearly state the contributions of their paper compared to the previous literature. The proposed inference method is novel, and the math derivations look solid. And the quantitative comparisons in the experiment results look competitive. Overall, I recommend this paper to be accepted.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "H6K1aQA1XOj", "reviewer": "Reviewer_d1x8", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I found the idea and its connection with the noise2noise approach interesting. However, I have concerns about the empirical evidence of the work and some comments on the structure of the paper:\n- Given that this area is an active area of research with a relatively large body of related work, I believe the related work deserves a separate sub/section. The current version of the paper has a list of related work in a long paragraph at the beginning of the paper. I suggest you move this to a separate section and provide a better overview of the relevant work (see “KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamics” for an example).\n-  Noise2noise method has been mentioned and used as a baseline, but no description has been provided for this related idea. Adding a few sentences or a subsection in the appendix can help the reader unfamiliar with noise2noise to connect the works better. \n- The “KalmanNet” paper, mentioned above seems to be closely related to the paper. As mentioned by the authors of KalmanNet, it can also be trained in an unsupervised fashion. Can you add the work to the \"related works\" section and describe the possible advantages of your recursive approach over theirs? \n- The experiments provide evidence for 1) model working in simple synthetic examples (passing sanity tests) and 2) model performing better than KF and EKF in both synthetic and real dataset. However, a more extensive comparison with other unsupervised methods is recommended. Given at least 9 other unsupervised methods in Table 1, adding at least one more unsupervised baseline to the audio denoting experiment can be helpful. If none of these methods is applicable to the experiment, justification can be provided instead. \n- The performance of SIN (Krishnan et al.) in figure 5 is a bit surprising to me. I’d expect a more gradual decline in MSE instead of a sharp drop. Can you provide some explanation for this behavior?  ", "summary_of_the_paper": "The paper proposes an approximate inference framework for state-space models with nonlinear latent dynamics. The approach uses the classical Bayesian update rules with neural network for estimating the parameters of local linear transitions.", "summary_of_the_review": "I found the proposed approach interesting; however, I think the paper can be further improved in terms of format and empirical evidence. \n################### POST-REBUTTAL COMMENTS ########################\n\nI found the authors' response adequate; hence, I'm adjusting my score. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "FmciCb-ZQS", "reviewer": "Reviewer_q53Q", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The manuscript's proposal is reasonable and interesting, but I have concerns about novelty. Also, I found some of the discussion a bit confusing, and the overall organization of the paper can be further improved. \n\nBelow are more specific comments : \n\n1. Introduction : The opening paragraph suddenly loads a lot of references to the reader, without really putting them into a context. I would suggest selecting some of them and going into a bit of a more detail, if they are helpful in describing the approach put forth in the manuscript.\n\n2. Introduction : I was also expecting a more concrete description of the proposal in the introduction. You are proposing to replace some of the parameters/matrices that occur in the Kalman filter with neural networks, which is perfectly fine, and is a simple idea (simple=good!). It would help to show the Kalman filter formulation (i.e., what it's solving), and outline what you're doing differently, without having to go into heavy notation.\n\n3. Introduction : in the list of the contributions, for the first item : can you make that into a precise statement (like a proposition) and refer to it in the paper, so it's clear what you're showing? Are you referring to the development in Section 3? If so, I don't see what's novel with that development (other than replacing the said matrices with observation dependent estimates obtained through neural networks) -- can you elaborate?\n\n4. Eqn 6 : What's the difference between the first line and the second line? Are you just replacing $<k$ with $\\leq (k-1)$? If so, I'd suggest removing this equation. In fact, since $<$ and $\\leq$ appear as subscripts, it may be good to stick to either one of them, so as to avoid confusion.\n\n5. As I noted above, I think the proposal is simple, but I'd expect that training the neural networks might be a challenge in practice (if not, that's great). However, the idea reminded me of the paper titled \"AI-IMU Dead Reckoning\", by Brossard, Barrau, Bonnabel (https://arxiv.org/pdf/1904.06064.pdf) -- see for instance the comments right before section III.A. Can you point out the departure of your idea from what Brossard et al. do in their paper?\n\n6. Another reference, which might be good to place the proposal into context is \"Discriminative training of Kalman filters\", by Abbeel, Coates, Montemerlo, Ng, Thrun, 2005. That work is certainly different from what you're doing, but also aims to find a good fixed set of covariance matrices from data.\n\n7. Audio Denoising Experiment : Is it possible to train the network on a portion of the signal, and then stop training, so as to switch to \"test\" mode? More generally, I found the description of this experiment a bit terse. I'd welcome it if you can give a more detailed description, possibly using equations to mark what the input/output or the noise term etc. is.   ", "summary_of_the_paper": "A Kalman filter typically requires specifying a state update equation, along with noise covariance matrices for state transition and observation. Determining these can be a challenge in certain scenarios. The manuscript proposes to replace these unknowns with trainable neural networks. The input to the neural networks are the observations, and the output are the unknown parameters (or their factorizations, to ensure positive definiteness). The manuscript validates the approach through numerical experiments.", "summary_of_the_review": "Overall, the manuscript discusses an interesting approach, and has wide application potential, given that it promises to facilitate the tuning of Kalman filters. However, I have concerns about novelty. I also think the paper could be better organized.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper presents a method for inference in state-space models with non-linear dynamics and linear-Gaussian observations. Instead of parameterizing a generative model, the paper proposes to parameterize the conditional distribution of current latent states given previous latent states and observations using locally linear transitions, where the parameters of the linear mappings are given by neural networks. Under fairly standard conditionally-independence assumptions, the paper uses known Bayesian filtering/smoothing tricks to derive a recursive estimation algorithm and a parameter-estimation method based on a simple maximum likelihood objective.  

Overall, the reviewers found the idea to be novel and interesting and I agree.  They also found the relation to the noise2noise objective worth highlighting. Several concerns were raised during the discussion period, which I believe the authors addressed satisfactorily. However, I think the authors should bring the assumed distinction between ‘supervised’, ‘self-supervised’ and ‘unsupervised’ upfront, as usually these types of models are trained using the noisy data (to which the authors refer to as unsupervised). 

Given the large body of literature on dynamical systems, filters and smoothers, I believe the paper will benefit significantly from more comparisons across a wider range of (and more realistic) datasets.
Decision: Accept (Poster)