Deep Bayesian Filter for Bayes-Faithful Data Assimilation
Authors: Yuta Tarumi
Keisuke Fukuda
Shin-ichi Maeda
Venue: ICML 2025 poster (2025)
Keywords: Data Assimilation
Variational Inference
State-Space Models
Koopman Operator
Primary area: probabilistic_methods->variational_inference
TL;DR: proposed a novel data assimilation method that achieves the best performance for nonlinear state space models

Abstract: Data assimilation for nonlinear state space models (SSMs) is inherently challenging due to non-Gaussian posteriors. We propose Deep Bayesian Filtering (DBF), a novel approach to data assimilation in nonlinear SSMs. DBF introduces latent variables $h_t$ in addition to physical variables $z_t$, ensuring Gaussian posteriors by (i) constraining state transitions in the latent space to be linear and (ii) learning a Gaussian inverse observation operator $r(h_t|o_t)$. This structured posterior design enables analytical recursive computation, avoiding the accumulation of Monte Carlo sampling errors over time steps. DBF optimizes these operators and other latent SSM parameters by maximizing the evidence lower bound. Experiments demonstrate that DBF outperforms existing methods in scenarios with highly non-Gaussian posteriors.

Deep Bayesian Filter for Bayes-Faithful Data Assimilation
Yuta Tarumi' Keisuke Fukuda'
Shin-ichi Maeda
Abstract
systems, such as those used weather forecasting, rely on methods like the ensemble Kalman Filter (EnKF) (Evensen, 1994; Bishop et al.. 2001) for sequential state filtering (i.e., p(zelo1:)) and the four-dimensional variational method (4D- Var) for retrospective state analysis (i.e., p(z) (1:1), ) < T). These approaches assume Gaussianity in their test distri- butions (1:r) or ( (1:T), a simplification driven by computational constraints. While exact methods such as bootstrap Particle Filters (PF) or sequential Monte Carlo (SMC) (Chopin & Papaspiliopoulos, 2020; Daum & Huang, 2007; Hu & van Leeuwen, 2021) could compute the true posterior, their performance degrades significantly when the number of particles is insufficient (Beskos et al., 2014). This issue is exacerbated in high- digensiimnsio systems, making SMC approaches impractical for many physical problems.
Data assimilation for nonlinear state space mod- els (SSMs) is inherently challenging due to non- Gaussian posteriors. We propose Deep Bayesian Filtering (DBF), novel approach to data assimi- lation in nonlinear SSMs. DBF introduces latent variables h2 in addition to physical variables 2z. ensuring Gaussian posteriors by (i) constraining state transitions the latent space to be linear and (ii) learning Gaussian inverse observation oper- ator (hylor) This structured posterior design en- ables analytical recursive computation, avoiding the accumulation of Monte Carlo sampling errors over time steps. DBF optimizes these operators and other latent SSM parameters by maximizing the evidence lower bound. Experiments demon- strate that DBF outperforms existing methods in scenarios with highly non-Gaussian posteriors.
To address these limitations, we propose a novel variational inference approach called Deep Bayesian Filtering (DBF) for posterior estimation. Our strategy consists of two main components: (i) constraining the test distribution remain Gaussian to ensure computational tractability, and, in cases where the original dynamics are nonlinear, (ii) leveraging a nonlinear mapping to enhance the expressive capability of the test distribution.
1. Introduction
Data assimilation (DA) is a crucial technique across various scientific domains. Its objective to estimate the current state and the trajectory of a system by combining partially informative observations with dynamics model. Specifi- cally, given a series of observations T time steps 01.7, the goal is to infer the posterior distribution of the system's ss physical variables 2e: p(2)(o1=)) DA has been widely ap- plied in fields such as weather forecasting (Hunt et al., 2007; Lorene, 2003; Andrychowicz et al., 2023), ocean research analysis (Ohishi al., 2024), sea surface temperature pre- diction (Larsen et al., 2007), seismic wave analysis (Alfonzo & Oliver, 2020), multi-senson fusion localization (Bach & Ghil, 2023), and visual object tracking (Awal et al., 2023).
Linear dynamics When the system's 's dynamics operator in p( (2)) is linear, DBF introduces the concept of the in- verse observation operator (IOO; see also Frerix et al. 2021) to construct Gaussian test distributions ((z))01z). The I0O and any unknown system parameters are trained to minimize the Kullback-Leibbe divergence between the test distribu- tion
and the true posterior p(2) (01:). The training maximizes the likelihood of the observed time series and therefore does not require teacher signals 21.
Nonlinear dynamics In the more common case of non- linear dynamics, DBF operates in a latent space, assuming Gaussianity in the latent variables ht. The original phys- ical variables are recovered through a nonlinear mapping function o, implemented via neural networks (NNs). This nonlinear mapping allows for more flexible representation of the test distribution q(=x/o1=). The 100 and other pa- rameters are trained in a supervised manner (i.c., is used during training).
A key challenge in DA arises from the non-Gaussian nature of the posterior distributions (())) which results from the inherent nonlinearity in both the system dynamies and observation models. Despite this, many operational DA
'Preferred Networks, Inc., Japan, Correspondence to: Yuta Tarumi <yuta.tarumi @ riken.jp>.
Proceedings of the nnnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
For state space models (SSMs) with nonlinear dynamics,

---

DBF for Bayes-Faithfuu DA
DBF functions as variational autoencoder (VAE) that ad- heres to the Markov property, expressing the posterior distri- butions of latent variables h, within a Bayesian framework. As subclass of dynamical VAEs (DVAEs, Girin et al. 2021 for a review), DBF leverages the VAE structure to model time-series data while distinguishing itself through its pos- terior design. Unlike other DVAEs, where Monte Carlo sampling required for inference (see Sec. 2.6.1), DBF recursively computes posteriors via closed-form analytical expressions, eliminating the need for sampling during the inference. Additionally, DBF can be interpreted as learning the Koopman operator (Koopman, 1931) using NNs. The discovery of such latent spaces and operators through ma- chine learning has been extensively studied (Takeishi et al., 2017: Lusch al., 2018: Azencot al., 2020) and will be experimentally validated through the handling of nonlinear filtering tasks involving chaotic dynamics.
M(=+i1i(( Q). where (a Σ) denotes Gaussian whose mean and covariance are μ and E. The nonlinear function f is the dynamics operator and Q is the system co- variance. The Markov property holds, as 21+1 depends only on 2t- An observation model p(o: |zz) = .N(r; h(zz); R) relates observations to physical variables via the observation operator and covariance R. The objective of sequential DA is to compute the posterior of 24 given 01:--
2.2. KF for Linear Dynamics, Linear Observations
In the KF, the dynamics and observation models are both linear Gaussian. Given that the dynamics and observation operators f. are linear, can represent them using matri- ces A and C, respectively, All matrices (A, C, Q. and Rt) are constant. The filter distribution p(za|o (1:))) remains Gaussian, provided that the initial distribution p(z: ) is Gaussian. We can recursively compute the posterior parameters (means 14: and covariance matrices Σ.) using the following equations:
In summary, our key contributions are as follows:
1t = IA + +Q= Apt-
DBF offers a novel -Bayes-Faithffe approximation for the posterior within the dynamical VAE, following the inference structure of an SSM with the Markov property.
+Kil0: HAH(-1);
(1)
E (A +Q) + HR-
(2)
where K; = (AΣ,-1A +Q)1I" (H(AAE-IA +Q)!*+++
For systems with linear dynamics, DBF extends the Kalman Filter (KF) to handle nonlinear observations through learnable NNs. The training process enables the model to infer unknown system parameters directly from data (see Sec. 3.1).
R 1-1 is the Kalman Gain.
2.3. DBF for Linear Dynamics, Nonlinear Observations
In this scenario, Gaussianity of the test distribution is lost during the KF update step. We introduce an inverse obser- vation operator (IOO) r(2 (see also Frerix et al. 2021):
For nonlinear dynamics, the posterior is maintained as Gaussian to ensure computational tractability while incorporating nonlinear transformations, allowing rep- resentation of a wide class of posterior distributions (see Sec. 3.2 and 3.3).
r(2 (0r)
p(z() x p(=))
(3)
)((=)=)))(=3) and p(z+) is a prior where ro)))))) p
virtually introduced for the I00. By approximating both the I00 and the virtual prior as Gaussians, r(2+(or) = N (fe(o))) Go(or) and p(z)) = (m,, V). respectieely, the posterior q can be analytically computed as a Gaus- sian, where the mean Pe and covariance E, are given as:
As a generative model, DBF estimates the uncertainty of the physical variables 2,, in contrast to 3D- and 4D- Var, which yield only point estimates (see Sec. 3.2 and Fig. 3).
The linear constraint on dynamics stabilizes the train- ing process, which is known to be unstable in standard recurrent NNs (see Sec. 3.3 and Fig. 6).
Pt = E,(AE-1A* + -1
+Golo.)- - -V-
(4)
E,1 = -AEE,IA' + -1+ -G
DBF has demonstrated superior performance over classical DA algorithms and latent assimilation methods in scenarios with highly non-Gaussian posteriors.
where fo(ot) and Go are NNs with parameters 0, and m and are constants set to m = and V 10 These values bias the NNs' outputs without affecting performance. The initial distribution ((2)) is taken to be Gaussian with i = 0 and Σ; 100/. With the Gaussian 100 r(zx in place, the recursive update in Eqs. 4 and 5 remains fully analytic.
2. Method
2.1. Physical Variable Inference in a State-Space Model
A physical system is defined by variables 24, with its evo- lution described by the dynamics model p(2441=+)) =
The recursive formula for the exact posterior (Equation 3) requires no approximation. Thus, DBF computes the exact
2

---

DBF for Bayes-Faithful DA
(a)
(b)
of observables g(=), such that Kg((z) = g(f(z)). This reformulates the system as hy41 Ah, in the latent space, where A is the dynamics matrix learned by DBF, While the physical dynamics f() are nonlinear, the Koopman oper- ator ensures the existence of an embedding that linearizes the dynamics, enabling recursive computation of test distri- butions. Discovering such embeddings finite dimensions has been widely studied (Takeishi et al., 2017: Lusch et al., 2018; 8; Azencot al., 2020). In high-dimensional simula- tions, the true degrees of freedom are often far fewer than the simulated variables, making surrogate modeling with the Koopman operator promising approach to reducing computational costs.
P:13
ht
1
A.
Q
h
A,
Q
h
r(h+ 100
4
4-1
Zt
Zt
+++1
2L-1
OL42
Or-1 Or 0c+1
Figure 1. Panel (a) shows the graphical model for the SSM as- sumed for SSM with nonlinear dynamics. Panel (b) shows the inference structure of our methodology for SSM with nonlinear dynamics.
2.5. Training
When assimilating in the physical space (i.c., when the dynamics are linear), we train the I00 (i.e., fo and Gg) by optimizing the evidence lower bound (ELBO) without using the physical variables 2, in data:
posterior when the true 100 Turue (z) is Gaussian, i.e., the SSM is a Garessin State Space model (LGSSM). In that case, the posterior update formula agrees with the KF (see Equations 1, 2 and 4, 5). The key difference is that non- linear functions are applied both the mean, So (ox), and the covariance, Go(o.). In the KF, fo(o:) is linear, and Go(oz) is a constant matrix (see Equations 1 and 2). Go(o:)'s depen- dence on observations allows flexible adjustment of the new observation's impact on state estimation. The importance of adjusting the internal state updates based on observations has also been discussed in recent SSM-based approaches (Gu & Dao, 2023).
logp(on:))
log logp(orlona-1) LFLBOO
Σ / log
LELBO
+KUlg(ajooua)lll((zloono
(6)
where KL[pllq| denotes the Kullback-Leii divergence
between distributions p and q (see Sec. A.I in the appendix
for the derivation). Here, q(21 l01:0) {(2)) is the initial distribution. If the SSM contains any unknown parameters, we can train these parameters as well.
2.4. DBF for Nonlinear Dynamics, Linear/Nonlinear Observations
In this scenario, the Gaussianity of the test distribution is lost during the predict step, making it impossible to apply the original dynamics over the physical variables 2, Therefore, we introduce a new set of latent variables h4 and assume a dynamics model over he p(ha+alla (he) = (hh | Q) (see panel (b) in Fig. 1). The I0O maps observations into
For SSMs with nonlinear or unknown dynamics, we have two approaches:
Strategy 1 Pretrain the Koopman operator, which consists of the nonlinear mapping from z₁ to ht, the linear dynamics between he and h4+1 represented by matrix A, and the re- verse nonlinear mapping from h, to 2, denoted by ø. With these components (A and Φ) of the Koopman operator, the method designed for linear dynamics can be applied. For pretraining, we require samples of 2, or the SSM for the physical variables to generate these samples. Pairs of 2, and og are not necessary, as the training for the linear dy- namies (A and Φ) and the IOO (r(hyloz) can be performed separately.
the latent variables hG: r(hyloe), The recursive formula
follows Equations& 4 and 5. To retrieve the distribution of the original physical variables 2,, we introduce emission model p(zz hhe) = (sz; ((he), It), where á is represented by a NN. By marginalizing over h4 with this emission model, a trained DBF can generate samples of 2, that follow the test distribution g (o.a) given observations 01:t-
Although the dynamics operator A for the latent variables h; linear, it can express any nonlinear dynamics if the latent space is sufficiently high-dimensional. The Koopman opera- tor (Koopman, 1931) provides a framework for representing nonlinear systems by mapping dusservblessfuuctioons of the system's 's state-inn into a dimensoodmee space where the dynamics are linear. For a system 2++1 = =((),, the Koopman operator K. is a linear operator acting on set
Strategy 2 Train all components (the matrix A, the stochastic mapping p(z, h.)) = N(=s; ((hx), diag[o=]), and the 100) simultancously. In this case, samples of (2t, o,) pairs or the SSM for both physical and observation vari- ables to generate these sample pairs are required during training. Note that the physical variables 2, are not required
3

---

DBF for Bayes-Faithful DA
for inference, ensuring that real-time applications are not hindered by the need for 2 during training. The parameters are optimized by maximizing a joint ELBO, CELBO.joimt joint via supervised training:
be estimated via Monte Carlo sampling. Consequently, dur- ing inference, successive Monte Carlo sampling ("cascade trick"; Girin et al. 2021) becomes unavoidable.
loss function DBF takes the ELBO from factorized den- sity log plor (01:--1) 1) in log p(ou:) = >, log logg(orlontt)))
T log logplor, 2:00:011
log
t=1
T
2 BOLBO,joiit.
logp(01-r)
Σ f
LELBO.joint
-Kilghhhlo::)))g//ooo------- (8)
+ (1
(7 the other hand, DVAEs take the ELBO from probabiliyy density with all the observations at once: log ((or))
(See Sec. A.2 in the appendix for the derivation). We have as
(loberrllom)lloppbutt o1:T)
replaced g(h, |o1:e: 21:a) with its special case our objective is to give the best estimate of oe given obser- vations 01.1-
- (hhg:rlooor
(9)
Therefore, DBF seeks for the filtered distributions (hhelo1a) whereas DVAES model the smoother distribu- tions ((h,)o1:)). Again, for DVAEs, to evaluate the ex- pected values in Equation 9, we need to undergo successive Monte-Carlo sampling over T variables (h1:r) (see also Sec. A.3).
Computational scalability, For nonlinear dynamics we parameterize the transition matrix A as 2 x 2 block- diagonal operator (Eq. 17, see Appendix C.I). This fac- torization decomposes each Kalman-style update into in- dependent blocks, so the arithmetic cost per time step is (da) in the latent dimension da and grows only linearly with the sequence length T. Such linear-in-d, behavior is a key advantage of DBF in the tigh-dimensional regimes typical of large-scale physical models.
Assuming linear Gaussian dynamics and a Gaussian I00, DBF allows for the analytical integration of q(helo1=-1). resulting in structured encoder. This structured posterior enables the recursive computation of the filtered distribution ((he(ora)) without relying on Monte Carlo sampling, setting it apart from other DVAEs. By constraining the dynam- ics to be linear, DBF ensures exact integration without the accumulation of Monte Carlo sampling errors across time steps.
2.6. Related Works
2. DYNAMICAL VARIATIONAL AUTOENCODERS
DVAES (see Girin et al. 2021 for a review) are broad class of models incorporating time-series architectures into VAEs, with DBF specialized subcategory. Key differences in- clude (i) the posterior design and realization of the dynamics step, and (ii) the loss function.
Moreover, the linear assumption helps DBF mitigate the instability issues commonly faced when training standard RNNs. The linearity of the latent dynamics is also assumed in normalizing Kalman Filter (de Bézenac et al., 2020) and Kalman variational auto-encoder (Fraccaro et al., 2017).
posterior design Our strategy for the test distribution is to incorporate an appropriate architecture that reflects the Markov property in the time dimension of the test distri- and the linear dynamics model
2.6.2. KF-BASED METHODS
Various upproaches have been explored to address LGSSM limitations, including linearizing the model via first-order approximations like the extended Kalman Filter (EKF), ap- proximating populations with a Gaussian distribution in the ensemble Kalman Filter (EnKF; Evensen 1994), and us- ing NNs to approximate the Kalman gain (Revach et al., 2022). EKFNet (Xu & Niu, 2024) assumes EKF for the con- struction of test distribution and train the SSM parameters. Auto-EnKF (Chen et al., 2022: 2023) leverages EnKF and train the model by maximizing the log model evidence. The EnKF and its variants (e.g., ETKF; Bishop et al. 2001) are commonly used in real-time data assimilation for weather forecasting. However, these methods rely on the KF's poste- rior update equations, limiting the expressivity of the distri- butions. Additionally, computations for covariance matrices
bution. The 100, r serve key instruments in constructing the test posterior distributions. A distinguishing feature of our methodology is that each component's role is defined with respect to the Markov property of the state-space model (SSM) and is clearly differentiated from other components involved in posterior construction. For example, the IOO influences only the update step and does not affect the prediction step. We refer to this methodology as "Bayes-Faite "aithfu- due to its tailored design for SSMs that exhibit the Markov property.
In contrast, the test posterior distributions in DVAEs are constructed using RNNs. The complexity of the transition model prevents the analytical computation of latent vari- ables across time steps. As a result, these values can only
4

---

DBF for Bayes-Faithful DA
become challenging in high-dimensional spaces, requiring specialized techniques for computational efficieney.
Github'.
Linear Dynamics: Moving MNIST In the moving MNIST task, the goal is to identify the images, positions, and velocities of two handwritten digits as they move within the observed frames. While the dynamics of these digit images and their observation processes are provided, the actual images, positions, and velocities are not available, making supervised learning impossible.
2.6.3. SAMPLING-BASED METHODS
The Particle Filter is a popular method for assimilating any posterior. However, achieving adequate particle density in high-dimensional state spaces poses significant challenges. Insufficient density of particles leads to particle degeneracy, where few particles explain the observed data (Beskos al., 2014). In contrast, DBF directly learns to position density through the I00, offering advantages for high- -igh-dimensional tasks. For the performance comparison of PF and DBF in terms of accuracy-computatey the Appendix. The Particle Flow Filter (PFF; Daum & Huang 2007; Hu & van Leeuwen 2021) addresses particle degeneracy by moving particles according to gradient flow and effectively scales to nonlinear SSMs with hidden state dimensions up to 1000 (Hu & van Leeuwen, 2021).
Nonlinear Dynamies: Double Pendulum and Lorenz96 For nonlinear dynamics problems, such the double pen- dulum and Lorenz96, DBF constructs a new latent space in addition to the original physical space. Here, we took Strategy 2 in Sec.2.5 for the training: we simultancously train NNs for the IOO, nonlinear observation operator Φ, the dynamics matrix A, and the emission model's standard deviation. We compare the performance of DBF with the classical DA algorithms (EnKF, ETKF, PF), state-of-the-art assimilation methodologies (PFF Daum & Huang 2007; Hu & van Leeuwen 2021, KalmanNet Revach et al. 2022), and DVAE-based approaches (deep Kalman Filter; DKF, Krish- nan et al. 2015; 2016, variational recurrent neural network; VRNN, Chung et al. 2015, and stochastic recurrent neu- ral network; SRNN, Fraccaro et al. 2016). DBF and other DVAEs are trained by optimizing the evidence lower bound (ELBO), as deseribed in Sec. 2.5.
trade-off, see Sec. E in
2.6.4. APPROXIMATE MAP ESTIMATION METHOD
MAP estimation is used to identify the high-density point of the posterior in nigh-dimensional space, such as in weather forecasting (Lorene, 2003; Frerix et al., 2021). Even if the computation of the posterior can optimize log p(h,(01z)) log p(o)//t)) + log p(h, (01.-11) if we can describe p(o, = J p(h, (he-1)p(hs 1 10:.--)ddh;-- ex-
p(h,)ora)) is intractable,
and p(h))01.t-1) 11 plicitly. In practice we cannot access p( and therefore the integral J p(h,
For all experiments, we generate random initial conditions and evolve them using the dynamics. Synthetic observations are produced by applying the observation operator with ad- ditive noise. Noise levels, observation operators, and further details are given in Sec. C.1, C.2, C.3, and C.4. Sec. also provides computationally efficient parametrization of the latent dynamics matrix.
1 we only compute the mean. The downside that sequen- tial computation of the covariance matrix of p(helo1:t-1) is impossible.
so
2.6.5. NN-BASED PDE SURROGATE
Recently, there have been attempts to approximate partial differential equations (PDEs) using NNs. In this study, we experimented with one of the latest methods, PDE-refiner (Lippe et al., 2023), but its performance was poor and was excluded from the experiments. We suspect this is because PDE-refiner, designed for constructing PDE surro- gates, does not handle noisy observations well, making it sensitive to noise. However, we confirmed that it performs well under noiseless conditions.
3.1. Linear dynamics: Two-body Moving MNIST
This experiment demonstrates DBF's ability handle linear dynamics with unknown observation operator parameters. The dataset consists of 2D figures containing two embed- ded images moving at constant speeds and reflecting off frame edges. The system state is defined by the positions and velocities of the images: 2n = (Ir,, M/, U2,t: Ug.e), with dynamics governed by a block-diagonal translation matrix Agr. Reflection is modeled using nonlinear observation operator (Sec. C.5). Observations are corrupted by Gaussian noise (a 50, where the original pixel values range from O to 255). See panel (a) of Fig. 2 for an example of the data.
3. Experiments
We evaluate the performance of DBF on three tasks: linear dynamics problem (moving MNIST) and two nonlinear dynamics problems (double pendulum and Loren296). An additional experiment on linear dynamics (object tracking) is presented in Sec. B of the appendix. The code is available
The goal is to show that DBF tracks linear dynamics while estimating unknown system parameters. DBF learns the pixel values of the images from noisy observations while maintaining consistency with physical motion. Classical
5

---

DBF for Bayes-Faithful DA
(b)
Table 1. The success rates and RMSE of the four methodologies for the two-body moving MNIST problem.
Iwd
Rer
Fer 300
foe 1936
Gial
m
Method
Success rate
RM
RMSE(vel)
F-12
1-16
5
DBF
100% (50/50)
0.39 = 0.027
0.45 - 0,042
EnKF
0% (0/50)
6.3 1.1
4.4 2.2
ETKF
0% (0/50)
5.7 1.4
6.5 + 8.4
Figure 2. Figures from the two-body Moving MNIST experiments.
PF
0% (0/50)
4.8 0.7
1.4 + 0.23
Panel (a) displays examples of the observation data. Panel (b)
illustrates the evolution of the observation model parameters (the embedded images) during training.
DA methods (EnKF, ETKF, PF) were adapted to infer these parameters by treating them as physical dimensions, but they fail due to high observation noise and non-Gaussianity. KalmanNet could not train due to the high observation di- mensions (xaim (44 x 44) even with the batch size of one, the training fails. While DVAE generates latent variables, they are different from the state variables of the original SSM: therefore, they cannot infer the position or velocity from those images.
(a)
(b)
b d
VRAN SRWN
0.4
DRF
0.2
15
20
40
50
60
70
Figure 3. A schematic figure (panel a) and results for double pen- dulum experiments. Panel (b) shows the RMSE of angle velocities (averaged over wi and w2) over time steps. Panels (c) and (d) show example histograms for normalized errors DBF and ETKF sam- ples compared against the unit Gaussian ((x;; μ = 0, o² = 1).
Fig. 2 summarizes the experiment. Panel (a) shows an ex- ample from the test set, illustrating the challenges posed by strong noise and overlapping images. Panel (b) presents the DBF learning process. In Table 1, we compare the success rates of DBF against model-based approaches (EnKF, ETKF, er-
PF). We define success as achieving a root-mean-squee ror (RMSE) of less than 1.0 for both position (x V1, z2, 3/2) and velocity (U2,, Uy: Upgy (222) ) of the two digits over the final ten steps (sub-pixel accuracy an error no larger than 1/44 2 2.3% of the 44 x 44 frame). DBF successfully performs assimilation without explicit knowledge of the im- ages, while all the other model-based approaches fail. The KF-inspired approaches (EnKF, ETKF) failed because of very strong Gon-Gaussi in the observation process and the high system dimension. Similarly, PF underperformed because the number of particles (10, (0,000) was insufficient for the problem dimension (Zdim = 8 and two digits images 2 x 28 x 28- 1,568). Figures for visualizing the assimila- tion results for all the algorithms are given in the appendix (Fig. 13).
of the setup is shown in panel (a) of Fig. 3.
We use the angles 01 and 0g. and the two angular velocities, and w2, as target physical variables. The latent dimension for DBF, VRNN, SRNN, and DKF is set to 50 (for the choice of the latent dimensions, refer to Sec. E.1). Observation data consists of the two-dimensional spatial positions of masses Pi and P2. corrupted by Gaussian noise. The observation operator combines trigonometric functions for 0, and 02 which are highly nonlinear. Experiments are conducted with noise levels of o 0.1, and 0.5 [m], with a time step of 0.03 [s] between observations. In the emission model p( 2 [hx), we assume von Mises distributions for 0₃ and 0z. while W1 and w/ follow Gaussians.
Table presents the RMSE between the physical variables and the mean of the filtered distribution. For both the angles 0 and angle velocities w, we compute the averages of the two variables across two pendulums. Training for KalmanNet was unsuccessful under all conditions. For the DVAEs, we exclude failed initial conditions (2/15 for VRNN and DKF, and 3/15 for SRNN) when calculating the RMSE. DBF out- performs both model- based and latent assimilation methods across all settings, showing significant improvements in estimating w, which cannot be inferred from a single obser- vation. Fig. 3 (b) illustrates an example of RMSE evolution during assimilation, where DBF consistently outperforms the other methods. The assimilation of w occurs within the first ~ 20 steps, maintaining an excellent estimation accuracy throughout the experiment.
Panel (b) also illustrates DBF's parameter updates. Initially, DBF assumes random shapes, identifying and refining the images over iterations. By the end of the training, DBF ac- curately estimates observation model parameters, including positions, crucial for reflecting behavior.
3.2. Nonlinear Dynamics 1: Double Pendulum
This section presents our experiments with double pendu- lum system, selected for its nonlinear and chaotic behavior. The pendulum consists of two 1 kg masses, PI and P2, con- nected by two 1 meter bars, B1 and B2. One end of the bar B1 is fixed at the origin ("O"), with the other end attached to P1. Mass P2 is connected to P1 via bar B2, A schematic
6

---

DBF for Bayes-Faithful DA
Table 2. RMSE the final ten steps of assimilation in double pendulum experiments.
0.1
0.3
0.5
0
0
0
DBF
0.03 = 0.01
0.21 + 0.04
0.05 = 0.02
0.26 + 0.05
0.06 + 0.01
0.36 + 0.04
EnKF
0.05
0.33 + 0.07
0.14
0.71 + 0.09
0.24 + 0.01
1.17 + 0.22
ETKF
0.05
0.46 + 0.08
0.22
1.41 0.41
0.36 - 0.08
2.70 + 1.25
PF
0.05 0.00
0.63 + 0.24
0.21 + 0.14
1.41 + 1.30
0.32 + 0.08
2.36 2.29
PFF
1.27 0.29
1.04 0.15
NA
5.99 + 1.09
5.88 0.67
NA
KNet
NA
NA 0.44 0.19
NA
NA 0.35 + 0.14
NA 0.08 0.04
NA 0.40 0.16
VRNN
=.01
1.02 0.06
SRNN
0.05 + 0.02
0.52 0.18
0.44 0.08
0.08 = 0.03
0.52 + 0.22
DKF
0.12 + 0.02
2.70 + 0.28
0.17 0.03
2.61 0.74
0.23 + 0.04
2.61 + 0.56
where 2; is the value at grid i, N is the number of grid points, and F is external forcing. For our experiments, we take (F, N) = 40).
Table 3. The Jeffreys divergence of normalized errors and the unit Gaussian between DBF, EnKF, and ETKF predictions for the double pendulum experiment.
We consider observa- tion operators. The first adds Gaussian noise to di- rect observations: Ouj = Rr.j + with noise levels a = 1,3,5. 5. The second uses a nonlinear operator: ot,j min(2t,,- 10) c, with the same noise lev- els. The dynamic range of æt, is around +10, and observations are capped at 10 when žt, exceeds 1.8. This makes highly chal- lenging for classical DA methods, each observa- tion offers limited infor- mation. The filter must integrate data over long timesteps, where nonlinear dynamics distort the probability distribution. Fig. 4 illustrates observations and target val- ues. All models use 80 observation steps with 0.03 time interval. The latent dimension for DBF, VRNN, SRNN, and DKF is set 800 (for the choice of the latent dimension in DBF, see Sec. E.2). For further details for the experiment, see Sec. C.4.
= 0.1
KLaym
DBF
0.02
observation
target
EnKF 10.2
ETKF 0.12
20
A key feature of DBF is its ability to generate samples of 2t and assess the uncertainty in state estimates. To evaluate this capability, we analyze the distributions of normalized errors defined as Enorm,t, (Zt.sample,i - 2,,5)/6; /6, where ž rep- resents the true value of dimension i at time and 8; is the standard deviation of somplee., We collect f, across all time steps, focusing on = wi and W2. since 0 and 0g follow von Mises distributions. If the uncertainty esti- mates are accurate, Enorm,t,f should approximate a Gaussian distribution with standard deviation of one. To quantify the accuracy, we compute the symmetric KL divergence (Jef- freys divergence) K T.symm [p, :] (K [KL|pllg] K [[gll))) between the histogram of Enorm,t, and unit Gaussian. DBF exhibits very low K Lagm values, indicating accurate error estimation. Panels (c) and (d) display example his- tograms of normtt, for DBF and ETKE.
50
Figure 4. A Hovmöller diagram for one initial condition in the test set. The observation op- erator is nonlinear, or min (=g.5> min 10) +
3.3. Nonlinear Dynamics 2: Lorenz96
In the final experiment, we focus on state estimation in the Lorenz96 model (Lorenz, 1995), benchmark for test- ing data assimilation algorithms on noisy, nonlinear obser- vations. The Lorenz96 model describes the evolution of one-dimensional array of variables, each representing physical quantity over spatial domain, like equilatitude circle. The dynamics are governed by the following coupled ordinary differential equations:
Table presents the assimilation performance across differ- ent noise levels and observation settings. DBF outperforms existing methods in direct observations with a 3, 5, and across all noise levels for nonlinear observation cases. the = 1 setting with direct observation, traditional algorithms like EnKF and ETKF outperform DBF.
The superior performance of EnKF and ETKF with direct observations at the lowest noise level can be attributed to
dz;
= (2i41 21-222-1 + F, i 1.....N... N, (10)
dt

---

DBF for Bayes-Faithful DA
Table 4. RMSE the final ten steps of assimilation in Lorenz96 experiments with (F, N)=(8 40).
direct observation
nonlinear observation
==1
==3 3
= 5
1
0= 5
DBF
0.53 : 0.04
0.82 = 0.03
1.16 = 0.07
1.08 0.15
1.29 = 0.18
1.65 + 0.17
EnKF
0.00
0.83 = 0.10
1.73 + 0.12
0.14
= 0.08
3.81 + 0.07
ETKF
0.30
1.06 0.15
2.42
4.57
4.28
4.23 =
PF
+.00
3.12 0.06
3.62
0.16
4.58 4.12
PFF
0.60
+.05
2.09
3.13 3.85 0.05
KNet
0.00
1.81 0.05
0.09
0.17 0.27
3.99 = 0.25
VRNN
3.67
3.67 0.06
3.67 0.06
3.69 0.04
2.79
3.67 0.06
SRNN
3.08 + 0.56
3.63 + 0.05
3.40 = 0.29
3.30 0.81
3.62 0.41
2.96 + 0.32
DKF
3.70
NA
NA
NA
NA
NA
(a)
setting
max[abs(cig)]
D,
1.016 0.002
ViAh
VRNV
2.2-
5MN
5158
D,
1.014 + 0.002
2.0
K46
Keet
D =
1.011 = 0.001
bid
516.
ITC
CTT
N.
1.012 + 0.003
0.
10
25
30
40
53
63
70
15
20
30
40
53
20
70
N, = 3
1.008 + 0.004
0.8
0.9 absteigenvaluu)
1.0
11
N = 5
1.004 + 0.001
Figure 5. RMSE results for Lorenz96 experiments. Panels (a) shows results for direct observation with = 1. Panel (b) shows results nonlinear observation with
Figure 6. Histogram of 800 eigenvalues of the dynamics matrix in Lorenz96. D for direct and N for nonlinear observations.
the minimal Gon-Gaussianity in the posteriors within phys- ical space. Non-Gaussianity can originate from both the dynamics model (predict step) and the observation model (update step). In this setting, the linearity of the observation operator prevents non-Gausiannity from being introduced during the update step, provided that the prior q (01:-1) Gaussian. Additionally, state estimation from each ob- servation is highly accurate due small noise. As a result, the prior ((.,))).-1) remains close to Gaussian distribu- tion, as the locally linear approximation of the dynamics adequately captures the time evolution of probability dis- tributions. The poorer performance of EnKF and ETKF in the experiment attributed the increased non- Gaussianity introduced during each predict step. Similarly, when the observation operator is nonlinear, each update step introduces substantial Non-Gaussianity. This results in sig- nificant drop in performance for traditional filtering methods across all noise levels. In these scenarios, DBF consistently maintains an advantage over classical DA algorithms.
the histogram of the absolute values of eigenvalues at the end of training, which are distributed around or below one, indicating stable training.
4. Limitation
DBF's learning of IOO requires a training phase, unlike clas- sical model-based data assimilation methods. Specifically, when dealing with nonlinear dynamics, DBF requires either: (i) pair of (z+,0t) generated from the original SSM, (ii) a pair of (zz, ot) obtained via, e.g., retrospective reanalysis (ERA5; Hersbach et 2020 in weather forecasting), or (iii) pretrained Koopman operator and observed data or-
In the Lorenz96 experiment, DBF's performance with direct observation with = falls short compared to EnKF and ETKF. In this setting, the non-Gaussianity of posteriors is weak, resulting in minor approximation errors due to Gaus- sian assumptions. Consequently, a model-based approach may be more advantageous in such situations, as it lever- ages complete SSM knowledge without introducing training biases.
We observe that training DVAE-based methods is highly unstable, while that for DBF exhibits stability. Dynamics in DVAEs are modeled by RNNs, which often suffer from unstable training due to exploding or vanishing gradients. In contrast, DBF employs matrix multiplication for dynam- ics. If the eigenvalues of the matrix exceed one by a large margin, the model predictions, and consequently the loss function, would explode irrespective of inputs. Fig. shows
5. Conclusion
We propose DBF, a novel DA method. DBF is a NN-based extension of the KF designed to handle nonlinear observa-
8

---

DBF for Bayes-Faithful DA
tions. While constraining the test distributions remain Gaussian, DBF enhances their representational capacity by leveraging nonlinear transform expressed by a NN. DBF is the first BBayes-Faithfie amortized variational inference methodology, constructing test distributions that mirror the inference structure of a SSM with the Markov property. This structured inference enables analytical computation test distributions, preventing the accumulation Monte Carlo sampling emors over time steps. DBF exhibits superior per- formance over existing methods in scenarios where posterior distributions become highly non-Gaussian, such as in the presence of nonlinear observation operators or significant observation noise.
2023. URL httpss:/aphs www.stiisscsseeeeeeeeeeeeeeeeese org/ /tg/CCoppuuTD:2512991111
Awal, M. A., Refat, M. A. R., Naznin, F., and Islam, M. A particle filter based visual object tracking: A systematic review of current trends and research challenges. International Journal of Advanced Com- puter Science and Applications, 14(11), 2023. doi: URL http: : /dx.
10. 01145690IACCSAA222220111111 doi. doi. / IJACSA 202..0141111 01411131.
Azencot, O., Erichson, N. B., Lin, V., and Mahoney, M. Forecasting sequential data using consistent koopman autoencoders. In III, H. D. and Singh, A. (eds.), Pro- ceedings of the 37th International Conference on Ma- chine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 475-485. PMLR, 13-18 Jul 2020. URL http://eooccceeeeeeeeeeeeeecccee mlr press/ v119 #119/azencot2000 html.
Reproducibility Statement We have provided the source code to reproduce the experiments for double pendulum (Sec. 3.2) and the Lorenz96 (Sec. 3.3) in the supplementary material. The hyperparameters for the training are provided in Table in the appendix. Generation method of the train- ing and test dataset, the dynamics model, the observation model, and the architectures are detailed in the appendix: Sec. C.2, C.3, and C.4.
Bach, E. and Ghil, M. A multi-model ensemble kalman filter for data assimilation and forecasting. Journal of Advances Modeling Earth Systems, 15(1):22022MS0003322 2023. doi: htt.://ooorrrr...../000222222111 URL
https: /ettps://agupuusii on i ne wiwrarrwriii..... wel...
Impact Statement
1ttp:/0....000///220000300 22022MS003123 2022MS003123.
Our Deep Bayesian Filter could improve numerical weather prediction by providing more accurate estimates of the at- mospheric state, enabling earlier and more reliable fore- casts of extreme events-scenarios that typically yield non- Gaussian posterior distributions. The same framework trans- fers cleanly to virtually any sequential nonlinear filtering task, opening broad avenues for data assimilation across disciplines.
Beskos, A., Crisan, D., and Jasra, On the stability of sequential Monte Carlo methods in high dimensions. The Annals of Applied Probability, 24(4):1396 - 1445, 2014. doi: 10. 10.1214/13-AAP955 URL https: / /ww./oooooooo org/ 10 15.111/113-AAP551
Bishop, C. H., Etherton, B. J., and Majumdar, S. J. Adaptive Sampling with the Ensemble Transform Kalman Filter.
Acknowledgements
Part I: Theoretical Aspects. Mon. Wea. Rev., 129(3):
420-436, March 2001. ISSN 0027-0644, 1520-0493,
This research work was financially supported by the Min- istry of Internal Affairs and Communications of Japan with scheme of "Research and development of advanced tech- nologies for user-adaptive remote sensing data platform" (JPMI00316). We thank the anonymous reviewers for help- ful comments earlier versions of this paper.
doi: 10. (01177773300003222011)0994444444
URL /ttp:///ttt///oorrraaaaeeetteooa
doi/10. Bol//00.175//5220043332238 12940400))12244422 ASWTET>2.0.com
Chen, Y., Sanz-Alonso, D., and Willett, R. Autodiffer- entiable ensemble kalman filters. SIAM Journal on Mathematics of Data Science, 4(2):801-833, 2022. doi: 10.11377/1M1433447. URL https: /dopi. http:////oooo.o1....// or1/ 1137/21M1 1147/71
References
Alfonzo, M. and Oliver, D. S. Seismic data assimilation with an imperfect model. Computational Geosciences, 24(2):889-905, 2020. Marine Environmental Monitoring and Prediction.
Chen, Y., Sanz-Alonso, D., and Willett, R. Reduced-ordd autodifferentiable ensemble kalman filters. Inverse Prob- lems, 39(1),, 10 2023. doi: 10. 1088/13611-642/acchff
Andrychowicz, M., Espeholt, L., Li, D., Merchant, S., Merose, A., Zyda, F., Agrawal, S., and Kalch- brenner, N. Deep learning for day forecasts from sparse observations. ArXiv, abs/2306.06079,
Chopin, N. and Papaspiliopoulos, O. An introduction to Sequential Monte Carlo. Springer series in statistics. Springer, 2020. URL hitps:/ /ci. nii ac. ip/ ncid/BC0 BC0333800.

---

DBF for Bayes-Faithful DA
Chung, J., Kastner, K., Dinh, L.,, Goel, K., Courville, A. C., and Bengio, Y. A Recurrent Latent Variable Model for Sequential Data. In Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc., 2015. URL. https://proceddingss newriii..sssnnnn...eo.......iiim cc httppppe_fillee/ppprr////////a//
Fraccaro, M., Kamronn, S., Paquet, U., and Winther, O. A disentangled recognition and nonlinear dynamics model for unsupervised learning. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https::/prrocedingas http://rooeeddinn...eeeeerrs..ss 0c cc/paper_fiieeeeeee /icaperrfiiee/ppppe///////LLLLL 7 1b7a558e2394000a11bbbbbbbe6a pdf.
3 html.
Abstract.
Daum, F. and Huang. J. Nonlinear filters with log-homotopy. In Drummond, O. E. and Teichgraeber, R. D. (eds.), Sig- nal and Data Processing of Small Targets 2007, volume 6699, pp. 669918. International Society for Optics and Photonics, SPIE, 2007. doi: 10.1117/12.725684. URL https: /tw. .rr. / 1117/12 725684.
Frerix, T. Kochkov, D., Smith, J., Cremers, D., Bren- ner, M., and Hoyer, S. Variational Data Assimi- lation with a Learned Inverse Observation Operator. In Proceedings of the 38th International Conference on Machine Learning, pp- 3449-3458. PMLR, July 2021. URL https: / http://pcccceeee..ee/e// mlr press/
de Bézenac, E., Rangapuram, S. S., Benidis, K., Bohlke- Schneider, M., Kurle, R., Stella, L.,, Hasson, H., Gallinari, P., and Januschowski, T. Normalizing kalman filters for multivariate time series analysis. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural Information Processing Systems, volume 33, pp.
v139/ 1139frerix21a. html. ISSN: 2640-3498.
Girin, L., Leglaive, S., Bie, X., Diard, J., Hueber, T., and Hameda--inedaa X. Dynamical variational autoencoders: A comprehensive review. Foundations and Trends@ in Machine Learning, 15(1-2):1-175, 2021. ISSN 1935- 8237. doi: 10.1 00/26/222000000088, URL http: /xx doi.org/11 101.112000001000..3
Curran Associates, Inc., neurips.
2020. URL htttps:/prroceeedings. neuriiis.snsn..ns..eoe....iiii.s oc
1f4 (447cef5c2300522f555666666688783 pdf.
Gu, A. and Dao, T. Mamba: Lincar-time sequence modeling with selective state spaces. arXiv preprint arXiv:23/2.00752, 2023.
Evensen, G. Sequential data assimilation with non-
linear quasi-geostrophic model using Monte Carlo methods to forecast error statistics. Journal of Geo- physical Research: Oceans, 191C5110014311016, doi: 10.1029/94/C00572..
Hersbach, H., Bell, B., Berrisford, P., Hirahara, S., Horányi, A., Muñoz-Sabater, J., Nicolas, J., Peubey, C., Radu, R., Schepers, D., Simmons, A., Soci, C., Abdalla, S., Abellan, X., Balsamo, G., Bechtold, P., Biavati, G., Bidlot, J., Bonavita, M.. De Chiara, G., Dahlgren, P., Dee, D., Diamantakis, M., Dragani, R., Flemming, J.. Forbes, R., Fuentes, M., Geer, A., Haimberger, L., Healy, S., Hogan, R. J., H6lm, E., Janisková, M., Keeley, S., Laloyaux. P., Lopez, P., Lupu, C., Radnoti, G., de Ros- nay. P., Rozum, I., Vamborg, F., Villaume, S., and Thépaut, J.-N. The era5 global reanalysis. Quarterly Journal of the Royal Meteorological Society, 146(730): 1999-2049, 2020. doi: https:///oi. org/ 10. 1ttp...000111000/..1313... 1002//i URL ht /tttps::/rmetstt htt:://ttssss///me///e www.eeewwnnnnnneeeeeeeeeeeeeeeee com/ hon//doi/aat 1ttp://bs/00.100///// qj 3803.
1994. ISSN 2156-2202.
URL https: /ww.innniibarrry....eraaaaaaaaee wiley. doi/abs/10 1029/943000572. http://wliillllllaaiiiweeecoooop
.eprint:
Fan, H., Lin, L., Yang, F., Chu, P., Deng, G., Yu, S., Bai, H., Xu, Y., Liao, C., and Ling, H. Lasot: A high-quality benchmark for large-scale single object tracking, 2019.
Fan, H., Bai, H., Lin, L., Yang, F., Chu, P., Deng, G., Yu, S., Harshit, Huang, M., Liu, J., Xu, Y., Liao, C., Yuan, L., and Ling. H. Lasot: high-quality large-scale single object tracking benchmark. Int. J. Comput. Vision, 129 (2):439-461, feb 2021. ISSN 0920-5691. doi: 10.1007/ s11263-020-011387-y URL https: / http//dioo///////// 020-0000000 01387-y.
wiley.
Hu, C.-C. and van Leeuwen, P. J. A particle flow filter for high-dimensional system applications. Quarterly Journal of the Royal Meteorological Society, 147(737): 2352-23744 2021. doi: https://doi.org// 1002//00000///42333 URL ht /tte::///mmet.////// on wwwlllineibbarnn wiley. com/ http//ab///...00//gg/ 4028.
Fraccaro, M., Sønderby, S. K., Paquet, U., and Winther, O. Sequential neural models with stochastic layers. In Proceedings of the 30th International Conference on Neu- ral Information Processing Systems, NIPS' 16, pp. 2207- 2215, Red Hook, NY, USA, 2016. Curran Associates Inc. ISBN 978- 1-5108-3881-5
Hunt, B. R., Kostelich, E. J., and Szunyogh, I. Efficient data assimilation for spatiotemporal chaos: A local ensemble transform kalman filter. Physica D: Nonlinear
10

---

DBF for Bayes-Faithful DA
Phenomena, 230(1): 2007. ISSN 0167-2789.
benchmark for object tracking in the wild. In Com- puter Vision ECCV 2018: 15th European Confer- ence, Munich, Germany, September 8-14, 2018, Pro- ceedings. Part I, pp. 310-327, Berlin, Heidelberg, 2018. Springer-Verlag. ISBN 978-3-030-01245-8 doi: 10. 1007/978-3--33--02446-5..1 URL https:/ /doi. org 10t.11007/9977 3-000---0--00 012-0-1211666-1 5_1..
doi:
1..08.
URL https://wwen http://wwww.cceeeneereeett.t..t. com/ science/artin (=-Bnc//artccee/1///987777666660
Data Assimilation.
Jocher, G., Chaurasia, A., and Qiu, Ultralyties
YOLO, January 2023, URL https::t//// https://githutt. com/ ultralytics/ulle htt.ic///utaaaaatis.......
Ohishi, S., Miyoshi, T., Ando, T., Higashiuwa- toko, T., Yoshizawa, E., Murakami, H., and Kachi, M. Letkf-based ocean research analysis (lora) version 1.0. Geoscience Data Journal, n/a (n/a), 2024. doi:
Koopman, B. O. Hamiltonian systems and transformation in hilbert space. Proceedings of the National Academy of Sciences, 17(5)311-3118 1931. doi: 10.1073/pnas..
5.315. URL https: /ttp:p///e /twp::///www http//wwpppaeeee..e pnas. org /o//do///aa abs/ 10. 10..1073/0 //paas/p//..... 1...5.15....
URL https:://mmettee on ne braar. ley. gtt.:.00022gg3...22
com/
Krishman, R. G., Shalit, U., and Sontag, D. Deep kalman fil- ters, 2015. Utthttps: http:/axxvvvv.. org/ abs///b////// abs/15ss 1511. 05121.
Revach, G., Shlezinger, N., Ni. X., Escoriza, A. L., van Sloun, R. J. G., and Eldar, Y. C. Kalmannet: Neural net- work aided kalman filtering for partially known dynamics. IEEE Transactions Signal Processing, 70:1532-1547, 2022. doi: 10.1 10110009.222..1155555555
Krishnan, R. G., Shalit, U., and Sontag, D. Structured in- ference networks for nonlinear state space models, 2016. URL https: //taa:///arreee /ttpvvaxvvvv////////v//eee//aaaa org/ abs// 1609 09869.
Takeishi, N., Kawahara, Y., and Yairi, T. Learning koopman invariant subspaces for dynamic mode decomposition. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Gareet,, R. (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.,
Larsen, J., Høyer, J., and She, J. Validation of hybrid optimal interpolation and kalman filter scheme for sea surface temperature assimilation. Journal of Marine Systems, 65(1):122-133, 2007. ISSN 0924-7963.
doi:
2017. URL http:://proceedings neurips cc/paper_filett cc//appe__filee/ppprr///////1//1 2.836222575c333effffe666222222 pdf.
URL https::/www..ciieene
com/
ocienee/arriiclee///////22/es Marine Environmental Monitoring and Prediction.
Lippe, P., Veeling, B. S., Perdikaris, P., Turner, R. E., and Brandstetter, J. PDE-Refiner: Achieving Accurate Long Rollouts with Temporal Neural PDE Solvers. In Thirty- seventh Conference on Neural Information Processing Systems, 2023. URL https: /wwpr//ieeeeeeeiee.... n forum? 4==vv46811v66
Xu, L. and Niu, R. Ekfnet: Learning system noise covari- ance parameters for nonlinear tracking. IEEE Transac- tions on Signal Processing, 72:3139-3152, 2024. doi: 10. 10.1099922220443333333
Lorenc, A. C. Modelling of error covariances by 4d-var data assimilation. Quarterly Journal of the Royal Meteorological Sociery, 129(595):3167- 3182, 2003. doi: 0ttp:///dbbb.rr//00.22666062002 URL
www.ennttss..innnnnrnenneeaeeeee wiley. com//ooiabs///ii comm//oi//ab//101115////g
Lorenz, E. Predictability: a problem partly solved. PhD thesis, Shinfield Park, Reading, 1995 1995.
Lusch, B., Kutz, J. N., and Brunton, S. L. Deep learning for universal linear embeddings of nonlinear dynamics. Nature Communications, 9:4950, November 2018. doi:
Müller, M. Bibi, A., Giancola, S., Alsubaihi, S., and Ghanem, Trackingnet: A large-scale dataset and

---

DBF for Bayes-Faithful DA
A. Derivation of the Evidence Lower-Bound and the Associated Monte-Carlo Sampling
A.1. Linear Dynamics Case
Following the definition of the probability density,
= p(o: ((oolo1:t1))((hooo)
(11)
Using Eq. the third equality,
T logp(oslou=-1)
log l(gp()))))
Σ
i ((h+l1r.)) log
plor, helo1-1)) dhu p(helo1:))
{ ((holoo)))) log
p(o,, Ny (1) ((:t)) dhg
Σ
a(hrlo1a) p(helo1z)
p(or, 1o dh, + KLig(hhoowwllll/llon))) g(holo1a))
((helora)) log
Σ LELBO + KLlg(talon:))lphhhloo..
Σ
LELBO
(12)
plor, helo1=-1) (hellrr))
LELBO,
q(h (1:)) ) log
dhg
/ ((hilon:)) log
p(helone (olloo---)p((oot )p(osshe)) ((hx)o11))
dhg
/
p(helora-11 dh, ((helo11))
q(hy (oa) log p(oe)))))) + a(helona))
/ ((hllor.)) log - KLlg(hhloowa]llhhloon--
(13)
The true prior at step t (p(h.lo1a-11) on the right hand side of Eq. 13 could be replaced with the prior computed from the test distribution (1 when training.
A.2. Nonlinear Dynamics Case
plor, 21, h4 214-1) = p(or, 2, 10...111 ( 21:))
(14)
12

---

DBF for Bayes-Faithful DA
The derivation proceeds parallel to the linear case. Using Eq. 14 at the third equality,
T
log p(p0o...22111 p(o, 24 21001111
(ogul 20000-0110
Σ
f ((hhlo1))) log
p(or, 2,, he 1e 21:t-1) 22:--1) dhi
p(he|01:1,
Σ) @ log
plors Ze, 11
(he lons)) p(he lorss =1xe)
dhg
g(helo1:))
Σ) / q(hhlo1:)) log
dhi + KLlg(halors)lllhhhlloaa plor, 21, hehhelos 1,
g(holo1a))
T
fELBO.joint,
2
Σ
LELBO.joint,
(15)
Loi /
plor, 2, 1, dhe
q(hy) o1:) log
(hello1a)
/ ((hylon log pliy 310- )p(or, 2e(11a) dhg
(holoraa)
f (hhglo: + (ogp(orlz))ddlt + g(helora)) p( 1ihhlo1a-1,,1:d dh,
((1)(o11)
/ (h, o1.) logp(z (L p(hp) 01---1, + logg(or(<2)
(16)
The true prior at step (p(h, -0 1, 1)) on the right hand side of Eq. 16 could be replaced with the prior computed from the test distribution q/h, (0.1-1) when training. The last term of the equation ((o)(2))))) can be neglected as it does not affect the new latent variables ht.
A.3. Comparison to Other DVAEs in terms of Monte-Carlo Sampling
The crucial difference from other DVAEs is that the Monte-Carlo samplings in DBF are not nested with each other. In DVAE,
we need to evaluate an integral term g(hrer (oi.r) log p(ou., )ddrrr where g(hir) ((h:r)ooLT)) = II. g(e ( o:). Although the log-term could be factorized as E, log p(o, (h)) - log p(h, ((-1) thanks to the Markov property, we need MC (nested) sequential sampling over hirr if we want to evaluate the term at f = T. On the other hand, ELBO in DBF
because DBF takes the lower limit of Σ, log (o1..o).)))... Thanks the analytic expressions ghe (01:) and (()))o) the KL term can be computed analytically. MC sampling needed compute (he/oor.) log p( 2 (hi)dh but this independent from other timesteps.
E. [[[(/h q(yy/o log p(=, (h,)dlu + K K[[g(he (he (0x:))) (l
A.4. Comparison to a Linearized Observation Model
DBF seeks the optimal Gaussian posterior distribution for the hidden state 2, (or h2) given an observation og. One alternative approach approximates the likelihood function by a Gaussian model: p(o, (zx) p' (or|2)) = Noor (=)),, G2())) where fo(:)) provides the mean and Go(z)) specifies the variance. Although this method preserves the Gaussian nature of the posterior, it requires the computation of the Jacobian (z))) to update the covariance matrix. In contrast, DBF parameterizes the IO0 instead of the observation model. This parameterization leads to more general and straightforward update equation, circumventing the need for calculating the Jacobian. The only downside is the introduction of a "virtual prior" -a theoretical construct that ensures the 100 represents valid probability distribution over 24. However, this virtual prior only slightly biases the neural network's output and does not impair overall performance.
13

---

DBF for Bayes-Faithful DA
B. Additional Linear Dynamics Experiment: Object Tracking
In single-object tracking problem, detector identifies bounding box for the object in each frame, and these boxes are then connected across frames. When the object determine its position. such scenarios, the KF aids by predicting and assimilating the object's true position. However, key limitation of the KF is its reliance fixed observation model throughout the tracking process. While low-confidence observations can provide valuable approximate position information, they may also mislead the tracker with inaccurate data, potentially degrading overall tracking performance.
fully visible or is obscured, the detector often fails to accurately
We demonstrate that DBF can enhance tracking stability without requiring additional training. During the computation of posterior p(=+|01:) from p(z, 01;t-1), ) the importance of the observation or is regulated via Go(ot). This allows the observational confidence be effectively incorporated into the posterior estimation. We evaluate the tracking performance using the "airplane" category from the LaSOT dataset (Fan al., 2019; 2021).
We the first 1,000 frames from 20 videos for evaluation. The first 10 videos serve validation set for determining filter parameters (see Sec. C.2), while the performance is assessed using videos 11 11-20. Each set of 1,000 frames is divided into 20 subsets of 50 frames. Filters are initialized at the ground truth coordinates of the bounding box in the first frame, after which each filter is responsible for tracking the bounding box throughout the subset. We employ the YOLOvån model (Jocher et al., 2023) as the object detector. The detector outputs the bounding box position, X, along with confidence score, c. detection threshold of 0.01 is applied. When multiple bounding boxes are detected, the one with the highest posterior probability is selected.
The bounding box coordinates used fo(o:) X. We experiment with linear confidence Go(o:) x and squared confidence Go(or) & e² and find that the squared confidence Go(o:) x e² perform better. For further settings, see Sec. C.2.
Figure presents the results. The left panel provides an illustrative example comparing the two tracking algorith ms. The KF tracker visibly influenced by false detections, being pulled toward a coordinate value of approximately 150 during frames 15-17. In contrast, the DBF tracker maintains stable predictions under the same conditions. The middle and the right panels offer a quantitative comparison of KF and DBF in terms of intersection estimating bounding box positions in frames without detections. However, DBF demonstrates significant performance advantage in frames with low-confidence (c < 0.1) detections. This improvement can be attributed to DBF's flexibility, allowing it to udaptively decide whether to trust low-confidence observations or disregard them. Mean IoU
union (IoU). Both filters perform well in
300
fraction of frames with IoU=0.1
250
87
1.2-
DBF
0.7
0.8
DBF
1.0
KF
KF default
200
0.6
payal
100
150
Ground Truth
64
0.8
0.6
default
0.4
0.5
0.3
0.4
0.2
DBF
0.2
0.1
0
5
10
15 frame
20
25
30
0.0
High
Low
Missed
0.0
High
Low
Missed
Figure Left panel: coordinate of bounding box center estimated
DBF, Colored dots show coordinates bounding box reported by the YOLO model. The red band (frames 18 - 20) shows frames where the detection network reports no bounding boxes. tracker. Detections with a confidence score greater than 0.1 are categorized as
Middle panel: fraction frames with loU 0.1 "High", those below 0.1 as "Low", and those below 0.01 as "Missed". Right panel: mean IoU for the three categories. The performance gain DBF KF considerable frames with low-confidence detections.
C. Settings and Additional Results for Experiments
C.1. General setting
parametrization of the dynamics matrix We have parametrized the dynamics matrix A following Lusch et al. (2018): we consider that hamm/2 complex eigenvalues 2,(0 S < hdim/2) characterize A. Namely, is block-diagonal matrix
14

---

DBF for Bayes-Faithful DA
of h blocks. Each block consists of 2 x 2 matrix, whose components are:
coss (w:) sin(wa))
sin(w:)
Aslock = exp(p:)
(17)
cos(w,)
where p; = ReeAA and w; = Am[A.] In contrast to Lusch et al. (2018), we apply the same dynamics matrix at any positions on the latent space. We consider that this representation is sufficiently expressive, as it can express any matrix on a complex number field that is diagonalizable.
One key advantage of DBF is that augmenting the latent dimension only results in a linear increase in computational demand. This scaling is due to the efficient parametrization of the dynamics matrix, where the Block-diagone structure allows operations to scale linearly with the latent dimension. In contrast, methods such as Sequential Monte Carlo (SMC) suffer from exponential increases in computational demand as the latent space grows, assuming that the same density particles must be maintained to capture posterior distributions. This makes DBF particularly well-suited for high-dimensional systems where traditional methods struggle with computational complexity.
Computational resources
We conduct experiments cluster of V100 GPUs. Each GPU has memory of 32GB.
hyperparameters for training For all experiments, we have used Adam optimizer with default parameters. Table 5 shows hyperparameters employed in our experiments. Trainings for moving MNIST and double pendulum are conducted with one GPU, while that for Lorenz96 is with eight GPUs.
Table 5. Hyperparameters for training
Ir
batch size
haim
Ndata train
Epochs
train time per model
moving MNIST
10
8
480,000
3hrx 1GPU
double pendulum
10-3
256
50
1.0 107
6hrx JGPU
Lorenz96
3 x 10-3
64
800
2.6 x 107
15hrx 8GPUs
object tracking
8
C.2. Object Tracking
"Airplane" movies in the LaSOT dataset (Fan al., 2019; 2021). contains 20 movies. Each movie has least 1,000 frames. We chop the first 1,000 frames into 20 sets of 50 frames. Airplanes numbered one to ten are considered validation set used to determine the model hyperparameters. We use the remaining data (airplane-1 11 to airplane-20) as test set to evaluate the performance of the filters.
Dataset:
Dynamics model: Constant velocity model. The (x, y) coordinates and (uz, my) velocities of the top left and bottom right edges are the latent (physical) variables.
h+1 Fh
(18)
0
0
0
f1,t
1
0
dt
22,
F
0
1
dt
Jue
1/,t
(19)
0
0
0
0
0
0
U2i.
0
0
0
0
0
0
Uys
0
0
0
1
0
Uza.
0
Uy2.1
Here, 1 and Vı, stand for the coordinates of the left top edge of the bounding box, and 22,r and y/,, are the right bottom edge the box. Uri, Uyr,s U2 Uy2,, are velocities of box edges. dt is the time difference between frames, which we take as (arbitrary).
15

---

DBF for Bayes-Faithful DA
Network architecture: We a pre-trained detector YOLOv8n model (Jocher al., 2023). The detector yields the bounding box's position, X, and the box's confidence score, c. We set the detection threshold at 0.01. In cases where the detector reports multiple bounding boxes, we choose the one with the highest posterior probability. We use the bounding box coordinates as fo: = X. Several choices for the relation between confidence score and Go(or) are possible.
We experiment with linear confidence Go(ot) x c and squared confidence Go(ot) xx c². We determine the system noise factor for either dependence with the validation set. We use normalized precision as the evaluation metric (Müller et al., 2018). Figure 8 shows the normalized precision score for the validation set for the system noise factor. The system noise factor of 10 is chosen for KF. For DBF, squared confidence with the system noise factor of 10-2 is employed.
validation set
0.84
0.83
0.82
Precision
default
KF
DBF
0.80
DBF, squared
0.79
0.78
10-4
10-3
10-2
10
10a
10a
system noise factor
Figure Normalized precision scores for validation samples.
C.3. Double Pendulum
The dataset consists of 2D coordinates representing the positions of two weights. The training set includes 10, 240, 000 initial conditions, while the test set contains 10 initial conditions. The number of training samples is sufficiently large to ensure that the training converges. During DVAE training, we observed that some initial conditions resulted in training failure due to instability; however, we maintained the total number of training samples since the training was successful for at least one initial condition. Both datasets comprise 80 time steps. Numerical integration is performed using the solve_ivp function in SciPy, with relative tolerance (rtol) set to 10 and absolute tolerance (atol) set to 10-2.
Dataset:
schematic figure explaining the problem setting presented in panel (a) of Fig. 3 in the main text.
Dynamics model is described in bars is 1 [m], and the positions of the two pendulum weights are observable with Gaussian noise of 0.1, 0.3, or 0.5 [m]. The observation interval is 0.03 [s]. The task is to predict the positions of the two weights in the successive ten frames.
hewplleeenduuuubhiee html. The length of the
Network architecture: fo: A sequence of ten "linear blocks" composed of fully connected layers, layer normalizations, and skip connections. Namely, each linear block has three components:
fe: (input dimension) x (output dimension) linear layer,
norm: layer normalization,
skip: skip connection.
Taking four observation variables input, the first linear block expands the dimensionality to 100. The intermediate linear blocks maintain these 100-dimensional variables. The final linear block reduces the 100-dimensional input to a 50-dimensional output, representing 50 latent space variables. The ReLU activation function applied throughout the network. The structure of Ge mirrors that of fe. while de serves as the inverse of fo. The initial eigenvalues are randomly sampled from the range between e and 0.01
16

---

DBF for Bayes-Faithful DA
Table List of hyperparameters for double pendulum experiment.
parameter
value
Rinit
diag 1
Q
diagle [e $1
initial concentration parameter
Training: All training variables (network weights for the 100 (fs. Gø), the emission model operator o, eigenvalues À for the dynamics matrix A, Gaussian noise parameter o for angular velocity w, and the concentration parameter for Von Mises distribution used for angular coordinate 0) are trained together.
Examples: Here, we show examples for assimilated 0 and w in Fig. 10. Also, we give an additional figure for the RMSE of for various methods.
C.4. Lorenz96
The dataset consists of physical and observed variables sampled at 40 grid points, The training set includes 25,600,000 initial conditions, while the test set contains 10 initial conditions. The number of training samples sufficiently large to ensure that the training converges in most cases. The original datasets comprise 80 time steps. Numerical integration performed using the solve.ivp function in SciPy, with a relative tolerance rtol 10-2 and an absolute tolerance of atol= 10-2, Gaussian noise with standard deviations of 1,3, is added to all measurements.
Dataset:
For KalmanNet, attempted to train with 25,600,000 and 400,000 initial conditions; however, the process was terminated due to memory limitations. Consequently, we report results using dataset size of 120,000. For DKF, VRNN, and SRNN, we also tried training with 25,600,000 conditions, but all models encountered a RuntimeError due to instability during the backward computation. To obtain results, we reduced the number of training samples to 512,000. With this adjustment, both SRNN and VRNN successfully completed the training procedure for some initial conditions.
physical quantity 2j is defined each grid point j(1 S j S 40). The time evolution of this quantity is described by the following of differential equations:
dz(t); = (25+1 - dt
40)
(20)
In this equation, the driving term F is set to 8. The first term models the advection of the physical quantity, while the second term represents its diffusion along a fixed latitude. With these parameters, the evolution of the physical quantity exhibits chaotic behavior.
Network architecture:
The NN So consists of ten convolutional blocks followed by a fully connected layer. Each convolutional block comprises ID convolution, layer normalization, and skip connection:
convid: nn.Convide Gin Cout, kernel size=5, padding=2, padding mode="circular".)
norm: layer normalization,
skip: skip connection.
The first convolutional block has On 1 and Cout 20, expanding the input by factor of 20 the channel dimension. flattened into
The subsequent eight layers maintain 20 channels. Finally, the 20 channels and 40 physical dimensions 800-dimensional variables, which are then fed into fully connected layer of size 800 x 800. For all layers, the activation function used is ReLU. The function Gg is structured identically to fe, while Φe represents the inverse of fe.
Training: All training variables, including the network weights for the inverse observation operator fo and Gg. the emission model operator o, the eigenvalues for the dynamics matrix A, and the Gaussian noise parameter a,, are trained concurrently.
17

---

DBF for Bayes-Faithful DA
PF
PF
PF
target
target
target
3-
-2
10
target
2.0
20
50
00
01
20
30
40
60
70
10
20
40
50
100
30
40
60
70
2300
stop
PF
7.5-
target
target
target
2.5
2-
target
-2.5
-5
-5.0
-10
-7.5
80
5109
step
FF target
FF
taret
taget
3
target
30
20
30
40
50
60
70
80
1.0
30
60
70
10
20
40
50
60
70
80
10
30
60
5189
s28p
target
target
target
target
-2-
sta9
step
PP
PP
Pf
0.
3
target
-8-
0
10
20
30
40 stap
50
60
70
80
10
20
30
40 step
50
60
70
00
01
10
20
30
40 stop
5/0
60
70
00
10
20
30
40
50
60
70
10
Step
Figure 9. PF results with 100,000 paticles for five example data in test set. Two left columns show evolution of 01 and O2 (rad) (, therefore, the values eyclic with the period of 6.3, we corrected those periodic shifts) and two right columns show wi wa (rad/s).
18

---

DBF for Bayes-Faithful DA
6-
target
target
target
DBF target
100
DBF
7.3
DEP
DEF
DBF
3
0.0-
-5.0-
-7.5
20
20
30
40
50
60
70
B0
0
10
20
30
40
50
60
70
10
20
30
40
50
60
70
80
0
10
20
30
40
50
60
70
80
S109
Step
step
step
6-
target
target
target
DBF
target
DBF target
DBF target
DBF
DBF
10
20
30
401
50
64
60
01
10
20
30
401
60
70
10
20
50
40
50
60
70
10
01
10
20
30
60
70
5109
Stop
step
Step
DBF
tarqut
target
tareet
3-
-6-
-6-
target
10
40
70
20
50
60
80
10
40
60
20
50
80
Figure 10. Same Fig. 9 but for DBF with the latent dimension of 20.
0.3
DBF
EnKF
ETKF
0.2
PF
RMSE
VRNN
SRNN
0.1
DKF
0.0
0
30
40 steps
50
70
Figure 11. Assimilation results for the angle variable 0. All models successfully determine the angle coordinate in spite of the strong nonlincarity the observation (trigonometric function), Among these, performance is best.
19

---

DBF for Bayes-Faithful DA
Table 7. List of hyperparameters for Lorenz96 experiment.
parameter
value
Rinit
diag( 1]
Q
diagle $]
Examples:
We show an example figure for assimilation experiment with DBF Fig. 12.
C.5. Moving MNIST
Dataset: The dataset consists a series of 2D images, where each pixel has dynamic range from to 255. The training set contains 480,000 initial conditions, while the test set consists of ten initial conditions, with both datasets comprising 20 time steps each. The number training samples and epochs sufficiently large to ensure that the training converges effectively. Gaussian noise with standard deviation of a 50 is added all pixels. The MNIST images of the digits "9" (data point 5740) and "5" (data point 5742)
at constant speeds until they reach the edges, where reflection occurs.
Training: The network weights for G9 are fixed during the first epoch to facilitate the learning of fo and the image tensor for the observation model. Subsequently, Go is trained during the second epoch. In total, DBF undergoes training for two epochs.
Dynamics model:
Constant velocity model. The exact dynamics matrix we have used is:
20+1
Fz
(21)
dt
0
0
11,t
0
dt
0
0
31,t
0
0
0
dt
0
22,
F
0
0
0
0
0
1
0
1
0
0
0
0
dt
0
=t
Uxr
1/2t
(22)
0
0
1
0
0
0
1
0
Ura
Uya.1
and true observation model:
3: (Ix mod 16)
if x//16 is even
same for y
(23)
(x, mod 16)
x//16 odd
h(z)), dim(o:) = 44 x 44 28 28 image is embedded at(ă,, Tt..
(24)
The formulation above addresses image reflection through the observation operator, resulting in linear dynamics while permitting multiple solutions for each observed figure. This approach presents significant challenges for the EnKF, which assumes single-peak Gaussian distribution in the assimilating space. To ensure fair comparison, we revise the dynamics and observation models to allow for single solution for each figure. This adjustment notably enhances the performance of the EnKF if the image is provided. However, even with this modification, the EnKF fails to accurately estimate the position, velocity, and the embedded image.
Network architecture: fe: Two-dimension convolutional NNs. Below is the list of layers.
convl: an.Conv2d(1, 2, kernel size=3, stride=2, padding=1)
20

---

DBF for Bayes-Faithful DA
obs(nonlin)
target
estimate
20
40
time
60
80
observation dim.
Figure 12. An example of assimilation output in the experiment with nonlinear observation operator. The observation is not very informative due low threshold for saturation observed as 10 c). In the first 20 steps, the model output resembles little with the target. However, the step proceeds, the estimated state begins to capture features of the true state. Even with such a poor observation operator, DBF finds latent space representation that captures the evolution of the true state.
observation operator (0t,j = min(21, 10) + e,, all cells with zt,5 >> are just
21

---

DBF for Bayes-Faithful DA
Table 8. List of hyperparameters for moving MNIST experiment.
parameter
value
diag(e
Q
diagle ']
conv2: nn.Conv2d(2, 4, kemne_size=33, stride=2, padding=1)
conv3: an.Con22,(4, 4, kemeeLsize=3, stride=1, padding=1)
conv4: nn.Conv2d(4, 4, kerneLsize=3, stride=1, padding=1)
fc: Linear((11 x 11 x 4, 8)
The input image, sized 44 x 44, is sequentially processed by convolutional layers (conv1, conv2, conv3, and conv4). The output is then flattened serve the input for the fully connected layer (fc). Ultimately, this process yields eight variables for (ot). The network Ge follows the same architecture as fa. but it produces only the diagonal components of Go(or) through the NN.
Example figures: In Fig. 14, we show example images for observations and all the algorithms in image-informed setting.
Table 9. The success rates of different methodologies in the two-body moving MNIST problem. For the model-based approaches, we used the same dynamics observation models that generated data. For DBF, the model was initialized random image tensors and trained solely the data.
Method
Success rate
DBF
100% (50/50)
EnKF
58% (29/50)
ETKF
0% (0/50)
PF
0% (0/50)
D. Training Stability
We observe that the training of our proposed method is stable compared to RNN-based models. Fig. 15 shows the evolution of the real parts of eigenvalues. Although we do not impose constraints on the real parts of eigenvalues, the values only marginally exceed one. Therefore, long-time dynamics is stable during training.
E. Hyperparameter Study on the Latent Dimensions
The dimension of the latent variables is hyperparameter. We have tested the performance and computation (both training and inference) time for nonlinear problems.
E. double pendulum
E.I.I. ACCURACY-COMPPTE TRADE-OFF DBF
For double pendulum problem, we test with the standard observation operator with the observation noise of 0.1. Figs. 16, 17 show the relation between the RMSE and the latent dimensions of the system. Here, show results with 1.0 x 107 training data. For the double pendulum problem, we have tested with 4, 20, 80, and 200 latent dimensions. All the latent dimensions tested were too small to observe the impact of the compute-latent dimension relationship. To observe the slowdown, we need to test with higher dimensions. Please also refer to the results for Lorenz96. The performance (RMSE at the final 10 steps) for the angles 0 and angle velocities w are poor if the latent dimension is four. By leveraging 20 latent dimensions, DBF achieves very good assimilation performance. Further enhancing the latent dimensions to 80 and 200
22

---

DBF for Bayes-Faithful DA
data
t=0
t=2
t=4
t=6
t=8
t=10
t=12
t=14
t=16
t=18
obs
S
DBF
5
6
6
EnKF
ETKF
PF
data 20
t=0
t=2
t=4
t=6
[=8
t=10
t=12
[=14
t=16
t=18
obs
DBF
5
S
S
S
9
S
EnkF
ETKF
PF
data 27
t=0
2=
t=4
t=6
t=8
t=10
t=12
t=14
t=16
t=18
obs
S0
DBF
9
6
EnkF
ETKF
PF
Figure 13. Example figures for two-body moving MNIST experiment. This is the setting explained the main text. For all algorithms, the two embedded images explicitly informed: algorithms need to deal with many unknown parameters in the observation model.
23

---

DBF for Bayes-Faithful DA
data
t=0
t=4
t=12
t=14
t=16
t=18
obs
D
DBF
S
9
9
9
EnKF
S
9
6
59
69
6
ETKE
3
5
9
8
5
P
PF
9
9
9
data 20
t=0
t=4
t=8
t=10
t=12
[=14
t=16
[=18
obs
DBF
S
5
5
5
S
EnKF
6
9
5
5
9
9
4
59
ETKF
95
S
9
6
9
5
9
95
9
9
PF
9
data 27
t=2
t=4
t=6
t=8
t=10
t=12
t=14
t=16
t=18
6
obs
DBF
39
60
9
9
5
9
6
6
EnKF
8
9
9
5
D
ETKF
5
9
9
5
9
e
2
9
PF
9
9
8
9
9
Figure 14. Example figures for two-body moving MNIST experiment. For model-based approaches (EnKF, ETKF, PF), contrary to the experiment reported in the main text, the true images are informed. In data 0, both DBF and EnKF successfully determine and follow the position of the images. On the other hand, data 20 and 27, EnKF estimate becomes unstable soon after the letters overlap. Even in that situation, DBF stably follows the positions of the embedded images.
24

---

DBF for Bayes-Faithful DA
abs(elgenvalue)
absleigenvalue)
(asteigenvalue)
abs(eigenvalue)
abs(eigenvalue)
abs(eigenvalue)
Figure Evolution of histograms for the real parts 800 complex eigenvalues Lorenz96 experiment. Initially, eigenvalues are taken as one. As the model learns the dynamics, eigenvalues lower than 1.0 appear. However, the largest eigenvalue Amar mostly remains less than 1.02.
double pendulum, DBF, RMSE-latent dim
double pendulum, DBF, RMSE-latent dim
double pendulum, time
0.20
0.4
2.0
Inference time (s)
0.18
0.3
1.5
0.16
0.2
1.0
0.14
0.1
0.5-
0.12-
0.0
109
10t
0.0
10a
0.10
Figure 16. Left panel: RMSE as function of the latent dimensionality of DBF Right panel: the inference time as a function of the latent dimensionality of DBF.
did not improve performance. The training gradually gets slower when we use latent dimensions higher than 80. As can be seen from Fig. 16, The performance rather insensitive to the latent dimensions the range [20, 200]: the RMSE for 0.036 dim(hg) 20, 0.053 at (h:) = 80, and 0.044 dim(h;) 200 and for w is 0.265 dim(h;) 20, 0.375 at dim(h.) 80, and 0.302 dim(h,) 200.
1.2. COMPARISON TO THE PF
The performance of PF depends on the number of particles used. We have tested with 20, 200, 2,000, 20,000, and 100,000 particles. The performance for improves significantly if we use more than 200 particles. The RMSE for the angle velocities w almost saturates at RMSE ~ 0.31 when we use particles more than 20,000. To achieve that accuracy, the inference time required for PF more than 200 seconds per initial condition. On the other hand, DBF achieves slightly better performance (RMSE 0.265) with the latent dimensions of 20. The inference time for DBF 0.1 seconds per batch.
E.2. Lorenz96
1. CECURACY-COMPULS TRADE-OFF DBF
For Lorenz96 problem, we test with nonlinear observation operator with the observation noise of = Figs. 19, 20 show the relation between the RMSE and the latent dimensions of the system. Here, we show results with 1.0 x 10" training data. The dimensionality of the latent variables can be either larger or smaller than that of the physical variables. but there is trade-off: up to certain latent dimensionality, increasing the dimension improves performance at the cost of longer computation time. Beyond that point, increasing the latent dimensionality no longer improves performance but only increases training time (although inference time remains relatively short compared to model-based approaches). Therefore, the optimal balance depends on the specific problem. For the Lorenz96 system, a dimensionality of 800 was a reasonable trade-off among 20, 80, 200, 800, and 2,000 dimensions. As shown in the figure, the RMSE changes by only percent (1.31 vs 1.23) in the range from 200 2,000 dimensions, indicating that the impact critical in this range.
25

---

DBF for Bayes-Faithful DA
double pendulum, DBF, train time
double pendulum, DBF, test time
5.0
0.20
4.5
0.18
4.0
inference time [s]
train time [x10
3.5
0.16
3.0
2.5
0.14
2.0
0.12
1.5
1.0
0.10
100
10
10
10"
10
10a
latent dimension
latent dimension
Figure 17. Left panel: the training time for 1.0 10" initial conditions function of the latent dimension. Right pancl: RMSE function of the training time five different numbers latent dimensions.
double pendulum, PF, RMSE-particles (0)
double pendulum, PF, RMSE-particles (w)
0.5
RMSE
0.3
RMSE
3
0.2
0.1
0.0
'00
102
10a particles
10*
10
101
102
10a
10
10
particles
double pendulum, RMSE-test time (0)
double pendulum, RMSE-test time (w)
DBF
DBF
10"
RMSE
10
RMSE
10
10
10
10-1
100
10a
10a
103
10-1
100
10
10a
10a
inference time [s]
inference time [s]
Figure 18. Left panel: the performance of PF function of the particles used. Right panel: RMSE function of the inference time for the DBF and the PF. For the DBF, the latent dimensions are 20, 80, 200, 800, and 2,000. For the PF, the number of particles are 20, 200, 2,000, 20,000, 100,000.
26

---

DBF for Bayes-Faithful DA
Lorenz96, DBF, RMSE-latent dim
Lorenz96, DBF, test time
2.5
0.20
2.0
0.18
inference time [s]
1.5
0.16
RMSE
1.0
0.14
0.5
0.12-
0.0
0.10
10a
102
10a
10t
10
10a
latent dimension
latent dimension
Figure 19. Left panel: RMSE function of the latent dimensionality of DBF. Right panel: the inference time function of the latent dimensionality of DBF.
Lorenz96, DBF, train time
Lorenz96, DBF, RMSE-train time
5.0
2.5
4.5
2.0
4.0
3.5
1.5
RMSE
time
3.0
1.0
2.0
0.5
1.5
1.0
0.0
10
10?
103
104
3x10
latent dimension
training time [s]
Figure 20. Left panel: the training time for 1.0 10 initial conditions function of the latent dimension. Right panel: RMSE function of the training time for five different numbers of latent dimensions.
E.2.2. COMPARISON TO THE PF
The PF also has the trade-off. Although RMSE improves slowly increase the number of particles, the RMSE poor (2.27) compared to the DBF results (RMSE = 1.3) even with massively large number of particles (100,000) with very long inference time (2,000 seconds per initial condition)
27

---

DBF for Bayes-Faithful DA
Lorenz96, PF, RMSE-particles
Lorenz96, RMSE-test time
7
PF
PF
6-
6-
DBF
5
RMSF
RMSF
3-
3-
2-
2-
1-
1-
102
10a
104
10
10
10"
10 10a
particles
inference time [s]
Figure 21. Left panel: the performance of PF as a function of the particles used. Right panel: RMSE as function the inference time for the DBF and the For the DBF, the latent dimensions 20, 80, 200, 800, 2,000. For the the number particles are 20, 200, 2,000, 20,000, 100,000.
obs
true
mode((hdim=20)
model(hdim=800)
0
10
10
20
20
20
20
30
40
40
40
40
50
50
50
50
60
60
60
70
70
70
70
20
0
20
0
20
20
Figure 22. the performance of DBF for low latent dimension (dim(h.) 20) and high latent dimension (dima(h)) 800).
Even with the latent dimensions (20) smaller than that of the original state space (40), DBF shows the skillful assimilation. With higher latent dimensions (800), the performance further improves.
28


Reviews:
[{"review_id": "eDMfoXTGMM", "reviewer": "Reviewer_zP6r", "summary": "The authors propose a novel variational method for data assimilation that constructs its variational family by replacing the non-linear observation model by a linear-Gaussian observation model whose mean and covariance are parametrized by a neural network.\nIf the dynamics of the prior are also linear-Gaussian, then the variational family is a linear-Gaussian state-space model in which is tractable using the Kalman filtering recursions. Inspired by Koopman operator theory, the authors extend the method to nonlinear dynamics by learning latent linear dynamics in a higher-dimensional space that map to the physical dynamics through a learned nonlinear transformation.\nThis transformation then makes the original method for linear dynamics with nonlinear observation models applicable.\nThe approach is shown to be competitive to classical DA methods (EnKF, ETKF) and multiple deep learning-based approaches on three benchmark problems covering both linear and nonlinear dynamics.\n\n## Update after rebuttal\nI am still very concerned with the introduction of the auxiliary prior in the derivation of the method, which I feel significantly impacts clarity (cf. review `VtfJ`). Given how pivotal this derivation is for the rest of the paper, I do not feel comfortable to raise my score without having seen the updated section of the paper.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank Reviewer zP6r for the thoughtful comments and constructive suggestions. \n\n\n- Clarification on Section 2.3\n\nWe would like to respectfully clarify a key point regarding Section 2.3, which may have caused confusion due to insufficient clarity in our original exposition.\nThe reviewer pointed out:\n\n“Section 2.3 has significant clarity issues as it seems that the likelihood p(ot∣zt)p(o_t \\mid z_t) is replaced by a posterior distribution r(zt∣ot)r(z_t \\mid o_t) over the unobserved quantity.”\n\nWe acknowledge that the current presentation may have inadvertently led to this misunderstanding, and we appreciate the opportunity to clarify our intention.\nTo be precise, our method does not replace the likelihood $p(o_t \\mid z_t)$ with the posterior distribution $r(z_t \\mid o_t)$. Rather, $r(z_t \\mid o_t)$ is defined via Bayes' rule as being proportional to the product of the likelihood and a prior distribution $\\rho(z_t)$:\n$r(z_t \\mid o_t) \\propto p(o_t \\mid z_t) \\cdot \\rho(z_t).$\nNo approximation is introduced in this definition itself. However, the challenge arises when we aim to perform recursive Bayesian inference, where we must evaluate:\n$p(z_t \\mid o_{1:t}) \\propto p(o_t \\mid z_t) \\cdot p(z_t \\mid o_{1:t-1}).$\nEven if $p(o_t \\mid z_t)$ is Gaussian, the dependence of its parameters on $z_t$ can make the posterior $p(z_t \\mid o_{1:t})$ analytically intractable, particularly when the mean or covariance is a nonlinear function of $z_t$. To address this, we exploit the identity:\n$p(z_t \\mid o_{1:t}) \\propto \\frac{r(z_t \\mid o_t)}{\\rho(z_t)} \\cdot p(z_t \\mid o_{1:t-1}),$\nand assume that $r(z_t \\mid o_t)$, $\\rho(z_t)$, and $p(z_t \\mid o_{1:t-1})$ are all Gaussian distributions. This ensures that $p(z_t \\mid o_{1:t})$ remains Gaussian and can be computed in closed form, thereby maintaining tractability of the filtering process.\nThe approximations in our method thus lie in the following modeling choices:\n\n1. We model $r(z_t \\mid o_t)$ as a Gaussian distribution whose mean and covariance are parameterized by neural networks, denoted as $f_\\theta(o_t)$ and $G_\\theta(o_t)$, respectively.\n\n2. The auxiliary prior $\\rho(z_t)$ is assumed to be a Gaussian with fixed mean and fixed (large) covariance.\n\nIn principle, both the parameters of $r(z_t \\mid o_t)$ and those of $\\rho(z_t)$ could be optimized during ELBO maximization. However, as shown in Equations (4) and (5), if the neural networks $f_\\theta$ and $G_\\theta$ are sufficiently expressive, fixing the parameters of $\\rho(z_t)$ does not limit the representational capacity. The variability needed for inference is effectively captured by the learned functions $f_\\theta(o_t)$ and $G_\\theta(o_t)$. The large covariance of $\\rho(z_t)$ ensures that its support covers the latent space broadly enough for effective approximation.\nWe acknowledge that this important design rationale was not sufficiently explained in the original manuscript. We will revise Section 2.3 to clearly state the role of $\\rho(z_t)$, clarify that it is not a replacement for the dynamics-induced prior $p(z_t)$, and explain why it can be treated as a fixed auxiliary distribution without loss of generality.\nWe truly appreciate the reviewer’s insightful suggestion. We are confident that incorporating this clarification will significantly enhance the clarity and theoretical rigor of the manuscript.\n\n- Clarification regarding the approximation in $q(h_t \\mid o_{1:t}, z_{1:t})$\n\nWe appreciate the reviewer’s insightful question regarding the approximation introduced in the variational distribution.\nAs correctly noted, in general, $q(h_t \\mid o_{1:t}, z_{1:t}) \\neq q(h_t \\mid o_{1:t})$. In our method, we do introduce an approximation by replacing the former with the latter—i.e., we approximate the richer distribution $q(h_t \\mid o_{1:t}, z_{1:t})$ using a simplified form $q(h_t \\mid o_{1:t})$ for tractability.\nThis choice restricts the expressiveness of the variational family, but it still provides a valid lower bound on the marginal log-likelihood, and therefore constitutes a legitimate ELBO formulation.\nThe equality $q(h_t \\mid o_{1:t}, z_{1:t}) = q(h_t \\mid o_{1:t})$ would only strictly hold if the mapping from $z_t$ to $o_t$ were deterministic and invertible—i.e., when $z_t$ is fully determined by $o_t$. Since this is not the case in our setting, we acknowledge that the replacement is indeed an approximation.\nNevertheless, this design allows for a computationally feasible training procedure, and we empirically find that it strikes a good balance between tractability and performance. We will clarify this point in the revised manuscript to avoid potential confusion.", "claims_and_evidence": "- page 2, lines 68-69: The way this is written makes it sound like the Kalman filter only supports time-invariant dynamics and observation models, which is clearly not the case", "methods_and_evaluation_criteria": "- page 6, lines 285-288, left: The definition of success seems arbitrary. What is the motivation for choosing exactly this threshold?", "theoretical_claims": "- There are no theorems that require a proof.\n- I did not check the derivations of the ELBOs.\n\n- page 4, lines 177-180, left: It is unclear what is meant here. Is this an additional approximation introduced to make training tractable? Normally one is only at liberty to drop $z_{1:t}$ from $q(h_t \\mid o_{1:t}, z_{1:t})$ in case $h_t$ and $z_{1:t}$ are conditionally independent given $o_{1:t}$, which does not seem to be the case here.", "experimental_designs_or_analyses": "While the experimental design seems sound, some of the choices made appear somewhat contrived. For instance, this includes the nonlinear observation operator $o_{t, j} = \\min(z_{t, j}^4, 10) + \\epsilon$, whose physical significance is unclear, and the choice of the moving MNIST problem as the main benchmark problem for linear dynamics. The object tracking experiment in the appendix seems to be a better choice, as it is much more interpretable.", "relation_to_broader_scientific_literature": "The related work section is extensive and gives a detailed account of the commonalities and differences of the present work to the broader scientific literature.", "essential_references_not_discussed": "None that I am aware of.", "other_strengths_and_weaknesses": "Section 2.3 has significant clarity issues as it seems that the likelihood $p(o_t \\mid z_t)$ is replaced by a posterior distribution $r(z_t \\mid o_t)$ over the unobserved quantity. While this is remedied by dividing by a prior $\\rho(z_t)$ \"virtually introduced for the IOO\", it is unclear what the significance of this prior is in the context of the dynamics model, which already induces a prior $p(z_t)$ over the state. The fact that the covariance matrix of $\\rho(z_t)$ is fixed at $V = 10^8 I$ suggests that it is in fact irrelevant for the algorithm (see also equations 4 and 5) and only needed for the internal logic of the exposition.\nI would hence like to suggest an equivalent exposition of the algorithm that avoids the virtual $\\rho(z_t)$ prior and seems somewhat easier to follow: The authors aim at deriving a tractable variational family for a state-space model with linear-Gaussian transition model and nonlinear observation model. As noted in the paper, the nonlinear observation model hinders tractability of the original model and should hence be replaced, e.g. by a linear-Gaussian one in the variational family. Hence, inspired by the idea of an inverse observation operator, the variational family approximates the observation model by $p(o_t \\mid z_t) \\approx q(o_t \\mid z_t) \\propto \\mathcal{N}(f_\\theta(o_t); z_t, G_\\theta(o_t)),$ which is linear-Gaussian. The dynamics of the variational family are simply given by the dynamics of the prior, i.e. $q(z_1) \\coloneqq p(z_1)$ and $q(z_{t + 1} \\mid z_t) \\coloneqq p(z_{t + 1} \\mid z_t)$. Then the conditional distribution $q(z_t \\mid o_{1:t})$ can be computed by the Kalman filter recursion as in the paper. And used to compute the ELBO.\nNote how this \"likelihood-approximation\" perspective renders the unintuitive virtual prior $\\rho(z_t)$ unnecessary.\nI'm very open to a discussion about resolving the clarity issues in this section and will likely raise my score should this be addressed satisfactorily.", "other_comments_or_suggestions": "- page 1, lines 14-15, right: Does the term \"test distribution\" refer to the approximate filtering/smoothing posteriors provided by the respective methods?\n- page 2, line 109, right: The use of $h_t$ in the context of linear dynamics is somewhat confusing, should this maybe read $r_\\text{true}(z_t \\mid o_t)$? This also applies to the first paragraph of Section 2.5.\n- page 3, line 127, left: Missing \"model\" in \"linear-Gaussian state-space *model* (LGSS*M*)\"\n- page 3, line 122, left: \"Panel (b)\" instead of \"Panel (c)\"\n- page 3, line 128, right: Why is the physical state referred to as a \"teacher signal\" here? This term is not previously defined.", "questions_for_authors": "- Strategies 1 and 2 don't seem mutually exclusive. Which one should be used in which case? An experimental analysis comparing the two strategies might grant a lot of insight into the differences.\n- Did you try to apply the approach to time-invariant dynamics? At least for linear dynamics or when using Strategy 2 for nonlinear dynamics, the method seems to be applicable to time-varying dynamics.\n- What is meant by \"marginalizing over $h_t$ with this emission model\"? Surely this marginalization can't be performed in closed form?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lOLhCgVjqe", "reviewer": "Reviewer_VtfJ", "summary": "This paper proposes a method for Bayesian filtering with nonlinear observations and dynamics, using a VAE specialized to Markov processes with linear latent dynamics. The result is a closed-form loss that can be optimized for both the encoder (latent state) and decoder (inverse observation operator, IOO).", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We gratefully acknowledge Reviewer VtfJ for the positive evaluation and insightful feedback.\n\n- Direct prediction of $o_t$:\n\n We agree and will explicitly clarify that Eq. (6) is particularly useful for cases where direct observation prediction is sufficient, even when the mapping from $z$ to $o$ is nonlinear.\n\n- Applicability to non-Markov settings:\n\n We confirm that even if $z_t$ is not Markov, DBF extends naturally if there exist a Koopman embedding that satisfies $h_t = g(z_t))$, where $h_t$ is Markov. We will clarify this explicitly in our revision. We would like to thank VtfJ again for the insightful comment.\n\n- Virtual prior $\\rho$:\n\n We recognize that our explanation was insufficient. We have now clarified this point in the main text and provided additional details in Appendix A.4.\n\nWe would like to thank VtfJ again for acknowledging the value of our methodology.", "claims_and_evidence": "The DBF shows strong improvement over relevant baselines.", "methods_and_evaluation_criteria": "The experiments are a challenging set of nonlinear filtering tasks.", "theoretical_claims": "I have convinced myself of the relevant derivations. The framework is very elegant.\n\nIn strategy 1, I think it would help to give more details on learning the map $z\\mapsto A$ or $A\\mapsto z$ from samples of $z_t$.", "experimental_designs_or_analyses": "No concerns", "relation_to_broader_scientific_literature": "Relevant literature is discussed and it’s clear how the present method advances over previous ones.", "essential_references_not_discussed": "None noted", "other_strengths_and_weaknesses": "Despite its flexibility the IOO still assumes the log-likelihood is quadratic in $z_t$ which is a major simplification for many applications.", "other_comments_or_suggestions": "There are many settings where we don’t care about $z$ and just want to predict $o$. In that case the linear method of eq 6 can be used even if the mapping from $z$ to $o$ is nonlinear, because we only need $h$ while $z$ can be ignored.\n\nLast line of page 2: I think you mean $h_t$ to be $z_t$ since $h$ has not been introduced yet.\n\nFig 1 caption: (c) should be (b)", "questions_for_authors": "I believe the Koopman operator works even for non-Markov processes: the sequence $h_t = g(z_t))$ can be made Markov even if $z_t$ is not (e.g., if $z$ is an AR-$k$ process for some $k>1$). Does this make DBF applicable in non-Markov settings?\n\nIs the virtual prior $\\rho$ necessary? Conceptually it’s strange because the prior is already present as $p(z_t|o_{1:t-1})$. Mathematically it’s redundant because it could be absorbed into $G$ as $(G_\\theta(o_t)^{-1} - V^{-1})^{-1}$.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oRjAAOWJ1Z", "reviewer": "Reviewer_gEdD", "summary": "In order to address the challenge of nonlinear filtering, the present work proposes _Deep Bayesian Filter (DBF)_, which leverages a learnable inverse observation operator (IOO) to transform the problem into a linear filtering problem, where one can apply standard Kalman methodologies. The cases of linear and nonlinear dynamical systems are treated differently: In the linear dynamics setting, having access to the IOO directly enables linear updates in physical space. In the nonlinear setting, an extra mapping to a high-dimensional latent space is required, where the hidden dynamics become approximately linear.  Training is performed once offline in a manner similar to VAEs. The model is tested on various benchmark dynamical systems, both linear and nonlinear, to demonstrate the robustness and performance gain achieved by DRF over classical filters.\n\n## Update after rebuttal\nThe authors have adequately addressed my main concern regarding the scalability of the algorithm during the rebuttal period. Therefore, I have increased my evaluation of the paper from a 3 to a 4.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We deeply appreciate Reviewer gEdD for their thoughtful and constructive feedback. Below, we specifically address the reviewer’s important concerns.\n\n- Scalability in High-Dimensional Settings:\n\nWe have conducted experiments varying latent dimensions in both the double pendulum and Lorenz96 settings, as reported in Appendix E of the original manuscript. Although we have not explicitly tested DBF in higher-dimensional physical systems, we expect that the latent dimensions required to accurately model the system dynamics would be substantially lower than the actual physical dimensions due to significant redundancy in degrees of freedom in large simulations.\nMoreover, even if higher dimensions were required, DBF remains resilient against explosive growth in computational demands. This aspect is detailed clearly in section C.1 of the Appendix, highlighting DBF’s efficient parameterization strategy that enables linear scaling with respect to latent dimension size, significantly alleviating the computational bottlenecks associated with traditional methods.\n\n- Partial Observations and Time-Varying Observation Operators:\n\nDBF finds no difficulty in handling partial observations. We need specific strategies to apply DBF for observation operators that vary over time. Amortizing a single inverse observation operator (IOO) across different observation operators can be indeed challenging. Future work could explore methods to learn conditional IOOs or multiple specialized IOOs to effectively handle scenarios with time-varying observation operators.\n\n- Criteria for Choosing Latent Space Dimension:\n\nThe choice of latent space dimension primarily depends on the complexity of the underlying dynamics. In practice, we recommend selecting the smallest latent dimension that achieves good validation performance. Indeed, we have already conducted an extensive ablation study on latent dimension sizes in Appendix E of our manuscript. The results offer valuable insights and suggest practical heuristics for balancing model performance and computational efficiency.", "claims_and_evidence": "The claims in this paper are supported by clear and convincing evidence. In particular, they identify examples where the proposed model, DBF, significantly outperforms both classical DA methods and Dynamic VAEs. Results are presented clearly and demonstrate the benefits of using DBF.", "methods_and_evaluation_criteria": "The datasets used to evaluate the method are appropriate for the application at hand. The authors mainly evaluate the RMSE, which, however, can be limiting as the ability to quantify uncertainties is also crucial in data assimilation. Regarding uncertainty quantification, only the Jeffreys divergence of normalised errors with the standard Gaussian is presented in the example of the nonlinear pendulum. However, I believe that it is also worth evaluating other UQ metrics, such as the negative log-likelihood and the CRPS, which are commonly used metrics to compare the UQ ability of DA methods.", "theoretical_claims": "The main contribution of the paper is in the model proposal and empirical evidence of it working in certain settings where classical DA methods tend to struggle. There are no proofs to check.", "experimental_designs_or_analyses": "The experimental setup is sound, evaluating the methods and baselines on commonly used benchmarks. The analysis of the experiments is also reasonable, and the arguments made are convincing. However, I would again point out the lack of comparison in the ability of the proposed model to perform UQ. In addition, some ablation study on the various components of DBF would be useful to have as well. For example, an ablation on the number of hidden state dimensions (in the case of nonlinear dynamics), a comparison of the two training strategies, etc... I would also think that having a discussion on what happens when you observe only a fraction of the components of Lorenz 96 model would be valuable too, as this is a common setting in DA (since it is rare in practice to observe the full state).", "relation_to_broader_scientific_literature": "The present work builds on a body of literature that aims to augment data assimilation/stochastic filtering with deep learning methodologies. However, in contrast to previous efforts like KalmanNet or methods based on dynamical VAEs, which use RNNs to model the latent dynamics, the present work uses explicit updates by assuming linearity of dynamics in the latent space. This helps prevent difficulties in training of RNN-based methods arising from problems such as vanishing gradient and Monte-Carlo approximation of the ELBO.", "essential_references_not_discussed": "The authors cover related literature in Section 2.6 fairly extensively. I would also point out that there is a line of works that has been coming out recently that use generative modelling techniques for data assimilation. For example,\n\n- A Score-based Nonlinear Filter for Data Assimilation, Bao et al. (2023)\n- DiffDA: A Diffusion Model for Weather-scale Data Assimilation, Huang et al. (2024)\n- Score-based Data Assimilation, Rozet and Louppe (2024)", "other_strengths_and_weaknesses": "__Strengths:__\n- The presentation and organization of the paper are well thought-out, making it easy to follow.\n- The experiments are fairly extensive, covering a range of examples where DBF can be beneficial over classical filters.\n- The \"Bayes-faithful\" property of DRF is appealing, which is missing in most deep learning-based approaches. However, I also wonder how \"faithful\" it is in terms of its ability to approximate the true non-Gaussian posterior.\n\n__Weaknesses:__\n\nI think the biggest flaw of the proposed method is in its scalability to high-dimensional settings, where computation of the filtered mean and covariance (1)--(2) becomes intractable due to the cubic/quadratric scaling (with respect to dimension) in computational cost/memory. This is also the problem with Kalman filters in high dimensions, which is why ensemble methods like EnKF and ETKF were proposed in the first place. Especially when considering SSMs with nonlinear dynamics, DBF requires lifting the dynamics to an even higher dimensional space, where applying (4)--(5) will become too expensive to run, or otherwise one has to settle for latent dynamics in a lower dimensional space, where results may not be very accurate. Hence, the applicability of the method lies in low-to-moderate dimensional systems, in which case other classical Kalman-based methods like the Extended Kalman filter (EKF) and Unscented Kalman (UKF) filter can be applied. In particular, the UKF is known to perform well and robustly in such settings, hence, I believe it is only fair to compare with this, too.", "other_comments_or_suggestions": "- I believe the $r_{true}(h_t | o_t)$ at the end of page 2 is supposed to be $r_{true}(z_t | o_t)$.\n- I think the \"rightmost table\" at the end of page 5 is referring to Table 1, which is not \"rightmost\".\n- It will be better to add standard deviations in the results in Table 3, as done in the other tables.", "questions_for_authors": "- Can the DBF framework deal with partial observations (i.e., only observing a few components of the L96) and observation operators that change with time (e.g., observing different components at each time step)? I believe these are important cases to consider as they occur commonly in practice. However, especially in the case where the observation operator changes with time, this does not seem straightforward, as one needs to amortize the inverse observation operator across various observation operators.\n- In the setting of nonlinear dynamics, is there a general criteria for choosing the latent space dimension? i.e., how do we know a priori what latent space dimension should be chosen for the hidden dynamics to be approximately linear? It would be interesting to see an ablation study in the hidden dimension size.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes a deep learning approach to Bayesian filtering in nonlinear discrete-time systems.
The introduction of a Gaussian inverse observation operator along with variational inference ensures the assimilation can take place in a linear space (thus Gaussian inference).

The paper is generally well received but some points should be addressed. Namely:

* The details regarding the likelihood approximation via inverse observation operator need to be significantly clarified. I.e. is the current formulation required to ensure that you know the approximation of p(z | o) up to and incuding normalization? or some other subtle point?
* The definition of "success" in the empirical validation ought to be more clearly motivated.
* The scalability of the method ought to be more carefully explained.
Decision: Accept (poster)