Implicit Maximum a Posteriori Filtering via Adaptive Optimization
Authors: Gianluca Bencomo
Jake Snell
Thomas L. Griffiths
Venue: ICLR 2024 poster (2024)
Keywords: Bayesian filtering
optimization
neural networks
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
TL;DR: We can define the Bayesian filtering problem as optimization over a time-varying objective, which makes it possible to define implicit filters based on optimization that perform well.

Abstract: Bayesian filtering approximates the true underlying behavior of a time-varying system by inverting an explicit generative model to convert noisy measurements into state estimates. This process typically requires matrix storage, inversion, and multiplication or Monte Carlo estimation, none of which are practical in high-dimensional state spaces such as the weight spaces of artificial neural networks. Here, we consider the standard Bayesian filtering problem as optimization over a time-varying objective. Instead of maintaining matrices for the filtering equations or simulating particles, we specify an optimizer that defines the Bayesian filter implicitly. In the linear-Gaussian setting, we show that every Kalman filter has an equivalent formulation using K steps of gradient descent. In the nonlinear setting, our experiments demonstrate that our framework results in filters that are effective, robust, and scalable to high-dimensional systems, comparing well against the standard toolbox of Bayesian filtering solutions. We suggest that it is easier to fine-tune an optimizer than it is to specify the correct filtering equations, making our framework an attractive option for high-dimensional filtering problems.

Published as a conference paper at ICLR 2024
IMPLICIT MAXIMUM A POSTERIORI FILTERING VIA ADAPTIVE OPTIMIZATION
Gianluca M. Bencomo', Jake C. Snell' & Thomas L. Griffiths
Department of Computer Science, Princeton University
Department of Psychology, Princeton University
{gb5435, jsnell, t tomg apmnneponnet
ABSTRACT
Bayesian filtering approximates the true underlying behavior of time-varying system by inverting an explicit generative model to convert noisy measurements into state estimates. This process typically requires either storage, inversion, and multiplication of large matrices or Monte Carlo estimation, neither of which are practical in high-dimensional state spaces such as the weight spaces of artificial neural networks. Here, we frame the standard Bayesian filtering problem as op- timization over a time-varying objective. Instead of maintaining matrices for the filtering equations simulating particles, specify optimizer that defines the Bayesian filter implicitly. In the laussina setting, we show that every Kalman filter has equivalent formulation using K steps of gradient descent. In the nonlinear setting, our experiments demonstrate that our framework results filters that are effective, robust, and scalable high-dimensional systems, compar- ing well against the standard toolbox of Bayesian filtering solutions. We suggest that it is easier to fine-tune an optimizer than it is specify the correct filter- ing equations, making our framework an attractive option for high-dimensional filtering problems.
INTRODUCTION
Time-varying systems are ubiquitous science, engineering, and machine learning. Consider multielectrode array receiving raw voltage signals from thousands of neurons during a visual per- ception task. The goal to infer some underlying neural state that is not directly observable, such that we can draw connections between neural activity and visual perception, but raw voltage signals are a sparse representation of neural activity that is shrouded in noise. To confound the problem further, the underlying neural state changes throughout time in both expected and unexpected ways. This problem, and most time-varying prediction problems, can be formalized as a probablistic state space model where latent variables evolve time and emit observations (Simon) 2006). One solution to such a problem is to apply a Bayesian filter, a type of probabilistic model that can infer the values of latent variables from observations.
Typically, Bayesian filters require matrix storage, inversion, and multiplication or the storage of particles, which are samples from the filtering distribution at every time step. In large state and observation spaces, the computational costs associated with both of these approaches render them impractical (Raitoharju & Piché 2019). In addition, since most time-varying systems do not have ground-truth states available, accurately estimating the process noise - the variability of the under- lying latent variables - is nearly impossible. This problem is exacerbated continuous systems where we need to perform numerical integration, which introduces additional uncertainty. Existing Bayesian filters are sensitive to process noise and dynamics misspecification, making them hard to use in these settings (Mehral 1972). Most filters are highly structured, requiring explicit specifica- tion of distributional assumptions. To be usable in practical settings, we need a system that remains effective when scaled to large state spaces (being applicable to weather systems, neural recording, or even modeling the evolving weights of an artificial neural network) and is robust to misspecification of the process noise and dynamics (which is almost guaranteed).

---

Published as a conference paper at ICLR 2024
The practical problems that arise when training deep neural networks (DNNss not dissimilar from those that make Bayesian filters difficult to work with: high- nigh-dimensional state spaces, nonlin- carities, and nonconvexity. These pressures have made way for adaptive optimizers (Duchi et al. 2011 Tieleman & Hinton 2012 Zeiler 2012 Kingma & Ba 2013) that offer a largely prescriptive solution for training DNNs, succeeding many previous approaches to optimizing neural networks that include applying Bayesian filtering equations (Haykin 2004). To efficiently and effectively overcome the difficulties of training DNNs (Glorot & Bengio 20T0), modern optimizers invariably make use of (i) the empirical Fisher information to crudely incorporate curvature at very little com- putational cost (Martens
2020 and (ii) momentum increase stability and the rate of convergence (Sutskever et al. 2013). We show that these same methods can be applied to Bayesian filtering in an framework.
extremely practical and theoretically-motivae
In the following sections, we cast Bayesian filtering optimization over time-varying objective and show that much of the information that explicitly defined by structured filters can be implic- itly internalized by optimizer hyperparameters. In doing so, we solve many of the scalability challenges associated with Bayesian filters and make Bayesian filtering easier to implement, espe- cially for the well versed deep learning practitioner. We show that our proposed method, Implicit Maximum a Posteriori (IMAP) Filtering, is robust to misspecification and matches or outperforms classical filters on baseline tasks. We also show that it naturally scales up to high-dimensional prob- lems such as adapting the weights of a convolutional neural network, and that it performs particularly well in this setting. We argue that is casier to specify an optimizer than it to correctly identify classical filtering equations, and our results illustrate the benefits of this implicit filtering approach.
2
BACKGROUND
We begin with an introduction to the filtering problem. Assume a discrete-time state space model with a Markov process over the states Xp.T A Xo, XI...., , XT and a sequence of observations YI:r A .2..2....,,> which are conditionally independent given the corresponding state. The joint probability of the states and observations is
p(xuT, p(xo) II
where p(xx) the initial distribution, p(x, xx-1) is the transition distribution, and p(y, xx) the likelihood. The posterior over the joint state X1:t of the system given all of the observations up until the current time Yıa given by:
t p(xialyaa)= - / p(xu) II
p(x,|x,-1)ptt ()((,(x)))) dxo:
P(y=1110-11
where p(y1)y1:0) p(y1). ). In sequential estimation, we are often only interested in the marginal of the current state p(x, i.r),, known as the filtering distribution. We normally want to compute or approximate this distribution such that the requisite computational resources do not depend on the length of the sequence. This can be accomplished by first initializing p(xoly - p(xo), and then forming a predictive distribution via the Chapman- Kolmogorov equation.
p(xely1a-1) / p(x/xx-3)p(xx-111.
(3)
and updating that predictive distribution, in light of measurements yc. by a simple application of Bayes' rule for every time step 1,...;
p(xe\V1=+)= (x)x)x))((xxxxee)
(4)
plyely1:--1)
2.1 KALMAN FILTER FOR LINEAR-GAUSSIA SYSTEMS
When the transition distribution and likelihood are linear and Gaussian, we can compute optimal estimates in closed-form via the Kalman filtering equations (Kalman 1960). Assume the following
2

---

Published as a conference paper at ICLR 2024
state space model:
p(x0) Mx No: En).
(5)
p(x, x(-1) Nx(x 11 Qt-1),
(6)
plyr xx) (y: H2,,, R,),
where F and H. are the transition and observation matrices, respectively, with process noise Qr-1 and measurement noise R. The Kalman filter propagates the filtering distribution from the previous timestep p(x--1 Y1a-1) ) = (x P4-11 Ex-1) with the predict step.
p(xgly1=-11) = (x; 14 _T). where μ4i 14-1 E and then updates that distribution as follows:
F
F
P
Q1-1,
(8)
((x,/y1:) = =x(x P1: Ee), where ui + K,Vi, E (2) + H, R2 H.
(9)
=== Hi is the prediction error, and K; E, H (H H, + R.) -1 known as the Kalman gain. We discuss further details in Appendix A.I.I
2.2 APPROXIMATE BAYESIAN FILTERING FOR NONLINBAR SYSTEMS
In real-world systems, guarantees for optimality break down due inherent nonlinearities and their (at best) approximate Gaussianity. The broader Bayesian filtering community has been motivated by developing methods that can operate in these more realistic settings (Särkkä & Svensson) 2023) Classical solutions include the extended Kalman filter (EKF) and iterated extended Kalman filter (IEKF) that make use of Jacobian matrices to linearize both the dynamics and observation mod- els, and subsequently most popular filtering implementations but only work well in mildly nonlinear systems on short timescales (Julier & Uhlmann 2004). The unscented Kalman filter (UKF) (Julier al. [995] Julier Uhlmann] 2004). like the particle filter (PF) (Doucet et al. 200Tp, is better equipped to handle highly nonlinear systems but both of these methods suffer from the curse of dimensionality and sen- sitivity to misspecified dynamics. Nonetheless, EKFs, IEKFs, UKFs, and PFs are the most promi- nent filtering solutions found throughout industry and research, leveraging expensive lincarization techniques or particle approximations to treat nonlinearities. We provide complete details for these classical approaches to nonlinear filtering in Appendix A.1
the Kalman filtering equations (Gelb 1974p. These are arguably the
BAYESIAN FILTERING AS OPTIMIZATION
The update step 5 in Bayesian filter is straightforward application of Bayes' rule with the predictive distribution p(x:\y1=-1) playing the role of the prior. In linear-Gaussian systems, this posterior distribution is available in closed-form
(9) but one could alternatively achieve an equivalent result via gradient descent due to its connection to regularized least-squares (Santos 1996). this section, describe this connection and how an optimization procedure
implicitly define
prior distribution, a kind of dual formulation of Bayes' theorem (see Figure 1). Such formulation reduces Bayesian inference to point estimates, but this can be more practical for nonlinear systems whose filtering distributions (xa)) are not analytically tractable in general.
3.1 FINDING THE POSTERIOR MODE VIA TRUNCATED GRADIENT DESCENT
The mode of the filtering distribution can be expressed the minimizer of the negative log posterior:
#4 arg min E, (x,), where (x,,) log p(y, xx,) log p(xp/((...))) (Y::.-)).
(10)
xi
In the case of the Linear-Gaussinn system, the regularized loss function is (up additive constant):
(x
2 ll - H R. + X 14 1|
(11)
where ||x||/-
z. Santos 1996) observed that the minimizer of a regularized least squares
problem in (1I) can be recovered by performing truncated gradient descent on the likelihood term alone. Specifically, let (1) = μ, and define the recursion
11 (k+1) +- (1 M.H. R, ty: H.ju for h: 1,..., K 1.
(12)
3

---

Published as a conference paper at ICLR 2024
Explicitly Defined Priors
Beyes' Rule
Optimization
Implicitly Detined Priors
p(s)
pixz)
pix))
pix.
p(x)
Licilooo
-2
-2
Explicit Bayesian Inference
Implicit Bayesian Inference
Figure 1: Explicit Bayesian inference (left) uses Bayes' rule balance information from the like- lihood and explicitly defined priors. x are MAP estimates three different posteriors, respective the priors shown blue, yellow, and purple. Implicit Bayesian inference (right) defines an opti- mization procedure that reverses the directionality of Bayes' rule by defining procedure to estimate (z) ....,,
the posterior mode, which corresponds to implicitly defined prior. Three trajectories
(r?",...) (r3 "...) of gradient descent are shown with a fixed learning rate over 3, 4, and steps, respectively. Taking the solutions produced by gradient descent as MAP estimates. we can use the corresponding gradient steps to determine the prior distributions that they implied (Santos 1996).
Let B, simultaneously diagonalize (E,)-1 and R, H2 such that ( (,, B, I and B, H. R, HH,B, diag(ri.... ,T. Such a matrix B, guaranteed exist since both (E, -1 Tx).
and H, "H, are symmetric and (E,) is positive definite (Golub & Van Loan 2013 Sec. 8.7).
The columns of B, the generalized eigenvectors of H, R, RH with respect (E, (- and
TI...., I' are the corresponding eigenvalues. By Theorem 3.1 of Santos (996) μ4 (K) defined in (12) the minimizer of 11) when M, = B,diag(AI,.... A, B,T where X; (1/r:)(1- (1))))))r----- 17k) r; # 0 and >; otherwise. This suggests an optimization wherein μ4, is computed via K steps of gradient descent on the likelihood, where M, is the learning rate matrix induced by M,
3.2 TRUNCATED GRADIENT DESCENT PRIOR SPECIFICATION
Instead of computing the learning rate matrix M, for given E,- consider specifying M, directly. For example, M, = pI recovers gradient descent on the likelihood with learning rate p. Let C, be chosen such that C M, C, I and C, H. R. H,C, diag(s1,..... :.)))............ As for B, above, such a C, will exist since both M, and H, R, H. symmetric and M,"' is positive definite. Then, by Theorem 3.2 Santos (196).. pi (K) as defined by (12) is the minimizer of (11) where Σ, C,diig((i,...... C, where a; = (1/8;)((1
-1 if 8; A and Ø. = otherwise. Thus, in linear-Gaussian system, specifying the learning rate matrix M. implicitly defines an equivalent predictive covariance E, and truncated gradient descent on the likelihood recovers the mode of the filtering distribution.
3.3 ALTERNATIVE INTERPRETATION AS VARIATIONAL INFERENCE
The procedure shown above can be alternatively motivated by positing a variational distribution q: (x,) = Nxx, m. M,) with fixed covariance M, We show in Appendix A.2 that the truncated optimization of [2) is equivalent to natural gradient ascent on the ELBOO. The choice of M, then specifies the covariance of the variational approximate to the filtering distribution, the learning rate matrix of the optimization, and the effective prior covariance. More generally, Khan & Rue (2023) demonstrates that many popular adaptive optimization techniques such as RMSprop and Adam can be interpreted as performing variational inference with different choices of qe (xx). In this light, the choice of the optimization algorithm when combined with & steps of optimization implicitly defines a corresponding prior distribution. In the next section, we propose a simple yet effective filtering algorithm based this interpretation of the update step.
4

---

Published as a conference paper at ICLR 2024
4 IMPLICIT MAXIMUM A POSTERIORI FILTERING
In this section, we state the Bayesian filtering steps of our approach, which (partially) optimizes the likelihood p(y=|x²) at every time step. The filtering distribution at each time is represented by its mean estimate fr. In addition to the likelihood, we also assume knowledge of a sequence of transition functions f((x;-1) - E[x, [x,1]- This is less restrictive than fully specifying the transition distribution, as done standard Bayesian filtering. The overall algorithm for the Implicit MAP Filter is summarized in Algorithm
Algorithm Implicit MAP Filter
Algorithm 2 Update step of Implicit MAP Filter: (MAP-UPDATE(4 K)
Input: Timesteps T, initial state estimate jo. sequence of loss functions . lT where ex)(-(((x) & p(y: X = x), sequence of transition functions f1...., fr where fx(x) E|x, Xx,/x,-1= = x]. optimizer M, number of opti- mization steps K.
Input: Initialization μt "",loss function L, op- timizer M, number of optimization steps
K.
Output: State estimate jt.
m
1
Output: Filtering state estimates f.,,..., frr
for
k 0 to
do
for 1 to do
g
(k) t Vx((x)] m
H4 +- fe(
(k+1) mkk)
g(0kk-))))
u < IMAP-UPDATE(44, (,,
end for
end for
f m() (K)
4.1 PREDICT STEP
The predictive distribution for the Implicit MAP Filter is obtained by applying the first-order delta method (Doriman 1938 Ver Hoei 2012) to the Kolomann-oooooroov equation [3]
/ p(xx x x(-1)p(Xx-1 \1o-1)dxx;11
p(x; Y1a-1)
p(x: X (1-1).
(13)
The mean of the predictive distribution is thus obtained by applying the transition function to the previous state estimate: E[x; [ As fe(p)-1). This negates the need for matrix-matrix products as is commonly required for the predict step.
4.2 UPDATE STEP
The update step is simply the application of optimizer M for K iterations, considering only an ini- tialization μ4 observation yı. and loss function &, Here, M takes as input the history gradients and outputs a vector to be added to the current state. This formulation captures many popular op- timizers, including SGD, RMSProp, and Adam. Our algorithm for the update step of the Implicit MAP Filter is provided in Algorithm 2]
When choosing M and K, the goal to specify an inductive bias that corresponds to the true filtering equations for the system. Choices such as increasing K and the learning rate increase Kalman gain (De Freitas et al. 2000). The nature of each optimization step, as well as K, defines the shape and entropy of the filtering distribution assumed (Duvenaud et al. 2016). Changing the initialization ,4 given by the predict step modifies the location of the implicitly defined prior distribution within the latent space. This concert of initialization 14, optimizer M, and number of iterations K produces state estimate fa that maximizes the posterior with respect to an implicitly defined prior. The aim when specifying these quantities is to define the correct posterior.
5 EXPERIMENTAL EVALUATION
The goal of our experiments is to assess whether the most popular adaptive optimizers can be used to design effective, robust, and scalable Bayesian filters. We begin with simple low-dimensional system with nonlinear dynamics to establish that our approach is competitive with standard filters. Then, more challenging system that has been used benchmark for these algorithms,
5

---

Published as a conference paper at ICLR 2024
True State
Observations
Mean
UKF
MAP
True State
30
20
24
-10
-20
-30
0
25
50
75
100
125
150
175
200
0
25
50
75
100
125
150
175
200
Timestep
Timestep
Figure 2: Toy nonlinear system (Q:-1 3, R = 2) for a single random trajectory (left) and the respective filter estimates (right). Implicit MAP Filter (MAP) shown uses the Adam optimizer with 50 steps at learning rate 1) = 0.1 and exponential decay rates B1. 32 0.1.
demonstrating our advantages. Finally, we show how Implicit MAP Filtering can be effective in high-dimensional settings, such as adapting the weight space of a convolutional neural network.
5.1 TOY NONLINEAR SYSTEM
The equivalence between Kalman filtering and steps of gradient descent elucidated in Section 3.2 suggests that the proposed framework should produce reliable estimates in approximately linear- Gaussian systems. To better understand filtering amidst nonlinearites, we first consider the one- dimensional example used in Douueet al. (200T): /ordon et al. (1933); Kitagawa (1996), originally proposed by Netto et al. (1978), which admits highly nonlinear periodic behavior (see Figure 2):
1 X1
X1-1
x
x + 25
+ 8 cos (1.2tAt) + qt-1;
+
(14)
2
1 1 +
20
N( Rt), and At 0.1. We simulate ground truth trajectories and their measurements for 200 time steps and evaluate performance using the root mean square errors (RMSEs) with respect to the simulated true trajectories for every combination of Qi-1 E 3,, 5) and R. {1, 2, 3). The means and 95% confidence intervals of the RMSEs are computed from 100 independent Monte Carlo (MC) simulations. For each optimizer, we report the best hyperparameters found by grid search using 5 separate MC simulations (see Appendix A.4
where Xg .V(0,)) 1), 4:-1 ~ Qr-))
Table shows the results for R = and Q = 1, There exist R.Sprrpp and Adam hy- perparameters that competitive with the unscented Kalman filter (UKF) under low-to-medium process noise and better under high process noise. Both the extended Kalman filter (EKF) and iter- ated extended Kalman filter (IEKF) have the tendency to diverge in this system. The particle filter (PF) performs the best universally, since 1000 particles can capture uni-modal and bi-modal filtering distributions in one dimension quite well. Nonetheless, our Implicit MAP Filter tends produce estimates that match the modes of the particle filter (see Appendix I only struggling on some parts of the sequence with rapid changes in the state. We describe the implementation details for these baseline methods in Appendix A. and show similar results for R, = 1,3 in Appendix A.5.1
The Implicit MAP Filter does not explicitly use Qt-1 or R. but internalizes both of these terms by the choice of optimizer. In fact, our Implicit MAP Filter can always arbitrarily set R. I, reducing the objective to the mean squared error loss (see Appendix A.3 for justification). Compared to methods that similarly use gradient information (EKF, IEKF), our Implicit MAP Filters with Adam and RMSprop see a significant performance boost despite the challenge a non-convex likelihood. We attribute this boost to the of momentum (see Appendix(A.5.1 In contrast to methods that use sigma points (UKF) or particles (PF), we see competitive performance while avoiding covariance calculations or maintaining many copies of the state.

---

Published as a conference paper at ICLR 2024
Table 1: RMSEs the toy nonlinear system (R, 2). Results show the average RMSE over 100
MC simulations with 95% confidence intervals.
Method
RMSE (Q:-1 = 1)
RMSE (Q:-1 = 3)
RMSE (Q = 5)
EKF
24.619 + 3.988
34.909 + 2.732
45.646 + 5.629
IEKF (K 5)
9.277 + 0.652
15.321 0.493
17.898 + 0.559
IMAP (Adadelta)*
32.008 8.650
23.152 + 3.973
26.462 + 5.058
IMAP (Gradient Descent)*
5.589 = 0.219
7.966 = 0.180
10.130 = 0.196
IMAP (Adagrad)*
5.569 = 0.215
6.549 = 0.223
9.264 + 0,227
IMAP (RMSprop)*
5.304 = 0.256
6.000 = 0.227
8.527 + 0.441
IMAP (Adam)
5.699 + 0.190
5.842 + 0.231
7.964 + 0.334
UKF
4.550 + 0.242
5.762 + 0.270
9.555 + 0.400
PF (n 1000)
1.565 + 0.047
2.800 + 0.108
4.518 + 0.160
* "Methods where the reported hyperparameters were found via grid search (see Appendix (..4).
5.2
STOCHASTIC LORENZ ATTRACTOR
In the previous section, show that approach is competitive with particle filter (PF) and the unscented Kalman filter (UKF) in system that is relatively easy for these methods. Now, we eval- uate mildly nonlinear chaotic system that is more complicated because the stochastic differential equation (SDE) is unreliable for accurately predicting the next state (Evensen, 2009).
Consider a stochastic Lorenz '63 system Lorenz 1963
et
2020 Zhao & Särkkä 2021
x1,t
ax2,, T1,t)
$1,t
I2.t =
Z1 - x3,t) - 22,t
dt + adW,
Y: =
22,t
+ Tt:
(15)
T3.t/ F1. T2,1 B3r3,
13,1
where X (No, I3) for μo = (10, 10, 10). T2 ~ ~ 13) 10. 28, 8
3 10,
and W, is a three-dimensional Wiener process with unit spectral density, Similar to Zhao & Särkkä (02TT, we simulate numerical ground truth trajectories and their measurements uniformly for 200 time steps with dt- 0.02 using 10000 Suler-Maruyan ment. We compute RMSES and 95% confidence intervals from 100 independent MC simulations and report the best hyperparameters found by separate grid search (see Appendix A.4
integration steps between each measure-
We report three experimental configurations where we only vary the method used for numerical inte- gration: 4th order Runge- Kutta (RK4) method, Euler's method (Fuler), and no integration, assuming a Gaussian random walk (GRW). These methods are presented in decreasing order accuracy with respect to the aforementioned Euler-Maruyama integration steps. The objective is to assess robust- ness to dynamics misspecification, which is an unavoidable reality wherever numerical integration is required. Unlike the toy nonlinear system, where the true process noise covariance is known exactly, numerical integration introduces additional unmodeled noise to the process. This presents a realistic challenge that significantly distort estimates (Huang al. 2017
Mehra 1972).
Table 2 shows the results of these experiments and the number of values stored to describe the state (N). For fair comparison, we show the EKF and UKF with Q--1 optimized by grid search (see Appendix [I..I) in addition to the EKF and UKF with the Q;-1 specified by the system. Performing such search for the PF is impractical due the cost of numerically integrating each particle. The Implicit MAP Filter with gradient descent and the EKF with optimized Qt-1 show the best performance. Both are relatively robust worsening estimates of the SDE. The EKF with optimized Q. is close to optimal due to the convexity of this system.
The performance of the Implicit MAP Filter and the optimized EKF are statistically equivalent but with several key differences. Our approach only computes point estimate over a minimal state space representation, which decreases space complexity and avoids expensive matrix-matrix computations. Classical filters such as the EKF are difficult to tune (Chen et al. 2018) whereas simple adaptive optimizers, like Adam, are fixed to 3 or 4 hyperparameters regardless of the state space and are faster to evaluate, making search quick and effective. The EKF is not well suited for nonconvexities, as shown in Section 5.1 whereas the Implicit MAP Filter overcomes this pitfall by its choice of optimizer.
7

---

Published as a conference paper at ICLR 2024
Table 2: RMSEs the stochastic Lorenz attractor (a 10). Results show the average RMSE over 100 MC simulations with 95% confidence intervals. RK4 indicates 4th order Runge- Kutta method. Euler indicates Euler's method. GRW indicates Gaussian random walk. N is the number of values stored to describe the state space (D = 3).
Method
RMSE (RK4)
RMSE (Euler)
RMSE (GRW)
N
EKF (original Q(-1)
0.890 + 0.094
2.308 + 0.155
3.057 0.037
D + D
EKF (optimized Q:-1)*
0.692 - 0.014
0.952 + 0.011
1.561 0.010
D + D
IEKF (K 5)
0.890 + 0.094
2.308 + 0.158
3.057 0.038
D + D
IMAP (Adadelta)*
6.034 = 0.634
7.649 = 0.254
10.754
D
IMAP (Adam)*
0.982 + 0.135
1.153 + 0.014
D
IMAP (Adagrad)*
0.907 + 0.144
1.096 + 0.017
1.765
D
IMAP (RMSprop)*
0.114 0,881 + 0.114
1.081 0.015
1.757
D
IMAP (Gradient Descent)*
0.701 + 0.018
0.960 0.012
L.561:0.0
D
UKF (original Q(-1)
2.742 + 0.059
1856=00066
5.628 + 0.067
D + D
UKF (optimized Q:-1)*
1.402 + 0.010
1.417
1.736 0.013
D D
PF (n = 1000)
1.568 + 0.027
1.725 + 0.031
14.346 + 0.365
nD
*Methods where the reported hyperparameters were found via grid search (see Appendix A.4).
Accommodating worsening dynamics estimates is straightforward by either increasing the number of steps or learning rate, each of which corresponds
weaker prior. In Table - our Implicit MAP Filter with gradient descent uses 3 gradient steps with learning rate v 0.05 with RK4, steps with 1) 0.1 for the Euler case, and 10 steps with r) 0.1 for the GRW case. However, the same optimizer settings generally show robustness from RK4 to Euler (see Appendix A.5.2).
When visualizing trajectories (see Appendix (A.5.2). our approach does a remarkable job fitting this system despite every initialization producing vastly different filtering problem. We report additional results for this system using a = 1, 5, 20 Appendix A.5.2
5.3 YEARBOOK
Suppose have a pre-trained neural network that performs inference in time-narying environ- ment. This could be a recommender system that must evolve with changing seasons or a facial recognition system that must be robust temporal shift. Let us assume that data curation is expen- sive, so we are only given small number of examples at every time step for adaptation. We can formalize this the following state space model (Haykin 2004):
W. ((x + 4t-1, Y: h(w;, X.) +
(16)
where w; is the network's ideal weight parameterization, q:-1 is some noise vector that corrupts estimates from the transition function f. y: is the network's 's desired response vector after applying the transformation h parameterized by w, with feature matrix X,, and r, is the measurement noise.
In this final experiment, we design time-varying prediction task described above using the Yearbook dataset (Ginosar et all 2015). which contains 37,921 frontal-facing American high school yearbook photos that capture changes in fashion and population demographics from 1905 -2013. We use the data from 1905-1930 to pre-train a layer convolutional neural network (CNN) to perform binary gender classification task. From 1931-2010, we adapt this weight initialization at each year sequentially using five simple approaches: static weights, direct fit, variational Kalman filtering (VKF), particle filtering (PF), and our Implicit MAP filtering. 32 randomly chosen training examples are available at every time step and a held-out test set of roughly 100 images per time step is used validate the approaches (see Appendix A.5.3 for complete details).
This system is complicated by (1) a state space of 28,193 parameters, (2) unknown weight-space dynamics equations, and (3) unknown process noise and measurement noise covariances. To un- derstand our capacity for modeling such system given VKF, PF, and Implicit MAP Filter, we test hyperparameter configurations of each method (see Appendix A.3.3). We report classification accuracies for all five methods in Table grouped into two 40 year blocks. Static weights yields the worst performance due to the problem being fundamentally sta-stationary. On the other extreme, attempting exactly fit the data at each time step severely overfits, which proves to be better but
8

---

Published as a conference paper at ICLR 2024
Table Classification accuracy with 95% confidence intervals over two 40 year blocks in the year-
book dataset.
Method
% Correct (Validation Years)
% Correct (Test Years)
Static Weights
82.416 + 2.270
60.897 + 1.615
Particle Filter
86.430 + 1.790
66.715 + 2.390
Variational Kalman Filter
93.087 = 1.038
79.967 = 2.204
Direct Fit
94.416 = 0.924
80.449 + 1.845
Implicit MAP Filter
94.973 + 0.837
84.747 = 2.030
still suboptimal. The VKF, PF, and Implicit MAP Filter attempt to internalize the system's 's true state space equations via a small grid search, but this is clearly difficult to do exactly in such state space. Our implicit approach is less sensitive to this inevitable misspecification. The Implicit MAP Filter with 50 steps of Adam optimization not only showed the best performance, but also highly desirable space and time complexity. In cases where we do not start with a pre-trained network, we can simply fit the first timestep to convergence, which is equivalent to uninformative prior the initial state.
6 RELATED WORK
Bayesian filtering, optimization, and neural networks share long history that dates back to early at- tempts of using the extended Kalman filter (EKF) algorithm to train multilayer perceptrons (Singhal & Wu 1988). Since then, Kalman filter theory for the training and use of neural networks saw rich development (Haykin 2004), but fell out of fashion gradient descent proved be the more effi- cient and scalable option for the training of deep neural networks (Bengro et al. (017). Ruck et al. [992]] realized a connection between optimization and Bayesian filtering, showing that the stan- dard backpropagation algorithm can be seen degenerate form of the EKF, Several works have since connected the EKF and the Gauss-Newton method (Bertsekas) 1996 Bell & Cathey/ 1993) and showed that the EKF can be seen
single iteration of Newton's method for a specific quadratic
form (Humpherys et al. 2012). Ollivier (2018) 2019) proved equivalence between online natural
gradient descent and the EKF when the dynamics are stationary the process noise is proportional to the posterior covariance over states. Aitchison (2020) used the Bayesian filtering formalism to derive adaptive optimizers like Adam and RMSprop. De Freitas al. (2000) showed that the process noise and measurement noise can be viewed as adaptive per-parameter learning rates. In the present work, we establish similar connections between optimization and Bayesian filtering but focus on optimization as tool for Bayesian filtering rather than Bayesian filtering tool for optimization.
Several works have similarly used popular optimization methods to address challenges with clas- sical Bayesian filters. Auvinen et al. (210) and [Bardsley et al.] (201)) used limited memory Deaeee---eecchee-GGoaaa---------
(LBFGS) and conjugate gradients, respectively, to derive vari- ational Kalman filters with low storage covariance, and inverse covariance, approximations. These methods scale up classical filters, like low-rank extended Kalman filters Chang et al. (202)). rather than providing general filtering approach that is painless to implement. Chen (2003) describes particle filters that move every particle down their respective gradient prior to sampling from the proposal distribution. This is reminiscent of our approach in that it uses gradients, but differs in that maintains particles, performs sampling, and again falls within the class explicit filters.
7 CONCLUSION
We have shown that Bayesian filtering can be considered as optimization over a time-varying ob- jective and that such perspective opens the door to effective, robust, and scalable filters built from adaptive optimizers. This framework, however, comes at the cost of interoperability and uncertainty estimates, which limits use in risk-sensitive environments situations where the filtering equations themselves are used for analysis. Nonetheless, our proposed Implicit MAP Filtering approach is an attractive option for the performance-drive practitioner.

---

Published as a conference paper at ICLR 2024
REPRODUCIBILITY STATEMENT
All results and figures reported can be reproduced with the code made available at: gilnbb.cm/gganuuaaacooooommmmmpe
ACKNOWLEDGMENTS
We thank R. Thomas McCoy, Jianqiao Zhu, and Logan Nelson and Ryan Adams for helpful dis- cussions. JCS was supported by the Schmidt DataX Fund Princeton University made possible through a major gift from the Schmidt Futures Foundation. GMB, JCS, and TLG were supported by grant N00014-23-1-2510 from the Office of Naval Research.
REFERENCES
Laurence Aitchison. Bayesian filtering unifies adaptive and non-adaptive neural network optimiza- tion methods. Advances in Neural Information Processing Systems, 33:18173-18182 2020.
Harii Auvinen, Johnathan M. Bardsley, Heikki Haario, and T. Kauranne. The variational Kalman filter and an efficient implementation using limited memory BFGS. International Journal for Numerical Methods in Fluids, 64(3):314-335 2010.
Johnathan M. Bardsley, Albert Parker, Antti Solonen, and Marylesa Howard. Krylov space approx- imate Kalman filterin. Numerical Linear Algebra with Applications, 20(2):171-184, 2013.
Bradley M. Bell and Frederick W. Cathey, The iterated Kalman filter update as Gauss-Newton method. IEEE Transactions on Automatic Control, 38(2):294-2971 1993.
Yoshua Bengio, Ian Goodfellow, and Aaron Courville. Deep learning. 2017.
Dimitri P. Bertsekas. Incremental least squares methods and the extended Kalman filter. SIAM Journal on Optimization, 6(3):807-822, 1996.
Peter Chang, Gerardo Durân-Martín, Alexander Y Shestopaloff, Matt Jones, and Kevin Murphy. Low-rank extended Kalman filtering for online learning of neural networks from streaming data. arXiv preprint arXiv:2305 230.::2330599535 2023.
Zhaozhong Chen, Christoffer Heckman, Simon Julier, and Nisar Ahmed. Weak in the NEES?: Auto-tuning Kalman filters with Bayesian optimization. 2018 2 International Conference on Information Fusion (FUSION), pp. 1072-1079, 2018.
Zhe Chen. Bayesian filtering: From Kalman filters to particle filters, and beyond. Statistics, 182(1): 1-69, 2003.
Nando De Freitas, Mahesan Niranjan, and Andrew H. Gee. Hicrarchical Bayesian models for regu- larization in sequential learning. Neural Computation, 12(4):933-953 2000.
R.A. Dorfman. A note the delta-method for finding variance formulae. Biometric Bulletin, 1938.
Arnaud Doucet, Nando De Freitas, and Neil James Gordon. Sequential Monte Carlo methods in practice. Springer, 2001.
John Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12(7), 2011.
David Duvenaud, Dougal Maclaurin, and Ryan Adams. Early stopping as nonparametric variational inference. In Artificial Intelligence and Statistics, pp. 1070-1077, 2016.
Geir Evensen. Data assimilation: the ensemble Kalman filter. Springer, 2009.
Arthur Gclb. Applied optimal estimation. MIT Press, 1974.
Shiry Ginosar, Kate Rakelly, Sarah Sachs, Brian Yin, and Alexei A. Efros. A century of portraits: A visual historical record american high school yearbooks. In Proceedings of the IEEE Interna- fional Conference Computer Vision Workshops, pp. 1-7, 2015.
10

---

Published as a conference paper at ICLR 2024
Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pp. 249-256, 2010.
Gene H. Golub and Charles Van Loan. Matrix Computations. The Johns Hopkins University Press,
Neil J. Gordon, David J. Salmond, and Adrian F.M. Smith. Novel approach to nonlinear/non- Gaussian Bayesian state estimation. In IEEE proceedings (Radar and Signal processing), volume 140, pp. 107-113, 1993.
Simon Haykin. Kalman filtering and neural networks. John Wiley & Sons, 2004.
Yulong Huang, Yonggang Zhang, Zhemin Wu, Ning Li, and Jonathon Chambers. A novel adap- tive Kalman filter with inaccurate process and measurement noise covariance matrices. 1EEE Transactions on Automatic Control, 63(2):594 601, 2017.
Jeffrey Humpherys, Preston Redd, and Jeremy West. A fresh look at the Kalman filter. SIAM Review, 54(4):801-823, 2012.
Simon Julier and Jeffrey K. Uhlmann. Unscented filtering and nonlinear estimation. Proceedings of the IEEE, 92(3):401-445 2004.
Simon J. Julier, Jeffrey K. Uhlmann, and Hugh Durrant-Whig Whyte. A new approach for filtering nonlinear systems. In Proceedings of 1995 American Control Conference-ACC volume 3, pp- 1628-1632, 1995.
Rudolph Emil Kalman. A new approach to linear filtering and prediction problems. Transactions of the ASME-Journal of Basic Engineering, 82(Series D):35-45, 1960.
Mohammad Emtiyaz Khan and Håvard Rue. The Bayesian learning rule. Journal of Machine Learning Research, 1(4):5, 2023.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR), 2015.
Genshiro Kitagawa. Monte Carlo filter and smoother for non-Gaussian nonlinear state space models. Journal of Computational and Graphical Statistics, 5(1):1-25, 1996.
Xuechen Li, Ting-Kan Kam Leonard Wong, Ricky T.Q. Chen, and David Duvenaud. Scalable gradients for stochastic differential equations. In International Conference on Artificial Intelligence and Statistics, pp. 3870-3882, 2020.
Edward N. Lorenz. Deterministic nonperiodic flow. Journal of Atmospheric Sciences, 20(2):130- 141, 1963.
James Martens. New insights and perspectives the natural gradient method. The Journal of Machine Learning Research, 21(1):5776-8811, 2020. 2020.
Raman Mehra. Approaches to adaptive filtering. IEEE Transactions on Automatic Control, 17(5): 693-698, 1972.
M.I. Andrade Netto, I. Gimeno, and M.J. Mendes. A new spline algorithm for non-linear filtering of discrete time systems. IFAC Proceedings Volumes, 11(1):2123-2133 1978.
Yann Ollivier. Online natural gradient as Kalman filter. Electronic Journal of Statistics, 12:2930- 2961, 2018.
Yann Ollivier. The extended Kalman filter is a natural gradient descent in trajectory space. arXiv preprint arXiv:1901.0069) 2019.
Matti Raitoharju and Robert Piché. On computational complexity reduction methods for Kalman filter extensions. IEFE Aerospace and Electronic Systems Magazine, 34(10):2-19, 2019.

---

Published as a conference paper at ICLR 2024
Dennis W. Ruck, Steven K. Rogers, Matthew Kabrisky, Peter S. Maybeek, and Mark E. Oxley- Comparative analysis of backpropagation and the extended Kalman filter for training multilayer perceptrons. IEEE Transactions on Pattern Analysis & Machine Intelligence, 14(06):686-691, 1992.
Reginaldo J. Santos. Equivalence of regularization and truncated iteration for general ill-posed problems. Linear Algebra and its Applications, 236:25-33, 1996.
Simo Särkkä and Lennart Svensson. Bayesian filtering and smoothing, volume 17. Cambridge university press, 2023.
Dan Simon. Optimal state estimation: Kalman, H infinity, and nonlinear approaches. John Wiley & Sons, 2006.
Sharad Singhal and Lance Wu. Training multilayer perceptrons with the extended Kalman algo- rithm. Advances in Neural Information Processing Systems, 1, 1988.
Ilya Sutskever, James Martens, George Dahl, and Geoffrey Hinton. the importance of initial- ization and momentum in deep learning. In International Conference on Machine Learning, pp- 1139-1147, 2013.
Tijmen Tieleman and Geoffrey Hinton. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. 2012.
Jay M. Ver Hoef. Who Invented the Delta Method? The American Statistician, 66(2):124-177, May 2012.
Matthew D. Zeiler. Adadelta: an adaptive learning rate method. arXiv preprint arXiv:1212.5701, 2012.
Zheng Zhao and Simo Särkkä. Non-linear Gaussian smoothing with Taylor moment expansion. IEEE Signal Processing Letters, 29:80-84, 2021.
12


Reviews:
[{"review_id": "2b4m1hyS62", "reviewer": "Reviewer_Y38s", "summary": "This paper proposes to do inference in non-linear, non-Gaussian state space models using an introduced technique termed the \"implicit MAP filter\". The key idea underlying the proposed method is to build on an observation of Santos (1996) that connects regularized least squares solutions to those obtained by running truncated gradient descent. Translated to the state space model inference context, the choice of steps implicitly defines a prior on the transition process, hence truncated optimization can be viewed as doing Bayesian MAP estimation in the state space model. The authors demonstrate their proposed method on two classical examples from the Particle Filter literature and an online inference task utilizing a pretrained network. It is shown that the proposed method is competitive with standard techniques, and works particularly well on the Lorenz-96 model.", "questions": "My questions mainly concern the practical uses and developing a better understanding of the proposed method. \n\nTable 1 states that the MAP methods are not dependent on the process noise covariance Q_t-1, which is true, but they are dependent on the number of steps K, which as stated earlier, in e.g., 3.3, implicitly defines a prior. The authors however state that there is no sensitivity to missspecification (last paragraph of Page 6). This should be clarified. \n\nHow robust is the method to the number of steps K? What if you run the optimizer essentially until convergence every time, truncating many steps ahead? What if you only do one step of the optimizer. Perhaps the method is not too sensitive to this, but this should be verified and made clear and this aspect of the method should be studied further. \n\nHow would the method compare to some of the more modern KF-based methods mentioned earlier, especially on the neural network problem and with some of the datasets that are used there?\n\nIn practical contexts, we often don't have access to a pretrained model. How would the method perform when the model is trained completely from scratch?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The question that the paper attempts to address, namely online learning of state space models, is important in a variety of contexts and the authors offer a concrete pathway to do so, via their proposed optimization framework. The paper provides a potential novel direction that can be followed and certainly developed further, due to, notably, the computational cost savings from looking at optimization as an alternative to (explicit) filtering approaches. The paper is clearly written and easy to read, and the results can be reproduced without much difficulty.", "weaknesses": "The interpretation of the number of steps as a prior (or as a regularizer) is not very intuitive. The authors should provide at least some minimal guidance on tuning the number of optimizer steps K and selecting the optimizer itself. \n\nThe authors should note some of the modern KF based research in this space (Chang et al. 2023 (https://arxiv.org/abs/2305.19535), Titsias et al. 2023 (https://arxiv.org/abs/2306.08448)) that successfully works on larger-scale problems than the ones considered in this paper.\n\nThe experiments chosen are somewhat small-scale given that the authors' argument for not having access to estimates of uncertainty is having a scalable method.\n\nSince the authors explicitly connect the choice of optimizer to defining a prior, they should for example compare different optimizers to one another on the same problem. Have you tried doing this?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lBy0HNXW1z", "reviewer": "Reviewer_zJkz", "summary": "This work proposes methodology for recursively obtaining point estimates of the states in a Bayesian filtering setting. Specifically, the authors first show that the Kalman filter can be equivalently phrased as a recursive truncated gradient-descent. They then explain that certain hyperparameters used by the optimiser (e.g., the number of gradient steps taken at each time step) can be interpreted as implicitly specifying a particular process-noise variance. Finally, the authors propose to linearise the mean of the state-transition equation in order to deal with models in which this mean is a non-linear function of the previous state. \n\nThe authors argue that their methodology is more robust than Kalman- or particle-filtering type methods because it does not need to know true the process-noise variance.", "questions": "Some of the \"true\" model parameters (especially the process and observation noise) in the examples from Sections 5.1 and 5.2 seem to be slightly different than those used in the references cited for these applications. Is there a particular reason for this change? And do the results in these sections differ qualitatively if the parameters from the literature were used? \n\nI am willing to raise my score if the authors can show that the results in Sections 5.1--5.2 are robust to the choice of model parameters.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "**Originality**\nAlthough it relies heavily on previous work by Santos (1996) (as the authors make clear), the proposed approach seems novel.\n\n**Quality**\nI believe that this work is of sufficient quality to be, in principle, publishable in ICLR.\n\n**Clarity**\nOverall, the presentation is clear. I only have some minor concerns detailed below.\n\n**Significance**\nIn higher-dimensional problems, in which Monte Carlo sampling may become prohibitively costly, I agree that optimisation-based approaches (like the present work) can be sensible. And the \"Yearbook\" example seems to support this view. In this regard, I believe that this work makes a worthy contribution.", "weaknesses": "**Choice of hyperparameters**\nI think Table 1 could slightly misleading because the hyperparameters of the proposed methodology are optimised based on a grid search using (other?) data based on (presumably) the same \"true\" process- and observation noise parameters. In other words, optimising these hyperparameters via a grid search is not dissimilar to letting competing methods (e.g., Kalman or particle filtering-type methods) learn the process- and observation noise parameters from other data sets. And if we did the latter, they would not suffer from the misspecification of $Q_{t-1}$ shown in Table 1.\n\n**Clarity**\nIt would be good if the authors could provide a bit more detail and intuition in the second half of Section 3.1 and in Section 3.2, e.g., about\n* the role of the matrices $B_t$ and $C_t$ and why we can be sure that these are well defined.\n* the learning-rate matrix (maybe just state the plain gradient-descent update equation which includes this matrix).\nThere are also a few minor typos, e.g. the notation $\\mathbb{E}[p(x_t|y_{1:t-1})]$ does not make sense (presumably $\\mathbb{E}[x_t|y_{1:t-1}]$ is meant)? Finally, in Section 4, I'm not sure Equation 13 and the discussion about the delta-method is really needed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qYaKKjBEx3", "reviewer": "Reviewer_KcGp", "summary": "The paper explains how sequentially optimising the likelihood of a nonlinear time-series estimation problem with any given optimiser can be interpreted as implicitly (point-)estimating the solution of a specific filtering problem.\nThe difference to existing interpretations of filtering algorithms through the lens of optimisation problems (e.g. the iterated extended Kalman filter) is that the proposed approach only requires access to an optimisation routine, a sequence of likelihood functions, and the mean of a temporal transition model.\n*The Kalman filter's need for prescribing process noise covariance functions of the temporal transition model is replaced by the need for prescribing optimisation algorithms.*\nMost popular optimisation algorithms scale to high-dimensional settings; most Kalman filter variations do not (because they use covariance information).\nThis approach takes inspiration from a result by Santos (1996) that states the equivalence between truncated gradient descent and solutions of least-squares-type problems.\nThe scalability of the resulting algorithm is demonstrated, among other things, by applying the method to a high-dimensional state-space model representing the weight space of a neural network.", "questions": "None", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is well written with a high level of clarity, including the experiment selection, and the algorithm is an enjoyable application of Santos' (1996) result to (nonlinear) filtering problems.\nThe resulting algorithm is so simple that anyone with previous exposure to optimisation or filtering algorithms could implement it.\n\nI appreciate the honest assessment of limitations; for instance, the method does not deliver any uncertainty quantification.\nOverall, I think this paper is in good shape.", "weaknesses": "While I do like the paper, I think that the algorithm/presentation has some (minor) issues:\n\n1. **From linear to nonlinear problems:** For linear problems, Santos (1996) shows that a choice of a (truncated) gradient descent corresponds to an implicit choice of predictive covariances.\nThus, for linear problems, estimating the filtering-mean by applying a suitably truncated version of gradient descent \"makes sense\" -- provided the prior model implicit to the length-scale in gradient descent \"makes sense\". Using the same idea for nonlinear problems, arbitrary optimisers, and possibly optimisers with momentum is, technically speaking, a leap of faith, and there is nothing to support this transition other than \"it seems to work well on the test problems\". Fortunately, it does seem to work well on the test problems. (Unless one chooses to use AdaDelta, that is.)\n\n2. **Sensitivity to misspecification:** I am unsure about how, in some parts of the manuscript, the paper describes the implicit MAP filter to be robust against misspecification of the prior. For example, the last paragraph on page 6 reads:\n\n    \"Since the Implicit MAP Filter does not explicitly use the process noise $Q_t$, there is no sensitivity to misspecification, as indicated by the empty columns in Table 1.\"\n\n    I think this statement might be somewhat misphrased because choosing the \"wrong\" optimiser is not unlike prescribing the wrong process noise covariance. \n    In other words, while the implicit MAP filter is, by construction, not sensitive to _explicit_ misspecification of the prior model, it can suffer from misspecification: according to Santos' (1996) results, choosing the wrong length-scale for truncated gradient descent leads to similarly inaccurate approximations as passing the false process noise to a Kalman filter. If we extrapolate a bit by replacing \"length-scale\" with \"optimiser\" in the sentences above, we could think of choosing the wrong optimiser as equivalent to selecting the wrong prior model -- this would be model misspecification. The perspective of the wrong optimiser not being unrelated to model-misspecification is corroborated by Table 1: In Table 1, AdaDelta performs *much* worse than other optimisers.\n        \n\nIn general, \nI think the first point is a clear disadvantage, but a disadvantage one has to live with (unless I have missed something, in which case I am open to correcting my assessment).\nThe second point is minor because it mainly affects the phrasing in Section 5.1. It should be relatively straightforward to fix.\n\n\nIn general, I think that the strengths outweigh the weaknesses, so I vote for accepting this submission.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)