Computational Doob h-transforms for Online Filtering of Discretely Observed Diffusions
Authors: Nicolas Chopin
Andras Fulop
Jeremy Heng
Alexandre H. Thiery
Venue: Submitted to ICLR 2023 (2023)
Keywords: diffusion
filtering
monte-carlo
particle-filters
BSDE

Abstract: This paper is concerned with online filtering of discretely observed nonlinear diffusion processes. Our approach is based on the fully adapted auxiliary particle filter, which involves Doob's $h$-transforms that are typically intractable. We propose a computational framework to approximate these $h$-transforms by solving the underlying backward Kolmogorov equations using nonlinear Feynman-Kac formulas and neural networks. The methodology allows one to train a locally optimal particle filter prior to the data-assimilation procedure. Numerical experiments illustrate that the proposed approach can be orders of magnitude more efficient than the bootstrap particle filter in the regime of highly informative observations, when the observations are extreme under the model, and if the state dimension is large.

Under review as a conference paper at ICLR 2023
COMPUTATIONAL DOOB'S h-TRANSFORMS FOR ON- LINE FILTERING OF DISCRETELY OBSERVED DIFFU- SIONS
Anonymous authors Paper under double-blind review
ABSTRACT
This paper is concerned with online filtering of discretely observed nonlinear dif- fusion processes. Our approach is based on the fully adapted auxiliary particle filter, which involves Doob's h h-transforms that typically intractable. We pro- pose a computational framework to approximate these h-transforms by solving the underlying backward Kolmogorov equations using nonlinear Feynman-Kan formulas and neural networks. The methodology allows one to train a locally optimal particle filter prior to the data-assimilation procedure. Numerical exper- iments illustrate that the proposed approach can be orders of magnitude more efficient than state-of-the-ar particle filters in the regime of highly informative observations, when the observations are extreme under the model, and if the state dimension is large.
INTRODUCTION
Diffusion processes are fundamental tools in applied mathematics, statistics, and machine learn- ing. Because this flexible class of models is easily amenable to computations and simulations, diffusion processes are very common in biological sciences (e.g. population and multi-species mod- els, stochastic delay population systems), neuroscience (e.g. models for synaptic input, stochastic Hodglyn model, stochastic Fitzhugh-Nauumo model), and finance (e.g. modeling multi assets prices) (Allen) 2010 Shreve al., 2004, Capasso & Capasso 202I). In these disciplines, tracking signal from partial or noisy observations is very common task. However, working with diffusion processes can be challenging as their transition densities are only tractable in rare and simple situations such as (geometric) Brownian motions or -unsteinUUhlenbeck (OU) processes. This difficulty has hindered the use of standard methodologies for inference and data-assimilation of models driven by diffusion processes and various approaches have been developed to circumvent or mitigate some of these issues, as discussed in Section 4
Consider time- hine-homogeneous multivariate diffusion process dX, discretely observed at regular intervals. Noisy observations yı of the latent process X1, are collected at equispaced times 2 T for A 1. We consider the online filtering problem which consists in estimating the conditional laws F (dx) = P(X., E dxlyy...... yk), i.e. the filtering distributions, observations are collected. We focus the use of Particle Filters (PFs) that approximate the fil- tering distributions with a system of weighted particles. Although many previous works have relied on the Bootstrap Particle Filter (BPF), which simulates particles from the diffusion process, it can perform poorly in challenging scenarios as it fails to take the incoming observation yı into account. The goal of this article to show that the (locally) optimal approach given by the Fully Adapted Auxiliary Particle Filter (FA-APF) (Pitt& & Shephard 1999) can be implemented. This necessitates simulating conditioned diffusion process, which can be formulated
((X)) =x(,) (X,) dB, that is
control problem involving an intractable Doob's h-trranffrrm (Rogers & Williams 2000, Chung & Walsh 2006). We propose the Computational Doob's 's (-Trrnsform (CDT) framework for efficiently approximating these quan- tities. The method relies nonlinear Peynman-Kac formulas for solving backward Kolmogorov equations simultaneously for all possible observations. Importantly, this preprocessing step only needs to be performed once before starting the online filtering procedure. Numerical experiments illustrate that the proposed approach can be orders of magnitude more efficient than the BPF in the regime of highly informative observations, when the observations extreme under the model, and

---

Under review as a conference paper at ICLR 2023
if the state dimension is large. A PyTorch implementation to reproduce numerical experiments is available at httpp::///// Tattlp:://nooyniee http///nnnnnnmssscm// 4open. //tennc/////oommooooooaaaat/////
Notations. For two matrices A, B E Id.,, their Frobenius inner product is defined as (A, B)F Ei Bij. The Euclidean inner product for u, v E Rd denoted as (u, v) = Ei= Uy For two (or more) functions and G, we sometimes use the shortened notation [xG to denote the product Fxx)
2 BACKGROUND
2.1 FILTERING OF DISCRETELY OBSERVED DIFFUSIONS
Consider homogeneous diffusion process {X, 1120 in x== Rd with initial distribution pu(dx) and dynamics
dx, = ((X,) dt o (Xt) dB,
(1)
described by the drift and volatility functions : Rd + Rd and o Rd - Rd.d The associated semi-group of transition probabilities p. (d x) satisfies P P(X..s A | x IA (dx |x)
for any 8, & > and measurable C X. The process (B, is a standard Rd-valued Brownian motion. The diffusion process (X, )20 is discretely observed at time is = kT, for & 2 1, for some inter-observation time > 0. The y-valued observation Y E y at time i, is modelled
by the likelihood function g X y R+ in the sense that for any measurable A C V, we have P(Y, E A
dy for some dominating measure dy on V. For a test function yp X' - R. the generator of the diffusion process {X,).20 is given by Lup = (j,, Vip) + 2 (aa" T (. This article is concerned with approximating the filtering distributions TA (dx) = P(X,, dx yI....,y'u Vkk). J For notational convenience, we set (d(x) = (o(x)) since there is no observation collected at the initial time t 0.
X = =x+) = -A
2.2 PARTICLE FILTERING
Particle Filters (PF), also known Sequential Monte Carlo methods, are set of Monte Carlo algorithms that can be used to solve filtering problems (see Chopin et
al. (020) for a recent text-
.x," xx") E X forward in time using combination of propagation and resampling operations. To initialize the PP, each initial particle X E X' for 1 S j < M is sampled independently from the distribution (o(xx) so that Tddxx) M b 8(dx; Approximations of the filtering distribution गą for k 2 built recursively follows. Given the Monte Carlo approximation of the filtering distribu- tion time lx. n (dxx
book the topic). PFs evolve set of M particles x! (x] ...,,
M
M M- 5(dx; x1,x the particles x!:M are propagated independently 1dt x using a Markov kernel 1 (d& x) specified by
forward in time by x/ the user. The BPF corresponds to the Markov kernel BPF (dx x) P E dx X,, x), while the FA-APF (Pitt Shephard 1999) corresponds to the (typically intractable) kernel FA-APF (dx x) = IP E dx Xx,, = X, 1++1 = =x+1). Each particle & is associated k+1 = W Eie W where the unnormalized weights W (by timehhomgennetty of (I) are defined
4k+11
with a normalized weight W
x(()
W K+1
(2)
ge+1 (dxc)e))
The BPF and FA-APF correspond respectively having
Wr..PF k+1 (k+1) and
W.J.FAAPP k+1
Elgg Nk+1) X1. x
(3)
The weights are such that T (dx) RS E/11 W k+1 8(dx; x x The resampling step consists in defining a new set of particles X with (xx = R) W This resampling scheme ensures that the equally weighted set of particles x x provides Monte Carlo approximation of the
1:M
2

---

Under review as a conference paper at ICLR 2023
filtering distribution at time t++1 in the sense that (x+1(dx)) = M EE== 6(dx; x Note that the particles 1:MM do not need to be resampled independently given the set of propagated particles RIMM We refer the reader to Gerber et al. (2019) for recent discussion of resampling schemes within PFs and to Del Moral 2004) for book-length treatment of the convergence properties of this class of Monte Carlo methods.
In most settings, the FA-APF (Pitt & Shephard 1999) that minimizes a local variance criterion (Doucet et al.] 2009) generates particles that are more consistent with informative data and weights that exhibit significantly less variability compared to the BPF. This gain in efficiency can be very substantial when the signal-to-noise ratio is high or when observations contain outliers under the model specification. Nevertheless, implementing FA-APF requires sampling from the transition probability 4k++
FA-AFF (dx x), which is typically not feasible in practice. We will show in the following that this can be achieved in setting by simulating conditioned diffusion.
2.3 CONDITIONED AND CONTROLLED DIFFUSIONS
As the diffusion process 1 is assumed to be time-homogeneous, it suffices to focus on the initial interval [0, I'] and study the dynamics of the diffusion X tejor)) conditioned upon the first observation YT y. It is standard result that the conditioned diffusion is described by diffusion process with the same volatility as the original diffusion but with a time-dependent drift
function that takes the future observation Yr- = y into account.
Before deriving the exact form of the conditioned diffusion, the notion of controlled diffusion needs to be discussed. For an arbitrary control function c X' x y x [0, + Rd and E y, consider the LG((x) + ((oc) (x. yx (x(())
controlled diffusion (X; )e(o, with generator Leytt p(x) t and dynamics
xx. (XX; dt + (X. dB, + [x t)
(4)
(original dynamies)
(control term)
Pl..11 and [0.7] denote the probability measures on the space of continuous functions C([0, 2T, Rd) generated by the original and controlled diffusions, Girsanov's theorem shows that
dP O.T (X (o.T) = exp { 1 2 L [cc(X,, dt 1-' yc. dB.)
(5)
d [0.7]
We now describe the optimal control function c, X' x [0, Rd that is such that, for any observation value }r, the controlled diffusion X x [.T has the same dynamics as the original
diffusion (o,7] conditioned upon the observation Y y. For this purpose. consider the function
1) = E[g(XT, y) X; = x] = / g(xT) y) pT p x)
that gives the probability of observing Y = when the diffusion has state x E X at time / E [o, [). Recall that the likelihood function g X'x x² 12, was defined in Section 2.1 Equation (6) implies that h X' x y x [0, + 14 satisties the backward Kolmogorov equation Oksendal 2013).
(0, + L)h 0 0,
with terminal condition h(x,y, T) = g(x,y) y) for all (x,y) X V. As described in Appendix the theory of Doob's h-transformed shows that the optimal control is given by
y,, Jo V log hl(x,y,t).
(8)
We refer readers to Rogers & Williams 2000) for formal treatment of Doob's 's h-transform.
METHOD
3.1 NONLINEAR FEYNMAN-KAN FORMULA
Obtaining the control function c, (x, y. = [o TV log y, by solving the backward Kol- mogorov equation in 7 for each observation y is computationally not feasible when filtering
3

---

Under review as a conference paper at ICLR 2023
many observations. Furthermore, when the dimensionality of the state-space X' becomes larger, standard numerical methods for solving Partial Differential Equations (PDEs) such as Finite Differ- ences or the Finite Element Method become impractical. For these reasons, we propose instead to approximate the control function c, with neural networks, and employ methods based on automatic differentiation and the nonlinear Feynman- Kac approach to solve semilinear PDEs (Hartmann al.
2017
2019; Kebiri et al.,
2017
E ct al. 20177 Chan- Wai-Nam et al. [019 Hutzenthaler & Kruse
2020
Hutzenthaler et al.
2020
Beck al. 2019 Han et al. 2018 Nüsken Richter 2021).
As the non-negative function h typically decays exponentially for large ||x|]. it is computation- ally more stable work on the logarithmic scale and approximate the value function y. [og[h(x, y, t)]. Using the fact that á satisfies the PDE (7), the value function satisfies
1 |1 T Vull?
(0, + C)u
v(x.y, T) = (oglg(x y)] for all (x.y) E X x
(9)
2
Let (X Jt be a controlled diffusion defined in Equation (4) for given control function c: X' x x x [0, - IRd and define the diffusion process (Vi)e as V = v(x;"), .y,t). While any control function c(x. y, t) with mild growth and regularity assumptions can be considered within our framework, we will see that iterative schemes that choose it as current approximation of c. (x. t)
tend to perform better in practice. Since we have that 0, + Lv + (oc, Vu) = (1/2) ||o Vul (c, Vu), Itō's Lemma shows that for any observation Yr y and 0
T, we have
VT = V2 + /." (1
| + (c, Zi) dt + [. (Ze, dB,)
with Z, = [o
.y.t) ,y. and VT = - loglo(X,4)) c.Y y)]. For notational simplicity, we sup-
pressed the dependence of (Vi, Z,) on the control and observation y. In summary, the pair of processes (V, Ze) defined as V2 = v(X," .y,t) and Z, -oo Vx](X ytt) are such that the following equation holds,
- -Xoglo(X7) y)] = V + / 12 + (c. Z,) dt + [. (Z,, dB.).
(10)
Crucially, under mild growth and regularity assumptions on the drift and volatility function μ : X - IRd and X - Rd.d, the pair of processes (V, Ze) is the unique solution to Equation (TO)
(Pardoux & Peng 1990: 1992; Pardoux & Tang 1999 Yong & Zhou 1999). This result can be used
as building block for designing Monte Carlo approximations of the solution to semilinear and fully
nonlinear PDEs (E et al. 2017 Han et al. 2018
Raissi 2018 Beck et al. 2019 Huré et al. 2020
Pham et al. 2)
3.2 COMPUTATIONAL DOoB'S 's h-TRANSFORM
As before, consider a diffusion { Jeejo,r controlled by a function C: X x x [0, 2] + gd and driven by the standard Brownian motion [Br) Furthermore, for two functions No : X' y + R and N X x y x [0, T) - Rdd, consider the diffusion process (Vi)ee(o.r) defined as
V = Vo + / {2 Zz 2 (c(x.) yy, Za) dt + J. (Z,, dB.),
(11)
where the initial condition Vo and the process (Zcheeo.ry are defined as
Vi = (x y) and Z, = - ttt).
(12)
Importantly, we remind the reader that the two diffusion processes x and V4 are driven by the same Brownian motion B, The uniqueness result mentioned the end of Section 3.1 implies that, if for any choice of initial condition x E X and terminal observation y E } the condition V (oglg(Xee e.y .y)] is satisfied, then we have that for all yx,y,) t) E X x y x [0,
N) = - log (xx, 0) and y,) = -Io V log t)) In particular, the optimal control is given by (x,,, -N(x, y, (). These remarks suggest parametrizing the functions No(-,-) and ((....) by two neural networks with respective parameters to O and e θ while minimizing the loss function
(13)
C(0,, 0;c) E. (Vr loglo(X9)) Y)) 27
(14)
4

---

Under review as a conference paper at ICLR 2023
The above expectation is with respect to the Brownian motion {Be the initial condition X 7x (dx) of the controlled diffusion, and the observation Y ~ 17x (dy) at time T. In (14). we fix the dynamics of X and optimize over the dynamics of V₁. The spread of the distributions 7xx and "Y should be large enough to cover typical states under the filtering distributions ug, k 2 and future observations to be filtered respectively. Specific choices will be detailed for each application in Section 5) For offline problems, one could learn in a data-driven manner by selecting TY as the empirical distribution of actual observations. We stress that these choices only impact training of the neural networks, and will not affect the asymptotic guarantees of our filtering approximations.
CDT algorithm. The following outlines our training procedure learn neural networks No and N that satisfy (3). To minimize the loss function (14), any stochastic gradient algorithm can be used with a user-specified mini-batch size of J 2 The following steps are iterated until convergence.
1. Choose control c: X x y x [0, T] - Rd, possibly based on the current neural network parameters (fc, 0) On O.
2. Simulate independent Brownian paths B/ (0,7). initial conditions X ~ nix (dx), and obser- vations Y 1/y (dy) for 1 S j.j... J.
3. Generate the controlled trajectories: the j-th sample path X [0.7] is obtained by forward integration of the controlled dynamics Equation c(., Y),.), .), and the Brownian path B (o.T)-
(4) with initial condition X control
4. Generate the value trajectories: the j-th sample path V is obtained by forward integra-
tion of the dynamics in Equation 11 12 with the Brownian path B/ (0.2) and the current
neural network parameters (0o, 0) Qg θ
5. Construct a Monte Carlo estimate of the loss function (1)
== (Vz + IXg Y
(15)
6. Use automatic differentiation to compute de, C and Qg and update the parameters (fu. 0)-
Importantly, if the control function e in Step. does depend on the current parameters (0,, 0), the gradient operations executed in t not be propagated through the control function A standard st stop-gradient operation available in most popular automatic differentiation frame- works can be used for this purpose.
dime-diseretizzatinn of diffusions. For clarity of exposition, we have described our algorithm in continuous-tims time. In practice, one would have to discretize these diffusion processes, which is entirely straightforward. Although any numerical integrator could potentially be considered, the ex- periments in Section 3 the standard Euler-Manuyama scheme (Kloeden & Platen 1992..
Parametrizations of functions No and N. In all numerical experiments presented in Section 5] the functions No and N are parametrized with fully-connected neural networks with two hid- den layers, number of neurons that grow linearly with dimension d, and the Leaky ReLU ac- tivation function except in the last layer. Future work could explore other neural network ar- chitectures for our setting. In situations that are close to Gaussian setting (e.g. Ornstein- Uhlenbeck process observed with additive Gaussian noise) where the value function has the form v(x, y,) = (x, a(y, !)x) + (b(y, ()) x) + c(y, (), a more parsimonious parametrization could cer- tainly be exploited. Furthermore, the function N (x, y, t) could be parametrized to automatically satisfy the terminal condition N(x,y, = -(o V log (x. y). A possible approach consists in setting y,x. (1 - t - (t/T)[o V log g1( y) for some neural network N x y x [0, - Rdd These strategies have not be used in the experiments of Section
Choice of controlled dynamics. In challenging scenarios where observations are highly informa- tive and/or extreme under the model, choosing a good control function to implement Step [ of the proposed algorithm be crucial. We focus two possible implementations:
5

---

Under review as a conference paper at ICLR 2023
CDT static scheme: simple (and naive) choice is not using any control, i.e. c(x, y.t) E Rd for all (x. y.t) E X' x y x [O,T].
CDT iterative scheme: use the current approximation of the optimal control c, described
by the parameters (ov, 0) E O x O. This corresponds to setting y, -x(x,y,,)
While using static control approach can perform reasonably well in some situations, our results in Section [ suggest that the iterative control procedure is a more reliable strategy. This is consistent with findings in the stochastic optimal control literature (Thijssen & Kappen, 2015; Pereira al.) 2019). This choice of control function drives the forward process X, to regions of the state- space where the likelihood function is large and helps mitigate convergence and stability issues. Furthermore, Section D reports that (at convergence), the solutions No and N can be significantly different. The iterative control procedure leads to more accurate solutions and, ultimately, better performance when used for online filtering.
3.3 ONLINE FILTERING
Before performing online filtering, we first run the CDT algorithm described in Section 3.2 to con- struct an approximation of the optimal control c, (x.
lo V log y,, For concrete- ness, denote by ट X x y x [0, + IR the resulting approximate control, i.c. č(x, = -N(x,y, where N ..))..) is parametrized by the final parameter O E O. Similarly, denote by V. : X y + I the approximation of the initial value function v(x, y, 0) () = yx) where No is parametrized by the final parameter 8g E Op.
log h(x, y. 0), i..
For implementing online filterigg with M 2 particles, consider current approximation T. (dx) x of the filtering distribution time t 2 0. Given the future observation YR+1 = Yk+1. the particles x are then propagated forward by exploiting the approximately optimal control tx, + c(x, tę). In particular, X is obtained by setting X
M- Ed 8(dx;
X where {X² JE follows the controlled diffusion
11
cx p (x)) + a (x)) AB:
yk+1, (x)
(16)
(original dynamics)
(approximatelly optimal control)
initialized at X x Each propagated particle & is associated with normalized
weight W / W where W = (dP (eete / (x++)))) (X) (.))).)) x
((%),,)+++) We recall that the probability measures F(ta,fu41) and ya [(a.fk+a] original and controlled diffusions on the interval [tx, tx+1). Girsanov's theorem, as described in Equation (5) implies that
correspond to the
W exp {
1
/ (Z) dB.) + log log (l Mer1)
| dt +
2
where Z2_ -(X NA+1, 1x). Similarly to Equation (1). consider the diffusion process (V2) he(eun 1] defined by the dynamics dV? - 2 1 1|2 dt + (B/) with initialization at v2g = Vo(x),, 1k-1). Therefore the weight can be re-written as
W = exp { ViL.. + log gogg(() )
M8+1)} } exp (-Vo(x), v
(17)
and computed by numerically integrating the process { (V.' Juelte.te=1l- Given the definition of the loss function in (14). we can expect the term within the first exponential to be close to zero. In the ideal case where y. = c, y, and Vx( = - log h(x, one recovers the exact AF-APF weights in 3 Once the unnormalized weights (17) are computed, the resampling steps are identical to those described in Section 2.2 for a standard PF. For practical implementations, all the processes involved in the proposed methodology can be straightforwardly time-discretized. To distinguish between CDT learning with static iterative control, we shall refer to the resulting approximation of FA-APF as Atatic-APF and Iterative-ALe respectively. We note that these APFs do not involve modified resampling probabilities as described e.g. in Chopin et al. 20200 p. 145).

---

Under review as a conference paper at ICLR 2023
4 RELATED WORK
This section positions our work within the existing literature.
MCMC methods: Several works have developed MCMC methods for smoothing and parameter estimation of SDEs; for example, Roberts & Stramer (2001) proposes treat paths between ob- servations as missing data. Our work concentrates on the online filtering problem: this cannot be tackled with MCMC methods.
Several methods have been proposed to reduce or eliminate the bias due to discretization (Beskos et al. 2006a b Fearnhead al. 2010 2008); these methods typically rely on the Lamperti transform that is only rarely available in multivariate settings. Furthermore, when filtering diffusion with highly-iforrmative observations, the discretization bias is often orders of magnitude smaller than other sources of errors. We also stress that our method is generic: it does not exploit any specific structure of the diffusion process being assimilated.
Exact Simulation:
Gaussian Assumptions: In the data-assimilation literature, methods based on variations of the En- semble Kalman Filter (EnKF) Evensen (2003) have been successfully deployed in applied scenarios and very high-dimensional settings. These methods do rely strong Gaussian assumptions and are inappropriate for highly nonlinear and non- Gausssan models. In contrast, our method is asymptoti- cally exact in the limit when the number of particles M + oo (up to discretization error). Indeed, we do not expect our method to be competitive with this class of (approximate) methods in very high-dimensional settings that common in numerical weather forecasting. These methods typ- ically achieve lower variance by increasing the bias. Our method is designed to filter diffusion processes in low or moderate dimensional settings. It likely that scaling our method truly high- dimensional settings with effective dimension D 10² would require introducing model-specific approximations (e.g. localization strategies).
Steering particles towards observations: particle methods pioneered by Van Leeuwen (2010) are based on this natural principle in order to mitigate collapse of PFs in high-dimensional settings. These methods typically rely on some model structure (e.g. linear Gaussian observation process) and have number of tuning parameters. They can understood as parametrizing a linear control, which is only expected to work well for linear Gaussian dynamics, admittedly very important in applications such as geoscience.
the method of Chorin et al. (2010) attempts to transform standard i.i.d Gaussian samples into samples from the optimal proposal density. To implement this methodology requires number of assumptions and requires solving non-convex optimization step for each particle and each time step. This can quickly become computational burdensome.
Implicit Particle Filter:
Guided Intermediate Resampling Filters (GIRF): the method of Del Moral & Murray (015)):
Park & Tonides (2020) propagates particles at intermediate time intervals between observations with the original dynamics and triggers resampling steps based a guiding functions that forecast the likelihood of future observations. The choice of guiding functions is crucial for good algorithmic performance. We note that GIRF in fact intimately related Doob's h-trrnsformm as the optimal choice of guiding functions is given by (6) (Park & Ionides 2020). However, even under this optimal choice, the resulting GIRF is still sub-optimal when compared an APF that moves particles using the optimal control induced by Doob's s h-traasform, i.e. it is better to move particles well rather than rely on weighting and resampling. The latter behaviour supported by our numerical experiments. Appendix A.5 details our GIRF implementation and the connection to Doob's ' h-transform.
5 EXPERIMENTS
We performed numerical experiments on three different models: an Ornstein-Uhlenber model, (nonlinear) Logistic diffusion model and a (nonlinear) diffusion model describing cell differentia- tion. This section presents experiments on the Unslein- model; the two other studies can be found in the Appendix A.3 and A.4 All experiments employed 2000 iterations of the Adam optimizer with learning rate of 0.01 and mini-batch size of B 1000 sample paths with 10 different observations each. Appendix A.6 describes how the CDT algorithm and the approximate control functions behave during training. Training times took less than minutes standard

---

Under review as a conference paper at ICLR 2023
0.125
0.25
0.5
Qy 1.0
100
100
100
100
Variance ELBO gap ESS%
60-
25
25
10
10'
107
10
17
10
100
10'
1.5
0.75
1500
0.50
8000
10-
0.5
0.25
500
0.0
30*20'
1.0
367
10l
107
10
10x
167
109
101-
104-
10a
103.
109
101
10
10-3
10-3
10
K
K
K
Standard deviation
BPF
kerative-APF
Exact-APF
Figure 1: Results for Monstinn-Uhlenbeck model with based on 100 independent repetitions of each PF. The ELBO gap in the second row relative to FA-APE.
4
32
Varlance ELBO gap E5S%
50
50
50
50
50
25
25
25
25
25
100
150
0.50
0.75
1.0
1000
0.5
50
500
aor 100
10
107
1o 107
107
10r
10r201
107
10r
20'
10p.
107
1.0
10F.
00
100
106
13.0
20
20-2
10P
10P
100
1.0
103
10a
10P
10a
10P
10p
K
K
BPF
Iterative-APF
Exact-APF
FA-APF
GIRF
Figure 2: Results for Ornstein -hattinnUhhlebeckk model with oY 1.0 based on 100 independent repeti- tions of each PF. The ELBO gap in the second row relative to FA-APE.
8

---

Under review as a conference paper at ICLR 2023
CPU: it is negligible when compared to the cost of running filters with many particles and/or to assimilate large number of observations. The inter-observation time was T === 1 and we employed the Euler-Maruyama integrator with stepsize of 0.02 for all examples. Our results are not sensi- tive to the choice of T and discretization stepsize as long as it is sufficiently small. We report the Effective Sample Size (ESS) averaged over observation times and independent repetitions, the evi- where
dence lower bound (ELBO) le[log Ply yx)], v. and the variance Var[log P(y Plyl.... YK) denotes its unbiased estimator of the marginal likelihood of the time-discretized filter P(yI...... yK). V.)). When testing particle filters with varying number of observations K, we increased the number of particles M linearly with K keep marginal likelihood estimators stable (Bérard et al. 2014). For non-toy models, our GIRF implementation relies on a sub-optimal but practical choice of guiding functions that gradually introduce information from the future observation by annealing the observation density using linear (Linear-GIRFF) or quadratic schedule (Quadratic- GIRF).
5.1 ORNSTEIN-UHLENER MODEL.
Consider a d-dimensional and the Gaussian observation model (x. = N(y;x,, We chose 7X =N(0 La/2) the stationary distribution and f/y = -O, (1/2+oł)LJ) as the implied distribution of the observation when training neural networks with the CDT iterative scheme. We took different values of dy E (0.125, 0.25, 0.5, 1.0} to vary the informativeness of observations and E {1, 2, 4, 8, 16, 32} to illustrate the impact of dimension. Analytical tractability in this example (Appendix (A2)) consider three idealized particle filters, namely an APF with exact networks (Exact-APF), FP-APF, and GIRF with optimal guiding functions (Appendix A.5). Comparing our proposed Iterative-APF to Exact- APF and FA-APF enables us to distinguish between neural network approximation errors and time- discretization errors. We note that all PFs except the FA-APF involve time-discretizatinn
process given by [ with μ(x) = -x, o(x) = Ia
Columns to 4 of Figure I summarize our numerical findings when filtering simulated observations from the model with varying oy and fixed 1. We see that the performance of BPF deteriorates as the observations become more informative, which is to be expected. Furthermore, when dy is small, the impact of our neural network approximation and time-discretization becomes more noticeable. For the values of ar and the number of observations K considered, Iterative-APF had substantial gains in efficiency over BPF and typically outperformed GIRF. From Column 5, we note that these gains over BPF become very large when we filter K 100 observations simulated with observation standard deviations that are multiples of σY = 0.25 which was used to run the filters. In particular, while the ELBO of BPF diverges as we increase the degree of noise in the simulated observations, the ELBO of Iterativ-APPF and GIRF remain stable
Figure shows the impact of increasing dimension dl with fixed ov = 1.0 when filtering simulated observations from the model. Due to the curse of dimensionality (Snyder et al., 2008 2015). it is not surprising for the performance of all PFs degrade with dimension. Although the error of our neural network approximation becomes more pronounced when d is large, the gain in efficiency of Iterative-APF relative BPF very significant in the higher dimensional regime, and particularly so when the number of observations K is also large. Iterative-APE also outperformed GIRF in most settings, with comparable performance when d is large.
DISCUSSION
This paper introduced the CDT algorithm, a Sequential Monte-Carlo method for online filtering of diffusion processes evolving in state-spaces of low to moderate dimensions. Contrarily to a number of existing methods, the CDT approach is general and does not exploit any particular structure of the diffusion process. Furthermore, numerical simulations suggests that the CDT algorithm is especially worthwhile when compared to competing approaches (e.g. BPF or GIRF) in higher dimensional settings or when the observations are highly informative. Ongoing work involves extending the CDT framework to parameter estimation and experimenting with alternative formulations and/or parameterizations accelerate the training procedures.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Linda JS Allen. An introduction to stochastic processes with applications to biology. CRC press, 2010.
Christian Beck, Weinan E, and Arnulf Jentzen. Machine learning approximation algorithms for high- dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations. Journal of Nonlinear Science, 29(4):1563-1619 2019.
Jean Bérard, Pierre Del Moral, and Arnaud Doucet. A lognormal central limit theorem for particle approximations of normalizing constants. Electronic Journal of Probability; 19:1-28, 2014.
Alexandros Beskos, Omiros Papaspiliopoulos, and Gareth O Roberts. Retrospective exact simula- tion of diffusion sample paths with applications. Bernoulli, 12(6):1077-1098, 2006a.
Alexandros Beskos, Omiros Papaspiliopoulos, Gareth O Roberts, and Paul Fearnhead. Exact and computationally efficient likelihood based estimation for discretely observed diffusion processes (with discussion). Journal of the Royal Statistical Society: Series (Statistical Methodology), 68 (3):333332,, 2006b.
Vincenzo Capasso and Vincenzo Capasso. Introduction Continuous-Time Stochastic Processes. Springer, 2021.
Quentin Chan-Wai-Namm Joseph Mikael, and Xavier Warin. Machine learning for semi linear PDBs. Journal of Scientific Computing, 79(3): 2019.
Nicolas Chopin, Omiros Papaspiliopoulos, et al. An introduction to sequential Monte Carlo.
Springer, 2020.
Alexandre Chorin, Matthias Morzfeld, and Xuemin Tu. Implicit particle filters for data assimilation. Communications in Applied Mathematics and Computational Science, 5(2):221-240, 2010.
Kai Lai Chung and John B Walsh. Markov processes, Brownian motion, and time symmetry, volume 249. Springer Science & Business Media, 2006.
Paolo Dai Pra. A stochastic control approach to reciprocal diffusion processes. Applied mathematics and Optimization, 23(1):313-32 1991.
Pierre Del Moral. Feynmun-Kat formulae: genealogical and interacting particle systems with ap-
plications, volume 88. Springer, 2004.
Pierre Del Moral and Lawrence Murray. Sequential Monte Carlo with highly informative obser- vations. SIAM/ASA Journal on Uncertainty Quantification, 3(1):969-997, 2015.
Brian Dennis and Robert F Costantino. Analysis of steady-state populations with the gamma abun- dance model: application to Tribolium. Ecology, 69(4): (9(4):1200-1213 200-1213, 1988.
Arnaud Doucet. Adam M Johansen, et al. A tutorial on particle filtering and smoothing: Fifteen years later. Handbook of nonlinear filtering, 12(656-704):3, 2009.
Weinan F, Jiequn Han, and Arnulf Jentzen. Deep learning-based numerical methods for high- dimensional parabolic partial differential equations and backward stochastic differential equa- tions. Communications in Mathematics and Statistics, 5(4):349-380 2017.
Geir Evensen. The ensemble kalman filter: Theoretical formulation and practical implementation.
Ocean dynamics, 53(4):343-367, 2003.
Paul Fearnhead, Omiros Papaspiliopoulos, and Gareth O Roberts. Particle filters for partially ob- served diffusions. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 70 (4):755-777, 2008
Paul Fearnhead, Omiros Papaspiliopoulls, Gareth O Roberts, and Andrew Stuart. Random Random-weight particle filtering of continuous time processes. Journal of the Royal Statistical Society: Series (Statistical Methodology), 4244)-497-512, 2010.
10

---

Under review as a conference paper at ICLR 2023
Mathieu Gerber, Nicolas Chopin, and Nick Whiteley. Negative association, ordering and conver- gence of resampling methods. The Annals of Statistics, (7744)2236622606 2019.
Jiequn Han, Arnulf Jentzen, and Weinan E. Solving high-dimensionall partial differential equations using deep learning. Proceedings of the National Academy of Sciences, 115(34):8505-852) -510, 2018.
Carsten Hartmann, Lorenz Richter, Christof Schütte, and Wei Zhang. Variational characterization of free energy: Theory and algorithms. Entropy, 19(11):626, 2017.
Carsten Hartmann, Omar Kebiri, Lara Neureither, and Lorenz Richter. Variational approach to rare event simulation using least-squares regression. Chaos: An Interdisciplinary Journal of Nonlinear Science, 29(6):063107, 2019.
Côme Huré, Huyên Pham, and Xavier Warin. Deep backward schemes for high- -igeessiian non- linear PDEs. Mathematics of Computation, 89(324):1547-1557, 2020.
Martin Hutzenthaler and Thomas Kruse. Multilevel Picard approximations of high-dimensional semilinear parabolic differential equations with gradient-dependent nonlinearities. SIAM Journal on Numerical Analysis, 58(2):929-996 961, 2020.
Martin Hutzenthaler, Arnulf Jentzen, Thomas Kruse, Tuan Anh Nguyen, and Philippe von Wurstem- berger. Overcoming the curse of dimensionality in the numerical approximation of semilinear parabolic partial differential equations. Proceedings of the Royal Society A, 476(2244)20010068) 2020.
Omar Kebiri, Lara Neureither, and Carsten Hartmann. Adaptive importance sampling with forward- backward stochastic differential equations. In International workshop on Stochastic Dynamics out of Equilibrium, pp. 265-281. Springer, 2017.
Peter E Kloeden and Eckhard Platen. Stochastic differential equations. In Numerical Solution of Stochastic Differential Equations, pp. 103-160. Springer, 1992.
Jonas Knape and Perry De Valpine. Fitting complex population models by combining particle filters with Markov chain Monte Carlo. Ecology, 93(2):256-263. 2012.
Nikolas Nüsken and Lorenz Richter. Solving high-dimensional Maanitton-Jcohh-Beellle PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path space. Partial Differential Equations and Applications. 2(4):1-48, 2021.
Bernt Oksendal. Stochastic differential equations: an introduction with applications. Springer Science & Business Media, 2013.
Etienne Pardoux and Shige Peng. Adapted solution of a backward stochastic differential equation. Systems & Control Letlers, 14(1):55-61, 1990.
Etienne Pardoux and Shige Peng. Backward stochastic differential equations and quasilinear parabolic partial differential equations. In Stochastic partial differential equations and their ap- plications, pp. 200-217. Springer, 1992.
Etienne Pardoux and Shanjian Tang. linear parabolic PDEs. Probability Theory and Related Fields, 1 114(2):13--150) 1999.
stochastic differential equations and quasi-
Joonha Park and Edward Ionides. Inference on high- sigh-dimensional implicit dynamic models using guided intermediate resampling filter. Statistics and Computing, 30(5):1477-152, 2020.
Mareus Pereira, Ziyi Wang, Ioannis Exarchos, and Evangelos A Theodorou. Learning deep stochas- tic optimal control policies using forward- backward SDEs. arXiv preprint arXiv: 2019.
Huyen Pham, Xavier Warin, and Maximilien Germain. Neural networks-based backward scheme for fully nonlinear PDEs. SN Partial Differential Equations and Applications, 2(1):1-24, 2021.
Michael K Pitt and Neil Shephard. Filtering via simulation: Auxiliary particle filters. Journal of the American Statistical Association, 94(446):590-59) 1999.

---

Under review as a conference paper at ICLR 2023
Maziar Raissi.
stochastic neural networks: Deep learning of high- nigh-dimensional partial differential equations. arXiv preprint arXiv:1804.07010, 2018.
Gareth O Roberts and Osnat Stramer. On inference for partially observed nonlinear diffusion models using the Metropolis-Hastinng algorithm. Biometrika, 88(3):603-621, 2001.
I. Chris G Rogers and David Williams. Diffussios, Markov processes and Martingales: Volume 2: Itó Calculus, volume 2. Cambridge university press, 2000.
Steven Shreve et al. Stochastic calculus for finance II: Continuous-time models, volume 11. Springer, 2004.
Chris Snyder, Thomas Bengtsson, Peter Bickel, and Jeff Anderson. Obstacles to high-dimensional particle filtering. Monthly Weather Review, 136(12):4629 06490,464000 2008
Chris Snyder, Thomas Bengtsson, and Mathias Morzfeld. Performance bounds for particle filters using the optimal proposal. Monthly Weather Review, 143(11):4750-4761, 2015.
Sep Thijssen and HJ Kappen. Path integral control and state-dependent feedback. Physical Review E, 91(3))032204, 2015.
Peter Jan Van Leeuwen. Nonlinear data assimilation in geosciences: extremely efficient particle filter. Quarterly Journal of the Royal Meteorological Society, 1366653):1991-1999 2010.
Jin Wang, Kun Zhang, Li Xu, and Erkang Wang. Quantifying the Waddington landscape and biolog- ical paths for development and differentiation. Proceedings of the National Academy of Sciences, 108(20):82778222.. 2011.
Jiongmin Yong and Xun Yu Zhou. Stochastic controls: Hamiltonian systems and HJB equations, volume 43. Springer Science & Business Media, 1999.
A APPENDIX
A.1 DOOB'S h-TRAASFORRM
This section gives an heuristic derivation of the Equation
8 that describes the optimal control. To simplify notation, we shall denote the conditioned process X (o.T] (Yr y) as [(o.))- Recall the function
y Elg(( Kr.yy X. =x] =x]== / y)xx, PT- x)
(18)
which gives the probability of observing Yr = y when the diffusion process has state x E X' at time t T). The definition in (6) implies that the function h X' x x x [0, + R4 satisfies the backward Kolmogorov equation [Kssndal] 2013).
(0, + L)h = 0, with terminal condition 1((x 7) g(x,y) for all (x,y) x V. For 4p X - R and an infinitesimal increment 8 0, we have
(19)
El(((1+6)))) x] Elsp(X++0) g( (XT. y) X, xx=x]/ /l//Eg(XT,, X, x]
= El(p(X++6)) 1(X y, + 8) X, =x] h(x,y,t)
(20)
Clip h)
((x) + 8
(x, t,t 0(0²)
h
Furthermore, since the function h satisfies (oo V h, Vip). Taking 6 + 0, this heuristic derivation shows that the generator of the condi- tioned diffusion equals Lip+ V log h, Vip). Hence X (ooT) satisfies the dynamics of a controlled diffusion with control function
some algebra shows that Llp h1/h Cup +
yx = o T V log h (x,
(21)
This proves Equation
g
12

---

Under review as a conference paper at ICLR 2023
A.2 ANALYTICAL TRACTABILITY OF THE ORNSTEIN-UHLENBEC UHLENBECK MODEL
The transition probability of the Ornstein-Uhlenbeck process considered in Section 5.1 is
Po(d x) = (x(x; t), xx (x) (x)
for time t > O, with mean (x t) x ex)(-t) and variance xx (t) = -11- -exp(-22)) From we have
,(x,y, = / XT, Ta) N -1) -()Ia)dxxr
| -t)
= -2x) 4/2 x
2 2 -0) d (T 1)008 of (T 1) exp
+ o
(T -
1 (x, T 1)113 ||y1
x exp
20 (T' -
20
Hence we can compute the value function y(x,y,t) - -log[1(X, (x, r)). Next, the optimal control function is
where (t) = (xx2(t) + 0x 2-
(x, y, 1) = TV log xt, of - 1) exp (T - {)}
[x T - 1) (T -
y expl- - [)}
x t)
+
2x -t)
/1x ex((x, -)-
The distribution of Xr conditioned on Xo= = Xo and Y = y N NNnn (Xo: v, of, (T))a with
(x(xo. y. (TT)
Hx 4x T)
+
y
xx (T)
2
0
A.3 LOGISTIC DIFFUSION MODEL
In this section we consider a logistic diffusion process (Dennis & Costantino 1988; Knape & De Valpine 2012) to model the dynamics of a population size (P (Pitezo. defined by
dP. = (03/2 2 01 P, dt + 0gP,
(22)
We apply the Lamperti transformation X, log(P, (Pt)/0g and work with the process
X, hrs that
satisfies [I] with p(x)= 0 - (02/0z) exp(0xx and (x) 1. Following (nnape & De Valpine
2012). wc adopt a negative binomial observation model g(x, y) = BN; 6g, exp(Ugx]) for counts
y No with dispersion 04 0 and mean exp(0xx), We set (01, 022 03 0a) as the parameter
estimates obtained in (Kaape & De Valpine 2012). Noting that (22) admits a Gamma distribution
with shape parameter 2 2 + 01)/0g
and rate parameter 202/03 as stationary distribution
(Dennis & Costantino] 1988), we select v/x as the push-forward under the Lamperti transformation and 7/ as the implied distribution of the observation when training neural networks under both static and iterative CDT schemes. To induce varying levels of informative observations, we considered 04 E {1.069, 4.303, 17.631, 17.6311,7..44 78.161).
Figure 3 displays our filtering results for various number of simulated observations from the model (Columns to 4) and for K 100 observations that are simulated with observation standard devi- ations larger than 04 = 17.631 used to run the filters (Column 5). In the latter setup, we solved for different values of 04 in the negative binomial observation model to induce larger standard devia- tions. The behaviour of BPF and Iterative-APr is similar to the previous example as the observations become more informative with larger values of 0₁. Iterative-APF outperformed all other algorithms over all combinations of 02 and K considered, and also when filtering observations that are increas- ingly extreme under the model. We note also that the APFs trained using the CDT static scheme can sometimes give unstable results, particularly in challenging scenarios.
A.4 CELL MODEL
This section examines a cell differentiation and development model from
Wang et al. 2011 Cel-
lular expression levels X, (X.,, Xx2) of two genes are modelled by with
1xt/(2-4 (x +
/2- x
m(x)=
x2/ -
- x1
(23)
X2
13

---

Under review as a conference paper at ICLR 2023
04
1.069
04
4.303
0,
17.631
84 78.161
100
E5S%
50-
50
50
50
20
1.00
1500
0.50
200
500
9.00
200
102
1.0
104
Variance
10a
10
10
10
107
10
107
16
10
10?
10
K
K
Standard deviation
BPF
Static-APF
Iterative-APF
Linear-GIRF
Quadratic-GIRF
Figure 3: Results for logistic diffusion model based on 100 independent repetitions of each PF. The ELBO gap in the second row is relative to Iterative-AP..
Oly 0.25
Gy 0.5
dy = 1.0
Oy = 2.0
100
100
100
100
100
Variance ELBO gap E5S%
40
10:
42000
6-0
0-:0
s0r 16
16'
10x 107
10a
107
16a
1or
167
16'
100
8
10
107
10
30"
1.0
100
10
101
10
10
10
102
10.7
10P
10'
10
10P
107
10
Standard deviation
BPF
Static-APF
Iterative-APF
Linear-GRF
Quadratic-GIRF
Figure 4: Results for cell model based on 100 independent repetitions of each PF, The ELBO gap in the second row is relative to Iterative-APF.
and (x) al. The terms in 23 describe self-activation, mutual inhibition and inactivation respectively, and the volatility captures intrinsic and external fluctuations. initialize the diffusion process from the undifferentiated state of X, (1, 1) and consider the Gaussian observation model g(x,y) Nyy; x, 12). To train neural networks under both static and iterative CDT schemes, selected Txx and 7/y the empirical distributions obtained by simulating states and observations from the model for 2000 time units.
numerical results for various number of observations K and σy (0.25, 0.5, 1.0, 2.0}. shows that Iterative-APF offers significant gains over all other algorithms when filtering observations that are informative (see Columns 1 to 4) and highly extreme under the model specification of ay = 0.5 (see Column 5). In this example, Static-APF did not exhibit any unstable behaviour and its performance lies somewhere in between BPF and Iterative-APF.
Figure 4 illustrates
A.5 GUIDED INTERMEDIATE RESAMPLING FILTERS.
We first describe our implementation of GIRF for online filtering. For 2 1 particles, let T& (dx) = ΣD 8(dx; xi,) denote a current approximation of the filtering distribution at time t4 2 0. Given the future observation YK+1 = Yk+1 at time f GIRF introduces a sequence of intermediate time steps 4A so 81 < 8p 1++1 between the observation times, and
14

---

Under review as a conference paper at ICLR 2023
sequence of guiding functions (Gy)p=o satisfying
Go Yk+1) II (X Yk+1) = ( p=1
(24)
For each intermediate step P {1, {I....,P), P), the particles xI:M are then propagated forward accord-
ing to the original SDE ① i.e. &,y11 ~ Pasp+i (d& x2,)) with stepsize Asp+1 = Sp+1 8p- In practice, this propagation step can be replaced by a numerical integrator. Each particle &2. is then
M associated with normalized weight W W p+1l E W p+1. where the unnormalized weight
W = Gp(x2 2o Yk+1), D (L,.,,,, P-1}.
Vk+2), if t the final observation time, W Gp(xL &pp yk+1), if tx41 is the final observation time.
w Gp(x, yk
After the unnormalized weights are computed, the resampling operation is the same as standard PF (see Section 2.2
From the above description, we see that the role of [Gp]p=o is to guide particles to appropriate regions of the state-space using the weighting and resampling steps. The optimal choice of guiding functions (Park& & lonides 2020) is
h(Xs, Vk+1,
Go(Xso= Vk+1) = YK+1, 80); Gp(Xs, Vk+1) = h(X Yk+1, 8p-1)
(25)
for E {1,.... where /:: X' x y x 0, 7 + R. defined in (6) is given by Doob's h-transform.
The condition
satisfied (XXe,...K++1 For the Omssein--Ulleneer
have telescoping product and h(x 11 Vk+1, (x+1)
model of Section 5.1 we leveraged analytical
tractability of (25 in our implementation of GIRF. When the optimal choice (25) intractable, one
sub-optimal but practice choice that gradually introduces information from the future observation by annealing the observation density
9(xap, V8+1) )
Go(Xo0 FK+1) = ((+1)) 2x, Xx g(X,,- 11 Vk+ A
for p E {1..... P}. where (Ap) p-ppppn is a non-decreasing sequence with Ap 1. This construction clearly satisfies the condition (24). It interesting note that under the choice A, = for P E f1...., P 1}. GIRF recovers the BPF. In our numerical implementation, we considered both linear and quadratic annealing schedules {Ap) which determine the rate at which information from the future observation is introduced.
Lastly, we explain why GIRF with the optimal guiding functions (25 is still sub-optimal compared to an APF that move particles using the optimal control X' x x [0 TJ Ed induced by Doob's h--ransform. We consider the law of {X.,) 1P" conditioned on X X and Y = Yk+1
II pAs, (dxs, Xx, (g((xp Vkt1). p=1
(26)
Under the condition 24
we can write the law 26 as
P
Go(Xso Yk+1)
Il Pas, (dx,, Xsy_ (X,, Xay; Yk+1).
(27)
p=1
GIRF can be understood as Sequential Monte Carlo (SMC) algorithm (Chopin et al. imating the law (27) with Markov transitions {pa,,} 1 and potential functions [ given by
2020) approx-
25
We can rewrite (27 as
P (6+11) II
p2ao (dx,, () p=1
(28)
15

---

Under review as a conference paper at ICLR 2023
where Markov transitions {r's,, 1 defined as
pha,, (dxx, (x PAsy (dx,, Xsy Nk+1 Sp)
(29)
h(xx,-11 Y k+1, 8p-1)
for p {I,..., P). By the Markov property, we have h(xs V:44+1 (4+1,8p-11) 1--1 .xx Pas, (dx:, Xap M hence (29 valid Markov transition kernel. Moreover, follows from Dai Pra 1191 Theorem 2.1) that 1r' are the transition probabilities the controlled dif-
fusion process in with optimal control (x, y, lo V Vogg yx, t). Hence APF prop- agating particles according to this optimally controlled process can be seen as SMC algorithm ap- proximating (28 with Markov transitions {p p and a single potential function Go. By viewing GIRF and APF specific instantaneous of SMC algorithms, it clear that the former is sub-optimal compared to the latter. Intuitively, this means that better particle approximations can be obtained by moving particles well instead of relying on weighting and resampling.
A.6 COMPUTATIONAL DOOB'S h-TRANSFORM ALGORITHM
In this section, provide figures to illustrate how our proposed CDT algorithm behaves. We report the training curves (i.e. loss v.s. iteration), as well as describe the evolution the ap- proximate control functions parametrized by the neural networks. In the analytically tractable Maastein-Uhlenbeck case, comparison with the optimal control is possible. See Figures S and for the AimsteinUUhlenbeck model of Section 5.1 Figures 7 and g for the logistic diffusion model of Section A.3 and Figure 9 for the cell model of Section A.4
16

---

Under review as a conference paper at ICLR 2023
0.8
0.6
0.5
10a
0.2
0.0
loss
10m4
-0.2
-10
1.5
1.0
0.5
0.0
0.5
1.0
1.5
1.5
1.0
6.5
0.0
0.5
1.0
1.5
10-4
X
(b) Evolution neural network (x(x,)) y) (black to copper) ap- proximating the initial value function vxx, y. 0) (red) over first 500 optimization iterations for typical (leff) and an extreme (right) observation y.
iteration
(a) Evolution of loss estimate over first 500 optimization iterations.
0.25
0.5
0.75
0.4
0.6
1.00
0.4
0.50
0.0
0.00
0.2
0050
-0.4
0.4
-0.6
-1.00
-1.5
1.0
-05
0.0
0.5
1.0
1.5
-1.
-1.0
-05
0.0
0.5
1.0
15
-1.5
1.0
0.5
0.0
0.5
1.0
1.5
2.0
2.5
1.5
2.0
1.0
1.5
1.0
0.5
10
0.5
0.5
0.0
-05
1.0
0.5
1.0
1.5
005
1.0
15
-.0
0.5
0.0
0.5
1.0
1.5
(c) Evolution of neural network -N(x,y,0) (black to copper) approximating the optimal control function C. y,, (red) over first 500 optimization iterations for a typical (upper row) and an extreme (lower row) observation y.
Figure 5: Results for Ornstein phase.
Omstein-Uhlenbeck Uhlenbeck model with d = 1 and = 1.0 during initial training
17

---

Under review as a conference paper at ICLR 2023
0.125
Static-CDT
0.100
0.5
109
Iterative CDT
0.075
loss
0.025
0.000
10a1
-15 115-10-00
0.0 0.5
1.0
1.5
-1.5-10-0.5 0.0
0.5
1.0 155
X
X
250
500
750 1000
1250 1500 1750 2000
iteration
Iteratie-CDT
(a) Evolution of loss estimate over 2000 optimization iterations under static and iterative CDT schemes.
(b) Neural network approximation No(x. y) initial value function v(x, y, 0) after training with the static and iterative CDT schemes for a typical (left) and an extreme (right) observation
t= 0.25
t= 0.5
= 0.75
0.2
0.1
0.2
0.0
0.1
0.2
-022
-0.4
-0.2
0.4
-0.5
-1.5
1.1
2.2
1.5
0.7
125
0.8
0.8
-0.
0.0
X
Static-CDT
Ieeatiie-CDT
Exact
(c) Neural network approximation -N -N(x, y. !) of the optimal control function c. (x,y, y, after training with the static and iterative CDT schemes for typical (upper row) and extreme (lower row) observation y.
Figure 6: Results for Ornstein-Uhlenbeee model with d = I and σx 1.0 after training.
18

---

Under review as a conference paper at ICLR 2023
14-
15.0
12.5
10.0
7.5
101
5.0
1os5
10P
3019
6.0
6.5
7.0
7.5
8.0
8.5
6.0
6.5
7.0
7.5
8.0
8.5
X
400
(b) Evolution of neural network No (x. y) (black to copper) ap- proximating the initial value function v(x. y, 0) over first 500 op- timization iterations for typical (left) and an extreme (right) ob- servation y.
iteration
(a) Evolution of loss estimate over first
500 optimization iterations.
t 0.25
0.5
t 0.75
0.6-
0.75
0.4
0.4-
0.25
-0.2
-0.
-3.25
-0.4
-0.4
0055
0.50
-0.8
6.0
6.5
7.5
8.5
6.0
6.5
7.0
7.5
8.0
6.0
7.0
7.5
6.0
8.5
2.0
1.0
1.0
0.5
0.0
0.0
0.0
0.5
-0.5
6.0
6.5
1.0
1.5
8.0
5.5
6.0
6.5
7.0
7.5
5.0
6.0
1.0
1.5
6.0
8.5
(c) Evolution neural network - N -Nxx, y, (black copper) approximating the optimal control function
C- (x, y, t) over first 500 optimization iterations for typical (upper row) and an extreme (lower row) observa- tion
Figure 7: Results for logistic diffusion model with 1.069 during initial training phase.
19

---

Under review as a conference paper at ICLR 2023
04
1.069
04 4.303
17.631
0. 78.161
10a
10a
10a
10a
10a
10a
10a
loss
10
10a
10-1
10
10-4
10-1
10
10
10
20020
500
1000 1500
2000
500
1000
1500
2000
500
1000
1500
2000
500
1000
1500
2000
iteration
iteration
iteration
iteration
Static-CDT
Iterative-CDT
(a) Evolution of loss estimate over 2000 optimization iterations under static and iterative CDT schemes and various levels of informative observations.
7.50
7.45
7.40
7.35
7.30
6.5
7.0
7.5
8.0
6.0
7.0
7.5
8.0
8.5
Static-CDT
Merative-CD1
(b) Neural network approximation (x, the initial value function v(x, y. 0) after training with the static and iterative CDT schemes for typical (left) and an extreme (right) observation
t 0.25
t= 0.5
t= 0.75
0.8
0.4
0.3
0.4
0.2
02
0.0
0.9
2.00
1.4
1.75
1.2
1.75
1.25
1.50
1.30
1.25
0.6
1.00
0.50
0.0
6.0
7.5 X
X
Static CDT
Iterative CDT
(c) Neural network approximation - -N(x, y. !) of the optimal control function c, (x, ,, () after training with the static and iterative CDT schemes for typical (upper row) and an extreme (lower row) observation y.
Figure 8: Results for logistic diffusion model with 04 = 1.069 after training.
20

---

Under review as a conference paper at ICLR 2023
dy 0.25
θy 0.5
= 1.0
Oy 2.0
103
10a
102
10a
10a
10a
10a
10a
loss
10a
10"
10a
10*
102
10
10
10
10-1
10"1
500
1500
1000
2000
són
1080 1500
iteration
iteration
iteration
iteration
Static-CDT
Iterative-CDT
(a) Evolution of loss estimate over 2000 optimization iterations under static and iterative CDT schemes and various levels informative observations.
(b) Neural network approximation No(x, y) initial value function v(x, y: 0) after training with the static (left column) and iterative (right column) CDT schemes for a typical (upper row) and an extreme (lower row) observation
t=025
X1
X1
X1
X1
(c) Neural network approximation -x, t) of the optimal control function c, (x, W, f) after training with the static (upper row) and iterative (lower row) CDT schemes for typical and (columns 1-3) extreme (columns 4-6) observation y.
Figure 9: Results for cell model with ax 0.5 after training.
21


Reviews:
[{"review_id": "u1AtX077FS", "reviewer": "Reviewer_rVdn", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a new filtering methodology based on computational Doob's h-transform. The novelty is to get more efficient proposals, at least aim at it.", "summary_of_the_review": "This paper has a number of problems before it can be published in a venue like ICLR. I think the current version is simply not interesting to the researchers in this field (ICLR crowd) - even to particle filtering people.\n\nThe paper phrases the problem of getting a good proposal as a control problem and approximates the relevant quantities using complex function approximators using neural networks. The claim is that the resulting method is more efficient under some extreme situations, like outliers or high-dimensions.\n\nThis paper could be an interesting contribution, it lacks very crucial experimental evidence:\n\n(i) While there is a sentence about runtime being two minutes and being less costly than BPF with \"many particles\", this claim was not made quantitative by *showing* that this is the case. This is not how computational efficiency claims are made.\n\n(ii) The dimensions are tested up to 32 are quite low. The claim in the earlier parts of the paper about efficiency in high-dimensions are completely unsupported. There are simply no true high-dimensional experiments, nothing like some of the published literature on PFs.\n\n(iii) \"Informative observations\" were tested by changing likelihood variance but another claim in the abstract \"when the observations are extreme under the model\" was not tested properly. What kind of extremes are we talking about? Is there a certain statistical setting for these extremes? This was not in the main text and might be pushed somewhere in the appendix, despite it looks like there was lots of space to use in the main text (e.g. page 8).\n\nGiven that there is almost no convincing experimental result, I won't discuss theoretical setting. It looks like this paper was rushed for the deadline and there was no time for experiments. It definitely needs comprehensive and rigorous updating and is not suitable for publication at this stage.", "strength_and_weaknesses": "Strength: Getting good proposals within particle filtering is an important problem\n\nWeaknesses: I think there were many (see below), mainly\n\n(i) Lack of clear and coherent description of the problem\n(ii) Experiments are low dimensional and are not interesting", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "T3L1SyoegB", "reviewer": "Reviewer_AGRQ", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies how to simulate conditional diffusion processes -- where given some observations, we wish to study what the resultant state of the unobserved proportions of the diffusion process may be. To solve this issue, the authors propose a  Computational Doob's transform, leveraging neural networks to more accurately observed the proportions of the diffusion process", "summary_of_the_review": "I don't really understand this paper -- I'm giving the paper a borderline accept but am complete unsure of my assessment.", "strength_and_weaknesses": "Disclaimer -- I found this paper rather challenging to read -- it seems to be targeted towards a more applied math audience rather than a deep learning/ICLR audience and I am unsure of what exactly is being learned by the neural network / training objective. \n\n# Strengths\n\n**Significance.** This paper appears to solve a significant issue where we seek to effectively determine unobserved portions of a diffusion process.\n\n**Clarity.** The paper seems to be relatively well written.\n\n# Weaknesses\n\n**Research Area.** I'm not sure if this paper is targeted to the ICLR audience. I had large amounts of difficulty reading the paper and could not understand what exactly was being learned.\n\n**Clarity.** The paper has a large number of mathematical formula and was very dense to read. It may be nicer to provide a high level overview of what the paper is trying to as well as the existing challenges and what explicitly the work is trying to tackle.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "LU-0MltxIRZ", "reviewer": "Reviewer_FHwj", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper deals with online filtering of discretely observed nonlinear diffusion processes which has rich applications in fields such as security pricing and modeling in finance. The authors propose a computational framework to approximate the Doob’s h-transforms that are typically intractable by solving the underlying backward Kolmogorov equations using nonlinear Feynman-kac formulas and neural networks. \nSimulations and numerical experiments show that this approach can be orders of magnitude more efficient than sota particle filters in difficult regimes. \n", "summary_of_the_review": "The overall solution to online filtering of discretely observed nonlinear diffusion processes is a promising direction in so many real-world applications. This paper provides approximations by leveraging neural network and Keynman-kac formulas which is interesting.\n\nThe work can be further improved by applying to real-world datasets and the network network’s architecture should be described and compared in detail for ablation study.\n", "strength_and_weaknesses": "Strong\n\n1. A novel computational framework with nonlinear feynman-kac formulas and neural networks to approximate these Doob’s h-transforms;\n\n2. Detailed background, method and related work expressions to better learn the motivations and solutions of the major novel ideas;\n\nWeak:\n\n1. Currently, could find less information about the neural networks used and how representative learning is performed – this paper aligns more with statistics and applied mathematics;\n\n2. More real-world datasets such as security pricing or portfolio changing experiments are preferred to be the application fields to better testify the ideas/algorithms proposed in this paper.\n\n\nDetailed questions and comments:\n\n1. Figure 1, 2, are curves – is it feasible to also list tables with specific numbers to learn the exact improvements (ranges) and differences?\n\n2. The OU process (Ornstein-uhlenbeck) is frequently used in real-world domains such as finance, do we have further experiments on real-world datasets and comparisons with baseline algorithms?\n\n3. ‘neural networks’ are mentioned around 30 times in this paper, prefer to learn the detailed architecture of the neural network and the complexity of it.\n\n4. In the reference, I can hardly find machine learning related references or articles, or even representative learning related. So this paper is more suitable for statistics or mathematics.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "D8WxfkGLnEQ", "reviewer": "Reviewer_dabH", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a mathematically-grounded method for solving nonlinear filtering problems in a particle-based manner. The method is based on Auxiliary Particle Filters and, compared to Bayesian Bootstrap Filter [1], requires substantial mathematical and ML engineering efforts to be practically implemented. The authors show the validity of the approach by modeling several synthetic nonlinear filtering routines. \n\n[1] Salmond et. al. Novel approach to nonlinear/non-Gaussian Bayesian state estimation, 1993\n", "summary_of_the_review": "The proposed method is theoretically interesting and mathematically rich. However, the applications section should be expanded with more convincing experiments, including real-world applications.", "strength_and_weaknesses": "Strength: The mathematical foundations of the proposed method are interesting, but the problem of nonlinear filtering also seems to have important applications. \n\nWeaknesses: From my point of view, the paper under consideration has a relatively poor experimental section. Actually, the authors consider several synthetic nonlinear filtering problems and compute the metrics which only indirectly assess the quality of the proposed nonlinear filtering approach. Therefore, I still don't know if the method outperforms the competitive ones. I understand that it is difficult to find nonlinear filtering problems with known ground truth. That is why a better way is to consider some real-world nonlinear filtering applications with domain - specific metrics (probably not directly related to the nonlinear filtering itself) and evaluate the proposed approach on such a task. \n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject