Computational Doob h-transforms for Online Filtering of Discretely Observed Diffusions
Authors: Nicolas Chopin
Andras Fulop
Jeremy Heng
Alexandre H. Thiery
Venue: NeurIPS 2022 Submitted (2022)
Keywords: diffusion
filtering
monte-carlo
particle-filters
BSDE

Abstract: This paper is concerned with online filtering of discretely observed nonlinear diffusion processes. We propose to approximate the Fully Adapted Particle Filter algorithm by solving a single auxiliary stochastic control problem prior to the data-assimilation procedure. The methodology relies on the non-linear Feynman-Kac approach to solving semi-linear partial differential equations. Numerical experiments suggest that the proposed approach can be orders of magnitude more efficient than the bootstrap particle filter in the regime when observations are highly informative.

Computational Doob's h-transforms for Online Filtering of Discretely Observed Diffusions
Anonymous Author(s)
Affiliation
Address
email
Abstract
This paper is concerned with online filtering of discretely observed nonlinear diffu- sion processes. Our approach is based on the fully adapted auxiliary particle filter which involves Doob's 's h-transforms that typically intractable. We propose computational framework to approximate these h--rranffrms by solving the under- lying backward Kolmogorov equations using nonlinear Feynman-Kac formulas. The methodology allows one to train a locally optimal particle filter prior to the data-assimilation procedure. Numerical experiments illustrate that the proposed approach can be orders of magnitude more efficient than the bootstrap particle fil- in the regime of highly informative observations and when the observations are extreme under the model.
2
4
6
10
Introduction
12
Diffusion processes are fundamental tools in applied mathematics, statistics and machine learning. This rich class of continuous-time models have been used model real-world phenomena in disci- plines as varied as life-sciences, engineering, economics and finance. However, working with dif- fusions can be challenging as its transition densities are only tractable simple and specific cases such as (geometric) Brownian motions, Ornstein- Uureinn--hlbbbckk (OU) processes and Cox-Ingersoll-RRe processes. This difficulty has hindered the use of standard methodologies for inference and data- assimilation of models driven by diffusion processes. Specialized methodologies have been devel- oped to circumvent or mitigate these issues [35, 4, 3, 14, 13, 5, 37].
14
15
16
17
18
19
20
Consider a tome-hooogeneuus multivariate diffusion process dX, = (x(,,) dt + (X,) dB, that discretely observed at regular intervals. Noisy observations Yı of the latent process X,, is collected at time l = A for k 2 1. We consider the online filtering problem which consists in estimating the conditional laws ne. (dx) as observations are collected. We focus on the use of Particle Filters (PF) that approximate the filtering distributions with a system of weighted particles. Although many previous works have relied on the Bootstrap Particle Filter (BPF), which simulates particles from the diffusion process, can perform poorly challenging scenarios it fails to take the incoming observation yı into account. This issue can be partially tackled by relying on resampling at intermediate times between observations [10, 31]. The goal of this article is to show that the (locally) optimal approach given by the Fully Adapted Auxiliary Particle Filter (FA-APF) [33] can be implemented. This necessitates simulating conditioned diffusion process, which can be formulated as control problem involving an intractable Doob's h-trrasffrmm [36, 8). We propose the Computational Doob's 's h-Transform (CDT) framework for efficiently approximating these quantities. The method relies on nonlinear Feynman- Kac formulas for solving backward Kolmogorov equations simultancously for all possible observations. Importantly, this preprocessing step only needs to be performed once before starting the online filtering procedure. Numerical experiments illustrate that the proposed approach can be orders
21
22
P(X,, E dxly1,,..., ; i.e. the filtering distributions,
24
25
28
27
28
29
30
31
33
35
Submitted to 36th Conference on Neural Information Processing Systems (NeurIPS 2022). Do not distribute.

---

37 of magnitude more efficient than the BPF in the regime of highly informative observations or when the observations are extreme under the model. PyTorch implementation reproduce our numerical experiments is available at https: / anonymous 4open. sccience/r/CommpooTTrma
38
Notations. For two matrices A, B e Rddd, their Frobenius inner product is defined as (A, B)F Dij=1 Big. The Euclidean inner product for u, E Rd is denoted as (u, v) Σim U22%-
40
Background
42
2.1
Filtering of discretely observed diffusions
Consider an homogeneous diffusion process (X, 122 in X' = Rd with initial distribution po(dx) and dynamics
44
dx, (Xx)) di a (X,) dB,
described by the drift and volatility functioos μ R2d + Rd and F² + Rd.dd We assume standard smoothness and growth conditions [27] for a unique strong solution of (1) exist for all times. The associated semi-group of transition probabilities p, P.(dR x) satisfies IP(X++s E A X, = x) = C X. The process {Br is a standard IR4. valued Brownian motion. The diffusion process {X, Fro0 is discretely observed at time t4 = AT, for k > 1, for some inter-observation time T >> 0. The V-valued observation Y y at time f2 is modelled by the likelihood function g X' x y IR., in the sense that for any measurable A C y, have P(Y A Xt, xx) = (Xx: : y) dy for some dominating measure dy on y. For test function X + I,, the generator of the diffusion process (X+)220 is given by
46
47
48
49 SA (ddd x) for any 8, > O and measurable
50
52
1
C (/. V4) + (ooT, V2
(2)
55
This article concerned with approximating the filtering distributions Tx P(X,, dx ,i....,Kk) For notational convenience, we set To (dx) (dxx)..
56
2.2
Particle filtering
58 Particle Filters (PF), also known as Sequential Monte Carlo methods, are a set of Monte Carlo algorithms that can be used to solve filtering problems (see [7] for recent textbook the topic). PFs evolve a set N 2 particles x ::NN (x.,..... x,¹) E X forward in time using a combination of propagation and resampling operations.
60
To initialize the PF, each initial particle x E X' for S S N is sampled independently from 62
83 the distribution po(xx) so that (d(xx) E'' o(dx; Approximations of the filtering distribution тT for 2 1 are built recursively follows. Given the Monte Carlo approximation 85 of the filtering distribution time lg, <dxxx) N - 8(dx; x²,)), the particles x!: are
64
x, using Markov kernel + (d& x) specified by the user. The BPF corresponds to the choice Markov kernel BPF (d&
propagatdd independently forward in time by 2 g (d& (
67
68 x) (X: E dx X., =x) x) while the FA-APF [33] corresponds to the choice
FA-APF (dx x) P(X 1 E dx X,, Y k+1 -k+1). g
89 Each particle X is associated with a normalized weight W h+1 = W Whi where the unnormalized weights W
70
V are defined
Pr(d x(x)
Wi R+1
g
(88d Mk+1).
(4)
Bh+ (dR) 2411 x2,,)
The BPF and FA-APF correspond respectively to having and
WPP.PF = (k+1))
J.P.AAP k+1
EEg Elg((X X1, = x/. (5)
k+1

---

The weights such that Tx++1 (dx) Ej=1 W k+1 8(dx; x ( The resampling step consists in 73 defining a new set of particles x with IP(x) x 14+1
) = W k+1- This resampling scheme ensures that the equally weighted set of particles x x provides Monte Carlo approximation of the
74
-==>i= 6(dx; x((_,). Note that the particles X do not need to be resampled independently given the set of propagated particles &1.N We refer the reader to [15] for a recent discussion of resampling schemes within PFs and to 1
filtering distribution at time 1R+1 in the sense that TT& R N 2j=1
75
77
78
[9] for a book-length treatment the convergence properties of this class of Monte Carlo methods.
79 In most settings, the FA-APF [33] that minimizes a local variance criterion [12] leads to better
80 performance when compared to the BPF. This gain in efficiency can very substantial when the
81 signal-to-noiss ratio is high or when observations contain outliers under the model specification.
82 Nevertheless, implementing FA-APF requires sampling from the conditioned transition probability in (3), which is typically not feasible in practice. We will show in the following that this can be
84 achieved our setting by simulating a conditioned diffusion. We note also that standard strategies to approximate the FA-APF do not apply to our setup as the latent state process evolves on higher
86 frequency relative to the observations.
87
2.3
Conditioned and controlled diffusions
86
As the diffusion process (1) assumed to be time-homogeneous, it suffices focus on the initial interval [0, T] and study the dynamics of the diffusion X [o,T] {X, (te(o,T) conditioned upon the first observation YT = y. The conditioned dynamies can also be described by a diffusion process. Contrarily to the original diffusion, the conditioned process is not himeehoogeeesuu in general. The conditioned process described by the same volatility function but with a different drift term that takes the future observation YT y into account.
89
90
91
92
94
Before deriving the exact form of the conditioned diffusion in Section 2.4, this section describes more general setting that will be of crucial importance in our proposed numerical scheme. For a Fd and a given observation y y, consider the controlled diffusion process {X, [E([T satisfying
85
96
control function e X' x [o, T]
97
dx:
((X. dt + (X. dB,
|o [X dt.
(6)
(original dynamics)
(control drift
The dynamics of the controlled diffusion is identical to the original diffusion, except for the additional a9 drift term (ar yx, y, E Rd described by the control function c. For y E ) and test function
98
100
: X + IR, the generator of the controlled diffusion is given by
( Cip(x) (x, y. t), Vy(x)).
101
[o.T] denote the probability measures on the space of continuous functions C((O, T], Rd) generated by the original and controlled diffusions respectively. Under mild growth as- sumptions on the control c, the two measures are equivalent and Girsanov's 's theorem [16] shows that
Let P P(O.T) and pay
103
exp { 1 / [c((X,,,, - 2
J' (cl (o((X,,, tB)
0.7 y
X (.T) [.
(8)
Our main objective is to construct a control function c, X' x y x [0, Rd so that, for any
104
observation value y E V. the controlled diffusion X cC. has the same dynamics as the original
105
(0.T]
diffusion 1 conditioned upon the observation Y = y, i.e. for any measurable set A C
106
C((0, Ti, Rd), we have
107
P P E A) = E [1 E A)g((T y)] (lg((xr, [.
108
We will give an exact expression of this control Section 2.4 and propose a numerical scheme to approximate it in Section 3.1.

---

110 2.4 Doob's h- i-trrasffrmm
111 To simplify notation, we shall denote the conditioned process (o.)) (Yr = y) as [ To
112 describe its dynamics, we introduce the function
y, (X X; = x / g(xT, pr p x)
(10)
113 which gives the probability of observing Yr = y when the diffusion has state x E X' at time EE [0. We recall that the likelihood function g X x y E+ defined in Section 2.1.
115
The definition in (10) implies that the function h X x y x [0, T] + R+ satisfies the backward
Kolmogorov equation [27].
1)+ -0 0,
(11)
with terminal condition 1(x, y, T) = y(x,y) for all (x, E X x V. For 0 X + R and an infinitesimal increment ô > 0, we have
118
El(((X+0)) x] Elp(((++)) XT.y) Xx x| (X = x
E[p(X++6)) h(X v, 1 X, =x] h,
(12)
+x(x + 8 Clp h|
(x, - ((02)
h
119
Furthermore, since the function h satisfies (II), some algebra shows that Lle hi/h = Cip (ao " V log h Vip). By taking 8 0, this heuristic derivation shows that the generator of the condi- tioned diffusion equals Li (ao T V log h. Vip). Hence X Jo.T] satisfies the dynamics a controlled diffusion (6) with control function c. tt) = [o Vlog (x,, y. t). We refer readers to [36, 8] for a formal treatment of Doob's s h-transform.
120
121
122
123
124
2.5
Nonlinear Feynman-Ka formula
Obtaining the control function C. (x, = lor T V log y,, y, by solving the backward Kol- mogorov equation in (11) for each possible observation y É y is computationally not feasible. Fur- thermore, when the dimensionality of the state-space X' becomes larger, standard numerical methods for solving Partial Differential Equations (PDEs) such Finite Differences or the Finite Element Method become impractical. For these reasons, we propose instead to approximate the control func- tion c, with neural networks, and employ methods based on automatic differentiation and the nonlin- car Feynman-Ka approach to solve semilinear PDEs [19, 20, 24, 17, 6. 22, 23, 1, 18].
125
126
127
128
129
130
131
As the non-negative funetion h typically decays exponentially for large |xx||. it is computationally more stable to work on the logarithmic scale and approximate the value function v(x, y, t) log[h(x y, {) Using the fact that h satisfies the PDE (11), the value function satisfies
133
(0, L)u= 2 ||o Vol?, v(x,y, = - loglg(x,))) for all (x, E X x V.
(13)
Let ( [E[[. be controlled diffusion defined in Equation (6) with given control function
135
c: : x y x 00, + Rd and define the diffusion process [Vehrejo. as V, = v(X. .y,t. Itō Lemma shows that for any observation YT = y and S T. we have
136
137
L-
( 11 (c (c. dt de+ {/= (Ze, dB,)
VT VT= V.
with Z. = [oT .y,t) and VT loglg .y), In summary, the pair of processes
138
139
(Ve, Ze) are such that the following equation holds,
- loglo ( y)] = V. J' {1 |z.11 + (c. Zi) jot + [ (Z,, dB.).
(14)
140 Crucially, under mild growth and regularity assumptions on the drift and volatility function μ :
X' + I and a X Rd.dd, the pair of processes (V,, Z,) is the unique solution to Equation (14)
142 [28, 29, 30, 40]. This result can be used as a building block for designing Monte Carlo approximations
of the solution to semilinear and fully nonlinear PDEs [18, 34, 21]

---

144
3
Method
145
3.1
Computational Doob's h--transform
148
As before, consider diffusion { hee(orr) controlled by a function c: X' x y x [0, T + R² and
147
driven by the standard Brownian motion (Br Furthermore, for two functions No X' x y - R
148 and N X x V x [0, + Rd, consider the diffusion process (V (Ve)ee(o.T) defined as
V, Vo + [- { 1z.l + (c(X .y.0) Zi) jdl + (Z..
(15)
where the initial condition Vo and the process hejory are defined
Vo- No .)
and Z. = N(X;) t,t).
(16)
150
Importantly, we remind the reader that the two diffusion processes X; and V, are driven by the
same Brownian motion B, The uniqueness result mentioned at the end of Section 2.5 implies 152 that, if for any choice of initial condition X E and terminal observation y E y the condition VT (Xg ey y)] is satisfied, then we have that for all (x, X x 3 [0, T
y) = - log y, 0) and N(x, y. = - Vo log hi yt.
(17)
In partieular, the optimal control is given by c, (x, y.t)
N(xx
155
These remarks suggest parametrizing the functions No(::-) and N(.,,; by two neural networks with respective parameters On E Oo and 0 E θ while minimizing the loss function
156
L(du. 0: c) E (V log x
(18)
157
The above expectation is with respect to the distribution of the Brownian motion (Be)e20. the initial condition X 1/x (dx) of the controlled diffusion, and the observation Y 7/y (dy) at time T. In practice, we will let the three sources of randomness be independent of each other. The spread
158
159
160
of the distributions 1/x and !!y should be large enough to cover typical states under the filtering distributions Tı, k 2 1 and future observations to be filtered respectively, Specific choices will be detailed for each application in Section For offline problems, one could learn in a data-driven as the empirical distribution of actual observations. Furthermore, any control function c : X x [0, T] + Rd with mild growth and regularity assumptions can be employed within our methodology: specific choices are discussed at the end of this section.
161
162
163
manner by selecting
164
185
166
CDT algorithm. The following outlines our training procedure to learn neural networks Ny and N that satisfy (17). To minimize the loss function (18), any stochastic gradient algorithm can be used with a user-spedified bati- size of J 2 1. The following steps are iterated until convergence.
188
1. Choose control X x y [0, T + Rd possibly based the current neural network parameters (0,,0) 0) E On x O.
169
170
2. Simulate independent Brownian paths B B initial conditions X 17x (dx), and observa- tions Y T> (dy) for 1 < < J.
172
3. Generate the controlled trajectories: the j-th sample path X X obtained by forward integration of the controlled dynamics in Equation (6) with initial condition X control c(,, Y),,), and the Brownian path B
173
174
175
4. Generate the value trajectories: the j-th sample path V (o.TT is obtained by forward integration of the dynamics in Equation (16) with the Brownian path By Bo and the current neural network parameters (0o, 0) E O, O.
178
177
178
5. Construct a Monte Carlo estimate of the loss function (18):
(v + loglo(X4,
(19)
5

---

6. Use automatic differentiation to compute do and 0gL and update the parameters (fv. 8).
180
Importantly, if the control function с in Step: does depend on the current parameters (0o, 0), the gradient operations executed in Step:ó should not be propagated through the control function c. A standard stop-gradi gradient operation available in most popular automatic differentiation frameworks can be used for this purpose.
182
184
dime-discretization of diffusions. For clarity of exposition, have described our algorithm in continuous-time In practice one would have to discretize these diffusion processes, which is entirely straightforward. Although any numerical integrator could potentially be considered, the experiments in Section 4 employed the standard Euler-Maruyama scheme [25].
186
188
189
Parametrizations of functions No and N. In all numerical experiments presented in Section the functions No and N are parametrized with fully-connected neural networks with two hidden layers and the Leaky ReLU activation function except in the last layer. Future work could explore other neural network architectures for our setting.
180
191
192
193
Choice of controlled dynamics. In challenging scenarios where observations are highly informative and/or extreme under the model, choosing good control function to implement Step: of the proposed algorithm can be crucial. We focus on two possible implementations:
194
195
196
CDT static scheme: a simple (and naive) choice is not using any control, i.e. y, =
0 E Rd for all yx E X x y [0,
197
198
CDT iterative scheme: use the current approximation of the optimal control described
199
by the parameters (0o, 0) E O x O. This corresponds to setting c(x, y, = -N(x y, t).
200
While using a static control approach can perform reasonably well in some situations, our results in Section 4 suggest that the iterative control procedure is more reliable strategy. This is consistent with findings in the stochastic optimal control literature [38, 32). This choice of control function drives the forward process X. to regions of the state-space where the likelihood function is large and helps mitigate convergence and stability issues. Furthermore, Section 4 reports that (at convergence), the solutions No and N can be significantly different. The iterative control procedure leads to more accurate solutions and, ultimately, better performance when used for online filtering.
202
204
205
206
207
3.2
Online filtering
208
Before performing online filtering, we first run the CDT algorithm described in Section 3.1 to construct
209
an approximation of the optimal control c, (x. y, F) = [a V log h], y, t). For concreteness, denote
by è: X' y x [0, T] + I& the resulting approximate control, i.e. C(x, ()) -N(x, y, () where ((....) -) is parametrized by the final parameter E θ. Similarly, denote by Vo X' x y + R the 211
210
approximation of the initial value function v(x, y. 0) = log h(x,y, 0), i.c. Vo(x,y) = yx where No(-.-) parametrized by the final parameter 6g E Og.
212
213
214
For implementing online filtering with N 2 particles, consider a current approximation T&. (dx) = N 1 Σソ= x(dx;x(,) ) of the filtering distributions at time t 2 0. Given the future observation
215
Y 1 Yk+1, the particles J:N are then propagated forward by exploiting the approximately
216
optimal control (x, t) + YEt1, fx). In particular, xe/ 1141 is obtained by setting x = X
217
1242
218
where [ follows the controlled diffusion xx
(x²) dt + +xx dB
|oc| X2 yk41, tx) dt
(20)
(original dynamics)
(approximately optimal control)
initialized at X x Each propagated particle & is associated with normalized
0/
weight W k+1
W W W where W Wit (dP (e / /dp
[te (R x
220
221 Vk+1). We recall that the probability measures Pitatees1 and p
correspond to the
original and controlled diffusions on the interval [tx, te+1). Girsanov's 's theorem, as described in Equa-
223 tion (8), implies that
{ 1 1x+1
(e+t (Z. + log gogg((X( +oggo(X Nk+11) [
W k+1 = exp
{
|2/1| di +
2 k/

---

where Z = -a(X Viti, tx). Similarly to Equation (15), consider the diffusion process
224
(V) ( (el defined by the dynamics dV? - -=1l 11 dt + (Z), dBB) with initialization at
V%. (x+x) Therefore the weight can be re-written as
226
W exp
{ V + log ( } exp { (o(xx)y
(21)
=00
and computed by numerically integrating the process { (V: he(te. [1]- Given the definition of the loss function in (18), we can expect the term within the first exponential to be close zero. In the ideal
227
228
case where č(x, t) = c. (x. y. and Vo(x. y) = log h(x, y. 0), one recovers the exact AF-APE weights in (5). Once the unnormalized weights (21) are computed, the resampling steps are identical
229
230
231
to those described in Section 2.2 for standard PF. For practical implementations, all the processes
involved in the proposed methodology can be straightforwardyy lime-diccreizzdd To distinguish
232
233
between CDT learning with static or iterative control, we shall refer to the resulting approximation of FA-APF as Static-APF and Iterative-APF respectively.
4
Experiments
235
236
This section presents numerical results obtained on three models. All experiments employed 2000 iterations of the Adam optimizer with a learning rate of 0.01 and a mini-batch size of 1000 sample paths with 10 different observations. Training times took around one to two minutes on a standard CPU. We note that this compute time is marginal compared to the cost of running filters with many particles and/or to assimilate large number of observations. Moreover, we can also benefit from
237
238
239
240
the use of hardware accelerators. We set the inter-observation time as T 1 and employed the Fuler- Maruyama integrator with a stepsize of 0.02 for all examples. Our results are not sensitive to the choice of T and discretization stepsize if it is sufficiently small. We examined the performance of each particle filter by computing its effective sample size (ESS) averaged over observation times and independent repetitions, the evidence lower bound (ELBO) Plyo...yy....................... yK)], ]. and the variance Var[log p(yl..... yk)], where p(yl...... Yk) denotes its unbiased estimator of the marginal likelihood of the time-discretized filter p(y1. .......... ..... .KK). When testing particle filters with varying number of observations K, we increased the number of particles linearly with to keep marginal likelihood estimators stable [2].
242
244
246
248
250
4.1
-aattinn-Uhleebeckk model
251
process given by (I) with p(x) -x. ox) = and the Gaus- sian observation model g (xx, = (y; ). ). We chose TΙX = (0, 1/2) as the stationary distri- 252
We considered an
253
bution and 7/y = =N 1/2 + x as the implied distribution of the observation when training neural networks with the CDT iterative scheme. We took different values of ay E {0.125, 0.25, 0.5, 1.0}
254
255
to vary the informativeness of observations. Analytically tractability in this example allows us to visualize the quality of our neural network approximations in Figure and consider two idealized particle filters, namely an APF with exact networks (Exact-APF) and the FA-APE. Comparing our proposed Iterative-APF to Exact-APF and FA-APF enables us to distinguish between neural network approximation errors and time-discretization errors. We note that all PFs except the FA-APF involve time-discren diserrcizztti.nnnn
256
258
280
281
Columns 1 to 4 of Figure 2 summarize our numerical findings when filtering simulated observations from the model. We see that the performance of BPF deteriorates as the observations become more informative, which is to be expected. Furthermore, when dy is small, the impact of our neural network approximation and discretttttiiatonn becomes more noticeable. For the values of σy and
262
263
264
285
the number of observations K considered, we obtained around an order of magnitude gain in efficiency
286
over BPF. From Column we note that these gains become very substantial when we filter K 100 observations that are simulated with observation noise that are several standard deviations larger than
268
oY= = 0.25 under the model specification. In particular, while the ELBO of BPF diverges as we increase the degree of noise in the simulated observations, the ELBO of Iterative- APF remains stable.
7

---

t- 0.25
Ł- 0.5
0.75
15
65-
-15-
62
22-
1.
1.5
24
65-
05-
-4-
-14-
(a) o(x, y. (lefi) and Vo(x. y) (right).
(b) c, (x, y, (upper) and v.t) (lower).
Figure 1: Neural network approximations for Omstein-UUlenbeed model with σ. 0.5.
Gy
0.125
Gy
0y
0.5
QY
105
102
655%
25-
25-
25-
25-
Variance ELBO gap
10
10'
15
1.0
800
3.0
207
10
105
10
Standard deviation
Figure 2: Results for Ornstein- -hleeben- model based on 100 independent repetitions of each PF. The ELBO gap in the second row is relative to FA-APF.
270
4.2
Logistic diffusion model
271
Next we consider a logistic diffusion process [11, 26] to model the dynamics of a population size 1220 defined by
dP. = 003/2+01-00
dl 0gP,
(22)
273
We apply the Lamperti transformation X, log (P.)(P))//)))) //03 3 and work with the process {X+ that satisfies (1) with pxx) = 0g /0, (02/0z) exp and (xx) = 1. Following [26], we adopt a negative binomial observation model g(x y) AB Oa, exp(6xx)) for counts y N, with dispersion 04 > ( and mean exp(fgx). We set (01, oz 0g 04) as the parameter estimates obtained in [26]. Noting that (22) admits Gamma distribution with shape parameter 2 /2 + 81)/63
274
275
276
277
and rate parameter 202/0g as stationary distribution [11]. we select v/x the push-forward under Lamperti transformation and v/y as the implied distribution of the observation when training neural networks under both static and iterative CDT schemes. To induce varying levels of informative observations, we considered 0₃ {1.069, 4.303, 17.631, 78.161].
279
281
282
Figure displays our filtering results for various number of simulated observations from the model (Columns 1 4) and for K 100 observations that are simulated with an observation model with several standard deviations larger than 04 = 17.631 under the model specification (Column 5). In the latter setup, we solved for different values of 6g the negative binomial observation model to induce larger standard deviations. The behaviour of BPF and Iterative-APF is similar to the previous example as the observations become more informative with larger values 04. Iterative-ADD APF outperforms both BPF and Static-APF over all combinations of 04 and K considered, and also when filtering observations that are increasingly extreme under the model. We note also that the APFs trained using
283
285
287
289
the CDT static scheme sometimes give unstable results, particularly in challenging scenarios.

---

8.
1.069
4.303
8, 17.631
78.161
102
100
200
100
E55%
20-
Variance ELBO gap
3.00
109
300
10
8.75
6.5
1000
000
200-
20n
201
104
L04
100
101
10
100
101
101
127
20
100
309
K
K
Standard deviation
BPF
-
Static
Iterative APF
Figure 3: Results for logistic diffusion model based on 100 independent repetitions of each PF. The ELBO gap the second row relative to Iterative-APr:
Qy
σy 1.0
102
100
E55%
Variance ELBO g90
127
10.070
103
30
20'
7.5
3.5
2002
0.00
100
201
120
20
000
100
10
Standard deviation
Static-APF
Iterative APF
Figure 4: Results for cell model based on 100 independent repetitions of each PF. The ELBO gap in the second row is relative to Iterative-APF.
291
4.3 Cell model
Lastly, examine cell differentiation and development model from [39]. Cellular expression levels X, = (X,,, x2,2) of two genes are modelled by (1) with
293
m(x)
x + +x)+))+22--
X1 XX2
(23)
and (x) VO.Ida. The terms (23) describe self-activation, mutual inhibition and inactivation respectively, and the volatility captures intrinsic and external fluctuations. We initialize the diffussion
295
296
process from the undifferentiated state of X. (1, 1) and consider the Gaussian observation model
297
g(x, y) Wyy; I2). To train neural networks under both static and iterative CDT schemes, we selected 7x and 7/y as the empirical distributions obtained by simulating states and observations from the model for 2000 time units.
298
numerical results for various number of observations K and ay (0.25, 0.5, 1.0, 2.0}- It shows that Iterative-APF offers significant gains over BPF and Static-APF when filtering observations that are informative (see Columns 1 to 4) and highly extreme under the model specification of øy = 0.5 (see Column 5). In this example, Static-APF did not exhibit any unstable behaviour and its performance lies somewhere in between BPF and Iterative-APF.
Figure 4 illustrates
301
302
303
304
References
305
308
[1] C. Beck, A. Jentzen, et al. Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations. Journal of Nonlinear Science, 1543:1563-1619 2019.
307
309
[2] Bérard, P. Del Moral, and Doucet. lognormal central limit theorem for particle approximations of normalizing constants. Electronic Journal of Probability, 19:1-28, 2014.

---

311
[3] A. Beskos, O. Papaspiliopoulos, and G. O. Roberts. Retrospective exact simulation of diffusion sample paths with applications. Bernoulli, 1276) 2006.
312
313
[4] A. Beskos, O. Papaspiliopoulos, G. O. Roberts, and P. Fearnhead. Exact and computationally ef- ficient likelihood -bsed estimation for discretely observed diffusion processes (with discussion). Journal of the Royal Statistical Society: Series B (Statistical Methodology), 68(3):333-382, 2006.
315
316
317
[5] Blanchet and F. Zhang. Exact simulation for multivariate Itô diffusions. Advances Applied Probability, 52(4):1003-1053 -004,, 2020.
318
[6] Q. Chan-Wai-Name Wai-Nam, J. Mikael, and X. Warin. Machine learning for semi linear PDEs. Journal of Scientific Computing. 1677:1167711712,, 2019.
320
321
[7] N. Chopin, O. Papaspilioppolos, et al. An introduction to sequential Monte Carlo. Springer, 2020.
322
323
[8]
K. L. Chung and J. B. Walsh. Markov processes, Brownian motion, and time symmetry, volume 249. Springer Science & Business Media, 2006.
325
[9] Del Moral. Feynman-Kac formulae: genealogical and interacting particle systems with applications, volume 88. Springer, 2004.
326
[10] P. Del Moral and L. M. Murray, Sequential Monte Carlo with highly informative observations. SIAM/ASA Journal on Uncertainty Quantification, 3(1)999--997, 2015.
328
329
[1]] B. Dennis and R. F. Costantino. Analysis of steady-state populations with the gamma abundance model: application to Tribolium. Ecology, 69(4): 1200-1213, 1988.
330
331
[12] A. Doucet, A. M. Johansen, et al. A tutorial on particle filtering and smoothing: Fifteen years later. Handbook of nonlinear filterin,, 12(656-704):3, 2009.
332
338
[13] P. Fearnhead, O. Papaspiliopoulos, and G. O. Roberts. Particle filters for partially observed
334
diffusions. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 70(4): 755 755-777,22 2008
335
336
[14] Fearnhead, O. Papaspiliopoulos, G. O. Roberts, and A. Stuart. Handom-weight particle filtering of continuous time processes. Journal of the Royal Statistical Society: Series B (Statistical Methodology). 72(4):497-51,, 2010.
337
338
339
[15] M. Gerber, N. Chopin, and N. Whiteley. Negative association, ordering and convergence of resampling methods. The Annals of Statistics, 47(4):2236-22) 2260, 2019.
341
[16] I. V. Girsanov. On transforming certain class of stochastic processes by absolutely continuous substitution of measures. Theory of Probability & Its Applications, 5(3):285-301, 1960.
342
parabooi [17] J. Han, A. Jentzen, et al. Deep learning- based numerical methods for high-dimensiooal parabolic partial differential equations and backward stochastic differential equations. Communications in Mathematics and Statistics, 5(4):349-33))) 380,08800000 2017.
344
346
[18] J. Han, A. Jentzen, and E. Weinan. Solving high-dimensional partial differential equations using deep learning. Proceedings of the National Academy of Sciences, (15(34):8505-85111 8510, 2018.
347
[19] C. Hartmann, L. Richter, C. Schütte, and W. Zhang. Variational characterization of free energy: Theory and algorithms. Entropy, 19(11):626, 2017.
349
350
[0 C. Hartmann, O. Kebiri, I.. Neureither, and I., Richter. Variational approach to rare event
351
simulation using least-squares regression. Chaos: An Interdisciplinary Journal of Nonlinear Science (966)0063107 2019.
352
[21] C. Huré, H. Pham, and X. Warin. Deep backward schemes for high-dimensional nonlinear PDEs. Mathematics of Computation, 1977217 2020.
354
355
[22] M. Hutzenthaler and T. Kruse. Multilevel Picard approximations of high-dimensional semilinear parabolic differential equations with gradient-dependent nonlinearities. SIAM Journal on Numerical Analysis, 58(2):929-961, 2020.
356
357
[23] M. Hutzenthaler, A. Jentzen, T. Kruse, T. Anh Nguyen, and P. von Wurstemberger. Overcoming the curse of dimensionality in the numerical approximation of semilinear parabolic partial differential equations. Proceedings of the Royal Society A, 47662244):20196630, 2020.
359
10

---

361
[24] O. Kebiri, L. Neureither, and C. Hartmann. Adaptive importance sampling with forward- backward stochastic differential equations. International workshop on Stochastic Dynamics out of Equilibrium, pages 261-281. Springer, 2017.
362
364
[25] P. E. Kloeden and E. Platen. Stochastic differential equations. In Numerical Solution of Stochastic Differential Equations, pages 103-160. Springer, 1992.
365
[26] J. Knape and P. De Valpine. Fitting complex population models by combining particle filters with Markov chain Monte Carlo. Ecology, 93(2):256-263, 2012.
367
368
[27] B. Oksendal. Stochastic differential equations: an introduction with applications. Springer
389
Science & Business Media, 2013.
370
[88] E. Pardoux and S. Peng. Adapted solution of a backward stochastic differential equation. Systems & Control Letters, 14(1):55-61, 1990.
372
[29] E. Pardoux and S. Peng. Backward stochastic differential equations and quasilinear parabolic partial differential equations. In Stochastic partial differential equations and their applications, pages 200-217. Springer, 1992.
373
374
375
[30] E. Pardoux and S. Tang.
barkward stochastic differential equations and quasilinear parabolic PDEs. Probability Theory and Related Fields, 14(2):123-150, 1999.
378
[31] J. Park and E. L. lonides. Inference on high-dimensional implicit dynamic models using guided intermediate resampling filter. Statistics and Computing, 30(5):1497-1522, 2020.
378
379
[32 M. Pereira, Z. Wang, Exarchos, and E. A. Theodorou. Learning deep stochastic optimal 2019.
380
control policies using forward-backward SDEs. arXiv preprint
381
[33] M. K. Pitt and N. Shephard. Filtering via simulation: Auxiliary particle filters. Journal of the
American statistical association, 94(446):590-577) 599, 1999.
383
[34] M. Raissi. forward-backward stochastic neural networks: Deep learning of high-dimensional
384
partial differential equations. arXiv preprint arxiv: 1e..1804.001111 2018.
[35] G. O. Roberts and O. Stramer. On inference for partially observed nonlinear diffusion models
386
using the Metropolis -astings algorithm. Biometrika, 88(3):003-621 2001.
387
[36] L. C. G. Rogers and D. Williams. Diffusions, Markov processes and Martingales: Volume 2:
388
Itô Calculus, volume 2. Cambridge university press, 2000.
389
[37] H. Sørensen. Parametric inference for diffusion processes observed at discrete points in time: a survey. International Statistical Review, 72(3):337-355 2004.
[38] S. Thijssen and H. Kappen. Path integral control and state-dependent feedback. Physical Review
392
E, 91(3):032044 2015.
[39] J. Wang, K. Zhang, L. Xu, and E. Wang. Quantifying the Waddington landscape and biological paths for development and differentiation. Proceedings of the National Academy of Sciences, 108520):8277-822, 2011.
394
395
[40] J. Yong and X. Y. Zhou. Stochastic controls: Hamiltonian systems and HJB equations, vol- ume 43. Springer Science & Business Media, 1999.
397
Checklist
398
399
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
401
contributions and scope? [Yes] See Introduction Section
402
(b) Did you describe the limitations of your work? [Yes] See Introduction Section
(c) Did you discuss any potential negative societal impacts of your work? [No]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
405
406
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
408
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments...
11

---

410
(a) Did you include the code, data, and instructions needed to reproduce the main ex- perimental results (either in the supplemental material or as URL)? [Yes] A Py- Torch implementation to reproduce our numerical experiments is available at https:
411
418
/ / anooymous.... . 4open. science/r/Connnneeeeeeeeeeee Cotae//////oooooorrreffoooo
414
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Experiment Section
416
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [N/A]
417
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] CPU and compute time reported
419
420
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [N/A]
422
(b) Did you mention the license of the assets? [N/A]
423
(c) Did you include any new assets either the supplemental material or as a URL? [N/A]
424
(d) Did you discuss whether and how consent was obtained from people whose data you're re using/curating? [N/A]
426
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
427
428
5.
If you used crowdsourcing or conducted research with human subjects...
429
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you deseribe any potential participant risks, with links Institutional Review Board (IRB) approvals, applicable? [N/A]
432
433
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
434
12


Reviews:
[{"review_id": "z3-ecEfTXMP", "reviewer": "Reviewer_AjSG", "summary": "This paper develops an approximation to the locally optimal particle filter in discretely observed diffusion processes under the assumption of a constant measurement rate. ", "questions": "\nQ1 - The difference between CDT static scheme and bootstrap proposals needs to be elucidated. Choosing c = 0 yield the original SDE dynamics, i.e. bootstrap proposals, no? ", "limitations": "Its fine. ", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "S1 - The method compares favourably to baseline solutions. \n\nW1 - Experiments are not entirely convincing. More specifically, only low-dimensional examples re considered (d <= 2), which undermines the rationale for their method in that we are not in a high-dimensional regime where numerical  solutions of PDEs become infeasible (paragraph 1 of sec 2.5). \n\nW2 - The method relies on fixed SDE dynamics so is not applicable to parameter inference. \n\nW3 - The method relies on constant sample rate, which is many times infeasible in practice due to e.g. missing measurements. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "fNvcdNKRZS", "reviewer": "Reviewer_q4F1", "summary": "The paper presents a novel method for online filtering where the latent process has a higher frequency than the observations. The authors define an algorithm, Computational Doob’s h-Transform, for iteratively learning a control function over the latent space and an observation to drive a stochastic process to a specific point at time T. The learned control function is then later applied in online filtering, where no further learning is required but the controlled dynamics can be applied to compute the particle filtering weights and to propagate the particles. CDT is applied to low-dimensional settings in the experiments section, where it is compared to bootstrap particle filtering for nonlinear dynamics, and to the exact analytical solution when available.", "questions": "Questions\n1. On the iterative method: how many iterations of the iterative method were required until satisfactory results/convergence? Can you \n      point to any theoretical results or give heuristics on the behaviour of the learned control through the iterative method? \n2. On dimensionality: How does the method scale when the dimensionality of the problem is changed? I’m concerned of the learning of \n       the control function, and how representative is the sampling scheme for observations during training when the problem is high- \n      dimensional\n3.  On experiments; Do they present a fair comparison? Is there some more recent work than bootstrap filters you could compare to in the \n        experiment settings where no analytical solution is available?\n4.  On training of neural networks; Did training the initial value and control networks simultaneously present any issues? \t\n5.   On the loss function: the loss function considers only the final state of the value function. Was no regularization needed on the \n         dynamics? Why, and are there some cases where it would be beneficial?\n6. On the observations: There is a comment in the experiments section (Line 242) that the results are not sensitive to choice of T as long \n        as the discretization has a small enough step size. What is small enough? A heuristic explanation/plot showing how long it takes the \n       control to steer to the observation would have been informative.\n7.   On clarity of text: (minor) some parts of the text had, to my knowledge, a number of typos and issues with grammar and notation. In \n      addition, there was some vague phrasing. Please consider going through the following lines\n\nLines to check:\nLine 13-14: applications listed, but no references provided\nLine 21: “is collected” while referring to multiple observations\nLine 29-30: “the goal of the article is…” This sentence felt too vague on my initial read\nLine 33: “these quantities” which quantities are you referring to here? Only the control function has been referred to so far \nLine 37: Efficient in what sense? \nLine 44: “an homogenous” should be “a”?\nLine 68: comma before while\nEquation 4: why p_T? It is confusing to the reader in the case k> 1. The notation makes sense after reading 88-90, but this equation is before that\nLine 136: “Ito lemma” should be “Ito’s lemma”\nLine 143: Period missing from the last sentence of the paragraph.\nEquation 15: t is used as time on the left-hand size, but it is the integrated variable on the right-hand side\nLine 217: why is the control evaluated at t-t_k?\nLine 238: odd to say that all experiments had 10 observations, when nearly all experiments vary K?\nLine 248: why was it required to grow the number of samples when the number of observations grows? Does this mean that when K=1000, there were 100k samples (since there were 1k samples for K=10)\nLine 255: “Analytically tractability” \nFigure 1: the figure text was quite vague, I didn’t get what I should be concluding based on the plots? I can see that they somewhat match, but was there some additional insight I’m missing here?\nFigure 2: The y-axis labels slightly overlap with the subplot borders\n", "limitations": "1. Societal impact of the work was not discussed. As the paper is presenting theoretical work, it does not require a deeper investigation into such matters. \n2.  Scalability to higher dimensional problems and convergence of the iterative method were not sufficiently discussed or empirically tested, even though scalability to high-dimensional problems was explicitly stated as a motivating factor for not using standard numerical methods for solving PDEs (Line 126-128)\n", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n1. An original method for learning a control function for particle filtering using a Doob’s h-transform\n2. Separation of learning the control and the filtering itself is an attractive idea for improving efficiency \n3. The work is reproducible (the code was provided and I was able to run it) \n\nWeaknesses\n1. Limitations and related work were not discussed enough, the paper would have benefited from a well-structured Conclusions/Discussion \n     section at the end instead of scattered remarks. It was not clear to me how the method presented compares to earlier work. \n2. While I agree that covering the theory in detail is well justified, having three pages of background section would have been more \n     comfortable to read if there was more text motivating why the topics covered are important or intuitive/heuristic explanations. \n3.   The convergence properties of the iterative scheme were not studied empirically or theoretically, I was not able to develop an intuition \n     into how the control behaves at iteration 1 versus iteration N.\n4.   Low dimensional experiments, only comparisons to a simple baseline method (bootstrap particle filter) and not to more recent work\n5.  The figures were focused on presenting error metrics, the behaviour of the learned control or the filtering distribution were hardly \n      visualized\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Kv7svwMjUCm", "reviewer": "Reviewer_qRy1", "summary": "Disclaimer: I am not expert in SDE theory so my review will focus on the high level and clarity.\n\nThe authors are interested in the filtering problem in SDEs, i.e. computing the conditional distributions x_t|y<t, with x the latent process and y the observations.\nThe focus is on particule filtering, where the filtering distribution is approximated via discrete particules.\nIn particule filtering, a guiding function q or proposal is used to propagate the values of the particules in time.\nA convenient guide is the 'Fully Adapted Auxiliary Particle Filter' which samples from the conditional p(x_t+1|y_t+1, x_t).\n\nDoob's theorem happens to give an expression for the SDE corresponding to this conditional p(x_t+1|y_t+1, x_t), which shares the same\ndiffusion as the a priori SDE and whose drift function is equal to the prior drift function plus an offest, or control function. \nSampling from this SDE for starting point x_t would give the desired sample of p(x_t+1|y_t+1, x_t) for the purpose of particule filtering\n\nThe authors propose an algorithm to find learn this control function c, without having to have to first solve a PDE first.\nThis function and the prior density is parameterized by neural network.\nAn objective is proposed to learn the control function.\nAn algorithm is proposed to optimize this objective.\n\nThe algorithm is evaluated to check the validity of the learned control in toy settings and in Particle filtering scenarios.\n", "questions": "Some comments / questions:\n\ntwo related comments first\n* eta_x should depend on the conditioning end observation. \nIf you drop this dependency, you are not sampling from the conditioned SDE. \nIn your experiment (L252), you drop this dependency. Is this a typo, a design choice, an approximation?\n\n* L159: \"In practice, we will let the three sources of randomness be independent of each other.\"\nWhat is lost with that approximation (is it an approximation)? If you choose T large, then the process reverts to the prior (independent of Y), stationary distribution of the uncontrolled SDE.\nIs it what you do?\n\n* The result section, shows that the learned control matches the exact control on a linear SDE scenario.\nMany details are missing, an appendix with those would be great. \nas a reader, I don't want to have to fill all the gaps. Even less as a reviewer.\nFor example, for OU, you could give the exact expression for h, and v.\n\n* The proposed objective seems to contain the control on both side of the square loss, the V_T and the log g(..).\nI do not really understand or see why the optimal control is a minimizer of objective.\nIt also feels like hitting a moving target.\n\n* It would be good to have a discussion of this optimization problem.\nFor example, how do the dynamics of learning look like?\n\n* Could you be more precise on which variables you are taking expectations over and why?\nThis is important for the loss.\n\n* Also could you give a justification for the 'stop gradient' comment?\nYou have a loss that has multiple dependence on c and you decide to only propagate through one part.\n\n* CDT algorithm, it would help to have a more detailed pseudo code for the algorithm.\n\n* Doob's transform: I suggest you either give more details or remove the sketch from the main paper. As is it is not useful.\n\n* Where is the conclusion? It is good to end a paper with the conclusion summarizing the method and results.\n\n", "limitations": "None", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "I find the proposed manuscript is well motivated.\nThe background on particule filtering is clear.\n\nMy SDE theory is less developed but I don't understand section 2.5,\nnot even at a high level. As a result I m not sure I trust the content of section 3.1 (the loss and algorithm)\nI m possibly not the best reviewer (hence the low confidence) \nbut I also think clarity could be greatly improved (see below).\n\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "3nOz-49IrCT", "reviewer": "Reviewer_mpC8", "summary": "The paper addresses the task of online filtering for discretely observed nonlinear diffusion approaches. The paper proposes a method based on the fully adapted auxiliary particle filter (FA-APF). The paper discusses how a control function can be used to construct a controlled diffusion that takes into account a future observation. The desired control function is related to Doob’s h-transform and can be obtained by solving the backward Kolmogorov equations. This is computationally intractable, so the proposed strategy is to work with the value function (the negative log of h). The optimal control can then be expressed in terms of the value function and approximated with neural networks. These networks can be trained by simulating the controlled diffusion trajectories. After training, the online filtering can be performed using the approximately optimal control. \n\nThe paper presents numerical results for three models: an Ornstein-Uhlenbeck model, a logistic diffusion model, and a cell model. The results are compared to a bootstrap particle filter. For the Ornstein-Uhlenbeck model, which is analytically tractable, there is also comparison with the exact APF. There superiority of the proposed method is clear in all three examples. \n", "questions": "1.\tHow many particles were used in the experiments (or what was the scaling factor with K)? Why was this value chosen? What is the sensitivity to this choice? Is it possible to achieve improved performance with more particles or is the computational burden already considerable, i.e. if I were trying to implement a real-time system with a reasonably powerful computer, with a reasonable time between observations, how many particles could I use?\n\n2.\tWhat is the computational overhead? How does the computation of the proposed method compare to the BPF? What are the key contributors to the computational burden? \n\n3.\tWhat are the state-of-the-art competing techniques? Why are these not considered in a quantitative performance comparison? Are methods like [R1] inapplicable for the studied settings? \n", "limitations": "Yes.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n1.\tThe paper addresses the important problem of online inference for diffusion processes (although it doesn’t make a compelling case as to why this problem is important) and provides an elegant algorithm for performing the filtering. To the best of my knowledge, this represents a novel approach to solving this problem and I would rate it as a highly original approach. I think the work could have a significant impact on research addressing online inference for diffusion processes.  \n\n2.\tThe paper is very well written. The material is relatively dense, but the authors do an excellent job of providing a clear description of the algorithm. It is developed step-by-step in an intuitive manner. \n\n\nWeaknesses\n1.\tRelated work and performance comparison: there is very little discussion of existing methods for filtering partially observed diffusions. Multiple methods are acknowledged (and dismissed) with the single sentence “Specialized methodologies have been developed to circumvent or mitigate these issues.” Other methods like [R1] are ignored. \nEven if some of the methods are more limited in their applicability, there should have been an effort to compare with them experimentally for examples where they are applicable. Some discussion of the limitations of existing work should be included in the paper. \n\n2.\tMotivation: The paper does not provide a compelling motivation for the analyzed problem. There are very general claims about the ubiquity of diffusion processes in the first few sentences but no citations to any literature to support these claims. There are no citations of work providing more practical examples of where the proposed filtering problem would prove beneficial. The studied examples, while providing a good illustration of the benefits of the technique, do not provide compelling evidence of the practical utility. The cell model and the logistic diffusion model for population size dynamics do not seem suited to a meaningful, practical filtering task.\n\n3.\tExperimental analysis: There is no performance comparison to any methods except for the basic bootstrap particle filter. There have been multiple approaches to address the filtering task, so there should either be a clear explanation as to why existing approaches are inapplicable or other methods should be included. I could not find the details of exactly how many particles are used in the simulations (apart from the specification that it increases linearly with K). This seems to be an important design choice and there should be an explanation as to how the value was selected. Preferably there should be an analysis of how performance changes with varying K. There is no reporting of computational requirements. It is not clear if the proposed filter and the BPF have the same (or very similar) computations. Presumably there are additional neural network evaluations for the proposed scheme, but these may be negligible compared to other computational overhead. If they are non-negligible, then a fairer comparison would use more particles for the BPF. \n \n[R1] Jasra, A., Law, K. J., & Yu, F. (2020). Unbiased filtering of a class of partially observed diffusions. Advances in Applied Probability, 1-27.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Reject