Variational inference for diffusion modulated Cox processes
Authors: Prateek Jaiswal
Harsha Honnappa
Vinayak Rao
Venue:  (2021)
Keywords: Cox process
variational inference
stochastic differential equation
smoothing posterior density

Abstract: This paper proposes a stochastic variational inference (SVI) method for computing an approximate posterior path measure of a Cox process. These processes are widely used in natural and physical sciences, engineering and operations research, and represent a non-trivial model of a wide array of phenomena. In our work, we model the stochastic intensity as the solution of a diffusion stochastic differential equation (SDE), and our objective is to infer the posterior, or smoothing, measure over the paths given Poisson process realizations. We first derive a system of stochastic partial differential equations (SPDE) for the pathwise smoothing posterior density function, a non-trivial result, since the standard solution of SPDEs typically involves an It\^o stochastic integral, which is not defined pathwise. Next, we propose an SVI approach to approximating the solution of the system. We parametrize the class of approximate smoothing posteriors using a neural network, derive a lower bound on the evidence of the observed point process sample-path, and optimize the lower bound using stochastic gradient descent (SGD). We demonstrate the efficacy of our method on both synthetic and real-world problems, and demonstrate the advantage of the neural network solution over standard numerical solvers.

Under review as a conference paper at ICLR 2021
VARIATIONAL INFERENCE FOR DIFFUSION MODU- LATED COx PROCESSES
Anonymous authors Paper under double-blind review
ABSTRACT
This paper proposes a stochastic variational inference (SVI) method for comput- ing an approximate posterior path measure of a Cox process. These processes are widely used in natural and physical sciences, engineering and operations research, and represent a non-trivial model of a wide array of phenomena. our work, we model the stochastic intensity the solution of a diffusion stochastic differential equation (SDE), and our objective to infer the posterior, or smoothing, mea- sure over the paths given Poisson process realizations. We first derive system of stochastic partial differential equations (SPDE) for the pathwise smoothing pos- terior density function, non-trivial result, since the standard solution of SPDEs typically involves an Itô stochastic integral, which is not defined pathwise. Next, we propose an SVI approach to approximating the solution of the system. We parametrize the class of approximate smoothing posteriors using neural net- work, derive lower bound on the evidence of the observed point process sample- path, and optimize the lower bound using stochastic gradient descent (SGD). We demonstrate the efficacy of our method on both synthetic and real-woris prob- lems, and demonstrate the advantage of the neural network solution over standard numerical solvers.
INTRODUCTION
Cox processess (Cox) 1955 Cox Isham 1980). also known as doubly-stochastic Poisson processes, are a class of stochastic point processes wherein the point intensity is itself stochastic and, condi- tional on a realization of the intensity process, the number points in any subset of space Poisson distributed. These processes are widely used in the natural and physical sciences, engineering and operations research, and form useful models of wide array of phenomena.
We model the intensity by diffusion process that is the solution of stochastic differential equa- tion (SDE). This is a standard assumption across range of applications (Susemihl et al., 201T] Kutschireiter et al. 2020). The measure induced by the solution of the SDE serves prior mea- sure over sample paths, and our objective is to infer a posterior measure over the paths of the under- lying intensity process, given realizations of the Poisson point process observations over fixed time horizon. This type of inference problem has been studied in the Bayesian filtering literature (Schup- pen 1977 Bain & Crisan 2008 Särkkä 2013), where is of particular interest infer the state of the intensity process at any past time given all count observations till the present time instant (the resulting posterior is called the smoothing posterior measure).
In a seminal paper, Snyder 1972) derived a stochastic partial differential equation (SPDE) describ- ing the dynamics of the corresponding posterior density for Cox processes. The solution of this smoothing SPDE requires the computation of an Itō stochastic integral with respect to the counting process. It has long been recognized (Clark 1978 Davis 1981 1982) that for stochastic smoothing (and filtering) theory to be useful in practice, it should be possible to compute smoothing posteriors conditioned on a single observed sample path. However, Itô integrals are not defined pathwise and deriving pathwise smoothing density is remarkably hard. 30 years after Synder's original work EI- liott & Malcolm (2005) derived a pathwise smoothing SPDE in the form a coupled system of forward and backward pathwise SPDEs. Nonetheless, solving the system of pathwise SPDEs, or well known, for example, that numerical techniques for solving these SPDEs, such the finite element
sampling from the corresponding SDE, is still challenging and intractable in general.

---

Under review as a conference paper at ICLR 2021
method (FEM), suffers from the curse of dimensionality (Han et al. 2018). Therefore, it is of con- siderable interest to find more efficient methods to solve the smoothing SPDE.
We take a variational inference approach to computing an approximate smoothing posterior measure. Variational representations of Bayesian posteriors in stochastic filtering and smoothing theory have been developed in considerable generality; There are a number of papers that consider the computation of an approximate posterior distribution over the paths of the underlying intensity process that is observed with additive Gaussian noise (Ar- chambeau et al. 2007| 2008 CSeke et al. 2013) [usemihl et al. 2011; [Sttter et al. 2016). Susemihh ct al. (2011) studied Bayesian filtering of Gaussian processes by deriving a differential equation characterizing the evolution of the equare eπror (MSE) in estimating the underlying Gaussian process. On the other hand, Sutter et al. (2016) compute a variational approximation to the smooth- ing posterior density when the underlying diffusion intensity is observed with additive Brownian noise. They choose their variational family to be a class of SDEs with an analytically computable marginal density. This setting is considerably different from our setting, where the observed process is a point process. Nonetheless, Sutter et al. (2016) provides methodological motivation for our cur- rent study. In the context of the computation of approximate amoothing/flttering posteriors for point process observations, Harel et al. (2015) developed an analytically tractable approximation to the filtering posterior distribution of diffusion modulated marked point processes under specific mod- eling assumptions suited for a neural incoding/decoding problem. In general, however, analytical tractability cannot be assured without restrictive assumptions.
(Mitter & Newton 2003) for a rigorous treatment.
We present a stochastic variational inference (SVI) (Hoffman et al. 2013) method for computing variational approximation to the smoothing posterior density. Our approach fixes an approxi- mating family of path measures to those induced by a class of parametrized SPDEs. In particular, we parametrize the drift function of the approximating SPDEs by neural network with input and output variables matching the theoretical smoothing SPDE. Thereafter, using standard stochastic analysis tools we compute a tractable lower bound to the evidence of observing a sample path of count observations, the so-called evidence lower bound (ELBO). A sample average approximation (SAA) to the ELBO is further computed by simulating sample paths from the stochastic differen- tial equation (SDE) corresponding to the approximating SPDE. Finally, by maximizing the ELBO, the neural network is trained using stochastic gradient descent (SGD) utilizing multiple batches of sample paths of count observations. Note that each sample path of the count observations entails the simulation of a separate SDE. We note that there are many problems in the natural and physical sciences, engineering and operations research where multiple paths of point process (over finite time horizon) may be obtained. For instance, we present an example in Section [ modeling the demand for bikes rented during a 24 hour, one day time period in a bike-sharing platform, where the underlying driving intensity is subject to stochastic variations, and demand information is collected over multiple days.
In contrast to the variational algorithm developed in Sutter et al. (0166). where the variational lower bound must be re-optimized for new sample paths of the observation process, our variational method is more general and our approximation to the smoothing posterior can be used as map for another (unobserved) sample path of count observations. Our computational approach can also be straight- forwardly adapted to solve the problem of interest in (utter et al. (2016).
In the subsequent sections, we describe our problem and method in detail and demonstrate the util- ity of our method with the help of numerical experiments. In particular, we show how the choice of approximating family enables us to use the trained neural network and in turn, the variational Bayesian smoothing posterior (VBSP), to compute smoothing SPDE in almost (3/4)th h of the com- putational time required to compute the original smoothing SPDE using FEM. Moreover, we also efficiently generate Monte Carlo samples from the learned VBSP and use them for inference on the bike-sharing dataset, whereas FEM failed to compute either VBSP or the true smoothing density for the given time-space discretization.
2 PROBLEM DESCRIPTION
Let N₂ be a Cox process with unknown stochastic intensity {z, E RR [o, We use Ne. to denote sample path realization of N, restricted to the interval [t', t), and use N, to denote N, No recall that No 0 by definition. As noted before, a Cox process conditioned the intensity is
2

---

Under review as a conference paper at ICLR 2021
Poisson process. Therefore, given realized sample path {z,, [0, of the intensity, and for any I < f S T, the marginal likelihood of observing No - No E counts in (t', tt]
(%. 2ods)
No-N,.
e e- [. 2,ds
N - No L(N. Nr N; No Ne
(I)
NN)!
where L denotes the Poisson likelihood. Rather than directly modeling the intensity 2, will bring a little more flexibility to our setting, and assume that 2; is a deterministic transformation of an another stochastic process x, through known mapping h =I Rt: is 2, h((x)). Note that the non-negative range of ensures that the Poisson intensity ž₁ = h((;,) is non-negative. Unless a E R*+ the mapping h cannot be identity function. We use the term intensity process to refer to either 2, or Tt.
We model the intensity process {æx Rd, Vt [0, with the following SDE,
dx; b(zz)dtt (x) B,, Vt < T and Zo 0,
where b RR + Fd is the drift function, (-) Rd + Rd is the diffusion coefficient, and B, is the d- dimensiionnl Brownian motion (or Wiener process). We assume that there exists a strong solution to the SDE above (Oksendal, 2013 Chapter 5). Moreover,
assume that b(-), h(.), and (.)) are fixed by the modeler apriori, and we are interested in inferring the unknown intensity process with their fixed definitions. Incorporating them will obscure our main contribution, and we leave it for future work.
The model of the count observations above forms diffusionn modulated Cox process. Diffusion modulated Cox processes are widely used to model the arrival process in various service systems such as call centers, hospitals, airports etc. (Zhang al., 2014 Wang al. 2020). Zhang & Kou (2010) use Gaussian process modulated Cox- Cox-process to infer proteins' conformation, in particular, they model the arrival rates of the photons collected from a laser excited protein molecule as a Gaussian process. [Schnoerr et al. (2016) model spatio-temporal stochastic systems from systems biology and epidemiology using Cox process where intensity is modelled with diffusions.
As stated in the introduction, we seek to infer the smoothing posterior measure over the un- known intensity process (æ,, E T)} using the count observations upto time T. Follow- ing terminology from the Bayesian filtering theory (Särkkä 2013), we smoothing to refer to inferring the unobserved intensity process at any past time given the observations upto the cur- rent time. Mathematically, the smoothing posterior is defined using the conditional expectation of the form E[f(x;) (=x (N,, u E TT)], where s(N,, u E [0, TJ) is the smallest sigma alge- bra (or filtration) generated by the Cox process (N,) from time 0 T. For brevity write El f( (NN,, u E [0, T]] as (x:) Interested readers may refer to Kutschireiter et al. 2020 for more details on non-linear filtering theory.
We now provide a formal derivation of the smoothing posterior using Bayes' theorem (Bain & Crisan 2008 Elliott & Malcolm (2005)). Observe the conditional expectation satisfies
[ (x:) ES [Ao.T
for any measurable function f(-) and Ag,t C(N..)) for any 0 S <<t T, where L is the
(N..))
unit intensity Poisson likelihood and [+[-] denotes the expectation with respect to Lt. Note that L does not depend on the stochastic intensity process æ and forms a reference measure. The marginal smoothing posterior density is defined as
(u, == Pixx E dx No,r),
which can be formally obtained from equation 3 by setting f(x;) = 444 (2x)) for any A E RRd where I[ay(() (v) is an indicator function that equals when y E A, otherwise 0. Now, define the unnormalized filtering density function (x) as the function satisfying
qe qi
P(x, E dx) No.)) = Jau (E(E)
(5)
3

---

Under review as a conference paper at ICLR 2021
and also define (x)) := E [A,,r|NNo, No, Then, it can be shown (Elliott & Malcolm 2005 that for any measurable function f.
[lf(2)))Noor) ET [ Saa (E)))
Et [
Jza 9e(
Next, recalling that li)) is the mapping to ensure the intensity process is positive, define the function f for given sample path of count observations (i.e., pathwise)
M (((()), 1, No) = exp [[1 - h(x))t + N₂ log h(x)), Vx E Rd.
Following Elliott & Malcolm 20055 Theorem 4) one may use Y to derive coupled system of
pathwise SPDEs that characterize (x) and (x)). In particular, they show that ai v qe is solution to the following SPDE
(.q((a) = L' (4 (x)), VI & T, (z() 62)(2))
where L is the adjoint of [[F(x)] = 2/22 Ei,; (2)5(2)0... F(x) E, b,(x)),,, F(x), which the infinitesimal generator of the prior process for any wiceediifferentabble continuous, and bounded function F IRd + I and a(x) (x)) and 8z, (I) is the Dirac delta distribution at To- Moreover, they also show that (x)) = (xr) satisfies the following backward parabolic equation
(,) V, V, [.],,,(()))
(8)
with terminal condition (x)) v)(x..
Now it follows from equation (6 using the solution these two SPDEs, the marginal smoothing posterior density for any E [0, satisfies
(c(E)o((E)dE
(e Ju (E)))
Using the SPDEs in equation[7] and 8 together with 9 can be shown that the marginal smoothing posterior density Pe ( (o.) satisfies its own SPDE: for any E ().
(xpo(c/No.r) = Σe Oz, [(a(z)
vx()))) + (x))) (o
Σ (x) 2 2,
(10)
and (o())/oorr = 82n (x) with xo 0. We present detailed derivation in Appendix A.1. Corre- sponding to this SPDE, there exists a smoothing posterior SDE, defined as
dē, = a (.) V log( ", v, (ā.)) + b(æ,)) dt + o ( and in = 0,
(11)
where { is modification of the process {x,} such that B, is independent of the Cox process Nr (and thus B,).
Observe that the entire sample path of the count observations No, is summarized through the path- wise function F, and together in the drift term of this SDE. Also note that the diffusion coefficient of the smoothing posterior SDE precisely the same as that of the prior SDE, The computation of the drift term in the smoothing posterior SDE requires the solving equation g for vz(x) which, in turn, makes the posterior computation challenging and computationally intractable in general. Con- sequently, the computation of the marginal posterior density (and hence the path measure). There- fore, we propose variational inference-based method to compute an approximation to the solution of the smoothing posterior SPDE, by computing approximate solution to the smoothing posterior SDE in equation II
VARIATIONAL BAYES FOR APPROXIMATING THE SMOOTHING DENSITY
Observe that the posterior path measure is the minimizer of the following variational optimization problem Mitter & Newton 20033 Proposition 2.1)).
(L(II||LLo) + / dil(20,,)) log L(No, ((No,r
max ((P())))P((
(12)
4

---

Under review as a conference paper at ICLR 2021
where P(C) is the space of all absolutely continuous measures with respect to Hh. the measure induced by a solution of the intensity SDE (equation 2 on the space C(O, T) of continuous func- tions with support [T, and KL denotes the Kullback-Leible divergence between two absolutely continuous measures. Note that P(C) also contains oT.
Solving this optimization problem over all measures in P(C)) is intractable. Therefore, choose the subset of absolutely continuous measures 2 C PP(C) induced by solutions the following SDE:
dæ: -aa No, t) + (dx), for ' and an 0,
(13)
where : N [0, + Rd is the drift function. We term this space of measures O; the variational family. The measures in 23 are absolutely continuous with respect ΠT they have the same diffusion coefficient, therefore Q5 C P(C). This choice of the variational family is not arbitrary, but rather motivated by the smoothing posterior SDE derived in equation []] where the diffuuson coefficient is a-)) and the drift coefficient has an intractable form (that depends on the prior drift b(.) and diffusion coefficient (-), and N₂ through F and (-)).. Notice that the choice of drift function spans the space of measures in the variational family Qi-
Since, 2g PCC), it follows from equation 12 that
(II + / dl((oo))) log ((No,r) ((Nor)1((Z0x)))
max P)P(C)
2
max QEQs
(L((Q))T)) + |dQ(zaT)) log (No ((x0,)))
(14)
The right hand side above is known as the evidence lower bound (ELBO). The corresponding ELBO maximization problem compute the optimal Q 25 (for given sample path No,r) is simply
Q' (:1No,r) arg arg max Eq (log L(No, (orr) (L(Q)[I1).
(15)
QEQ;
Note that absolutely continuous measures on path space correspond to changes in the drift function, for fixed diffusion coefficient (else, the measures are singular). As consequence of Girsanov's ' theorem (Oksendal 2013 Theorem 8.6.8) (see Appendix A.2 for the proof) we have,
[
J' 10 (z=)()))(((z))) b(x,, Nr, +)
EQ (Q( = 2
(16)
where recall -) -) is the drift of the prior SDE defined in equation 2 and b(.,.,) is the drift of the variational SDE. Substituting this into equation 15 yields
Q" (=((NNo..)) = arg max EQ log C(No, - 1 -1/[" 110 (0, t) b(, ))] QEQ;
(17)
We denote Q" (-I.o.))) as the variational Bayesian smoothing posterior (VBSP) path measure Next, lay down the details of the SVI algorithm to solve the above optimization problem to compute the VBSP.
4 STOCHASTIC VARIATIONAL INFERENCE OF THE VBSP
It is evident from the ELBO in equation 17 and the definition of the variational family Og that com-
puting the VBSP measure entails the computation the unknown drift function 6(-.:-) ) in equa- tion 13 We further restrict the family of measures Q by assuming the drift functions belong to a class of parametrized, smooth functions. A feasible way to model this class drift functions is through neural network. However, it is possible to use simpler approximation function classes as done in, for example, Sutter et al. (2016), who fixed ensure that the marginal distributions of the variational smoothing SDE belong to specific exponential family of distributions. We note
5

---

Under review as a conference paper at ICLR 2021
that in choosing parametrized class, must still ensure that the resulting drift functions are Lip- schitz continuous and satisfy sufficient regularity so that a solution to the SDE equation 13 exists. Furthermore, restricting the drift functions in this way entails a further restriction of the class of (ap- proximating) path measures. An open question is here is how much of a loss this entails (in terms of the Kullback-Leibler divergence from the 'true" posterior path measure in this instance).
To fix the idea, we assume that b(:,..) lies in a general class of functions parametrized by 8. Henceforth, we write the drift coefficient b(:;,,0)) use stochastic gradient descent (SGD) to maximize the ELBO, which requires the computation of stochastic gradients of the ELBO with respect 8. To compute the gradients, we generate sample paths of x", the solution of the variational SDE equation for a given 6, using a first-order Euler- Maruyama integration of the SDE. We do this for convenience, though higher order approximations could be used. Specifically, we partition the time interval [0. 7] in K equal sub-intervals of length A T/ K, (1, .tk), where to and lg- T and then generate the sequence of {a. )=1 using the following recursive equation and initial condition x = 0:
make its dependence on 0 explicit. We
i=1
ii; =x + b(al, 1 Nr, 11 1:4--,0)4 + Ao Ao(a) IE (1,2,... K}
(18)
where {Z is the sequence of K independent and identically distributed (i.i.d) d-dimensional standard Gaussian random vectors.
We generate M independent sample paths of the discrete- -inrette--imeeeee process in equation 18 denoted as {a); (x);",, ), for E f1,2,... .), compute sample average approximation (SAA) of the ELBO over the partition {lo, 4,, (x) as
M
K
1
1 log C(N [Maa) (0 )) [|o((a?;")) "(((a",," ...) -ball"" Lc, 0]][
ELBO
M
1
M
Naw log(h(a?) h(m?;"") A ( 9.m (10m......))) (.) b(al."" 0.m NN, tx 0)]1 + +C,
M
(19)
where we used the definition of the Poisson likelihood L from equation I and C a constant independent of 0. Now, to compute gradients of ELBO with respect to 6, observe that the gradient operator can be exchanged with the expectation, since the only source of randomness in each sample path of a; are i.id. Gaussian random vectors { which are independent of 9. Notice too that this is pathwise analog of the reparametrization trick, and has been used recently to learn deep latent models (Tzen Raginsky 2019 Li et al. 2020). Also, note that thus far the ÉLBÖ is defined for single sample path of the count observations No,r. However, as noted before, in our method we will also take a sample average of ELBÖ over multiple batches of sample paths of count observations at each epoch of the training algorithm.
Ji=1.
5 NUMERICAL EXPERIMENTS
We present three experiments demonstrating the efficacy and utility of our proposed SVI method. First, we consider a setting when the underlying stochastic intensity process is 1-dimensional. We compare the SVI approximation with the 'true" smoothing posterior density computed using the solution of the forward and backward SPDEs, defined in equation [7] and equation g We solve the SPDEs using a standard finite element method (FEM) solver (Skeel Berzins 1990 Matlab solvers for 1-D PDEs). In a second experiment, we demonstrate the performance of our algorithm on sub- set of Bike-sharing dataset obtained from the UCI machine learning repository (Fanace-T & Gama 2013). In this experiment we estimate smoothing posterior density for the observed counts of the demand for bikes in a 24 hour period, assuming that the demand process is well-modeles by Cox process. In our third and final experiment, we apply our method compute approximation to 4- dimensional smoothing posterior density. We note that despite being low dimensional, the standard FEM solver does not scale this setting, while method can be straightforwardly adapted.

---

Under review as a conference paper at ICLR 2021
5.1 VARIATIONAL APPROXIMATION OF UNIVARIATE SMOOTHING POSTERIOR DENSITY
As defined in Section we are interested in learning the posterior measure over an unknown process far E ] where the intensity process {z,} satisfies 2; h((z)). We set b(z) = -I and a (z))
in the prior SDE as defined in equation 2 Furthermore, motivated from the mathematical structure
of the true smoothing SDE equation [1] we fix our variational family Q to be the class of measures induced by solutions to the class of SDEs in equation 13 with drift and diffusion coefficient set as
Vi V
b( N.) =
+ (ar, NT - N₂; - x,
( = 1,
where V is the derivative of V, with respect to x. Here V(z, t, Nr Ne; f) is modeled using a neural network with 2 hidden layers whose parameters we call @ (see Appendix A.3 for more details on the architecture).
5.1.1 SIMULATED DATASET
We generate sample paths of the count observation No.7 from non-homogeneous Poisson process,
where the intensity process {z(}} is the solution of the following ordinary differential equation,
dæ 20 t)
- +)?), and =8 = 0. We fix the map h(a) = * cxp(-.08 .08 * (a - 5)
di
this experiment. To train the neural network V, we use 150 samples paths of the count observation
between time interval [0, 2] and optimize ELBO in equation 19 using Adam (Kingma & Ba 2014).
To demonstrate the efficacy of our approach, we first generate 20 test sample paths of count obser- vations and compute the true smoothing posterior density (defined in equation (9) using the solution of the forward and backward SPDEs (defined in equation and equation [ reespectively) We do this using the FEM method. Then for the same test observations we compute the VBSP density by FEM using the trained drift coefficient of VBSP SDE (see equation TO 11).
Comparative results are presented in Figure [ are very similar each other, with our variational approximation capturing the sharp rises in density with high fidelity. Note from the first two plots in the bottom row that as the ELBO decreases the gap between the the true and VB smoothing posterior reduces too. Moreover, the time required to compute the smoothing density using the forward and backward SPDEs on the test data is about 2.2 seconds which is approximately fifty percent higher than the trained VBSP density, which required 1.6 seconds (on an 3.1 GHz Intel i5 CPU). This is due to the fact that we are required to solve one SPDE in the latter case instead of two in the former case.
We clearly see from the the top row plot that these
Notice that the learnt drift of the smoothing SDE equation 13 is a map which can be used with any sample path of count observations to generate Monte Carlo samples from the an approximate smoothing posterior density. In contrast, it is challenging to sample from the true smoothing SDE as it involves computing the solution of the system of SPDEs in equation 7 and Furthermore, this solution must be recomputed for each new sample path of count observations.
5.1.2 BIKE SHARING DATASET
In this experiment, we compute the VBSP density for the hourly counts of demand in a bike-sharing system. The experimental setting remains unaltered, albeit the diffusion coefficient is set to (z))
1.1, to capture the increased variability in counts of bike demand than the variability in simulated counts in the previous experiment. Notice that fixing the map h is a modeling question, and we consider mappings of the form h(x)) = a exp(-b(x - -))) parametrized by a, b, and c. We take a simple empirical Bayes heuristic to fix their values after observing the count observations, based on the fact that h(())) A is the mean of the Poisson counts in the interval A. We set in such a way that h(x)A cquals the maximum of the median observed count. For the Bike-sharing data we re-scaled 1050 as a. The
the problem to interval [0, 2] and fixed A = 0.083 and thus choose 90/0.083 choice of b and depends on the diffusion coefficient a (zz) and ze. the SDE should be able to explore the relevant domain of h to appropriately model the actual count observations. Thus, after looking at the count data, we chose h(z) = 1050 cxp(-0.001 (y 50)2).
The empirical results are summarized in Figure 2 We note here that the FEM approach (our imple- mentation) to compute the VBSP density was numerically unstable and failed; this may be attributed
7

---

Under review as a conference paper at ICLR 2021
Smoolerg Postaton Toe PON
0.0
04
04
60
LR.0.032
Tea--60
decasss boterme ane
Dours-00g 5m Do Beg
50
Tel: 10g
G R. 6: &. e
Adial
40
Counts
30
20
20
10 20 30
50
Epochs
Epocha
Figure 1: Variational vs. True smoothing posterior density on the 1-D simulated dataset. The top row compares the variational approximation with the true smoothing density. The bottom-left plot shows the ELBO as a function of epochs of the training algorithm. We also compute the L distance between the VB and true smoothing density over 20 sample paths of both training and test count observations as the training progresses and plot the 10th, 50th and 90th quantile in the bottom- middle figure. The bottom-right plot depicts that 95% of the test count observations within the 5th and 95th quantile of the simulated counts, where counts are simulated using the learned VBSP SDE for that test sample path of count observations.
the concentrated nature of the VBSP, as visible in the plots. It is evident from the last plot in Fig- ure 2 that for the current h, VBSP density captures the trend in the real count observations well, however we anticipate that a different choice of the map can better model the count observations.
Note that in smoothing problem, the prior intensity process (specifically the drift coefficient b(-), diffusion coefficient (-) and the map (u))) are sourced from an expert, and the objective is to update the modeler's 's beliefs with count observations to compute the smoothing posterior density. In many settings, the functions b(.), aT-) and (()) are known only up to some unknown parameters. It is fairly straightforward to combine the parameters of h and with the neural network parameters 0, and learn them all in a data-driven manner. We choose not to do this to keep the discussion simple. Learning the parameters of presents slightly greater challenge since the path measures for different settings of a are singular. This is crucial difference from the finite dimensional setting, where the Lebesgue measure is common reference. We leave solving this problem for future work.
5.2 VARIATIONAL APPROXIMATION OF MULTIVARIATE SMOOTHING POSTERIOR DENSITY
We demonstrate method on 4-dimensional smoothing problem. In this we fix the map h(a) = 25 * cxp(-.08 * ||a 2) where || the L-2 norm, and a E R.. We also choose the prior density to be induced by an SDE defined in equation with b(x:) where æ E I and I is a d x d identity matrix. Furthermore, we choose our variational family to be family of SDE defined in equation 13 with drift and diffusion coefficients, b(ar, 1, N.)
and (xx) I,
vV, (a 1, Ny N₂; 0) W.
and (xx)
To train the neural network V. interval [0. 2] and optimize the ELBO defined in equation 19 using Adam (Kingma & Bal 2014)
150 samples paths of the count observation between time
We plot the empirical results in Figure
8

---

Under review as a conference paper at ICLR 2021
Cavel
Courst
No.r of Day
8
0.0.
10 30
10
VBSP smoothing posterior density for Bike Sharing Data. The left plot shows the ELBO as the training progresses. The second and third plots depict the empirical VBSP density computed test sample path of count observations, using 1000 simulated sample paths of the learnt variational smoothing SDE. In the right plot, we use the same test sample path of count observation, compute the VBSP using the trained drift and demonstrate that most count observations (of the test sample path) ( 70%) lie within the 97.5th and 2.5 quantile of the simulated counts.
Figure 2:
VSS demty-
vaEP daratt-3
Marginal danalty-4
1.5
Marginal
12
Hour
140
Heu -10y
efD
10-0000
200
8
100
Figure 3: Multivariate VB smoothing posterior density: The first two rows of Figure 3 show the marginals of the (empirical) VBSP density using the trained drift coefficient for given test sample path of count observations. The bottom-lefi figure plots the ELBO value as function of the number of training epochs. Then, for test sample path of count observations, the right hand plot shows that more than 95% of these observations are within the 95% confidence interval of the simulated count observations generated from the trained 4-dimensional VBSP density computed for that test path.
REFERENCES
Cédric Archambeau, Dan Cornford, Manfred Opper, and John Shawe-Taylor. Gaussian process approximations of stochastic differential equations. Journal of machine learning research, 1: 1-16, 2007.
Cédric Archambeau, Manfred Opper, Yuan Shen, Dan Cornford, and John Shawe-Taylor. Varia- tional inference for diffusion processes. In Advances in Neural Information Processing Systems, pp. 17-24, 2008.
Alan Bain and Dan Crisan. Fundamentals of stochastic filtering, volume 60. Springer Science & Business Media, 2008.

---

Under review as a conference paper at ICLR 2021
JMC Clark. The design of robust approximations to the stochastic differential equations of nonlinear filtering. Communication systems and random process theory, 25:721-734, 1978.
David R Cox. Some statistical methods connected with series of events. Journal of the Royal Statistical Society: Series (Methodologicul), 17(2):12-157,, 1955.
David Roxbee Cox and Valerie Isham. Point processes, volume 12. CRC Press, 1980.
Botond Cseke, Manfred Opper, and Guido Sanguinetti. Approximate inference in latent gaussian- markov models from continuous time observations. In Advances in Neural Information Process- ing Systems, pp. 971-979, 2013.
Mark HA Davis. pathwise solution the equations of nonlinear filtering. Theory of Probability & Its Applications, 27(1):167-175, 1982.
MHA Davis. Pathwise non-linear filtering. In Stochastic Systems: The Mathematics of Filtering and Identification and Applications, pp. 505-528. Springer, 1981.
Robert J. Elliott and W. Paul Malcolm. General smoothing formulas for Markov-modulated poisson
observations. JEEE Transactions on Automatic (00)) 2005. ISSN 001 80189286. doi: 10.1009/AAC22200.855222 255.85565....
Hadi T and Joao Gama. Event labeling combining ensemble detectors and background knowledge. Progress Artificial Intelligence, pp. 1-15, 2013. ISSN 2192-6352. doi: 10.1007/
s13748-013-0040-33 URL [WebLink]
Jiequn Han, Arnulf Jentzen, and E Weinan. Solving high-dimensional partial differential equations using deep learning. Proceedings of the National Academy of Sciences, 115(34)88505-851) 85100 2018.
Yuval Harel, Ron Meir, and Manfred Opper. A tractable approximation to optimal point process filtering: Application neural encoding. Advances in Neural Information Processing Systems, 2015-Januayy: 1603-111 2015. ISSN 10495258.
Matthew D Hoffmann David M Blei, Chong Wang. and John Paisley. Stochastic variational infer- ence. The Journal of Machine Learning Research, 14313113 2013.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 14X2:644/2.6980 2,9900 2014.
Anna Kutschireiter, Simone Carlo Surace, and Jean-Pascal Pfister. The hitchhiker's guide to nonlinear filtering. Journal of Mathematical Psychology, 94:102307, February 2020. doi: 10. 11101///..pp22111102222
URL/ttps: wttp::////oo..g.......0....
jmp. 2019. 102307
Xuechen Li, Ting-Kam Kam Leonard Wong, Ricky TQ Chen, and David Duvenaud. Scalable gradients for stochastic differential equations. arXiv preprint arXiv:2001.00 0. 2020.
Sanjoy K Mitter and Nigel J Newton. A variational approach to nonlinear estimation. SIAM journal on control and optimization, 4215):1813-1833, 2003.
Bernt Oksendal. Stochastic differential equations: an introduction with applications. Springer
Science & Business Media, 2013.
Simo Särkkä. Bayesian Filtering and Smoothing. Institute of Mathematical Statistics Textbooks. Cambridge University Press, 2013. doi: 114.077/CB00781113994990
David Schnoerr, Ramon Grima, and Guido Sanguinetti. Cox process representation and inference for stochastic reaction-diffusion processes. Nature Communications, 7(1), May 2016. doi: 10. 1038/ncomms 1729. URL
/10
10. 1038 acomms 1 1179
J. H. Van Schuppen. Filtering, prediction and smoothing for counting process observations, martingale approach. SIAM Journal on Applied Mathematics, 32(3):552-570 May 1977. doi: 10.1 10.1137/0132045 URL wttpeee..................1
10

---

Under review as a conference paper at ICLR 2021
Robert D. Skeel and Martin Berzins. A method for the spatial discretization of parabolic equations in one space variable. SIAM Journal on Scientific and Statistical Computing. 11(1):1-32, January 1990. doi: 10.1137/0911011, htttts::://doi 022/11.1137/991100
Snyder. Smoothing for doubly stochastic poisson processes. IEEE Transactions on Information Theory, 18(5)558-566, 1972.
Alex Susemihl, Ron Meir, and Manfred Opper. Analytical results for the error in filterigg of Gaus- sian processes. Advances in Neural Information Processing Systems 24: 25th Annual Conference on Neural Information Processing Systems 2011, NIPS 2011, pp. 1-9, 2011.
Tobias Sutter, Arnab Ganguly, and Heinz Koeppl. A variational approach path estimation and parameter inference of hidden diffusion processes. Journal of Machine Learning Research, 17: 2016. ISSN 15337928.
Belinda Tzen and Maxim Raginsky. Neural stochastic differential equations: Deep latent gaussian models in the diffusion limit. arXiv preprint arXiv:1905.00 09883, 2019.
Ruixin Wang, Prateek Jaiswal, and Harsha Honnappa. Estimating stochastic poisson intensities using deep latent models. arXiv preprint arXiv:2007.06033 2020.
Tingting Zhang and S. C. Kou. Nonparametric inference of doubly stochastic poisson process data via the kernel method. The Annals of Applied Statistics,, 1913:1913-194,, December 2010. doi: 10. 201211410--0as352
httts:ts::///tt http://....../11.....11..1.. httpg//////oooorg///001/////00-0 0000533355
Xiaowei Zhang, L Jeff Hong, and Jiheng Zhang. Scaling and modeling of call center arrivals. In Proceedings of the Winter Simulation Conference 2014, pp. 476 485, IEEE, 2014.

---

Under review as a conference paper at ICLR 2021
A APPENDIX
A.1 DERIVATION OF SMOOTHING SPDE
According Theorem [sic] in Elliott & Malcolm (2005), K, (Sez ((()))())d))) is almost surely constant for & < T. Using this result it tollows that
p.ps. = (114(1)00 + (()r()))()((z)))
= (x) + (2))(z)
1, L' ps.t v W,L ( + ((((T)
PS,
(
v V L'V,
p9. (z)
vx(x)
Ps. Vi(x)
[[V](2)] + Vi(x) |
Ps.t Vo(2))
(20)
where Vi(x) = (x) and PS. = (o are introduced for brevity. Now observe that
V*(x)L*
Ps,t
Vi(x))
1 Σ
Oman (aig(())))))))(())
ps,t Vi(x))
Σ Oz.
(1((z)
Ps.t
Vi(+:)
2
Vi(x))
Vi(x))
1
Ps.t (ats(T)
Oz, Velor)b Ví(x)
2
Vi(x))
1
Σ
((,,,,(())ss + b,
(21)
Consider the summand in the first term of equation 21 and observe that Oa, (x()))
(z)(() = 0z, Vi(2),,, Vi(x)
az,w,
(x²(z)
Vi(x)ae.a, (x)s()) 0r Vi(z) (als())ps.) Vi(())
20,, Vz(
1
[2z, Vi(x) (ar.s (x) (s),s
dz. (0z, (
(x)
V2(x))
Now the second term in equation can be expressed as
a ( V
0z, Ve(x)0,, (z)s())
= Onia (a:. (x))5.))
Vi(z)
Vi(x))
20,, V((x)
+
[Oo, (x( (an5(z)
dr. [0x, (()
(22)
V2(())
Vo(x)
Substituting the above expression in equation 21 we have
O2, Ve(x))o, (x)s())
V*(x)L*
P5,t
Σ
(er (ais(z)ps.)
Vi(x))
2
Vi(x)
20,, (z)
[d., Vz(x) (an, (x)sss)
1 Vi(x)
dz [0z, (i(2)a }
V7(x)
On, (x)),,
Σ
-100,,(())ss + (zx)0,,
(23)
V((x)
Now consider the first term in the RHS of equation 20
Ps. [[V((>)]
= Σ as (1)ps
Oz,a, Vi(x) Σ
bi(r)ps. or Vo(x). Vi(z)
(24)
Vx(x)
2
f.j Vi(x)
12

---

Under review as a conference paper at ICLR 2021
Now substituting equation 23 and equation 24 into equation 20
we obtain
Ot
PS.
Vi(x)
ps.t
[[V[(2)]] + Vz(x))
L
Vi(æ)
Ps.t
Σ a..I
bx())s..
Deja; Ve(x))
Σ
Oz. Vi(x)) +
Σ
Deray (at,(r)Ps..)
2 2
Vi(x)
Vi(x)
2
O2. Ve(x))22 (x)Ps.)))
20z, Vi(x)
100, Vi(xx) (at,
Oz. 10, ((()))s }
Vi(z)
V2(x))
Vi(x)
Sr. (x)))
Σ (
Vi(z)
dea, Dea, Vi(z)
Σ
Dziny (as.) (2)ps.)
Σ
Dos Dz: [b((s)
Σ
(ar (x)ps.
Vi(Ø)
i,
22 (xxx)0,, (x)pss)
20,, Vi(z)02, Vi(x)) (x)
+
(I)i(())ss.))
Vz(x)
[n,, Vz(x)),,, (an,) (c)ps,) + Ozeay (=()))r,(=)psss)
(x)
Oeia, (x))s))) Don
Dn. [x:(p))s
Or [Oz, log ((xx)aa,(=)pss.) + Oz. Vi(1))s,, (an,(x)
2 1
Vi(())
Oz Vi(z) Vz(
(T))(s) Vi(x) Do,,, (z) }
(i(())
Since, On,n, Vi(x) = Ozjar Vx( therefore
0
Σ
O2izy (ai, (x)).s) Σ On (x)ps.)
Hr
Pstt
1
Oz. [O., log Vi + :ng 10,, log (x(r))aa,,())ss. }
2
Σ
Ozizy (o.,(z)ps.)) Σ dz. [b((2)ps.l
Σ fzx [Ox, log Va(z)(a,, 2.j
=
2
=
Oziay (ai5(1) Σ dz. (l(a(x)[Vlogg bo
(25)
2
A.2 KI-DDIERGENCE BETWERN A MEMBER OF VARIATIONAL. FAMILY AND PRIOR SDE
We derive a pathwise expression for (o) for a given count observation path No
Theorem A.I. Define u( I, N₂; 0) == (Ix)) -1 (b(x,, 1) b(xr, 4, Ne; and suppose that
satisfies strong Novikov's 's condition:
1 cxp
6/ |1 Nz; ]
< +00 V0
2
Then,
1 T
KL(QQII) EQ
|| t, Ne:
(26)
13

---

Under review as a conference paper at ICLR 2021
Proof. Given samples path of count observation No.r using the definition of and under Novikov's condition, using Girsanov's 's theorem (Oksendal 2013 Theorem 8.6.8), we have
dQ = exp (-/²" u( No: dd % [,u(r,, N:: ds1 dll
and
B, : fCu(ere L, No; 0)ds (())
u((
(27)
is a Brownian motion w.r.t Q. Furthermore, we also have
dæ, b(x,, t)dt + (x, (BB..
(28)
The following expression is obtained by substituting for 4Q atl
dQ(Zor)) (aTo((a:))
Ee log
(29)
1 -BQ 1/" (u(zest; Nz; + Ill((r (, N,; 0)12 ds
Now, applying B(t) = 1o u(ae, t, Ne; 0)ds + B, in Equation equation 29 we have
%
Eq
u(z Nr;
Eq
N1; u(xx Ne:
EQQ
N,;
No; Nz;
EQ
||((1,54, 0):
(30)
Substituting equation 30 into equation 29 yields
dQ (
<l.(QI| = EQQ
log
dlz (Xo:r)
(31)
= E2Q
||u(se, 1, Ne: 0):
2
and thus concludes the proof.
14

---

Under review as a conference paper at ICLR 2021
A.3 NEURAL NETWORK ARCHITECTURE
For all the experiments, we use the neural network architecture depicted in Figure A.3 activation functions between fully-connected hidden layers.
with ReLU
Input Layer (d+2)
Fully connected layer (50)
Relu
Fully connected layer (10)
Relu Activation function
Output Layer (d)
Figure 4: Neural network architecture used in the numerical experiments.
A.4
COMPARING COMPUTATIONAL TIME REQUIRED TO NUMERICALLY COMPUTE VBSP AND TRUE SMOOTHING DENSITY USING FEM
Epochs
6
8
9
10
13
14
13
16
17
1.59
T.51
True SD
2,07
2
2.22
2.17
2.16
2.16
2,23
2.12
2.12
2.15
2.12
2.17
2.14
2.11
2.14
2,18
2.15
Epochs
18
19
20
21
22
23
24
23
26
27
30
31
32
33
34
185
2.06
1.81
1.84
True SD
209
2.18
2.25
2.37
2.22
2.18
2,22
2.23
2.11
2.24
2.33
2.29
2.36
2.39
2.41
2,17
2.12
Epochs
33
36
37
38
39
40
41
42
43
44
43
46
47
48
49
30
183
1.91
True SD
214
214
215
2188
222
217
214
212
22
2115
223
217
217
2.28
2.25
217
Table 1: Comparing computational time (in sec) required compute 1-D VBSP and smoothing density using FEM. The time reported here are median over 20 test sample paths of count observa- tions each epoch
15


Reviews:
[{"review_id": "keGz9unhkJE", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper under review proposes a variational inference procedure for a specific class of Cox processes whose intensity is derived from a stochastic differential equation. The methodology relies on a restriction of candidate solutions the the subset for which the drift depends on $x_t$, $N_t$ and $t$; the drift is then modelled with a neural network. By simulating from the candidate model, a sample average approximation of the ELBO is used to compute a stochastic gradient descent algorithm, optimize the bound and thus estimate non-parametrically the drift.\n\nThe paper is clear and well written with original and significant contribution. The paper would benefit from clarifications about the following points:\n\n\n## Major points\n* It is not clear how equation (13) is a subset of the solution of (2) in general more parameters implies more degree of freedom and thus the drift in (13) could look more general than $b(x)$ (the use the $\\bar{b}$ notation suggesting it refers more to (11) than (2)). This probably comes from the fact that all variables are indexed by $t$ but more details would be nice here.\n* The proposed framework requires to specify both $b$ and $\\sigma$ a priori. It would be nice to explain what is the impact and restrictions of doing so. For instance, the first experiment set $\\sigma$ to $1$ and the second to $1.1$ without further justification. Could the author(s) elaborate(s) on this point? Similarly, the authors use a very flexible model (NN) for $\\bar{b}$, so the main restriction comes from the choice of the `target' $b$: the impact of this choice should be discussed in more detail.\n* Similarly to the previous points, in the methodology, the link function $h$ must be set a priori (and its uncertainty is not taken into account). Would it be possible to have an inference methodology estimating not only the drift but also $h$ (and $\\sigma$?) at the same time? If not, is it possible to explain why?\n\n\n## Minor points\n* p.1 l.1: the reference for Cox processes is not the 'original' work.\n* p.2: 'in particular, we show that...' either 'that' or 'how'.\n* p.2: VBSP has never been defined at this stage of the paper.\n* p.3: Wiener.\n* p.3: the notation $\\mathbb{E}^\\dagger$ is undefined.\n* p.3: defined the indicator function that you are using.\n* p.3: shouldn't the $x$ in the definition of $\\bar{v}(x_t)$ be bold?\n* p.3: which 'section'? Simply 'above'?\n* p.3: $f$ denotes first bounded measureable functions and then twice-differentiable ... Use two different notation to avoid confusion.\n* In general, use eqref for equations.\n* p.5: We call $Q^\\star$ as the VB ... Defined the meaning of VB.\n* p.5: specify the class of parametric functions used in Sutter et al.\n* p.6: the notation $\\psi_t'$ is not defined. The link with equation (13) could be made clearer.", "extra_scores": {}, "extra_text": {}}, {"review_id": "BpJCfvKaThx", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper provides a stochastic variational inference method for\napproximating posterior path measures for doubly-stochastic Poisson processes\nconditioned on realisations of paths of the Poisson process. The intensity\nprocess is modelled as the solution to a diffusion stochastic differential\nequation. The authors compare their method experimentally to the numerical\nsolution of the associated system of stochastic partial differential\nequations using the finite element method.\n\nThe paper, although technically difficult, is well written and clearly\npresented. While I cannot validate the mathematical content in detail, the\npaper seems technically correct. It deals with a well-defined problem in a\ndifficult mathematical setting. \n\nThe idea of using variational inference for approximating the smoothing\nposterior density seems well-founded. The method is experimentally validated\non simulated and real data.\n\nIn summary, though my knowledge of point process theory is not sufficient to\nevaluate all aspects of the paper in detail, I find the paper to be a solid\ncontribution worthy of publication at ICLR.", "extra_scores": {}, "extra_text": {}}, {"review_id": "BL6joWsS59", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes an interesting point process named diffusion modulated cox processes, which generalizes the stochastic intensity to a stochastic differential equation. The variational inference method looks sound. \n\nPros:\n+ The generalization of SDE-type intensity is novel. The proposed stochastic variational inference makes sense. Especially the neural network solution is meaningful and will have an impact on the learning of point processes. \n+ Good empirical performance and analysis. \n\nCons:\n- I strongly recommend the authors to further improve the presentation of the current draft. Sometimes the notations are not consistent, like $\\mathbb{E}$ and $E$, $\\mathbb{R}$ and $R$. The definition of some of the notations are not very clear, such as Eq(3), what is the definition of the conditional expectation? Is it the expectation of the intensity function given all the information about the process?  Besides, please give a brief introduction of the mathematical background, such as the smoothing posterior. It would also be better if the authors can give some examples to illustrate the advantages of the proposed diffusion modulated cox process.\n- Please discuss the influence of the non-negativity of the intensity function. The paper claims that $h$ is non-negative and thus can be an identity function, which is false.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper presents a stochastic variational inference method for posterior estimation in a Cox process with intensity given by the solution to a diffusion stochastic differential equation. The reviewers highlight the novelty of the approach. Some of the concerns with regards to clarity have been addressed by the authors satisfactorily. 

However, an important issue of the approach is that of estimating model parameters, which the authors do not address explicitly by simply referring to that as the task of the modeller. I believe this is an important issue and, although some of the parameters can be estimated along with the neural network parameters, this has not been shown empirically. Along a similar vein, the paper only presents results on a single real dataset (the bike-sharing dataset), which questions the applicability of the approach and no other baseline method is presented. At the very least, the authors should have provided an objective evaluation to other doubly stochastic point process models, e.g. based on Gaussian processes, where modern stochastic variational inference algorithms have been presented. 

Decision: Reject