Efficient Training of Neural Stochastic Differential Equations by Matching Finite Dimensional Distributions
Authors: Jianxin Zhang
Josh Viktorov
Doosan Jung
Emily Pitler
Venue: ICLR 2025 Poster (2025)
Keywords: neural stochastic differential equations
Markov process
scoring rule
Primary area: learning on time series and dynamical systems
TL;DR: We propose a novel class of scoring rules to compare laws of Markov processes, leading to an efficient algorithm to train neural SDEs.

Abstract: Neural Stochastic Differential Equations (Neural SDEs) have emerged as powerful mesh-free generative models for continuous stochastic processes, with critical applications in fields such as finance, physics, and biology. Previous state-of-the-art methods have relied on adversarial training, such as GANs, or on minimizing distance measures between processes using signature kernels. However, GANs suffer from issues like instability, mode collapse, and the need for specialized training techniques, while signature kernel-based methods require solving linear PDEs and backpropagating gradients through the solver, whose computational complexity scales quadratically with the discretization steps. In this paper, we identify a novel class of strictly proper scoring rules for comparing continuous Markov processes. This theoretical finding naturally leads to a novel approach called Finite Dimensional Matching (FDM) for training Neural SDEs. Our method leverages the Markov property of SDEs to provide a computationally efficient training objective. This scoring rule allows us to bypass the computational overhead associated with signature kernels and reduces the training complexity from $O(D^2)$ to $O(D)$ per epoch, where $D$ represents the number of discretization steps of the process. We demonstrate that FDM achieves superior performance, consistently outperforming existing methods in terms of both computational efficiency and generative quality.

Published as a conference paper at ICLR 2025
EFFICIENT TRAINING OF NEURAL STOCHASTIC DIF- FERENTIAL EQUATIONS BY MATCHING FINITE DI- MENSIONAL DISTRIBUTIONS
Jianxin Zhang
Josh Viktorov, Doosan Jung, Emily Pitler
EECS, University of Michigan
Cisco Systems
Ann Arbor, MI 48109 j anxi nzę umi ch edu
New York, NY Lakewood, CO San Jose, CA {joviktor, doojung, epitler) iecceiicc..oo
ABSTRACT
Neural stochastic differential equations (Neural SDEs) have emerged as powerful mesh-free generative models for continuous stochastic processes, with critical ap- plications in fields such as finance, physics, and biology. Previous state-of-the-an methods have relied on adversarial training, such as GANs, or on minimizing dis- tance measures between processes using signature kernels. However, GANs suf- fer from issues like instability, mode collapse, and the need for specialized train- ing techniques, while signature Cornel-based methods require solving linear PDEs and backpropagating gradients through the solver, whose computational complex- ity scales quadratically with the discretization steps. In this paper, we identify a novel class of strictly proper scoring rules for comparing continuous Markov pro- cesses. This theoretical finding naturally Icads to novel approach called Finite Dimensional Matching (FDM) for training Neural SDEs. Our method leverages the Markov property of SDEs to provide computationally efficient training ob- jective. This scoring rule allows us to bypass the computational overhead asso- ciated with signature kernels and reduces the training complexity from O(D²) O(D) per epoch, where D represents the number of discretization steps of the process. We demonstrate that FDM achieves superior performance, consistently outperforming existing methods in terms of both computational efficiency and generative quality.
INTRODUCTION
Stochastic differential equations (SDEs) modeling framework used to describe systems in- fluenced by random forces, with applications spanning finance, physics, biology, and engineering. They incorporate stochastic terms to allow the modeling of complex systems under uncertainties.
A neural stochastic differential equation (Neural SDE) (Kidger et al. 2021 Issa et al. 2023 al.
Izen & 2020 Morrill et 2020)
Raginsky 2019 Jia & Benson| 2019 Hodgkinson al. [202]] Li
is an SDE where neural networks parameterize the drift and diffusion terms. This model acts as mesh-free generative model for time-series data and has shown significant impact in financial
applications (Arribas et al. 202T Gierjatowicz et al. 2020 Choudhary el al. 2023 Hoglund et al. 2023
Training Neural SDEs typically involves minimizing distance measure between the distribution of generated paths and the distribution of observed data paths. State-of-the-art performance has been achieved using signature kernels to define distance measure path space (Issa et al. 2023). Al- though effective, this approach requires solving linear partial differential equation (PDE) whose computational complexity scales quadratically with the discretization step, which becomes imprac- tical for long time series. An alternative is training these models adversarially as Generative Ad- versarial Network (GAN) (Kidger al. 2021) However, GAN-based training be fraught with issues such as instability, mode collapse, and the need for specialized techniques.
Most of the work done during internship Cisco.

---

Published as a conference paper at ICLR 2025
In this paper, present theoretical result that extends scoring rules for comparing distributions in finie-ddimensional spaces to those for continuous Markov processes. This extension forms the basis of novel algorithm, Finite Dimensional Matching (FDM), designed for training generative models of stochastic processes. FDM exploits the Markovian nature of SDEs by leveraging the two-time joint distributions of the process, providing an efficient training objective that bypasses the complexities of signature kernels. Notably, FDM reduces the computational complexity from O(D²) OD)) per training step, where D represents the number of discretization steps.
The key contributions of this paper are as follows:
Our main theorem shows that scoring rules to compare continuous Markov processes can be easily built upon scoring rules on finite-dimensional space.
Our main theorem suggests an efficient training method, FDM, for Neural SDE.
Our experiments show that FDM outperforms prior methods on multiple benchmarks.
The rest of the paper is organized as follows: Section 2 provides review of the relevant literature. In Section 3] we present preliminary results that lay the foundation for our main contributions. Sec- tionAinnrooduee our main theorem, which extends scoring rules for finite dimensions to continuous Markov processes and leads to the development of our novel Finite Dimensional Matching (FDM) algorithm. In section 5 we analyze the sample complexity and sensitivity of FDM. fection[ddetall the experimental setup and results, demonstrating the superiority of FDM in terms of both compu- tational efficiency and generative performance across several benchmark datasets. Finally, Section 7 concludes the paper by summarizing the contributions, limitations, and directions for future work.
2 RELATED WORK
We begin by reviewing prior applications of scoring rules in generative modeling followed by an exploration of the literature on training Neural SDEs.
2.1 SCORING RULES
Scoring rules offer method to measure discrepancies between distributions (Gneiting & Raftery 2007) and are especially appealing for generative modeling and have been employed in training various generative models (Bouchacourt al. 2016: Gritsenko et 2020 Pacchiardi al. 2024 Pacchiardi & Dutta 2022, Issa et al.) [0233] Bonnier & Oberhauser [024]. Notably, Pacchiardi et al. (2024) apply scoring rules to discrete Markov chains, although their extension to continuous-time processes has not yet been explored. Issa et al. (2023) and Bonnier & Oberhauser (2024) construct scoring rules for continuous processes by utilizing signature kernels.
2.2 NEURAL SDES
Several methods have been proposed for training Neural SDEs generative models, each differing in how they define the divergence or distance between distributions on path space. In Table [ we compare different methods for training Neural SDEs, highlighting their divergence measures and the corresponding discriminator or training objectives. Our approach, Finite Dimensional Matching (FDM), introduces a novel scoring rule specifically designed for continuous Markov processes.
One method to train Neural SDE is the latent SDE model introduced by Li et al. (2020), which trains Neural SDE using variational inference principles (Opper 2019). In their framework, training involves optimizing the free energy that includes the Kullback-Leebler (KI.) divergence between the original SDE and an auxiliary SDE. These two SDEs share the same diffusion term but have different drift terms. The KL divergence between their laws be computed using Girsanov's change of measure theorem. However, the performance of latent SDEs is generally inferior to SDE-
GANs due to lower model capacity (Kidger et al. 2021 Issa al. 2023).
A prominent method is the SDE-GAN introduced by Kidger et al. (2021) I), which employs adversarial training to fit a Neural SDE, in Wasserstein GANS (Arjovsky al. 2017). This approach relies on the 1- W-Wsssesstein distance, with the discriminator parameterized by neural controlled differential equations (Kidger al. 2020 Morrill et al. 2021). However, SDE-GANs are notoriously difficult
2

---

Published as a conference paper at ICLR 2025
Table 1: Methods for training Neural SDEs. SigKer stands for signature kernel (Issa et al. 2023), TruncSig is for truncated signature (Bonnier & Oberhauser, 2024). SDE-GAN is proposed by Kidger al. (202I). and Latent SDE is proposed by Li et (020).
Methods
Divergence or distance
Discriminator or training objective
Latent SDE
Kl.-divergence
Monte-Carke simulation of free energy
SDE-GAN
I-Wasserstein distance
Optimizing discriminator nets Solving Goursat PDEs
SigKer
Signature kernel score
TruncSig
Truncated signature kernel score A novel class of scoring rules for continuous Markov processes
Truncated approximation of signature Standard ERM of the expected scores
FDM (Ours)
to train due to their high sensitivity to hyperparameters. Another major challenge the need for Lipschitz discriminator, which requires techniques like weight clipping and gradient penalties to enforce this constraint (Kidger, 2022). Adversarial training for time-series generative models has
also been explored in the context of discrete data (Ni et al. 2022 Yoon et al. 2019).
Another key contribution to training Neural SDEs is the signature kernel method (Issa et al. 2023
Bonnier & Oberhauser 2024). which minimizes distance measure based signature kernels (Lee & Oberhauser] (023) of paths. However, evaluation of the signature kernel requires solving Goursat partial differential equations (PDEs) and backpropagating gradients through the solver (Salvi al., 2021]. The computational complexity of solving Goursat PDEs scales quadratically with the number of discretization steps, which can be prohibitive for long time series data. Bonnier & Oberhauser (2024) approximates the signature kernel as inner products of truncated signature transforms, called truncated signature. However, the scoring rule based on truncated signature not strictly proper and has (dN) ) memory complexity where d is the number of features and N is the truncation size.
The concurrent work of Snow & Krishnamurthy (2025) introduces a novel technique that leverages Wiener space cubature theory to bypass Monte Carlo simulations in Neural SDE training.
3 PRELIMINARIES
In this section, we set up the notations and introduce the following preliminary concepts: Neural SDEs, Markov processes, and scoring rules.
Background and Notations Let {S, FF be probability space where 22, F, P denote the sample space, sigma-algebra, and probability measure, respectively. For a random variable & the function Pg = Po 5-1 is the induced measure on its range space. In particular, for random process X, Px denotes its law. We use the superscript T for the transposition of a matrix or vector.
Rdooise be Brownian motion on Ed where dnoise E N. We define a Neural SDE as in Issa et al. (203) and Kidger et al. (202T):
Neural SDE Let B [0, 7
Zo = 5 dZ, = μ p((tt, Zt) + o (t, dW,, x" = A Z 5" where a is sampled from a cinitiar-dimensiona standard Gaussian distribution.
5" Iddinsiial + Rdd, 1 :0, T' x Rd= + Rdd σa [0, x Rd= dgd Xdmouse
along with A E Rd.xd. 80 E Rd=, are functions parameterized by neural networks, and dimitiats, du, d2 E N. We assume additionally that 1 and の are Lipschitz continuous in both argu- ments and (a) has finite second-order momentum. These conditions ensure that the SDE for Z. has a unique strong solution. Suppose Y, is the data process, we' 'dd like to train the neural networks 0 on data sampled from Py so that p matches Py.
Markoy Process We say a continuous process X, with filtration {Fi} is Markov if X,, is in- dependent of F, for all u > given X, (Kallenberg) 2021). For an SDE of the form dX, (tt, X.) dt + (t, X₂) dB,, with the filtration generated by the Brownian motion B,, X, is Markov as long as the SDE has unique strong solution (Theorem 9.1, (2007)).
Scoring Rules Given a measurable space (9g. Fo) and wi E 9n scoring rule Gneiting & Raftery 2007) s( s(P, wb) maps probability measure P on On and sample wp R.. The expected score is
3

---

our main claim, Published as a conference paper at ICLR 2025
defined as S(P.Q) Eqls(P, wo)] = Sn, s(P, wb) dQ((w)), where is the predictive distribution
and Q is the true distribution. The scoring rule s is said to be proper if the expected score satisfies S)) < S(Q. Q). is strictly proper Q) = SQQ.Q) Q) = P = Q. For example, let IR R + be the RBF kernel defined as k(z,y) 0 a parameter that determines the width of the kernel, then s(P.2) = 2. Z, Z')
exp 1000~711a y/²) where 7 >
Ez-pk(Z, z) is a strictly proper scoring rule for distribution on 12d (Gneiting & Raftery 2007).
Let pd be distribution controlled by a generative model 0, and let Q be the true distribution accessed through data. Given strictly proper scoring rule sufficient model capacity of 6, and sufficient data points from Q, pd can be trained by maximizing S(P", (Q) over 0, leading to p Q. While many scoring rules for finiteddimensional spaces have been proposed, we lack strictly proper scoring rules for random processes that can be evaluated efficiently.
we show that strictly proper scoring rule for two-time joint distribution, i.e., the distributions {(X,,, X²,), f2 E [0, TT}. for a random process X, can be converted into a strictly proper scoring rule for continuous Markov processes.
4 FINITE DIMENSIONAL MATCHING
In this section, we present our main theorem, which converts a scoring rule for a two-time joint distribution into scoring rule for Markov process. Specifically, if we have a scoring rule for 2o 2d then Theorem 2 allows us to convert it into a scoring rule for Markov processes X,Y: Y
[0, [] Rd where d N and T E Rxo.
4.1 SCORING RULE FOR MARKOV PROCESS
In this section, we present our main theorem which shows that strictly proper scoring rule for the two-time joint distributions can be converted to a scoring rule for two Markov processes. Let continuous Markov processes X, Y on [0, T) take values in a Polish space & endowed with its Borel o-algebra. Let be any strictly proper scoring rule defined on E x E. Let ((P, Q) = Eq[s(P,w)) < oo, Y measures P, Q on E E. We define the scoring rule for continuous Markov processes as following:
X1,,X,,)) (yt, Vt2)), where P(x,,xX,)) is the joint marginal distributions at times 41, l2 of X, and U(O TJ*) is the uniform distribution on [0, TT².
Definition 1. s(xx, v) = ((1.,)))
Let S(Px. Py) = ((Pxx Now we present our main claim, with its proofs deferred to the appendix.
Theorem 2. If is strictly proper scoring rule for distributions on E x E, is a strictly proper scoring rule for E-valued continuous Markov processes on [0, where T E Ryo. That is, for any E-valued continuous Markov processes X, with laws Px, Iyy, respectively, S(Px,Py Py) S S(Py,Py) with equality achieved only PPy
In the appendix, we present more generalized version of Theorem that does not require the timestamps l and t2 to be sampled from ([0, T?²) in the definition of s. Nonetheless, to maintain clarity and simplicity, we focus our discussion the uniform sampling case in the main paper.
Suppose X is a Markov process parameterized by neural net parameters 0 with sufficient capacity. Therefore, maximizing S(I x=, Pv) BB [s(IP x., Y)), which can be achieved by maximizing the corresponding empirical average, will result in P Xx== = Py
We present conerete example on the application of Theorem 2 Consider continuous Markov processes X, Y on Jo. T] taking values in Rdd Let k IR2d x 2d - R be the RBF kernel, recall that (P,, JE 2 pk Z, Z') EEzpk(2, is strictly proper scoring rule for distribution on ad (Geiiiig & Raftery
2007 p. By Theorem 2
5 ( y) -00,000000000111
x(,)) Ex .x,, [B6, Vcal))
(1)
is strictly proper, where [ the concatenation of two vectors. 5(Px', Py) = Ey [P(x' Y)) can be estimated through empirical average and optimized efficiently.
4

---

Published as a conference paper at ICLR 2025
4.2 FDM ALGORITHM
We consider an expected score s(Pxx,Pyy) = Ey-py [3(Px,, Y),, which can be estimated using an empirical average S. For example, unbiased estimator of S for s defined in ①1) can be con- structed using batches of generated paths Bx = {x*) |==1 and data paths By {v²) /i=1- For each independently sample two timestamps and f.. The empirical estimator is then given by:
S(Bx, By) 2B(B - 1)
Σ ijj
lo (,,
B
Note that the above estimator S only requires each data path to (potentially irregularly) observed at two distinct timestamps, and by the Neural SDE model. Alternative empirical objectives are provided the appendix.
observe the z' at any timestamps since they are generated
present the concrete finite dimensional matching (FDM) algorithm derived from In Algorithm I
Theorem to train a Neural SDE X
Algorithm 1: Finite Dimensional Matching (FDM)
Input: Neural SDE X", data paths {y' [N]], strictly proper scoring rule batch size B repeat
Generate batch of simulated paths 3x {x* [B]] using the Neural SDE model 0; Randomly sample batch of data paths By C {y' : [N] with |By| B
Compute the empirical estimate B of
S(Px PPy);
Maximize Å with respect to & using optimizer of the user's choice;
until stopping criterion is met;
THEORETICAL PROPERTIES
In this section, we investigate the sample complexity and sensitivity of the proposed scoring rules 8. All proofs are deffered to the appendix.
5.1 SAMPLE COMPLEXITY
We show that the sample complexity of the estimator 51 By) retains the classical sample com-
plexity of kernel-based scoring rule s (Gretton al. 2012). Let h be kernel associated with Reproducing Kernel Hilbert Space (RKHS) and s(P.z) _ Ez,z' pk (2, Z') Ez-pk(Z) be (x,, X.,) (Yt,, Mtt)) and
strictly proper scoring rule. Recall that 3(Px,y) E(t..t))-C(10TPT) ((Px, Pr) = Ey-sy (5(P (i)xx y)]-
Theorem 3. Let k(., satisfy S k(.,)) K and the batch size B 2 2. For any e > 0,
8BE²
P (I$ EP$1 2 S 2exp
47K2
Equivalently, with probability at least 5, the deviation of $ ffrom its expected value S(PP x, Py) is bounded as
47 47
SS By) - 5(P Py) < K
8B
exhibits a sample complexity analogous to the classical sample complexity of kernel based scoring rules s. Section B of the appendix, we extend this analysis to an alternative estimator where all sample paths are evaluated at n shared timestamps.
5.2 SENSITIVITY
Let X be an Itô diffusion on R., i.c., dX, = u(t,X,)dt ot,, Xt) The following theorem shows how perturbations in y and affect the value of the scoring rule s(Px,y))
5

---

Published as a conference paper at ICLR 2025
Theorem 4. Let X satisfy d.x, Xi)t ot, Xi)dB, R.. Let X satisfy d.x,, itt,, + Xj)dBr Red where I. [|l(t,, x) (((,,))l2 I 6 ||o(t, ő (1,,)]12 S Sa. and 6n, 80 are constants. Assume the scoring rule s( 2) is Lipschitz in terms of distribution P with respect to the Wasserstein-2 distance. Assume both X and X have unique strong solutions and share the same initial conditions, then (8(Px.y) - x. S 1,, + dz), where Lig is the Lipschitz constant of the constant C depends on Lipschitz constants of and a.
we allow different sampling distributions of 4, la than uniform as in the generalized main theorem Theorem 7 in the appendix, C may also depend the sampling distribution of ( 12. This result provides a theoretical guarantee that small changes in the dynamics of the process result in changes the scoring rule that are linear with respect 6 6g.
EXPERIMENTS
Table 2: Average KS test scores (lower is better) and chance of rejecting the null hypothesis (%) at -iisinnife level on marginals (lower better) of metal prices, trained on paths evenly sampled at 64 timestamps.
Dim
Model
1= 6
19
1= 32
1 ==
57
SigKer
144, 23.9
.144, 14.4
.13,, 11.5
126, 9.20
.122, 7.96
TruncSig
.274, 97.9
.277, 98.7
2293, 99.3
.34, 99.6
.31,, 99.6
SILVER
SDE-GAN
.300, 70.0
.647, 100.
.789, 100.
.813, 100.
.828, 100.
FDM (ours)
.118. 9.76
.114, 7.24
.112, 6.08
114.7.22
.117, 8.16
SigKer
.129, 10.7
.127, 9.40
.128, 10.1
.126, 9.80
.123, 8.16
TruncSig
24.7
.274, 98.4
.298, 96.8
316,999
.300, 99.8
GOLD
SDE-GAN
.244 90.6
.299, 94.8
.318, 96.8
.336, 96.8
.352, 91.7
FDM (ours)
1.3,9.32
.117,8.00
.15,,
.116, 7.66
.118, 8.52
Table Average KS test scores and chance of rejecting the null hypothesis (%) 5%6-significance level on marginals of U.S. stock indices, trained on paths evenly sampled at 64 timestamps. "DOL- LAR", "USA30", "USA500", "USATECH", and "USSC2000" stand for US Dollar Index, USA 30 Index, USA 500 Index, USA 100 Technical Index, and US Small Cap 2000, respectively.
Dim
Model
1=
1=19
1=32
1=44
1=57
SigKer
.262, 76.4
.316. 82.1
.322, 83.7
.314, 84.4
.266, 83.9
TruncSig
2799, 98.3
.303, 99.4
.323, 99.7
.339, 99.8
.35,, 99.9
DOLLAR
SDE-GAN
.389, 93.0
.544, 98.3
.605, 99.5
.599, 99.8
.553, 99.8
FDM (ours)
.143, 25.6
.151, 29.6
.153, 30.7
.155,
.156, 33.0
SigKer
200,
.239, 78.8
.244, 91.8
279, 94.2
.211 93.7
TruncSig
51.5
61.9
.213, 70.9
.288, 79.1
.250, 89.5
USA30
SDE-GAN
$11,80.9
.402, 91.6
.428, 90.6
.550, 99.9
.666, 100.
FDM (ours)
.132, 15.6
.123, 10.0
.124,
.124,9300
.121, 8.04
SigKer
227, 288,,6.8 86.8
350,
.367, 94.0
365,93.5
.35, 93.1
TruncSig
57.7
.204, 63.7
.221,
.31,
.24, 86.1
USA500
SDE-GAN
310,97.0
.488 93.3
.625. 100.
.713, 100.
.746, 100.
FDM (ours)
.122, 9.82
.117, 6.84
.117, 6.56
.177, 6.30
.188, 6.52
SigKer
.12, 80.6
.240, 91.0
.22, 90.9
239,90.6
245, 92.2
TruncSig
.197, 74.1
.27, 86.1
.247, 91.3
265, 94.8
.280, 97.6
USATECH
SDE-GAN
420, 99.9
.786. 100.
.100, 100.
.500, 100.
.969, 100.
FDM (ours)
.123, 9.92
.119, 7.48
.118, 6.68
.118, 6.54
.118, 6.74
SigKer
225,, 70.1
.312, 88.3
.328, 94.5
.325, 94.2
.314, 93.6
TruncSig
.180, 46.5
.199, 62.5
.211 78.8
.233 89.0
.28. 96.4
USSC2000
SDE-GAN
.37, 75.4
.522, 98.2
.664, 100.
.843, 100.
.887, 100.
FDM (ours)
.134. 16.6
.126. 11.1
.122. 8.80
.124,
.122, 8.34

---

Published as a conference paper at ICLR 2025
Table 4: Average KS test scores and the chance of rejecting the null hypothesis (%) 5%- significance level on marginals for different currency pairs (EUR/USD and USD/IPY, trained on paths evenly sampled at 64 timestamps.
Dim
Model
t=6
t=19
t=32
t=44
t=57
SigKer
2251, 66.5
.293, 70.3
.288, 66.4
.271, 55.4
.248, 38.5
TruncSig
97.9
.313, 99.6
.340, 99.8
.354, 99.9
.369, 99.9
EUR/USD
SDE-GAN
.529, 89.2
.665, 95.8
.723, 96.0
.754,
99.8
FDM (ours)
.125, 12.9
.113, 6.26
.109, 5.04
.110, 5.46
.111, 6.10
SigKer
.165, 34.3
.189, 38.1
199,,444
.188, 30.5
.185, 29.3
TruncSig
.252, 87.8
291,98.0
.37, 99.4
.34, 99.7
.354, 99.9
USD/JPY
SDE-GAN
.12, 73.4
.267, 85.2
.309, 88.6
359,91.2
.425, 92.8
FDM (ours)
.120, 9.54
.111, 6.04
.110,
.111,
.111,
Table 5: Average KS test scores and the chance of rejecting the null hypothesis (%) at 5%- significance level on marginals energy prices, trained on paths evenly sampled at 64 timestamps. We reserve the latest 20% data as test dataset and measure how well the model predicts into future. "BRENT". "DIESEL" "GAS", and "LIGHT" stand for U.S. Brent Crude Oil, Gas oil, Natural Gas, and U.S. Light Crude Oil, respectively.
Dim
Model
t=19
t=32
1=44
L=57
SigKer
.284, 69.4
.339, 70.7
.343, 68.0
.328, 65.4
.302, 60.0
TruncSig
254,
.264, 95.4
.273, 96.7
.292, 98.1
.303, 98.8
BRENT
SDE-GAN
487,97.
.812, 100
.929, 100
.911, 100
.981, 100
FDM (ours)
127,12.5
.123, 10.2
.125, 11.5
.124, 11.6
.124, 11.4
SigKer
.187,77.1
218. 55.4
.223, .2.3,49.6 49.6
.222, 42.9
.219, 38.1
TruneSig
21.9
.244 93.9
.262, 97.9
.274, 98.9
.305, 99.6
DIESEL
SDE-GAN
77.9
.522, 97.6
.664, 99.8
.735, 100
.793, 100
FDM (ours)
.122, 10.2
.117, 7.82
.117, 8.20
.123, 10.5
.123, 10.7
SigKer
244,
298. 77.1
305, 71.5
.295, 69.1
.273, 63.7
TruneSig
224, 82.0
.800, 93.4
.301, 97.6
.328, 99.4
.342, 99.7
GAS
SDE-GAN
337, 86.2
.586, 99.8
.717, 99.9
.80,, 100
.877, 100
FDM (ours)
.116, 7.52
.116, 7.36
.123, 11.7
.127, 14.4
.126, 13.7
SigKer
.184, 60.1
200, 66.9
.15,, 59.1
186, 53.0
.173, 43.6
TruneSig
245, 91.7
.211, 94.6
.272, 96.4
.292, 98.7
.308, 99.5
LIGHT
SDE-GAN
2666, 74.8
.403,
.464, 85.7
.604, 98.6
.717, 99.9
FDM (ours)
.121, 9.82
.122, 10.4
.131, 15.5
.131, 15.2
.132, 15.5
We evaluate our method, FDM by comparing it three existing methods for training Neural SDEs: the signature kernel method (SigKer, Issa et al. (2023)). the truncated signature method (TruneSig, Bonnier Oberhauser (2024)) ), and SDE-GAN (Kidger al., 202Ip. Our experiments are conducted across five real-world datasets: energy prices, bonds, metal prices, U.S. stock indices, and exchange rates, as well as one synthetic dataset, the Rough Bergomi mode The weal-world datasets are historical price data for variety of financial instruments. The rough Bergomi model is widely used stochastic volatility model and has been extensively described Issa et al. (2023). For Neural SDE.
all datasets, we model all features jointly with single
Consistent with Issa et al. (202)). we use the Kolmogorov-Smine (KS) test to assess the marginal distributions for each dimension. Specifically, unseen batch from the real data distribution and calculate the KS scores and the chance of rejecting the null hypothesis, which states that the distributions are identical. This process is repeated
compare batch of generated paths against an
code available at https::/// / wtwpthwwwwwwwwwwwiiioo/oooa/aaae All real-world datasets are obtained from https: http:///wwwww www duc.duukassoooo www.wwwwwwoopppcooooooooooo com foom/aw ss/ https//wssssssss////////e/g wkktwatcchhisstrr....

---

Published as a conference paper at ICLR 2025
Table Average KS test scores and chance of rejecting the null hypothesis (%) 5%-significance level marginals of bonds, trained on paths evenly sampled at 64 timestamps. We reserve the most latest 20% data as test dataset and measure how well the model predicts into future. "BUND", "UKGILT", and "USTBOND" stand for Euro Bund, UK Long Gilt, and US T-BOND, respectively,
Dim
Model
t=
t=19
t=32
1=4
[=57
SigKer
.210, 49.0
.24, 52.8
.251, 54.7
.252,
.245, 54.3
TruncSig
95.0
.296, 99.4
.328, 99.7
.350, 99.9
.362, 99.9
BUND
SDE-GAN
.399, 97.8
.582, 100
.613, 99.3
.764, 100
.831, 100
FDM (ours)
.119, 10.2
.120, 8.80
.124, 8.66
.125, 9.72
.118, 8.20
SigKer
.158, 26.0
.183, 27.5
.197, 30.5
.201, 31.8
.201, 32.0
TruncSig
.200, 67.8
224, 89.8
.25,, 98.9
.12,
.337,99
UKGILT
SDE-GAN
.366, 84.3
.640, 99.8
.862, 100
.902, 100
.921, 100
FDM (ours)
.127, 12.6
.113, 6.62
.109, 5.28
1.9,
.109,
SigKer
.189, 40.6
.207,
.213, 35.7
.233 35.3
.114, 36.2
TruncSig
.229, 73.5
228. 81.3
.284, 95.0
.310, 99.3
.334, 99.8
USTBOND
SDE-GAN
351,8444
.688, 100
.853, 100
-903, 100
.916, 100
FDM (ours)
.137, 17.9
.124, 10.8
.115, 6.60
.113, 6.14
1111,566
Table 7: Average KS test scores and the chance of rejecting the null hypothesis (%) 5%-
significance level marginals for different currency pairs (EUR/USD and USD/JPY), trained on paths evenly sampled at 256 timestamps.
Dim
Model
t= 25
f = 76
t == 128
t t= 179
== 230
SigKer
.535, 100.
.535, 100.
.536, 100.
.546, 100.
.540, 100.
TruncSig
.137, 18.9
.184, 67.1
.222, 99.6
.290, 100.
.318, 100.
EUR/USD
SDE-GAN
.134, 21.6
.11, 100.
.569, 100.
.548, 100.
.338, 99.9
FDM (ours)
.136, 23.0
.112, 5.70
.123, 12.7
.132, 17.8
.141, 26.6
SigKer
.535, 100.
.534, 100.
553,, 100.
.538, 100.
.541, 100.
TruncSig
.114,
.152, 28.3
.199, 82.8
.232, 97.9
.22,, 99.2
USD/JPY
SDE-GAN
201,,2.6
.47, 100.
.405, 100.
.338, 100.
FDM (ours)
.124, 13.8
.112, 6.30
.118, 6.90
.122,
.115, 6.20
for all the test batches and we report the averaged KS scores and the chance of rejecting the null hypothesis across all the batches.
For all experiments, we use fully connected neural networks to parameterize the drift and diffusion terms, with hyperparameters and preprocessings suggested in Issa et al. (023)) We choose to be s(., = 1E2 ( Z') Ez-pk( Z, 2) where is the rbf kernel with unit kernel bandwidth. In particular, following Issa et al. (2023).
let method and TruncSig train for 10000 steps, while SDE-GAN trains for 5000 steps and SigKer for 4000 steps, to normalize the training time. Despite the differences in training steps, our method remains the fastest in terms of wall-clock time. All models are trained and evaluated on a single NVIDIA H100 GPU.
first follow Issa al. (2023) to train and evaluate the models on three datasets-metal prices, stock indices, and exchange rates rates-using using sequences with 64 timestamps and random train-test splits. This training and evaluation process is repeated with five different random seeds, and the average KS scores and rejection rates are reported in Tables 2 and 4 respectively, with corresponding standard deviations provided in the appendix. For the energy price and bonds datasets, we reserve the latest 20% of the data for testing, evaluating the trained models via the KS test on generated sequences against unseen future sequences. These results are presented in Tables (and We repeat the experiments on these two datasets five times, with standard deviations also reported in the appendix. Additionally, for the exchange rates dataset, we trained and tested the models using sequences with 256 and 1024 timestamps, reporting KS test results in Tables 7 and 8 and training time in Table 16 in the appendix. For the synthetic rough Bergomi model, we generated sequences with 64 timestamps across both 16 and 32 dimensions, with results reported
For our experiments,
8

---

Published as a conference paper at ICLR 2025
Table 8: Average KS test scores and the chance of rejecting the null hypothesis (%) 5%- significance level on marginals for different currency pairs (EUR/USD and USD/IPY, trained on paths evenly sampled at 1024 timestamps. thread limit error encountered during the training of the SigKer (Issa et al. 2023), which relies on dedicated parallel PDE solver.
Dim
Model
102
307
512
716
921
SigKer
TruncSig
.476, 100.
.718, 100.
.99,, 100.
.996, 100.
.887, 100.
EUR/USD
SDE-GAN
.280, 98.4
.888, 100.
.96,, 100.
.846, 100.
.85, 100.
FDM (ours)
.117, 11.1
.117, 9.00
.138, 25.1
.153, 36.2
.191, 66.5
SigKer
TruncSig
.766, 100.
.743, 100.
.670, 100.
.998, 100.
1.00, 100.
USD/IPY
SDE-GAN
.528, 100.
.291, 100.
.389, 100.
.530, 100.
.655, 100.
FDM (ours)
.138, 20.9
.124, 14.3
.150, 32.1
.199, 74.9
.2600 97.9
Table 9: Average KS test scores and the chance of rejecting the null hypothesis (%) 5%-
significance level on marginals across all dimensions, trained on paths evenly sampled at 64 times- tamps from 16-dimension rough Bergomi model.
Model
t == 6
t = 19
t = 32
t = 44
t = 57
SigKer
.112, 6.60
.188, 7.80
.124, 10.8
.132 16.3
144, 25.5
TruneSig
.450, 100.
.458, 100.
.462, 100.
.461, 100.
.460, 100.
SDE-GAN
.30,, 99.8
.374, 99.4
.393, 99.5
.406, 99.6
.430, 99.7
FDM (ours)
.113, 7.20
.116, 7.80
.119, 8.80
.124, 11.8
.131, 15.8
in Tables and 10 where report the average KS scores and the chance of rejecting the null hypothesis across different dimensions. We also compare the computational efficiency of the models terms of training time for different dimensions of the Rough Bergomi model, with detailed results summarized in Table 17 in the appendix. We highlight the best-performing model across all tables.
We include qualitative studies in Figure which compare the dynamics of joint distributions of real and generated data points for the metal price dataset. We compare the sample paths of the metal price dataset Figure Due to space constraints, we provide further qualitative studies comparing pairwise joint distributions and sample paths, along with tables comparing computational efficiency, in section G in the appendix. Our results demonstrate that our method outperforms competitors in an overwhelming majority of cases in terms of KS test results, qualitative results, and computational efficiency.
7 CONCLUSION, LIMITATIONS, AND FUTURE WORK
Our main theorem demonstrates that any strictly proper scoring rule for comparing distributions on finite dimensions be extended strictly proper scoring rules for comparing the laws of con- tinuous Markov processes. This theorem naturally leads to the FDM algorithm for training Neural SDEs. We empirically show that FDM outperforms current state-of-the-ar methods for training Neural SDEs, both in terms of generative quality and computational efficiency. However, the ap- plicability of our main theorem is currently constrained by the assumptions of continuity and the Markov property. Although this lies beyond the scope of Neural SDEs, we provide a straightfor- ward extension of the main theorem to Cädlàg Markov processes in the appendix. This extension broadens the applicability of FDM to wider range of models, including jump processes. Fur- thermore, an intriguing direction for future work would be relax the Markov assumptions, for instance, by incorporating hidden Markov models.

---

Published as a conference paper at ICLR 2025
Table 10: Average KS test scores and the chance of rejecting the null hypothesis (%) at 5%c significance level on marginals across all dimensions, trained on paths evenly sampled 64 times- lamps from a 32-dimension rough Bergomi model. TruncSig runs out of GPU memory.
Model
t=6
1=1
1=32
1-44
t=57
SigKer
.120, 11.1
.137, 18.5
.149, 26.5
.157, 35.3
168, 45.2
TruncSig
SDE-GAN
284, 99.8
.288. 99.7
.298, 99.8
.311, 99.9
.326, 100.
FDM (ours)
.117, 9.10
.119, 10.2
.122, 11.4
.124, 13.0
.128, 15.4
t=6
t=19
t=32
t=44
t=57
FDM SDE-GAN Truncated Signature
Kernel
-10
Signature
Gold
-10
-5
-5
5
-5
-5
5
-5
5
Silver
Figure 1: Blue points real samples and orange points generated by Neural SDEs. The dynamics of the joint distribution of gold and silver prices in the metal price data. Each row of plots corresponds method and each row corresponds to timestamp. For each plot, the horizontal axis is the silver price and the vertical axis is the gold price.
X
Pxte
Fxton
Pxtns
-6
Pxe
4x
Mx
Pxo
Pxton
Pxtmn Px
Pxtne Px
Px
Px
Px
X
Figure 2: Sample paths for silver (top) and gold (bottom) prices from metal dataset. Blue lines represent real samples, while red lines represent those generated by Neural SDEs. From left right, the plots correspond to signature kernels, truncated signature, SDE-GAN, and FDM, respectively. The horizontal axis represents time, and the vertical axis represents metal prices.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial net- works. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Con- ference on Machine Learning, volume 70 of Proceedings of Machine Learming Research, pp. 214 223. PMLR, 06-11 Aug 2017. URL htttp::/pproceddiiii wtwp:eeeddinn............m
ml wtw.sm.......s/////mom
ar jrjovsky1aa.he html
Imanol Perez Arribas, Cristopher Salvi, and Lukasz Szpruch. Sig-sdes model for quantitative fi-
nance. In Proceedings of the First ACM International Conference on Al in Finance, ICAIF '20, New York, USA, 2021. Association for Computing Machinery. ISBN 9781450375849, doi: 10.1145/3334555:44255555 URL btttt::://oott org/10. / 114453384555.34255533 3422553
Patric Bonnier and Harald Oberhauser. Proper Scoring Rules, Gradients, Divergences, and Entropies for Paths and Time Series. Bayesian Analysis, pp. 32, 2024. doi: 10. 10.121//44-B14455. URL
/tttps://dii.rre http////....//11111111122221 1214/24 1111//44-8144333 BA1435
Diane Bouchacourt, M. Pawan Kumar, and Sebastian Nowozin. Disco nets: dissimilarity coefficient networks. In Proceedings of the 30th International Conference on Neural Information Process- ing Systems, NIPS' 16, pp. 352-360, Red Hook, NY, USA, 2016. Curran Associates Inc. ISBN
9781510838819.
Vedant Choudhary, Scbastian Jaimungal, and Maxime Bergeron. Funvol: A multi-asset implied volatility market simulator using functional principal components and neural sdes, 2023, URL
https://arxix: http://wwwwrra///////////// 00859
Patryk Gierjatowicz, Marc Sabate-Vidales, David Šiška, Lukasz Szpruch, and Zan Zurič. Robust pricing and hedging via neural sdes, 2020.
Tilmann Gneiting and Adrian Raftery. Strictly proper scoring rules, prediction, and estima- tion. Journal of the American Statistical Association, 102(477):359-33) 378, 2007. doi: 10.1 198/ 016214506000004477 htt.ttts:://dot /0 621 01111///000011110000000000000000
Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. Journal Machine Learning Research, 13(25):723-773, 2012. URL
hattp://mmi..rgg
Alexey A. Gritsenko, Tim Salimans, Rianne van den Berg, Jasper Snoek, and Nal Kalchbrenner. A spectral energy distance for parallel speech synthesis. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS '20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546.
Liam Hodgkinson, Chris van der Heide, Fred Roosta, and Michael W. Mahoney. Stochastic contin- uous normalizing flows: training SDEs as ODEs. In Cassio de Campos and Marloes H. Maathuis (eds.), Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence, volume 161 of Proceedings of Machine Learning Research, pp. 1130-1 140. PMLR, 27-30 Jul 2021. URL
.tml
Melker Hoglund, Emilio Ferrucci, Camilo Hernandez, Aitor Muguruza Gonzalez, Cristopher Salvi, Leandro Sanchez-Bett Betancourt, and Yufei Zhang. A neural rde approach for continuous-time non-markovian stochastic control problems, 2023. URL http://////awww.ooo//b//////////
14258
Zacharia Issa, Blanka Horvath, Maud Lemercier, and Cristopher Salvi. Non-adversarial training of neural sdes with signature kernel scores. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural In- formation Processing Systems, volume 36, pp. 11102-11126. Curran Associates, Inc., 2023. URL. htttps:/pprocess ngs. neurips co http:/pppprrrfllsspppppppp/p File
www.adlllee....... o2gp Canapprr--ooffeenceeeeee .ddf
Junteng Jia and Austin R Benson. Neural jump stochastic differential equations. In H. Wal- lach, H. Larochelle, A. Beygelzimer, F. d'Alché-Bus E. Fox, and R. Gareet (eds.), Ad- vances in Neural Information Processing Systems 32, pp. 9847-9858. Curran Associates, Inc.,

---

Published as a conference paper at ICLR 2025
differential-equaaiins htt.pp/...opppp....aaaaaaaaaooar
2019. URL
Olav Kallenberg. Foundations of Modern Probability. Number 75 in Probability Theory and
Stochastic Modelling. Springer Cham, 3 edition, 2021. ISBN 978-3-030-61871-L doi: 10.1007/
http:/t:::/ /otp...rr/...11777777-6---------
Patrick Kidger. On neural differential equations, 2022.
URL https://arxiv.org:
22C2.02433
Patrick Kidger, James Morrill, James Foster, and Terry Lyons. Neural Controlled Differential Equa- tions for Irregular Time Series. Advances in Neural Information Processing Systems, 2020.
Patrick Kidger, James Foster, Xuechen Li, and Terry J Lyons. Neural sdes as infmiitediimensional gans. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Confer- ence on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 5453-5463. PMLR, 18 24 Jul 2021. URL https:t//pooceeeeeeee kidger21b. html
Darrick Lee and Harald Oberhauser. The signature kernel, 2023. URL https::/arxiv..o abs/2305. 04625
Xuechen Li, Ting-Kam Kam Leonard Wong, Ricky T. Q. Chen, and David K. Duvenaud. Scalable gra- dients and variational inference for stochastic differential equations. In Cheng Zhang, Francisco Ruiz, Thang Bui, Adji Bousso Dieng, and Dawen Liang (eds.), Proceedings of The 2nd Sympo- sium on Advances in Approximate Bayesian Inference, volume 118 of Proceedings of Machine Learning Research, pp. 1-28. PMLR, 08 Dec 2020. URL httpp:://pooceeeiinnggg .mm..... mlr.. press/v118/1i206 html
Xuerong Mao. Stochastic Differential Equations and Applications. Horwood Publishing, Chichester, UK, 2nd edition, 2007. ISBN -77-5-994475--2-55.
Colin McDiarmid. On the method of bounded differences. In Surveys in combinatorics, 1989, volume 141 of London Mathematical Society Lecture Note Series, pp. 148-188. Cambridge Uni- versity Press, 1989.
James Morrill, Cristopher Salvi, Patrick Kidger, James Foster, and Terry Lyons. Neural rough differential equations for long time series. In International Conference Machine Learning, 2020. URL htttps:/:// wtt.:pp...smmaanncccccccceeceoee hat.aaar..orr/ooppsDD:223333
James Morrill, Cristopher Salvi, Patrick Kidger, James Foster, and Terry Lyons Neural rough differential equations for long time series. International Conference on Machine Learning, 2021.
Hao Ni, Lukasz Szpruch, Marc Vidales, Baoren Xiao, Magnus Wiese, and Shujian Liao. Sig- wasserstein gans for time series generation. In Proceedings of the Second ACM International Conference on Al in Finance, ICAIF '21, New York, NY, USA, 2022. Association for Computing Machinery. ISBN 47814503114
doi: 10. 10.11455/449035444444933
URL https://dol.
orq/10. 1145/11555 34903544334..4 1104144433344444333333
Manfred Opper.
Variational inference for stochastic differential equations. Annalen der Physik, 13003):1800333 2019.
doi: http://ddo..org//1...00///////// 20180033.... 800233.
URL https://
onlinel ibrary. www.wwlww....ooo.. wwt. http:////o////aa///../////////aa
Lorenzo Pacchiardi and Ritabrata Dutta. Likelihood-free infereeee with gemnerttttt neural networks via scoring rule minimization, 2022. URL htt.:pp://aaxxv.orrr//oo////////
Lorenzo Pacchiardi, Rilwan A. Adewoyin. Peter Dueben, and Ritabrata Dutta. Probabilistic fore- casting with generative networks via scoring rule minimization. Journal of Machine Learning Research, 25(45)11644 2024. URL
wttp://.rrrppppps/////222222 20388 htm)
Cristopher Salvi, Thomas Cass, James Foster, Temry Lyons, and Weixin Yang. The signature kernll is the solution of a goursat pde. SIAM Journal on Mathematics of Data Science, 3(3):873- 899, 2021. doi: 10. 0111/2001336667 URL URR.tttss:... 0tte//1..11333//213333333
12

---

Published as a conference paper at ICLR 2025
Luke Snow and Vikram Krishnamurthy. Efficient neural sde training using wiener-space cubature, 2025.
Belinda Tzen and Maxim Raginsky. Neural stochastic differential equations: Deep latent gaussian models in the diffusion limit, 2019, UtL/tttps://axxx..orrs http://wwww...r///a///////9999 09883
Jinsung Yoon, Daniel Jarrett, and Mihaela van der Schaar. Time-series generative adversar- ial networks. In H. Wallach, H. Larochelle, A. Beygelzimer, F. /Alché-Buc, E. Fox, and R. Gamett (eds.), Advances Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https:: Saaprr/000044iseeeeee/aaaaaaeee2 Paw.ee.....pp...............
wwwwips.c./paaaeeeeeeeee files/
13


Reviews:
[{"review_id": "JXEcfWl4J4", "reviewer": "Reviewer_GLJM", "summary": "This paper proposes a new method called Finite Dimensional Matching (FDM) for training Neural SDEs by identifying a class of strictly proper scoring rules for comparing continuous Markov processes. Using this scoring rule, they claim to reduce the training complexity from quadratic in discretization timesteps to linear.", "questions": "In Theorem 2, does the result hold for any scoring rule $s$, or does $s$ also need to be strictly proper? Could you clarify if there are specific conditions on $s$ required for Theorem 2 to apply?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "This paper addresses an important problem: the high computational complexity (quadratic in time steps) associated with training Neural SDEs using scoring rules. The authors propose a reduced complexity method, aiming for linear complexity to enhance performance. They also provide a theoretically grounded approach in designing a new scoring rule. However, despite this strong motivation, the results are somewhat unconvincing due to certain weaknesses noted below.", "weaknesses": "This paper has several notable weaknesses. While the authors propose a reduced complexity approach for training Neural SDEs, they do not adequately explain key concepts, making the paper difficult to follow, especially for readers without a strong background in this area. For example, despite experience with SDEs in score-based generative models and deep learning theory, I found the explanations lacking in detail and context and some results are not convincing.\n\nThe authors do not provide sufficient preliminary material on scoring rules or background on how these rules are used to measure divergence between two Markov processes. A concrete example of the scoring rule $s(P, z)$ with an RBF kernel, presented early on, would have improved clarity.\n\nAdditionally, the complexity reduction claim is unconvincing. For example, at the beginning, the authors claim they reduce the complexity to linear using $D$ to denote the time steps, however, this notation $D$ is never used again in the rest of the paper. Instead, in Section 4.2 Algorithm, they compare two stochastic processes and use $B$ to denote the total number of time steps. However, the nested summations in the top equation on page 5 suggest quadratic rather than linear complexity, i.e., $B^2$.\n\nFinally, the paper uses the **ICLR 2024 format rather than the ICLR 2025** format.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LJD22sR0Oz", "reviewer": "Reviewer_SGBZ", "summary": "This paper introduces a very simple yet elegant way of comparing the similarity of the laws of two homogeneous Markov processes. The idea follows from the fact that the distribution of the process is completely determined by the transition kernel. Hence the ability to compare the transition distribution should give enable a comparison between the laws. \n\nThe authors then use this finding as a scoring rule to learn to simulate from a neural SDE, given the observations from a ground truth process. Then, they test their scoring rule and their FDD matching procedure in numerical experiments, showing that their method outperforms existing law matching techniques.", "questions": "The main concern I had was in regard to the proof. Please help me to understand or fix the issue. \n\nIf this and the writing issues I raised above can be addressed, I will happily give you at least a 6. However, since I feel that your main result is wrong, I have to give a low score for now.\n\n**Update: I was wrong about the validity of the proof. The result is sound. **\n\nIf this (or a variant of the) scoring rule for Markov processes is indeed correct, I think the authors could improve the paper by exploring the sensitivity properties of this scoring rule; for example, what kernel to use and how the score behaves when P and Q are close? Is there a simple formula to compute the gradient?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 2, "contribution": 3, "strengths": "The method is very simple and elegant and the implication of this result, **if true**, could be impactful to the neural SDE and generative modeling community.\n\n**Update: I was wrong about the validity of the proof. The result is sound. **", "weaknesses": "I think there is a significant issue in the proof that could invalidate your main theorem 2. In the middle part of page 14, you have the integral over $t_1,t_2$ of the $S(P_{t_1,t_2}, P_{t_1,t_2}')$ scoring rules being equal, but then you conclude that $$S(P_{t_1,t_2},P_{t_1,t_2}') = S(P_{t_1,t_2},P_{t_1,t_2})$$ a.e.. This is not true or I am missing something? \n\n**Update: I was wrong about the validity of the proof. The result is sound. **\n\nMy intuition is that this is not an easy fix: You want to match the distribution over all $t_1,t_2$, so, in some sense, you want the expectation equality to hold for all test measures $\\nu$ instead of just one particular $\\nu$. I would be very happy to raise the score and rewrite my review if I am wrong. However, it does seem that proving a result like yours is possible, given that the generator or the resolvent will determine the law of the Markov process (see Either and Kurtz). \n\nI appreciate the rigor in defining the math notations and results. But the writing and explanations in the paper need improvements. For example, what do you mean by \"Update the model parameters $\\theta$ through backpropogation to maximize $\\hat S$\" (inside the algorithm)? Also, there is no explanation of what the data is. What are the \"Average KS test scores\"? Are you repeating the experiment across multiple batches and producing the percentage of rejection \"chance of rejecting the null hypothesis (%) at 5%-significance level on marginal\"? \n\nOther minor issues include: \n1. I think you need $\\mathcal{E}$ to be Polish. \n2. The radial basis function (RBF) kernel is not defined. \n3. The $\\pi$ notation seems a little distracting, why not just use $(x_t,x_s)$ for $\\pi_{t,s}(x)$. \n4. It would be nice to emphasize that the Markov processes you define are homogeneous Markov processes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tzblZicoc8", "reviewer": "Reviewer_DrUo", "summary": "The authors propose a scoring rule for continuous time stochastic processes that is directly derived from a scoring rule on a generic space. They show that this rule is proper (i.e. injective from the space of paths to the space of laws), which is a non-trivial contribution. Experiments show that this method outperforms existing concurrent methods based on signature kernels and SDE-GANs. I stress that the experiments are carried out on a vast array of datasets.", "questions": "* Please include confidence intervals in your tables: variability of your results is a very important aspect.\n\n* Could you please include vizualizations of the full generated time series, rather than only plotting the time marginals ?\n\n* Could you add experiments on at least one high dimensional and one non euclidian real-world dataset ?\n\nI would considered increasing my score if a significant number of concerns on the experiments are addressed. Similarly, I could consider lowering my score during the discussion phase if other reviewers relativise the strength of the theoretical contribution --- which again seems sufficient to me.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 2, "contribution": 3, "strengths": "* The paper is well written and structured, making it easy to read and to follow. The introduction is sound and features a thorough literature review.\n \n* The main technical contribution is Theorem 2, which allows to convert any scoring rule on a generic space. I believe this contribution to be non trivial and novel, although simple to prove.  \n\n* The experiments are carried out on a vast array of time series datasets and show overall superior performance of the proposed approach. The authors compare themselves to all relevant baselines to the best of my knowledge. Large generative models such as diffusion models are not included ; however, I do believe that they do not belong to the same class of models and do not require a comparable computational budget.", "weaknesses": "* While the contribution made in Theorem 2 is elegant and novel, one could object that it is slightly insufficient --- this is not my point of view, but another reviewer might disagree and I am willing to discuss this point. In order to strengthen their theoretical part, I encourage the authors to consider for instance the sample complexity of their kernel i.e. how fast does the empirical divergence they define through a kernel on $\\mathcal{E}$ converge through the expected divergence ? Do sample complexities in $\\mathcal{E}$ carry over to the space of paths ? See Gretton (2012) for an example. \n\n* A blind spot of the paper is in my opinion the choice of the kernel. The authors do not seem to consider other kernels for $\\mathbb{R}^d$ valued processes, which could yield an interesting extension. This might especially be interesting since some kernels are sometimes used because of their specific properties (invariances, ...). I would suggest that the authors consider this point, at least in the Appendix. \n\n* This approach nicely extends to kernels defined on any space - this could be graphs, images, etc. and could allow to generate time series in these spaces. This would provide, in my sens, a extremely valuable extension to the paper. \n\n* The experimental section is hard to read and a tad unstructured. I encourage the authors to use less tables, add more comments and broaden the analysis of their experiments. Also, a notable restriction of this part is that experiments are only carried out on $2$ dimensional time series. I strongly encourage the authors to extend their experiments to high dimensional datasets. Also, there are no confidence intervals in the tables. \n\n* Regarding this last point, I believe that a valuable extension could be to consider random feature approximations to the kernels for high dimensional generation, which is still a major hurdle in the field. Similarly, the authors could consider sliced kernels on $\\mathcal{E}$ when the dimension is high.  \n\n* Concerning experiments, an interesting task to consider could be the augmentation of a time-series dataset, and the analysis of the gain in performance for any model trained on this dataset. \n\n* Concerning experiments, I believe that it would be highly valuable to extend applications beyond finance. Generating time series is a major hurdle in many domains with great social impact such as neuroscience, healthcare, biology, climatology, economics ... \n\n* A valuable extension of this work would be the investigate the use of the devised score for other purposes than training generative models, such as two-sample tests for instance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hfTFiU93yU", "reviewer": "Reviewer_4f46", "summary": "This paper presents a novel approach to training neural stochastic differential equations by introducing Finite Dimensional Matching, a new scoring rule designed for continuous Markov processes. The proposed method leverages the Markov property of stochastic processes to reduce the computational complexity of training neural SDEs, with the goal of enhancing both efficiency and generative quality. Theoretical contributions establish that the new scoring rule provides a strictly proper method for comparing two-time joint distributions. Experimental results demonstrate improved performance in training efficiency and generative quality when evaluated against competing methods.", "questions": "Please see weaknesses.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 1, "presentation": 2, "contribution": 2, "strengths": "The paper tackles a crucial challenge in training neural SDEs by introducing a new scoring rule that optimizes efficiency for continuous Markov processes. The proposed FDM method is backed by mathematical proofs, providing a strictly proper scoring rule that extends from finite-dimensional distributions to continuous Markov processes. This theoretical contribution is valuable to the literature on neural SDE training methods. Experiments show that FDM offers computational efficiency gains, reducing training complexity from quadratic to linear in the number of discretization steps. The approach outperforms prior methods in computational efficiency, as shown in multiple experimental benchmarks.", "weaknesses": "The paper’s main theorem relies on strong assumptions regarding the Markovian properties and continuity of the processes involved. These assumptions may limit the applicability of the FDM algorithm in more complex, non-Markovian stochastic processes or those with jumps, which are common in real-world scenarios. Explicitly discussing these limitations and potential ways to relax these assumptions would make the contributions more transparent.\n\nAlso, the writing template is for ICLR 2024, not ICLR 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)