Higher Order Kernel Mean Embeddings to Capture Filtrations of Stochastic Processes
Authors: Cristopher Salvi
Maud Lemercier
Chong Liu
Blanka Horvath
Theo Damoulas
Terry Lyons
Venue: NeurIPS 2021 Poster (2021)
Keywords: kernel mean embedding
stochastic process
filtration
maximum mean discrepancy
signature transform.

Abstract: Stochastic processes are random variables with values in some space of paths. However, reducing a stochastic process to a path-valued random variable ignores its filtration, i.e. the flow of information carried by the process through time. By conditioning the process on its filtration, we introduce a family of higher order kernel mean embeddings (KMEs) that generalizes the notion of KME to capture additional information related to the filtration. We derive empirical estimators for the associated higher order maximum mean discrepancies (MMDs) and prove consistency. We then construct a filtration-sensitive kernel two-sample test able to capture information that gets missed by the standard MMD test. In addition, leveraging our higher order MMDs we construct a family of universal kernels on stochastic processes that allows to solve real-world calibration and optimal stopping problems in quantitative finance (such as the pricing of American options) via classical kernel-based regression methods. Finally, adapting existing tests for conditional independence to the case of stochastic processes, we design a causal-discovery algorithm to recover the causal graph of structural dependencies among interacting bodies solely from observations of their multidimensional trajectories.

Higher Order Kernel Mean Embeddings to Capture Filtrations of Stochastic Processes
Cristopher Salvi
Maud Lemercier maud. lemercier@rarwick. ac. uk
cristopher halviomathon ox. ac uk
Chong Liu 3
Blanka Horvath
chong wwwwmaths..... o..
hanka..oovathhkee.. ac.
Theodoros Damoulas 2
Terry Lyons terry 1yons@maths ac. uk
t. www.ddaaalassscc www.woiwsl@wwwwickkkkkk.k.. ac. uk
Abstract
Stochastic processes are random variables with values in some space of paths. However, reducing a stochastic process to path-valued random variable ignores filtration, i.e. the flow of information carried by the process through time. By conditioning the process on its filtration, we introduce family of higher order kernel mean embeddings (KMEs) that generalizes the notion of KME and captures additional information related to the filtration. We derive empirical estimators for the associated higher order maximum mean discrepancies (MMDs) and prove consistency. We then construct filtration-sensitive kernel two-sample test able to pick up information that gets missed by the standard MMD test. In addition, leveraging our higher order MMDs we construct family of universal kernels on stochastic processes that allows to solve real-world calibration and optimal stopping problems in quantitative finance (such the pricing of American options) via classical kernel-based regression methods. Finally, adapting existing tests for conditional independence the case of stochastic processes, design causal- discovery algorithm to recover the causal graph of structural dependencies among interacting bodies solely from observations of their multidimensional trajectories.
Introduction
The idea of embedding probability distributions into reproducing kernel Hilbert space (RKHS) via kernel mean embeddings (KMEs) has become ubiquitous many areas of statistics and data science such as hypothesis testing [1. 2], non-linear regression [3, 4], distribution regression [5, 6] etc. Despite strong progress in the study of KMEs, most of the examples considered in the literature tend to focus on random variables supported on some finite (possibly high) dimensional euclidean spaces like Rdd. The study of KMEs for function-valuee random variables has been largely ignored.
Stochastic processes are random variables with values in some space of paths. However, reducing stochastic process to a path-valued random variable ignores its filtration, which can be informally thought as the flow of information carried by the process through time. question that naturally
University of Oxford The Alan Turing Institute
2 University of Warwick & The Alan Turing Institute
University of Oxford
4 'King's s College London, The Alan Turing Institute & Technical University Munich
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

sample path from X
filtration Fx.
sample from PX|Fx,
X
V
time
Fx
Px
PX|Xx
Hs(V)
Figure I:
Schematic overview of the construction of the 1st order predictive KME | (Sec. 3.2). Here X is stochastic process with sample paths taking their values in V. The red contours indicate the portion of its filtration Fx upon which the conditioning is applied, i.e. the available information about X from start up time f. As explained in Sec. 3.2, the 1st order predictive KME 1x|=x is path whose value at time (V)---ed random variable representing the law of X conditioned on its filtration Fx,- Equivalently xX|=x a stochastic process with sample paths taking their values in (V)
emerges from the study of many random, time-evolving systems like financial markets is how does the information available up to present time affect the future evolution of the system?
Formally, this question can be addressed by conditioning a process on its filtration (Sec. 3.1 and 3.2). In this paper we introduce a l'amily of higher order KMEs that generalizes the notion of KME to capture additional, filtaationrrelaedd information (Sec. 3.3 and 3.5). In view of concrete applications, we derive empirical estimators for the associated higher order MMDs and use of them to construct a filtration Altration-sensitive sensitive kernel wwo-sample test (Sec. 3.4) demonstrating with simulated data its ability to capture information that otherwise gets missed by the standard MMD test (Sec. 4.1). Furthermore, we construct a family of universal kernels on stochastic processes (Sec. 3.6) that allows to solve challenging, real-world optimisation problems in quantitative finance such as the pricing of American options via classical kernel-based regression methods (Sec. 4.2). Finally, we adapt existing tests for conditional independence to the case of stochastic processes in order to design a causal-dine -diso-drsy algorithm able to recover the causal graph of structural dependencies among interacting bodies solely from observations of their multidimensional trajectories (Sec. 4.3).
1.1 Related work
The notion conditioning a powerful probabilistic tool allowing to understand possibly complex, non-linear interactions between random variables. As their unconditional counterparts, conditional distributions can also be embedded into RKHSs [7]. Recently, conditional KMES have received in- creased attention, especially in the context of graphical models [8], state-space models [9], dynamical systems [10], causal inference [11, 12, 13], two-sample and conditional independence hypothesis testing [14. 13, 15] and others. Embeddings of distributions via KMEs have also shown their success in the context of distribution regression (DR), which is the task of learning a function mapping a collection of samples from probability distribution to scalar targets [16, 17, 18]. More recently, a framework for DR that addresses the setting where inputs are sample paths from an underlying stochastic process is proposed in [6]. The authors make extensive of the signature transform [19, 20) and of the signature kernel [21, 22], two well established tools in stochastic analysis.
When it comes to stochastic processes, it was first shown in [23] that weak convergence of random variables does not always account for the information contained in the filtration, as highlighted by means of numerous numerical examples in [24, 25]. This limitation is addressed in [23, 26] through

---

the construction of a sequence of so- so-called adapted topologies (Tn) that become progressively finer as n increases (with 71 coinciding with the weak topology). In particular, higher order adapted topologies are shown capture more filtration-related information than their weak counterpart. This characteristic becomes relevant for example in some optimal stopping problems such as the pricing of American options, where the pricing function can be shown to be discontinuous with respect to the weak topology, but is continuous with respect to the second order adapted topology² [24, 28, 27]. Leveraging properties of the signature transform, it has been shown that adapted topologies are intimately related to a family of higher order MMDs [29]. However, providing empirical estimators for these discrepancies that can deployed on real-world tasks remains a challenge. we propose to address this challenge by presenting an alternative construction to this higher order MMDs using the language of kernels and KMEs. The results in [29] serve as a strong theoretical background for the present paper.
this paper
2
Preliminaries
We begin with a brief summary of tools from stochastic analysis needed define higher order KMEs. Let (((I = {i [0, + R² compact set of continuous, piecewise linear, R -aluee paths defined over a common time interval [0, 7". obtained for example by linearly interpolating multivariate time series. For any path æ E X' (R we denote its kthh coordinate by x! [0, 7" + R.
for & {1..... d}. More generally we denote by (VV) {z [0, T] + V} a compact set of continuous, piecewise linear paths with values in a Hilbert space with a countable basis.
2.1 The signature transform
The signature transform S xVV) + Hs(V) is a feature map defined for any path x E XV) as the following infinite collection of statisties
S(x) = (1, (s)))
M1=1" { {((x))(i d Ny J
J
(1)
where each term is a real number equal to the iterated integral
((z))kammkk))
dx
The signature feature space Hs(V) is defined as the following direct sum of tensor powers of V
Hs(V)=
+
R + V 4 (V) +
(3)
k=
where & denotes the standard tensor product of vector spaces [30, 19].
2.2 The signature kernel
Because V is Hilbert Hs(V) is also Hilbert [21]. The signature kernel ks X(V) x X(V) + R is a characteristic kernel defined for any pair of paths x, y X(V) as the following inner product
ks (s(z, y) ( Hs(V)
(4)
The recent article [22] establishes a surprising connection between the signature kernel and certain class of partial differential equations (PDEs), culminating in the following kernel trick for ks.
Theorem 1. /22, Thm. 2.5] For any x, y E X(V) the signature kernel satisfies the equation ks( y) = Ur (T, (T,T), where Uz.9 10, T x [0, IR is the solution of the hyperbolic PDE
8 Uz.y = (i.s Dtl Uz.g
(5)
Osôt
with boundary conditions Uz,,, (0,-) = U2 (z = and where i.
dec dr
r=="
We say that sequence of random variables ( converges to random variable X in the topology if and only if for every 7-open neighbourhood (U of X there exists N N such that X.. U as
2 N.
A topology Ti is said to be finer than a topology To if every 72-open set is also 7i -oen..
? The second order adapted topology equivalent to the adapted Wasserstein distance [27].

---

Hence, evaluating ks at pair of paths (z, is equivalent to solving the PDE (5); in this paper we solve PDEs numerically via finite difference scheme (see Appendix for additional details). In what follows, we denote by (V) the RKHS associated to ks.
2.3 Stochastic processes and filtrations
We take PP) as the underlying probability space, A (discrete stochastic process X is random variable with values on X(V). We denote by Ixx = Po X- the law X. Assuming the integrability condition Ex [ks (X, x)] < oo, the 1sd order kernel mean embedding (KME) of X is defined as the following point in (V)
x\ (x = - =EX(V))
(s(x,-)D (dx)
(6)
Accordingly, given two stochastic processes X, Y, their 1s order maximum mean discrepancy (MMD) the standard MMD distance with kernel ks given by the following expression
D) = || py (ls(())
(7)
Because the signature kernel ks is characteristic, it is a classical result 31] that the 1st order MMD a sufficient statistics to distinguish between the laws of and Y, other words
(), = + Px===
(8)
Despite the fact that stochastic processes are valued random variables, they encode much richer structure compared to standard I2 -aluud random variables, that goes well beyond their laws. This additional structure is described mathematically by the concept of filtration of process X, defined as the following family of o-algebras
Fx (===Fxx)tee )EJ
where for any E [x. is the o-algebra generated by the variables {x..e. Note that Fx is totally ordered in the sense that Fx, C Fx, for all s < t, which naturally explains why filtrations are good mathematical descriptions to model the flow information carried by the process X.
In the next section, we will present our main findings and introduce family of higher order KMEs and corresponding higher order MMDs as generalizations of the standard KME and MMD respectively. We will do so by conditioning stochastic processes on elements of their filtrations.
3 Higher order kernel mean embeddings
We begin by describing how KMEs can be extended to conditional laws of stochastic processes.
3.1 Conditional kernel mean embeddings for stochastic processes
Let X,Y be stochastic processes. For given path XVV), define the 1st order conditional kernel mean embeddings 1r|x== E (V) and -bxx Hs(V) + Hss(V) as follows
x f vEX(v) ks kx
My x = El )] (Y,,(
(10)
/ ) ks(:, Bs(::))Pyxx(dl
My x = (Y, |x|]
(11)
Note that whilst => is single point in HV)(V, the 1s order conditional KME pbyxx deseribes a cloud of points on (V) Each point in this cloud is indexed by path x E XVV). Equivalently, " constitutes a IPxx measurable, H.s vva-ule random variable.
These embeddings allow to extend the applications of conditional KMES to the case where the random variables are (possibly multidimensional) stochastic processes. particular one can directly obtain conditional independence criterions for stochastic processes (see Appendix A.1), enabling to deploy standard kernel-based causal learning algorithms [13], as we demonstrate in Sec. 4. Next we describe how in the case of stochastic processes, conditioning filtrations is important mathematical operation to model real-world time-evolving systems.
'The 1s order KME is the standard KME with the signature kernel ks.

---

3.2 Conditioning stochastic processes on their filtrations
Financial markets are examples of complex dynamical systems that evolve under the influence of randomness. An important objective for financial practitioners is to determine how actionable information available up to present could affect the future market trajectories. The task of conditioning on the past to describe the future of stochastic process X can be formulated mathematically by conditioning X on its filtration Fx, for any time E [0, T).
More precisely, consider the 1st order KME of the conditional law Px|Exx which is defined as the following F Fx, -measurable, H.s H.s(V)-valeed random variable
XX|Fx. (X, (o,)) (ZEX(V)
ks(: ks(:,2)xxi7xx (dz)
(12)
where X [1o denotes the stochastic process X restricted to the sub-interval [0,4] 4] C [0, By varying the time index i, we can form the following ordered collection of 1st order KMEs
MXIFx (MXX xE[0,T]
(13)
that we term 1 order predictive KME of the process X. By construction, Fx|/F describes path taking its values in the space of H.s (Vss((V--vlledd random variables, in other words it is itself stochastic process 9 (see Fig. 1). Hence, the law of |xx] can itself be embedded via KMEs into a "higher-order RKHS" (see next section), making the full procedure iterable, as we shall discuss next.
We note that for each time t, the random variable 1x|Tx, in eq. (12) is the Bochner integral of ks(+, with respect to the probability measure Px|Ex, Since we assumed that is a compact set, the path space XVVV) is also compact. Hence, the function x + ks(:;2) 2) is continuous, the set K = (+s(; x XVV) is compact continuous image of compact set, and therefore its Bochner integral Mx|xx, takes values in the closed convex hull of K, which is again compact subset in the RKHS HV) Consequently the path + px),,, belongs to a compact subset of (V)), which satisfies the assumptions introduced in Section 2.
3.3 Second order kernel mean embedding and maximum mean discrepancy
The 2nd order KME is the point in (ss(Hs(V)) defined as the KME of the 1st order predictive KME
x [
( ks(, Pk p (dx)
(14)
The 2nd order MMD of X, Y is the norm of the difference in (s()ss(V) of their 2nd order KMEs,
D)(X = || 12 (H(s(()))
(15)
The next theorem states that the 2nd order MMD of two stochastic processes X, Y is stronger discrepancy measure than the 1st order MMD. Theorem 2. Given stochastic processes X,Y +
D) =0 + Px|Fx = Pr
(16)
Furthermore
D3(X,Y)=0 =0 Y))
(17)
but the converse is not generally true.
Proof. All proof' are given in Appendix D.
Next we make use of Thm. 2 in the context of two-sample hypothesis testing [1. 31] for stochastic processes, In Sec. 4 we will show by means of a numerical example that the 2 order MMD is able to capture filtration-related information otherwise ignored by the 1st order MMD.
Because Px PxIFxy. all the information about the law of X is contained in just the terminal point of the trajectory traced by pxxTx (Fig. l).
5

---

3.4 A filtration-sensitive kernel two-sample test
Suppose we are given m sample paths {- X and n sample paths (1/ Y. A classical two-sample test [1] for X, Y tests null-hypothesis
Ho : Ixx = Py against the alternative HA Px
(18)
The probability of falsely rejecting the null is called the type I error (and similarly the probability of falsely accepting the null is called the type I| error). If the type error can be bounded from above by constant a, then say that the test is of level a. In [31, Sec. 8] it is shown that rejecting the null if (X, Y)² > Ca (for some ca that depends on m, n and a)) gives a test of level a, where (X, Y) denotes the classical unbiased estimator of the 1st order MMD [1]. This choice of threshold is conservative and can improved by using data-dependent bounds such as in permutation tests (we refer to the MMD testing literature for extra details [1, 32, 33)).
However, discussed in Sec. 3.3 comparing the laws xx: Py via the estimator above might be insufficient to capture filtration-related information about X, Y. To overcome this limitation we propose instead test the null-hypothesis
(19) Using Thm. 2, one can immediately construct a fllrraton-seensiivve kernel two-sample test for (19) provided one build empirical estimator of the 2nd order MMD (X, Y). In the rest of this section we explain how to obtain such an estimator and ultimately show its consistency.
Ho IP Px|xx = Pylrr against the alternative НA : Px|Fx # Pylrr
Assuming availability of m sample paths (3 Me from the stochastic process "X]=x and sample paths {7 1i from Hyps. an estimator of the squared 2nd order MMD is given by
m.n
(x(X = m(m
ks
ks(F,
+ n(n 1)
ks
Computing this estimator boils down evaluating the signature kernel ks D) on sample paths 1xxFxx and ÿ ~ ply Fy By Thm. 1, the signature kernel solves the following PDE
Pua.ū = (( Tc-8) Ho()) - (2.-6. D6)-6.Di)nocv (o) (2 (2..
+ (To, H ur.y
Osôt
where the two derivatives in eq. (5) have been approximated by finite difference with time increment It remains to explain how to estimate, for any 8, ( E [0. 7]. the inner product (T ) Hs()) from
sample paths of X and Y. This can be achieved using the formalism of coss--oovriance operators [34] as thoroughly explained in Appendix A, which yields to the following approximation
T St Bt)Ha((V)
(KV. + .TAA)) k. (K* + -AAm))) T.T
(20)
Hv((V
where k. E Rm k" E R are the vectors
[ (s (o.o).
[K]] = kss (v H).4)
and K E RR mmxmm K K E Bmxn K E Buxa are the matrices
K K = ks ks( 2 )..)),
K 1k7 = ks ks )orr))
[K la, ks (s v
and where Im (resp. I,,) is the m x m (resp. n x n) identity matrix. The corresponding algorithm and its complexity analysis are provided in Appendix B.
The next theorem ensures that the estimator Yx, is consistent for the 2nd order MMD. Theorem 3. (X,Y is a consistent estimator for the 2nd order MMD, i.e.
-D(Xx)))D3XX,Y) 2 O as m. + x0
(21)
with for 1 X. 1 Y and where convergence is in probability
We now iterate the procedure presented far to define higher order KMEs and MMDs.
Here use the notation 2(0.s] denote the restriction of the path to the sub-interval [0, s] 2].

---

3.5 Higher order kernel mean embeddings and maximum mean discrepancies
One can iterate the procedure described in Sec. 3.3 and recursively define, for any N31, the n order KME of X as the following point in HV(V)
ks( (s( "xixx
(dx)
(22)
(ZE
n-- where "X[Fx
is the (n - 1)tt predictive KME of X and
Hz(V)= Hs(Hs HVs(V)
(23)
n. times
The associated n order MMD between two processes X, is then defined as the norm of the difference in Hy of the two n order KMEs
D Y) = llxk
(24)
The following result generalizes Thm. in that it shows that the n² order MMD is stronger (i.e. finer) discrepancy measure than all the tthh order MMDs of lower order 1 < k < n.
Theorem 4. Given stochastic processes X, Y
YX, = =
= 0 for any 1 < k <
(25)
but the converse is not generally true.
Other than hypothesis testing, another important application relying on the ability of distinguishing random variables is distribution regression (DR) [5]. In the next section we make use the nh order MMD in the setting of DR on path-valued random variables presented in [6] and propose a family of kernels on stochastic processes whose RKHSs contains richer classes of functions than the RKHS associated to the universal kernel proposed in [6].
We note that since V is Polish space (i.e., separable, complete metric space) and the signature maps is continuous, in view of [35, Lemma 4.33] one can easily check that all RKHSs appearing in the present paper separable Hilbert spaces by an induction argument and therefore all regular conditional distributions are well-defined.
3.6 Higher order distribution regression
DR stochastic processes describes the supervised learning problem where the input is collection of sample paths and the output a vector of scalars [6]. Denote by (V)( the set of stochastic processes with sample paths (V)). Following the setup in [6], the goal is to learn function F : (V) + from a training set of input-output pairs {(,,, yt)} with X, E (V)(()) and Y: E IR, by means of classical two-step procedure [16, 17, 18].
Firstly, stochastic process X E (V)((V)) is embedded into its KME p (Vs(V) via the signature kernel k.s. Secondly, another function G HVs(V) -> IR is learnt by solving the minimization arg minge Hzar E, C(g(u),) ).))) where C is a loss function, and HRBF is the RKHS associated to the classical Gaussian kernel KRAF H.s HVsVV) x Hs(V) + R. This procedure materialises into kernel on stochastic processes whose RKHS is shown to be dense in the space of functions F P()(V)) > that are continuous with respect to the weak topology [6, Thm. 3.3].
However, a class of approximators that is universal with respect to some topology is not guaranteed to well approximate functions that are discontinous with respect that topology (but potentially continuous with respect to finer topology). For example, financial practitioners are often interested in calibrating financial models to market data or pricing financial instruments from observations of market dynamics. These tasks can be formulated as DR problems on stochastic processes (see experiments in Sec. 4.2), but the resulting learnable functions are discontinuous with respect the 1s order MMD whilst being continuous with respect to the 2nd order MMD [27]. This motivates the need to extend the kernel-based DR technique proposed [6] to situations where the target functions are not weakly continuous, which is what Thm. 5 addresses. A function f R - IR is called globally analytic with non non-negative coefficients if admits everywhere a Taylor expansion where all the coefficients are strictly positive, i.c. for any R we have f(x) Eiz a,z" with a, 0.
7

---

Theorem 5. Let R - I be globally analytic finction with non-negative coefficients. Define the family of kernels K'S P(X(V)) ((X(R")) + E. as follows
()(X,Y) = (X))(X)) n E
(26)
Then the RKHS associated to k is dense in the space of functions from P(X((R)) to R which are continuous with respect to the 1sth order MMD for any < S.
In Sec. 4 we will take f(x) = exp(=2² /o) with a > 0. This result marks the end of our analysis.
Next we apply our theoretical results in the contexts of two-sample testing, DR and causal inference.
4 Applications
Here we demonstrate the practical advantage of using 2nd order kernel mean embeddings, and evaluate the conditional kernel mean embedding for stochastic processes on causal discovery task. Additional experimental details can be found in Appendix C and the code is available at http://ggttauu.ccommaaaa...//// com/maudd3116/ hetir/////.//iggerrrrrr....
https://ggt thub.
4.1 Hypothesis testing on filtrations
We start by considering two processes X and X with transition probabilities depicted in Fig. 2. Although the laws Pr, and IP get arbitrarily close for large n, their filtrations are very different. Indeed, the two processes have different information structures available before time / = 1. Indeed, for any 0 < t < 1, the trajectory X is deterministic, whilst the progression of X remains random until t 1. Being able to distinguish two such stochastic processes is crucial in quantitative finance: if
P 0.5
p 0.5
p 0.5
p 0.5
Figure 2: The supports of P.,, (left) and (right).
P., and are the laws of two traded assets, P, gives an arbitrage opportunity. As shown in Fig. the 2nd order MMD can distinguish these two processes with similar laws (n = 5 10%) but different filtrations, while the s order MMD fails do so.
200
200
No
Count
Count
100
0.34
0.06
0.10
0.50
0.50
DJ vy
D3 1
Figure 3: Left: Empirical distribution the 1s order MMD. Under Ho the two measures are equal to and we use 500 samples from cach. Under Ha with P and P., where n 5- 10", and we use 500 samples. Right: Same for the 2nd order MMD. Histograms are obtained by computing 500 independent instances of the MMD.
4.2 Applications of higher order distribution regression to quantitative finance
In this section we use kernel Ridge regression and support vector machine (SVM) classification equipped with the kemnel K3 from Thm. to address two real-world problems arising in quantitative finance, notably the calibration of the rough Bergomi model [36] and the pricing of American options [37]. We benchmark our Eltration-sensitee kemel K3 against range of kernels, including K}.

---

The rough Bergomi model is rough volatility model [38] satisfying the following stochastic dynamics
ds, = V.S,W, Vi2 f (ss,t)d.., Ze = pW, + v V1-p2W?
(27)
where W. W are two independent Brownian motions and (s.t) (t 8)a where here we take = 0.2. The model in eq. (27) is non- non-Markovian the sense that the conditional law of S Fs, depends pathwise on the past history of the process. Of particular importance is
the correct retrieval of the sign of the correlation parameter P [39]. We consider 50 parameter values [ chosen uniformly at random from [-1,1] 1]. Each μi is regressed on a collection of m 200 sample trajectories. We use an SVM classifier endowed with different kernels (Table 1).
One of the most studied optimal stop- ping problems is the pricing an American option with non-negative payoff function g : RRd + I. Stock prices assumed follow d-dimensional stochastic process X. The price of the corresponding option is the solution of the optimal stopping problem sup, (X+) Xo]. where the supremum is taken over stopping times 5. Despite significant advances. pricing American options remains one of the most computationally challeng- ing problems in financial optimization, in particular when the underlying process X is non- Markvannov This is the setting we consider, modelling stock prices as sample paths from fractional Brownian motion (fBm) [40] with different Hurst exponents h E (0. 1). True target prices are obtained via expensive Monte Carlo simulations [41]. We consider 25 values { 25 sampled uniformly at random in [0.2, 0.8] and use 500 samples from each fBm. As shown in Table our kernel K
Table 1: Quantitative finance examples. Average performances with standard errors in parenthesis.
Rough Bergomi model calibration (Acc.)
American option pricing (MSE x10
Kernel
RBF Matérn
87% (5%)
1.07 (0.75)
87% (3%)
2.75 (3.05)
K
91% (3%)
0.90 (0.34)
K'S
93% (3%)
0.52 (0.07)
yields the best results on both tasks (rough Bergomi model calibration and American option pricing), systematically outperforming other classical kernels as well as the kernel K3 introduced in [6].
4.3 Inferring causal graph for interacting bodies
Finally, we consider the task of recovering the causal relationships between interacting bodies solely from observations of their multidimensional trajectories. We employ the multi-body interaction simulator from [42] in order simulate an environment where N balls are connected by invisible physical relations (e.g. a spring) and describe 2D trajectories (see Fig. 4a with N = 3 and 2 springs). At the beginning of simulated episode, the initial positions the balls are generated at random, and during the episode, the balls are subject to forces with random intensity and direction. By simulating m episodes we end up with m sample trajectories for each of the N balls. We use the kPC algorithm [13]-which relies on conditional independence testing- with the signature kernel and evaluate its ability to recover whether any two balls are connected or not. We vary m and N and report the results in Figs. 4b and 4c. Each experiment is 15 times, 30% of the runs are used to chose the hyperparameters, and the reported results have been obtained on the remaining runs. We note that for finite datasets conditional independence testing is hard without additional assumptions, as discussed in 44].
C2
Aceuescy
6.76
Accnnaeee
0.50
0,50
0:20
0.25
61
Nuncher of samples
Narubee of bells
(a) 3 interacting balls describing trajectories in the 2D plane over time.
(b) Accuracy on binary classifica- tion of edges with a varying num- ber of sample episodes (5 balls)
(c) Accuracy on binary classifica- tion of edges with varying num- ber of balls (100 samples)

---

5 Conclusion
In this paper, we introduced a family of higher order KMEs by conditioning a stochastic process on its filtration, generalizing the classical notion of KME. We derived an empirical estimator for the 2nd order MMD and proved its consistency. We then proposed a fllraaiio-sensitive kernel two-sample test and showed with simulations its ability to capture information that gets missed by the standard MMD test. addition, we constructed a family of universal kernels on stochastic processes that allows to solve real-world calibration and optimal stopping problems in quantitative finance via Ridge regression. Finally, we designed a causal- discovery algorithm using conditional independence tests to recover the causal graph of structural dependencies among interacting bodies solely from observations of their multidimensional trajectories.
6
Future work
Regarding the choice of kernel hyperparameters, in the setting of two-sample tests, we can use various hyperparameter selection methods which have been proposed in the kernel literature, including approaches aiming at maximizing the test power using the Signal-o-nnisse-e [45, 46, 47J. In the distribution regression setting, we made use of cross validation approach. Developing hyperparameter tuning methodologies for higher order KMES is an interesting future work direction and we will note that [48, 49] certainly a good starting point for such investigation.
as an objective
Higher order KMEs have the potential to be used beyond two-sample tests and distribution regression. For example [6] recently investigated the use of the 1st order MMD to derive approximate Bayesian computation (ABC) algorithm for irregular time series. Another idea that is currently being investigated is using higher order MMDs as discriminators in autoregressive generative models for time series, where conditioning the future trajectories on past observations is key.
We conclude with theoretical remark. All paths considered in the present paper are piecewise linear. Consequently, all sample paths from higher order predictive KMES are also piecewise linear and their KMES are well defined. Such a nice property will not hold anymore if one considered more generic continuous sample paths, because such regularity of sample paths from the corresponding higher order predictive KMEs might break as noted in [29, Remark 1]. The study of how the regularity changes by taking higher order kernel mean embeddings is an interesting direction for future work.
Acknowledgments and Disclosure of Funding
ML and CS were respectively supported by the EPSRC grants EP/L016710// and EP/R513295/1. TD acknowledges support from EPSRC (EP/T004134/1) UKRI Turing AI Fellowship (EP/V02678X/I), and the Lloyd's 's Register Foundation programme on Data Centric Engineering through the London Air Quality project. ML, CS and TL were supported by the Alan Turing Institute under the EPSRC grant HPP/511019// and DataSig under the grant EP/S026347/1.
References
[1] Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. kernel two-sample test. The Journal of Machine Learning Research, 13(1):223-733 2012.
[2] Kun Zhang, Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Kernel-based conditional independence test and application in causal discovery. arXiv preprint arXiv:1202.3775. 2012.
[3] Bernhard Schölkopf, Alexander Smola, Francis Bach, et al. Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT press, 2002.
[4] Thomas Hofmann, Bernhard Schölkopf, and Alexander J Smola. Kernel methods in machine learning. The annals of statistics, pages 1171- 2008.
[5] Zoltán Szabó, Bharath K Sriperumbudur, Barnabás Póczos, and Arthur Gretton. Learning theory for distribution regression. The Journal of Machine Learning Research, 17(1):5272-5311, 2016.
10

---

[6] Maud Lemercier, Cristopher Salvi, Theodoros Damoulas, Edwin Bonilla, and Terry Lyons. Distribution regression for sequential data. In International Conference on Artificial Intelligence and Statistics, pages 3754-3762. PMLR, 2021.
[7] Le Song, Kenji Fukumizu, and Arthur Gretton. Kernel embeddings of conditional distributions: unified kemnel framework for nonparametric inference in graphical models. IEEE Signal Processing Magazine, (0(4):98-111, 2013.
[8] Le Song, Arthur Gretton, and Carlos Guestrin. Nonparametric tree graphical models. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pages 765-772. JMLR Workshop and Conference Proceedings, 2010.
[9] Kenji Fukumizu, Le Song, and Arthur Gretton. Kernel bayes' rule: Bayesian inference with positive definite kernels. The Journal of Machine Learning Research, 14())3753-3783,2000 2013.
[10] Le Song, Jonathan Huang. Alex Smola, and Kenji Fukumizu. Hilbert space embeddings of conditional distributions with applications to dynamical systems. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 961-968, 2009.
[11] Jovana Mitrovic, Dino Sejdinovic, and Yee Whye Teh. Causal inference via kernel deviance measures. arXiv preprint arXiv: /80200424.22... 2018.
[12] Robert E Tillman, Arthur Gretton, and Peter Spirtes. Nonlinear directed acyclic structure learning with weakly additive noise models. In NIPS, pages 1847-1855, 2009.
[13] Xiaohai Sun, Dominik Janzing, Bemhard Schölkopf, and Kenji Fukumizu. A kernel-based causal learning algorithm. In Proceedings of the 24th international conference on Machine learning, pages 855-862, 2007.
[14] Kenji Fukumizu, Arthur Gretton, Xiaohai Sun, and Bernhard Schölkopf. Kernel measures of conditional dependence. In NIPS, volume 20, pages 489-496, 2007.
[15] Junhyung Park and Krikamol Muandet. A measure-theoretic approach to kernel conditional mean embeddings. Advances in Neural Information Processing Systems, 33, 2020.
[16] Ho Chung approaches to distribution regression. In International Conference on Artificial Intelligence and Statistics, pages 1167-1176 PMLR, 2018
Leon Law, Dougal Sutherland, Dino Sejdinovic, and Seth Flaxman. Bayesian
[17] Krikamol Muandet, Kenji Fukumizu, Francesco Dinuzzo, and Bernhard Schölkopf. Learning from distributions via support measure machines. In Advances in neural information processing systems, pages 10-18, 2012.
[18] Alex Smola, Arthur Gretton, Le Song, and Bernhard Schölkopf. A hilbert space embedding for distributions. International Conference on Algorithmic Learning Theory, pages 13-31. Springer, 2007.
[19] Terry Lyons. Rough paths, signatures and the modelling of functions streams. Proceedings of the International Congress of Mathematicians, Korea, 2014.
[20] Patric Bonnier, Patrick Kidger, Perez Arribas, Cristopher Salvi, and Terry Lyons. Deep signature transforms. Advances in Neural Information Processing Systems, 2019.
[21] Franz Király and Harald Oberhauser. Kernels for sequentially ordered data. Journal of Machine Learning Research, 20(31):1-45, 2019.
[22] Thomas Cass, Terry Lyons, Cristopher Salvi, and Weixin Yang. The signature kernel is the solution of a goursat pde. arXiv preprint arXiv:2006.14779 14794, 2020.
[23] D. J. Aldous. Weak convergence and general theory of processes. Unpublished draft of monograph, 1981.
[24] G. C. Pflug and A. Pichler. A distance for multistage stochastic optimization models. SIAM J. Optim., 22(1):1-23, 2012.

---

[25] J BBcchhoff- Veraguas, D Bartl, M Beiglböck, and J Wiesel. Estimating processes in adapted wasserstein distance. Annals of applied probability, 2021.
[26] D. Hoover and J. Keisler. Adapted probability distributions. Trans. Amer. Math. Soc., 1984.
[27] J Backhoff- Vackkhoff-Vrraguss, Bartl, M Beiglböcck, and M Eder. Adapted wasserstein distances and stability in mathematical finance. Finance and Stochastics, 2019.
[28] Julio Backhoff- Veraguas, Daniel Bartl, Mathias Beiglböck, and Manu Eder. All adapted topologies equal. Probubility Theory and Related Fields, 178(3):1125-1177 2020.
[29] Patric Bonnier, Chong Liu, and Harald Oberhauser. Adapted topologies and higher rank signatures. arXiv preprint arXiv:2005.0888 08897, 2020.
[30] Terry J Lyons. Differential equations driven by rough signals. Revista Matemática Iberoameri- cana, 14(2):115310,, 1998.
[31] Ilya Chevyrev and Harald Oberhauser. Signature moments to characterize laws of stochastic processes, arXiv preprint 18X0: 10971, 2018.
[32] Bharath K Sriperumbudur, Arthur Gretton, Kenji Fukumizu, Bernhard Schölkopf. and Gert RG Lanckriet. Hilbert space embeddings and metrics on probability measures. The Journal of Machine Learning Research, 11:1517-1561, 2010.
[33] Dino Sejdinovic, Bharath Sriperumbudur, Arthur Gretton, and Kenji Fukumizu. Equivalence of distance-based and rkhs-based statistics in hypothesis testing. The Annals of Statistics, pages 2263-2291, 2013.
[34] Krikamol Muandet, Kenji Fukumizu, Bharath Sriperumbudur, and Bernhard Schölkopf. Kernel mean embedding of distributions: A review and beyond. arXiv preprint arXiv: 1oXv::6005009222 2016.
[35] Andreas Christmann and Ingo Steinwart. Support Vector Machines. Springer verlag, 2008.
[36] Christian Bayer, Peter Friz, and Jim Gatheral. Pricing under rough volatility. Quantitative Finance, 16(6):887-906 2016.
[37] H Buehler, L. Gonon, Teichmann, and B Wood. Deep hedging. Quantitative Finance,
19(8):1211-1291 2019.
[38] Jim Gatheral, Thibault Jaisson, and Mathieu Rosenbaum. Volatility is rough. Quantitative Finance, 18(6):933 949, 2018
[39] Paul Gassiat et al. On the martingale property in the rough bergomi model. Electronic Communications in Probability, 24, 2019.
[40] Tyrone E Duncan, Yaozhong Hu, and Bozenna Pasik-Duncan. Stochastic calculus for fractional brownian motion i. theory. SIAM Journal on Control and Optimization, 38(2):582 612, 2000.
[41] Francis A Longstaff and Eduardo S Schwartz. Valuing american options by simulation: simple least-squares approach. The review of financial studies, 14(1): 11))1,,114471111111111 2001.
[42] Yunzhu Li, Antonio Torralba, Anima Anandkumar, Dieter Fox, and Animesh Garg. Causal discovery in physical systems from videos. Advances in Neural Information Processing Systems, 33, 2020.
[43] Rajen D Shah and Jonas Peters. The hardness of conditional independence testing and the generalised covariance measure. The Annals of Stotistics, 48(3):1514-1555) 2020.
[44] Anton Rask Lundborg, Rajen D Shah, and Jonas Peters Conditional independence testing in hilbert spaces with applications to functional data analysis. arXiv preprint arXiv:2101.07108 2021.
12

---

[45] Arthur Gretton. Dino Sejdinovic, Heiko Strathmann, Sivaraman Balakrishnan. Massimiliano Pontil, Kenji Fukumizu, and Bharath K Sriperumbudur. Optimal kernel choice for large-scale two-sample tests. In Advances in neural information processing systems, pages 1205-1213. Citeseer, 2012.
[46] Danica J. Sutherland, Hsiao-Yu Yu Tung, Heiko Strathmann, Soumyajit De, Aaditya Ramdas, Alexander J. Smola, and Arthur Gretton. Generative models and model criticism via optimized maximum mean discrepancy. In 5th International Conference on Learning Representations, ICLR, 2017.
[47] Feng Liu, Wenkai Xu, Jie Lu, Guangquan Zhang, Arthur Gretton, and Daniea J Sutherland. Learning deep kernels for non-parametric two-sample tests. In International Conference on Machine Learning, pages 6316-6326. PMLR, 2020.
[48] Seth Flaxman, Dino Sejdinovic, John Cunningham, and Sarah Filippi. Bayesian learning of kernel embeddings. In Proceedings of the Thirty-Second Conference on Uncertainty in Artificial Intelligence, pages 182-191, 2016.
[49] Kelvin Hsu, Richard Nock, and Fabio Ramos. Hyperparameter learning for conditional kernel mean embeddings with rademacher complexity bounds. In Joint European Conference Machine Learning and Knowledge Discovery in Databases, pages 227-242. Springer, 2018.
[50] Kenji Fukumizu, Francis R Bach, and Michael Jordan. Dimensionality reduction for supervised learning with reproducing kernel hilbert spaces. Journal of Machine Learning Research, 5(Jan):73-99, 2004.
[51] Calypso Herrera, Florian Krach, Pierre Ruyssen, and Josef Teichmann. Optimal stopping via randomized neural networks. arXiv preprint arXiv:2104. 13669, 2021.
[52] Sebastian Becker, Patrick Cheridito, and Arnulf Jentzen. Deep optimal stopping. Journal of Machine Learning Research, 20:74, 2019.
[53] Peter Spirtes, Clark N Glymour, Richard Scheines, and David Heckerman. Causation, prediction. and search. MIT press, 2000.
[54] J Park and K Muandet. Regularised least-squares squares regression with infinite difnite-dimensinna output space. arXiv preprint arXiv:201O. 10973, 2021.
[55] Andreas Christmann and Ingo Steinwart. Universal kernels on non-standard input spaces, In Advances in neural information processing systems, pages 406-414, 2010.
[56] Charles Walkden. Ergodic theory. Lecture Notes University of Manchester, 2014.
13


Reviews:
[{"review_id": "zjcJIEtjybt", "reviewer": "Reviewer_zRy1", "summary": "The paper proposes (higher order) kernel mean embeddings (KMEs) for stochastic processes -- the novelty is in considering the filtration induced by the process rather than the sample path, which allows us to consider historical dependence.  \n", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "My overall impression of the paper is positive. \nThe paper introduces an interesting way of summarising stochastic processes, reflecting their history information (given in the form of filtration). \nAs demonstrated by the authors with examples, the maximum mean discrepancy (MMD) defined by the new embedding allows us to characterise differences between processes, which are not captured by the KME defined by the law of the path-valued random variable corresponding to the process. The latter approach is, to my knowledge, common in areas like functional data analysis, whereas we often need to exploit past information (dependence). \nThe example applications make the KMEs look promising, although some of the applications are not fully established (e.g., a two-sample test is actually not proposed); the proposed approach would lead to new applications of the MMD (or the KMEs) for sequential data, as done for the original MMD. \n\n## Comments \n* Higher-order KMEs are defined in terms of conditional distributions. Is the space of the recursive RKHS regular enough so that regular conditional distributions are defined? It would be great this point (e.g., with kernel's continuity and separability of $V$ )  \n* How does the process in (14) satisfy the assumption on $\\mathcal{X}$ given in the first paragraph of Section 2? \n* The conditional independence testing with continuous conditioning variables is known as hard [Shah and Peters, 2020]  Although the experiment result in Section 4.3 looks reasonable, it might need to be replaced with some other results. \n\n\n## Minor comments: \n* L133: $\\mathcal{H}(\\mathbb{R}^d)$ -> $\\mathcal{H}(V)$\n* Some equations do not have punctuation marks, such as the period. \n* The Markov process figure (Figure 2) is missing the X-axis; adding time indices would help. \n* Numbering to equations would be better if it is limited to the ones that are actually referenced (I assume the authors did it for the reviewing purpose). \n* Is mentioning Eq. (12) necessary? On the RHS, the CME acts on the kernel; how is this defined? As done in Theorem 4 in [Song, Huang, Smola, and Fukumizu, 2009]? The operator definition is less general than the Bochner integral definition. \n\nReferences: \n* Rajen D. Shah, Jonas Peters \"The hardness of conditional independence testing and the generalised covariance measure,\" The Annals of Statistics, Ann. Statist. 48(3), 1514-1538, (June 2020)", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "NA", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "10 hours"}, "extra_text": {}}, {"review_id": "oquim5Bxe8-", "reviewer": "Reviewer_V4uV", "summary": "This paper aims to construct a family of kernel mean embeddings that capture filtration information from stochastic processes. This is achieved through the use of the signature kernel, a characteristic kernel defined on pairs of paths that can be evaluated by solving a hyperbolic PDE, and by forming conditional kernel mean embeddings of the stochastic process conditioned on its own filtration up to some time in the past.\n\nThe resulting conditional kernel mean operator itself is a stochastic process, and thus this construction can be repeated to produce higher ordered conditional kernel mean embeddings of the original stochastic process. The corresponding MMD defined from the conditional kernel mean embeddings at each order can then be used to measure distances between two stochastic processes at each order.\n\nThe first major contribution shows that higher ordered MMDs are stronger discrepancy measures than lower ordered MMDs (Theorem 2 and 4). They can then be used to perform a kernel two-sample test between paths from two stochastic processes. In particular, the paper focuses on using the second order MMD and derive consistent estimators for it (Theorem 3).\n\nThe second major contribution applies these constructions for distribution regression on stochastic processes, where the input is a collection of sample paths and the output is a vector of scalars. This is done by embedding the sampled paths from the stochastic process into a kernel mean embedding through the signature kernel, then learning the mapping between this kernel mean embedding to the vector of scalar outputs. The authors pay strong attention to the application in financial markets where the functions of interest are often discontinuous with respect to the first order mean embedding, and show they can still be learned by proving that the RKHS induced in higher orders is dense in the space of functions of interest (Theorem 5).", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, the contributions of the paper is well motivated, well articulated, and technically interesting. The developments presented in the paper is clear and well ordered. It is clear why the signature kernel was chosen and the motivation behind constructing conditional kernel mean embeddings of stochastic processes conditioned on their filtrations. The subsequent developments in constructing higher ordered kernel mean embeddings and their applications to kernel two-sample testing and distribution regression is also easily to follow.\n\nTo me, the major novelty presented in this paper is the notion of recursively constructing the conditional kernel mean embedding of stochastic processes conditioned on its own filtration into high ordered versions of the original, by making use of the fact that the resulting conditional kernel mean embedding (or more exactly the conditional kernel mean operator, with the conditioned variable left as a stochastic process not as a realised path) is itself a stochastic processes. Combined with the fact that discrepancies become stronger at higher orders, as measured by the MMDs of the respective order, this is a very general notion for learning tasks and relationships involving stochastic processes with an ordered nature to its entirety, without losing information from filtration.\n\nThe two example applications presented in this paper, kernel two-sample testing and distribution regression, are rather straight forward in consequence by applying what we know about these applications of KMEs in general to this specific setting where KMEs are constructed as previously described. However, they are well developed nonetheless.\n\nThis paper can be further improved by some discussion on how the kernel hyperparameters can be learned or selected. Especially at higher orders, hyperparameter learning does not seem trivial and presumably performance can be quite sensitive to the kernel hyperparameters unless we can argue otherwise.\n\nOverall, the notions introduced in this paper can be quite valuable to the wider community in extending adoption of KME based methods to also tasks involving stochastic processes where filtration information is important to capture. I would tend towards acceptance.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The work is motivated by applications in financial markets. However, its contributions are general in nature in terms of the technique and not limited to financial markets.\n\nThe paper can be improved by highlighting potential wider impacts in being able to capture filtration information through kernel mean embeddings. I believe these concepts are more general and can be applied to settings other than kernel two-sample tests and distribution regression.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "TaAGeYfEY1", "reviewer": "Reviewer_cs6X", "summary": "This article study stochastic processes through the lens of kernel methods. The authors propose to use a generalization of kernel mean embeddings to define metrics between stochastic processes. In particular, they used these metrics to define a filtration-sensitive kernel two-sample test. Moreover, they showed how to use this new class of kernels in some learning tasks. ", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This work is built upon existing theoretical tools (signature transform, signature kernel). Nevertheless, the use of this kernel seems to be novel in the ML community. From an empirical point of view, the use of this family of kernels seems to improve significantly upon existing methods in the field. However, the evaluation of the kernel is very consuming since it relies on the identity (5), which requires solving a PDE along with some heuristics such as identity (21). It would be interesting to comment in detail on the complexity of such an approximation scheme.  \n\nI enjoyed very much reading the paper, and I have some questions: \n\n- In practice, can we go beyond 2nd order MMD with the approximation scheme based on the identity(5)?\n-What is the intuition lying behind this transform? \n-Is there a toy example where the evaluation of the kernel has an analytical form?", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "No. The contribution is a general-purpose framework.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "7"}, "extra_text": {}}, {"review_id": "QQ9nHBXbfBy", "reviewer": "Reviewer_SVrX", "summary": "The paper proposes a generalized approach to kernel mean embeddings – higher-order mean embeddings --  and new two-sample test based on the newly proposed kernel which can be applied to stochastic processes and in particular is designed to capture filtration information in stochastic processes. Various theoretical results are derived, including consistency. \n", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper proposes an interesting new approach which can be used, in particular, in quantitative finance. The paper also presents an application to causal discovery. While I found the approach and motivation relevant, the applications described in the paper appeared to me rather limited. I believe more extensive experiments have to be performed to fully understand the advantages of the proposed method. The current experiments have to be explained in more detail and clearly motivated. \n\nSome detailed comments:\n-\tI found it a bit hard to follow at times, perhaps describing the outline of the paper in the end of the introduction with the main contributions could be helpful;\n-\tFinance application could use more description;  it is really not a classical finance model and since the finance audience is also not targeted audience of the paper it would be good to provide more extensive intuition about what it is used for, what are particular challenges etc. \n-\tRelated to the previous point: the authors do mention that the correlation parameter \\rho is of particular interest; in the simulation studies the authors draw values for \\rho from the interval [-0.4;0.4]. Is this reasonable? I would have expected a higher (positive and negative) correlation be possible as well. \n-\t$h$ is not introduced (I suppose it is related to the frequency of sampling of the data)? And the choice of this parameter for the simulation study is not motivated. \n-\tThe choice of the kernels for the comparison seems to be not ideal; in particular I would indeed expect the RBF kernel to be not appropriate since it is high-frequency financial data (so it is unlikely to be smooth). \n-\tConsidering different numbers of rough volatility trajectories also could strengthen this application (see [6]).  In particular, if the advantage is the same over different numbers of trajectories would be of interest.\n-\tHow is the choice of $h$ motivated in the simulation studies? \n-\tIn the causal discovery application I do not see a competitive method to which we could compare the proposed method. How does it perform in comparison to $K_{S}^{1}$, for example? Comparison to the state-of-the-art conditional independence test perhaps could highlight the strength of the approach.\n\nUPD: Most of my original concerns were about empirical applications and the choices the authors made for the kernels/parameters range etc. During the rebuttal and discussion, the authors clarified the things I missed and provided additional results which are in line with the literature for the parameter choice of the financial model. These results confirmed earlier findings. Taking into account the addressed concerns and novelty of the proposed method I change my score to 7 and recommend the paper for acceptance. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I do not see obvious negative societal impact.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3,5"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Our opinion is that this paper should be accepted though there are some weaknesses: it would have been easy in the experiment section to use real-world data from quantitative finance instead of draws from a fractional BM; comparisons to CI tests from the causal inference area; and readability.  On a more detailed level it would be good if the authors would cite a disintegration theorem for their family of measure P_{X|F_{X_t}}.
Decision: Accept (Poster)