Learning with Expected Signatures: Theory and Applications
Authors: Lorenzo Lucchese
Mikko S. Pakkanen
Almut E. D. Veraart
Venue: ICML 2025 oral (2025)
Keywords: Probabilistic Machine Learning
Signature
Expected Signature
Time Series
Rough Paths
Primary area: theory->probabilistic_methods

Abstract: The expected signature maps a collection of data streams to a lower dimensional representation, with a remarkable property: the resulting feature tensor can fully characterize the data generating distribution. This "model-free"' embedding has been successfully leveraged to build multiple domain-agnostic machine learning (ML) algorithms for time series and sequential data. The convergence results proved in this paper bridge the gap between the expected signature's empirical discrete-time estimator and its theoretical continuous-time value, allowing for a more complete probabilistic interpretation of expected signature-based ML methods. Moreover, when the data generating process is a martingale, we suggest a simple modification of the expected signature estimator with significantly lower mean squared error and empirically demonstrate how it can be effectively applied to improve predictive performance.

Learning with Expected Signatures: Theory and Applications
Lorenzo Lucehese' Mikko S. Pakkanen'
Almut E. D. Veraart'
Abstract
of iterated integrals of X, i.e. for t E [0, k 220
The expected signature maps a collection of data streams a lower dimensional representation. with remarkable property: the resulting fea- ture tensor can fully characterize the data gener- ating distribution. This "model-free" embedding has been successfully leveraged to build multi- ple domain-age -agnostic machine learning (ML) al- gorithms for time series and sequential data. The convergence results proved in this paper bridge the gap between the expected signature's is pirical discrete-time estimator and its theoreti- cal continuous-time value, allowing for a more complete probabilistic interpretation of expected signature-baat based ML methods. Moreover, when the data generating process is a martingale, we suggest a simple modification of the expected sig- nature estimator with significantly lower mean squared error and empirically demonstrate how it can be effectively applied to improve predictive performance.
(x) = 1. dx @ ... @ dx (1)
0S.15...5..5
In many practical applications the path X is taken to be the piecewise linear interpolation of discrete-time stream of data, which is of bounded variation by construction. Signature-based machine learning (ML) approaches (Lyons & McLeod, 2024) thus often restrict the theoretical frame- work to paths in ( T). IR.). In this setting, two funda- mental properties of the signature that make a desirable non-parametric feature extraction method for sequential data are the characterization result of Hambly & Lyons (2005) and the universality approximation theorem of Levin et al. (2016). Moreover, when the path X is understood as a (realization of a) random process with distribution over BV( (o, T), R.)), the shuffle property of the signature implies that all moments of the random variable S SX%) [o.T] are determined by its expectation
((T) = B{S(X) pu.21l E TT(R))).
A natural question, known as the Hamburger moment prob- lem (Fawcett, 2003), is thus whether the expectation of the signature characterizes its law (and thus the law of the path). When imposing a probability distribution on X the assumption of bounded variation paths becomes quite restrictive: Brownian motion, the basic building block of many stochastic models, has paths of infinite variation al- most surely. Even if we observe a discrete seecreetetimeeeeee stream of data, we often still would like to define the process X as latent stochastic process of which we observe the linear interpolation over some partition æ of [o, 7), hereafter de- noted by X We hence wish make sense of the signature of a stochastic process X with paths of unbounded variation. For given path X E C(O0, T), Rdd) of finite p-variation, once we "lift" the process to a p-rough path (Lyons et al., 2007, Definition 11) then the signature S of is uniquely defined'. Without delving into the details of rough path theory, for our purposes it suffices to interpret the choice of lift as fixing notion of integration with respect to X: the higher order signatures terms are then understood as iterated integrals of the path X defined in this sense.
1. Introduction
The signature transform of stream of data is an infinite but countable sequence of its "iterated integrals" summarizing the input in top-down fashion, meaning the informational content of its terms decays factorially. Originally introduced by Chen (1954) and serving as fundamental object of rough path analysis (Lyons et al., 2007), the signature
(0.1)) E T((R)), t [0,
of a path = (X,, E [)} E [[0o,T,,R)
lift (in the sense that embeds X) to the space of continuous functions over the tensor algebra T' T((E))) possessing some nice algebraic and geometric properties. When the path is of bounded variation, the signature is defined as the sequence
Depaartment of Mathematics, Imperial College London, Lon- don. United Kingdom. Correspondence to: Lorenzo Lucchese Zlorenzo.luccheee 17 @imerrialacccu llucchese6 crrcchsso@ggmai...o com?.
Proceedings of the 4dnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
This is the first fundamental theorem in the theory of rough paths (Lyons et al., 2007, Theorem 3.7).

---

Learning with Expected Signatures: Theory and Applications
Motivated by the fact that we can only ever observe the process X over discrete partition æ of [0, 7] we restrict our attention to the class of stochastic processes whose lift (and hence signature) can be approximated by the lift (and hence signature) of the bounded variation path x Following the rough path literature we take such approximation in the p- variation metric to define the notion of canonical geometric stochastic process, cf. Definition 2.1. In Chevyrev & Lyons (2016); Chevyrev & Oberhauser (2018) the authors provide characterization results for the expected signature of canon- ical geometric stochastic processes, i.e. conditions under which the map IP + [x is injective. Such charac- terizing property of the expected signature has found practi- cal use in a wide range of applications, ranging from classic ML tasks (Lemercier et al., 2021; Triggiano & Romito, 2024; Schell & Oberhauser, 2023) mathematical finance (Lyons et al., 2021; Futter et al., 2023).
across the samples X " ..... X The first main contri- bution of this paper is thus bridge the gap between the empirical expected signature estimator and the expected signature of latent continuous-ting stochastic process, un- locking a more general probabilistic interpretation of several ML algorithms and effectively moving beyond the expected signature as a simple feature extraction method. This natu- rally leads to the second theoretical contribution: by starting from the continuous of the expected signature estimator with significantly bet- ter finite sample properties when the latent data generating process is martingale. The superior performance of this modified estimator is empirically verified through various experiments with expected signature- based MI. algorithms from the literature.
setting we devise modification
Latent path distribution IP (oT))
Observed values
X
%
N.
The expected signature is thus a highly informative quan- tity and, consequently, methods for computing (TT) have received considerable research interest. Such methods can be broadly categorized into two classes: those employing an analytical approach and those following statistical one. An- alytical methods aim to develop exact formulas for specific classes of models. A first step in this direction was taken in Ni (2012, Section 4) showing that the expected signature of an Itô diffusion satisfies an explicit partial differential equation (PDE). This result was subsequently generalized in Cuchiero et al. (2023) to the class of signature-SDEs and in Friz al. (2022; 2024) to (discontinuous) semimartin- gales. On the other hand, the statistical approach aims to estimate ((T) directly from observed data, preserving the model-free nature of the expected signature. For given set of observations X 1, N,= one can form the estimator
X
x
T
time,
partition, ST
Figure I. Estimating the expected signature estimation from finite collection of discretely-observed paths.
2. Theory
Let (X,, E [0. denote d-dimensional stochas- tic process over the probability space (,, IP).
N (oTT,
N.T (T) = N
Σ S(X*)))
Definition 2.1. We say X is a canonical geometric stochastic process of rough order p if there exists sequence of parti- tions p with |e| - O such that the limit in the p-variation metric of the canonically lifted linearly interpolated process X exists in probability. Convergence in probability implies almost sure convergence (along a subsequence) and hence we can almost surely define the lift of X as such limit.
as illustrated in Figure 1, and study its in-fill |x| - 0 and large-sample N - oo asymptotics. This line of work in- cludes the explicit results of Ni (2012, Section 3.2) for Brownian motion and of Passeggeri (2020) for fractional Brownian motion with Hurst parameter H > 1/2 as well as the preliminary results in Friz. & Victoir (2010) for more general semimartingales. Additionally, Schell & Oberhauser (2023, Section 8) develops asymptotic results for processes of bounded variation. In this work we provide a unifying set of general conditions under which the expected signature estimator J.M-T (T) displays important asymptotic statistical properties, namely consistency and asymptotic normality. Our results allow for irregular2 observation partitions æ possibly varying across samples and for dependency
Remark 2.2. The definition of lift suggests this might de- pend on the choice of the sequence of partitions p. In any case, for a wide range of stochastic processes there exist canonical lifts that satisfy our definition of canonical geo- metric rough path. These include:
Semimartingales: For p E (2,3) any semimartingale can be lifted to geometric p-rough path by defining the lift via Stratonovich integration; the signature of X then coincides with iterated Stratonovich integrals. For any sequence of partitions p the lifts of the linear
"Clerrly for the estimation problem to be well-posed, the se- quence of partitions needs be signature defining in the sense of Definition 2.5.
2

---

Learning with Expected Signatures: Theory and Applications
interpolations converge in p-variation metric the Stratonovich lift (Friz&& & Victoir, 2010, Chapter 14) and hence X is canonical geometric stochastic process in the sense of Definition 2.1.
Remark 2.4. Taking p to be a sequence such that Defini- tion 2.1 holds, by continuity of the extension map (Lyons et al., 2007, Theorem 3.10), it immediately follows that the signature of X' (truncated at level K 2 [p_) converges in probability to the signature of X (up to level K) in the p- p-variation topology. In particular, this implies that, for any finite collection of words I,
Gaussian processes: Many Gaussian processes ad- mit canonical lifts to geometric p-rough paths (Friz & Victoir (2010, Theorem 15.34, Definition 15.35) and Coutin & Qian (2002)) with the existence criterion for such canonical lifts easily stated in terms of the co- variance function. The definition of the lift implicitly requires P to be any sequence such that X converges uniformly to X almost surely. For example, fractional Brownian motion with Hurst parameter H > 1/4 can be lifted to a geometric p-rough path with p > 1/H by choosing p to be the sequence of dyadic partitions.
S'X)) 3 (X)
(4)
Similar arguments imply that, when convergence to the lift along p holds almost surely in the p-variation metric, then also the higher order signature terms converge almost surely, and, in particular, (4) holds in the almost sure limit.
In the following sections, we will be estimating the expected signature at fixed time horizon T >> O. To develop the properties of these estimators, it will be thus sufficient to work with pointwise limits like (4) without having to deal with the stronger pathwise p-variation convergence used to define canonical geometric stochastic processes. This mode of convergence will thus be sufficient consider sequence of partitions as signaurredefining
In what follows, we will assume the canonical geometric stochastic process X has canonical lift (i.e. a canonical sequence of partitions p along which the lift is defined) and unambiguously refer to it as the lift of X.
Let denote a partition of (0, T] with mesh |e| and X²- = [Xe, E [0] the linear approximation of X over p. i.c.
Definition 2.5. Let X = {X., t E []] be a canonical geometric stochastic process, we say that sequence of partitions и of the interval [ C [0, T] with | - is signature-definnng if for any collection of words I,
t X. E [u, E.P. X²- X. +
V - u
with X..u X. X.. The signature of the bounded varia- tion path X up to time i [0, defined by Equation (1) through classic Riemann- -ttelltjss integration and can thus be computed by
S* (x.) E. ())))
| -
(5)
S(X²) [0.1] =
X exp&
2.1. Expected Signature Estimation
S
[u.o]EP(o.a]
In this section, we assume we have access to N copies of X discretely observed over possibly different partitions of the interval [0, 7]. i.c. each x", is an observation over n of a continuous- latent process X", for n 1,.... ....... N. We will focus on two observational schemes:
where P]o.d] denotes the restriction of p to 4.. The canoni- cal lift of X to a (geometric) p-rough path is
(I, S: (x²))..). S.P. (x*) E Tiel ((R)) (3)
for E [o, 7]. Definition 2.1 requires that there exists se- quence of partitions for which this sequence of geometric p-rough paths converges in probability in the p-variation metric. key result from rough path theory is that a ge- ometric p-rough path has a full signature. Fixing the lift of X via Definition 1. we thus have uniquely specified signature for X.
(ind) Repeatedly observe X through N independent ex- periments, in which case the "underlying" signatures 5(x"))o for n = ,,...,N,, N, are independent and identically distributed.
(chop) Chop-up (and shift in time) a single observation of the process (X,, t 2 0} over a partition
Definition 2.3. The signature of a canonical geometric stochastic process X,
I[N) NN,1 U... ((N 1)T + TN,N),
S = (S()) E E((R))) e [0,T]}
of [0, NT]. In this setting, we assume that the latent sequence {x", n 2 1) taking values in C(O0, :R:) stationary, i.e. for k E N, n1,..., 74 E and n
is defined pathwise (on a set of full measure) the unique extension of the lift of X to multiplicative functional of arbitrary order in the sense of (Lyons et al., 2007, The- orem 3.7). The elements of the signature are the rough iterated integrals of X.
R
X & (x"i+n ...., (..,,"+ttt))
(6)
3

---

Learning with Expected Signatures: Theory and Applications
and hence the signatures S(x") [0,2] form a stationary sequence. This assumption ensures the task of esti- mating (T") is well-posed. Note this condition is slightly stronger than necessary but weaker than requir- ing (X,, 1 2 0} to be stationary, cf. Proposition 2.13. (A,5) |E X
Assumption 2.6. For aorall =
(Aa) 1x S s|
Xu,
Lo/3 2 1t s|-
The first observational framework can be recast in the sec- ond by appropriately pasting the x" into single process (X,, t 2 0}. Going forward we hence focus on the second setting and refer to the large sample asymptoties N as long-span asymptotics. For any finite collection of words I, we thus consider the estimator
(Ar) |E X
2// S 11 - 8
u,t
82
Lx/3 x 8
(A6) [ X S s| s|
Remark 2.7. By the contraction property of the conditional expectation, the strongest form of (A,3), (Ar) and (Aó) is obtained by setting Fs,, = o(Xs,ux u [s. t]).
(T(M) (T)
Σ st (X".aN. p.M N
1
Theorem 2.8. Let k = maxte1 [I] and, for m 2 2, set mk. Assume % is a canonical geometric stochastic process that satisfies one of the following:
We will be interested in the double asymptotics where, as the number of signature evaluations N increases, the granularity of the discretized paths from which such signatures are computed also increases, i.c.
(i) (Ar) 1/22;
(ii (Aa), fAr) 1/2,6
[[I(N)
max |N,al 00 N + co. ISASN
(iii) (Aa), (A8) (1/3,1/2),8
We can decompose
(iv) (Aa), (AB), (Ar) (
69
(I(N)
(T) - (I(
with
1
(x( (a. -s'("))
N
(8)
2a - 1,
if), (ū), if (iii), iiv),
(2a - 11/2) A (a
N
(9)
[0.T] - [ [o.
3a 1,
40ABA7-1,
Under suitable conditions, we shall prove @1
(N) (T) is con-
and consider signature-definng cf. Definition 2.5, se- quence of refining partitions {T n 2 1) of the interval [0, T) such that
sistent and asymptotically normal for (T')) by showing
1. each summand in the first term converges to zero in m the in-fill asymptotics nol + 0;
Σ n21
| < 00,
2. the second term, when inflated by vN, converges in distribution to a normal random variable in the large sample asymptotics N 00.
then the stronger convergence holds
S*XX"") (0 L s (x),,
a n 00,
(10)
2. IN-FILL ASYMPTOTICS
with rate O |T 19).
The convergence in probability (5) is not sufficient to show consistency the expected signature estimator. In this section, we thus explore continuity conditions on the process X ensuring the convergence holds in a stronger d sense.
Proof. See Appendix B.I.
Remark 2.9. Note that, if {T.. n 2 1} is a sequence of dyadic partitions with [ 2T= then
Let (Fs.ts [s, C [0. T}} be a family of sigma-algebras such that, for [u, w] C [s C [0. Fu, S F.,t and, for [s, :] C [0, 2J, X.,u is measurabe for all u E [s,
Te
mn
c Σ nere
2-neee
1 2-c < 00,
n21
The following continuity assumptions will be used to state the in-fill asymptotics.
and the rate of convergence is O(2-me).

---

Learning with Expected Signatures: Theory and Applications
2.1.2. LONG-SPAN ASYMPTOTICS
with by No/2 non-overlapping wun.oueetapinnggrss-vvvvni crosscovvriinssss
Theorem 2.10. Fix T
and let {X,, t 0} be a stochastic process such that X (X,, E [0. TT} satisfies
M
S.. 21
1
SS X N |0.T]
91 2u(N) (T)
M
the assumptions of Theorem 2.8 with > 2. Assume {X", 2 1) is stationary and ergodic and the sequence of partitions {II(N) N 2 } is such tha, for each n 2 1, n..a = {TN.as N 2 n is signature -effinnng sequence of refining partitions, and
m=1
x S (x ((
11.TT]
1 (T))
for M = [NV(n2 + 1)] and
I S-.JI( : (Sp:
1. 1= N
Σ [U(NN) +0, + N'
(11)
is consistent for En, i.e. Σ I[((N 4 Σ as N + oc, and hence the CLT result of Theorem 2.10 can be made feasible.
Then the expected signature estimator (7) is
1. consistent, i.e. P1 (T)
(T((N) (T) E (T N 00.
Proof. See Appendix B.3.
If. moreover, {x", 2 }} is strongly mixing with mixing coefficient {a)(),, 2 1 such that, for m - > 0,
Requiring {x", n 2 }} to be stationary and ergodic or strongly mixing are high-level conditions. The following results give stronger but easier-to-interpret conditions.
Σ n21
a(()c/122 9 <0,
Proposition 2.13. Fix T O and let (Xt, t 2 0} be (12)
stochastic process. Then
and
{ t 2 0f is stationary
VN N Σ [I(N') 0, N
(13)
xx, 2 0} has jointly stationary increments
N'NN
{x n 2 1} is stationary.
where is given in Equation (9), then the estimator is also
If any of the above holds, and X is a canonical geometric stochastic process, then, for any collection of words I,
2. asymptotically normal, i.e.
NN T
(T) (T) (((T)) E(0,E,)) N + oc,
(X,, 1 0) is strongly mixing
{x" 2 1) is strongly mixing.
as long as Σg is strictly positive definite, where
Σ Var (S (xx (x))
Proof. See Appendix B.4.
+ Σ Cov (s*(()) )'
One might expect a similar statement to hold for ergodicity, but Remark B.6 shows that
n22
Proof. See Appendix B.2.
(X,, 0} ergodic + [X", n 2 1} is ergodic.
Remark 2.11. IF (L)(N), N 2 1} is sequence of ex- panding dyadic refinements, i.e. for each 1, nn is sequence of dyadic partitions with NN 22-NT, N 2 n, as in Remark 2.9, then [TT[N] = 2 N-N and, hence,
Strong mixing implies ergodicity and hence the second part of Proposition 2.13 yields sufficient condition (as far as (X", n 2 1} is concerned) for both the consistency and asymptotic normality results of Theorem 2010. Strong mix- ing is a somewhat restrictive assumption and hence one might wish to find a set interpretable conditions weaker than strong mixing ensuring at least consistency of the esti- mator. The following theorem gives such condition when [X,, 2 0} is Gaussian process.
VN Σ [[(''( 0 N2 N +0 N 00.
N'NN
Corollary 2.12. Assume the conditions of Theorem 2.10 hold with Theorem 2.8.(ii) satisfied for some > 4 and for any T > O. Assume furthermore rate of convergence of Theorem 1 as p(N) ~ - for some v & (0, 1). Then the kernel estimator
characterize the
We say (X,, 12 0) has jointly stationary increments for all v E N. 0 < 8; & 1, with = I...., n, and / 12 0,
(X X.nitt) (Xeto).ttor Xetsnitta)) (14)
(T(() 1
Σ
(aNN(N
1
5

---

Learning with Expected Signatures: Theory and Applications
Theorem 2.14. Fix T and let {X,, L 0} be a Gaussian process such that X (X,, t E [0, is canonical geometric stochastic process satisfying* (Aa) with a 2 1/2 and p 2. Assume the sequence of dyadic partitions of [0, T) is signature-eefning for X and for each N 2 1 let N, be the dyadic partition the interval [0,T]
The correction term S (X 1o,ry is inspired by considering
the continuous-time signature
(2) = S S (X) dx!()))
with mesh JT,, = 2 2-TT.
where the integral is defined in the Stratonovich sense. To preserve the estimator's unbiasedness while reducing the variance we aim to find a mean-zero control variate St (x)77 that is highly correlated with St (X) A nal- ural candidate is
Suppose {X,, 2 0} has constant mean and time- homogeneous increment covariance, iee. Vu, U, u,8,t,r 0
Cov Ku,,;Xa.)) Cov (Xutrutr: X xxtrttr);
S (X) = [-" (x), dx
satisfying, for some decreasing 0: RR+ + R2 with 0(t) + 0, + xo and Jo f(t)dt < 0o and m E N,
where the outermost integral is now interpreted in the Itô sense. If X is square-integrable martingale satisfying the conditions of Jacod & Shiryaev (1987, Theorem I.4.40), {S' (x)) [o,
(A0) |/Cov (Xu,, x, II S 0i1s- v1)l0 ulll s/.
for all S u S < S t with |s | 2 (1t- (1 Then the expected signature estimator (7) is consistent, i.e.
E [0, Т}} is also square-integrable martin- gale with E (u,77] O. Going back to the discretized setting, we note that, when X is a martingale, the discretized correction term 5xx*) (o.T) is also mean-zero and, hence,
61 ( (T) F (T) as N co.
the control variate estimator @) (TT) has the same bias as NNNrT (T), but, when picking the optimal
Proof. See Appendix B.5.
2.2. Variance Reduction via Martingale Correction
Cov(S5(XX) [. St
(X* (o))
In Section 2.1 we developed the necessary theory to es- tablish the asymptotic properties of the estimator (7) for the statistic o
(XX*) (3*) (0.r))
EiS" (.z7]. for any word I (f1.-.., .ie). This section aims find alternative estima- tor with better finite sample properties when the process X = -X,, E [0. T}} is a martingale. We restrict ourselves to the independent observation setting, with the same par- tition across samples, í.e. NN. = π for n = I....,N. N. We will hence be considering the estimator
it has reduced variance
V (T)) = (1- pÝ..) Var( NNN (T)),
where Pl,x == Corr(S' (X")(())))
[o. (x")
S (x (om))
In practice, to estimate c; the most straightforward ap- proach would be to the sample variance and covariance. In this case the estimator for ca is the slope of the simple lin- ear regression of (S' (x**"")) (o.)) n 1.....N) against { (x".*) ()..)) n = N} or, exploiting the mean zero property of the control,
1 N N
NN. (T) =
Σ St (X n=1
Jo.T|
(15)
where the X are i.i.d. piecewise linear observations of X over the partition ग. We introduce the cortrol-variate modification of the estimator (15),
c ΣN S'(X"" [.
" S. (XX")) 1o.T
S! (( Po. ΣN
N (.T) 0
1 Σ
10.11 -S'(x" )o,r), (16)
[.7]
In Appendix C.2 we propose an alternative estimator for ct derived using the properties of the signature.
where, setting I_1 = (di,..., 11-1).
Remark 2.15. This variance reduction technique is not lim- ited to processes X that are full martingales but can also be applied to partial martingales, i.e. X such that only a subset of the components is a martingale. this case, we can use the control variate expected signature estimator for any word I = (f1 (f1...., . such that X(i) is a martingale.
(x((**) |0.T
Σ Ju
s (x* 10. X
(tt)
"When a = 1/2, assume furthermore X satisfies (A5) with 8 2 and p- 2k where k = maxxer
'Note that by Friz & Victoir (2010, Chapter 14) any sequence of partitions with vanishing mesh size is signature-defning.
"We assume throughout Var(S! (X.) )p,r)) E (0, oo).
6

---

Learning with Expected Signatures: Theory and Applications
Even when the data generating process X is not a martingale, the variance reduction achieved by the corrected estimator (16) may outweigh the bias it introduces, leading to better performance - in terms of mean squared eтror (MSE) - than the classic estimator (15). In cases where the underlying pro- cess cannot be assumed to be a martingale we thus suggest to treat the martingale correction as a data transformation applicable in the learning pipeline (a model pyyper-paammeee in similar spirit the add-time or the lead-lag transform in the signature context) whose usefulness may be empirically ascertained via cross-validation.
CAR A bidimensional Continuous-ting Autoregressive
(CAR) process (Y,, t 2 0} of order p = 2 driven by standard Brownian motion with drift A = (Ai, A2) e (IR2x3)2. The CAR process is defined as the first d = 2 entries of its pd = 4-dimensional state space representation (X,; t 2 0}: an FinsteinUUhlenbeck process with drift and diffusion
02xx A2
- -I2x2
(02x2
02x2 12x2
AA
E
A
02x2
(Lucchese et al., 2023; Marquardt & Stelzer, 2007). We can apply the first set of conditions in Appendix D.1.2 to deduce that (X,, t 2 0} (and hence {Y,, t 2 0}) satisfies (Aa) and (A6), for a = 1/2, 8 and any p 2 2. Under (ind) sampling we can hence apply Theorem 201 Moreover, when AA has positive real parts of all eigenvalues and the process is started in its stationary distribution, (X,, t 2 0} and (Y,. t 2 0} are stationary, ergodic and strongly mixing with strong mixing coefficient a(l) = O(e=s), for some a > 0 (Marquardt & Stelzer, 2007). We can hence apply Proposition 2.13 to deduce that {X", n P 1] is stationary and strongly mixing with strong mixing coefficient (n)) O(e-anT), ), i.e. satisfying Equation (12), for (any) Ç >> O. Under (chop) sampling we can thus apply" the consistency and asymptotic normality results of Theorem 2.10.
3. Applications
3.1. Examples
We now consider a few concrete examples of continuous- time stochastic processes satisfying the assumptions of The- orem 2.10 and Theorem 14. Note that BM, CAR and Heston are semimartingales and hence, by Remark 2.2, they are canonical geometric stochastic processes such that any sequence of partitions with vanishing mesh size is signature defining. fBm is instead an example of a process that is not a semimartingale but is a canonical geometric stochastic pro- cess with dyadic signature-defining sequence of partitions (Remark 2.2). Taking (II(N), N 2 }} to be sequence of expanding dyadic partitions thus ensures the observa- tional assumptions of Theorem 2. 10 and Theorem 2.14 are satisfied by all four processes, cf. Remark 2.11.
The joint price- variance dynamics of a Heston model under the nestral measure Q with zero interest rate and no dividends, i.c. {(S,, Vo), t 2 0} such that
Heston
ds, = V.S.aw..
BM A standard Brownian motion (Bt, L > 0}. It can be easily checked it satisfies (Aa) and (A6), for any a 2 1/2, Z 1 and 2 2. Moreover, (B,, t 2 0} has stationary and independent increments and, hence, the (ind) and (chop) sampling schemes are equivalent: in both cases we can apply? Theorem 2.10 to deduce consistency and asymptotic normality of the expected signature estimator.
dV. = n(0 - V:)dl + EV V.W.Y.
where (W;S, 2 0} and (WY, 1. 0} are standard Brow- nian motions with correlation (W'S , = pt. Under the Feller condition 2x0 > <², tte variance process is strictly positive (and so is {Sc, 2 0}). The Heston model is thus an Itô diffusion with Lipschitz drift f R. x R + R and 1 /2-HHllder continuous diffusion o IR., x R. + 2 We can thus apply the third case of Appendix 1.1.2 to prove that {(S, V/),, Z 0} satisfies (Aa) and (A6) with a = 1/2. 8 = and any p > 2 for deterministic initial conditions So 8p and Vo vo- When paths are sampled under (ind) we can hence apply Theorem 2.10 to deduce consistency and asymptotic normality of the expected signature estimator.
[Bm A fractional Brownian motion (B.", t 2 0} with Hurst parameter H > 1/2. H satisfies (Aa) with a = H (Appendix E.2.2) and, hence, Assumption 2.6 is fulfilled. Under (ind) sampling, {IB",",, 2 1} is trivially stationary and strong mixing and, hence, we can apply Theorem 2. 10. When instead paths are obtained under (chop) we can apply* Theorem 2. 14, cf. Example E.2.2, to deduce consistency.
3.2. Experiments
7 'Brownian motion is Gaussian process with constant mean function and hime-homogencous covariance of the increments triv- ially satisfying (A6) with / = 0 and 0, it thus also falls under the scope of Theorem 2. 14.
Quite wide range of learning algorithms has been devel- oped leveraging the properties of the expected signature. The theory for such algorithms is usually developed under
'The increments of fractional Brownian motion are not strongly mixing (Mandelbrot & Van Ness, 1968) and, hence, we cannot apply the second part of Theorem 10 deduce asymptotic nor- mality.
'The CAR process is Gaussian process satisfying (A0), cf. Appendix E.2.1, and hence also falls under the scope of Theo- rem 2.14.
7

---

Learning with Expected Signatures: Theory and Applications
the assumption of bounded variation paths for the input pro-
Predictive Accuracy [%]
cess X, assumed to be piecewise linear. The results in Sec- tion 2.1 give the theoretical foundation for their probabilistic interpretation when the underlying process X is an, arguably more realistic, continuous-time stochastic process such as the ones discussed in Section 3.1. In this section we review a few algorithms from the literature, showcasing the practical relevance of the asymptotic results of Section 2.1 and the po- tential improvements achieved by the martingale correction introduced in Section 2.2. Code and examples demonstrat- ing the integration of the martingale correction into machine learning algorithms, along with the simulation results from the previous section, are available at ht https: / /ggitti thub. com/ 1 orenzol ucchese es g. The code designed to be compatible with Python-based ML pipelines, support- ing both numpy arrays and torch tensors.
FBM
OU
Bidim
GPES
95.62 (0.18)
62.20 (0.70)
79.33 (0.46)
GPES-MC
95.26 (0.70)
88.26 (0.31)
88.97 (0.44)
1-stat
1.49
-101.92
-45.52
p-value
0.15
0.00
0.00
Table I. Synthetic data experiments of Triggiaan & Romito (2024): GPES model without and with martingale correction (MC).
universality of the signature to show that a large class of path- payoffs F can be arbitrarily well approximated by a linear payoff on the signature, i.c.
price F) = [Z ~ (1, 2T (S( )),r1))))
3.2.1. TIME SERIES CLASSIFICATION
for set of linear coefficients f T((R))) where Q is pricing measure for X. Zr deterministic discount factor over [0, 7] and XLL denotes the add-time lead-lag transform of X. In Appendix F.2.2 we also discuss the corresponding hedging problem.
The first model we consider, introduced in Triggiano & Romito (2024), falls under the general task of time series classification, mapping an input path x E dR to a class label с C. The input stream is interpreted as a discrete- time realization of a Gaussian process, whose conditional mean and covariance pected signature of the latent Gaussian process, used as input to classification layer, is estimated by super-sampling the process. Theorem 2. 14 ensures this approach consistently es- timates the expected signature of the latent continuous- time Gaussian process, fundamental step for the probabilistic interpretation of the algorithm.
Given a pricing model Q for X, we can hence price via Monte Carlo simulations. This provides a classic setting for applying the martingale correction described in Section 2.2 since, under Q, the (discounted) price process X is martin- gale. In Figure 2, we compare the finite sample properties of the expected signature estimator with and without martin- gale correction when the price process is assumed to follow a Brownian motion (BM); in the context of option pricing, this is known as the Bachelier model. Similarly, in Figure 3, we plot the densities of the two estimators under the Heston dynamics 10 (Heston). Both figures suggest the martingale correction (blue) materially improves the classic estimator (red), and hence more accurate pricing is achieved by the modified estimator introduced in Section 2.2.
learned parametrically. The
We replicate the synthetic data experiments of Triggiano & Romito (2024) the (FBM). (OU) and (Bidim) datasets. The performance on the out-of-sample testing datasets of the Gaussian Process augmented Expected Signature (GPES) classifier with and without martingale correction is reported in Table 1. The output of the GPES model is by construc- tion stochastic and, hence, we repeat the evaluation of the model with 10 different seeds. In Table 1 we report the mean accuracy and standard error of the model with and without martingale correction (MC), as well as the results of an independent samples (-test between their accuracies. The martingale correction significantly improves the perfor- mance of the GPES model, a remarkable result considering that most processes in the three datasets are not martingales.
N 20
40
80
100
1-1110 10 1-
(2)) (id)
6 Molrgg (id)
(V")
3.2.2. PRICING PATH- DEPENDENT DERIVATIVES
Figure 2. Distributions of expected signature estimators for BM. The y-axis is in log-scale.
The next application we consider is purely financial one. The objective is to price (and hedge) path-dependent deriva- tives by decomposing them into a set of atomic Arrow- Debreu-like securities. Let X = (X,, t E [0, T}} be a price process, i.e. a semimartingale over some probability space. In Lyons et al. (2021, Proposition 4.5) the authors use the
IIn both simulations we fix T = 1 and consider x to be uniform with mesh |x| = 2 2- [N/10]+1 This choice ensures the sequences of partitions are signature-defining for both processes and satisfy the conditions necessary for consistency and asymptotic normality, cf. Remark 2. 1.
8

---

Learning with Expected Signatures: Theory and Applications
N 20
N
80
Predictive MSE [x 10 10-3)
1= 1=10,00 0g
N = 20
N = 50
N = 100
SES
1.49 (0.39)
0.33 (0.13) 13)
0.20 (0.08)
SES-MC
1.26 (0.48)
0.31 (0.09)
0.19 (0.05)
(1.100
t-stat
0.87
0.63
0.29
p-value
0.43
0.56
0.79
Edu
(T) (id)
(Tr) (iid)
Table 3. Rough volatility experiment Lemercier al. (2021):
SES model without and with martingale correction (MC).
Figure 3. Distributions of expected signature estimators for the Heston process with parameters so = L, vo 0.1,0 = 0.1, 0.6, = 0.2 and -0.15. The y-axis is in log-scale.
4. Conclusions
In this paper, we established new estimation results for the expected signature, model-free embedding for col- lections of data streams. Our consistency and asymptotic normality results bridge the gap between the theoretically "optimal" continuous -time expected signature and the em- pirical discrete-time estimator that can be computed from data. Morcover, we introduced a simple modification of such an estimator with significantly better finite sample properties under the assumption of martingale observations. Our empirical results suggest that the modified estimator might improve the performance of models employing ex- pected signature computations even when the underlying data generating process is not necessarily a martingale.
3.2.3. DISTRIBUTIONAL REGRESSION FOR STREAMS
Introduced in Lemercier et al. (2021), the Signature of the pathwise Expected Signature (SES) model aims to learn map from collection of paths, understood as an empirical measure on path space, to a scalar value, a task known as distributional regression. Under appropriate conditions, the authors show that linear functionals on the signature of the pathwise expected signature are universal for weakly continuous functions (Lemercier et al., 2021, Theorem 3.2).
We repeat two of the synthetic data experiments conducted in Lemercier et al. (2021), analyzing the performance of the SES model without and with martingale correction (MC). We report the average out-of-sample mean-squared error (MSE) and its standard deviation in Table 2 and Table 3, as well the f-statistic and p-value of pairwise t-test between the MSEs of the two models. While the results do not yield statistical significance there still seems to be mild benefit in using the martingale correction, especially considering that the processes of both experiments are not martingales !!
A Aknowleedemenns
This research has been supported by the EPSRC Centre for Doctoral Training in Mathematics of Random Systems: Analysis, Modelling and Simulation (EP/S023925/1), The authors would like to thank Nicola Muca Cirone and Will Turner for helpful discussions on the topic, as well as the three anonymous reviewers for their insightful comments.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Predictive MSE [x 1x100= 2]
0.35 x V/VN 72 0.65 x V/NN
SES
1.27 (0.23)
0.09 (0.03)
SES-MC
1.31 (0.45)
0.07 (0.02)
1-stat
0.29
1.41
p-value
0.79
0.23
Table Ideal gas experiment of Lemercier al. (2021): SES model without and with martingale correction (MC).
IIn the first experiment, when the particle radii are large and collisions are more frequent, one could argue the motion of the gas particles to be amenable to that of pollen grains in water, the original experiment which led to the "discovery" of Brownian motion by Scottish botanist Robert Brown 1827.
9

---

Learning with Expected Signatures: Theory and Applications
References
Friz, P. K., Hager, P., and Tapia, N. On expected sig- natures and signature cumulants in semimartingale mod- els, 2024. URL https: /arxivi.vv/..////// org/ abs/2408 05085.
Buchler, H., Murray, P., Pakkanen, M. S., and Wood, B. Deep Hedging: Learning to Remove the Drift under Trad- ing Frictions with Minimal Equivalent Mear-Martingale Measures, 2022. URL https::///// petttps://axiie aaaiaiiivvoo.oo org/ abs/ 2111. 07844.
Futter, O., Horvath, B., and Wiese, M. Signature Trad- ing: A Path- Hath-Dependent Extension of the flean-Variane Framework with Exogenous Signals, 2023. URL https: //arxiv. wwg. abs////b/// aaaaaass/a 2200 15135.
Burkholder, D. L.,, Davis, B., and Gundy, R. F. Integral inequalities for convex functions of operators mar- tingales. In Le Cam, L. M., Neyman, J., and Scott, E. L. (eds.), Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability, volume 2, pp. 223 223-240, Berkeley, California, 1972. University of California Press. URL https::////m / httpp//oo/ccccciiidcc arg/euclicn idd. www.b//sss.........m 2100551121200142211
Graham, B. Sparse arrays of signatures for online charac- ter recognition, 2013. URL abs /1308. 0371.
Hambly, B. and Lyons, T. Uniqueness for the signature of a path of bounded variation and the reduced path group. Annals of Mathematics, (7tprti 171(1):109-16) 2005.
Chen, K.-T. Iterated Integrals and Exponential Homomor- phisms. Proceedings of the London Mathematical Society, s3 53-441):502-512, 1954, doi: 10.1 1.551
Ibragimov, A. Some Limit Theorems for Stationary Pro- cesses. Theory of Probability Its Applications, 7(4): 349-382, 1962. doi: 10.1137/1107036.
Chevyrev, I. and Lyons, T. Characteristic functions of mea- sures geometric rough paths. The Annals of Probabil- iry, 44(6):4049-400 2016. ISSN 00911798.
Isserlis, L. On Formula for the Product- Coeffi- cient of any Order of a Normal Frequency Distribution in any Number of Variables. Biometrika, 11 1918. ISSN 0006-3444. doi: 10. 1093/biomet/12.11 1-12.. 134.
Chevyrev, I. and Oberhauser, Signature moments to characterize laws of stochastic processes, 2018. URL atttps://arxian org/ abs/ 10971.
Coutin, L. and Qian, Z. Stochastic analysis, rough path analysis and fractional Brownian motions. Probability Theory and Related Fields, 122:108-140, 01 2002. doi: 1..0077/000440000015..
Jacod, J. and Shiryaev, A. N. Limit Theorems for Stochas- tic Processes, volume 288 of Grundlehren der math- ematischen Wissenschaften. Springer Berlin, Heidel- berg, 1987. ISBN 9783662025161. doi: 10.1007/
978-3-662-02514-7.
Cuchiero, C., Svaluto-Ferro, S., and Teichmann, J. Sig- nature SDEs from an affine and polynomial perspec- tive, 2023, URL httph:t////// httpe:://axxv...oo.. absgg//bb/23000 01362.
Kallenberg, O. Foundations of Modern Probability. Prob- ability theory and stochastic modelling. Springer, 2021. ISBN 9783030618728.
Dragomir, S. S. Some Gronwall Type Inequalities and Ap- plications. Nova Science, New York, 2003.
Kiraly, F. J. and Oberhauser, H. Kernels for Sequentially Ordered Data. Journal of Machine Learning Research. 20(31):1-44 2019.
Fawcett, T. Problems in stochastic analysis. Connections be- tween rough paths and non-commutative harmonic anal- ysis. PhD thesis, University of Oxford, 2003.
Kulik, A. Ergodic Behavior of Markov Processes. De Gruyter, Berlin, Boston, 2018. ISBN 97831 10458930. doi: 10.1515/97831114588990
Friedman, A. Partial Differential Equations of Parabolic Type. Prentice-Hall, 1964.
Lē, K. A stochastic sewing lemma and applications. Elec- tronic Journal of Probability, 25:1 55, 2020. doi: 00.121420-EJJP44..
Friz, P. K. and Victoir, N. B. Multidimensional Stochas- tic Processes as Rough Paths: Theory and Applica- tions. Cambridge Studies in Advanced Mathematics. Cambridge University Press, 2010.
Lemercier, M., Salvi, C., Damoulas, T., Bonilla, E. V., and Lyons, T. Distribution regression for sequential data. In 24th International Conference on Artificial Intelligence and Statistics (AISTATS 2021), Proceedings of Machine Learning Research, pp. 3754-3762. Journal of Machine Learning Research, 2021.
Friz, P. K., Hager, P. P., and Tapia, N. Unified signature cumulants and generalized Magnus expansions. Forum of Mathematics, Sigma, 10:e42, 2022. doi: 10.10s./fms. 2022.20.
10

---

Learning with Expected Signatures: Theory and Applications
Levin, D., Lyons, T., and Ni, H. Learning from the past, predicting the statistics for the future, learning evolv- ing system, 2016. URL httph::t:/// aawaa.iiiv..oo org/ abs/ 1309. 0260.
Schell, A. and Oberhauser, H. Nonlinear independent com- ponent analysis for discrete-time and continuous-time signals. Annals of Statistics, 51(2):487-518. 2023.
Triggiano, and Romito, M. Gaussian Processes Based Data Augmentation and Expected Signature for Time Series Classification. IEEE Access, 12:80884-80895 2024. doi: 10. 200...33
Lucchese, L., Pakkanen, M. S., and Veraart, A. D. Estimation and Inference for Multivariate Continuous- time Autoregressive Processes, 2023. URL https:
(/aaxii.org/aiiai /207. 13020.
Willett, D. W. Nonlinear vector integral equations as con- traction mappings. Archive for Rational Mechanics and Analysis, 15:79-86, 1964. doi: 10. 10.10007/b002254000
Lyons, T. and McLeod, A. D. Signature Methods in Machine htttst:/a/xxiiv.oo /ttps///axxvvvooooee abs/
Learning, 2024. URL https::////o 2206.14674.
Xie, Z., Sun, Z., Jin, L.,, Ni, H., and Lyons, Learning Spatial-Semand Context with Fully Convolutional Re- current Network for Online Handwritten Chinese Text Recognition. IEEE Transactions on Pattern Analysis and Machine Intelliggnce, 40(8)) 2018. doi: N0.1099PPAMI.221.2222777
Lyons, T., Caruana, M., and Lévy, Disserential Equations Driven by Rough Paths: École été de Probabilités de Saint-Flour XXXIV-2004. Number no. 1908 Differen- tial Equations Driven by Rough Paths: École D' ééé de Probabilités de Saint-Flour XXXIV-2004. Springer, 2007.
ISBN 9783540712848.
Lyons, Nejad, S., and Pérez Arribas, Non- Kon-parametric pricing and hedging of exotic derivatives. Applied Math- ematical Finance, 27(6):457-494, 2021.
Mandelbrot, B.
Van Ness, Fractional Brown- ian Motions, Fractional Noises and Applications. SIAM Review, 10(4))422-45) -37, 1968. ISSN 00361445. URL http: / hwwp//wwww/a.///////////// jstor.rooooooo ottg/stablee//2711155
Marquardt, and Stelzer, R. Multivariate CARMA pro- cesses. Stochastic Processes and their Applications, 117 (1):96-120, Jan 2007. ISSN 03044149. doi: 10.1016/j. spa.2006.05.014.
Newey, W. K. and West, K. D. A Simple, Positive Semi- Definite, Heteroskedastictty and Autocorrelation Consis- tent Covariance Matrix. Econometrica, 55(3):703-708 1987. ISSN 00129682, 14680262.
Ni, H. The expected signature of a stochastic process. PhD thesis, University of Oxford, 2012.
Passeggeri, R. On the signature and cubature of the frac-
tional Brownian motion for H > 1/2. Stochastic Pro- cesses and their Applications, 130(3):1226-125 2020. ISSN 0304-4149 494.. doi: 10. 10.1066/j.spa.2019 3...000......
Pérez. Arribas, I., Goodwin, G. M., Geddes, J. R., Lyons, T., and Saunders, K. E. A. A signature-based machine learning model for distinguishing bipolar disorder and borderline personality disorder. Translational Psychiatry, 8, 2018.
Salvi, C., Cass, T., Foster, J., Lyons, T., and Yang, W. The Signature Kernel Is the Solution of a Goursat PDE. SIAM Journal on Mathematics of Data Science, 3(3):873-88) 899, 2021. doi: 10.1137/0M1166699.

---

Learning with Expected Signatures: Theory and Applications
Contents of the Appendix
Informal Glossary
B Proofs of Section 2
B.1 Proof of Theorem 2.8
B.1.1 Proof of Theorem 2.8 under (i), (iii) or (iv)
B.1.2 Proof of Theorem 2.8 under (ii)
B.2
Proof of Theorem 2.10
B.2.1
Proof of Theorem 2. 2.10, consistency
B.2.2
Proof of Theorem 2. 2010, asymptotic normality
B.3
Proof Corollary 2.12
B.4
Proof Proposition 2.13
B.4.1
Proof of Proposition 2.13, stationary implications
B.4.2 Proof of Proposition 2.13, strong mixing implications
B.5 Proof of Theorem 2.14
C Variance Reduction via Martingale Correction
C.I Martingale Continuity Criterion
C.2 Estimating cx
C,3 Proof of Lemma C.I
D Itō processes and diffusions
D.1 In-fill conditions
D.1.1 Itō processes
D.1.2 Itō diffusions
D.2 Long span conditions
D.2.1
Itô diffusions
E
Gaussian Processes
E.1
Gaussian Processes Continuity Criterion
E.2
Gaussian Processes Covariance Decay Condition
E.2.1
Cinstein-Uhlenber Process
E.2.2
Fractional Brownian Motion
F
Machine Learning Algorithms with Expected Signatures
F11
Martingale Correction in Applications
F.2
Algorithms
F.2.1
Time Series Classification (Triggiano & Romito, 2024)
E.2.2
Pricing Path- Deeh-Dependent Derivatives (Lyons et al., 2021)
F.2.
Distributional Regression for Streams (Lemercier et al., 2021)
F.2.4
Systematic Trading (Futter et al., 2023)
G
Controlled Linear Regression
G.1
Controlled Ordinary Least Squares (OLS) estimation
G.2
Simulation study
12

---

Learning with Expected Signatures: Theory and Applications
A. Informal Glossary
This informal glossary provides accessible explanations of selected technical terms and notational conventions used in this paper, aimed at readers with little or no background in rough path theory. These intuitive definitions are intended to aid the understanding of the theoretical framework presented in Section 2. particularly Definition 2.1. However, they remain closely tied more technical definitions such as those of multiplicative functionals, rough paths, and geometric rough paths which require more rigorous exposition of rough path theory. For concise introduction to rough paths, we refer the reader to Lyons et al. (2007), and for treatment in the stochastic setting, to Friz & Victoir (2010).
The p-variation of a path is a measure of its regularity. For the purpose of our discussion it suffices to note that paths that have finite p-variation for low p are more regular. A bounded variation (BV) path a path with finite 1-variation (also known as total variation). This regularity ensures there exists a well-defined notion of integral against this path (e.g. a piecewise linear paths or continuously differentiable path) and, hence, we can easily define its signature in Equation (2). Many interesting stochastic processes (e.g. those driven by Brownian motion) have infinite 1- l-variation (i.e. not BV) but have finite p-variation for all p >> 2 and, hence, defining their signature requires rough path theory.
p-variation
Convergence in p-variation Convergence in the p-vaiiation metric/topology is a pathwise mode of convergence (i.e. over all points E [0, I" simultancously) that is (much) stronger than the pointwise (i.e. at fixed t E [0, T') convergence required to state and prove our results. See, for example, Remark 2.4.
Spaces of paths We denote by C(10, TJ,,T..R).. Rd), resp. (VO, R*), the space of Rd-valued continuous, resp. bounded variation, paths over the interval T..
Mesh of a partition For a partition π = {0 to < t1 ... < T3 of [0, 21. we define its mesh as |T| max s.,]e It c-st where the maximum is taken over all sub-intervals of the partition.
Shuffle property The shuffle property of the signature an algebraic property stating that the product of two signature terms is a linear combination of higher-order signature terms. More precisely, the product of the signature terms correspond- ing to words / and J is the sum of all signature terms indexed by words K of length |7] + |J] obtained by interleaving I and J. In the context of the discussion on page this means that all moments of the signature can be written as linear combinations of higher order expected signature terms.
Signature indexing A word = (fi....,k, in) with i1,..., in E (1....,,) d) is a multi-index used to denote an entry of the signature, i.c. real-valued number. The length of the word, i.e. |4| = n, denotes the signature level, i.c. an n-dimensional tensor, to which such entry belongs. For example (' where I (1, 2), denotes the (1, 2n-ertyy of the second level of the signature (a matrix), while (2, 1, 1) denotes the (2, 1)-entry of the third level of the signature (a three-dimensional tensor).
Stochastic processes A continuous stochastic process X= (X,, E [0, T) over probability space (2, FP)) is such that, for each w E Ω, the realization X(w) = (w), E [0, E 11, Rd). If one takes Ω2 = C(O,, 21,RR IRd) and IP a probability measure over this path space then each w E Ω denotes a possible path realization of X. We thus say a property holds pathwise or almost surely if the set of E for which that property holds has probability one.
Canonical geometric stochastic process We define a canonical geometric stochastic process continuous stochastic process whose "higher order structure" can be approximated by the iterated integrals of its piecewise-lings interpolations (in probability in the p-variation metric). Its signature is then defined the limit of the signatures of its piecewise-linee -inear interpolations, i.e. the iterated integrals given in Equation (3). For clarity, we emphasize that canonicity here refers to the aforementioned construction of the signature, not to the underlying probability space on which the process is defined.
B. Proofs of Section 2
B. Proof of Theorem 2.8
Sketch of proof. The main idea of the proof to show the sequence of discretized signatures (S* (xx" )o.r). n 2 13 is Cauchy in T. Since T a Banach space this implies the sequence converges in I. By uniqueness of limits, we can
13

---

Learning with Expected Signatures: Theory and Applications
deduce this limit the same its P-limit, i.e. St (o. show the sequence is Cauchy in L. we proceed inductively on The main ingredient of the inductive step is signature level k' E {L.....,k) k3 under the progressively weaker norm L mmijk' manipulation of the discrete-time time signature (2), ensuring
SK (x*=+) ((oor11) Sh (x** (Toim))) [To (O,T).
can be written as a sum over time intervals kn41.[ru.ra) The inductive assumption is then verified by bounding this summation using Lemma B. I. when simple Minkowski bound is too weak. We use two different manipulations of the discrete-time signature under assumptions (i), (iit) or (iv) and under assumption (ii): In the former case we use the classic representation given in (2), while in the latter we rely on the "causal" representation of Lemmn B.3. For clarity of exposition we thus divide the proof of Theorem 2,8 into two parts.
We first establish couple of useful lemmas which will be used repeatedly in the proof of this in-fill asymptotic results, The first is basic result which is also applied in the proof of the stochastic sewing lemma (L.è 2020). In the following, let E denote Banach space.
Lemma B.1. Let (Zn, n= I.,,...,NN } be finite sequence of E-valued random variables in L with m [2, 00) and let 19.. n ,,...,,.... N} be filtration such that, for each {1....,N. N). the variables Zz, ....... Zu-1 Gn meaaurabae... Then
1/2
Σ
Zn
Bg., [Zn]]l + 20m
Σ
1z l1 L
L
Proof.
N
N Zn Eg, (2nl)
Zn
Eg, [Zn]
mm
n=1
mm
n=1
Lm
(a))
N
1/2
Egg, [Zm]
+ Cm
Σ
7n R.g. [?,]1
n=1
L
1/2
(Hi)
N
N
[Eg. [Zm]|l + Cm
Σ
2r, Eg,, 12..117.m
n=l
n=1
N Σ
1/2
Eg,, [2 C
(1) 1 m + |Bo (mmll
n=1
n=
N
1/2
[G, [Zn]]l + 20m
7. 112
by using in (i) the triangle inequality, in (ii) triangle inequality and the Burkholder-DDavis- Gundy (BDG) inequality (Burkholder et al., 1972) applied to the martingale {Mn, = ,,....,N' N with M, = >m (Z; (2]),, in (did) and (iv) the triangle inequality and in (v) the contraction property of conditional expectation.
Lemma B.2. Let P.P....... Pt E (0, co) U }+oo} be such that pi +... -pt p-1, then, for any set of tensors A₁ E L ((R )...., A₁ E L (Rdd)ook)
||A 8 ... @ Arllzz S d' | ... ||
Proof.
A| @ ... ....8A. ....&Alla Aillor
Σ
||A A 11
(w Jww
Σ
|A 1M1 ... ||A LM
14

---

Learning with Expected Signatures: Theory and Applications
ddutt.. AAlllee ". " A
where W3- = (I,...,dd d3 denotes the set of words of length k and, in (*), we applied the classical Hölder inequality.
We also prove useful lemma that allows to write the k-th level signature of piecewise linear path "causal" sum of lower order signature terms, i.e. preserving time order. This will allow to derive in-fill result with assumptions the regularity of Fu,, [x),), more natural object than VF., X X,, when 1/2, i.. Theorem 2.8 under (ii).
Lemma B.3. Let be a partition of [0, I and let π. Then, for k 2 O, we can write
g*+1 (x |0.1 =
Σ
s ' |0,0| & x u
(x
(1 +1+10 i)!
1=0
[u,
Proof. Note that for k 2 O.
ghtt1 (x" (0.l
[skt] (x*) (.,0) - -k - [o.w]]
@i
gk+ gh+1 (X" [0,, 2
u,v
sk+1
(x" [,u)
1==0
k+1
X
si
Σ
5k+1-1 (x"
2,0
00000 8
i!
[u,o|Emuor]
k
sk-
x @1++)
11 i)!
[0.0]
&
u,v
1=0
[u,0] [m.
where, in (+), we use Chen's relation and ([X))(u,)) exps X since X is linear over (u, r) E
B.1.1. PROOF OF THEOREM 2.8 UNDER (i), (iii) OR (iv)
Denote by (m.; n 2 1} the signature-defining sequence refining partitions of the interval [0, T). Without loss of generality, we can consider {T,, n 2 1} be such that T441 is obtained from n. by adding at most one refinement in each sub-interval, i.e., for each [s, t] W,, either [s, E #n+1 or (s, ul, [u, t] TT+1, for E (s, If not, super-sequence satisfying this property and then pass to the original subsequence.
consider
In the following, for any n 2 1 and [s, t] E nne denote by [n.[s,(] the restriction of t, to [s, f] and, abusing notation slightly, S ((**") ||s1)| -(% (1..)))))
Let (To: T1] TT. for N 2 1, and note that, for any and N, we can write
sk (x [oo)) (x [Tom))
[ [ (X ( ((7o7T))
S* an Mamll
(17)
[s.t]Emm..foovn)
where the partitions n,,s are defined as Tn.a = I,+1, U [s.Ts. i.e., for each [s, E T.e the partitions Th,s and Tn, differ by at most one point u E (s, t). Using Chen's 's relation and the definition of the tensor product, we can write for each [s, E] nn with refinement u (s,
Sh (X ) - -S (x*a..) (Fo.t.)
Σ 2..22.200 fitittis=k
Shh (x |ooel & )(mg)) S (xx(((())) @ Stx(X ).n)
(18)
Note that, for 1g E 1).
5ta (x - (xx(nn ) =
15

---

Learning with Expected Signatures: Theory and Applications
and applying again Chen's relation when ig yields
82
Sth ( [0,00
(x* [anuel @ 5t= ( [a,0]
1
12
2y 3,a
@
x
i2!
u,t
1=0
==
where we used the fact that, if Y is linear over [s, t]. then ((1) = expo Ye, which also implies
8 nm) 8.8
(Xe,u + xx,,)
X u )
i2!
iz!
5,u
[E(0,1) iEI
denoting by I E {0, 1 a binary number of length i2 with [Z| x E IRd. We hence have that
Eiez and recalling that x00 11x x, for any
(x*n++))))
St2 (x
(1.1]
Σ
C1
@i &
x
suu
u,t
IE(0,1)
A
where for I {0, 1)
i2
if = (I...., 1,0 0).
i2!
|Z]
CI
otherwise.
iz!'
Plugging this into Equation (17) via (18) and noting that Cr = 0 for I E {(0,.... .0, (I,...,1)) we can write, for any N 2 1, [oo TI] TN, n 2 N and k 2 2,
5 (X [man
S (x [man
Σ IEJ0.1122 i1t (I....))
Cz (X
X
x
x & (xx((())
u,t
22.22
uE(e.t)
We now proceed inductively to show that, for any E k} and any [To, Tıl aN with N 2 1, the sequence
(5tt (x" [ra.s)): n 2 N} converges in mm/fis with rate O 22 (.) and
sup sup N21 TToY
||5 (x**)* (Tom)) < co.
(19)
k' Note that for (To, TI] E nN with N one has 8 (%*T)) 170.71] = X for n 2 N, and
X Ta mm $ Im Tol% T < <0,
by Assumption (Aa). Hence S (X) = X E muk and the statement holds trivially.
Assume the inductive hypothesis holds for all E (I.....,k) R'} with k' E fl....,k 1}. Then, for each [To, Tı] E TN with N and N, let
|| sk (x (88m1)) -8 +1x+
(x Ira 1mb/444
Σ
|rrl
Z Io.t)Emn./To..al uE(s,t)
Σ
(20)
2, i &
IEfo.112
L
(0.0....)
where, for each [To, T1] E ny, n. with 2 N, 12 2 0, 2 with i + +
+ + is = and (0, f, we define
Zt.n == -XI'X(=-1! (To.)) & @i 8.1 & (
8 Sth (X (..))) [s, ol E Tn.(ro)))) with a E (s, t),
IET
16

---

Learning with Expected Signatures: Theory and Applications
keeping only the dependence on I for notational convenience. Note that, by applying Lemma B.2, the inductive hypothesis and Assumption (Aa),
(Z (1,0))1 (.mmar) S | "
[2] (x,l Lmnk
X
||S 1
I
(x
Lmkkia 2 1t - s
and, hence, each [%,11] L Moreover, by simple application of the triangle inequality,
Σ
Z [c.a]
| - s/a
(21)
[8.0]EmT..boo.11 aE(s,f)
7.. wE(s.t)
Assumption (i) Hence, if a > 1/2. we have for each (To, ril TN. Tn with n 2 N, 21, 2 0, i2 2 with
+ i2 = k'' + and I (0, 1)
Σ [8.1)Enn.Itoirll wE(s.t)
[sstt
[an|2a-1pp I mll-
(22)
Assumption (ii) If E (1/3,1/2) note that if is such that iz 2 3, then
Σ
| |30-0 |n
Tol
(23)
(5.1)E mE(s.f)
Lonk(M
but if i2 = 2 then the bound (21) is not strong enough. We can instead apply Lemma B. with the filtration (9),,), [s, f] n defined by
9/e.4| == F, V (X w. : [u, w| E <Tn.(t,)))
by noting that each z Ton with is -je.jj-eeaurabbl and mk/ (I' 1) for all K 1 S k. This implies
1/2
Σ
ZT.
Σ
Eg [2Teo] [mAA(44))
+
Σ
1z
[s. uE(e,()
(1(+))+
[st uE(s.))
Ts. uE(=,0)
1/2
Σ
1t- 5/ +
Σ
s
s/aa
[6.1)an..Imm uE(s.f)
(a.d)EFo..torr1) (E(sAt)
|nl
(3-1)((20-1/2
(Im ( Tol + |71 - -ol -20
(24)
where we used the fact that for I E {0, (1,0)),
12
1..4
gmn/(e111 11)
(
Sii (x [owo 2 Egq..
Ei &
x unt
2 S )
IET
(48)
| 11 )). mm///
Egr.t
000 s,u
x x, x((-1))
||Sts (x (x ) pma/t
JET
Lm2/2
(163)
Bggnnn Xx, & Xusellemers
(6x)
BJo VJar [x Xugill
17

---

Learning with Expected Signatures: Theory and Applications
by using in (i) measurability of 5t (x [rois] and S*a [t.ra)] with respect to 9/s.t). in (ii) Hölder inequality for
tensors Lemma B.2, in (iri) the inductive assumption (19) and the fact that A @ B|| = | @ All for any A, B E IR, in (iv) the tower property and the contractive property of conditional expectation applied to 9 S Fo. V FI.T and in (v) Assumption (A,5). Combining bound (24) when i2 and bound (23) when in 2 3 with a E (1/3, 1/2], follows that for
each (TO: TI] E nN, aa with 2 N, i 23 0, i2 2 2 with + i2 is k' and I {0. 1 33²
Σ [sL)|ERniITom1]] NE(aA))
x nx (8 1300011)111111)1
1/1 Tul
(25)
Assumption (iv) A similar reasoning can be applied when E (1/4, (and k 3) by considering the cases i2 2 4. d2 = 3 and iz = 2 separately. The case 12 2 4 follows directly from (21), the case i2 = 2 follows from (24) and the case i2 = 3 can be shown in the same way as i2 with the only difference being that we require Assumption (A-) to show
that, for {(0,, 0, 1), (0, 1, 0), (1,0,
F (TTT))) 12) & Fo. [x,N @ X
Lmk/3 S - 8/7,
and, for I E {(0, (1,,,, 01, (1,10))
Eg 12 + 2 EFo. [ & Xutlll 2 s|7,
so that applying again Lemma B.1,
1/2
Σ
[
Σ (5 uE(s.))
R (2aal 1mnk((44)
+
Σ
[s.])E=n..tro.1)] uE(a.))
uE(a,))
1/2
||- 1| +
8- Ga
[s:1)Emn, nE(,,t)
(s.t)En uE(a.t)
< Tnl (7 (3-1)/33-1122)
(I7 17--70 -ol + |
To 1/2
(26)
Combining the cases i2 2, i2 = and i2 2 4 when a E (1/4, 1/3] yields, for each (To, Ti) wN, on with n N, i,, is 2 0, ig 2 2 with i + + i3 k' +1 and {0,1)*
Z (a. uE(a.t)
Σ
Tn (8- (1--)))-- T1 Tol.
(27)
Lmk/('+1))
Defining c as in Equation (9), we can plug bounds (22), (25) and (27) into Equation (20) to deduce that
s (x Iroil - S* 24'+1111 (x ) l S |m -70||TTl
and, hence, under the assumption that Enz1 |Tr < oo, for any [T0.T]] TN with N 2 1, the sequence 1st'-1 (oom))) n 2 N} is Cauchy in L mk/k(kk+ Since mkk/(k+3 (K++)) is a Banach space the sequence converges in mmkk(kk+ +1) st'+l (x) L +1) (by uniqueness of limits) with rate
18

---

Learning with Expected Signatures: Theory and Applications
(x [no.n
8h 395+1(X*0 +X""""""" ) 1m
SI - T0
Σ
Tal 1
n'2n
n'2n
And, to complete the inductive step for k' + 1, note that for all N 2 and [T,, TI ETN
S 10
(x*M | 1.2//0/+1) $ || +x)+
(8) (7amm))) (.meree41 + Tol
Σ
l
n'2N
s* +1 (x) [mm +T
|
n/21
B.1.2. PROOF OF THEOREM 2.8 UNDER (ii)
In what follows we shall simplify notation and denote Exo,, by E,,
Denote by {In: n 2 1} the signature-defining sequence of refining partitions of the interval 7). Without loss of generality, we can consider (T n 2 13 to be such that TT+1 is obtained from n by adding at most one refinement each sub-interval, i.e. for each [s, 1] E ni, either [s, 1] E #a+1 or [s, u], [u, E nn+1 for a E (s, If not, one can consider a super-sequence satisfying this property and then pass to the original subsequence.
We start by showing inductively that, for any E (1....,k).
sup sup (X )1 60. n21 TEA
(28)
Note that the case i 1 is trivial since, for any 2 and Tn, by (Axr)
(x [0 10.1llmm Imme 11 S -Ta S T
Next, for the inductive step, assume that (28) holds for all i E [l,....' K} with 1/ S k 1. Then, by using Lemma B.3, we can bound for any n 2 and E An-
| +1 (X*")
(0.r) mm
(3)
1
Σ
Sk' & x av
(1+ i)!
1=
Σ
|| (X" ((0oi) 8 E. Xu,e (iak/ce-1)
1/2
+
| (x*T 10,1) (0.))
& X
1ma//e 12)
k
Sk S )
2 x x(++1)
(1 + i)!
0,
[u.t]Em..jo.vl
1/2
(10)
2
En X uul +
Σ
X2,0ll2 112
X Il 1+1
[u.ujEx..0o..)
[u.t]En...jo.1
1=1 [u.e]En...jo..)
1/2
k'
1w- u0 +
Σ
10 u1200
Σ
10 - u (++
i=1
[u,0]
T+
19

---

Learning with Expected Signatures: Theory and Applications
where in (i) we applied the triangle inequality, in (ii) we bounded the i = 0 term by applying Lemma B.1 to the sequence of random variables
Z|,,o| ;= Sk (x Th Ju.u u..) @ Xu, E L mm,//(k+1)
with filtration (Fa, [u, o] E [o.[0.-]] and we bounded the i l..... k'' terms by applying the triangle inequality, in (dit)
we applied the Hölder inequality given in Lemma B.2 and the inductive hypothesis Equation (28) for all signature levels up to k', in (iv) we used Assumptions (Aa) and (Ao).
Proceeding again by induction, we will show the conclusion of the theorem holds by proving the stronger statement: For each i E ......,.. for all N 2 T E z and n. 2 N,
|X TH 1) 10,0r) S*(*" (0.1) & |Tpl"
(29)
The case K' 1 is again trivial since, for all N 2 L, T E NN and 2 N, S () Xu,r, and hence
|| (x m (0.) s [0, me = O.
For the inductive step, assume Equation (29) holds for all E {l...., k} with W' k. Fix N 21, AN and 2 N, then we can write the telescoping sum
sk +1 Tn+1 )p.-) sk +1 +1(**" )u,r) =
sk''1 (x=+)x))))00 |0,71 - Sk +1 +x )0.0)) (x.m)))u.7)
(30)
(s.t)Em..to.rl
where the partitions Tn.s are defined as Nn.s = [00.]] U [n, i.e. for each [s, E Tn, the partitions Tn, and The differ by at most one point u E (s, Note that, for each [8, 4] E Ta with refinement u E (8, t), we can apply Lemma B.3 to write
Sk 1 (x [0,rl
5t 5x 10.7
2 x0(++1) + S Sk' s,u
S* 50''1(X 5m )0,01 ((++)) )0,o) & u,t
x 9(I+)
(1 + i)!
E
(x"mi) - Sk -3''i(Xmm. [000
x
x1++)
6,
F-i
Σ
gk (x' )0.1 2 ((++) - @(++)
+ Σ
Sk'
(x )o.0l X
8(++)
(1 + i)!
j!
a.t
j=
Σ
S -0 [,ol - 5th " -1x )1000 8
x
@(1+))
6,0
[0,0]] (fni
gk' 5k' )0.1) 2
x x(++))
x (0(++)) (1++]
(1 + i)!
x' Wー1-1
+
Σ
shi (x (
10,00 & (1++)) @ x(
(1 j)!
1=0
Σ
( ) - Sh -2''i(x -xx )10.01 2
(
+
2,00
8k' -t (xx )aoo)
Σ
x! s,u
& x x(()-1)
1)!
IEI
I=(.....,)) ().1.....
+
sk ao'-i-j-1 (= (x* 10.0| 8 x(++))
@ (1++)
(1 j)!
3,
1=0
20

---

Learning with Expected Signatures: Theory and Applications
Σ
sh (x (..1) - 5* - -x )10.v]
x@
0,30
[0,w]] (t..
by noting that, for all [e, w| E Tn with w Σ 8, S( " [0.0] = 5(x = S(X )(o.)) and when applying
Chen's 's relation to S(x*=*)) jo.aa. setting S(x = exps X since n is linear over [s, u| E An,t. Plugging this
expression into Equation (30) and exchanging the orders of the summations we obtain
8 +1 (% )0.r) - - -h''+1 (x) )000T]
1
Σ
Σ [e.d
Sk'
10.s)
[0,s] @
X s,u
Xu, x((l-))
(1 + i)!
[0(0
wE(s,f)
-
Σ
10,0] 8 x (1++))
@ x0(+))
+
(1+j)!
s.a
==0
uE(a,))
E
Σ
( )0.0) -xx-1(x=m..))
x x
+
[u,
K'
Σ
Σ
sk' 00.0] K
xol @l 8.1 & x
t)!
IE(0,1)
IEZ
(),
(1.(1.....)
wE(a,t)
----1
S0'-i-j-1(X )|0 2 (x(+)))
X
x((+)
j)!
1=0
[s.t)E=n.(orr]
mE(x,))
+
Σ
sk' [0,0) Sk' (xx )a.1) & x @(++)
0100
k'
W--111
Σ
Σ
4 +
Σ [ [0.r] aE(s.))
Z2,.. [..1]
23,
(1+ i)!
(1+
[0,10]
[s. NE(x.))
[o.u]E#n..oor))
We thus proceed to bound each of the summation terms over [0,(0,r] using Lemma B.1 and Assumptions (Aa) and (A5).
Let I E {0, 13 with I + (O....,0) 0), (I,..., with i E {l,...,,' K'.. Note that for all [s, f] E Tn,[U,7]
1. [
11 < sh- -xx=i(((xm))) (mm//(a-))) | l
r,mk
|X b.
1+i-| x lt 5) (1+1)a
(31)
by applying Lemma B.2, the uniform bound (28) and Assumption (Aa), hence [sst) E Lm1/((1-1) When i = we can thus apply Lemma B.1 to the sequence (Z/ [s, E Tn.[0.r), (s, with filtration {Fu, [s, E Tn.(0.r). E (s,t)} to bound
1/2
ZII
|| [ pmeeaenn +
Σ
1/ 12
[s.])ETT.((o.r) uE(+.))
(A(+)
[s.t)E uE(a,))
(s.t]Emn.(a.r)] uE(s,))
1/2
Σ
|t-
s
- 40
[s.t]E
[s.t)Emm..o.r]] uE(a.))
aE(a.))
S |m
1
2a 1/2 1/2
(T (7)
(32)
21

---

Learning with Expected Signatures: Theory and Applications
where we used Equation (31) with i and
| [ (5.1) 42))
g -xx=x11) x, @ Eu(Xu, Lmarart
gk -x |0.00 (maa/ar11)
| [ |[Eu (x pm S -|s+t
by applying Lemma B.2, the uniform bound (28) and Assumptions (Aa) and (A6), When i 2 2 we can directly apply the triangle inequality and Equation (31) to bound
Σ
I.II
[..d]
(a.)) uE(a,))
Lind 11)
(o. uE(e.))
< |t - =|s((11))))
nm 10-00-177 S |
(33)
[o. NE(a.t)
Next, let E {O....,k's 3 and j E KO,...,k'--2
- i--1- 1}. We can proceed exactly as for L (applying Lemma B.1 when -1).
i == = or the triangle inequality when i + j 2 1) to show that under Assumptions (Aa) and (A6),
II Σ
[x.t]]
S |T VT))
(34)
[o.1)EN... =E(x.))
Finally, let i (O....,k)) K). We proceed in similar way for 2 and z but using the inductive hypothesis (29) instead of the bound (28). Note that for all [s, t] E TT.[0,r]
[m.1] [s,t]
2 [e.0] Lmm//4+1 & (x )o.1 S )0.a) pmaeasss) eXeellltte 1+i
S | (1++)
(35)
by applying Lemma B.2, the inductive hypothesis (29) and Assumption (Aa), hence 7 T.mb/(((1)), When 0 we can hence apply Lemma B.I the sequence (7(o4)) [s, 1] E :n,10o=1] with filtration {Fx, [s: [n.10,11) bound
1/2
Σ
1E.1Z 2mm//// +
[1,, 112 mm2///+))))
[e.d)Exn.to.rl
[0.1EEm..10.))
1/2
|Tal(t - s|
| 10 - s|20
[e.0)Emn.to.vl
[0.t)En..1o.r)
| ( VT),
(36)
where we used Equation (35) with i and
E. 12 [s. pon//ore1 r.
= [ (x*++)) 5* (x*=))00))
8 E. [X, (me/(a
< ||s (x Sk (x**)
E, (x, (1,,)]l Imm S |Tm 8/5
LonkN
by applying Lemma B.2, the inductive hypothesis (29) and Assumption (Ao). When i > 1, we can instead directly apply the triangle inequality and Equation (35) to bound
(o.)) | 1m 2
Σ
|TT/' s|(1+()a 2 (T. (37)
[s.t)Emm..o.r)
[s.t]EnT..oor1
22

---

Learning with Expected Signatures: Theory and Applications
Combining bounds (32), (33), (34), (36) and (37) yields
SWh (x* [0.r)
5 (X [0.r]
Σ
ZLI [
11 t)!
(0.1
1s.
mE(s.))
1
Σ
Z
(1 j)!
j=0
[e.t)Emn.(oorf uE(a,))
Likk(a-
Σ
2 Lmbe(++))
[s.])ET..Io.v
nal (T + VF) x nn (T vT),
which proves (29) with i = k' + 1, completing the inductive step. Setting i = k = and r = in (29) yields
5 . (x=+))) P.T
5h (x ( 10. 11 S Imal' n21, 1,
and hence, assuming En21 | l 0o, the sequence {S (x 10,27, n 2} is Cauchy in L Since L a Banach space, the sequence converges in L to 5 (X) m (by uniqueness of limits) with rate
|| "x""))or - S* (x) 1x) 1m S Σ
|| (x 10.07 - 5*xx"**** )1.2711m
S
Σ |Tal".
n'2n
n 2n
Remark B.4. When (Mn, 2 1} is sequence of refining partitions with En>1 |T]] co, the proof actually yields the following (stronger) result
sup rEKn
[0.r]
Σ n22n
| - 00
B.2, Proof of Theorem 2.10
Sketch of proof. The proof of this result relies on decomposition (8). We can combine Theorem 2.8 with assumption (11) to show first term in the decomposition vanishes for m. > To show the full consistency result it thus suffices to show second term in (8) also vanishes L?, which follows by Birkhaff's s ergodic theorem under the stated assumptions. Similarly, for the asymptotic normality result, we combine Theorem 2.8 with assumption (13) show the first term the decomposition vanishes in L. when inflated by /N. The asymptotic normality of the second term can then obtained by a simple application of a central limit theorem (CLTT) for dependent random variables.
Under the assumption that {X", n 2 1} is stationary and x! satisfies the assumptions of Theorem 2.8 with m >> 2, we have that, for each n 2 1,
s' Momm 5 s (x) N oc,
with rate OlEN (n))) And hence
1
Σ
St (x a 10. )..7)
St (x"
(o.m)
Σ TN S Σ II(N')S
max SSSNN
N
mm
N'2N
Nº2N
since [II[N)] = maxignsN' |. Under assumption (11), we have thus established the first term in decomposition (8)
vanishes in T as N oo and therefore can focus on showing the second term also vanishes. Note that, under the stronger
23

---

Learning with Expected Signatures: Theory and Applications
assumption (13), similar reasoning can be applied when "blowing up" decomposition (8) by VN, and hence it suffices show the second term, when rescaled by vN, converges Gaussian random variable establish the asymptotic normality result.
We first prove the following somewhat technical result. In what follows we abuse notation slightly and write (0) to denote (')).
Proposition B.5. Let X (X,, E [0, denote canonical geometric stochastic process defined on the probability space (f(o.T) = R)) P(o.T)) by X = (io.Ti(t) for E [0, T] and wp,ry E Q.. Then for any collection of words I there exists measurable map St C( 7): 1 TR
[ such that
(w)or)) 5 10.M)
Plo.1 a.s.
Proof. By Remark 2.4 every canonical geometric stochastic process has at least one signature-definie sequence of partitions over any [s, f] S [0, 2 given by the sequence in Definition 2. Moreover, by passing to subsequence if necessary, this also guarantees the existence a sequence of partitions p. along which (5) holds almost surely. Hence, there exists a Borel set o.11 B) and sequence of partitions p* with vanishing mesh such that for all w[o,T] (o.7).
st S (o.))
St (Wjo.T)) 10.r7'
lp.1 + 0,
[0.7]
and (I..T((9%O.T)) = 1. For every partition the map
WP, E No.TI + St (
IR 10,2]
is 9 1o.r n measuralll (by measurability of the sums and products of coordinate maps appearing in the discretized signature) and hence also
Wj..21 E No.r + ST ((wo.21)
ER
[o,3]
is 2 f% O Bi
[o - measurable (by measurability of the pointwise limit of measurable functions). We can hence extend
ST Ω 0.7] + ( to a measurable map on the whole Q ([ Rd).
B.2.1. PROOF OF THEOREM 2. 10, CONSISTENCY
The consistency result then follows by a simple application of Birkhoff's ergodic theorem (Kallenberg, 2021, Theorem 25.6). Let (12 = C(OO, oo);; = Bo) P) denote the canonical space on which (X,, t 2 0} is defined, i.e. IP is the law of [X., 2 0}. Consider (x", n 2 1} as the sequence of C(10, 12:: random variables on the space obtained by pushing forward IP by the measurable (C([0, R ) To. (o..))) where the probability measure P [0.7]
mapping
w E 1- (u%O.T n n E C(O,TT::R)))
where
-(b.r] = (w((n 1)T + w(nn 1YT), E [0, 7]}.
If [X", n 1) is stationary and X X is canonical geometric stochastic process, then each X" is also canonical geometric stochastic process (C((0, T): RRdd) Bo P. with signature given by the same measurable mapping St C(O. Rd) + | given in Proposition B.5. We can then apply Birkhoff's ergodic theorem (Kallenberg, 2021, Theorem 25.6) conclude
s
[o.
P-ga F(S*(%)
N - +0000 0o.
N
n=1
Reecll that if ((E) is Borel measurable space and we equip 0 i.c. the space of sequences with values in E, with the product topology then
BEE*) = (o2) (B(2)(((E)" E²)
as long as E second-countable.
24

---

Learning with Expected Signatures: Theory and Applications
B.2.2. PROOF OF THEOREM 2.10, ASYMPTOTIC NORMALITY
We apply the dependent central limit theorem (CLT) given in Ibragimov (1962, Theorem 1.7) and extended to the multivariate setting via the Cramér-Wold theorem. To apply this result require {[* (x).7.) n 2 1} be stationary with St p. E I,2+6 and strongly mixing with strong mixing coefficient a(n), n E N satisfying
Σ a(() < 00;
so that
Σ1= = Var (5' +2 Ecor
Cov (x'( ) (x))) < co, n22
and, if En is strictly positive definite,
VN
N Σ
St (X" 10.7 - EES (x) (000027))
-
Note that S"(X") [.,j E L2+5 for <> 0 is immediately obtained by applying Theorem 2.8 with m > 2. By measurability of ST C(O0, R) + R (Proposition B.5), (SS (X")) S (xx). for all 2 which implies (S*((")) (0,7] 2 1} is also strongly mixing with strong mixing coefficient at most a(n), n E N. The assumptions of Theorem 2.10 are thus sufficient to deduce asymptotic normality of the expected signature estimator.
B.3. Proof of Corollary 2.12
Sketch of proef. In order to show that the CLT result of Theorem 2.10 can made feasible we need to prove the kernel extimator SBIN) consistent for the long-run covariance matrix Σx. do so, we first show that the kernelized estimator is consistent for Σt if for each 2 O, the cruss-covariance estimator term So)) consistent for VT (with "fast enough" convergence rate). To show consistency of each cross-covariance term, we introduce an auxiliary process Y" obtained by appropriately "stitching" the processes X' ,...,% z144 together. This choice of Y", along with the shuffle property of the expected signature, implies that each term in 5h can be expressed as a combination of terms from the expected signature of Y", The rest of the proof is thus devoted to showing that, under the assumptions of this Corollary, the process V satisfies the conditions of Theorem 10. ensuring we can consistently estimate its signature terms and, in turn, the entries of T.T.
"..,,%,+*
To make the CLT feasible one requires consistent estimator for the long-run covariance of the sequence of random variables {S"(X)) (o.)) 1),
E 20 + Σ 27, where 27 Cov (***" n 0.
We consider the non-overlapping sample (evvrieees of the sequence {St (x No.T 1, 1,...,,,,, i.c. for |n| < N --. 1,
[N(n+ Σ 1=0
S.,I((N)
)o.21 (T))
(I( (I((N) ) (xFx..mm))) )0.21 01 (PLN)(TT)
T
[N/ 1)J
as estimators for ET Note that, for fixed observation partition II(N), we are only able to estimate ET up to n = N -
with the quality of the estimator decreasing as n ! natural choice would thus be to put less weight on E )
In this context, one might exploit the available data more efficiently by considering the full sample cross- covariinvss of the sequence (g )o,, 1, .N. i.c. for [n| I N 1,
S Jo.m 31 (H)))(T)
(5 ml )0,r - -2 -2)())(T))"
N
25

---

Learning with Expected Signatures: Theory and Applications
with n large than on 11 (T,TT(()
with small n. To do so, one can consider the kernel estimator
S
I
Σ (N-
N-1
&
n hw
1
ST f = (ST (E
T,
for n2 = 1,...,, NN 1,
where k R + [0, 1] is a decreasing kernel function continuous at zero with with k(O) and by is an appropriately chosen band-width parameter". In what follows, and in the statement of Corollary 2.12, we set k be the truncation kernel {(x) 1 (-111 for simplicity, but other choices, such as the Bartlett kernel, might lead to better finite sample properties.
For each I, E I, if 2 ST.TI(N) is consistent for Σ9.J in L2 with monotonically decreasing rate r rMM) - 0 as the effective sample size M = (n + 1)J + co, then
1,, E1,511 S
( PI
+
Σ
12
|m/>AN
Σ ml/s
(IN/(n+1]]) +
Σ |m|>dy
If the band-wides hy at which Nn to be slow enough ensure also the first term converges to zero, then consistency of the estimators (.,, for 2 0, is inherited by Si),")) Under the assumption r(M) M for (0, (0,1), one can set hv NV/22 11
N oc, then Σ1 ensures the second term vanishes. If, moreover, set the rate
We can hence focus on determining under which conditions, other than those of Theorem 2.10.2, the estimator 11,J En.II(NN consistent for E7 for any and I,J I.
Note that, we can apply the shuffle identity to write, for I E I,
(O.INN) E
Σ
[u(N)
(
(T) 06 (T) (T),
mI(M) (
ØK
KETWJ
and show consistency of this estimator for ΣJ., by applying the consistency result for the estimator of the expected signature terms to K E I -ww. for I e I. We now attempt to apply a similar approach to the cross- cross-covariance terms. To do so, we introduce an IRnd_valued auxiliary process!5 defined as Y (Y,, E [0, (n + 1)7]}, where
X
X x2
X
Y
te (0, = Y
X
t= iT +, sis n.
r++1 (1-
0
0
By construction, we have that, for any two words J,J E over the letters (1.....d).
5' (x |o.: = S (Y"") (o.(x+11))) and S S (xx*+)) (0.7 = 5nd+J (Y" (0
Le. we have re- re-written the two signature components of x and X" over [o, T] as time-overlapping signature components of y over [0, (n + 1)T]. We can thus apply the shuffle product to deduce
S (x) 1 S (x"))1
|0,21 St (1.000111 (Y"))t"("""")""/""")""""
Σ
S (K("")ou
KEIu(nd+J)
estimators for Ef with 0. The reason for using the nve-overlapping sample covariance will become apparent once we introduce the process Y", which will be used to show consistency of the estimator (.n. for ET by re-applying Theorem 2.10.1. To show consistency of the full sample covariance estimator would instead require the generalization of such result for time-overlapping expected signature estimators.
"In the presence of conditional heteroskedasticity such estimators for the long-run covariance are known as Heteroskedasticity and Autocorrelation Consistent (HAC) estimators (Newey& & West, 1987). Note that decomposing the estimation of 2ET into the sum of fox. (NN) and its transpose ensures the resulting long-run covariance estimator 2 (1I(N), is symmetric.
We would like to thank Nicola Muca Cirone for suggesting this clever trick.
26

---

Learning with Expected Signatures: Theory and Applications
We thus re-write the (1, J-th entry of Eq
Df E Ss (x
(0. (x [a.n] - - (x*
(x [o.r]] E [S ( (x 1o.l
[S (1'
(1 Sma+t
S (t" (0, SS (x) E (x)
E
E
[SK (0 E [s (x) E SS (2)
KElw(nd+J)
Σ
5x + (TT)
Kelu(nd+J)
and estimate by
Sp.
Σ
((n + 1)T) 37 ( (T)Ø) T (T),
ØK
Ke/w((ddJ)
where, setting M (N/(n 1)1,
M
((n 1)T) = M
Σ
(k(xXX,mm)) (p [1(+1]]]
m=1
and
II(N:)) NNJIm U 1)(n+ 1)T + aN.M.))
where for each I,
N,an = (f U.-11)1--...... ....U (nT *N,m()+1))
partitions [0, (n + 1)7] and
[I(Ns=)]
max I5m5M
TN.mml
max M (5i5(n+1)
max
(N S [[[[N]]
[(())))) (n''), we need to understand under which additional In order to apply the consistency result of Theorem 2.10 to y
conditions on (X,, t 2 0}, other than those of Theorem 2.10, the process (Y., 2 0}, defined by
1) X (-
Y Σ
1 120
m21
m
(t-(m--n1)77+ta
satisfies itself the assumptions of Theorem 2010. Note that canonical geometricity of the stochastic process X (with lift-
defining sequence of partitions p. lel - 0) and stationarity of (xm m 2 1} imply that each Xm is a canonical geometric stochastic process and, hence, so is Y (with lift-defining sequence of partitions p" = (T +p) -UN. + +p), , lp| + 0), Moreover, stationarity and ergodicity of {xmh, m 2 1} imply stationarity and ergodicity of { ..... X m 2 1} and hence of the measurable transformation {Y f(x" .....) m 2 1}.
For fixed n 2 0 and m 2 each n. T.,m, N m.n: [N/(n + 1)I 2 m] a signature-definnn sequence for === y", over [0, (n 1)T) since each T.. (TT,,, N n) signature-defining sequence for X over [0, T). Moreover, by construction of nn, [I][Nnn)] < [I(N) + 0 whenever [I(N) + 0 as N +
It thus remains to check that Y y satisfies (Aa) and (Ao) over [0, (n + 1)T] with a 2 1/2, 8 2 and > 4k with = max [7|- Clearly, as we are now considering the process over an arbitrarily long time span [0, (n + 1)7], we will require (X,, t 2 0} to satisfy (Aa) and (Ao) with a 2 1/2, 8 2 1 and p >> 4k not just over [0,T] but for any S s S t with [t - s| < T. This condition automatically fulfilled when (X,, 2 0} is stationary (or, more generally, has jointly stationary increments) and satisfies (Aa) and (A6) over [0, with a 2 1/2,
1 and p 4k with k max [ It turns out that these conditions, only slightly stronger than those already required in Theorem 2.10 for the asymptotic m1(())))
normality of 01 (I((N) (I), are sufficient to show that v ((n + 1)T) is consistent for u/x ((n + 1)T), and thus (n. consistent for Σy, for any n 2 and 7, I.
27

---

Learning with Expected Signatures: Theory and Applications
To show (Aa) holds for y with the same 1/2 (X,, 12 0} note that for 0 S s (n (1
Y
I
X i+1 (T- AT
s.t
km0
XiT+(s (TT6----)1)A777744---0) ATll
(1 i AT - iT)* ATT 1 5 5/
Next, to show that (A6) holds for Y with the same ā 2 1 as (X,, t 2 0}. Note that for 0 558515 (n+ 1)7,
E. E. LF S
E, x X (---+
(3..:+ AT (
1=
(1- (1)+ AT - iT) ATI S 51
k=
B.4. Proof of Proposition 2.13
B.4.1. PROOF OF PROPOSITION 2. 13, STATIONARY IMPLICATIONS
The first implication follows directly from the definitions of stationarity and joint stationarity of the increments. It remains to show the latter implies stationarity of {x", n 2 1), i.e. Equation (14) implies Equation (6). Under joint stationarity of the increments, for all E N. ,1,..., E and n 2 0,
P( E , x E As)
=P(X (m (IT 1/7+ E Bl.-. ....... (m B 17 Bi...., -1n-ta-17//nttt----T E B
=P(X++* E A xma-n E Ak),
for all Ai...., Ag cylinder of the form
A₁ = (wo, E C(O,TT):R*) w(th) e Bf....,,u(P,, E BynB
for BI...., m, E B(Rd), 4..... Mn, E [0, T). m, 2 1. Noting that the collection of the sets A x... x A₁ is a semi-ring that ge the olagebraa B/o.rr.
apply Caratheodory's extension theorem conclude that (6) holds.
Rccall that for a set I C IR+, the Borel alagebrra B, (3C((;R))) (w.r.t. the topology induced by || (loo) can be equivalently defined by
Bi = (WE C(;:") w(th) A (t....,(ta)) An, th E 1, AI,...,AA An E B(R 2 1).
Moreover, if (F,, (3F)) is Borel measurable space and equip E with the product topology, then
B(F²) ((E)* == a(Ai x .... Az, AI...., A E B(E)) a({ x X A A As eg).
where G is a generating collection for B(F), i.c. B(F) = (9).
28

---

Learning with Expected Signatures: Theory and Applications
Remark B.6. One might expect similar statement Proposition 2.13 for ergodicity, but the following counterexample
shows that
(X,, & 2 0} is ergodic + {x", n 2 13 is ergodic.
Let (X,, t 2 0} be an R-valued process such that
X, = sin
+ Φ
2 0,
and Φ ~ ([(0, 2r)), inducing probability measure 0.2] on ( 1o.r)) where Rpor1 (C([O,T],R)) The process
is stationary and ergodic. Stationarity of (X,, & 2 0} implies stationarity of {x", 2 1} by Proposition 2.13. But the shift-invariant set
1 = II (woo. E 120 U Ign) E [
n21
where 120. Igo E Bi are given by
120 [Wp,2]] (1) 2 0, Vi E [0,7]]
Lgo == [Wj,rr] Wj. (t) S 0, Vt E [0,
has measure Piorr (1) = PPO E [x/2, m| U [3z /2, 2m)) = 1/2 $ (0, 1) and hence {x", n 2 13 is not ergodic.
B.4.2. PROOF OF PROPOSITION 231 STRONG MIXING IMPLICATIONS
We start by showing that strong mixing of (Xt, t 2 0} with strong mixing coefficient a(s), s E R4 implics strong mixing
of the progressive increment process {X?" 1 0} where
x = X UITTTT 2 0.
with strong mixing coefficient a'(s) S a(s 27), 2 27. This follows immediately from the definition of strong mixing and the fact that for t 2 0,
o(? u S S ((... u t),
and for 2 27,
OX u (2 a [(t +)//T//T C Co u uzt (2 (2--())- 2T)).
Next, we show that strong mixing of (XT, 2 0} with strong mixing coefficient (s(s), s E R, implies strong mixing of
{ n 2 13 with strong mixing coefficient a" (n) s a' ((n 1)T), n N, and thus a a"(n) S a((n - -3)T), n 2 3. Let
x = a (x., u E [a, b))
for a, b E R4 with a < b and let
St o m < < n), for m, n E N with m & n. Then, by definition, for each n E N,
T (n-1)7
11T++ 1 E [0,T]) = =S
Thus for any m, n E N with m En n
S = puf
(m-1)T-
Letting N and A E Sk -00 B E Sktn we thus have
(P(A P(A))(B) 10 +0, 00.
29

---

Learning with Expected Signatures: Theory and Applications
B.5. Proof of Theorem 2.14
Sketch of proef. As in the proof of Theorem 2.10, c.f. Appendix B.2, we can apply the in-fill result given in Theorem 2.8 to show the first term in decomposition (8) vanishes. For the second term, we show the sequence of random variables n 2 1/ satisfies a (weak) law of large numbers by verifying the auto-covariance decay condition (38). For each fixed lag n 2 1, we start by bounding the auto-covariance of lagged discretized signatures (S" (o.T, n 2 1}. This step crucially relies on the Gaussian assumption when using Isserlis' theorem to compute the expectation of the product of (arbitrarily many) path increments. The required auto-covariance decay condition is then obtained in the in-fill limit along a sequence of signature deffiing partitions p by an application of Theorem 2.8.
(S*("")) (x"))) [o.r;
Note that for any n E N,O 8 S 1/ for i ,,...,, and 2 0
(Xtte, Xeten lte (Xer. Xendals
since both vectors normally distributed with means
E ([tts, E[X,,, O,
for 1,....,h and covariances
Cov (Xitsi,,th, X(ts),4t1,) = (.,, C(It; - s,1, 1s; -tl. -syl),
i,j 1,... .,...,,,, This implies (X,, t 2 0} has jointly stationary increments and hence, by Proposition 2.13, {X", n 2 1} is stationary. By Proposition B.5 the sequence [S (x"), 1} is defined P-a.s. and is stationary.
Note that by Appendix E. X satisfies (Aa) for any 2 2. When a 1/2, it also satisfies (A6) with ō > 1 for all p 2 2 by assumption. We can thus apply Theorem 2.8 to obtain an L in-fill asymptotic result along a sequence of dyadic refinements. By the discussion at the start of Appendix B.2, we can thus focus on showing the weak law of large numbers holds for (X"),, n 2 1}.
To apply the weak law of large number for dependent random variables, note that:
1. (1((") 1o.r T.2, by the in-fill asymptotic result.
2. To show Cov (8*(X)) (X*) [027y; St (x)=))))))))))))) 0 as |m oc, by stationarity of (Xn, 2 1}, it is sufficient to show that
Cov (s*(()))o (x))),17, SK (x") (x")) 0, - oc,
(38)
for any S
k' < max //].
To show (38), we wish to find a bound on Cov (x²))..77> Sk (X" (1.1)) vanishing to zero as n co. Note that, by the in-fill asymptotic result, along the signature-defiing sequence of dyadic refinements, for all n 2 1 and < k < k.
10.27, sk (xP')) (0.)) + Cov (s* (x) J.27, sk (x") )om), lp + 0. Cov (sk (xx:1) (xx11))0
(39)
To bound Cov(S* (x)) (0.77 SK' (X") (OOT)) we can hence start by bounding the covariance between the discretized signa-
tures Cov(Sk (x |o, 5k (xx*"))o and then let |p] + O.
Let be dyadic partition of [0,2] and note the form of the discretized signature
S(X)) 10
exp@
X
[u,oeE
implies
X
((/,,)) u,
SK'
(XP)
(0,7
Σ
&EMMY Ju.n|E
30

---

Learning with Expected Signatures: Theory and Applications
where
M tip = (u 0 0. [u,,o S K', [u
Σ
k},,
[u,u]Ep
is the set of multindices over p with sum of components k'. Note that we can rewrite this as
X
Shh (x') 10
Σ
Σ
Σ
it!
j=1
$1ownyy 21 [11.11] <---s-eeeee Slujsuee 1=1
II it!
ucg 1lg?
j=1 11mng22 21 [un,e]| < uysuylEp
x=1
$1 Hi
where we first split the sum over the number of non-zero M[..o] for each ip E MM and then rewrite the tensor product by
introducing (4, ) = ,,, 2,,, 2,............................ j...,, j), where the index is repeated i times, the index 2 is repeated 22 times, and so on. We introduce the notation [u1. 01 <m/ [u2, Ø22 denoting at least m'' intervals between [u1, v1 and (u2, 02] in p. When m' = 0, we equivalently write < or <n to denote the interval [u2, 22] being after [w1, 21) in p.
We then group the above summations over the intervals [u], vil,--.. [uy, uy] over all possible combinations of time intervals with at least pair less than steps away in the partition p. To do so, introduce the set variable I @o J E j.m where I (1) (0, 135-1 is such that [[] Iı + + Ij = j' and J (1....,m)-- where m is such that (A0) holds. We can then recursively define, for = ,,....5.
interval Ji steps after [w-1, (I Bo J) [up VIZral
if I 0, if I = 1,
[w, vT] Go J)
where Inal=
In +... +..+ I,, and write
k'
8 (
10.2
Σ Σ
Σ.'
II L==l
X Jas .n, ((DBo))
in!
21 =ky=
='=1 IeaJ EE, a.m a.m [m1.n]] Smy ""Snnn-ll
i
where, for each = =1, ......j' 1, we have [ur, ur] Smp (1 11) i.e. there are at least my intervals between [up, my.] and 41, +1] in p. where
my = mr (I Do J) =
Ji 1l (1Z,)==1 Σ 1=1
The structure of the tensor products in the summation is thus
X Sh Un.e
X
Sig
Sing
Ging +2 x
X
Giny +n2
us of
where [u1, v. | [u] vfi,.... ... [u])) o(')] (and so on)
are all less than m intervals apart and [m2, Uy] is at least m intervals after [u]] of
Let n 2 and define Pn = (n 1)T + p. Then, for each word = (W1,.... wx) of length k', we can write
Cov (s' ) s' (x²"")).rr)
Here I tb I denotes pairing elements of J to the elements of I equal to O.
To help intuitive understanding consider the where kú' 2 and
1, then the above expression reduces
82 x
22t X |o.
Σ
Σ [ML-)]EP
X @ X
Σ
Σ [ug.9a]Ep
X
X
2!
[spel]ee
[1
31

---

Learning with Expected Signatures: Theory and Applications
Σ
Σ
hiz= M1mny 21 4t t-
11-1/
1/
I 1
1 22.32mm
"....
II
Cov
II x (we)
14g oy [c BoJ1)
x (ma)
log 12 (g
"
19-1 "/ le
x=l
x=1
[siitile mi
g m²
-/-1
Σ
Σ
Σ
Σ..
it!
$1uml1 Ar 1-1 Bo fi ln of "myy "og " JEp
4 =k ext =t
I2 32 bn 5m3 mm 33
Σ
II
Cov
(x(wa 161 lug "g 12 1o3 log, 1/g ((330055)
X (Wan ((1-11)
x (Weg ((1-82)
101 11 (1
1912, 112, (2
PE PEMPa (2.M)) (f2:9g))Ep
(161.11)
by using the fact that, for two collections of zero normal random variables (Zo.,.... Zo and ,1, we can apply Isserlis' theorem (Isserlis, 1918) to show
Cov (Zo,1 ... Zo, Z1,1 .. (I,)))
= E |0,1 ... Zu, Z,, ... Z1,x - E /u,1 ... Zo, [ [21 ... Zz,xrl
= Σ
E
Σ
E ZZZo
Σ II E MEPG (A.j)r
((P()
(2,K)) (dy
((11)))
MEP (i.jeeg
Σ
II
E BiZ Synizll
PEMPa (B,)) da)le
Σ
II
Cov (Zi 2ja.ja)
PEMP2
(0,0) (a..eee
where P (2.12 denotes the set of all the pairings of (0, [l....,k}). P7 denotes the set of all the pairings of {1...., k'}
p E MP2 (2.07 denotes the set of all the pairings of {0, 1} x (1....,k²
and MP²
R) that contain at least one "mixed" pair, i.e. for all (2,4") there exist ((11, 12), (21,12) p such that i # j1-
Note that each [ur, my] for = 1,.....i A appears in at least one covariance term. We proceed by cases:
If [y] appears in pair with an interval [u., v.] such that [uv-1, uv- < [u_, < [ur41, vv+ then
|Cov(X
(w) [urr,y]] eu.,e.))l S. loy- - url
x(())
by applying Assumption (Aa) with a 1/2 and the fact that p is dyadic (and hence uniform).
If [ur, uy] appears pair with an interval [u., v.] such that [w., v.] S [ur-1, -V-1), 1]. then we have luy - u.l 2
where [0i, wi] is the interval coght-con to [uı, vı] in p. If = 2 and m = 2, instead
X
02
S |0.7
Σ
Σ
Xun.nn & X +
Σ Instlee
Xun @ Xwn.n
2
Σ
Σ [M2
Xuym Xuging
M
where [u1, v1]. (V1, w1], (w,, 21] are consecutive intervals in p.
32

---

Learning with Expected Signatures: Theory and Applications
m/2(11. - uul + loy upl) and
(Cov(X(ur) (wm) pupi
x/O) (u.....) 2 0(wur- U. 1/v.. u.ller - w.l S 0(wux - 11-10 - us -ullerr upl
by applying (A0).
Similarly, if [uv. vr] appears in a pair with an interval [ua, v.] [a) such that [ur+1, Uy+11 I [T:: valy then we have lu. - url 2 m/2(lur - uvl + l0+- u.l) and
|Cov(X (w)
x [u..o.)
) x 0(uu- ov Dey lll ual 2 0(lu+1 wrller uw llv ual-
If [uv,, my] appears in a pair with an interval (sa, tx] E Pr. then
[Co(( (oree)) pur (sa.tt))] I S 0((n - w -urlit sxl,
(w)
X (1)
by applying (A0).
A similar reasoning applies to each (sp, tr]. for l' = 1, I..... ... Noting that at least one pairing is mixed across p and Pa. we can hence bound
1Cov
(xx) )p.rry S¹ (xx" )o...)))
i
Σ
Σ
Σ
$1-22=1
f1
$1n... 21
1-11/=1
In B.JiEs, f fm fu mf 5m "31-" "1 lEp
1/1 =k/
2 =k
I 12 52.22. [m,.i1 m² Sm
3/2 1og lEpn
0((n 2)T) 11
10l...0((u) 0(1;; - 2ズ- 110b nz11 un 10[122 1 122 u21 ...
PEMP2
(2,0)
x 141 - -1 - 41)1622 22 ..001 4/ -1l43 55g1
Σ
((n 2)T)
jid2=1 iimoty 21 einmegy? 21
4 =k' o1 t.2
Σ |11 (um-)))) mil)
Σ
([22])/022 wal)
(
Σ
000s - -0ss)
Eup.
[ngt 10gle
x
Σ
It1 1--s) s1l)
Σ 0((021)t2 - 02)
Σ
0113) - 1558
[22.t2|Epn
lesg 1seen
h
50
0it-6-2
+((n (((1-2)2)
Σ
Σ
Σ
ΣΣ
Σ
1
, lel 0.
f1-2=1
21
92 21
111
4ot WM ex mintt =K²
Combining this bound with (39), we can conclude that, for all n 2 3,
Coov (s" (x) )o.r) St (x")))))r I. 0((n - 2)T) + 0, n - 00,
ie. we have shown (38). Hence, we can apply the weak law of large numbers of dependent random variables and the in-fill asymptotics to obtain the desired consistency result.
C. Variance Reduction via Martingale Correction
In this Section 2.2 we considered single control obtained by substituting the outermost Stratonovich integral with an
Itô integral. In principle, for a word of length | = k 2 2, one could consider 2 distinct controls: for any subset IC f2.... k - 1). one can obtain a control by changing each of the integrals with index in I U {k) to Itô integrals. One can then apply the controlled linear regression estimator (with only the intercept term as regressor) described in Appendix G. This family of controls will likely be highly correlated and hence:
33

---

Learning with Expected Signatures: Theory and Applications
the improvements provided by each additional control might be quite marginal compared to the considerable increase in computational cost;
the estimator of the (inverse) variance matrix of the controls, needed to make the estimator feasible, might quite unstable.
Hence, for clarity of exposition and computational ease, throughout the rest of this work we only consider the control variate estimator introduced in Section 2.2, i.e. for a fixed word
Ny Sh (T)
1 N
Σg
(x(x.)))
0.21 (x
(40)
where T-1 (61,-... 1k- and
(xx)))00)
Σ
s 5 )000
=
[u,0|E=
C.1. Martingale Continuity Criterion
If X is martingale, then, by the Burkholder- Saaider--vvi--Guueee (BDG) inequality (Burkholder al., 1972), we can write stronger version of assumptions (Aa) in terms of the quadratic variation of X, for all 0 S SIST
(Aa.M) (X) $ 20
Note that for many martingales assumption (Aa.M) holds with a = 1/2 and hence, since (Ao) holds trivially, we can usually apply Theorem 2.8 under (ii). Some non-trivial degenerate cases exist: For example, consider a one-dimensional mean-zero Gaussian martingale over [0, 1] with covariance function C(s A t) where C' is the Cantor function. This process has quadratic variation (X)r C(t) and hence since the Cantor function Hölder continuous with Hölder exponent loga(2) assumption (Aa.M) is satisfied with á loga 2 (1/4, 1/3]. In this case, (Ay) hold by combining the independent increments and martingale property of X.
easily check that (A33) and
C.2. Estimating cx
In Section 2.2, we considered the following estimator for cx:
EN 5 (X".7)
1. St (x"**) [o.
Σ St. (X..5 ) (O.T)
Alternatvely, exploit the explicit form of the covariance and variance of the infeasible estimator and approximate
Cov(S'(X*)) (o,77 S: (X* (x)))))))) x Cov(s' (X) p.77. St (a))
Σ (T( 2
1
Σ
PT(T)
Jelw!
Jelwi 2*((t 1.6))
1
Σ (T)
Σ
(T)
Jetw!
JelwI--2 ((h--
N. (T)
1
Σ
3 (T)
2
JeTw!
Σ JETul
/N (T)
Σ
(T)
JE (elui---((ie-sstt))
where
N Σ
(T) Σ 5 (x, (x))))))) (
(T)
S ((X, (X))
= N
34

---

Learning with Expected Signatures: Theory and Applications
and
N (())))))))))))
(N
(T)
Σ
S ((X
u.2l
with (x)"= ((x)7, e [0, defined as
[
E
X X
-- u x²
u,v
t [],
u
Similarly, we can approximate
Var( (x* (x))))))))) se (
Σ
( 2
Σ
(all(ceda))) (T),
Naa
JEI-1UI111
JEI-1H1-
and hence we define the second estimator for cg
2 Eserui (T) (( VJ (T)
Eue (z(lin((e)) (T)
N...
((Xui), En ('EELLS'((X ((Xm)) (2)) Dom - (( ('X (2))), (8)m))))..)))))
Σ EJeL,w,, (1=((in))))) (2)m.)))o.7
Whether estimator 1, or estimator G. is more precise, in terms of MSE, depends on the properties of the process X and the expected signature word being estimated by (40).
Lemma C.I. Let X = (X,, E T}} be square-integrable martingale satisfying Assumption (Ao.M), for some a 2 1/2 and p = 4k where = I Assume that a is part of sequence of refining partitions with mesh vanishing fast enough, i.e.
2a 1/2 < <0.
n21
Then the difference between the mean square errors of the two estimators &r,,y and é for of is approximately
B[(C=,2 -()))
E(G
JlY
PLY
B(Z 2(E E(YZ1))
N 1
1zz
where
Y = ('
Z (x) [.T])
Zz = Σ
((x
JEI-
and py [Y), 4z = = E[22]
Proof. See Appendix C.3.
In practical applications, the above expression cannot be evaluated exactly, but we can approximate it by its sample estimate
2
1
El(ct,2 -0]]_((67,,)
2
Σ
((Zg,) 2
(TT,))
( Z
Y
ZT,.)
N2
Øzz
ja
1=1
where
N
By
1
Σ
v 1,
+
Y
P2
Σ
Z
Z
2N
2N
35

---

Learning with Expected Signatures: Theory and Applications
and 1, Lin 2,n Z given in Appendix C.3.
Another important discriminant when choosing between estimators 6 and G,22 is usually computational cost. To compute G,, suffices to regress {S' (x"*")) (o.T)) n ............ N against {S] (o.T).
........, N). Both samples need to be computed to evaluate the control-variate estimator (40) and thus the extra computational cost of " is just the cost of simple linear regression with sample size N, namely ((N). On the other hand, to compute 7, one needs compute all the higher-order expected signature estimates // ())."((T)) with J E TTI,, E Tw I-2 ((ik-,, ix) and J = J ((ig, ix)) for J' E [- E I-1, which has (naive) extra computational cost O((=)- + (d++ 1))*-1)) when parallelizing across the N samples. In the in-fill limit, | IT > N and, hence, computing 6 is significantly more expensive than computing G.
C.3. Proof of Lemma C.I
Sketch of proof. To compare the two estimators , and č,22 we exploit their structure as rutio estimators based on i..d observations of numerator random variables Yi,, Y2' and denominator random variables Zi, T22 respectively, We first show that the two estimators are both (biased) estimators for c = = = -.,2 where
E G B
[E
and Cr22 E 25
The first equality trivial while the second requires several applications of Thearem 2.8 which detailed in Lemma C.3. We then derive simple formula for the mean squared error ratio estimators in terms the means and variances the mumerator and denominator random variables. Taking the limit [ O and applying Theorem 2.8 to show the second order statistics of Yf, Y Z7, Zz converge those of Y1, Y2, Z1, Za yields the desired result.
Note that both ε., and 6, are ratio estimators of the form
Yx
Vx
C =
===1
7
m=
for random variables y2 ....... J.N and Z
NjN with j = 1,2 given by i..dd copies of
Y 10,17
z = (x(x') Po.
and
Y = Σ ( (2)
2 Σ
((X*x((X))))
JELLT
2
Σ
(Jal((erda))( ((X", ).
JEI
By the standard theory of ratio estimators, these are biased estimators for
B C
B
36

---

Learning with Expected Signatures: Theory and Applications
and applying first-order Taylor expansion" 19 we can approximate the mean squared error
E 1
E| |=
BI(
-G,)
~
Var
+
Var (2,)
Cov(Y,, 2n)
E ||
1 Var (2)) 2 ][
Cov(Y .27)
~ N
1|
Var
B|Z
Note that
E[S (X*)
1,27
S (X*)
10.71
Cov( (x*) 10.T) (xx) )U.TT) cx
10.
( (0.7))
and, as ||| + O.
E EsEEUr (X",, (x)* (x))))(o)) (O.T)) 2/2 2 EJemm E[S (x)))) pom1 Eser_ B[
G22
(46) Eserur E[s ((X, (x)) (0.r))]]] 1/ Ietuu )(r)))
Els ((X, (X)))) p.1
EsEl 1w1 ((SU~(Gnnta)) (x, (o.rl (43) EyEEUl Els ((X, (X)) - 2 E ( Evenir (( (xx (x))) 10.11]
E[S (XX, (x)) p,7)
Cov(s' (2) p.rj. St (X))..))
Var(S! (X)) ( (o)))
(42) Cov(S' (X") /o.ry. (x (o.1) c
Var (ar (o.n)
We refer to Lemma C.3 for the rigorous justification of the approximations (41), (42), (43), (46). Moreover, as |ズ| + 0, by (41) and (42),
YT 4 (x =: Yi, z7 3 5 (x) 0.7 =: 21,
and by combining (43) and (46),
13 Y
Σ
s ((X, (()))
1
Σ
S'((X, (X)))_T) =: YY.
2 JEIuI-2-((e-11
Jetnl
2 12
Σ
(J.((benb)))( ((X, (x))) =: 7z.
JEL
Note that Y1 = Y/222 =: Y but Zi A Z2 even though B[Z]] E[Zs] uzz When |-| is small we can thus approximate the
MSEs of ē and 2 with respect to of as
1
1
23 Var Var(Zi)= - 2 Cov(Y.Z2))
E E( 1
+ 2
oy
N
2z
2
19On the set 123 - -E|2]] < |B[25]|.
1 y,- E|
2J -
En, = C
14
B(TF)
B[Z5]
1+ YI - EYTT)
27 -R(3]
+0 ( (25-B1Zz) R[25]
Cr.
B[Y3T]
B|23]]
B1777
E Y B
E[2,1) +0 ( Y2 Ely,J|
-E(Y
[ (2-EZ211) ))
(2J
B
1
B[Z7]
For the approximation to be rigorous one needs assume the probability of the set (w : (w)(w) (w) faster than the speed at which the MSE conditional on this set explodes as N - oc.
E|Zz11
approaches 0
37

---

Learning with Expected Signatures: Theory and Applications
and
1 N
1
oy
Var (Z2) -
Cov(Y, (2)(Y.Z)))
11
fz
p1
which differ by
BI - c ~
jy
PY
(B|Z)] - B(Z 2 22] B|Y 2111
N
pz
Pz
Remark C.2. Note that we can "mix" the two estimators and form
Yr 1,2 2
and
The discussion above ensures that, as |/|| + O, 2, and 1 have the same MSES as & = Ōr and C = 2x respectively.
Lemma C.3. Assume that satisfies (Aa.M) for some a 2 1/2 and p = 4k where I1). Assume that πT is part of sequence of refining partitions with mesh vanishing fast enough, i.e.
Σ R21
nn 2a 1/2 < 00.
Then the approximations (41), (42), (43), (46) hold as |x|
+.0.
Proof. By Appendix C.1 X satisfies (Aa) with 2 1/2 and 4k. Note that since X is a martingale (Ao) holds trivially and hence we can set e = 1/2.
We can apply Theorem 2.8 to deduce that
(x") 4 5 (x)
| + 0.
Moreover,
(( 10.7 4 S. (.
| + 0.
since
||S (x) 10.21 (xx,))1,))
5 (K) Σ
(t2) S1- 1x. us
+
Σ
(S-- (S"=1(X)oon) -5 j,,) (
ju
[u.u)TT
(00
(- S--11 (X) (o.j
Σ
( 1., 0., X(ie)
[u,
1/2
+ +C2
||st (X) -t (x" 10,00 L
|| 11
|u,o|Ex
(10)
{
T
5l- (1))
Σ [u,jeT
s (x),0x).. X((i
1/2
+ 2C2 sup uET
88-1 ('11 - S1 (xx ) 10.0
Σ
1o u/20
J0,0jET
(60)
0,
| 0,
38

---

Learning with Expected Signatures: Theory and Applications
by applying in (i) the triangle inequality, in (ii) Lemma B.1 with the natural filtration of X, in (iii) Theorem 2.8 the word 1-1 with = 2k/ 1) and in (iv) the definition of the Itô integral and Remark B.4.
Under these assumptions, we thus have
Sx
(o.T 4 s' (X)
|<| + O,
(41)
and
(x*) Po.7
[..7y
- S (x) [.77
| + 0.
(42)
Next, we consider the approximation (43). Note that X satisfies (Aa) (and trivially (Ad)) with 2 1/2 and p 4k
while (X) satisfies (Aa) and (Ao) both with exponent 2x 2 1 and p = 2k. By using slightly more general version 20 of
Theorem 2.8 applied to the process (X, (2)). for any word J E w or J E E I * ((ix-1, ix)) or J = J' ((is, ix))
with J' E T-1 I-1 we have
(x))
4 5 (x)))
| + 0,
(43)
and hence for fixed N 2 1.
(T) 4 3y (T), |T|| 0.
Finally, making approximation (46) rigorous is a bit more challenging. Let us start by considering the case where (X) only appears in the outermost integral, i.e. J = (J....,, Jkr). where Je.-..,Je-1 E {I....,d] d} and ju e ((1, ()...; (,,)).... (d, ] for some k {1, (l...., 2k - 1}. Then, for any T E n,
( ((X, (x.))) 5² ((X (2) )0.=1 (1//(r11)
Σ
s² )0, (1--++)) x ((x)15) - (8))
[u.r)
14//(1 (1)
(10
Σ
5 Jonm No. (41) (iie--)) (x)(lr)) - - (x1(ee))
i!
1=1
L4N///4 +33
1/2
(1))
( 10ml
(x))lte) -
(((g)))
u.e
[a.
sd )0-.(x")o.w) xifor-u+ x
(x) - (xx(irr)
i!
(a.
1/2
(x/( - -0--xxxx (x((ee1)2 (xx LAR/CC-1)
للي (x* 1xx1( (x*)
[..))
(e (x0))
X lig -140)) x(g
(x) (ig)) - (x!(fe))
i!
1=2
[u.
1/2
(te)
sup [EF
|0.0]
X)uux - x02 gok
k
sup (0,z]
| (xx) )0.))
Σ
[Xu.ell (X) x
i!
1
uE
2A
VF +
|
0,
|
To estimate the expected signature term corresponding to the word 7 = (f1,..., (x), it is sufficient to require (i), (i)) (iii), or (tv)
he satisfied by x(i)) ,-.., (ik) with Pl. Pi.....PP Ps such that p. + + Pt m1. In Theorem 2.8 considered the case where
Pi = = P = p for clarity.
39

---

Learning with Expected Signatures: Theory and Applications
by using in (i) Lemma B.3, in (ii) the triangle inequality and the definition of (x)= in (ti) Lemma B. with the natural
filtration of X (with respect to which X (X) is a martingale) for the 1 term and the triangle inequality for the
i i= 2...... J terms, in (iv) Hölder inequality, in (v) Remark B. applied to X and J-; with 4k, m 4k/(k' - i) and
J-s = k' - i for i = I....,'' k'' and
(
X
u. paN S (1(x) + 1I S 10 u/ un?n,
(44)
by assumption (Aa.M) with 2 1/2 and p = 4k. We have thus shown
5 ((X, (x.)") 5 ((X", (x) M).)) marwon) 2 $1 sup TEn
(45)
This extends to any J (ji..... Jkk _...... Ju +m)) with m' 2 0 and I K''k's + +m s 2k 1, where, as above, ju {(1, 1),...... (1,d)..... (d, ) and ji E {I....,d} df for all other i & k. We proceed inductively on m' = o...., 2k k' to show Equation (45) holds for any J of this form in the L L/a/(kk +m +1) norm. The case m' = 0 has been covered above. Assume (45) holds for Jw with m {O....,, m'} and S m' 2k: K' 1. then
|| swwwii m² 12 ((2, (x)) (0.-1 SWM 10iimn 1 (x))))) +0/44++4+2))
(X))) (o..) S /wwtt//t----// ((x", (). Nu Sww+m----- (X),
xlgg+m(42-)) x
Xu,0
W 4mm +1
Nu, +m ((x)(ee) (er) (X8) (8)))(5))) ... X. (0//
(88)
Sdartm (2)))) sh ((X (x)) Manu xel x
Ju,0/ [l
m/4
Σ
S'040't1- (()())))) SJ 5/04m'41-1((" ( p.u]
i!
x X 40244)))2 x (ie+mm43))
Σ
s'x ),x)
i!
i=m +2
[w,0)ETT.r]]
Xu,e +2 ... (x)) - (x())"))
LAN/CW + 420
1/2
(aa)
| mn (x))) )po.)) S ((x", (2)) )))))) pou) + (u.))TT...)
Σ
sup
w wET(p.))
((x Jo.. 5/w4mm411-1( (2)) )o..
sup [
R 4 +1
dl
sup nE
shiiii )o.s) +
[ 11 [(())..
- X 92 11
[u.
| / |m|- |, 0,
where in (i) we have used Lemma B.3, in (di) triangle inequality, in (ii) Lemma B. with the natural filtration of X for the term, traingle inequality for the i 2.................. W + ml + terms and Hölder inequality across all terms, in (iv) the
40

---

Learning with Expected Signatures: Theory and Applications
inductive hypothesis for Jk with 0,... m' for the first two sups, Remark B.4 applied X and Jj with p 4k, 4k/i and Ji| for i .....::.. k' 1 for the third sup and bounds (44) and (Aa) with p 4k and a: 2 1/2 for the
summations over -[
We have thus shown that for any word J E I w I or J E I ((ix-1, ix)) or J = J' ((ik. ix)) for J E I w I-1 we have
('X, (x)) [0.0T] ('X", (.77 3 | -
(46)
and hence for fixed 2 1,
35 له x (T) 4 -
D. Itō processes and diffusions
In this section consider Itô processes and Itô diffusions: two common classes of models for continuous-time stochastic processes. We start by providing sufficient conditions ensuring Assumption (2.6), needed for the in-fill asymptotics, holds. We then focus on time-homogcneous Itō diffusions and discuss general conditions under which these processes are stationary and strongly mixing, ensuring stationarity and strong mixing of {X", 2 1} under (chop) observations (cf. Proposition 2.1
D.1. In-fill conditions
D.1.1. ITÔ PROCESSES
We consider the case where X is an Itô process, i.e. satisfies
x, = Xo + / b, ds + / V.dW,, te (0,
where b (b,, E [0, and V = (Vi t [0, are progressively measurable d- and x g-dimensional processes such that
sup <E(O.T]
||bاl
sup sE(O,T]
||VilL. <0,
(47)
W- (, 8 2 0} is g-dimensional Brownian motion and Xo E LP, The assumptions b and V imply that for all
0 S StST,
| buddy
$ |." lbulll. du
(48)
LP
sup [E(07.]
[|bull |t- s/,
(49)
by Minkowski's integral inequality, and
|1 V.dw.
( tr
1/2 = -1% |[all |all
1/2
$
V. du
LP/2
1/2
V. 11z, du
(50)
sup wE
||Vullz>
1t 8- 5//22
(51)
by Burkholder-Davie Davis-Gundy (BDG) inequality (Burkholder et al., 1972), the formula for the quadratic variation of the Itô
integral, and Minkowski integral inequality. We can show (Aa) holds with 1/2 by noting that for all 0 < L
& $ -/ bud
+ -11 V.dW, 4
Sit - s + =/1//2 S s|
LF

---

Learning with Expected Signatures: Theory and Applications
by combining bounds (49) and (51).
Next, we can show (A6) holds with d by noting that for all 0 S a ssstsT,
Eg X
bx
Es
b du
bu
21 - s/.
where we use the martingale property of Itö integrals, contractive property of conditional expectation and the bound (49).
1.1.2. ITÔ DIFFUSIONS
Next, assume X is (possibly time- time-inhomogenccus) Itô diffusion, i.. satisfies
dX; = f(t, + ot, d,,,,, E (0,7),
where W (W,, e T) a q-dimensional Brownian motion, : T] IRd - R R [0, x R2d - Rgdx and Xo LP. Itô diffusions form a subclass of Itō processes, which we already covered in Appendix D.I.I. Here, we give conditions specific Itô diffusions i.e. in terms of and which imply condition (47).
If f and a are uniformly bounded on [0, 7 R4, then condition (47) immediately holds.
Assume f and a are Lipschitz continuous, i.c for all s, E [0, T] and x,y E R.
|S( x) (8.. ((t, (s. < Ky([t s| + ||x- - yl),, -o(t.)) vs) (tt,xx (s,y)) K. (|t s| yl))
Then, for all 0 585157 T, we can bound
- -I1% f(u, (,),K.) + [ alu, X(W,..)d....
LP
2A [( [1](u, X.),,, du (f( (lo(u, [oo(u.X.)1l33 X.)ili du
[lf(s, (t s)= + f: [1f(u, - f(s, (.s,xx)llzzz du
+ 10((s, X,)l (t - 1/122 (f [lo(u, 06, -oos (s,,x,)111 du
1/2
(02)
[I.(0, 0)1 + Ky(ss X (t - s) + K₁ (t-s)² s)2
+ S K₁ 1| du
2
+ ||0(0, 0)1 K X -//1/2 + Ko (f: [(u, - (s, X.) du
1/2
8) K3 (t - s)²
+ S. Ky ||Xx,ulle du
[1 0)1 Ky(s |x,|
+ [000000]|+ (s |X, [ (t s) (t-)) -oo (f, ju-s/Rdu
1/2
+ Ko (J" Ne 112 du
1/2
233 (1V (t - s/)1/2 S'¹ (X du du+
(
1/2
[ 11. du
(tt - -/) (f. [Xx du
1/2
(1V X ((
42

---

Learning with Expected Signatures: Theory and Applications
by using in (i) triangle inequality and Equations (48) and (50), in (ii) triangle inequality, in (iis) Lipschitzianity of and in (iv) triangle inequality, in (v) for all 0 Σ S t T, I T & 1 and (t- s)1/2te S x (t-s) for 2 0, and in (vi) Jensen's 's inequality. Setting 0, since assume Xo LP, this is
(L (1 1 du
1/2
X S 1/2
and can apply Willett (1964, Lemma 2.2), nonlinear generalization of the Gronwall inequality, deduce for all 0 stt T,
(1o exp(--Cssssd 1/2
|Xo,oller S 11122 +
V expl -C1]
T
12
- -CT] < 00.
We can hence show the condition for Itô processes (47) holds by noting that X. E L and Lipschitzianity of f and a imply
[t, X,) || b < [l,(0, + ( + |Xol| + (o,ellze)) < 0o,
[t, X,) ) IP - |0((0, 0) + (t) + [Xo L + (o,+ < 00,
uniformly in t E [0, 77, and hence (Aa) and (AA) hold with a 1/2 and = 1, respectively.
Assume f and a are time-homogeneous such that f is Lipschitz continuous and a is 1/2- 1/2-Hölder continuous, i.c. for all x. y.y.e Rd.
ll - f(y)l E KyI[x - -yl, |o(x)) - -yy)l - Ko |x y11
Then, for all 1 T, can bound
S ((X,)du+ ['o(x.)JwW)
|X,,
12
[( (1((X))lerr du (L. lo(((xx))]] du
1/2
|1((X))l (1 8)+ f( (1s(x.) -(xx))l du
+ +loo (t- s/22 (ʃ" (1o((x)) du [I((0]] K2 [1 8) + S Ky ||x du
1/2
1/2 + +|10(0>]| + K. |x.11/22 (t- ()- K!/² ( ((IXxaallerdd)
(w0) (1VIXX,lI> v || 1/222 )( (1- 5) J' o du (
[[ (Xxeullledd))
1/2
by proceeding in the previous case. Setting O, since we assume Xo E IP, this is
| Ie $ 5112 '"' I du + (S [IXo,ull dul du),
1/2
43

---

Learning with Expected Signatures: Theory and Applications
and we can apply Dragomir (2003, Theorem 41), another nonlinear generalization of the Gronwall inequality, deduce for all 0 - TST.
Io f(t) oc.
We can hence show the condition for Itō processes (47) holds by noting that Xo E L and the conditions on f and imply
lx & [[((0)] + (o(lolll + [Xo,rlle)) < oc, [xx < ||(0)]| + K. (I |Xo,,l 1/2 00.
uniformly in t E [0, and hence (Aa) and (Ao) hold with 1/2 and respectively.
D.2. Long span conditions
D.2.1. ITô DIFFUSIONS
When developing conditions ensuring stationarity and ergodicity of an Itō diffusion it is natural to restrict (X,, t t 03 to the case where it is time-homogeneous, i.e. satisfies
dX. d( + o (X,)dW t 2 0,
where W- = (, 2 0} is g-dimensional Brownian motion, f Ed - Rd and a Id - Rdxg Assume:
The diffusion coefficient TR Rdxq is Lipschitz continuous and Σ ooT RRd Rdxd is bounded and uniformly elliptic, i.e.
(5, (())())) O.
inf nERM,EER" (0)
|
This a classic PDE condition which ensures the transition densities are "nice", i.e. continuous and bounded away from zero (Friedman, 1964).
The drift f Rd + Rd is Lipschitz continuous and has negative radial part at oc, i.e.
lim sup
f(x),
-Cp E 1-,0,,0,
| |0|
||
pushing the process towards the origin with strength controlled by [-1, oo). When -1 assume further that 20-1 suPzeR= Tr (x)) and define
80,
if K > 0,
2Co [|]]
if A = 0,
n/ =
2Cx/((1 =III,
if K (-1.0), O),
(2C-1- - suupper Tr E(x)) /||||||
k = 1,
where ||]||]]] == supzerd [[()]].
These conditions are enough to ensure there exists a unique invariant probability measure y on IR with
Sas الll (((x)) co, JEU edlli|| u(dx) < oo, SEx ((dx) < oo,
if h 2 if k E (-1,0), k =
Note that, for large [|æ]], one has ( (f(x), fin) and decays as ||a||| increases when A
C. || " and henee the strength of the pull grows as ||æ||| increases when > 0
44

---

Learning with Expected Signatures: Theory and Applications
for all y (0, nj.)), such that for any o E Rdd the transition probabilities (Pi(s,),, 1 1 0} converge to j in total variation distance with rates
(e + ca), ( (e + +
if 0, if E (-1,0), if = -1,
[P,((r,)) uIlTV <
(1(1 + c2t) // (| cs):
with C1, C2, Cs > 0 (Kulik, 2018, Theorem 3.3.4, and 3.3.6).
Assuming Xo ~ 14, the Itô diffusion {X,, / 2 0) defines a stationary Markov process with Xo E LP for all p 2 2 when > -1 and for all 2 2 p < 7j.x when ^ = -1. Recall that, by the discussion in Appendix D.1.2, when f and are Lipschitz continuous, is enough to have X E LL" to ensure the process X (Xt, E [0, T]} satisfies Assumptions (Aa) and (A6) with a 1/2 and it remains to establish strong mixing of {x", 1] to apply Theorem 2.10. The strong mixing coefficient of the stationary Markov process (X., i 2 0} can be easily bounded by
1. implying
1/2. By Proposition 2.13 the chain {x", n 2 1} is stationary and, hence,
if 2 0, if E (-1,0), (1 (1+00) (1t)=))-n/2 nn/ =-=-1,
ext
[ a(t) $ Jd [|P((z,)) pllr p(dx)
c -
x
and hence the process is strongly mixing. By Proposition 2.13 the chain {x", 2 1} is also strongly mixing with coefficient an(n) I a((n 3)T), n 2 3. follows immediately that (X", n 2 1} is ergodic and hence, we can apply Theorem 2.10.1 to deduce that, letting [ + 0 as N + oo the expected signature estimator (7) consistent for any expected signature term when & > 1 and for all expected signature terms with [7] < 27, when 11
Finally, we note that for any <> >>
e < 00, En e ( c0, Enzi( + - 00;
if 2 0, if x E (-1,0), if K 11
Σ (m)))//22))) 820
S
and hence, if K > - and [IN is sequence of expanding dyadic refinements, we can apply Theorem 2.10.2 to show that the expected signature estimator (7) is also asymptotically normal.
For the time- tomoohoomeeesuus Itō diffusion (X., 2 0} these are defined by
(r A) P(X, E A|X. 2) P(X.++ E A|X, = =x),
for all s 2 0, E R2, A B(R*).
2'rr (Xa, t 2 0}) is a stationary Markov process with stationary distribution and E F' B E F t, 2 0.
IP(An B) P(A)P(B)
| 1l B[1A][(1
[ 11 - B[1
LA
B[ X Xx++I]]]]
LA (
[ ha ha())a(dr) )
E
Rd
A { () dz) m(dr))
E
2d
< [1] Pa (X,,,) -) - 11|]v]
= f (P, :) 1V|TV p(dz),
where hp Rd Rd [0, 1] is the measurable function defining conditional expectation F(1 X++s] = h(x++s).
45

---

Learning with Expected Signatures: Theory and Applications
E. Gaussian Processes
We first exploit the properties of Gaussian random variables to show that, if a Gaussian process satisfies (Aa) for p = and some a > 1/4, then it satisfies (Aa) for any p 2 2 and the same a. Next, we show that two common examples of Gaussian processes, FrsteinnUhlenneck processes and fractional Brownian motion, satisfy the assumptions of Theorem 2.14, i.e. (Aa) with p 2 and 1/2, (Ad) when 1/2 and (A0) for some decreasing 0 RR+ R+ with 0(t) 0, 00 and So' ((t) < 0o and m E N.
E.1. Gaussian Processes Continuity Criterion
Let X be a Gaussian process with mean function p. [0 + -R. If X satisfies (Aa) with p - 2 and or > 1/4 and is a a--ölder continuous, then it satisfies (Aa) for any p 2 2 and exponent a. Note that, by the inclusion of norms in L spaces, suffices to show this holds for arbitrarily large p. Choosing p 2q even, we can write
| L29 2 x,,t Marell 2 ||Ma,e||
97
1/2g
(9) M8.t
2
)
24
E
x
+ |t - 8/00
1/24
Σ
q
E
x!
(1)
24
x (d)
24a
91.... qdl
01 01 toy 0120
1/24
Σ
)
Σ
II
Cov (x(1
s.t. (x
+ +|t 8)
91...., qd
pe Ph Phe.....2 {istEp
91.00.9200
1/29
2A3
Σ
Σ PEPa
II {p
t - 5/20
S s|--
91 t.t- I...... 0220
9d
PE 201 2mg
24 the set of all the pairings of S [1]2g U {2]2n U...U {d] 2na i.e. all distinct ways of partitioning S into +... + qa = pairs, and in (ii) the fact that Assumption (Aa) with p = 2 and a > 1/4 implies for all j = l,...,d.
where in (i) we apply Isserlis' theorem (Isserlis, 1918) denoting by P2 P
Cov(x 8,c
5. - [Cov (0)
att x!!i!
S |I[ 2,1 x( 1
$ x
1 G)
|x,1| + ||, < s|20
E.2. Gaussian Processes Covariance Decay Condition
E.2.1. ORNSTEIN-UHLENBEER PROCESS
If [X,, t 2 0} is stationary mean-zero d-dimensional Ornstein- Uhlenbbek process, i.e. mean-zero Gaussian process with covariance
C(s,t) == Cov(X,, X,) - A
where Σ= = Var(X,) and the drift matrix parameter A E d has positive real parts of all eigenvalues, then we can show that:
(Aa) holds with = 1/2 and p = 2. Note that for all 0 OSsSt<?
Xx,,, s,t l2z- = Eltr(X,,t 8 x.,))
46

---

Learning with Expected Signatures: Theory and Applications
(rr(Va(XX)) + Var(X,) 2Cov 2Cov(X,, x,)) = 2tr((Ta - e -
I 2 [T - |ل|||
I (|||||t - s|el|all(ed)) |Σ|
|t - s/-
(Ad) holds for any p 2 2 with 1. Using the integral representation of the OU process, one can easily verify that for
a 0 f s - T. E, [X.] : [F] = Alt-s| X,, where {Fe, t E [o, the natural filtration of X. Then, for all 0 st T,
E X = ||(e= Ale-A 1a)x 1 11 - e Alt-1 |X 1e S lt - s/.
The covariance of the increments is homogeneous since, for all u, U, s, 0,
Cov (X x.,) Cov (Xp: X,) - Cov (X, x Cov (X X,) + Cov (Xa, X,) Al-Allt e- -Ae-ool - - + e Alo--a))
depends only on the relative distances f v1, Is u/,1 |t - ul and |s u|.
The covariance of the increments satisfies Assumption (A!) with m = and ((1) = -dat where a is a constant
depending on the drift matrix A E Rd For all 0 I a S v < $ t,
||Cov (Xu,v, Xx,z)
|le Ale-Als-l (e
(e-Ait-sl Alt-s|
La Alt Alv Alp-ul + +e A1v A111o=1
||e Ale-Alseole A|t- -1))111)((1 (Ia - E A
||e_ Als-cl || e A s| La AAll-ul |Σ||
$ -- - s||e - u/,
where, in the last step, the fact that A aadd has positive real parts of all eigenvalues to find da (0, (A) Re(A)) such that ||e-A(]| $ e- for all 2 O.
Note that (Aa) and (Ad) could have been alternatively established by noting that the OU process is an Itô diffusion with Lipschitz continuous coefficients and applying the results of Appendix D.1.2.
E.2.2. FRACTIONAL. BROWNIAN MOTION
If (XH, 1 2 0} is (one-dimensional) fractional Brownian motion with Hurst parameter H 1/2, i.e. mean-zero Gaussian process with covariance
H (= Co ) 2 (1 + 1s|2 - It- s| )
1
then can show that:
(Aa) holds with a = H > 1/2 (and p = 2) since |X |t "|
The covariance of the increments is homogeneous since for all U, 8, 2 0,
Cov (x noo x Cov (x## x Cov (x x! Cov (X x + Cov (XH, x!!)
2 2 + It u1211 - |t - v 2 Is u 2
depends only on the relative distances |t. 21, Is - w/. It ul and |s u|.
47

---

Learning with Expected Signatures: Theory and Applications
The covariance of the increments satisfies Assumption (AO) with 3 and O(1) =12#-2, For all 0 S with |s r| 2 3/2([t
+ |u
ul).
CCov (X u,
k.
2 ss w/ 2H
w12H
+ +1t 2
22 -It 1|t-00 10-00= 2H - |s 2H
8 v 22
2
(2H) (2H- + 1)
|s - vl 2H (1t - u] - -1ss - -1s-21)"
n!
(2H) ... (2H n + 1)
21 2H 2n2 (|t
la |s =1)
n!
==0
(2H) ... (2H- - n + 1)
18 r 2 n (1 m| -
n!
n=0
1
00
(2H)
(2H
n + 1)
Σ
|s 2 2H (1t
s + 1/0 ul)"
|t
m
2
n!
n=2
1
20 (2H) (2H n + 1)
2H 2
(n-2) Σ
() 21 //e w/--d
8
2
n!
n=2
j=1
2H 2 2/6
$| |U - ml
2
(2H) ... (2H n + 1)1
2
2)
vl- (n-2)
|1 8/3 m
m 2)-
X
(n 2)
j
n=
1
2H 2
s|| s||u
u| |2H 12H 1
Σ
|s 0
n ([t 8| + 10 ul)a
2
==0
1
It -
s + 10 21
< - 8 0 2H 2 - 2||0 u| |2 |2
1
2
vl
|2H 2H 11|s 2 2 t s | 2
by using the fact that that x + f(x) y 2h is analytic for any x >> andd| |s - o| 2 3/2((6 -| s|+11 +|u ud).
F. Machine Learning Algorithms with Expected Signatures
Signatures of paths have found widespread use in the machine learning community, with applications ranging from character recognition (Graham, 2013; Xie et al., 2018) to medical diagnosis (Pérez Arribas et al., 2018). Taking the signature of a stream of data is essentially a feature extraction method mapping raw stream-like data to a lower-dimensional but highly-informativy latent space. The theoretical foundations for their efficacy range from the characterization result of Hambly & Lyons (2005) to the universal approximation theorem (Levin et al., 2016, Theorem 3.1). As discussed in the introduction, when dealing with collections of paths, the characterization results of Fawcett (2003), Chevyrev & Lyons (2016) and Chevyrev & Oberhauser (2018) 8) give strong theoretical justification for the use of the expected signature as a feature extraction method.
While dealing with a collection of paths is arguably a less common setting than a single stream of data, the literature still provides a wide range of machine learning algorithms leveraging expected signatures These cover many different tasks (from distributional regression to generative modeling) and applications (from ECG classification to option pricing). In this section, we review five of these machine learning algorithms discussing how the martingale correction introduced in Section 2.2 can be applied to the expected signature computation step to improve performance. Before diving into each
The GPES algorithms, discussed in Section 3.2.1, actually takes input single stream of data and applies data augmentation technique to form collection of paths.
48

---

Learning with Expected Signatures: Theory and Applications
specific application in detail, we discuss some general considerations on the use of the martingale correction term in practice.
F.1. Martingale Correction in Applications
In Section 2.2, we considered the same framework as in the rest of this work, namely the setting where X is continuous-time stochastic process and X is piecewise- linear interpolation of the discrete-time observation of such process along the partition л. It is important to note that, while some applications have natural underlying latent continuous tomtitimuuueeeueeeeo model (Lyons al., 2021) others do not (Lemercier et al., 2021). In any case, we do not necessarily require the "background" continuous- time model X to be defined to apply the control-variate estimator (40) with = ỘT Letting = {0 to < 4 < ... < aM T), one can easily see that it is sufficient for the discrete-time process {X.. = X m = 0,..., MY to be a discrete-time martingale with respect to Fm = (X for the control variate estimator to have the same bias but lower variance than the naive expected signature estimator. In what follows all path observations are inevitably sampled at discrete points in time and hence, abusing notation slightly, in some places we drop the dependence on the partition x. Whether X is a continuous-time process or a sequence of observations in discrete time should be clear from the context.
Machine learning methods based on signature methods often apply augmentations to the raw streams of data before computing the signature, cf. Lyons& & McLeod (2024, Section 2.5). For example, a path augmentation which is often found to improve model performance is the lead-lag transform. Combining the previous observation on discrete-time martingales with Remark 2.15, we can easily see that the control variate expected signature estimator can also be employed when the lead-lag augmentation is applied to the raw data, i.e. when the d-dimensional discrete-time martingale (Xm; m. =1,, I..... :....... M3 is embedded into the ad-dimensional process
X = (X X,), 2X2, x1); (X2, X2)..... (XM, M-))), M, XM) = (Xm'' m' 0/=2, 2 2M 2}.
Note that for each m' = .,....... 2M + 2, if m' = 2m then x = (Xm, Xm): X = (Xm+1, Xm) and
E| n/ +1 X X = (E[ (I...., Xm], Xm) = (Xm, Xm):
and if m 2m + 1 then X m' = (Xm+1, X,); X m' (Xm+1. X xm+1) and
E E[X
X X = (Xm+1, Xm+1).
and hence the leading components, i.e. the first d entries of X', form a discrete-time martingale with respect to the natural filtration of X'. By Remark 2.15, we can hence apply the control variate expected signature estimator (40) for any word 1= (i....., iz) E f1....,,44 2d] with in E dl.
Finally, in some upplications, we may not have strong prior on whether the process being modeled is martingale or not. In this case, we may consider the martingale correction as a model configuration hyperparameter to be tuned, just like the lead-lag path augmentation discussed above. In the model training phase we can then apply a cross-validation procedure to learn whether applying the martingale correction to (some of) the expected signature terms improves the performance of the model.
Remark F.1. Both the signature transform and the expected signature transform are general methods applicable to any machine learning task dealing with (collections of) streams of data. These can thus always be used as out-of-the-box feature extraction methods when little domain knowledge is available. On the other hand, when task-specific information is known, incorporating such knowledge in the machine learning model will most likely improve performance.
Remark F.2. It is important to note there is also a wide range of machine learning methods based on signature kernels (Kiraly & Oberhauser. 2019; Chevyrev & Oberbauser, 2018; Lemercier et al., 2021; Salvi et al., 2021). This bypasses the need to explicitly estimate the expected signature and hence we cannot directly apply the martingale correction developed in Section 2.2.
F.2. Algorithms
1. TIME SERIES CLASSIFICATION (TRIGGIANO & ROMITO, 2024)
In the Gaussian Process augmented Expected Signature (GPES) classifier, the input stream x E Rddx is interpreted as a discrete-time time realization of a Gaussian process X ~ GP() E(t) at points Tı = {0 = t1 <... < fM = T),
L.e a realization of Xx_ Values of the process over a fixed set of in-fill points 92 {81 < ... < SM,) can thus be
49

---

Learning with Expected Signatures: Theory and Applications
sampled from the conditional distribution of x given the input X i.e. the conditional distribution x (1 Tas 32)- The expected signature of the process X is then estimated from collection of samples X Unu such that X x and x² .V (Uxx.m (x) for 1, I, In Triggiano & Romito (2024), the authors emphasize the theoretical and empirical importance of the tensor normalization introduced in Chevyrev & Oberhauser (2018), ensuring the resulting expected robust signature is characteristic for a larger class of processes. An important component of the GPES model is thus the (truncated at level k) tensor normalization A T (BR)) - T (R.), controlled by the hyperparameter C. For more details on the effect of the tensor normalization procedure and sensitivity analysis with respect to the slyeer-prameete C' we refer Triggiano & Romito (2024). When applying the martingale correction the GPES model?? we subtract the correction term ( (x )o.TT] each S' (T( ( i.e before taking the empirical expectation over the paths. This modification of the original algorithm
highlighted in green in Algorithm 1. The final layer of the GPES model then maps the expected signature class by combination of linear transformation and softmax output activation. The forward pass through the GPES model is summarized in Algorithm 1.
Algorithm 1 Gaussian Process augmented Expected Signature (GPES) classifier, forward pass
hyperparameters Signature truncation level & E N, tensor normalization parameter C' IR,,, data augmentation size N E N, in-fill partition i2 E A [0.2]] s.t. M2 E N.
parameters Biases by R by E R boa Rdu and weights W₁, Rd Wy E R Xdu, Wout E IR x where din + (d Mi + Mi + M2). du + d M2, dy, + d M2(d M2 1)/2, daig + (d 4 ... + d²) and doot + [C].
input x RddxM and æ A Mj
10.27"
1: Mx, + b + (x, T2)
2: Lx f- by + Wx (x, Ti, π2) and Ex.ra + Lx LT 22,
3: for (1....,, N) do
4: Xn.51 4 x.
5: Sample x N (1x 122 121 Dx 11
6: Signature of i ; S" = 5 (x (XMMLUU (xm.)Umma)) po.T (o.T E BR4.. 7: Tensor normalization: St t (S()
8: end for
9: Expected signature ES- + Σm= S"
N
output è < softmax (Dou Wont out ES).
Note that, unlike classic Gaussian process regression where the prior mean is assumed to be constant vu()= and the prior covariance function
Σ: [0, [] - dxd
is parameterized by a kernel and posteriors are computed by combining standard properties of the multivariate normal distribution and the kernel trick in the GPES model the conditional mean and covariance functions
Mx, E &RdxM² & Rd M2
(x, Ti, x2) IR d1 x A 10.21 10..7
Ed M
Ex E MM(dxMMa) s M2ddM
are parametrized by linear transformations
Px, b, + Wn, (X, nI, T2)
Ex = Lx,m, .2 m
Lx by + WE T1, T2),
SSuper-samlling the input data collection of realizations from Gaussian process, can he effectively understond as regularization by noise technique.
"Chossin very large value of C is in practice equivalent to not applying a tensor normalization. 27 Note that the GPES algorithm estimates the expected signature conditional on xn. x, so technically the martingale correction biasing the estimator.
26e Gaussian process regression model is then fitted by tuning the kernel hyperparameters (either via maximum likelihood cross-validation).
50

---

Learning with Expected Signatures: Theory and Applications
where ma is lower triangular matrix. The parameters b by, Wn Ws are learned along with the output layer parameters bow; Won in the training phase via numerical optimization, in Triggiano & Romito (2024) the authors use simple stochastic gradient descent (SGD). If the timestamps и of the observations are not provided, they need to be fixed and can be regarded as hyperparameters of the model. The (way the) in-fill partition (is chosen) is instead always chosen a-priori, with a natural choice for 22 being the set of mid-points of тΙ Other model hyperparameters include the signature truncation level k, the size of the augmentation N and the constant C' controlling the strength of the tensor normalization, as well as the training procedure's 's hyperparameters (learning rate, batch size etc.).
In Section 3.2.1 we replicate the synthetic data experiments of Triggiano & Romito (2024). These consist of three datasets:
(FBM) Two equally balanced classes with samples generated according to a standard Brownian motion and fractional Brownian motion with Hurst parameter H = 0.26 (both in dimension d = 1).
(OU) Two equally balanced classes with samples generated according to two different Comstein-Uhlenbed (OU) processes (both in dimension d = 1).
(Bidim) Six equally balanced classes with samples generated according to six different bi-dimensional stochastic processes (d=2) 2).
When fitting the models we take the optimal hyperparameters cross-validated by Triggiano & Romito (2024) and apply cross-validated SGD to the training dataset. That is, we use 80% of the training dataset to iterate through SGD parameter updates, while keeping the remaining 20% of the training dataset (the validation set) to determine when the procedure has converged without overfitting. As described in (Triggiano & Romito, 2024) the presence of the tensor normalization step often leads to exploding gradients in the training procedure. We thus repeat the SGD routine over 5 different parameter initializations and pick the model with best validation performance.
2.2.2. PRICING PATH-DEPENDENT DERIVATIVES (LYONS ET AL., 2021)
While the authors of Lyons et al. (2021) consider more general setting, for brevity, we focus on the case where the (discounted) price process X is assumed to be a semimartingale. Let X = {X,, t E [0, }}} be a semimartingale on the probability space (22 = C([0,, T], R), F, and denote by 12/ the set of realized (time and lead-lag augmented) price signatures, i.e.
0 = (S( 10.2] E T((R))) X (11, 1 12 03),
where we refer to Lyons et al. (2021, Definition 2.14 and Example 2.15) for the definition of the lead-lag augmentation but, for the purposes of our discussion, it is sufficient to note that it is uniquely defined through Stratonovich integration. The authors then consider the market (Ω2, B(QH), {Fi, t E [0, T)}. PPI.)) where pil. is the push- mush-fo of P onto DH By defining the set of derivative payoff's as all measurable F: : 24 - R. i.c for given price realization X the holder of the derivative receives F(S(X!")))oo )o.T)) in Lyons et al. (2021, Proposition 4.5) the authors use the universality of the signature to show that any continuous payoff F can be arbitrarily well approximated by a linear payoff In particular, this implies the price of any such F can be decomposed as
E 2r F(S
(o.Ti)] Ri (J, 2r (2.)))
for a set of linear coefficients f E T( where Q pricing measure for X and Z7 is a deterministic discount factor over [0, TJ. The set of signature payoffs (x* I E W([1, 2. 4})} can thus be understood as a set of Arrow-Debreu securities spanning the set of continuous path path-dependent derivatives F. Similarly, in Lyons et al. (2021, Proposition 4.6) the authors show that linear trading strategies are dense in the space of admissible trading strategies A, here defined as the set of all continuous functions 0: ((.)) - ((S())) over the stopped at E [0, T] augmentddeteed price path signatures, i.e.
0 (()))) ~ (e, S()))..)) Vt E [0,7],
The only hyperparameter we modify is the truncation level which we set to for computational reasons (in the original paper the optimal value was found to be 5 or 6, depending on the dataset). The hyperparameters an the training and testing routines used to produce the results in Table I can be found at ht / /gi thub. com/ lorenzo lucchese/ htpp/gpesssee cop-esig-clan sos--caassiiie
Note that the approximation of F by f does not depend on the choice of probability measure TP, i.c. it is a pathwise density result. In Algorithm 2 and Algorithm 3 we thus assume that f has been estimated offline (i.e. for any model) by linearly regressing F(w) against w for a large set of w E 244.
51

---

Learning with Expected Signatures: Theory and Applications
for of linear coefficients é T((()))). These two results are then combined in Lyons et al. (2021, Theorem 4.7) show that the solution of the quadratic IP-hedging problem
[ (
- Po - S." (0,(X) )
27
0*
argmin E²
(52)
BEA
can be arbitrarily well approximated by the solution a linear signature quadratic hedging problem. i.e.
0 (s(())))))) (e", ((7)))))a)))) E [0,
where le E (((2))) can be computed by
argmin ((f - Pol + 64)*² "4) p.rl)). (ET(()))))
p,r7))
Algorithm 2 Pricing Path- Path-Dependent Derivatives with Expected Signatures
hyperparameters Signature truncation levels k number of Monte Carlo samples N E N. parameters Risk-neutral measure Q, deterministic discount factor Zr E R+, linear approximator f E TTT*((()))
input Derivative payoff F S2 +R R.
1: Sample N trajectories X ,...,."' X Q
2: Compute time-augmented lead-lag transforms LL for n E (I,...,Ny, N).
3: Compute S E (R²)) s.t. ST + S (x (o.ry (2.7)) I s forn E N). (T),
4: Estimate Φ T* ((R') s.t. F IN (T), IT| from ( 1==1
output Price (f, D).
Algorithm 3 Hedging Path- Path-Dependent Derivatives with Expected Signatures
hyperparameters Signature truncation levels k, number of Monte Carlo samples N E N. parameters Real-world measure I, initial capital po e R, linear approximator f E T ((R))")
input Derivative payoff F Q4f RR.
1: Sample N trajectories X ..... XN P.
2: Compute time-augmented lead-lag transforms L.LL for E (I,...,,N,
3: Compute S ((R')) s.t. ST + ('x"'LL)) [o.7]. [| S k for n E {I,...,.1 N).
4: Estimate & E TT((())) s.t. t, - 8) N (T). [|] S from (XM.LI 1 J==1-
5: 8" <- (())) (lf Po@ 14) Φ).
output Hedging strategy ł + (e, S())))))..) E [0,7].
These theoretical results suggest both a pricing and a hedging strategy for path-dependent derivatives based expected signatures summarized in Algorithm 2 and Algorithm 3. Both algorithms make use of expected signature estimation via Monte Carlo simulations, an approach that provides a classic setting for applying the martingale correction described in Section 2.2 (recall that by Remark 2.15 and the Lead-Lag discussion in Appendix F.I we apply the correction only to signature terms with the process X appearing in the outer integral). Note that price processes under P (as considered the hedging setting) are usually not martingales and hence is not clear whether the variance reduction for the Monte Carlo estimator of the expected signature obtained via the martingale correction offsets the introduced bias. On the other hand, under Q. the fundamental theorem option pricing ensures the discounted asset price process is a (local) martingale and hence the martingale correction is exact.
31 For conciseness here we only discuss the quadratic hedging problem, Lyons (2021) the authors obtain results general polynomials which allows them to also approximate the optimal hedge under exponential utility.
As discussed in Pérez Arribas al. (2018, Section 6.1) the expected signature under the measure ! can alternatively be estimated in model-free way from the market prices of a large enough set of exotic derivatives, yielding the implied expected signature. Here assume measures and Q have been appropriately calibrated to market data.
52

---

Learning with Expected Signatures: Theory and Applications
F.2.3. DISTRIBUTIONAL REGRESSION FOR STREAMS (LEMERCIER ET AL.,, 2021)
The machine learning model we consider in Section 3.2.3 perhaps the most natural one when working with expected signatures. Introduced in Lemercier et al. (2021), the Signature of the pathwise Expected Signature (SES) model aims to learn map from collection of paths X!.... ..... .......NNN E X' S Lip([0, 2], I,),, understood path space μ N Em=1 ōg- E P(t), to a corresponding scalar value f(u) E R. The task of learning such function : P())) R. from finite number of (possibly noisy) observations {(4,, (i) JeTum is known distributional regression. Under appropriate conditions, by combining the characterizing property of the expected signature and the universality of the signature, the authors show that linear functionals on the signature of the pathwise expected signature are universal for weakly continuous functions / (())) - R (Lemercier et al., 2021, Theorem 3.2). The training and testing of the SES model is summarized in Algorithm 4 with the step at which can apply the martingale correction highlighted in green.
empirical measure on
Algorithm 4 Signature of pathwise Expected Signature (SES), training and testing hyperparameters Signature truncation levels: K1, ho E N. Linear regression regularizers. input [({X") Ne,, BE) [[(X)) IeE, Jieza where X" E RdxM and yı E IR.
1: d1 f ......... d and d2 t d1 + +....... d
2: for i E Immin U Test do
3: Pathwise expected signature of (x") nEE: D' RRdIM,,y I,m t Naci (tm), 11sk, 1 ISmSM.
4: Signature of Φ': S + S
Rd
5: end for
6: Fit linear regression: A = (Ba. ..... . Bdx)) t LinearRegresionFii(((S (S:: vizzm))))
7: Predict using fitted linear regression: (úr JEI LineanRggessiooPPreeccc/t/ (S* JiEIma).
output Performance metric: L((ūi (ie (vI
In Section 3.2.3 we repeat two of the synthetic data experiments conducted in Lemercier et al. (2021), analyzing the performance of the SES model without and with martingale correction (MC). In the first experiment (Lemercier et al., 2021, Section 5.2), the task is to infer the temperature of an ideal gas from the paths of N = 20 particles moving in a box. The dynamics of the system are inevitably linked the radius of the particles, with larger particle radii resulting in more frequent collisions. Two settings* are therefore considered, one with smaller particle radii rı 0.35 x V/VN and one with larger particle radii 12 = 0.65 x R//N.. The second experiment (Lemercier et al., 2021, Section 5.3) concerns the estimation of the mean-reversion parameter in a rough volatility model. More precisely, the task is infer the value of a E [10-9, 1] from sample {a I==1 of (discretely observed) paths Tº {of'. t E T) over the partition æ {0, 0.01, .., 2] with continuous-time dynamics
dz, = -a - + vdB or = exp Zz, t E [0,2]
where (B E [0, is a fractional Brownian motion with Hurst parameter H 0.2, p 0.5, 0.3 and Zo- 0.5. The performance of the model is evaluated with increasingly large collections of paths inputs, i... N 20, 50, 100. As expected, the model becomes more accurate as the number of paths increases. We refer to Lemercier et al. (2021) for more details on the two experimental setups.
In both experiments, we keep the same training-evaluation pipeline as the one considered in the original paper, namely nested k-fold cross-validation with 5 outer folds for evaluation and inner folds for hyperparameter selection (including the signature truncation ką and Lasso regularization parameter). The code used to produce the results of Table and Table 3 is available at https:/ // / tigt......... com/ oorenzo wistteeeeeeeetteeeeeeeeeeeeeeeee on regress on- reams.
F.2.4. SYSTEMATIC TRADING (FUTTER ET AL., 2023)
The last application we consider is also motivated by financial application and can be understood as natural extension of the quadratic hedging problem (52), In Futter et al. (2023) the authors consider the same setup as in Lyons et al. (2021) but allow the trading strategy E A to depend on the signature of the augmented process
-= (tt, X, f.), t [0, T)} E C((0, T), IR R++d+
WV= = 3cm denotes the volume the box in which the particles are moving.
53

---

Learning with Expected Signatures: Theory and Applications
where X , E [0, E C([O, T Rd) is d-dimensional price process and F {f,, L [0, C( T) R9) is set of q observable but not tradable factors (or signals) influencing X, and trade in d assets. Again motivated by universality result of the signature the authors approximate any such trading strategy by linear one, i.e.
0, (s())) ~ (e,, (o)). i (I,...,,dh d}, t t [0,2],
for some set of linear functionals ,a E T((R)td99))) The authors then show that an explicit solution to the path-dependent vern-variance problem
[ (ei, ((2)
argmin G ((24441))) )S
E [nnLr]
where
PnLr
Σ 1=
for arbitrary truncation level & E N can be read from the entries of
1 D Msig
Rd dsig d d.....+. d(1 + +d++++g 4)
ad
where Haig E Rd and Esig E Rd Xdey and Ar R_ only depend on the expected signature E (S((2!))) (om))
standard way of applying the sig-trading strategy in practice is given in Algorithm 5. Note that in real financial markets price and signal paths cannot be resampled and hence the collection {(*",F")) ,F"))/" can only be obtained by chopping-od shifting a single long observation {(X,, f.), t E NT]]. In this respect, the sig-trading algorithm provides a striking example of setting where the sampling scheme cannot be considered i.i.d. and hence one needs to resort to the results of Theorem 2.10 obtain theoretical guarantees for the expected signature estimator.
A silent but fundamental assumption is that the market dynamics of the collection of past price-factor paths {(X", [(""F")) F"))"= used to estimate the expected signature will be the same as those of the future price-factor process IF*) to which the trading strategy will be applied. Using the martingale correction for estimating (some of the entries of) the expected signature, induces a bias to "ignore" the drift component of the signature term. For example, the first level of the martingale-corrected expected signature is always zero. This might be a desirable feature avoid over-fitting the trading strategy to spurious drifts in the data (for example in the price processes X (Buehler et al., 2022)) while capturing higher order effects. As with other applications discussed in this section, the usefulness of the martingale correction in Algorithm 5 can be empirically cross-validated.
Algorithm 5 Signature Trading
hyperparameters Signature truncation level k, maximum variance A E R.. input Collection of price-factor paths {(X", F²) } where X = (X, EE [0, C([O, TJ. Rd) and F" = (f., t E
[0, E C(10, IR9) for n E NY.
1: Set 2 X,, ft). E [).
2: Compute time augm-nug lead-lag transforms II for (1....,,N) N).
s.t. 2 + &/ (T), | < k from (2T.1L J I==1. 4: Compute As Msig; Ssig from corresponding entries of 4b
3: Estimate & e T ((
5: Extract é from corresponding entries of (2\a - Usg for i E {l.....dh d}.
output Trading strategy t + ( ((2)))))))) i E
(1....,d} d), t E [0, for new F*),
G. Controlled Linear Regression
In this section, we introduce the notion of controlled linear regression. The main rationale is to exploit as much information as possible in the training phase to make the coefficient estimators as precise as possible. We start by considering the following linear model
y 3|21 + ... Bpo, + - +
44 As in the mean- variance optimal portfolio, the variance scaling parameter da E RR also depends on the target variance A. Cleearly, this assumption is not specific the sig-trading strategy but applies to any trading strategy that tries learn patterns from the past to profit in the future.
54

---

Learning with Expected Signatures: Theory and Applications
and assume we observe the training data {Won Fn,1, Fnpp) ,,..., N..ie. i.e.
yn 31 + ... + Byenp + En = =x + Ens L...,,,
where, given the design matrix X IR. NXP, the enrors fcns n == I,..., N are
mean zero, i.e. El X] 0 for n ,,...,N, N,
homoskedastic, i.c. B[e, X o² [0, 00) for n ,,..., N, and
uncorrelated, i.c. Em X = 0 for n. m I...., N with n o m.
For any test observation x. = (z+,1
E R' we thus have the best possible prediction (in terms of MSE) for is
Elu.|x.] = x =: 0.())
and plugging in an estimator 3 for 33 yields the predictor
ú. (. x
Note that, under the assumptions introduced above, the mean squared error of such predictor can be decomposed as
E (3) - y.)² x. = E X,x, E (i. 8. (3))2
= + E (6. (3) (B)) X x.]
Under the assumptions discussed above minimizing the mean squared error of the predictor relative to the target is thus equivalent to minimizing the mean squared error of the predictor relative to the infeasible best prediction ÿ. (3).
G.1. Controlled Ordinary Least Squares (OLS) estimation
Classic OLS estimation The usual OLS estimator for B = -S1...., Bp) E RP is given by
Bx (X"x)
which, by the Kauss-Markov theorem, is known be the best linear unbiased estimator (BLUE): for any À (A1...., Ap) RP,
E - -
min EXELUE(X.y)
- -
where LUE(X, y) is the set of all linear and unbiased estimator for B, i.c 3x = C(X)y for some X-measurable matrix C(X) E R and E[ [X] = B. By applying the BLUE property, we can show that J. (3x) ) is the best predictor across all predictors formed from linear and unbiased estimators, i.e.
E (i- (3x)
e. (.)) S (0.(8x) (.))) ( x,
for all 3x E LUUB(X,y) y). Note that here we applied the mean-zero uncorrelated errors assumption.
Controlled OLS estimation Let us now assume we can additionally observe the "control" random variables {z, (En.1...... 2n,k) E A n = 1,...,,, N}. We shall assume the controls are available for training, i.e. when estimating B, but for predicting, i.e. when forecasting y, we will have access to x, but not to 2,, Given the original design matrix X E Nx. we now assume the errors and the controls are jointly
=,.......
mean zero i.e. (G,, x X = 0 E R++1 for n = 1,.., N,
*Here, and in all other estimators discussed in this section, the dependence on y is dropped from the notation. 37 terms mean squared error (MSE). Recall that unbiased estimators, the MSE is equal to estimator's 's variance.
55

---

Learning with Expected Signatures: Theory and Applications
homoskedastic, i.e. [[[(en 2n 82 X Σ E ((++))((++1) for 1, ...... N for some E symmetric positive definite,
uncorrelated, i.e. [B[(en; Zn (Em; Zm X = E [((+1)x(k++) for n, m 1,..., N with n o m.
In what follows we partition
0²
Σ
E
En.y
Σx
Remark G.1. Throughout the whole section, unless stated otherwise, we consider the original design matrix X and the new observation x, IRP be fixed with random controls Z and errors c.
As discussed above, fixing the design matrix X NR and test observation x, (Fo,1, Ta.p) E IRP, the predictor
ÿ- (3xx) x,,3x.
is unbiased for the statistic d. (3) = =x,3 E R. We hence introduce the control variate predictor
8. 0. Z. X) 8. AT
where Z E N is the control design matrix while A E R and X2 E R are measurable in X and x.. Under the assumptions discussed above the controlled predictor can be shown to be unbiased and attains minimum in variance when
xj X(X"x)=xxx and X - Σー Ez.y
Note that since Z and are assumed to be jointly spherical given X (and x.) the variance of the controlled predictor is given by
Var(O.(Bx Z, x.)
-Var(x! x X3 A
-xI(XXx) x BiEE X, x. XX x, 2x Xx ' XXE//A/ + A x.xIA>
=x'x Xx + 22 x
X1 AEAAA.
Setting partial derivatives in A1 and A2 equal to zero
0x Var (ū- (8x, Z, A) x.) 2XI x x 22
ExA2d1) = 02g Var( (8x. Z. A) 2X x. En.y 2A Ezd2 -AT 'x, Ex.y EzA2) 0,
yields as non-trivial (i.e. such that A1 A2 * 0) stationary point A X(X*x)- xx and AA compute the Hessian at the stationary point to be
Exz Ex.g. By taking second derivatives
=2 Ey. E ELNINNN -xxxxxXXXX) 'x"
X xx Ey.2 x. (X"x)
ax0x Var(j. (8x A)(X A=
-xxxxx1
which is positive (semi)definite since E is positive definite and hence A' is minimum. To show the assumption that Σ is positive definite implies the Hessian is positive (semi)definite the following result from linear algebra theory: symmetric block matrix
A B
B)
M
where square is positive (semi)definite and only and A²A-"B are positive (semi)definite. Applying result to the Hessian at A. note that A = Eg En.IIN b..INNx xNN is trivially positive (semi)definite since My. 2i Ez.v 2 0 by positive definitness of Σz, and
D D- D-B'A-'B- B XXXX"
En
Σx.w Ey.a Eyy F.g S
positive (semi)definite since x, (x"x) -1 1x,x 2 by positive definitness ' and E, (E Si 15 D...y) - EE,yy En, a positive (semi)definite by applying the converse of the previous statement to the positive (semitdetinite matrix
M'
En
1 E.z.
5-
Eg Σn
Σn.y
56

---

Learning with Expected Signatures: Theory and Applications
For any X, (vx,1 (3.,1....,; xpp RP we thus have
V. (.x3 2,
-xxxxxxxxxx Eny "x, = (xx22)))))
yielding the infeasible (in the sense that it depends on the unknown quantity 2) estimator
= (x'==
-ا 3x
Note that, given X, 3x.z.v is unbiased (by mean property of the controls). Moreover, for any test observation X, (2+,1..... pp)p) RP,
Var(x, T Z, X x.) -Var(x, / 3x [X, x.) Var(x
Z 1 1 Ezny y
XX, x.)
-2Cov (X X " (,,
= =o'x] xx + 2y 1 2z (xTX) xx ,3 12 15 En, (X"x) xx
= =(o² - - "i x (X X)
S s0²x (x xxxx) Var (x 3x X, X.);
with equality iff Σz.y O. In other words, as long as the controls are correlated with the target, we obtain a better prediction by using 3x,z. instead of the OLS estimator 3x and the quality of the prediction increases the correlation between y and increases. The variance reduction factor is constant across test observations and is given by
Var(x 3x x, n. X, xx,
(1 (1- En Dx E=1 1E..
Var(x X. X.x.)
g²
Remark G.2. Note that when X = 1 R x, = 1 and Z E RN, we estimate μ. Elvl with the OLS estimator
P.. B y and the simplest control variate estimator
Cov(y, 2) Var(z) 7,
=
1 since Xo 1 FIERN RN and A
Cov(y,2) 2) Var(z)
which has reduced variance by a factor of (1 Corr(y, =)).
In practice, the correlation matrix E is usually unknown; hence, make the estimator 6x.z.x feasible, we need to estimate it. Under the assumptions discussed above, the most natural candidate is given by the sample estimates
1 - z"y N
1 zzz N
and Σn
yielding the feasible estimator
= -x-x) 2(1 'Z)yy
This can be cquivalently understood regressing y the control Z (i.e. projecting y onto the space spanned by Z) and then regressing the residual onto X. The resulting estimator will likely be biased (given X), with its exact finite sample properties depending on the distribution of (X, Z)
When e depends linearly on Z, i.c.
6 Za + 7,
we can write
y= + Za + 7,
and hence we know that the joint OLS estimator obtained from the design matrix (X Z) E R Nx(p+A), i.e.
xx.z
X Z
X T
X Z
X
2)
6x
=
Z
ZTZ
57

---

Learning with Expected Signatures: Theory and Applications
By using some simple algebraic manipulations for block matrices, we extract the first p entries of the joint OLS estimator, Le. the estimator for [3,
3x.2= -
=
Le.. xx,,,x with estimated by
1 N - (XX-X(X"))'x'))
Σ 2,V
N X and Σ
Note these are the covariance and variance estimators obtained when projecting Z onto the orthogonal complement of the space spanned by X. Under the assumptions introduced above, these are also unbiased for Σn,y and Ez. This provides a second feasible controlled estimator for [3.
If we fix both X and Z and assume 17 satisfies the (B, a). Extending the classic OLS estimator 3x to (3x, Ōx,z)) obtain another linear and unbiased estimator for (B, a)). It follows from the Gauss-Markon theorem that for any RR, setting À= = (x., Elz. X Z, x.] 0), one has
conditions, then the joint OLS estimator is the BLUE for
E (0. (8x.z) - ú. (3)) Z,
E (0. (0x) D- (3)) (x, X,
and hence, by the tower property of conditional expectation,
E
(i- (fx,z) ú. (3)) x.]
E (x
Using the forms of the variances of 3x and 3x,z we can compute
IF
(Bx.)) (B)) X, x.
( xXxxx
E (5- G. (3)) X, x.
=x 'x.,
and thus quantify the MSE reduction factor as
(j. (3x.z) - - x.
X -IZZZ
X]x, -Xxx
E
(0x(3x)
ő. (8))2
x!(X"))
which we note does not depend on o².
Given X and Z, augmenting the first feasible controlled estimator 3 xx,255 to an estimator for (3, a),, yields linear but biased (at least in the components) estimator. Whether 3x,2,5 or 3x,z yields better predictor cannot thus be deduced from the Gauss-Markov theorem. As we will see in the numerical experiments discussed in the next section, which estimator performs better depends on the properties of the data generating process.
G.2. Simulation study
In the previous section we introduced two feasible control estimators, B xx,2,9 and Bx,z, ,. for the parameters (3 E RP, We showed that when
6. = Za + '7,
and r) z X is mean-zero, uncorrelated and homoskedastic then ÿ. (33.z) is always better predictor than the one formed
by the classic OLS estimator ÿ. (3x). This leaves unanswered the question of how 3x,z,8 performs relative to 3x,z (and 3x). We address this question empirically in Table 4 with the following experimental setup.
We consider N 1, 000 i.i.d. samples from the model
y = 30 + 3101 + 3222 +
58

---

Learning with Expected Signatures: Theory and Applications
with 80- 1, 6, 32 8 and
C op2 + vI-p?n).
with ~
(0, 1) independently of X. We thus have Corr(y, 2 2/x) Corr(c, 2) and c (N X. We fix 000 samples æ1 ~ (0, 1) 1 D2 ~ (111 1) to obtain X E IR Ngxx and evaluate the performance of the predictor on the new observation of = 0, m² 1. For each estimator, we report an estimate of the root mean square error
RMSE(Ú. (3).0 = VB (0. 6. (3)) X.
obtained over 10, 000 Monte Carlo samples (i.e. keeping X and x* fixed and resampling Z E R. and 7 E R-)). We highlight with a single asterisk (+) RMSES that are lower than the uncontrolled OLS estimator's RMSE with high statistical significance (t-test p-value across the Monte Carlo samples less than 0.001). When one of the two proposed estimators outperforms the other with high statistical significance we highlight the corresponding RMSE with double asterisks (*4).
3x
X (% of 3xx
3x.z
3x,2,1
P
RMSE
RMSE
RMSE
(% of (3x)
RMSE
(% of 13x)
0.00
0.1572
0.1579
100.46%
0,1573
100.03%
0.1572
100.00%
0.25
0.1588
0.1550
97.57%
0.1538*
96.86%
0.1489
93.75%
5
0.50
0.1581
0.1368*
86.48%
0.1358#*
85.86%
0.1186
75.00%
0.75
0.1592
0.1052
66.08%
0.1043 #.
65.52%
0.0697
43.75%
1.00
0.1606
0.0164
10.21%
0
0.01% %
0.0000
0.00%
0.00
0.3
0.3146
100.06%
0.3145
100.03%
0.3144
100.00%
0.25
0.3177
0.3082
97.03%
0,3077
96.86%
0.2978
93.75%
10
0.50
0.3163
0.2719
85.97%
0.2715
85.86%
0.2372
75.00%
0.75
0.3185
0,2088
65.57%
0,2086
65.52%
0.1393
43.75%
1.00
0.3211
0.0164*
5.11% 1
0.0000
0.00%
0.0000
0.00%
0.00
0.6289
0.6286
99.96%
0.6291
100.03%
0.6289
100.00%
0.25
0.6353
0.6154
96.86%
0.6154
96.86%
0.5956
93.75%
20
0.50
0.6325
0.5429
85.83%
0,5431
85.86%
0.4744
75.00%
0.75
0.6369
0.4169*
65.46%
0.4173
65.52%
0.2786
43.75%
1.00
0.6422
0.0164*
2.55%
0.0000**
0.00%
0.0000
0.00%
0.00
1,2578
1.2569
99.93%
1,2582
100.03%
1,2578
100.00%.
0.25
1.2707
96.80%
1.2307
96.86%
1.1913
93.75%
40
0.50
1.2651
1.0853**
85.79%
1.0862*
85.86%
0.9488
75.00%
0.75
1.2738
0.8336** **
65.44%
0.8346*
65.52%
0.5573
43.75%
1.00
1 1.2845
0.0164*
1.28%
0.0000**
0.00%
0.0000
0.00%
Table 4. RMSES of the classic OLS estimator 3x, the two feasible control estimators 3x x and 3x, and the infeasible optimal control estimator 3x,z.5> A single asterisk (*) indicates feasible RMSEs that are lower than the uncontrolled classic OLS estimator's RMSE with high statistical significance (1-test p-value across the 10,000 Monte Carlo samples less than 0.001). double asterisk (**) indicates the feasible control estimator's RMSE is lower than the other feasible control estimator's RMSE with high statistical significance.
As expected, as the correlation between the control and the target p increases, the performance gain obtained by using either of the feasible control estimators increases. The joint-OLS estimator outperforms the standard OLS estimator by the same amount across different signal-to-noise ratios while 3 x,,z,t 's relative performance changes. Comparing 3xx xx.z,5 and 3x.z we note the results suggest that for high signal-to- -oiise ra the joint-OLS estimator 3 slightly outperforms , x while for lower hignal-to-noiss ratios the latter estimator performs marginally better. Consider the two edge cases:
Here we define the signal-to-noise ratio as std(x std(x where std(xT, (3) measures the variablity of the explainahle part of the model (signal) and measures the variability in the error (noise). Table the signal component of the model is kept fixed and hence higher a denotes lower signal-to-noise ratio.
59

---

Learning with Expected Signatures: Theory and Applications
when there is error, i.e. a 0. 1.,3x x,2 performs worse than 3x (= 3x,z) is adding uninformative variability to the estimator;
when there is no signal, i.. X 1 RR xx,z, and 3x.z both reduce to the classic control variates estimators, cf. Remark G.2, but the former is more precise as it estimates X -2g D'g'E,,, using the knowledge that the control is mean zero.
Next, investigate the empirical performance of xx.z,5 and 3x.z when the dependency between Z and is non-linear but (Z, c) X are still jointly mean-zero, uncorrelated (across samples) and homoskedastic. Keeping the same design matrix X in Table 4 and the parameters (-1,6, 6,8) now let
E 6= oxf + 1
with 17 1), ~N
1)) independently of X. By choosing f(2) such that E[f(2)] and E[f(2)?] 1 have E[c] = and E[{²]= o². Moreover, choosing Cov(z, ((z)) ensures that Corr =/x) Corrle, p. We investigate the following three dependence functions:
(sq) f(2) 2² +
V
23
(cube)
f(2) V15"
(exp) f(2) e² Ve Vea e
The code used to produce the results of Table and Table can be found at https: httpp://github... com/
We also experimented with multiplicative noise (e [(z) n)) heavier tailed emrors (n (3) and different dataset sizes (N {100, 1000, 10000}). The results are similar to the ones reported in Table 4 and Table 5: the two control estimators outperform the classic OLS estimator as long > 0 while the differences in performance between the two control estimators are statistically significant but small.
60

---

Learning with Expected Signatures: Theory and Applications
3x
xx,z,5
xx,z
fx,Z,5
f
P
RMSE
RMSE
1% of 3xx
RMSE
(% of 3x)
RMSE
(% of 3xx
0.00
0.3144
0.3146
100.06%
0.3145
100.03%
0.3144
100.00%
0.25
0.3174
0.3084
97.14%
0.3075
96.88%
0.2976
93.75%
(sq)
0.50
0.3202
0.2772
86.57%
0.2761 **
86.25%
0.2401
75.00%
0.75
1.00
0.00
0.3144
0.3146
100.06%
0.3145
100.03%
0.3144
100.00%
0.25
0.3184
0.3088
97.00%
0.3083
96.83%
0.2985
93.75%
(cube)
0.50
0.3184
0.2736
85.95%
0,2734
85.86%
0.2388
75.00%
0.75
0.3222
0.2125
65.96%
0.2123*
65.90%
0. 0.1409
43.75%
1.00
0.00
0.3144
0.3146
100.06%
0,3145
100.03%
0.3144
100.00%
0.25
0.3178
0.3085
97.07%
0.3078 **
96.85%
0.2979
93.75%
(exp)
0.50
0.3182
0.2741
86.13%
0.2734
85.92%
0.2387
75.00%
0.75
0.3191
0.2091
65.53%
0.208 **
65.20%
0.1396
43.75%
1.00
Table 5. RMSES of the classic OLS estimator 3x. the two feasible control estimators 3x 22 and 3x and the infeasible optimal control estimator (x,2,s- A single asterisk (*) indicates feasible RMSEs that lower than the uncontrolled classic OL.S estimator's RMSE with high statistical significance (t-test p-value across the 10,000 Monte Carlo samples less than 0.001). double asterisk (**) indicates the feasible control estimator's RMSE is lower than the other feasible control estimator's 's RMSE with high statistical significance. Empty values indicate setups that are not achievable for the given correlation p and dependence function f.
61


Reviews:
[{"review_id": "0xhkrFG7mp", "reviewer": "Reviewer_YgTV", "summary": "An empirical estimate of the expected signature of a stochastic process depends on the number of observed paths $N$ and the partition $\\pi$ on which the paths are observed. This paper shows that under suitable conditions, the empirical estimate of the expected signature of a canonical geometric stochastic process is consistent and asymptotically normal in the double asymptotic limit of $N\\rightarrow\\infty$ and $|\\pi|\\rightarrow 0$. Brownian motion, fractional Brownian motion (fBM), a bidimensional continuous-time autoregressive process, and the price-variance dynamics of a Heston model are shown to meet the conditions for the empirical estimator to be consistent and all but fBM are shown to meet the conditions for the empirical estimator to be asymptotically normal. Furthermore, this paper shows that when the stochastic process is a martingale, you can introduce a zero-mean correction term which reduces the variance of the empirical estimator. The paper concludes by highlighting that the theoretical results deepen our understanding of existing machine learning approaches which use the expected signature and by demonstrating that the martingale correction leads to improved empirical performance, even for processes which are not martingale.\n\n## Update after rebuttal\n\nThe authors addressed my minor stylistic comments and concerns regarding applying the method to non-martingale processes. I am happy to recommend acceptance after the rebuttal.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for the comments. We believe the reviewer has well understood the main contributions of the paper, which they summarized in \"Theoretical Claims\". \n\n**Minor comments**\n- Line 37: typo, the informational content (i.e. the norm) of signature terms decays factorially.\n- Line 39: we agree this was poorly worded, we shall update the references.\n- Line 88: we will fix notation to be consistent. \n- We will add references to the proof locations in the main text (e.g. Proof of Theorem 2.8. See Appendix A.1.)\n- We will add short outlines to the proofs (e.g. the main idea in the proof of Theorem 2.8 is to show the sequence of signatures along $\\pi\\_n$ is a Cauchy sequence in $L^m$). We shall also add a better partitioning between sections of this proof that concern conditions $(i)$, $(ii)$, and $(iv)$.\n- Line 296: we will spell out Multivariate Continuous-time Autoregressive Process (MCAR).\n- Line 827: typo.\n\n**Questions For Authors**\n1. Yes, there is a theoretical barrier to progressing to $\\alpha\\leq 1/4$. This arises when applying Lemma A.1 to bound the sum of $Z^{\\mathcal{I}}\\_{[s,t]} $ with $i\\_2=4$. More precisely, let $\\alpha\\leq 1/4$ and assume we follow the same proof strategy of Theorem 2.8 up to line 794 under assumption $(iv)$. Then, when $\\alpha\\leq 1/4$, we cannot apply (19) to bound the sum of the $Z^{\\mathcal{I}}\\_{[s,t]}$'s with $i\\_2=4$ and so this term needs to be treated separately in the same way the $i\\_2=2$ and the $i\\_2=3$ had to be treated separately when $\\alpha\\leq 1/2$ and $\\alpha\\leq 1/3$ respectively. In a similar spirit to the $i\\_2=2, 3$ cases we can thus attempt to apply Lemma A.1 to bound the sum of the $Z^{\\mathcal{I}}\\_{[s,t]}$'s. In this case though, even if we impose a condition similar to (A$\\beta$) and (A$\\gamma$) to bound the first term by $|\\pi\\_n|^{\\zeta -1}$ for some $\\zeta>1$, the second term, i.e. the sum of the $\\\\|Z^\\mathcal{I}\\_{s,t}\\\\|^2$, will be bounded by $|\\pi\\_n|^{8\\alpha - 1/2}$ which does not necessarily vanish when $\\alpha \\leq 1/4$. \n2. When the data generating process is not a martingale the corrected estimator introduces an additional bias which, depending on how \"far\" the process $\\mathbb{X}$ is from being a martingale, might offset the benefits of the variance reduction. When $\\mathbb{X}$ is \"close\" to being a martingale though, the corrected estimator is still theoretically better (in terms of MSE) than the classic estimator. \t\n\tMoreover, in practical applications, the corrected estimator might turn out to be a more informative quantity than the classic estimator. Intuitively, one might expect this to be the case when the drift of the underlying process is not relevant for the downstream task. In cases where the underlying process cannot be assumed to be a martingale we thus suggest to treat the martingale correction as a data transformation applicable in the learning pipeline (a model hyper-parameter in a similar spirit to the add-time or the lead-lag transform in the signature context) whose usefulness may be empirically ascertained via cross-validation. This is discussed in Appendix E.1 (paragraph starting on line 2436). If the reviewer feels this is an important point to include in the main text we would be happy to move the relevant discussion to Section 2.2. \t\n3. Fixing the sampling rate leads to an irreducible lower bound on the precision with which we can learn the expected signature of the continuous-time process (in a similar spirit to the Nyquist–Shannon sampling theorem). If $\\pi$ is fixed (e.g. because of constraints on the sampling rate) and we let $N\\rightarrow \\infty$ the empirical expected signature $\\hat{\\phi}\\_{\\mathbf{I}}^{\\Pi(N)}(T)$ where $\\Pi(N) = \\pi \\cup (T+ \\pi) \\cup \\dots \\cup((N-1)T+ \\pi) $ is a consistent (and asymptotically normal) estimator for the expected signature $\\phi^\\pi\\_{\\mathbf{I}}(T) = \\mathbb{E}[S^\\mathbf{I}(\\mathbb{X}^\\pi)\\_{[0,T]}]$ of the *piecewise linear* process $\\mathbb{X}^\\pi$ under the condition that $\\\\{\\mathbb{X}^{n, \\pi}, n\\geq 1\\\\}$ – or sufficiently $\\\\{\\mathbb{X}^{n}, n\\geq 1\\\\}$ – is stationary and ergodic (and strongly mixing). This follows directly from the application of Birkhoff's ergodic theorem given in A.2.1 (and the dependent CLT in A.2.2) with $\\mathbb{X}$ replaced by $\\mathbb{X}^\\pi$. The estimator $\\hat{\\phi}\\_{\\mathbf{I}}^{\\Pi(N)}(T)$ is thus asymptotically (in the $N\\rightarrow\\infty$ limit) biased for the expected signature $\\phi(T) = \\mathbb{E}[S(\\mathbb{X})\\_{[0,T]}]$ of the underlying stochastic process $\\mathbb{X}$, with bias $\\phi^\\pi(T) - \\phi(T)$ (this is the \"irreducible error\" due to $\\pi$-sampling). The in-fill asymptotic $|\\pi|\\downarrow 0$ is crucial to show such error vanishes, i.e. the empirical expected signature is a consistent estimator for the expected signature $\\phi(T)$.", "claims_and_evidence": "The main claims of this paper are theoretical, and I discuss these and the provided proofs in more detail in the relevant section. The evidence provided to demonstrate the empirical benefits of the martingale correction is clear and convincing. However, the paper does not provide a theoretical or intuitive justification for the improved empirical performance observed on non-martingale processes.", "methods_and_evaluation_criteria": "A good demonstration of the benefits of the martingale correction is the comparison of pricing path-dependent derivatives. However, the other two classes of problem considered both contain non-martingale processes. This means that the martingale correction introduces a bias into the empirical estimator. Although the martingale correction is shown to be beneficial on these datasets, the broader implications of applying the correction to non-martingale paths and in which situations it is safe to do so are not discussed in the paper.", "theoretical_claims": "The majority of this paper is dedicated to proving four theoretical claims.\n\n1) Under suitable continuity conditions on $\\mathbb{X}$, $S^{\\mathbf{I}}(\\mathbb{X}^{\\pi_n})$ converging in probability to $S^{\\mathbf{I}}(\\mathbb{X})$ as $n\\rightarrow\\infty$ and $|\\pi_n|\\rightarrow0$ implies that it converges in $L^m$.\n2) That if the sequence of paths $\\\\{\\mathbb{X}^n, n\\geq 1\\\\}$ is stationary and ergodic and you have a set of refining partitions as in (1), then the empirical estimator is consistent. If further the sequence $\\\\{\\mathbb{X}^n, n\\geq 1\\\\}$ is strongly mixing, then the empirical estimator is asymptotically normal.\n3) That under suitable conditions, when $\\mathbb{X}$ is a Gaussian process the empirical estimator is consistent. \n4) If $\\mathbb{X}$ is a martingale, then there exists a mean-zero correction term which reduces the variance of the empirical estimator, and can itself be estimated.\n\nI have thoroughly reviewed the proof of claim (1) and found no errors. However, due to time constraints, I was only able to briefly review the proofs of claims (2), (3), and (4). No apparent issues were identified during this cursory examination. These claims are a significant theoretical contribution, providing an important advancement to the existing literature. However, given that the paper is highly theoretical and 55 pages long, I am concerned that a conference with an eight-page limit does not offer a suitable venue for adequately presenting or rigorously reviewing these theoretical results.", "experimental_designs_or_analyses": "The design of all experiments seems sound.", "relation_to_broader_scientific_literature": "This paper directly extends the work of Ni et al. (2012) and Passeggeri et al. (2020), who studied the asymptotic behaviour of the expected signature empirical estimator for Brownian motion and fractional Brownian motion respectively. Additionally, this paper gives a theoretical framework for understanding and extending existing machine learning applications of the expected signature, such as those in Lemercier (2021) and Triggiano et al. (2024).", "essential_references_not_discussed": "I am not aware of any missing essential references.", "other_strengths_and_weaknesses": "I have no other strengths or weaknesses I wish to highlight.", "other_comments_or_suggestions": "## Minor Comments\n\n- On line 37, it is not clear to me what the informational content of the signature is and why this decays exponentially, when the terms of the signature decay factorially.\n- On line 39, the signature was not introduced in rough path analysis. It dates back to `Iterated Integrals and Exponential Homomorphisms' by Chen (1954).\n- On line 88, you swap the order of $n,\\pi$ and $\\pi,n$.\n- The Theorems and their proofs in the Appendix are not well connected, with no references in the main body of the text to the location of the proofs.\n- The proofs can be difficult to follow. The presentation would benefit from short outlines of the proof strategy at the beginning of the relevant sections and subsequent headings to section the proof. For example, this could be used to highlight the areas of the proof of Theorem 2.8 which concern conditions i), ii), and iv).\n- On line 296, the abbreviation MCAR is not defined.\n- On line 827, a second by is accidentally included.", "questions_for_authors": "I have three questions for the authors.\n\n1) The structure of the proof of Theorem 2.8 under i), iii), and iv) suggests it may be possible to continue extending to $\\alpha\\in(1/(n+1),1/n]$ by assuming additional higher order bounds on the path increments. Is there a theoretical barrier to progressing to $\\alpha\\leq1/4$ or just progressively tedious algebra?\n2) How should the martingale correction being empirically beneficial on non-martingale processes be interpreted? Is this a complete surprise or something that has a theoretical (or intuitive) explanation?\n3)  A common setting in application is the ability to continue increasing the number of samples $N$, but being limited by the sampling rate $|\\pi|$. Is it possible to comment on the asymptotic behaviour of the empirical estimator as just $N\\rightarrow \\infty$? Would you expect the estimator to be inconsistent in this limit?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tZTT0VzjrN", "reviewer": "Reviewer_v9sL", "summary": "The authors explore an interesting and young topic with in ML, that is, signature-based methods for ML. Signature methods have been quite useful as a sort of preprocessing stage in ML pipelines to synthesize long time-series data among other applications.\nThe authors theoretically explore the expected signature and it's relation to the empirical expected signature providing a useful bridge between theory and practice.\nIn this work they explore how to improve the consistency of such empirical estimators.\nLastly, the explore a few relevant applications.\n\n## Update after rebuttal\nI think this is a significant work that can establish future downstream applications making use of the promising tool of signatures from rough paths theory. I increased my score to 5 - strong accept.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for the comments. We believe the reviewer well understood the main contributions of the paper, which they highlighted in \"Summary\". \n\n**Other Strengths And Weaknesses**\n- Conclusions were not added due to space constraints, in case of acceptance we shall include the following short conclusion reiterating the main contributions of the paper. \"In this paper we established new estimation results for the expected signature, a model-free embedding for collections of data streams. Our consistency and asymptotic normality results bridge the gap between the theoretically \"optimal\" continuous-time expected signature and the empirical discrete-time estimator that can be computed from data. Moreover, we introduced a simple modification of such estimator with significantly better finite sample properties under the assumption of martingale observations. Our empirical results suggest the modified estimator might improve the performance of models employing expected signature computations even when the underlying data generating process is not necessarily a martingale.\" \n- Inspired by the provided reference [1], in case of acceptance, we shall include a simple diagram to visually illustrate the main contributions of the paper in the introduction (unfortunately we cannot upload the image in this rebuttal).\n- We generally followed the convention that all the equations that are cross-referenced in the paper are numbered and the others are not.\n- See answer to question 3. below.\n- The results in the appendix provide the detailed proofs of the results discussed in the main paper. These details though are not strictly necessary to understand the main contributions of this paper but we still believe a good mathematical proof should leave as little details unexplained as possible. This is why we would like to keep the level of detail presented in the appendix as is.\n\n**Other Comments Or Suggestions**\n- We will move the discussion contained in this footnote to the main text as it provides the main motivation for using the martingale correction term, see also the answer to question 3. below.\n- As discussed above, we will highlight key points and contributions further by adding an intuitive diagram and a short conclusion.\n- This is addressed in the answer to question 3. below.\n\n**Questions For Authors**\n- In our notation $S^k(\\mathbb{X})\\_{[0,T]} \\in (\\mathbb{R}^d)^{\\otimes k}$ is the $k$-th level of the signature, i.e. $$S(\\mathbb{X})\\_{[0,T]} = (S^0(\\mathbb{X})\\_{[0,T]}, S^1(\\mathbb{X})\\_{[0,T]}, S^2(\\mathbb{X})\\_{[0,T]}, \\ldots),$$ where the signature $S(\\mathbb{X})\\_{[0,T]}$ lives in the tensor algebra $T((\\mathbb{R}^d)) = \\oplus\\_{i=0}^\\infty (\\mathbb{R}^d)^{\\otimes k}$. We believe the reviewer was referring to the notation used in [1] where $S^k(\\mathbb{X})\\_{[0,T]}$ denotes the $k$-th level *truncated* signature, i.e. the mathematical object living in $T^k((\\mathbb{R}^d))= \\oplus\\_{i=0}^k (\\mathbb{R}^d)^{\\otimes i}$, and hence $S(\\mathbb{X})\\_{[0,T]}$ is indeed retrieved by taking the limit as $k\\rightarrow\\infty$.\n- This equation was left unnumbered due to formatting reasons. We agree with the reviewer that this is quite an important equation and thus we will number it.\n- This corrector (aka control) was chosen based on a control variate argument: by adding an appropriately signed (and scaled) mean-zero control to the estimator we preserve the original estimator's bias while reducing its variance (the stronger the correlation between the control and the original estimator, the larger the reduction in variance). The form of the signature of a martingale yields a natural candidate for the choice of such control variate: by substituting the outermost Stratonovich integral with an Ito integral we obtain a mean-zero term which, by construction, should be highly correlated with the original estimator, thus leading to a significant reduction in variance. This is the content of footnote 6 on page 6. As this provides the main motivation for choosing the form of the corrected estimator we shall move the discussion contained therein to the main text.", "claims_and_evidence": "To the best of my knowledge all the claims are supported, however, I am not a expert in rough path theory and only performed a quick review of the arguments and proofs.", "methods_and_evaluation_criteria": "The authors explore several interesting and relevant experiments. The evaluation seems to be fair and illustrates the strength of the theory as they are able to reduce the variance of their estimator.", "theoretical_claims": "I performed a preliminary read through of the argument, however, I did not have the time nor expertise to extensively review the 40+ pages of appendices in detail.", "experimental_designs_or_analyses": "From what I read for all three time series experiments they seem sound.", "relation_to_broader_scientific_literature": "This work seems quite relevant to research which uses long time-series data.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "## Strengths\n* Well written paper\n* Addresses an important area of research\n* Theoretical results could be quite important for future research in the coming years\n\n## Weaknesses\n* No conclusion at the end to reiterate contributions\n* Paper is quite dense (to be expected) and the organization will good is not exemplary\n* I think [1] is a good example of illustrating complicated topics to the layman.\n* Numerous equations in the main paper are unnumbered which is bad practice for future citations.\n* The motivations for several key design decisions are either missing or not communicated clearly. Especially, for work of this type it is highly important that you make your key ideas as easy to understand as possible.\n* While the appendices are quite detailed and show important derivations, this results in the paper being extremely long (I understand, however, that this is the current trend in math-heavy ML research).\n\n[1] Morrill et al., *Neural Rough Differential Equations for Long Time Series*, ICML 2021 https://arxiv.org/pdf/2009.08295", "other_comments_or_suggestions": "* The footnotes in page 6 are quite long particular footnote 6. Perhaps 6 could be moved to the appendix (or part of it).\n* I think highlighting key points and contributions would make the paper easier to read.\n* Likewise, a further discussion of motivation for design decisions in constructing the estimator would improve readability.", "questions_for_authors": "1. Does $S(\\mathbb X)\\_{[0,T]}$ denote $S^k(\\mathbb X)_{[0,T]}$ as $k \\to \\infty$? This is my understanding but I don't see it stated clearly in the main text.\n2. Why is the equation on line 285 unnumbered? It seems quite important to the paper.\n3. What is the motivation for constructing the equation mentioned above, why do you choose this corrector? I have some thoughts, but this motivation should be included in the writing.\n\nN.B., I generally like this paper and enjoyed reading it; however, the lack of clarity and explanation of motivations prohibits me from giving this paper the highest rating.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lCT8cGhJMQ", "reviewer": "Reviewer_d8FL", "summary": "This paper establishes a rigorous framework for the expected signature of stochastic processes, proving consistency and asymptotic normality under a double asymptotic regime—where the discretization mesh ($\\pi$) tends to zero (in-fill asymptotics) and the number of observations $N$ increases (long-span asymptotics). It also introduces a martingale-based variance reduction technique that further improves finite-sample performance, with empirical validations.  \n\n## Update After Rebuttal\nI raised my score as a result of a discussion with the authors (see below).", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for the comments. We believe the reviewer well understood the main contributions of the paper, which they highlighted in \"Claims and Evidence\". \n\n**Methods And Evaluation Criteria**\n\nIn practical applications the (expected) signature transform may face the curse of dimensionality. Nevertheless estimation of the expected signature is still a relevant topic as, at least for low dimensional time-series, it provides a theoretically motivated domain-agnostic baseline model. Moreover, ongoing work is being carried out to \"a priori\" select a transformation of the $\\sum\\_{k=0}^p d^k$ truncated (expected) signature feature vector retaining its essential characteristics. For example in [2] the authors introduce a linear state-space transformation such that redundant information can be easily removed from the signature vector by (a further) truncation.\n\nUsually, a domain-specific architecture – for example, in the case of image/video data, a convolutional layer targeting the known topological structure of the input – will likely improve empirical model performance (cf. Remark E.1). In such cases, the (expected) signature can be used as a layer in a neural network, see [3]. When working with video streams, one might consider a sequential architecture with an initial embedding layer mapping the data to a lower dimensional space before computing the (expected) signature. The empirical applications discussed in this paper provide a set of simple benchmark tasks on which to evaluate the performance of the martingale correction (such simple models do not and should not be understood as trying to achieve SOTA results). The scope of this work was not to develop new expected signature-based architectures but rather, as well understood by the reviewer, to fill a gap in the theory and, under suitable conditions, provide an improved empirical estimator for the theoretical expected signature.\n\n[2] Bayer and Redmann, Dimension reduction for path signatures, preprint, https://arxiv.org/pdf/2412.14723\n\n[3] Bonnier et al., Deep Signature Transforms, NeurIPS 2019, https://arxiv.org/pdf/1905.08494\n\n**Other Strengths And Weaknesses**\n\nWe shall add intuitive definitions of technical terms in the appendix, the following is an extract from said glossary.\n\n*Informal glossary*\n\nThe $p$-variation of a path is a measure of its regularity. For the purpose of our discussion it suffices to note that paths that have finite $p$-variation for low $p$ are more regular. A bounded variation (BV) path is a path with finite 1-variation (also known as total variation), this regularity ensures there exists a well-defined notion of integral against this path (e.g. a piecewise linear paths or continuously differentiable path) and hence we can easily define its signature as in Equation (2). Many interesting stochastic processes (e.g. those driven by Brownian motion) have infinite 1-variation (i.e. are not BV) but have finite $p$-variation for all $p>2$ and hence defining their signature requires rough path theory.\n\nThe shuffle property of the signature is an algebraic property stating that the product of two signature terms is a linear combination of higher-order signature terms. More precisely, the product of the signature terms corresponding to words $I$ and $J$ is the sum of all signature terms indexed by words $K$ of length $|I|+|J|$ obtained by interleaving $I$ and $J$. In the context of the discussion on page 1 this means that all moments of the signature can be written as linear combinations of higher order expected signature terms.\n\nA word $I=(i\\_1, \\ldots, i\\_n)$ with $i\\_1, \\ldots, i\\_n\\in\\\\{1, \\ldots, d\\\\}$ is a multi-index used to denote an entry of the signature, i.e. a real-valued number. The length of the word, i.e. $|I| = n$, denotes to which level ($n$-dimensional tensor) of the signature the entry belongs. For example $S^I(\\mathbb{X})\\_{[0,T]}$ where $I=(1,2)$ denotes the $(1, 2)$-entry of the second level of the signature (a matrix).\n\n[...]\n\n**Questions For Authors**\n\nYes, we will contribute public code in the following two ways:\n- An independent code repository containing methods to estimate expected signatures using both the simple empirical estimator and the martingale corrected estimator (compatible with `numpy` and `torch` arrays, using `iisignature` and `signatory` for signature computations). This directory will also contain the scripts needed to reproduce the option-pricing experiment of Section 3.2.2.\n- Forks of the original repositories for the other two experiments adding the option to use the martingale corrected estimator, which can be used to reproduce the results of Sections 3.2.1 and 3.2.3.\n\nWe will refer to the public repositories in the final version of the paper. This information was redacted from the first submission in adherence with the conference's double blind review policy.", "claims_and_evidence": "This paper makes several notable contributions that extend and deepen their theoretical utility in machine learning.\n\n(i) Bridging Discrete and Continuous Frameworks:\nThe paper rigorously establishes conditions under which the empirical expected signature estimator—computed from discretely observed, piecewise linear approximations of a continuous-time process—converges (in a double-asymptotic sense) to the true expected signature of the underlying latent continuous-time stochastic process. The authors establish consistency and asymptotic normality (Thm. 2.8 / 2.10) under regularity, mixing, and moment conditions. Detailed proofs in the Appendix use techniques from Rough Path theory and stochastic analysis.\n\n(ii) Variance Reduction via Martingale Correction:\nRecognizing that the classical expected signature estimator can suffer from high variance in finite samples, the paper introduces a modified estimator that employs a control variate derived from the properties of the signature in the martingale setting. By leveraging the Stratonovich integral’s properties, this martingale correction leads to significantly lower mean squared error. The authors back up this theoretical finding with empirical experiments demonstrating improved performance in tasks such as time series classification and option pricing.", "methods_and_evaluation_criteria": "While the authors evaluate their approach on a classification task, there are practical concerns regarding scalability in modern machine learning settings due to the curse of dimensionality. Specifically, when a $d$-dimensional time series is mapped to its signature truncated at level p, the resulting feature vector has dimension $\\sum_{k=0}^{p} d^k$. This growth can become prohibitive for high-dimensional and multimodal data—for example, video streams where d might be on the order of $256 \\times 256$ or more.\n\nMoreover, for many classification tasks, task-specific representations (such as those learned by deep neural networks that selectively discard irrelevant information) may achieve superior performance compared to employing a full sufficient statistic like the expected signature. \n\nDespite these practical limitations, it is important to note that the primary focus of the paper is on theoretical contributions. Given this theoretical orientation, the experimental evaluations serve primarily to validate the underlying mathematical results rather than to claim state-of-the-art performance in classification. Consequently, while the potential practical challenges of scaling signature methods to very high-dimensional data are acknowledged, they should not be considered grounds for rejection of the paper.", "theoretical_claims": "I have reviewed the high-level theoretical claims and proof sketch but there remains a possibility that oversights exist.", "experimental_designs_or_analyses": "See the above “Methods and Evaluation Criteria” section.", "relation_to_broader_scientific_literature": "This paper bridges the gap between the theoretical framework of expected signatures—defined via nested iterated integrals in continuous time—and their practical estimation from discrete time series data. By rigorously establishing unbiased estimation and asymptotic properties for the expected signature, the authors connect foundational ideas from Rough Path theory with estimation techniques common in practice, thus broadening the future research of signature methods in machine learning.", "essential_references_not_discussed": "Essential references are adequately discussed.", "other_strengths_and_weaknesses": "The paper employs several technical terms—such as BV, the shuffle property, p-variation, and words $I$—without providing sufficient introductory guidance or intuitive explanations. While these concepts may be well-known to experts in stochastic processes and Rough Path theory, the ICML audience is broad and interdisciplinary. To enhance clarity and accessibility, the authors are encouraged to include a concise glossary or an appendix that offers reader-friendly definitions and intuitive explanations of these key terms.", "other_comments_or_suggestions": "The paper was an enjoyable read. Please note that my comments represent my initial impressions and may include misunderstandings. I welcome further discussion on these points and am open to revising my score once my questions and concerns are adequately addressed.", "questions_for_authors": "Do you plan to make your implementation code publicly available in the near future? Given the complexity of the theory, having accessible code would greatly help in ensuring reproducibility.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper establishes central limit theorems of empirical risk-based estimators of signatures for representing time-series for certain classes of generative processes that satisfy a martingale property, as well as corrections for processes that do not. They establish that this model applies to variants of Brownian motion and other stochastic differential equations that arise in options pricing, especially the rough Heston model. Numerous theoretical and experimental contributions support the key claims of this work, which the reviewers unanimously appreciated.
Decision: Accept (oral)