SigDiffusions: Score-Based Diffusion Models for Time Series via Log-Signature Embeddings
Authors: Barbora Barancikova
Zhuoyue Huang
Cristopher Salvi
Venue: ICLR 2025 Poster (2025)
Keywords: diffusion models
path signatures
time series
Primary area: learning on time series and dynamical systems
TL;DR: We introduce SigDiffusion, a novel Lie algebra preserving diffusion model operating on log-signature embeddings of a time series; we also provide new closed-form signature inversion formulae.

Abstract: Score-based diffusion models have recently emerged as state-of-the-art generative
models for a variety of data modalities. Nonetheless, it remains unclear how to
adapt these models to generate long multivariate time series. Viewing a time
series as the discretisation of an underlying continuous process, we introduce
SigDiffusion, a novel diffusion model operating on log-signature embeddings
of the data. The forward and backward processes gradually perturb and denoise
log-signatures while preserving their algebraic structure. To recover a signal from
its log-signature, we provide new closed-form inversion formulae expressing the
coefficients obtained by expanding the signal in a given basis (e.g. Fourier or
orthogonal polynomials) as explicit polynomial functions of the log-signature.
Finally, we show that combining SigDiffusions with these inversion formulae
results in high-quality long time series generation, competitive with the current
state-of-the-art on various datasets of synthetic and real-world examples.

Published as a conference paper at ICLR 2025
SIGDIFFUSION: SCORE-BASED DIFFUSION MODELS FOR TIME SERIES VIA LOG-SIGNATURE EMBEDDINGS
Barbora Barancikova Department of Computing Imperial College London
Zhuoyue Huang
Cristopher Salvi
Department of Mathematics Imperial College London
Department of Mathematics Imperial College London
ABSTRACT
Score-based diffusion models have recently emerged as state-of-the-art generative models for a variety of data modalities. Nonetheless, it remains unclear how to adapt these models to generate long multivariate time series. Viewing time series as the discretisation of an underlying continuous process, we introduce Si gD fffffuuii ion, novel diffusion model operating log-signature embeddings of the data. The forward and backward processes gradually perturb and denoise log-signatures while preserving their algebraic structure. To recover signal from its log-signature, we provide new closed-form inversion formulae expressing the coefficients obtained by expanding the signal in a given basis (e.g. Fourier orthogonal polynomials) as explicit polynomial functions of the signsitte Finally, we show that combining SigDi croofffsiinn ions with these inversion formulae results in high-quality long time series generation, competitive with the current state-of-the-- on various datasets of synthetic and real-world examples.
INTRODUCTION
Time series generation has gained significant attention in recent years due to the growing demand for high-quality data augmentation in fields such as healthcare (Trottet et al.] (023) and finance (Hwang et al. 2023). Since sampling rates are often arbitrary and non-uniform, it is natural to assume that the data collected from measurements of some underlying physical system evolving continuous time. This perspective calls for modelling tools capable of processing temporal signals as continuous functions time. We will often refer to such functions as paths.
The idea of representing path via its iterated integrals has been the object of numerous mathematical studies, including geometry (Chen 1957 1958) control theory (Fliess et al. 1983), and stochastic analysis (Lyons| 1998). The collection of these iterated integrals known the signature of path. Thanks to its rich algebraic and analytic properties summarised in Scction 2 the signature is an efficienttto-comppute universal feature map for time series evolving in continuous time. As result, signature methods have recently become mainstream in many areas of machine learning dealing with irregular time series, from deep learning (Kidger al. 2019 Morrill
[021 Cirone et al. 2023, (2024) to kernel methods (Salvi et al. [02 Lemercier et al. |202 lb: Issa et al. (024)) with applications quantitative finance (Arribas al., 2020 (Salvi al. [022b]] Horvath et al. 2023 Pannier & Salvi 2024). cybersecurity (Cochrane et al.) [2021), weather forecasting (Lemercier et al. 2021a), and causal inference (Manten et al.) (024). For a concise summary of this topic, we refer the interested reader to Fermanian et al. (2023b).
Score-based diffusion models have emerged as a powerful framework for modelling complex distri- butions in computer vision, audio, and text (Song et al. 2020; Bilos et al., 2023 Popov et al.) 2021] Cai et al. 2020 Voleti et al.) 2022). These models perturb the observed data distribution through forward diffusion process, adding noise until sasy-to-sample base distribution is reached. A new sample from the learned data distribution is then generated by reversing the noising process using a learned score function of the data conditional on the noise level. Despite recent efforts summarised in Section it remains unclear how to adapt score-based diffusion models for generating long signals in continuous time.

---

Published as a conference paper at ICLR 2025
Contributions In this paper, make use of the log- sinnsignatee, compressed version of the sig- nature, as a parameter- free Lie algebra embedding for time series. In Section 2.3 we introduce SigDiffure ffusion, new diffusion model that gradually perturbs and denoises lig-signatures pre- serving their algebraic structure. To recover a path from its log-signature embedding, we provide novel closed-form inversion formulae in Section 3 Notably, we prove that the coefficients of the expansion of path in a given basis, such as Fourier or orthogonal polynomials, can be expressed as explicit polynomial functions on the log-signature. Our results provide major improvement over existing signature inversion algorithms (see Section 5.1) which suffer from scalability issues and are only effective on simple examples of short piecewise- paths. Section [ we demonstrate how the combination of Sigg gD i f fusi ons with our inversion formulae provides time series generative approach, competitive with state-of-the-art diffusion models for temporal data on various datasets of synthetic and real-world examples.
GENERATING LOG-SIGNATURE WITH SCORE-BASED DIFFUSION MODELS
We begin this section by recalling the relevant background material before introducing SigDi f ffusii i on model. We will limit ourselves to reporting only the key properties of signa- tures and the notation necessary for the inversion formulae in Section 3 Additional examples of signature computations can be found in Appendix A.3
2.1 THE (LOG)SIGNATURE
Let x : [0. - Rd be a smooth d-dimensional time series defined on a time interval [0, 1]. We will equivalently refer this object path. The step-n signature SS"(x) of is defined as the following collection of iterated integrals
= (1, S,(x))
where
Sx(x)= = I JostIe..<t451
dr,, 8 ... @ dap, for S S
and denotes the tensor product. Intuitively, increasing dimension, where the value of the m-th tensor at the index in. i.. im represents a measure of "interaction" between the i1, ig...... -th channels of This makes the signature transform particularly effective at capturing information about the shape of multivariate time series.
view the signature
set of tensors of
Example 2.1. Assume = 2, and denote the two channels of x as x = (x² x²²)) Then (1 (2) tensors with shape [2] and [2,2] respectively
Si J drn ['drm
(Sg dot dx?,
S2(1) = S.' 10 dx,, @ da
(Sa p'a dat dxt lg
Sa Sez dey dx
So So dxt, dx?,
So So dot do?
Denoting the standard basis of Id as c1, C2 ... ed we define basis of the space of k-dimensional tensors as
Giri2...ie = Ci2 @ Ci2 @ @ City for 1 i1...... da S d and < S n.
We refer to these basis elements as words. In Section we will make use of the notation (Ei, (*=(())) E E to extract the (ii..., 8thth element of the k-th signature tensor Sz)z).
Words can be manipulated by two key operations: the shuffle product wu and right half-shuffle product is defined as the sum over the (r+*) ways of interleaving the two words. For a formal definition, we refer readers to Reutenauer (2003 Section 1.4). Much of the internal structure of the signature is characterized by the shuffle identity (see Lemma which uses the shuffle and half-shuffte products to describe the relationship between elements of higher and lower-order signature tensors. This identity is crucial
>. The shuffle product of two words of length and (with +
2

---

Published as a conference paper at ICLR 2025
in our derivation of the inversion formulae in Appendix C A rigorous algebraic explanation of these concepts provided in Appendix(A.]
Moreover, it turns that the space of signatures has the structure of step-n free nilpotent Lie group G" (IR)). We denote by -d (IR) the unique Lie algebra associated with 9 (IR",, and we call its elements log-signatures. 9 (Rd) ) is the image of the 2 (TR.) under the exponential map
(²() e0 (R*))
where, in the case of signatures, exp denotes the tensor exponential defined in Appendix A.2 Furthermore, one can use the tensor logarithm (see Equation (3) to convert log-signatures to signatures. These two operations mutually inverse.
We note that the Lie algebra L (Rd) ) a vector space of dimension 3(d, n) with
((d,n) Σ
d'
14
where je is the Möbius function (Reutenauer 2003). Crucially, the Lie algebra is isomorphic to the Euclidean space (dd,)) which motivates the diffusion model architecture Section 2.3
2.2 SIGNATURE AS A TIME SERIES EMBEDDING
The (log)signature exhibits additional properties making it an especially interesting object in the context of generative modelling for sequential data. In this section we summarise such properties without providing technical details, as these have been discussed length in various texts in the literature. For a thorough review, we refer the interested reader to Cass & Salvi (2024) Chapter I).
At first glance, computing the (log)signature seems intractable for general time series. However, an elegant and efficient computation is possible due to Chen's 's relation
Lemma 2.0.1 (Chen's relation). For any two smooth paths [0, 1] + Rd the following holds
SS" (x v) = (z"(()).5 (5)())
where * denotes puth-concatenatiion and is the signature tensor product defined Equation
II|
Combining Chen's relation with the fact that the signature of linear path is simply the tensor exponential of its increment (see Example A.2) provides us with an efficient algorithm for computing signatures of piecewise linear paths. This approach eliminates the need to calculate integrals when computing signature embeddings. See Appendix A.3 for simple examples of computations.
Furthermore, the (log)signature is invariant under reparameterissations. This property essentially allows the signature transform to act as filter that removes an infinite dimensional group of symmetries given by time reparametcrisations. The action of reparameterising a path can be thought of as sampling its observations at different frequency, resulting in robustness to irregular sampling. Another important property of the signature the factorial decay the magnitude of its coefficients (Cass & Salvi 2024| Proposition 1.2.3). This fast decay implies that truncating the signature at sufficiently high level retains the bulk of the critical information about the underlying path.
The signature is unique for certain classes of paths, ensuring one-to-one identifiability with the underlying path. An example of such classes given by paths which share an identical, strictly monotone coordinate and are started the same origin. More general examples are discussed in Cass & Salvi (204 Section 4.1). This property is important if one is interested, as we are, in recovering the path from signature. Yet, providing viable algorithm for inverting the signature has, until now, been challenging; valid but non non-scalable solutions have been proposed only for special classes of piecewise linear paths (Chang Lyons, 2019: Fermanian al. 2023a Kidger et al. 2019). In Section 3 we provide new closed-form inversion formulae that address this limitation.
2.3 DIFFUSION MODELS ON LOG-SIGNATURE EMBEDDINGS
As described in Section 2.1
any element of (R) corresponds to the step-n signature of smooth
path. Taking the tensor logarithm in Equation 113 then implies that arbitrary element of 2 ("(R))
3

---

Published as a conference paper at ICLR 2025
Closed-form operations
Learning-baal based operations
Calculate arp-nniinnauuresol of time ser les dataset
2) Convert to log-signatures logerinn tgp. 1131
3) Train diffusion model on log-signatures
Signedrre Pap.
Forward deiuson Egg.
6) Invert samples back time
5) C Conver samples to step- skynatures
Sample the diffusion model
Sigratire inveriinn fonrilin Section
Torsor roporettialn in (13)
Probebiitt flw Ole Eip.
Figure I: SigDi f ffusions ons pipeline. The signatures of time series dataset are points distributed in non Euclidean space (Lie group). Converting to log-signatures maps them to a Euclidean space (Lie algebra) where standard diffusion models operate. Calculating the log-signature embedding and its inverse (blue box) are fully deterministic operations, which greatly simplifies the learning task. The log-sinnatures serve as imputs score-based diffusion model (orange box). Step 6 enabled by our newly derived closed-form inversion formulac.
corresponds to the step-n log-signature of smooth path. Because the Lie algebra 4 (TR))) is a linear space, adding two log-signatures yields another log-sinnature. Furthermore, the dimensionality d++1_1 making the log-signature more compact repre- 3(d, n)) of (RR)) strictly smaller than d-1 sentation of path than the signature while retaining the same information. We can leverage these two properties to run score-based diffusion models on L" (IR"), followed by an explicit signature inversion, which we discuss in the next section. Figure presennt an overview of the proposed idea, which we demonstrate experimentally in Section 5 to be an efficient and high-quality method for generating multivariate time series.
We briefly recall that score- score-based diffusion models progressively corrupt data with noise until it reaches a tractable form, then learn to reverse this process to generate new samples from the underlying data distribution p(x). A neural network is used to estimate the gradient of the log probability density, known as the score (Song & Ermon 2019), se (t, x) z Vx log pr (x) at each injected noise level t. We model the forward data perturbation process with stochastic differential equation (SDE) of the form
dx 3t)xxdt + v B(t)dw, x(0) data
(4)
where (t) is linear over t [0, 1]. The corresponding reverse diffusion process follows a reverse-
time SDE (Anderson) 1982) dx = (t)x Vx lo di + //(())dw, where flows backwards from 7 to and w is Brownian motion with negative time step dt. The initial point x(T) is sampled from a standard Gaussian distribution. Following previous work (Ho et al.. 2020 Bilos
al. 2023) (uuan & Qiao 2024), we use a simple transformer architecture with sinusoidal positional embeddings of / to model 8g.
SIGNATURE INVERSION
In this section, we provide explicit signature inversion formulae. We do so by expressing the coefficients of the expansion of a path in the Fourier or orthogonal polynomial bases as a linear function on the signature. The necessary background material on orthogonal polynomials and Fourier series can be found in Appendix B
Throughout this section, [0, 1] + R will denote 1 1-dimensional smooth path. The results in the sequel can be naturally extended to multidimensional paths by applying the same procedure channel by channel. Depending on the type of basis we chose to represent the path, we will often need to reparameterise the path from the interval [0. 1) to a specified time interval (a, b) and augment it with
time as well as with additional channels c², c2 :.... C [a, b) - IR mailor-made for the specific type of inversion. We denote the augmented path by z(1) these transformations are fully deterministic and do not affect the complexity of the generation task outlined in Section 2.3] Furthermore,
(t, c ((), .... c² x(t)) Rr+, Note that
will use the shorthand notation ((t) for the step-n signature SS" (.) throughout the section, and assume that the truncation level n is always high enough to retrieve the desired number of basis coefficients. All proofs can be found in Appendix
4

---

Published as a conference paper at ICLR 2025
3.1 INVERSION VIA FOURIER COEFFICIENTS
In this section, we derive closed-form expressions for retrieving the first n Fourier coefficients of path from its signature. First, recall that the Fourier series of 27-periooic path æ(() up order v E is F (1) = 40 + > (ag (os( + bn sin(n!)) where do anz in are defined as
1
2x
00
J
xt))tt
(5)
2T
2m
Og
xxt) cos(nt)dt,
(6)
Jo
1
2x xxt) sin(nt)dt.
b
(7)
Theorem 3.1. Let æ [0, R be a periodic smooth path such that z(0) and consider the angmentation E(t) = (t, cos() - 1, x(t)) E IR. Then the following relations hold
1
00
(e + c1 S())).
2x
cos 2
nn - ((e) ل we w eg > e1. S(z)),
(8)
q=0
k
bn
sin
(n
((e) we - ل e3 > E1, S(t))
k=0
3.2 INVERSION VIA ORTHOGONAL POLYNOMIALS
To accommodate path generation use cases for which a non-Fourier representation more suitable, next derive formulae for inverting the signature using expansions of the path in orthogonal polynomial bases. Recall that any orthogonal polynomial family (Pn)neN with a weight function w: (a, b] + R satisfies three-term recurrence relation
(t) (A,t + B.)p (t),
n 2,
with (t) = and (t) = + B₁. Also, note that any smooth (or at least square-integrable) path at)) with xa) n-th orthogonal polynomial coefficient
be approximated arbitrarily well xx() En anPr (1) where On, the
Ohp (pr: Pr)) S.
(10)
and (---) denotes the inner product (f.g) S." f(t) We include several examples of such polynomial families in Appendix B
Theorem 3.2. Let x (a. b) + R.. smooth path such that s<a) O. Consider the augmentation
a(t) = (t, (t)x((()) E TR?, where w(t) corresponds to the weight function of a system of orthogonal polynomials (Pa)neR and is well defined on the closed and compact interval [a, b). Then there
exists linear combination En of words such that the th coefficient in Equation (10) satisfies
an = -4n, ((i)). Furthermore, the sequence (En)NEN satisfies the following recurrence relation
ln = An (pn-1,Pm-1). E1 ln-1 + (Ana + Br) (Pn-1.Pn-1) l-- + Cn (pn,Pa) (po,Pn)
(Pn2), 1n- Ln-2
(pn,Pn)
with
An (po. Po)
A (P1.P1)
(e12 6211 + Aia + Bi
6o
621 and
(p1, P1) 621-
Remark. The results in Theorem 3.2 require signatures of (t, w(t)=(t)). However, sometimes one may only have signatures of ã (t, x()).. In Appendix C.2 approximating the weight function as Taylor series.
we propose an alternative method by
5

---

d-1 giving the inversion a Published as a conference paper at ICLR 2025
3.3 INVERSION TIME COMPLEXITY
The inversion formulae all boil down to evaluating specific linear combinations of signature terms. Evaluating a linear functional has time complexity linear in the size of the signature. Since this evaluation is repeated for each of the n. recovered basis coefficients, the total number of operations nu, where level n + 2. For a d-dimensional path, length of a step-N signature dx+1 time complexity of 22
the number basis coefficients and m the length of the signature truncated at
4 RELATED WORK
Multivariate time series generation Generating multivariate time series has been an active area of research in the past several years, predominantly relying on generative adversarial networks (GANs) (Goodfellow et al. 2014). Simple recurrent neural networks acting generators and discriminators (Mogren 2016] Esteban et al., 2017) later evolved into encoder-decodes architectures where the
adversarial generation occurs in learned latent space (Yoon et al. [019 Pei et al. [202] Jeon et al., 2022). To generate time series in continuous time, architectures based on neural differential equations in the latent space (Rubanova al., 2019 Yildiz et al., 2019) have emerged as generalisations of RNNs. More flexible alternatives have since been proposed in the forms of neural controlled
differential equations (Kidger al. 2020) and state space ODEs (Zhou al. 2023).
Diffusion models for time series generation There are number of denoising probabilistic diffusion models (DDPMs) currently at the forefront of time series synthesis, such as DiffTime (Ho) 2020), which reformulates the constrained time series generation problem in terms of conditional denoising diffusion (Tashiro et al. 202T). Most recently, Diffusion-TS (Yuan & Qiao 2024 has demonstrated superior performance on benchmark datasets and long time series by disentangling temporal features via a Fourier-basse training objective. To learn long-range dependencies, both the aforementioned methods use transformer-basee (Vaswani al. 2017) diffusion functions. Many recent efforts attempt to generalise score-based diffusion to infiniteddimensional function spaces (Kerrigan al. 2022. Dutordoir al. 2023 Phillips et al. 2022 Lim et al. 2023). However, unlike their discrete- time counterparts, they have not yet been benchmarked on a variety of real-worke temporal datasets. An exception this a diffusion framework proposed by Bilos et al. (2023). which synthesises continuous time series by replacing the time- time-independent noise corruption with samples from a Gaussian process, forcing the diffusion remain in the space of continuous functions. Another promising approach for training diffusion models in function space is the Denoising Diffusion Operators (DDOs) method (Lim et al., 2023). While has not been previously applied to time series, our evaluation in Section 5 demonstrates its strong performance in this context. Additionally, there is a growing body of recent literature focusing on application-specin
time series generation via diffusion models, such speech enhancement (Lay et al. 2023 Lemercier et al. 2023). soft sensing Dai et al. 2023). and battery charging behaviour (Li et al 2024).
Signature inversion The uniqueness property of signatures mentioned in Section 2.2 has motivated several previous attempts to answer the question of inverting the signature transform, mostly as theoretical contributions focusing on one specific class of paths (Lyons & Xu 2017] Chang et al.) 2016, Lyons & Xu, 2018). The only fast and scalable signature inversion strategy to date is the Insertion method (Chang Lyons) 2019), which provides an algorithm and theoretical error bounds for inverting piecewise linear paths. It was recently optimised (Fermanian et al. (023a) and released as a part of the Signatory (Kidger et al. 2019) package. There are also examples of inversion via deep learning (Kidger et al.) (019)) and evolutionary algorithms (Buchler et al., 2020), but they provide no convergence guarantees and become largely inefficient when deployed on woaldworld time series.
5 EXPERIMENTS
In Section 5.1 we demonstrate that the newly proposed signature inversion method achieves more ac- curate reconstructions than the previous Insertion (Chang & Lyons 2019 Fermanian et al.) (023a) and Optimisation (Kidger et al., 2019) methods. We also analyse the inversion quality and time complexity across different families of orthogonal polynomials. In Section 5.2 we show that (log)signatures,

---

Published as a conference paper at ICLR 2025
5.0
25
0.0
2.5
-5.0
-10.0
mathcd
Legendre polynomials
Original
Figure 2: Comparison of different inversion methods.
combined with our closed-form inversion, provide an exceptionally effective embedding for time series diffusion models. Section 5.2.1 visualises the trade-off between the precision of time series representation and model complexity introduced by the choice of the signature truncation level. In Section [.2.2] we present experiments demonstrating that generating step-4 log-signatures via the pipeline outlined in Figure oottperfros other recent time series diffusion models across several standard metrics.
5.1 INVERSION EVALUATION
We perform experiments to evaluate the proposed analytical signature inversion formulac derived in Section via several families of orthogonal bases. Using example paths given by sums of random sine waves with injected Gaussian noise, we reconstruct the original paths from their step-12 12 signatures. Figure compares inversion of these paths via Legendre and Fourier coefficients the Insertion method (Chang & Lyons 2019 Fermanian et al., 2023a) and Optimisation method (Kidger et al. 2019), showcasing the improvement in inversion quality provided by our explicit inversion formulae. Figure 3 presents the time consumption against the L2 error as the degree of polynomials increases. Notably, the factor holding the most influence over the reconstruction quality is the truncation level of
50
L, emrr
10"
2.2 error
10
Hemm 0)
Jacobi1
10'
10
Hemmite 0.05
12
order
time
Figure 3: L2 error of signature inversion via orthogonal polynomials with respect to the polynomial order N and time. Error and time are calculated by an average over 15 paths with 200 sample points.
the signature, as it bounds the order of polynomials a discussion about inversion quality in Appendix [ Namely, Figures and 12]shww more examples of inverted signatures using different types of paths and polynomial bases.
can retrieve. We refer the interested reader to
5.2 GENERATING LONG TIME SERIES
In this section, we introduce the by the following strategy, also illustrated in Figure I
pipeline for generating multivariate time series
Choose an orthogonal basis and order sufficient to represent the time series with enough detail to retain its meaningful components while smoothing out unnecessary noise.
2. Compute the lig-signatures of the time series, augmented as described in Section and truncated at level N
7

---

Published as a conference paper at ICLR 2025
3. Train and sample score-based diffusion model to generate the log-signatures.
4. Invert the synthetic log-signatures back time series using our formulae from Section 3
5.2.1 TIME SERIES REPRESENTATION AND MODEL CAPACITY
Here, we highlight the trade-off between model capacity and the faithfulness the time series representation given by its truncated Fourier series. Since the signature inversion formulae are exact, the inversion quality depends only on how well the underlying signal is approximated by the retrieved Fourier basis coefficients. When the primary goal is to model the overall shape of multivariate time series, it is sufficient to truncate the signature, and thus the Fourier series, at a low level. Low truncation levels capture most of this information, smooth out high-order noise, and simplify the generation task. In contrast, modelling signals with high-frequenge components requires higher signature truncation levels, raising the complexity of the diffusion task. As the signature size grows exponentially with the truncation level, generating highly oscillatory time series with high fidelity becomes increasingly constrained by the model's capacity. To illustrate this, we show the generated samples from Si SigDi E fus ions trained on a noisy Lotka-Volters system with different log-signature truncation levels in Figure + At each truncation level, log-signatures become progressively more difficult to generate accurately.
Figure 4: Time series representation and model capacity trade-off. Left to right: real sample from noisy Lotka-Voltern system, sample generated by SigDt ffusion with signature truncation level 6,
5.2.2 GENERATING STEP-4 -IGSSIGNATS
We now demonstrate the exceptional ability of signatures to capture the shape and cross-channel dependencies of time series at low truncation levels. We show that SigDi f fus ons applied to step-4 log-si signttgats combined with Fourier inversion, outperform state-of-the-art diffusion-based models on the task of generating 1000-point-long time series.
Datasets We perform experiments on five different time series datasets: Sines YYoon et al. 2019)
dataset of 5-dimensional sine curves with randomly sampled frequency and phase, Predator-
prey (Bilos et al. 2023) a two-dimensional continuous system evolving according to a set of ODEs, Household Electric Power Consumption (HEPC) (UCI Machine Learning Repository 2024) the univariate voltage feature from a real-world dataset household power consumption, Exchange Rates (Lai al. 2018 Lai
2017)
real-world dataset containing daily exchange rates
8 currencies, and Weather (Kllle 2024) a real-world dataset reporting weather measurements.
Metrics We use the metrics established in Yoon et al. (201)). The Discriminative Score reports the out-of-sample accuracy of a recurrent neural network (RNN) classifier trained distinguish between real and generated time series. To improve readability, we report values offset by 0.5, so that in the ideal case, where real and generated samples are indistinguishable by the classifier. this metric will approach 0. The Predictive Score measures the loss of next-point predictor RNN trained exclusively on synthetic data, with the loss evaluated on the real test data set. the case of univariate time series, the RNN predicts the value of the time series 20 time points ahead, instead of predicting the value an unseen channel. We also run the Kolmogorov-Smirna (KS) test on marginal distributions of random batches of ground truth and generated paths. We repeat this test 1000 times with a batch size of 64 and report the mean KS score with the mean Type error for 5% significance threshold. Since the cross-channel terms of the log signature are not necessary for the inversion methods, we generate a concatenated vector the log-signatures of each separate dimension plus the augmentation described in Section
8

---

Published as a conference paper at ICLR 2025
Table 1: Results for generating time series of length 1000.
Dataset
Model
Discriminative Score
Predictive Score
SigDiffusion (ours)
0.100+:0026
0.191 +.044
DDO (7 =1
0.040+-0025
0.187+-0003
Sines
Diffusion-TS
0.291+-070
0.365++0006
CSPD-GP (RNN)
0,468+.030
0.244#.012
CSPD-GP (Transformer)
0,4931.003
0.3901.023
SigDiffusion (ours)
0.1841.058
0.056:1.000
DDO (1 10) Diffusion-TS
0.211#.080
0.056#.000
Predator-prey
0,500#0000
0.482#
CSPD-GP (RNN)
0.168+.024
0.057+.0000
CSPD-GP (Transformer)
0.498+-003
0.919+-002
SigDiffusion (ours)
0.070+.032
0.050+.012
DDO (? = 1) Diffusion-TS
0.081+.019
0.044+.001
HEPC
0.438110057
0.0661:0222
CSPD-GP (RNN)
0.415::.045
0.108:1.002
CSPD-GP (Transformer)
0.500++.000
0.551#.028
SigDiffusion (ours)
0.2781.062
0.0571.001
DDO (7 = 1)
0.326 102
0.094
Exchange Rates
Diffusion-TS
0.4
0.120
CSPD-GP (RNN)
0.500+0001
0.273+.100
CSPD-GP (Transformer)
0.500++000
0.432+
SigDiffusion (ours)
0.350+t.080
0.168
DDO (7 10) Diffusion-TS
1.3
0.307
Weather
0,4981.0033
0,4381.0035
CSPD-GP (RNN)
0.000
0.505
CSPD-GP (Transformer)
0.500#
0.490=
Benchmarks Table lists the time series generation performance metrics compared with four recent diffusion model architectures specifically designed to handle long or continuous-time paths:
This model introduces novel Fourier-based training objec- tive to disentangle temporal features of different seasonalities. This interpretable decomposition strategy makes the model particularly robust to varying time series lengths, demonstrating strong performance relative to other benchmarks as the training time series become longer.
Diffusion-TS (Yuan & Qiao. 2024)
CSPD-GP (Bilos et al., 2023) This approach replaces the Lime-independens noise corruption mechanism with samples drawn from a Gaussian process, effectively modelling diffusion on time series process occurring within the space of continuous functions. CSPD-GP (RNN) and CSPD-GP (Transformer) refer to score-ber score-based diffusion models with the score function either being RNN or transformer.
Denoising Diffusion Operators (DDO) Lim et al. 2023) These models generalise diffusion models to function spaces with a Hilbert space-valued Gaussian process to perturb the input data. Additionally, they use neural operators for the score function, ensuring consistency with the underlying function space formulation. DDO's s kernel smoothness hyperparameter y is tuned and reported for each dataset.
We compute the metrics using 1000 samples from each model. Table shows the model sizes and training times, demonstrating that S SigDi fusion outperforms the other models while also having the most efficient urchitecture. Table in the Appendix evaluates the time series marginals using the KS test. More details about the experimental setup be found in Appendix F

---

Published as a conference paper at ICLR 2025
Table 2: Comparison of model sizes.
Dataset
Model
Parameters
Training Time
Sampling Time
SigDiffusion (ours)
229K
8 min
11
DDO (1=
4.12M
3.6
42 min
Sines
Diffusion-TS
4.18M
57 min
15 min
CSPD-GP (RNN)
759K
9 min
min
CSPD-GP (Transformer)
973K
15 min
5 min
SigDiffusion (ours)
211K
8 min
12 sec
DDO (y 10) Diffusion-TS
4.12M
3.5 h
42 min
Predator-pir prey
4.17M
55 min
14 min
CSPD-GP (RNN)
758K
8 min
1 min
CSPD-GP (Transformer)
972K
16 min
5 min
SigDiffusion (ours)
206K
8 min
12 sec
DDO (y 1) Diffusion-TS
4.12M
2.6
42 min
HEPC
4.17M
50 min
15 min
CSPD-GP (RNN)
758K
min
min
CSPD-GP (Transformer)
972K
9 min
5 min
SigDiffusion (ours)
247 K
9 min
11 sec
DDO (y 1) Diffusion-TS
4.12M
3.6 h
42 min
Exchange rates
4.29 M
1.2
20 min
CSPD-GP (RNN)
760 K
11 min
1 min
CSPD-GP (Transformer)
974 K
15 min
min
SigDiffusion (ours)
282K
8 min
12 sec
DDO (7 10) Diffusion-TS
4.12M
3.6 h
42 min
Weather
4.3 M
1,4 h
20 min
CSPD-GP (RNN)
763 K
14 min
min
CSPD-GP (Transformer)
975 K
17 min
6 min
CONCLUSION AND FUTURE WORK
In this paper, we introduced SigDiffusiin new diffusion model that gradually perturbs and denoises log-signature embeddings long time series, preserving their Lie algebraic structure. To recover the path from its log- signaauue, proved that the coefficients of the expansion of a path in a given basis, such as Fourier or orthogonal polynomials, can be expressed as explicit linear functionals the signature, equivalently polynomial functions the log-signature. These results provide explicit signature inversion formulae, representing a major improvement over signature inversion algorithms previously proposed in the literature. Finally, we demonstrated how combining SigDi f fusi on with these inversion formulae provides a powerful generative approach for time series that is competitive with state-of-the-art diffusion models for temporal data.
As this the first work on diffusion models for time series using signature embeddings, there are still many research directions explore. To mitigate the rapid growth the number of required signature features for high-fre frequency signals described in Section future work could explore alternative embeddings to the signature. For example, other types of path developments derived from rough path theory, which embed temporal signals into (compact) Lie groups, such as those proposed by Cass Turner (2024), may offer more parsimonious representation. These alternatives retain many of the desirable properties of signatures, including the availability of a flat-space Lie algebra. However, it remains unclear how an inversion mechanism would work in these cases. Finally, it would be interesting to understand how discrete-tine signatures (Diehl et al. 2023) could be leveraged to encode discrete sequences on Lie groups and potentially perform diffusion based generative modelling for text.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
Barbora Barancikova is supported by UK Research and Innovation [UKRI Centre for Doctoral Training in AI for Healthcare grant number EP/S023283/IJ.
REFERENCES
Brian DO Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12(3)311-336, 1982.
Imanol Perez Arribas, Cristopher Salvi, and Lukasz Szpruch. Sig-sdes model for quantitative finance. In Proceedings of the First ACM International Conference on Al in Finance, pp. 1-8, 2020.
Kendall. Atkinson. Theoretical Numerical Analysis A Functional Analysis Framework. Texts in Applied Mathematics, 39. Springer New York, New York, NY, 3rd 2009. edition, 2009. ISBN 1-282-33318--
Marin Biloš, Kashif Rasul, Anderson Schneider, Yuriy Nevmyvaka, and Stephan Günnemann. Modeling temporal data as continuous functions with stochastic process diffusion. In International Conference Machine Learning, pp. 2452-242242 2470. PMLR, 2023.
Hans Buehler, Blanka Horvath, Terry Lyons, Imanol Perez Arribas, and Ben Wood. A data-driven market simulator for small data environments. arXiv preprint arXiv:2006. 14498, 2020.
Ruojin Cai, Guandao Yang. Hadar Averbuch-Elor Zekun Hao, Serge Belongie, Noah Snavely, and Bharath Hariharan. Learning gradient fields for shape generation. In Computer Visi 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part III 16, pp. 364 381. Springer, 2020.
Thomas Cass and Cristopher Salvi. Lecture notes rough paths and applications machine learning. arXiv preprint 24X1v:2404.06583, 2024.
Thomas Cass and William Furrner. Free probability, path developments and signature kernels as universal scaling limits. arXiv preprint arXiv:2402. /2311, 2024.
Jiawei Chang and Terry Lyons. Insertion algorithm for inverting the signature of a path. arXiv preprint arXiv: 08423, 2019.
Jiawei Chang, Nick Duffield, Hao Ni, and Weijun Xu. Signature inversion for monotone paths, 2016.
Kuo-Tsai Chen. Integration of paths, geometric invariants and generalized haker-fausdorff formula. Annals of Mathematics, 66( 1957.
Kuo-Tsai Chen. Integration of paths a faithful representation of paths by noncommutative formal power series. Transactions of the American Mathematical Society, 9922)3995-407, 1958.
WL Chow. On system of linear partial differential equations of the first order. Mathematische Annalen, 117(1)98-1055, 1939.
Nicola Muca Cirone, Maud Lemercier, and Cristopher Salvi. Neural signature kernels as infinite- width-depth-limits of controlled resnets. In International Conference Machine Learning, pp. 25358 25425. PMLR, 2023.
Nicola Muca Cirone, Antonio Orvieto, Benjamin Walker, Cristopher Salvi, and Terry Lyons. The- oretical foundations of deep selective state-space models. arXiv preprint arXiv:2402. 19047, 2024.
Thomas Cochrane, Peter Foster, Varun Chhabra, Maud Lemercier, Terry Lyons, and Cristopher Salvi. Sk-tree: a systematic malware detection algorithm on streaming trees via the signature kernel. In 2021 IEEE international conference cyber security and resilience (CSR), pp. 35-40. IEEE, 2021.
Andrea Coletta, Sriram Gopalakrishnan, Daniel Borrajo, and Svitlana Vyetrenko. On the constrained time-series generation problem. Advances in Neural Information Processing Systems, 36, 2024.
11

---

Published as a conference paper at ICLR 2025
Yun Dai, Chao Yang, Kaixin Liu, Angpeng Liu, and Liu. Timeddpm: Time series augmentation strategy for industrial soft sensing. JEEE Sensors Journal, 2023.
Joscha Dichl, Kurusch Ebrahimi-Fard, and Nikolas Tapia. Generalized iterated-sums signatures.
Journal of Algebra, 632:801-824, 2023.
Vincent Dutordoir, Alan Saul, Zoubin Ghahramani, and Fergus Simpson. Neural diffusion processes. In International Conference on Machine Learning, pp. 8990-9012. PMLR, 2023.
Cristóbal Esteban, Stephanie L Hyland, and Gunnar Rätsch. Real-valued (medical) time series generation with recurrent conditional gans. arXiv preprint TxXiv:/70.00263:: 2017.
Adeline Fermanian, Jiawei Chang, Terry Lyons, and Gérard Biau. The insertion method to invert the signature of path. arXiv preprint 231.:22004.0862, 2023a.
Adeline Fermanian, Terry Lyons, James Morrill, and Cristopher Salvi. New directions in the applications of rough path theory. IEEE BITS the Information Theory Magazine, 2023b.
Michel Fliess, Moustanir Lamnabhi, and Françoise Lamnabhi-Lagariin nonlinear functional expansions. IEEE transactions on circuits and systems, 30(8):554 570, 1983.
An algebraic approach to
Peter K Friz and Nicolas B Victoir. Multidimensional stochastic processes as rough paths: theory and applications, volume 120. Cambridge University Press, 2010.
Ian Goodfellow, Jean Pouget-Abadic, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Blanka Horvath, Maud Lemercier, Chong Liu, Terry Lyons, and Cristopher Salvi. Optimal stopping via distribution regression: higher rank signature approach. arXiv preprint arXiv:2304.01479, 2023.
Yechan Hwang, Jinsu Lim, Young-Jun Lee, and Ho-Jin Choi. Augmentation for context in financial numerical reasoning over textual and tabular data with large-scale language model. In NeurIPS 2023 Second Table Representation Learning Workshop. 2023.
Mourad Ismail. Classical and quantum orthogonal polynomials in one variable / Encyclopedia of mathematics and its applications v. 98. Cambridge University Press, Cambridge, 2005. ISBN 9780521782012.
Zacharia Issa, Blanka Horvath, Maud Lemercier, and Cristopher Salvi. Non-adversarial training of neural sdes with signature kernel scores. Advances in Neural Information Processing Systems, 36, 2024.
Jinsung Jeon, Jeonghak Kim, Haryong Song, Seunghyeon Cho, and Noseong Park. Gt-gan: General purpose time series synthesis with generative adversarial networks. Advances in Neural Information Processing Systems, 35:36999-3700 -70100, 2022.
Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and François Fleuret. Transformers are rnns: Fast autoregressive transformers with linear attention. In International conference machine learning, pp. 5156-5111 5165. PMLR, 2020.
Gavin Kerrigan, Justin Ley, and Padhraic Smyth. Diffusion generative models in infinite dimensions. arXiv preprint arXiv:2212.00880 2022.
Patrick Kidger, Patrie Bonnier, Imanol Perez Arribas, Cristopher Salvi, and Terry Lyons. Deep signature transforms. Advances in Neural Information Processing Systems, 32, 2019.
Patrick Kidger, James Morrill, James Foster, and Terry Lyons. Neural controlled differential equations for irregular time series. Advances in Neural Information Processing Systems, 33:6696-677 6707, 2020.
12

---

Published as a conference paper at ICLR 2025
Olaf Kolle. Documentation of the weather station top of the roof of the institute building of the
max-planck-instiuun for biogeochemistry. https: www.bcc- mpc. de /t/ wetter/
2024. Accessed: 2024-08-24.
Guokun Lai. Multivariate time series data sets. htttpa://qithub com/ www../ooo/maa.uuui..aa
mul titaiiattteeemeeee-ee seri ieprres-doe es-data 2017. Accessed: 2024-08-24.
Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long-and short-term temporal patterns with deep neural networks. In The 41st international ACM SIGIR conference on research & development in information retrieval, pp. 95-104, 2018.
Bunlong Lay, Simon Welker, Julius Richter, and Timo Gerkmann. Reducing the prior mismatch of stochastic differential equations for cnffusion-basee speech enhancement. arXiv preprint arXiv:2302.14748, 2023.
Jean-Marie Lemercier, Julius Richter, Simon Welker, and Timo Gerkmann. Storm: A diffusion- based stochastic regeneration model for speech enhancement and dereverberation. JEEE/ACM Transactions on Audio, Speech, and Language Processing, 2023.
Maud Lemercier, Cristopher Salvi, Thomas Cass, Edwin V Bonilla, Theodoros Damoulas, and Terry Lyons. Siggpde: Scaling sparse gaussian processes sequential data. In International Conference on Machine Learning, pp. 6233-6243 PMLR, 202
Maud Lemercier, Cristopher Salvi, Theodoros Damoulas, Edwin Bonilla, and Terry Lyons. Distri- bution regression for sequential data. International Conference on Artificial Intelligence and Statistics, pp. 3754-3766 PMLR, 2021b.
Siyang Li, Hui Xiong, and Yize Chen. Diffcharge: Generating ev charging scenarios via denoising diffusion model. IEEE Transactions Smart Grid, 2024.
Jae Hyun Lim. Score-based diffusion models in function space. https://github. com/
11m0605/ddo 2023. Accessed: 2024-09-21.
Jae Hyun Lim, Nikola B Kovachki, Ricardo Baptista, Christopher Beckham, Kamyar Azizzadeneshelii Jean Kossaifi, Vikram Voleti, Jiaming Song, Karsten Kreis, Jan Kautz, al. Score-based diffusion models in function space. arXiv preprint arXiv:2302.07400 2023.
Terry Lyons. Differential equations driven by rough signals. Revista Matemática Iberoamericana. 14(2):215-3 -10, 1998.
Terry Lyons and Weijun Xu. Hyperbolic development and inversion of signature. Journal of Functional Analysis, 272(7):2933 2255,,255,,0 2017.
Terry Lyons and Weijun Xu. Inverting the signature of path. Journal of the European Mathematical Society, 20(7):1655- 1687, 2018.
Benoit B Mandelbrot and John W Van Ness. Fractional brownian motions, fractional noises and applications. SIAM review, 10(4):422-437, 1968.
Georg Manten, Cecilia Casolo, Emilio Ferrucci, Søren Wengel Mogensen, Cristopher Salvi, and Niki Kilbertus. Signature kernel conditional independence tests in causal discovery for stochastic processes. arXiv preprint arXiv:2402. 18477, 2024.
Olof Mogren. C-mn-gan: Continuous recurrent neural networks with adversarial training, arXiv preprint 111.99661,,09944, 2016.
James Morrill, Cristopher Salvi, Patrick Kidger, and James Foster. Neural rough differential equations for long time series. In International Conference on Machine Learning, pp. 7829-7838, PMLR, 2021.
Alexandre Pannier and Cristopher Salvi. A path-dependen. pde solver based on signature kernels. arXiv preprint 240::2403.//755 2024.
13

---

Published as a conference paper at ICLR 2025
Hengzhi Pei, Kan Ren, Yuqing Yang, Chang Liu, Tao Qin, and Dongsheng Li. Towards generating real-world time series data. In 2021 IEEE International Conference on Data Mining (ICDM), pp- 469-478. IEEE, 2021.
Angus Phillips, Thomas Seror, Michael Hutchinson, Valentin De Bortoli, Arnaud Doucet, and Emile Mathieu. Spectral diffusion processes. arXiv preprint arXiv:2209. 14/25, 2022.
Vadim Popov, Ivan Vovk, Vladimir Gogoryan, Tasnima Sadekova, and Mikhail Kudinov. Grad-tts: A diffusion probabilistic model for text-to-speech. In International Conference Machine Learning, pp. -599-8808. PMLR, 2021.
Rimhak Ree. Lie elements and algebra associated with shuffles. Annals of Mathematics, 68(2): 210-220, 1958.
Jeremy Reizenstein. Calculation of iterated-integrat signatures and log signatures. arXiv preprint arXiv:1712.02757, 2017.
Christophe Reutenauer. Free lie algebras. In Handbook of algebra, volume 3, pp. 887-903, Elsevier, 2003.
Yulia Rubanova, Ricky TQ Chen, and David K Duvenaud. Latent ordinary differential equations for irreuulrll-samppled time series. Advances in neural information processing systems, 32, 2019.
Cristopher Salvi, Thomas Cass, James Foster, Terry Lyons, and Weixin Yang. The signature kernel is the solution of goursat pde. SIAM Journal on Mathematics of Data Science, 333)8873-899 2021a.
Cristopher Salvi. Maud Lemercier, Chong Liu, Blanka Horvath, Theodoros Damoulas, and Terry Lyons. Higher order kernel mean embeddings to capture filtrations of stochastic processes. Advances in Neural Information Processing Systems, 34: 14416635-1100 16647, 2021b.
Cristopher Salvi, Joscha Diehl, Terry Lyons, Rosa Preiss, and Jeremy Reizenstein. A structure theorem for streamed information. Journal of Algebra, 634:91 1-93,, 2023.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. basre-based generative modeling through stochastic differential equations. arXiv preprint UrXiv:2011.134455 2020.
Morgan Stanley, Msml: Morgan stanley machine learning.
https://github.cr com/
Norganstanneey/MMM.
2024. Accessed: 2024-05-21.
Yusuke Tashiro, Jiaming Song, Yang Song, and Stefano Ermon. Csdi: Conditional score-based diffu- sion models for probabilistic time series imputation. Advances in Neural Information Processing Systems, 34:24804 24816. 2021.
Cécile Trottet, Manuel Schürch, Amina Mollaysa, Ahmed Allam, and Michael Krauthammer. Gener- ative time series models with interpretable latent processes for complex disease trajectories. In Deep Generative Models for Health Workshop NewrIPS 2023, 2023.
UCI Machine Learning Repository. Electric power consumption dataset. https::/ kagg le dwwaaaatttttsss.. httpttssscmmmeeeeeeeeeceee comersocoopppeeercoooosssstttooe data- 2024. Ac-
cessed: 2024-05-21.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones. Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
Pascal Vincent. A connection between score matching and denoising autoencoders. Neural computa- tion, 23(7):1661-1674 201 I.
14

---

Published as a conference paper at ICLR 2025
Vikram Voleti, Alexia for prediction, generation, and interpolation. Advances in neural information processing systems, 35:23371-23333 2022.
and Chris Pal.
conditional video diffusion
Cagatay Yildiz, Markus Heinonen, and Harri Lahdesmaki. Ode2vae: Deep generative second order odes with bayesian neural networks. Advances in Neural Information Processing Systems, 32, 2019.
Jinsung Yoon. Timegan: Temporal generative adversarial networks for time-series synthesis main script. https://githin: cww. http:///wwooo22////maooo//////// timegan. py 2024. Accessed: 2024-05-21.
Jinsung Yoon, Daniel Jarrett, and Mihaela Van der Schaar. Time-series generative adversarial networks. Advances in neural information processing systems, 32, 2019.
Xinyu Yuan. Diffusion-ts: Diffusion models for time series. https:/gitthb.cco TS 2024. Accessed: 2024-05-21.
Xinyu Yuan and Yan Qiao. Diffusion-TS: Interpretable diffusion for general time series generation. In The Twelfth International Conference on Learning Representations, 2024. URL. https
//openreview.oo net www./eooommaaaahleee.. j099
Linqi Zhou, Michael Poli, Winnie Xu, Stefano Massaroli, and Stefano Ermon. Deep latent state space models for time-series generation. In International Conference on Machine Learning, pp- 42625-42643. PMLR, 2023.
15

---

Published as a conference paper at ICLR 2025
APPENDIX
This appendix is structured in the following way. In Appendix A we complement the material presented in Section 2 with additional details on the signature. In Appendix B we provide examples of orthogonal polynomial families one can use for signature inversion due to the derived inversion formulae in S In Appendix we provide proofs for the signature inversion Theorem [.1 and Theorem 3.2 Appendix []cottains additional examples and discussion about the quality of signature provides details on the implementation of experiments.
inversion by different bases. Appendix E
A ADDITIONAL DETAILS ON THE SIGNATURE
In this section, we establish the foundational algebraic framework for signatures Appendix |A. We then provide mathematically rigorous definition of the (log)signature in Appendix [A.2 building upon the introduction in Section 2 The section concludes with illustrative signature computation examples Appendix A.3
A.1 ALGEBRAIC SETUP
For any positive integer N, we consider the truncated tensor algebra over Rd
T (BR)
(R4))AA
k==
where & denotes the outer product of vector spaces. For any scalar a E R, we denote by Ta (R*) {A 7 (R'): As a) the hyperplane of elements in T (R.) with the 0t term equal to
T" ( is a non-coomnnnnee commutamuteeive algebra when endowed with the tensor product - defined for any two elements A (Ag.A1, A,.....A, A,) and B (Bo, BI,..B. of T" (RR) as follows
B (Co, CI,... E E TTT
where
C2 C==
Σ 1=0
A Bx- (Rd)
(11)
The standard basis of IRd is denoted by c1, C2 .... Cd. We will refer to these basis elements as letters. Elements of the induced standard basis of 7 (B") are often referred to as words and abbreviated
62..2.. = ci, X 6₁2 @ City for 1 ii... ig < and s S n.
We will make use of the dual pairing notation (Et, A) E R to denote the (d.. ( element of a tensor A E (TR))).. This pairing is extended by linearity to any linear combination of words.
Following Reutenauer 2003), the truncated tensor algebra 7 (TR)) carries several additional algebraic structures.
Firstly, is Lie algebra, where the Lie bracket is the commutator
[A, B] A B B B.A for e, (RR).
We denote by L (TR') the smallest Lie subalgebra of 7 (R.)) containing Rd., We note that the Lie algebra _n (Ted) ) is vector space of dimension ((d, n) with
nd
11
d'
A
where f is the Möbius function Reeutenauer 2003). Bases of this space are known as Hall bases (Reutenauer) 2003 Reizenstein 2017). One of the most well-known bases is the Lyndon basis indexed by Lyndon words. A Lyndon word is a word occurring lexicographically earlier than any word obtained by cyclically rotating its elements.
Secondly, T (R) is also a commutative algebra with respect to the shuffle product w. On basis elements, the shuffle product of two words of length and (with n) is the sum over the
16

---

Published as a conference paper at ICLR 2025
(*t²) ways of interleaving the two words. For more formal definition, see Reutenauer (2003 Section 1.4).
Related to the shuffle product is the right half-shuffte product > defined recursively as follows: for any two words Ciy.... and Ej....j. and letter Ej
Ci... x ej Cs....eJ and Ct...,r X Cys...ja (Ct, X Cj....a-1 + Cf....--1 > Ci Cj.-
The right half-shuffle product will be useful for carrying out computations in the next section. Note that the following relation between shuffle and right half-shuffle products holds (Salvi et al. (023)
Ci... j..... Ci.... > Cj....a + Cj..... > Ci....r
Equipped with this algehraic setup, now introduce the signature.
A.2 THE (LOG)SIGGAATURE
Let [0, 1] -> Idd be smooth path. The step-n signature 5*5(()) of at is defined as the following
collection of iterated integrals
(z) = (L.)())............
TT (
(12)
where
Josn dxa, 2 dx (R") for 1 S k k
(x)x)
An important property of the signature is usually referred to the shuffle identity. This result is originally due to Ree (1958). For a modern proof see (Cass & Salvi 2024 Theorem 1.3.10).
Lemma A.0.1 (Shuffle identity). Ree 1958) Let : [0, 1] + Rd smooth path. For any two words Ei..,, and Eji-nds' with 0 S r, nI, the following two identities hold
(Cir.de W Wen..... s²*(x)) (eri..... s5*..)))s"( (ej...... S5 (x))
= S/' (Ei (z))) d (f S (5"(x)))
(Eignnir > en.... S (5m(x)) (x)) =
where 85"(z)) is the step-n signature of the path x restricted to the interval [0.
An example of simple computations using the shuffle identity is presented in Appendix A.3
Moreover, it turns out that the signature is more than just generic element of T (IR)): in fact, its range has the structure of a Lie group as we shall explain next. Recall that the tensor exponential exp and the tensor logarithm log are maps from 7 (IR") to itself defined as follows
(-1)
exp(A)
and log(1 + +A) A)=
(A)
(13)
k!
k
200
where 1 = (1, 0, .... 0) E T" ( It is a well-known fact that exp log: T (Rd) + TV (I2) are mutually inverse.
1D (IR)) + (R and
The step-n free nilpotent Lie group is the image of the free Lie algebra under the exponential map
g" (R.) = exp((" c (RR).
(14)
As its name suggests, g" ("(Rd) is a Lie group and plays a central role in the theory of rough paths (Friz & Victoir 2010).
Here comes the connection with signatures. It is established by the following fundamental result due to Chen (1557 1958), which can also be viewed as consequence of Chow's results in Chow (1939).
(Chen 1957 1958 Chow 1939) The step-n free nilpotent Lie group (IRd) is precisely the image of the step-n signature map in Equation (12 when the latter is applied to all smooth paths in gd
Lemma A.0.2 Chen-Chhonl
9d (TR) (SS"(x) (x) x [0, 1] + Rd smooth).
17

---

Published as a conference paper at ICLR 2025
A.3 SIMPLE EXAMPLES OF SIGNATURE COMPUTATIONS
In the following examples, we alter the notation so that for path x [a,4 - Rd, the tensor representing the k-th level of the signature computed on interval [a, is denoted
( a. = (S( :i....., i fl.....,d) d)) (RR))*
(15)
Furthermore, we can express the value of S(x)a, (k) at a particular set of indices ......,.. as k-fold iterated integral
ds E (1,...d) .d)
5(x)ar = S
dr
duy
(16)
Jastscnette <t
We assume that the signature is always truncated at sufficiently high level n, allowing to denote the step-n signature simply
S(z)a,: (1, S(x) S(z) .............. ((=),.......())))) (P*)..
(17)
Example A.1 (Geometric interpretation of a 2-dimensional path). Consider path ē: [0.9] + R2, where 2 x;) (t, x(t). Here, x(t) defined as
3
t E [0. 2] t E [2, 8] t [8.9]
x =(t)==
2/3
- v
which is continuous and piecewise differentiable. In this case, it 1, and ii can be expressed as
v3
E (0,2)
2 = =())
0
E (2
v3
E (8,9)
One can compute the step-n signature â as
S($)0.9 = (1, (L.S( [L.
S(0) , ((.,,5())
where
/ du! = vng In 9
S())
[ dx² = zn I = 3/3
S(4)
Jocaco
8(8)) [
2
81
de de!
x d
(x.1
Jocreeco
[
/
/
v3 2
v3
9
21
S(0)
I dr dr,
Jocoso
sdx
st d
2
v3
J
2
2
S(2)0.)
JnereseD d=žda = / J 2
|
v3
s²
12
+
+233
+
v3
8
6/38
33
V3
2
2
2
0
2
2.2 S(i)2.3
Jocreseo de,,dx? /
I
J 22dx3
=
(12)
27
10
From Figure 5 let A_ and A4 represent the signed value of the shaded region. The signed Lévy area of the path is defined as A_ + A4. In this case, the signed Lévy area is -3 Surprisingly,
(S()) (()) = /3
33 V3
--33 A_ + A
2
which is exactly the signed Lévy area.
18

---

Published as a conference paper at ICLR 2025
(9,3
(4)))3)
A
(8.2/3)
A
Ax
AX
2
x
Figure 5: Path in Example A.1 The shaded region represents the signed Lévy area.
Another important example is given by the signature of linear paths.
Example A.2 (Signatures of linear paths). Suppose there a linear path [a, b] + Rd. Then the path x is linear in terms of i.e.
It = Ia + a
(ob - xa)
follows that its derivative can be written as
(Ib Vaa) dt.
dæ
b
Recalling the definition of signature, holds that
[
8(x)
dati ... .dott
Jastiemetees
Ibb
x'd
dt....dty dts
a)k
Jostenettes
T.b
F.
a)
(b - a)k
h!
Tb
Fa
==1
k!
Therefore, the whole step-n signature can be expressed tensor exponential of the linear increment
Tb
To
5(x) (8)
(It Ia)
Ok
bd
(Ib Fa) 69 =exp
S(x)a,b =
k= k!
(xr
F.)
Chen's identity in Lemma 2.0.1 is one the most fundamental algebraic properties of the signature it describes the behaviour of the signature under the concatenation of paths.
19

---

Published as a conference paper at ICLR 2025
Definition A.1 (Concatenation). Consider two smooth paths [a. - R and y: [b, + R Define the concatenation of and y, denoted by x * y as a path [a, c| R
Zt Tb
if st fb
(x y)
yb + yt
Chen's identity in Lemma 2.0.1 provides a method to simplify the analysis of longer paths by converting them into manageable shorter ones. If we have smooth path x [a) + Rd then inductively, we can decompose the signature of x: to
tz)),, (()),..1 S(z)
((z))
Moreover, if we have a time series (tg, zo), .... (tn In) E Rd+1, we can treat piecewise linear path interpolating the data. Based on Example A.2 one can observe that
t( exp3 (It, - Ito) expo (T In1) expo (It, - Itn-1),
which is widely used in Python packages such as esig iss gnature. Example A.3 (Example of shuffle identity). Consider smooth path (I), [a,bb] + RR. By integration by parts, we have
I del I Juceed
(e1, ((z)),)) ((x)ab)
dxž
Jacke
= Hedt / iedt
Jacteb
Jacteb
by parts
Jackbb (() + / Jackeo
(el, ((),)())
= (e2,1, + (E1,2 ((z)ab)
By the shuffle identity, we have
((1,S(z) (()))
= S(x))(()at)) ()) S(z)abb
(ei e2, S(z)a.b)
= (c1.2 + 62,1 ((z)a,))
(())a)) + (e2,1, S(r)a,)),
which is exactly the same what derived via integration by parts.
Example A.4 (Example of half-shuffle computations). Consider two-dimensional, real-valued smooth path & = (I, xx())= (a, RR2 with x<aa) = The elements of the first truncation level of
the signature can be retrieved as follows:
(e1, S())..)) = [ ds ds=t-an a, (e2, (())a1))
[ d (=((s)) = x(t) =(a)) xx)).
Then, one can express all integrals in terms of powers of a and x(t) by signatures of ä. For example, let n, m E No.
1 (1 - a)"
' ( (00
/ ((e)) ((e)) S(T)a.)) ((er,S($)a.))
((etin w ) > Ei, S())a.).
B ORTHOGONAL POLYNOMIALS AND FOURIER SERIES
In this section, we introduce the background material on orthogonal polynomials and the Fourier series necessary for the signature inversion formulae presented in the next section.
20

---

Published as a conference paper at ICLR 2025
B.1 ORTHOGONAL POLYNOMIALS
B.1.1 INNER PRODUCT AND ORTHOGONALITY
Consider dot product (x, v) = E Litti, where y E R For some weights wi, IR+, we can also define the weighted dot product (x, y). = ビニ W: where (:-)) can be written as (5-) for simplicity.
For E oo), LP (92) is the linear space of measurable functions from Ω to E such that their weighted p-norms bounded, i.c.
L (0) = is measurable in 2e /, lu(t) < 20)
For example, let do be non-negative Borel measure supported on the interval [a, bj and V = L (a, b). One can define (f.9) f (())(())d(()) Stieltjes integral for all f.g g V. Note that if at))) is absolutely continuous, which will be the setting throughout this section, then one can find weight density w(() such that (I = w(()dt. this case, the definition of the inner product over a function space reduces to integral with respect to weight function, i.e.,
(f.g) - f. f(t)g())w(t)dt.
We can then consider an orthogonal polynomial system to be orthogonal with respect to the weight function w. We denote P[t] c (St) as the space of all polynomials. A polynomial of degree n, E [t] is monic if the coefficient of the n-th degree is one.
Definition B.1 (Orthogonal polynomials). For an arbitrary vector space V, and are orthogonal if (u, = 0 for all u, E V. When V P((), sequence of polynomials (Po)DEN E PP(( is called orthogonal with respect to a weight function if for all m + n,
(pr. Pm. = (t)))pm())((()dt
where deg(pn) the degree of polynomial. Furthermore, we say the sequence of orthogonal polynomials is orthonormal if (pn, Pa.) for all E N.
For simplification, the inner product notation (:--) will be used without specifying the integral formulation for the orthogonal polynomials. To construct a sequence of orthogonal polynomials in Definition B.I one can follow the Gram-Schmin orthogonalisation process, which is stated below.
Theorem B.1 GGram-Schmida orthogonalisation) The polynomial system (px) (PN)DEN with respect to the inner product (.,-) can be constructed recursively by
Ps) for 222 1. (pr,p))
1, = t
(18)
From the orthogonalisation process in Theorem B.1 we can see that the n-th polynomial Pr. has degree n exactly, which means (Pn)neN a basis spanning P/t). Furthermore, the orthogonal construction makes the orthogonal polynomial system an orthogonal basis with respect to the corresponding inner product. The following proposition forms an explicit expression for coefficients of (ps ke .n) in an arbitrary n-th degree polynomial.
Proposition B.1.1 (Orthogonal polynomial expansion). Consider an arbitrary polynomial x(t) E P.]t]. One express ætt) by sequence of orthogonal polynomials (pk)ke mn). i.e.,
n
(pE,
rr))=
px(().
(pk. Pk)
Remark. We have stated the orthogonal polynomial expansion for x. E Pn[(). In general, by the closure of orthogonal polynomial systems L (a, b), arbitrary f E 12 (a, b) can be written as an infinite sequence of orthogonal polynomials.
(pk: f) (pk.Ps)
f(t) = Σ
(t))).
21

---

Published as a conference paper at ICLR 2025
The N-th degree approximation of is the best approximating polynomial with degree less equal to N, denoted by
N (pks f)
PN)(
Pk Pk (D).
(19)
k=0
B.1.2 BASIC PROPERTIES
Here, will list the main properties of orthogonal polynomials significant for our application.
THE THREE-TERM RECURRENCE RELATION
Theorem B.2 (Three-term recurrence relation). A system of orthogonal polynomials (Pr)DEN with respect weight function w satisfies the three-term recurrence relation.
(I() = 1, (1( Ail + B1, Pn+1(() = (An+1l 1 + Ba+1)Pa(I) + Cn+11Pa- for all n E N, and A; 0 for all e No-
Before proving the recurrence relation, we will first show that an orthogonal polynomial is orthogonal to all polynomials with degree lower than that of itself.
Lemma B.2.1. polynomial q(l) P.l(4] satisfies (q,r) 0 for all r(1) E IP [1] with if and
only if (()= = Po (1) up to some constant coefficient, where pr (1) denotes the orthogonal polynomial with degree n.
Proof. =: Consider (t) = t =(t==l) and (t) = ān th O(t"-1). Then we define On
s(1) = g(1) Pr (1) (11-1),
on
which has a degree at most n - 1. Therefore, for m < n2
On (pn: Pm 0.
(m.) = (4.
The former inner product (g, Pm 0 by assumption, while the latter inner product (Pro Pm.) by
orthogonality. By Proposition B.1.1
(Pm.
ān
s(t)
Pm =0
((t)
pn. (t).
(pms Pmm)
On
m=0
-: Consider r(t) = Σ kk= (t).(t). Let q()) = cP.(().. Using the linearity of the inner product and orthogonality of (pm MEN, for all m < n,
m
(g.r) = cpn (t),
(xPK(t) =
Th (t), = 0.
km
km
Now, we have enough tools to prove the famous three-term recurrence relation.
Proof of Theorem B.2 Consider a sequence of orthogonal polynomials (p.).eN. When n P1 can be expressed as Art + B₁ for A1, B1 E R. This is because p1 is an element in an orthogonal basis with degree 1. Based on the inner product of orthogonal polynomials,
f ( (px(()), = (tpx.P.)-
(pu.tpm) =
Therefore, for 0 < k < 1, we have (Pt. tp..) 0 by Lemma B.2.1 Since tp has degree n + 1, by Proposition B.I.I.
(pk. tpm)
n+1
(pe, tp))
tpn(t)
pp) (t(
pk (t) an- + anpx(t)) + an+ (t)
(k).
k=n-1
t an
ap
= Pat1
(t)))=
Pn (t).
an+1
On+11
On+1
22

---

Published as a conference paper at ICLR 2025
Remark. Recurrence is the core property of orthogonal polynomials in our setting, the orthogonal
find
higher-order coefficients based on lower-order coefficients given the analytical form of form
polynomials. This idea coincides with the shuffle identity of signatures. As stated in Theorem 3.2 can construct an explicit recurrence relation for the coefficients of orthogonal polynomials by linear functionals acting on signatures.
APPROXIMATION RESULTS FOR FUNCTIONS IN L
Without loss of generality, consider f E (-1,, [a, b] linearly into the interval [-,, Recall the N-th degree approximation PN f(t) defined in Equation (19). The uniform convergence of the N-th degree approximation ft) f can be found in Atkinson (209)). where we obtain
always transform arbitrary interval
1ls PN S |l - P S (1+ (1 0||اo
E PN,
2
where |x relates to the system of orthogonal polynomials, and | depends on the smooth- ness of f. In the case of Chebyshev polynomials, where the weight function is w(t) = 1/v1 8
|N = log n + ((1) (Atkinson 2009). For some a E (0,1]. 1].
log N Ilf PN-P flflle S c Nk+a
for N 2
This bound result is shown numerically in Figure 6
10
10
orgen p4h
-1.00
-0.75
-050
-025
0.00
0.25
0.50
0.75
100
10
N
16
51 16
10"
orgiral pth
0.00
-0.75
-0.50
-020
0.00
0.26
0.50
0.76
1.00
10
12
N
5
-1.0
orginal pah
1.00
0.75
10.00
0.25
0.00
0.25
0.50
0.75
1.0
1.0
12
N
Figure 6: The approximation quality (left) and convergence of the T- error (right) for Chebyshev polynomials of increasing degree N. As N increases, the colours change from blue red in the left column. Paths are given by (top to bottom): z (t) = cos 10t) sin(2 (t sin din(10t)= + -t, (t)(t) 2t 11
B.1.3 EXAMPLES
In this subsection, will provide two general orthogonal polynomial families, Jacobi polynomials and Hermite polynomials, which will be used for signature inversion in the next section. Figure visualises the first few polynomials of these two kinds.
23

---

Published as a conference paper at ICLR 2025
Legendre Polynomials
Hermite Polynomials
1.0
20
mPQ
mn
n12
pwlt)
Pull
0.0
605
1.0
-1.00
-0.75
-0.50
0.00
0.5
0.75
1.00
-10
Figure 7: Visualisation of the first 7 Legendre and Hermite polynomials.
JACOBI POLYNOMIALS
Jacobi polynomials pa (a.9) are system of orthogonal polynomials with respect to the weight function w: () + R such that
w=t: a: 3) = (1 - () 1)
There are many well-known special cases of Jacobi polynomials, such as Legendre polynomials ph and Chebyshev polynomials pa (-1/2.-1/2) In general, the analytical expression of Jacobi polynomials [smail 2005) is defined by the hypergeometric function FF:
(0.0)
pn ()
(a + 1)n 2 Fii-n,11 a + m. 1: (1- ()-
1
n!
where (ar + 1), is the Pochhammer's symbol. For orthogonality, Jacobi polynomials satisfy 2 2*+ Ta 1) U
[ (1-t) (2. (t)pl (1.8) (t) (2n + a + 8 + ( 3 1)n!
6nm
a, 1.3>-11
where om is the Kronecker delta. For fixed a; B, the recurrence relation of Jacobi polynomials is 2n + 1 a 3-2) (2n a + a 2)t a² -3²) p (1)))()
Pn (a (t) 2n(n + a +
an+a-1)(n=3- 1) + B) Pn (a.9)
m(n a(n+a++ () 2)
HERMITE POLYNOMTAL.S
Hermite polynomials
system of orthogonal polynomials with respect the weight function Da
w. (-o0, x)) + R such that w(l) = (xpp--1/2). These are called the probabilist's Hermite polynomials, which will use throughout the section. There is another form called the physicist's Hermite polynomials with respect to the weight function w(t) = exp(-t?). The explicit expression of the probabilist's Hermite polynomials can be written as
[
t 2m
(t) nl!
Σ
ml(n 2m)!
with the orthogonality property
(m)
2
(20)
Lastly, we state the recurrence relation of Hermite polynomials as Hn+1(1) xHn(() nHH. 1 (1).
Note that the weight of Hermite polynomials can be viewed as an unnormalised normal distribution. If we are more interested a particular region far away from the origin, we can define "shift-and-scale" version of Hermite polynomials with respect to the weight
to (t) = exp((1 2
where to denotes the new centre, and is the standard deviation. Let (Hi MeN denote the shift-and- scale Hermite polynomials. Then, the orthogonality property is
H (t) Ho (()e
1e-10)2 det dt = E =
H (to + ey) H.o (to + ez)e " dy.
24

---

Published as a conference paper at ICLR 2025
by substitution y to)/c. Hence, if H (to + ey) (v). n N,
(21)
then (H MeN is an orthogonal polynomial system with orthogonality
(t-ta)²² 2nd dt =E = Hm Hm.(y)Hnn % dy evvve2
(t) (t)e
which follows from the orthogonality Hermite polynomials in Equation 20). connection between Hermite and shift-and-scale Hermite polynomials in Equation 21 provides way find the explicit form and recurrence relation of Hoiee) JEN, which are
Similarly, the
[
1
t to
n-2m
H. n!!
Σ
(22)
ml(n 2mm)
m=0
1 (t) = (t - to) H (1) n
(23)
Remark. Note that there is simple expression for (H that
mee at to. One can easily observe
1 (to) = (-2)
al 2
for even
for odd n
B.2 FOURIER SERIES
One can also represent function by trigonometric series. Here, we only present a brief introduction to the Fourier series, providing complementary details to the main result in Theorem 3.1
B.2.1 TRIGONOMETRIC SERIES
Let f E (-((-,,))) r). The Fourier series of f is defined by
40 2
F(t)
+
(ax cos(kt) + by sin(kt)),
where
4% = ["
f(t) cos(kt)dz, k 0
1
bk
f(t) sin(kt)dr,
h: 2 1,
which can be derived from the orthogonal bases {cos k and {sin kt}x. More generally, we can extend the period to 2/ R. For f L' (-1,t) and k Z.
F(t) Σ
che e thee
C = 1 f(t)e-1* k7 dt.
(24)
In the setting of the Fourier series, the expression for the k-th coefficient e& in the exponential form can be defined as a linear functional (*(x) = of on the space of Fourier series, for (-1,)
B.2.2 CONVERGENCE
Under some regularity conditions, F(t) converges to f(t) (Atkinson, 2009). To examine the conver- gence of the Fourier series, we define the partial sum of the Fourier series as
Sx1(()
ag 2
(ap cos(kt) by sin(k)))
Now we present pointwise convergence and uniform convergence results (Atkinson) 2009) of the Fourier series for various functions.
25

---

Published as a conference paper at ICLR 2025
Theorem B.3 (Pointwise convergence for bounded variation). For 2T-periodic function f of bounded variation on [-T, n]. its Fourier series at an arbitrary converges to
1 +
Theorem B.4 (Uniform convergence for piecewise smooth functions). f is 2m-periodic piecewise smooth function,
(a) if f also continuous, then the Fourier series converges uniformly and continuously to
(b) if f is not continuous, then the Fourier series converges uniformly f on every closed interval without discontinuous points.
Theorem B.5 (Uniform error bounds). Let f E (la((x)) be 2x-perio k-times continuously differentiable function that is Hölder continuous with the exponent a E (0, Then, the 2-norm and infinity-norm bound of the partial sum Sy I can be expressed
1
log N for N 2 2.
llf - SNffl2 & ff SN S Ck Nk+a
2
For functions only defined on an interval (a, b). we can always shift and extend them to be 2x-periodic functions. These theorems guarantee the convergence of common functions will use in later experiments. To illustrate this, Figure 8 compares the convergence theorem bounds to numerical approximation results. Note that compared with the path =2(() sin( (10)) + e t, the other paths have better empirical convergence results. The main reason is that the Fourier series of at t #1 does not pointwise converge x=( Since the Fourier series treats the interval [- as one period over IR, by Theorem B.3 the series will converge to (x2(-1) + (2))) at = +1, leading to incorrect convergence at boundaries. This property also arises in real-world non-periodic
time series, leading to introduce the mirror augmentation later in Appendix F
Comparing Figures one can observe that orthogonal polynomials are better at approximating continuously differentiable paths, while the Fourier series is better at estimating paths with spikes, and computation is more stable in the long run. a later section, Figurep]prove provides a summary of convergence results for different types of orthogonal polynomials and Founer series, which also match the results shown here.
B.3 APPROXIMATION QUALITY OF ORTHOGONAL POLYNOMIALS AND FOURIER SERIES
Finally, we present a numerical comparison of the approximation results given by the methods introduced above.
In particular, we will visualise the approximation results of the following objects:
Legendre polynomials: w(t) =
types of Jacobi polynomials: wtt) v + w(() -
three types of thift-nd-scaal. Hermite polynomials with different variance for pointwise approximation: = 0.1, 0.05, 0.01
Fourier series
For pointwise approximation via the Hermite polynomials, each sample point of the function will be approximated by a system of Hermite polynomials centred at the point 4;, i.e., (H ")ENe test the approximation quality, we simulate random polynomial functions and random trigonometric functions. The L2 error is then obtained by an average over 10 functions of each type.
nen To
B.3.1 APPROXIMATION RESULTS
Figure demonstrates the reduction in L2 error as the order of orthogonal polynomials and Fourier series increases. Among all the bases considered, the Fourier series provides the least accurate approximation for both path types. This is due to its inability to guarantee pointwise convergence at +1, which is caused by boundary inconsistencies. Three types of Jacobi polynomials, including
26

---

Published as a conference paper at ICLR 2025
oighal
-0.00
-0.25
0.50
0.75
1.00
2x100
omo
orginal path
3x
1.00
0.50
0.75
1.0
S 10"
otghillphh
-1.0
-1.00
0.75
0.50
0.25
0.50
0.25
0.80
0.75
1.00
10"
12
Figure 8: Approximation (left) and L2 convergence (right) results for Fourier series by increasing order N, with the same experimental setting Figure
Legendre polynomials, show comparable approximation results, with slight edge in convergence observed for Legendre polynomials. On the other hand, Hermite polynomials exhibit much lower approximation error due their shifting focus on the point of interest. However, decreasing to sharpen the focus sample points can cause the coefficients of Hermite polynomials to inflate rapidly. This behaviour consistent with the analytic form and recurrence relation of the shift-and- scale Hermite polynomials, as detailed in Equation (22) and Equation (23). As result, Hermite polynomials with é = 0.01 do not outperform those with = 0.05. The step-like pattern decrease observed in Hermite polynomials can be traced back to Remark B.1.3] Figure I0 shows the L2 approximation error of different bases for two our real-world datasets. Here, we see that apart from using shift-and-scale Hermite polynomials which come with additional complexity (see Figure 3). the Fourier series seems to be the best candidate for approximation.
Legerdre
Jacobil0,
30
Jacobric.5,
Hermite e 0.05
Harmite
10
10
order N
order N
Figure 9: L2 approximation error using different bases. The figures (from left right) the corresponding error averaged over 10 random polynomial and trigonometric functions.
To mitigate computational expense, we henceforth use Hermite polynomials with e = 0.05 as the representative of the Hermite family. The findings presented in Figures 9 and 10 play crucial role
27

---

Published as a conference paper at ICLR 2025
Fourler
0.22
Lagendre
Jacab4(0,
Jacabr(O.5,
025
Hermite == 0.05
0.2
030
Figure 10: Real data L2 approximation error using different bases. The figures (from left to right) are the corresponding error averaged over 10 random samples from the HEPC and Exchange rates datasets.
our signature inversion method, as they establish benchmark for the best possible performance attainable in path reconstruction from signatures.
C PROOFS OF SIGNATURE INVERSION
In this section, present the formal proofs of the signature inversion Theorem 3.2 and Theorem 3.1
along with the remark in Section 3.2 about Taylor approximation of the weight function.
C.I PROOF ORTHOGONAL POLYNOMIAL INVERSION THEOREM 3.2
Recall the statement Theorem 3.2 deriving the n-th polynomial coefficient C,, (see Equation 10 via recurrence relation:
Let x. : (a, b] > be a smooth path such that s(a) = 0. Consider the augmentation a(t) (t, (t) TR2, where w(t) corresponds to the weight function of system of orthogonal
polynomials (Px)neN. and is well defined on the closed and compact interval [a, 6). Then, there
exists linear combination én of words such that the th coefficient in Equation (10) satisfies
= (Ln, S(i)). Furthermore, the sequence (En)neN satisfies the following recurrence relation
in An (Pn-1:Pr-1) ej en-1 + (A, + Br) (Pn-1.Pn-1 ln-1 On (Pn-2, Pn-2) ly 2 (pn)Pr)
(pn.Pn)
(Pn,
with
Ao (po,
A
(e121 + e211) + Aia + B
to
and 4 = (p1,PI)
(pI,PI) e21
28

---

Published as a conference paper at ICLR 2025
One express the first two coefficients in orthogonal polynomial expansion of by the signature:
Proof.
L: Anz(())(((dd
00
(po,Po)
Po)
Ao
C2 61, S(t)
(Po Po)
Ao
E21 S())
(po, po)
(L,, S(t)),
1
(A1t+ (i)x(t)((()dtt
a1
(1).P))
A
+ Aia + B₁ db
(PI,P1) [ x(()w(()d
(1 (L-a)x((w((dd)
(P1,P1)
Ai
((ei (2222 el, S()) + Aia Bi
(e21, S())
(P1,P1)
(p1,P1)
Ai
(e121 + 6211) + Aia + B
621 S(i))
(PI,Pi)
(P1.P1)
(f1, (()).
Then one can find e, recursively by multiplying both sides Equation 99 by xtt) and integrating on Ja, b]:
1' pm (((())())((()) = S(
/6 (Anl 10)) 1. Coln-2(0)a(())(())))
(t- a)d (. (m-n-s())a(()))(o)) An ( (t
+ (Ana+ + BR) [- (n-1(t)x(()w())d
Cnn 1 (n-2
By definition of On.
[ (_()))(t)w(()d=== = (n),Pn)a, (pn, (en,
An [ (t - ajd (.. (nn((ooaa) = An 1pn- - 1 S(i))
(Ana+ Br) S Pn-1(0)(() (Ana+ 1
Cn pn-2(=)x(t)u((jdt Cn Pn- D 24 S()).
Therefore, the recurrence relation of the linear functions on the signature retrieving the coefficients of orthogonal polynomials is
(Cn, S(b)) = AA, (Pn-1,Pa-1) (El > S(8))
(pu,Pa)
+ (An + Brd (Pn-1,Pa-1)
(L ((n=1,
pn)
+ Cn (Pn-2.Pn-2)
(1 (2n-2, S(i))
(pnsVn)
29

---

Published as a conference paper at ICLR 2025
Several assumptions made in order derive the recurrence relation. Namely, the interval defined on the inner space must be compact, and the weight function (t) should be well-defined on the closed interval (a, b]. These assumptions can limit the range of applicable orthogonal polynomial families. For example, since the range of Hermite polynomials is unbounded, they are not suitable for our inversion. However, by using shift-and-scale version of these polynomials, where most of the weight density is concentrated at a particular point, their weight can be numerically truncated to compact interval. One can centre the weight density using small enough and shift it to point of interest f;. Since the non-zero density region is now concentrated in a small interval, Theorem 3.2 can be applied to the truncated density over this interval. The relationship between the original Hermite polynomials and the shift-and-scale Hermite polynomials is given by Equation 21
C.2 REMARK ON THE TAYLOR APPROXIMATION OF THE WEIGIIT FUNCTION
The results in Theorem 3.2 require signatures of (t, w(t)x((t)) However, sometimes one may only have signatures of (4, Here, we propose a theoretically applicable method by approximating the weight function as a Taylor polynomial.
Consider the Taylor approximation of w around / = a, i.e.,
d'w
w(t) = 0
(t - a)' =
Σ wi(t - -a)'.
dti
Letting ã; = (()) and
G == (e2 we > >e i!(e21.... +1121...1 + + 12...
we have
1
L (o)(()u(t)dt
au = (po, po)
Ao
[ ph
Σws
Wi
(t- a)' (t)dt
(po, Po)
1=0
Ao
=(po. Po)
wi(e2 we e1, S(3)
Ao
M
WiCs S(i))
(po. Po)
1=0
(Co, S(T)),
1
(Ail+ (i)x(()((()d
P1),
1
(A1
Bi) (t))
(P1.P1)
M
I ())
Σ
Wi
(Ai(t - -a) - (Aia + (t)(t - a)') x(t)dt
(P1,P1)
e
M
wi (Ai(ez ) + (Aia Bi)(e > S())
(p1PP))
1=0
wi(Acc (Aia+ B()c)),(())a))
(p1.P1)
(4,. S(z))
By induction, the same relation Theorem 3.2 holds
( x ln-1 + (An + Br) (Pn-1 1, Pn-1)
en-1 + On (Pn-2, (2..Pn-2) Ln-2. (pn, pn)
ln = An (pr)Pn)
(pn, Pn)
30

---

Published as a conference paper at ICLR 2025
There several reasons why consider the Taylor approximation method "theoretically applicable." The expansion of the weight function around point a can be hard to find analytically, and even if the series is found, it may diverge. If the series does converge, still need to determine the number of terms required to meet a certain error tolerance. Furthermore, if the convergence rate is slow, more terms are needed in the series. This leads to higher necessary signature truncation levels and consequently also higher computational complexity.
C.3 PROOF OF FOURIER INVERSION IN THEOREM
Recall the statement in Theorem 3.1 deriving the Fourier coefficients an, an. bn (see Equations 5
( 7 of path as follows:
Let [0, 2m] + R be periodic smooth path such that z(0) O, and consider the augmentation a(t) (t, sin cos(t)
1, xx))) E IR Then the following relations hold
(es - e1, S(t)),
2x
k
On
cOs(
(n
((e) Iu e w e/ e1, 5(0)),
9
q=
k
hn
sin
(n ((e) w w wez > S()))
2
Proof. By Multiple-Angle formulas, we have
sin(nt) Σ
n
n2 k
cos x (f) sin" siin""(tt sin( 5 k))),
(25)
n
ni h
cos (1)sin (in)nn)(())) cost 1 ((1 k)))).
cos(nd)
(26)
We can now connect Equation 77. Equation 25 and the shuffle identity of the signature described in Lemma 1 to obtain expression for b,
1
25
xt) (nt)
π
2x
xt(t)
n2
cos" (t) sin"">(t) sin S k)r)ddt
k
k=0
2m
sin( ク (n - ((k-))))
a(l) cos * (1) (i)sinn" sin
k
sin ( (n b)T)
x(t)((cos(t)) 1) + 1)* sin" -
(27)
2
h (cos(t) - 1)
sin 2 (n - k)))
x(t) ((t)sinn-3 m-)
9=0
k
sin 5 (n -k)T) /-² x(t) (tt - 1)
k
k
sin (n (()) -un w )4 E1, S()))
31

---

Published as a conference paper at ICLR 2025
Table 3: Polynomial/Fourie basis orders used in Figures 11 and 12
Approximation method
Order
Level of truncated signature
Legendre Jacobi(O, 0.5)
10
n+2=12
10
n+2=12
J0c5
10
n+2=12
Hermite (e = 0.05)
2
n+2=4
Fourier
10
n+2=12
Similarly, rearrange Equation (6 with Equation (26) obtain the formula for an
1
2m (t) cos(nt)dt
On
π
0
(28)
k
cos( / (n ((T) Wez w ex ) X e1. S(t)).
k=
Q=
Finally, we get ag immediately
G0 =
x(t)dt = (es > S(8))
(29)
2π
D VISUALISING INVERSION BY DIFFERENT BASES
We demonstrate the quality of signature inversion paths generated from fractional Brownian motion (Mandelbrot & Van Ness, 1968) (see Figure []) and two of our real-world datasets (see Figure 12) using five different polynomial and Fourier bases. The orders of the bases recovered in these figures are listed in Table D establishing a relationship with the levels of truncated signatures. The reconstruction results are influenced by three main factors:
1. the degree/order of the bases, n, and the corresponding levels of truncated signatures;
2. the complexity of the paths, such as frequency and smoothness;
3. the weight function of the orthogonal polynomials.
Factor plays a crucial role in path reconstruction by providing a theoretical bound on the inversion error, the order of the retrieved polynomial and Fourier coefficients is constrained by the truncation level of the signature. Recall Figure 9 which demonstrates that increasing the basis order improves the approximation quality, implying that reconstructions from higher-level signatures will more closely approximate the original path.
Factor 2 also plays crucial role in the quality of approximation. A comparison between the first and second columns of Figure 11 shows that all bases can approximate the simpler path in the second column accurately. Consequently,
complex paths may lead to less satisfactory inversion results due to the limitations of the bases, as previously discussed in Section 5.2.
Compared the previous factors, factor 3 has a relatively minor impact on the reconstruction process. As shown in Figure [2] the left tail of the Jacobi(O, 0.5) approximation and the right tail of the Jacobi 0) approximation tend diverge as their weight functions approach zero at - +1. Meanwhile, the signature inversion using Hermite polynomials, performed on a pointwise basis, remains precise even at lower polynomial degrees, as each sample point estimated centre of the weight function.
E EXPERIMENT DETAILS
In this section, we provide additional details about the experimental setup. The accompanying code be found at https: / //g tigh.u...... com hata://abbrra/aaaaa fusi ons.
32

---

Published as a conference paper at ICLR 2025
1.00
05
0.75
0.85
0.00
02
-0.00
1.00
0.25
0.00
1.00
0.75
0.00
025
0.4
0.50
03
0.25
01
10.25
80th
00
DOT
-0.50
M
xcnakress
1.00
0300
0.25
0.00
0.25
0.75
0.75
0.00
1.00
05
0.75
0.88
0.00
02
0.1
0.50
8
0.25
03
02
-0.25
0.1
-0.50
M
stnatures
0.0
gnaturts
1.000
0.75
0.00
0.25
0,00
0.25
0.00
0:76
1.00
-1.00
0.76
-0.00
0.20
0.00
0.25
0.50
0.76
1.00
1.00
0.5
0.75
-0.25
0.1
0.50
0.0
Figure 11: Inversion results on fractional Brownian motion with Hurst 0.5 and 0.9, with approximation bases (from top to bottom) Legendre (Jacobi(O, 0)), Jacobi(O, 0.5), Jacobi(O.,, 0), Hermite (c = 0.05) and Fourier.
33

---

Published as a conference paper at ICLR 2025
08
0.4
pach
parid dm
0.5
0.4
0.3
0.2
0.
0.0
0.0
1.50
0.75
0.50
0.25
0.00
0.25
0.50
0.75
1.00
1.00
0.75
0.50
0.25
0.00
0.25
9.0
0.75
1.00
MA
0.6
0.4
nwch signouures
gnazurcs
0.5
wash MM
0.4
0.3
32
0.2
0.1
oourees
gnoturcs
2.50
0.75
0.50
0.25
0.00
0.25
0.50
0.75
1.00
1.00
0.75
0.50
0.25
0.00
0.25
9.
0.75
1.00
M
0.6
32 0.4
pah
signoures
1.00
Inversion results real-world time series. The left column is sample from the HEPC dataset. The right column is a sample from the Exchange rates dataset (for readability, we only plot one of the eight dimensions). The approximation bases (from top bottom) are Legendre (Jacobi(O, 0)), Jacobi(O, 0.5), Jacobi(0.5, 0), Hermite (e 0.05) and Fourier.
Figure 12:
34

---

Published as a conference paper at ICLR 2025
Table 4: KS Test average scores and type I errors on the marginals of time series of length 1000.
Dataset
Model
1=300
(=500
L=700
1=900
SigDiffusion (ours)
0.25, 50%
0.22, 31%
0.24, 40%
0.23, 38%
DDO ( =) 1)
0.17, 7%
0.16, 5%
0.20, 15%
0.23, 33 %
Sines
Diffusion-TS
0.77, 100%
0,50, 100%
0.48, 100%
0.48, 100%
CSPD-GP (RNN)
0,62, 100%
0,45, 82%
0.48, 95%
0.55, 99%
CSPD-GP (Transformer)
0.57, 100%
0.49, 100%
0.51, 99%
0.60, 100%
SigDiffusion (ours)
0.20, 20%
0.29, 76%
0.23, 39%
0.22, 35 %
DDO = 10)
0,34, 92%
0.30, 85%
0.36, 96%
0,40, 100%
Predator-prey
Diffusion-TS
1.00, 100%
1.00, 100%
1.00, 100%
0.99, 100%
CSPD-GP (RNN)
0.27, 56%
0.25, 47%
0.32, 74%
0.31, 79%
CSPD-GP (Transformer)
0.79, 100%
0.78, 100%
0.79, 100%
0.74, 100%
SigDiffusion (ours)
0.20, 16%
0.18, 9%
0.19, 12
0.21, 22 %
DDO (7 1) Diffusion-TS
0.25, 46%
0.23, 38%
0.25, 46%
0.25, 51%
HEPC
0.85, 100%
0.87, 100%
0.82, 100%
0.88, 100%
CSPD-GP (RNN)
0,52, 100%
0,53, 100%
0.55, 100%
0.56, 100%
CSPD-GP (Transformer)
1.00, 100%
1.00, 100%
1.00. 100%
1.00, 100%
SigDiffusion (ours)
0.31, 80%
0.28, 67%
0.28. 65%
0,31,
DDO (y 1) Diffusion-TS
0.24, 41% %
0.24, 42%
0.25, 45
0.25, 45 %
Exchange rates
0.72, 100%
0.71, 100%
0.70, 100%
0.69, 100%
CSPD-GP (RNN)
0.59, 100%
0.56, 100%
0.55, 100%
0.56, 100%
CSPD-GP (Transformer)
1,00, 100%
0.99, 100%
0.98, 100%
0.99, 100%
SigDiffusion (ours)
0.35, 82%
0.34, 80%
0.33, 76%
0.34, 78%
DDO (7 = 10) Diffusion-TS
0.26, 45%
0.27, 52%
0.26, %
0.26, 47% %
Weather
0.49, 100%
0.50, 100%
0.50, 100%
0.49, 100%
CSPD-GP (RNN)
0,57, 100%
0.56, 100&
0.56, 100%
0,56, 100%
CSPD-GP (Transformer)
0.91, 100%
0.92, 100%
0.91. 100%
0.91, 100%
We follow the score-based generative diffusion via variance--reeserving SDE paradigm proposed in Song & Ermon (2019). We tune 3min and mmg in Equation (4) to be 0.1 and 5 respectively. We use a denoising score-matching (Vincent 201 objective for training the score network 8p- For sampling,
we discretise the probability flow ODE
dx,-
(()(x, + so x:)ddt E [0,
(30)
2
with an initial point X₁ ~ (O, 1). To solve the discretised ODE, we use a Tsit solver with 128 time steps. We adopt the implementation of the Predictive and Discriminative Score metrics from TimeGAN (Yoon) 2024). To satisfy the conditions for Fourier inversion, we augment the paths with additional channels described in Theorem 3.1 and we add extra point to the beginning of each path, making it start with O.
The model architecture remains fixed throughout the experiments layers, a hidden size of 64, and 4 attention heads. Note that other relevant works (Yuan & Qiao 2024) Coletta et al. [2224] Bilos et al.) (203)) follow a very similar or bigger architecture. We use the Adam optimiser. We run the experiments on an NVIDIA GeForce RTX 4070 Ti GPU. Table details additional KS test performance metries (see Section 5)
transformer with 4 residual
For the task of generating time series described in Section [..2 we fix the number of samples to 1000, the batch size to 128, the number of epochs to 1200, and the learning rate 0.001. The details variable across datasets are listed in Table 5 We reserve 1000 points from each dataset as an unseen test set for metric calculation.
The mirror augmentation For many datasets, we might not wish to assume path periodicity as required in the Fourier inversion conditions in Theorem 3.1 We observed that a useful trick in this case is to concatenate the path with reversed version of itself before performing the additional augmentations. We denote this the mirror augmentation. Table iindiictss the datasets for which this augmentation is performed.
35

---

Published as a conference paper at ICLR 2025
Table 5: Datasets for long time series generation.
Dataset
Mirror augmentation
Data points
Dimensions
Sines
Yes
10000
5
HEPC
No
10242
1
Tredator-prey
No
10000
2
Exchange rates
No
6588
8
Weather
No
10340
14
Datasets As previously described in Section 5 we measure the performance of SigDi ffusions on two synthetic (Sines and Predator-pe prey))ppeyye)) and three real-world (HEPC, Exchange Rates, Weather) public datasets. We generate Sines the same way the Sine dataset is generated in TimeGAN (Yoon) jet al., 2019), by sampling sine curves at a random phase and frequency but changing the sampling rate to 1000. Predator-prey is a dataset consisting of sample trajectories of a two-dimensional system of ODEs adopted from Bilos et al. (2023)
2
2
ry.
3
(31)
ÿ zy y.
We generate Predator-prey time grid of 1000 points on the interval t E [0, 10). HEPC (UCI Machine Learning Repository 2024) is a household electricity consumption dataset collected minute- wise for 47 months from 2006 to 2010. We slice the dataset to windows of length 1000 with stride of 200, yielding a dataset of 10242 entries. We select the voltage feature to generate as a univariate time series. We use the Exchange Rates dataset provided in Lai et al. (2018); Lai (2017) and slice with stride of 1, yiclding 6588 time series. Lastly, the Weather dataset was measured and published Planck-Institune
by the this dataset describing the pressure, temperature, humidity, and wind conditions, and we slice the time series with stride of 5 to get 10340 samples.
for Biogeochemistry (Kolle, 2024). We take the first 14 features from
Benchmarks We compare our models to four recent diffusion models for long time series genera- tion: Diffusion-TS (Yuan & Qiao 2024). DDO (Lim et al. 2023), and two variants of CSPD-GP (Biloš) with transformer. For Diffusion-T's TS and al. 2023) one with an RNN for score function and
CSPD-GP, we keep the model configurations as they were proposed in the authors' implementations for datasets with similar dimensions and number of data points. One exception this halving architectures due to memory constraints. We also halve the
the batch size for transformer-bat number of epochs to preserve the proposed number of training steps. As DDO has previously only been implemented on image-shaped data, we alter the code in the authors' GitHub implementation to generate samples of shape (time series length x x number of channels). We always report the performance for the RBF kernel smoothness hyperparameter
|0.05, 0.2, 1,5, 10) corresponding to the highest predictive score. We train the model for 300 epochs (see Appendix Section J of the DDO paper by Lim et al. (2023)) with batch size of 32 and keep the remaining hyperparameters as proposed for the Volcano dataset. Table shows the number of parameters and computation times for each model. We use the publicly available code run the benchmarks:
https//gghhubbccommoooggggaaaaaa (Stanley 2024)
Yuan 2024)
https//github.com//imm66666 (Lim 2023)
Experiments with the truncation level To produce Figure we use dataset generated by Equation (31). discretised to 25 points. To add non-trivial high-frequency components, we include sine and cosine waves of the first eight Fourier frequencies, with random amplitudes drawn from standard normal distribution. To keep the model architecture consistent and avoid memory issues at higher truncation levels, we use the linearised attention mechanism from Katharopoulos al. (2020) with relu kernel, instead of the full attention mechanism. We also reduce the model's hidden size from 64 to 32.
36


Reviews:
[{"review_id": "LljWCqc9Aw", "reviewer": "Reviewer_ry7B", "summary": "Authors propose a generative model for multivariate time-series data, SigDiffusion. The diffusion processes generates samples from a distribution in a (latent truncated) log-signature space (coming from Rough Path theory), which serves as an efficient embedding of a time series. The authors discuss improved methods for recover from the log-signature to time-series path space — namely they propose some new closed-form inversion formulas to recover the time-series trajectories from the log-signature as polynomial series. \n\nThe two main contributions are:\n1. Application of score-based generative diffusion model to space of truncated log-signatures, which is \n2. Improvements to signature inversion algorithm going from the log-signature space to trajectory space. \n    1. This is done by path as expressed as a series in different polynomial bases, which can be shown to converge.\n\nThese two claims are backed up by empirical experiments:\n- For the signature inversion: synthetic evaluation using example paths random sine waves + gaussian noise,\n- For the diffusion model application time-series: Two synthetic data (sines, Lotka–Volterra system) and three real data: Household Electric Power Consumption, Exchange Rates, and Weather.", "questions": "- Could authors provide some results on signal recovery for real signals? (e.g. some signals have non-noise high frequency characteristics that may not be well captured without higher order terms of polynomial series, you could back this with up with some synthetic signals as well).\n- In Appendix E of experimental details could authors provide some discussion about which inversion scheme (and how many basis authors used) authors chose for the real data and why?\n- Why is the order of the log-signature order chosen to be 4 in your experiment? Since the runtime growth is highly unfavorable in the order $\\mathcal{O}[(d^{n+1}-1)/(d-1)]$ and mildly unfavorable in the dimension of the signal.\n- On the runtime aspect, to make work self contained, could authors provide some discussion (extending Figure 1) about how much runtime is required to transform between the truncated log-signature space (extending discussion in lines 198-205), and go from the log-signature space to the path space using Theorems in 3.1 and 3.2 (i.e., extending Table 2) and comment on runtime as compared to previous method (e.g., Inversion method).\n- Connection to long sequences specifically is unclear to me, could you elaborate on why SigDiffusion is more suitable for long sequences?\n\nHappy to adjust my score if these concerns are addressed", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is clear, well organized, and well written (see later comments on section 2)\n- The overall scheme using log-signature is grounded in sound mathematical theory from rough path theory and signature method literature. The claims in Theorems 3.1 and 3.2 seem reasonable (though I did not check in high detail) and appear to be new contributions that address a known problem in literature.\n- Efficacy of doing diffusion in log-signature space backed up by empirical evidence\n    - Per Table 2. The speedup by doing the generative process in the truncated log-signature is an advantage with comparable or better performance to other diffusion method for continuous-time paths.", "weaknesses": "### Contribution, Theory, and Experiments:\n\n- Doing the diffusion in log-signature (or the space of any integral transform/transformed feature or embedding) is a relatively straightforward idea (similar ideas are discussed in related literature of section 4). Primary contribution seems to me to the closed form signature inversion formula, which then enables one to do the diffusion processes in the log-signature space. I am more or less convinced that the presented polynomial series would approximate the original signal. But all numerical illustrations done in this aspect are only with synthetic data, and not using real data (though the synthetic illustrations are quite extensive, they seem to mostly cover of the case of a low frequency signal + noise).\n    - Since the signature inversion is a primary claim, it may be useful to provide some illustrations using the real datasets about reconstruction quality (e.g., in terms of $\\ell_2$ error akin to Fig 8.) from log signature (e.g., number of basis, type of basis, properties of the data analogous to Appendix D.)\n    - The runtime to recover using the closed form inversion formula in terms of number of basis and number of truncated signatures is not explicitly stated.\n- It may be useful to have at least a comparison to one non-diffusion baed generative time series model (i.e., pick one from section 4, following from TimeGAN paper).\n\n### Organization and Writing:\n\n- Disclaimer: I not an expert in signature method or rough path theory:\n    - Not coming from this background, I think presentation in full generality of Section 2 is, in my opinion, not very accessible.\n    - The present development may be more suitable for an appendix. The exposition in provided references (e.g., Fermanian et al. 2023) was more clear to me.\n\n### Errata: \n- Line 213: acronym ST is not defined, assume it refers to (log)-Signature Transform.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "prG0YmNBzG", "reviewer": "Reviewer_U13T", "summary": "This paper proposes an algorithm for generating particularly long multi-series time series. The proposed method, SigDiffusion, embeds time series into a Lie algebra and is a diffusion model constructed from the perspective of Lie algebra structure.", "questions": "- What is the claim you want to prove in Table 1, and after looking again at the most advanced methods, please tell us why the comparison you have chosen is appropriate as a way of proving your claim. There may be cases where you change the comparison. Table 1 in [1] will be useful for a list of cutting-edge methods.\n\n[1] Y. Yang et al., A Survey on Diffusion Models for Time Series and Spatio-Temporal Data, arXiv:2404.18886\n\n- Please clarify the definitions of the evaluation indicators in Table 1.\nIf the above points are clarified, I will raise the score.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- This paper proposes a new method for constructing a time series generation model for multiple series. The model method is score-based diffusion, but it is a method that is theoretically robust because it is constructed based on Lie algebra analysis.\n- The proposed method is also very useful for objects with long series lengths.\n- It is constructed with theoretical proof, so I think it will be useful in terms of leading to a wide range of future expansion. Personally, I am very interested in the possibility of applying it to texts and leading to the construction of an LLM.", "weaknesses": "This paper is strong in the theoretical part, and that is important, but on the other hand, the description of the experimental part is poor. It is not clear what the experiment proves in relation to the paper's claim. It is unclear whether it is showing that it is excellent as a generative model for time series, or whether it is showing that it is excellent in score-based diffusion, but since it is being compared with TS-Diffusion, I think it is showing that it is excellent in the overall generative model for multiple series. Since time series generation models are being actively developed, it is necessary to compare the most advanced methods, and it should be explained that appropriate methods have been selected for comparison. There are so many different new methods, and the evaluation metrics and data sets are all different, that it is impossible to judge whether they are producing sufficient results.\nAlso, the Discriminative Score is generally a better indicator when the value is larger, and this can be recognized even by reading the explanation, but it seems that a smaller value is better. I think it would be better to clarify the definition. It seems that the predictive score is defined in the same way as the loss used in training, but the loss in training does not always match the loss in prediction performance, so the definition should be clearly stated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BiTNmCgFX7", "reviewer": "Reviewer_nEHh", "summary": "This paper presents a new score-based diffusion modeling trick for long multivariate time series.\nTheir motivation is to recover the underlying dynamics of the discretized time series data via log-signature embeddings.\nHere, log-signature embeddings corresponds to some algebraic structure of time underlying continuous time dynamics.\nThe key contribution is the derivation of a closed-form for signal recovery.\nThey support their theory and proposal with numerical experiments.", "questions": "Per my understanding, log signature is path integral. A few clarification questions are in order:\n\n* Is log-signature deterministic or stochastic path?\n\n* If it's deterministic, please justify why it's a good idea to use deterministic dynamics in stochastic diffusion process?\n\n* How to justify the robustness of the algebraic structure against noise? Especially it's well-known time series is one of the most noisy data type.\n\n* In `line 182`, why you describe the score function as it's not continuous? In score-based DMs, while the time is discretized for implementation purposes,  score function is continuous.\n\nFrom above, I find it difficult to see the connection between DMs and log-signature embedding in Sec. 2.3.", "limitations": "", "rating": 1, "confidence": 5, "soundness": 2, "presentation": 1, "contribution": 1, "strengths": "This work suggests a new approach to time series generation from signature inversion algorithms. The problem of interest is important. The method employed is refreshing, at least in ML.", "weaknesses": "## **Updated after rebuttal: Changed to strong rejection due to claiming non-original results as their own.**\n\n---\n\nI don't believe I'm the best person to judge this work, as I'm not well-versed in log-signature. However, I am familiar with Lie algebra, group theory, and some algebraic and topological algebra. I find this paper challenging to read and understand, even with my background. The paper feels more like a math paper than an ML paper, especially in the first half. While I appreciate the mathematics, I believe there are valuable insights here, but I suggest the authors polish the presentation to better engage the general ML community.\n\n* Are Thm 2.1 and Thm 2.2 original results? If not, consider treating them as lemmas and specifying the reference in the title. For example, \"Lemma 2.1 (Shuffle Identity, Ree 1958).\"\n\n* Poor motivation. For theory-heavy papers, motivation and intuition are important to communicate with a broader audience, including non-theorists and mathematicians. For example, in Appendix A, while the authors provide some examples, they remain very dry and don't help the reader appreciate the connection between log-signature and ML.\n\n* As a background section, Sec. 2 is very dry and difficult to learn from, beyond memorizing definitions.\n\n* As a methodology paper, the experiments are somewhat limited.\n  -  Ablation. The paper does not include enough ablation studies. Additional ablation studies are needed to support the findings.\n  - For common time series data, why only use exchange rates and weather? Why not include electricity and traffic data?\n\nGiven above, I find this paper not ready for publication. I encourage the authors to improve its accessibility for the broader ML community.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)