PCF-GAN: generating sequential data via the characteristic function of measures on the path space
Authors: Hang Lou
Siran Li
Hao Ni
Venue: NeurIPS 2023 poster (2023)
Keywords: Generative adversarial networks
time series generation
rough path theory
Lie group

Abstract: Generating high-fidelity time series data using generative adversarial networks (GANs) remains a challenging task, as it is difficult to capture the temporal dependence of joint probability distributions induced by time-series data. Towards this goal, a key step is the development of an effective discriminator to distinguish between time series distributions. We propose the so-called PCF-GAN, a novel GAN that incorporates the path characteristic function (PCF) as the principled representation of time series distribution into the discriminator to enhance its generative performance.  On the one hand, we establish theoretical foundations of the PCF distance by proving its characteristicity, boundedness, differentiability with respect to generator parameters, and weak continuity, which ensure the stability and feasibility of training the PCF-GAN. On the other hand, we design efficient initialisation and optimisation schemes for PCFs to strengthen the discriminative power and accelerate training efficiency. To further boost the capabilities of complex time series generation, we integrate the auto-encoder structure via sequential embedding into the PCF-GAN, which provides additional reconstruction functionality. Extensive numerical experiments on various datasets demonstrate the consistently superior performance of PCF-GAN over state-of-the-art baselines, in both generation and reconstruction quality.

PCF-GAN: generating sequential data via the characteristic function of measures on the path space
Hang Lou Department of Mathematics University College London hang. lou. 19@ucl. ac. uk
Siran Li Department of Mathematics Shanghai Jiao Tong University s s14025@nyu.ed edu
Hao Ni Department of Mathematics University College London h. M.ni@uccl ac uk
Abstract
Generating high-fidelity time series data using generative adversarial networks (GANs) remains challenging task, as it is difficult capture the temporal de- pendence of joint probability distributions induced by time-series data. Towards this goal, key step is the development of an effective discriminator to distinguish between time series distributions. We propose the so-called PCF-GAN, a novel GAN that incorporates the path characteristic function (PCF) as the principled representation of time series distribution into the discriminator to enhance its gen- erative performance. On the one hand, we establish theoretical foundations of the PCF distance by proving its characteristicity, boundedness, differentiability with respect generator parameters, and weak continuity, which ensure the stability and feasibility of training the PCF-GAN. On the other hand, we design efficient initialisation and optimisation schemes for PCFs to strengthen the discriminative power and accelerate training efficiency. To further boost the capabilities of com- plex time series generation, we integrate the auto-encoder structure via sequential embedding into the PCF-GAN, which provides additional reconstruction fune- tionality. Extensive numerical experiments on various datasets demonstrate the consistently superior performance of PCF-GAN over state-of-the-art baselines, in both generation and reconstruction quality.
Introduction
Generative Adversarial Networks (GANs) have been a powerful tool for generating complex data distributions, e.g., image data. The original GAN suffers from optimisation instability and mode collapse, partially remedied later by an alternative training scheme using integral probability metric (IPM) in lieu of Sensen-Shanne divergence. The IPMs, e.g., metrics based Wasserstein distances or Maximum Mean Discrepancy (MMD), consistently yield good measures between generated and real data distributions, thus resulting more powerful GANs on empirical data ( (144 24)).
More recently, ( proposed an IPM based on the characteristic function (CF) of measures on Rd which has the characteristic property, boundedness, and differentiability. Such properties enable the GAN constructed using this IPM as discriminator ("CF-GAN") to stabilise training and improve generative performance. However, ineffective in capturing the temporal dependency of sequential data, such CF-metric fails to address high-frequency cases due to the curse dimensionality. To tackle this issue, take the continuous time perspective of time series and lift discrete time series to the path space ([2)) 29 )) This allows us to treat time series of variable length, unequal sampling, and high frequency in unified approach. We propose path characteristic function (PCF) distance to characterise distributions on the path space, and propose the corresponding PCF distance as a novel IPM quantify the distance between measures the path space.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Built on top of the unitary feature of paths ((26), ). our proposed PCF has theoretical foundations deeply rooted the rough path theory (7Z), which exploits the non-commutativity and the group structure of the unitary feature encode information on order of paths. The CF may be regarded as unitary matrix. We show that the PCF
the special case of PCF with linear random path and distance (PCFD) possesses favourable analytic properties, including boundedness and differentiabiiliy in model parameters, and we establish the linkages between PCFD and MMD. These results vastly generalise classical theorems on measures on E (II), with much more technically involved proofs due to the infiiie-diimnniionaltyy
path space.
On the numerical side, we design an efficient algorithm which, by optimising the trainable parameters of PCFD, maximises the discriminative power and improves the stability and efficiency of GAN training. Inspired by 225 41, we integrate the proposed PCF into the IPM-GAN framework, utilising an auto-encoder architecture specifically tailored to sequential data. This model design enables our algorithm to generate and reconstruct realistic time series simultaneously, which has advantages diverse applications, including privacy preservation
(35) and semantic representation extraction for downstream tasks (IIO). To assess the efficacy of our PCF-GAN, conduct extensive numerical experiments on several standard time series benchmarking datasets for both generation and reconstruction tasks.
We summarize key contributions of this work below:
proposing new metric for the distributions on the path space via PCF;
providing theoretical proofs for analytic properties of the proposed loss metric which benefit GAN training;
introducing novel PCF-GAN generate reconstruct time series simultaneously; and
reporting substantial empirical results validating the out-performance of our approach, compared with several state-of-the-art GANs with different loss functions on various time series generation and reconstruction tasks.
Related work. Given the wide practical use of, and challenges for, realistic time series synthesis (3 4) various approaches are proposed to improve the quality of GANs for synthetic time series generation. Several works, e.g., [3 45 36), are devoted to improving the discriminator of GANs to be better suited to distributions induced by time series. Among them, COT-GAN in 4R) shares similar philosophy with PCF-GAN by introducing novel discriminator based on causal optimal transport (which can be seen
improved variant of the Sinkhorn divergence tailored to sequential data), while TimeGAN ((45) shares similar auto-encoder structure, which improves the generator's 's quality and enables time series reconstruction. Unlike PCF-GAN, the reconstruction and generation modules of TimeGAN are separated, whereas it has additional stepwise supervised loss and discriminative loss. In different vein, C GT-GAN [17] COSCI-GAN 39 and EWGAN focus primarily on the design of network framework and generator architecture, which achieve state-of-the-an art results on several benchmarking datasets.
2 Preliminaries
The characteristic function of measure on Rdd, namely that the Fourier transform, plays a central role probability theory and analysis. The path characteristic function (PCF) a natural extension of the characteristic function to the path space.
2.1 Characteristic function distance (CFD) between random variables in Rdd
Let X be an -aluue random variable with the law μ Po X The characteristic function of X, denoted as 4x :Rd -> C, maps each E TRd to the expectation of its complex unitary transform:
Φ + Ex~ [e (A.)). Here Ux IRd + C. x + e ((A..)) is the solution to the linear controlled
differential equation:
(x) iU)())(A dx),
(0 1,
where 0 is the zero vector in d and (.,-) is the Euclidean inner product 4

---

References [TI [6] studied the squared characteristic function distance (CFD) between two R valued random variables X and Y with respect to another probability distribution A on R
CFD = EZ-a fx - (x((2)0y(2)))
(2)
It is proved in 25 1 that if the support of A is Rd, then CFDA is a distance metric, so that CFD (X, = if and only if X and Y have the same distribution. This justifies the usage of CFD as a discriminator for GAN training to learn finte-ddimensional random variables from data.
2.2
Unitary feature of a path
Let BV ((0, T]; Rd) be the space of -valued paths of bounded variation over [0, T-. Consider
(3) For a discrete time series x = (4, =)) where 0 = to < 4 < <.. < Ix = T and x; E TRd we can embed it into some x E X whose evaluation at (4:) coincides with æ.
X' = {x: [0, T] + Rd+1_ x() = (1, x()) for & E [0, x E BV (10, T); IR)) x(0) - 0}
.N,,, ,... This is well suited for sequence-valued data in the high-frequency limit with finer iine-iiscrttisation and is often robust in practice (127 26).. Such embeddings are not unique. In this work, we adopt the linear interpolation for embedding, following [23 [3].
(i {0.....
Let Cm == {m x m complex matrices), ), Im be the identity matrix, and be conjugate transpose. Write (m) and u(m) for the Lie group of m x unitary matrices and its Lie algebra, resp.:
U(m) = {AA CRXmm :A"A = Lm
u(m) = (A E m A + A=
Definition 2.1. Let E BV (10, () be a continuous path and M I - u(m) be a linear map. The unitary feature of x under M is the solution y [0, T] (m) to the following equation:
dy: Yr M(xx)), Yo Im-
We write (x) yr, Le.. the endpoint the solution path.
By slight abuse of notations, UM (x) is also called the unitary feature of x under M. Unitary feature a special case of the Cartan/path development, for which one may consider paths taking values in any Lie group G. We take only G = (m) here; e d in general (16) 30)..
Example 2.2. For M E L ,u(m)) and x E BV (10,T); particular, when m u(1) is reduced iP and M (y) i (AM, v) for some Xas E Rdd
(Rd) linear, (M) Mxx xo), In
Motivated by the universality and characteristic property of unitary features (77) see Appendix A.3) constructed a unitary layer which transforms any d-dimensional time series I (Zo. zN to the unitary feature of its piecewise linear interpolation X. It is a special case of the path development layer (26] when Lie algebra is chosen u(m). In fact, the explicit formula holds: Uss exp (M(Ax:)) where Ar; = Iz -i-1 and exp is the matrix exponential.
Convention 2.3. The space (RR,, ((())) in which M of Eq. (4) resides isomorphic to u(m)4, where u(m) is Lie algebra isomorphic to mim=1 For each E u(m)d given by anti- Herm-iemit J linear map M is uniquely induced: M M(x) >í (00) (x, Vx E RRd.
matrices {
3 Path characteristic function loss
3.1 Path characteristic function (PCF)
The unitary feature of a path x E X' plays role similar that played by (((x)A) to valued random variable. Thus, for a random path X, the expected unitary feature can be viewed as the characteristic function for measures the path space ((7)).
Definition 3.1. Let X be an X-valued random variable and IPx be its measure. The path characteristic function (PCF) of X of order N is the map & x
(m) L (IR u(m)) - Cm given by
tx (M) == (X)M(X)) = (x)
3

---

The path characteristic function (PCF) Φx ① @m=o L (R* u(m)) the natural grading: xx (x = 4b (m) for each m E N.
① m=0 @m is defined by
In the above, UM (x) E ((m) is the unitary feature of the path x under M. See Definition 2.1
Similarly to the characteristic function of - random variables, the PCF always exists. Moreover, we have the following important result, whose proof is presented in Appendix A Theorem 3.2 (Characterissicityy. Let X and Y be Xavuleed random variables. They have the same distribution (denoted as X d Yf and only if 4x DY.
3.2 A new distance measure via PCF
We now introduce a novel and natural distance metric, which measures the discrepancy between distributions on the path space via comparing their PCFs. Throughout, dus denotes the metric associated with the Hilbert-Schmice Schmidt norm IHs on Cm
dus (A, B) = V 14 Blis tr [A BA ()
Definition 3.3. Let X, Y [0, - dRd be stochastic processes and PMM be a probability distribution on u(m)d==: := L (R., u(m)) (recall Convention 2.3) Define the squared PCF-based distance (PCFD) between X and Y with respect to PM as
PCFD Y) = EM-PA ( &y (M))].
(5)
We shall not distinguish between M and P.M for simplicity.
PCFD exhibits several mathematical properties, which provide the theoretical justification for its efficacy as the discriminator on the space of measures on the path space, leading empirical performance boost. First, PCFD has the characteristic property.
Lemma 3.4 (Separation of points). Let X, P(X) and X + Y. Then there exists N, such that if M is -valued random variable with full support, then PCFDM (X, Y) * O.
Furthermore, PCFDM has a simple uniform upper bound for any fixed m E N: Lemma 3.5. Let M be u(m)d-valuee random variable. Then, for any BV (I0, ([,,T): RT) -valued random variables and Y, holds that PCFD) (X,Y) S 2m?.
Under mild conditions, PCFD is a.e. differentiable with respect to a continuous parameter, thus ensuring the feasibility of gradient descent in training.
Theorem 3.6 (Lipschitz dependence P) be a metric space, Q be a Borel probabiiiy measure on 2, and M be a Borel probability measure on umm)". Assume that θ x Z + 2, (0, Z) + go (Z) is Lipschitz in 0 such that Var. [ga (Z) ger (Z)) I w (0, 0²). In addition, suppose that EM-PE [||M]]] < oo
continuous parameter). Let X and 2 be subsets of
BV ([o, TR.) (9,p)
and Ez~q [w) [Z]] < co. Then PCFD M (go (Z), X) is Lipschitz in Moreover, it holds that
|PCFD (ge (Z), X) PCFD (gor (Z). 0). S /EM-PM [|] [[]M[| [Z)] p (0.0")
for any 0, E θ, Z E Z, X E X, and M E P (u(m)²).
Remark 3.7. The parameter space (9, p) is usually taken to be æ for some d E N. In this case, by Rademacher's theorem PCFDM (ga (Z), , X) is a.e. differentiable in 0.
construct metric based PCFD, denoted as PCFD, on the space P(X') of Borel probability measures over the path space, and we prove that it metrises the weak-star topology on P(x). Throughout, 4 denotes the convergence in law.
Similarly metrics measures dRd (cf. 2 24
Theorem 3.8 (Informal, convergence in law). Let ) and X be X-valued random variables
with measures supported in compact subset of X. Then PCFD(X,,. X) 0 X, 4x
The formal statement and proof can be found in Lemma B.2 and Theorem B.S in the Appendix.
Similar to 40 for Rd we prove that PCFD can be interpreted as an MMD with specific kernel (see Appendix B.3). Example B. 12 iillustrates that the PCFD has the superior test power for hypothesis
testing stochastic processes compared with CF distance on the flattened time series.

---

3.3 Computing PCFD under empirical measures
Now, we shall illustrate how to compute the PCFD on the path space.
Let X = {x+)=_ and Y == {y")== be i.i.d. drawn respectively from X'-valued random variables X and Y. First, for any linear map M E u(m)", " the empirical estimator of ( is the average of unitary features of all observations X = {x: i.e. (x = Ei=1 (x,). We then parameterise the
random variable .M via the empirical measure Moz., i.e., Ma = E dM. where Oar [M, E u(m)dxx are the trainable model parameters. Finally, define the corresponding empirical path characteristic function distance (EPCFD) as
EPCED& (X.Y)
2 (Mi) D D (M+)l
(6)
Our approach to approximating M via the empirical distribution differs from that [25]. where M is parameterised by mixture of Gaussian distributions. In $4.1 and $5 it is shown that, by op- timising the empirical distribu- tion, moderately sized & is suf- ficient for achieving superior per- formance, in contrast a larger sample size required by [25]
2
(x)).. 2(0P(0.T)
R
au, U, M, 8, =
manrics a In 3 Ermaatee parameters
A
Aug (g
(r(M) E gp50
Figure 1: Flowchart of calculating the PCF <x
PCF-GAN for time series generation
4.1 Training of the EPCED
In this subsection, we apply the EPCFD to GAN training for time series generation as the discriminator, We train the generator to minimise the EPCFD between true and synthetic data distribution, whereas the empirical distribution of M characterised by 0ы E ([()ddx is optimised by maximising EPCFD,
By abuse of notation, let X' = Rdxer (Z = Rexer, resp.) denote the data (noise, resp.) space, composed of IRd (R resp.) time series of length nt. As discussed in $2.2 X' and Z can be viewed as path spaces via linear interpolation. Like the standard GANs, our model comprised of generator Ge, Z + _dx and the discriminator EPCFD&,, P(x') IP()) - R where 0м u(m kxd the model parameter of the discriminator, which fully characterises the empirical measure of M. The pre-specified noise random variable Z (Zz, ( the discretised Brownian motion on [0, with time mesh 1. The induced distribution of the fake data is given by Ga, (Z). Hence, the "y"
min-max objective of our basic version PCF-GAN is
min max EPCFD,,, (Ga, (Z). X).
8,
We apply mini-batch gradient descent to optimise the model parameters of the generator and discrimi- nator in an alternative manner. In particular, to compute gradients of the discriminator parameter 0xg we use the efficient backpropagation algorithm through time introduced in [26]. which effectively leverages the Lie group-valued outputs and the recurrence structure of the unitary feature. The initialisation of Oar for the optimisation is outlined in the Appendix B.4.1
Learning time-dependent Wrastein-Uhhlebedd process Following 19), we apply the proposed PCF-GAN to the toy example of learning the distribution of synthetic time series data simulated via the time-dependent
(OU) process. Let (X1) be an IR-valued stochastic process described by the SDE, i.e., dX,, (ut AX,) + adB, with X. NoOO, 1), where (Bt) is 1D Brownian motion and N (0 1) is the standard normal distribution. We set μ = 0.01, 0 0.02, σ = 0.4 and time discretisation 51 = 0.1. We generate 10000 samples from = 0 to ( t= 63, down-sampled at each integer time point. Figure shows that the synthetic data generated by our GAN model, which uses the EPCFD discriminator, is visually indistinguishable from true data. Also, our model accurately captures the marginal distribution at various time points.
5

---

Figure 2: Left: Sample paths generated from the time-dependent OU process and synthetic paths from PCF-GAN. Right: The marginal distribution comparison at E {10, 20, 30, 40, 50, 60).
4.2 PCF-GAN: learning with PCFD and sequential embedding
In order to effectively learn the distribution of high- figh-dimensional or complex time series, using solely the EPCF loss as the GAN discriminator fails to be the best approach, due to the computational limitations imposed by the sample size k and the order y of EPCFD. To overcome this issue, we adopt the approach [41 25), and train generator that matches the distribution of the embedding of time series via the auto-encoder structure. Figure 3 iillustrates the mechanics of our model.
To proceed. let us first recall the generator Go Z - X' and introduce the embedding layer Fø,, which maps X to Z (the noise space). Here o, is the model parameters of the embedding layer and will learned from data. To this end, it natural to optimize the model parameters fg of the generator by minimising the generative loss Lgenerator, which is the EPCFD distance of the embedding between true distribution X and synthetic distribution Go (Z); formula,
Lngenerator (6g. 8M, 0y) = EPCFD (Fo, (Ge, (Z)). Fo, (X))).
(7)
Encoder( ) ddecoder(Go, struc-
Reconctruct X,e
Real
ture: The motivation consider the auto-encoder structure is based on the observation that the embedding might be degenerated when optimiz- ing I.generntor- For example, no matter whether true and synthetic distribution agrees or not, Fe, could be simply a constant function to achieve the per- feet generator loss O. Such a degen- eracy can be prohibited if Fe, is in- jective. In heuristic terms, the "good" embedding should capture essential information about real time series of X and allows the reconstruction of time series X from its embedding Fo, (X). This motivates us train the embedding Fo, such that Fo, Go, is close the identity map. If this condition is satisfied, it implies that Fe, and Ga, are pseudo-inverses of each other, thereby ensuring the desired injectivity. In this way, Fo, and Go, serve as the encoder and decoder of raw data, respectively.
Ver
Fxy
GO
Flyy
antedtnn
None 2yp
Reconetruceed muke 2,r
Requdartation los Angeale
Reco struction
Gererator
Figure 3: Visualization of the PCF-GAN architecture
To impose the injectivity of Foy, consider two additional loss functions for training O follows:
Reconstruction loss Ireecnery: It is defined as the 12 samplewise distance between the original and reconstructed noise by Pø, Ga,, i.c., Lrecovery El|2 F'e (Gs, (Z). Note that Lreaweery 0 implies that Fo, (Go, (z)) = 2, for any sample z in the support of Z almost surely.
Regularization loss Treeularizzaion: It is proposed to match the distribution of the original noise variable and embedding of true distribution X. is motivated by the observation that if the perfect generator Go (Z) X and Fe, Go, is the identity map, then Z = F's (X). Specifically,
Luegularizatinn EPCFD& R3 (Z, Fo, (X)),
where we distinguish 8%/ from AN in L Lgenenatoe- The regularization loss effectively stabilises the training and resolves the mode collapse [41) due to the lack of infectivity of the embedding.
Training the embedding parameters Oy: The embedding layer Foy aims to not only discriminate the real and fake data distributions as critic, but also preserve injectivity. Hence optimise the

---

embedding parameter 8 by the following hybrid loss function:
max (ggenerator Arcoowey
-2 Aogulaaaiiiii
By
where A and X2 are hyper- parameters that balance the three losses.
Training the EPCFD parameters (0,, 0/4)): Note that Legeenator and Tregulariration have trainable parameters of EPCFD, i.e., Om and 0's- Similar to the basic PCF-GAN, we optimize Ous and 0yr by maximising the EPCFD to improve the discriminative power.
max generators
max Leegularization
(10)
8
By doing so, we enhance the discriminative power of EPCFD0 and EPCFDg,, Consequently, this facilitates the training of the generator such that the embedding of the true datä aligns with both the noise distribution and the reconstructed noise distribution.
Differentiability of EPCFD with respect to parameters of the embedding layer and generators are guaranteed by Theorem 3.6 as long as Fo, Ge, satisfies the Lipschitz condition thereof. Let us also stress on two key advantages of our proposed PCF-GAN. First, it possesses the ability to generate synthetic time series with reconstruction functionality, thanks to the auto-encoder structure in PCF-GAN, Second, by virtue of the uniform boundedness of PCFD shown in Lemma 3.5 our PCF-GAN does not require any additional gradient constraints of the embedding layer and EPCFD parameters, in contrast to other MMD-based GANs and Wasserstein-GAN. It helps with the training efficiency and alleviates the vanishing gradient problem in training sequential networks like RNNs.
We provide the pseudo-code for the proposed PCF-GAN in Algorithm
Algorithm PCF-GAN.
1: Input: Pa (real time series distribution), P. (noise distribution), 0 00. y. 0g (model parameters for EPCFD, critic F and generator G), λ1, A2 IR (penalty weights), (batch size), 1) E IR (learning rate), ne the iteration number of discriminator per generator update, 0M,,
2: while fM 0. tc, 0g not converge do
3: for E (1...., ng. do
4:
train the unitary linear maps in EPCFD
5: Sample from distributions: ~ Pa. 7 ~ PPz.
6:
Generator Loss: Lygnerator EPCFD& (Fo, (X), Fo, (Ge, (7)))
7:
Update: 0M + 00 + 17 VeN Leenerator
8:
Regularization Loss: Lregalorization
EPCFD
24 (2, Fo, (X))
9:
Update: 8/ + 8'y 17. V (Inegulatizzatonn)
10:
train the embedding
11:
Reconstruction loss: 1:rccovery = E (Go, (2) Loss on critic: La Ligenerator
A1 Liecovory X2 Lregularization
13:
Update: 0, + O + 17- Vo. Le
end for
15:
# train the generator
Pa. P. Generator Loss: Lignnrrtor EPCFDM (X), Fo, (Go, (Z)))
Sample from distributions:
17:
18:
Update: 0g f- 0g Vo,Ly L
19:
end while
5
Numerical Experiments
To validate its efficacy, we apply our proposed PCF-GAN to a broad range of time series data and benchmark with state-of-the-an aht GANs for time series generation using various test metrics. Full details on numerics (dataset, evaluation metrics, and hyperparameter choices) are in Appendix Additional ablation studies and visualisations of generated samples reported in Appendix D
Baselines:
We take Recurrent GAN (RGAN) TimeGAN 45), and COT-GAN 43 as bench-
marking models. These are representatives of GANs exhibiting strong empirical performance for time series generation. For fairness, we compare our model to the baselines while fixing the generators and embeeding/diserrminator to be the common sequential neural network (2 layers of LSTMs).
7

---

Dataset: We benchmark our model on four different time series datasets with various characteristies: dimensions, sample frequency, periodicity, noise level, and correlation. (1) Rough Volatility: High- frequency synthetic time series data with low noise-to-signal (2) Stock: The daily historical data on ten publicly traded stocks from 2013 to 2021, including as features the volume and high, low, opening, closing, and adjusted closing prices. (3) Beijing Air Quality 477 An UCI multivariate
time series on hourly air pollutants data from different monitoring sites. (4) EEG Eye State [38]: An UCI dataset of high frequency and continuous EEG eye measurement. We summarise the key
statistics of the datasets in Table
Table I: Summuary statistics for four datasets
Dataset
Dimension
Length
Sample rate
Auto-cor (lag 1)
Auto-cor flag 5)
Cross-cor
RV
2
200
0.967
0.916
-0.014
Stock
5
20
Iday
0.958
0.922
0.604
Air
10
24
Thour
0.947
0.752
0.0487
EEG
14
20
8ms
0.517
0.457
0.418
Evaluation metrics: The following three metrics are used to assess the quality of generative models. we compare the true and fake/reconstructed distribution by Go, Fo, via the below test metrics. (1) Discriminative score 45]: : We train post-hoc classifier to distinguish true and fake data. We report the classification error on the test data. The hetter generative model yields lower classification error, it means that the classifier struggles to differentiate between true and fake data. (2) Predictive score [45 12: We train a post-hoc
For time series generation/reconstrittto
regression model predict the latter part of a time series given the first part from the generated data. We then evaluate and report the mean square error (MSE) on the true time series data. The lower MSE indicates better the generated data be used to train predictive model. (3) Sig-MMD We use MMD with the signature feature as generic metric on time series distribution. Smaller the values, indicating closer the distributions, are better. To compute three evaluation metrics, we randomly generated 10,000 samples of true and synthetic (reconstructed) distribution resp. The mean and standard deviation of each metric based on 10 repeated random sampling are reported.
D 12
5.1 Time series generation
Table 2 indicates that PCF-GAN consistently outperforms the other baselines across all datasets, as demonstrated by all three test metrics. Specifically, terms of the discriminative score, PCF- GAN achieves a remarkable performance with values of 0.0108 and 0.0784 on the Rough volatility and Stock datasets, respectively. These values are 61% and 39% lower than those achieved by the second-best model. Regarding the predictive score, PCF-GAN achieves the best result across all four datasets. While COT-GAN surpasses OCF-GAN in terms of the Sig-MMD metric on the EEG dataset, PCF-GAN consistently outperforms the other models in the remaining three datasets. Additionally, to assess the fitting on auto-correlation, cross-correlation and marginal distribution, we include the corresponding numerical results in Table in Appendix D.4 For a qualitative analysis of generative quality, we provide the visualizations generated samples for all models and datasets in Appendix D without selective bias. Furthermore, to showease the effectiveness of our auto-encoder architecture for the generation task, we present an ablation study in Appendix D
Table 2: Performance comparison of PCF-GAN and baselines. Best for each task shown in bold.
Task
Generation
Reconstruction
Dataset
Test Metries
RGAN
COT-GAN
TimeGAN
PCF-GAN
TimeGAN (R)
PCF-GAN(R)
Discrininative
.0271+.048
.0499++068
.0827=0019
.0108
5000+
.2820=.082
RV
Predictive
.0393+t.000
.0395++000
.0395=0001
.0390
.000
.0590+.005 000
-0398=.001
Sig-MMD
.016311004
.0116.1.003
.00271.004
.000 .0024
3.308
.0960_..50
Discriminative
.188++0015
4966+.002 -002
.3286+003
.0784 .028
.4433+ .002
.3181 -038
Stock
Predictive
0132-1.000
.014:1..00
.01391.000
.000 .0125
1180:11012
.0127
Sig-MMD
0244-1.008
.0029 .000
.0272.1006
_0017_.000
.75871.186
.0078_.004
Discriminative
45492.012
.992+.0022
.360=.025
.2326=.055
.49944.000 000
.4140=.013
Air
Predictive
.02611.001
.0260.1.001
.02561.000
.0237 .000
.06191.004
.0289-1.00
Sig-MMD
045611.015
.0128:1.02
0146:1.02
.0126_000 .005
A1411.078
.03591.012
Discriminative
4908:1.003
.9311.007
.771
.3660-0225
5000+.000
.959-.003
EEG
Predictive
.0315+.000
.0304+.000
.0342-0001
.0246-000
.0499+.001
.0328-0001
Sig. Sig-MMD .0602+.010
.0102+-002
.0640+ 025
.0180
.0700+-021
.0641--019

---

5.2 Time series reconstruction
As TimeGAN is the only baseline model incorporating reconstruction capability, for reconstruc- tion tasks we only compare with TimeGAN. The reconstructed examples of time series us- ing both PCF-GAN and TimeGAN are shown in Figure see Appendix D for more samples. Visually, the PCF-GAN achieves better reconstruction results than TimeGAN by producing more accurate recon- structed time series samples. No- tably, the reconstructed samples from PCF-GAN preserve the temporal de- pendency of original time series for all four datasets, while some recon- structed samples from TimeGAN in EEG and Stock datasets are completely mismatched. This is further quanti- fied in Table 2 on the reconstruction task, where the reconstructed samples from PCF-GAN consistently outper- form those from TimeGAN in terms of all test metrics.
Real
PCF-GAN
TimeGAN
RV
Stock
Air
EEG
Figure 4: Examples of time series reconstruction using PCE-GAN and TimeGAN.
5.3 Training stability and efliciency
Figure demonstrates the training progress of the PCF-GAN on RV dataset. Compared to the fluctuating generator loss typically observed
16
gone
recovery
in traditional GANs, the PCF-GAN yields bet- 10
ter convergence by leveraging the autoencoder structure. This is achieved by minimising re- construction and regularisation losses, which ensures the injectivity of Fe, and enables pro- duction of semantic embedding throughout the training process. The decay of generator loss in the embedding space directly reflects the im- provement the quality of the generated time series. This is particularly useful for debugging and conducting hyperparameter searches. Fur- thermore, decay in both recovery and regulari- sation loss signifies the enhanced performance of the autoencoder.
-200
10000
15000
20000
25000
30000
Figure 5: Training curves for PCF-GAN of real (left) and generated (right) time series distributions on the Rough Volatility dataset at different training iterations. Plotted by moving average over window with 500 iterations.
By leveraging the effective critic For- we achieve enhanced performance with a moderate increase in parameters (ranging from 1200 to 6400) within Ox of EPCFD. The training of these additional parameters is highly efficient in PCF-GAN, while still outperforming all baseline models. Specifically, algorithm approximately twice fast TimeGAN (using three extra critic modules) and three times as fast as COT-GAN (with one additional critic module and the Sinkhorn algorithm). However, it takes 1.5 times long RGAN due the extra training required on Ow-
6 Conclusion & Broader impact
Conclusion We introduce a novel, principled and efficient PCF-GAN model based on PCF for gener- ating high-fidelity sequential data. With theoretical support, it achieves state-of-the-ar generative performance with additional reconstruction functionality in various tasks of time series generation.
Limitation and future work In this work, we use LSTM-based networks for the autoencoder and do not explore other sequential models (e.g., transformers). The suitable choice of network architecture for the autoencoder may further improve the efficacy of the proposed PCF-GAN more complicated data, e.g., video and skeletal human action sequence, which merits further investigation. distance metric time series, PCFD can be flexibly incorporated with other advanced generators of time

---

series GAN models, hence may further improve the performance. For example. one can replace the average cross-entropy loss used in [7 39 and the Wasserstein distance [66 by PCFD, with some simple modifications the discriminators. Furthermore, although we establish the link between PCFD and MMD, interesting to design efficient algorithms compute the kernel specified in Appendix B.3
Broader impact Like other GAN models, this model has the potential to aid data-hungry algorithms by augmenting small datasets. Additionally, it can enable data sharing in domains such as finance and healthcare, where sensitive time series data is plentiful. However, it is important to acknowledge that the generation of synthetic data also carries the risk of potential misuse (e.g. generating fake news).
Acknowledgments and Disclosure of Funding
The research SL supported by NSFC (National Natural Science Foundation of China) Grant No. 12201 399, and the Shanghai Frontiers Science Center of Modern Analysis. This research project also supported by SL's visiting scholarship at New York Sniveesity-Shanghlee
HN is supported by the EPSRC under the program grant EP/S026347/1 and The Alan Turing Institute under the EPSRC grant EPPN510129// LH is supported by University College London and the China Scholarship Council under the UCL-CSC scholarship (No. 201 201908060002). SL and HN are supported by the SITU-UCL joint seed fund WH610160507/067. HN and HL are supported by the Ecosystem Leadership Award under the EPSRC Grant OobfJ22/ 100020 and The Alan Turing Institute in part. HN is grateful to Jiajie Tao and Zijiu Lyu for proofreading the paper.
10

---

References
[1] Abdul Fatir Ansari, Jonathan Scarlett, and Harold Soh. A characteristic function approach to deep implicit generative modeling. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 1478-7487 2020.
[2] Martin Arjovsky and Léon Bottou. Towards principled methods for training generative adver- sarial networks. arXiv preprint arXiv: 704622 2017.
[3] Samuel A Assefa, Danial Dervovic, Mahmoud Mahfouz, Robert E Tillman, Prashant Reddy, and Manuela Veloso. Generating synthetic data in finance: opportunities, challenges and pitfalls. In Proceedings of the First ACM International Conference on Al in Finance, pages 1-8, 2020.
[4] Steven M Bellovin, Preetam K Dutta, and Nathan Reitinger. Privacy and synthetic datasets. Stan. Tech. Rev., 22:1, 2019.
[5] Lukas Biewald. Experiment tracking with weights and biases, 2020. Software available from wandb.com.
[6] Horatio Boedihardjo and Xi Geng. S1_2 (t)-developments and signature asymptotics for planar paths with bounded variation. arXiv preprint arXiv:2009. 13082, 2020.
[7] llya Chevyrev, Terry Lyons, et al. Characteristic functions of measures on geometric rough paths. The Annals of Probability, 44(6))4049-4082, 2016.
[8] Uya Chevyrev, Vidit Nanda, and Harald Oberhauser. Persistence paths and signature features in topological data analysis. IEEE transactions on pattern analysis and machine intelligence, 42(1):192-2022 2018.
[9] lya Chevyrev and Harald Oberhauser. Signature moments to characterize laws of stochastic processes. Journal of Machine Learning Research, 23(176): 2022.
[10] Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre. Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN encoder- decoder for statistical machine translation. arXiv preprint /406. 1078, 2014.
[11] Kacper P Chwialkowski, Aaditya Ramdas, Dino Sejdinovic, and Arthur Gretton. Fast two- sample testing with analytic representations of probability measures. Advances in Neural Information Processing Systems, 28: 1981-1989, 2015.
[12] Cristóbal Esteban, Stephanie L Hyland, and Gunnar Rätsch. Real-valued (medical) time series generation with recurrent conditional gans. arXiv preprint arXiv:1706.0263. 2017.
[13] Marián Fabian, Petr Habala, Petr Hájek, Vicente Montesinos Santalucía, Jan Pelant, and Václav Zizler. Functional analysis and aefiiit-dimmnsional geometry, volume of CMS Books in Mathematics/Ouwrager de Mathématiques de la SMC. Springer-Verlage New York, 2001.
[14] Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Improved training Wasserstein GANs. Advances in Neural Information Processing Systems, 30:5769-55 5779, 2017.
[15] Ben Hambly and Terry Lyons. Uniqueness for the signature of path of bounded variation and the reduced path group. Annals of Mathematics, pages 109-167, 2010.
[16] Christopher R Heathcote. The integrated squared error estimation of parameters. Biometrika, 64(2):255-264, 1977.
[17] Jinsung Jeon, Jeonghak Kim, Haryong Song, Seunghyeon Cho, and Noseong Park. GT-GAN: General purpose time series synthesis with generative adversarial networks, Advances in Neural Information Processing Systems, 35:36999-37010, 2022.
[18] Patrick Kidger, Patric Bonnier, Imanol Perez Arribas, Cristopher Salvi, and Terry Lyons. Deep signature transforms. Advances in Neural Information Processing Systems, 32:3082-3092, 2019.

---

[19] Patrick Kidger. James Foster, Xuechen Li, and Temry J Lyons. Neural SDEs as infinite- dimensional GANs. In International Conference on Machine Learning, pages 5453-5463. PMLR, 2021.
[20] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 2014.
[21] Achim Klenke. Probability theory: comprehensive course. Springer Science & Business Media, 2013.
[22] Erich Leo Lehmann, Joseph P Rooano, and George Casella. Testing statistical hypotheses, volume 3. Springer, 2005.
[23] Daniel Levin, Terry Lyons, and Hao Ni. Learning from the past, predicting the statistics for the 2013.
future, learning evolving system. arXiv preprint
[24]
Chun-Liang Li, Wei-Cheng Chang, Yu Cheng, Yiming Yang, and Barnabás P6czos. MMD GAN: Towards deeper understanding of moment matching network. Advances in Neural Information Processing Systems, 1112200022100 2017.
[25] Shengxi Li, Zeyang Yu, Min Xiang, and Danilo Mandic. Reciprocal adversarial learning via characteristic functions. Advances in Neural Information Processing Systems, 33:217-228, 2020.
[26] Hang Lou, Siran Li, and Hao Ni. Path development network with finitediimensional Lie group representation. arXiv preprint arXiv:2204.00740, 2022.
[27] Terry Lyons. Rough paths, signatures and the modelling of functions streams. arXiv preprint arXiv:1405.4555 2014.
[28] Teny J Lyons. Differential equations driven by rough signals. Revista Matemática Iberoameri- cana, 14(2):215-310, 1998.
[29] Terry J Lyons, Michael Caruana, and Thierry Lévy. Differential equations driven by rough paths. Springer, 2007.
[30] Terry J Lyons and Weijun Xu. Hyperbolic development and inversion of signature. Journal of Functional Analysis, 272(7):9933-2955, 2017.
[31] Hao Ni, Lukasz Szpruch. Mare Sabate-Vidae Vidales, Baoren Xiao. Magnus Wiese, and Shujian Liao. Sig. Wasserstein GANs for time series generation. In Proceedings of the Second ACM International Conference on Al in Finance, pages 1-8, 2021.
[32] Hao Ni, Lukasz Szpruch, Magnus Wiese, Shujian Liao, and Baoren Xiao. Conditional sig- wasserstein gans for time series generation. arXiv preprint arXiv:2006.05422 2020.
[33] Kalyanapuram R. Parthasarathy. Probability measures on metric spaces, volume of Probability and Mathematical Statistics. Academic Press, Inc., New York-London, 1967.
[34] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury. Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An Imperative Style, Pigh-Peerormmace Deep Learning Library. In Advances in Neural Information Processing Systems 32, pages 8024-8035. Curran Associates, Inc., 2019.
[35] Vibhor Rastogi and Suman Nath. Differentially private aggregation of distributed time-series with transformation and encryption. In Proceedings of the 2010 ACM SIGMOD International Conference on Management of data, pages 735-746, 2010.
[36] Carl Remlinger, Joseph Mikael, and Romuald Elie. Conditional loss and deep euler scheme for time series generation. Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 8105, 2022.
12

---

[37] Jinfu Ren, Yang Liu, and Jiming Liu. EWGAN; Entropy- based Wasserstein GAN for imbal- anced learning. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 1001 1-10022, 2019.
[38] Oliver Roesler, Lucas Bader, Jan Forster, Yoshikatsu Hayashi, Stefan Heßler, and David Suendermann-Oers Comparison of eeg devices for eye state classification. Proc. of the AIHLS, 2014.
[39] Ali Seyfi, Jean-Francois Rajotte, and Raymond Ng. Generating multivariate time series with COmmon Source CoordInated GAN (COSCI-GAN). Advances in Neural Information Process- ing Systems, 35:32777-327888, 2022.
[40] Bharath Sriperumbudur, Arthur Gretton, Kenji Fukumizu, Bernhard Schölkopf, and Gert RG Lanckriet. Hilbert space embeddings and metrics on probability measures. The Journal of Machine Learning Research, 11:1517-1561, 2010.
[41] Akash Srivastava, Lazar Valkov, Chris Russell, Michael Gutmann, and Charles Sutton. VEEGAN: Reducing mode collapse in GANs using implicit variational learning, Advances in Neural Information Processing Systems, 30:3310-3320, 2017.
[42] Csaba Toth and Harald Oberhauser. Bayesian learning from sequential data using Gaussian processes with signature covariances. In International Conference Machine Learning, pages PMLR, 2020.
[43] Tianlin Xu, Li Kevin Wenliang, Michael Munn, and Beatrice Acciaio. COT-GAN: Generating sequential data via causal optimal transport. Advances in Neural Information Processing Systems, 33:8798-8809, 2020.
[44] Yasin YaNııı, Chuan-Sheng Foo, Stefan Winkler, Kim-Hui Yap, Georgios Piliouras, and Vijay Chandrasekhar. The unusual effectiveness of averaging in GAN training. International Conference Learning Representations, 2011. 2019.
[45] Jinsung Yoon, Daniel Jarrett, and Mihaela Van der Schaar. Time-series generative adversarial networks. Advances in Neural Information Processing Systems, 32:5509- 14.5509-5519, 2019.
[46] Kósaku Yosida. Functional analysis. Sixth edition, volume 123 of Grundlehren der Mathema- tischen Wissenschaften [Fundamental Principles of Mathematical Sciences]. Springer-Verlag Berlin-New York, 1980.
[47] Shuyi Zhang, Bin Guo, Anlan Dong, Jing He, Ziping Xu, and Song Xi Chen. Cautionary tales on qir-quality improvement in beijing. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 204447777 2017.
[48] Robert Zimmer. Essential results of functional analysis. University of Chicago Press, 1990.
13

---

we collect some notations and properties for paths and unitary feature of path. Appendix [B] gives a thorough introduction to the distance function via the path characteristics function. Detailed proofs for the theoretical results on PCFD are provided. Appendix C discusses experimental details and Appendix D presents supplementary numerical results.
In Appendix A
A Preliminaries
A.1 Paths with bounded variation
Definition A.I. Let X [0, Rd be continuous path. The total variation of X on the interval [0,7] defined by
Σ Xte -xx2 .1
Tot. VXr.(X)
sup DC(0,
Xt2
(11)
where the supremum is taken over all finite partitions D {te) of [o, T). When Tot. Var.(X) is finite, say that X is path of bounded variation (B (Byptth on [0, and denote X E BV (Io, [[: R*).
B V-pattss can be defined without the continuity assumption, but we shall not seek for greater generality in this work. well-known that
|| |BBv := || (1.77) + Tot. (Xr.
defines norm (the BV-norm). There is a more general notion of paths of finite p-variation for corresponds to BV-paths discussed above. We restrict ourselves to 1, as this is sufficient for the study of sequential data in practice as piecewise linear approximations of continuous paths.
p 2 (see 22 ). where the case p
Definition A.2. (Concatenation of paths) Let X [0, + 12d and Y [s + R be two continuous
paths. Their concatenation denoted as the path Y [], + Id is defined by
Xuu: Y Y, X,, u [s, t].
[0, s].
(X + Y)
Definition A.3 (Tree-like equivalence), A continuous path [0. + Rd is called tree-like there is an R-tree T, a continuous function Φ [0, T) T, and function &b :T - Rd such that (0)) = (T) and X = &b Φ.
Let X [0, 7] JR denote the time-reversal of continuous path X, namely that T(() X(T t).
We say that X and are tree-like equivalence (denoted as X ~rp Y) if X * Y is tree-like.
Y. That is, for X e BV ((0, T]; It)), take nondecreasing surjection À [0, + [Ti, T2], and take X(I) (A(I)).
An important example is when path X is a time re-parameterisation
A.2 Matrix groups and algebras
The unitary group and symplectic group are subsets of the space of m m matrices:
U (mm) = {A E Cm A A = AA" Im Sp(2m, C) = {A E 222x2m A Jm A = Jm
where Jm : (-hm and Im E mm the identity. Their corresponding Lie algebras are 0 Im
u(m) = {A Cmxm A" + A= 0}.
sp(2m, C) == {A 22mx2m Jm + JmA= 0}.
The unitary group is compact and is group of isometries of matrix multiplication with respect to the Hilbert-Schii Schmidt norm. Such properties are crucial for establishing theorems and properties related to the path characteristic function (PCF), as discussed in subsequent sections.
The compact symplectic group Sp(m) is the simply-connected maximal compact real Lie subgroup of Sp(2m, C). It is the real form of Sp(2n, C), and satisfies
Sp(m) Sp(2m, C) n (2m).
Note that U (mm) and Sp(m) are both real Lie groups, albeit they have complex entries in general.
14

---

A.3
Unitary feature of a path
Recall Definition 2.1 for the unitary feature, reproduced below:
Definition A.4. Let M Rd + u(m) be a linear map and let X E BV (); (2) be BV-path.
The unitary feature (a.k.a. the path development on the unitary group U(m) ] of x under M the solution to the equation
dy: = Yr (dxx)
for all t E [0, with Yo = 4m
We write UM (x) yT.
Definition 2.1 is motivated by 7 $4]. Consider M E L (IR*, u(Haa) with Hia ranging over all fmite-dimensional complex Hilbert spaces. Extend M by naturality to the tensor algebra over R4, that is, define M : ((R))) = Dk-o (I2) 8k + u(Haa) by linearity and the following rule:
M1 ... 2 Uk)) == ((1) M (1k) for any k N and Ui...., Uk Rd
Then denote by A ( the totality of such M. Any element in A (R.) is unitary representation of the Lie group 9 ((R)) grooupllkke elements in T (. See 77] p.4059].
The following two lemmas are contained in 26
Lemma A.5. [Multiplicativity)eee Let X E BV (I0, s) Rd) and Y BV ([s, t), 12). Denote by Y their concatenation: (v) (w) for u E [0, s) and - + X() for [s, t). Then (M Y) (X) Mas for all M L [R4 u(m)).
We shall compute by Lemma A.5 and Example 2.2 the unitary feature of piecewise linear paths.
Lemma A.6 (Invariance under time-reparameerristtoon) Let X (V T), T),2d) and let : t+ A be non-decreasing C dddffeeomorphism from [0, T] onto (), Define X² == Xx, fort E [0, T7. Then, for all M E C (TRd, u(m)) and for every 8, E [0, T), , it holds thut UM (Xx,,a)) Uns (X2.)=
A key property of the unitary feature is that it completely determines the law of random paths: Theorem A.7 (Uniqueness of unitary feature). For any two paths X1 * X2 in X, there exists M E C [Rd, u(m)) with some m E such that Uu (X1) # Uyx((2))
Proof. For X, *X₂ in X, by uniqueness of signature over VV-paths (cf. 15 one has Sig(X,) o Sig(X2) in & (RR)). Here we use the fact that the signatures of BV-paths are group-like elements in the tensor algebra. Then, as A (Rd) separates points over G ((Re)) (ef. 7 Theorem 4.8)), there is M E (RR u(m) such that M [1) # M [Sig(X2)]: hence M (X,)) * (X2). Therefore, by considering the (m))-vluued equation dY, Y, M(X,)) with Yo J... conclude that (Xi) & Uy (X2).
Theorem A.8 (Universality of unitary feature). Let K. C BV (10,T); Rd) be compact subset. For any continuous function f K. and any > O, there exists an m, N and finitely many M1 MN E L (R3 uumm)) as well as Li,..., LN (U(m.); C), such that
sup Xek
f(X)
Σ
L Lo (X) <E
(12)
Proof. It follows from 26 Theorem A.4] and the universality of signature in 8 that Eq.
12
holds with M, E L [ + EmE u(m)) and Lj E (
((mmy men Umm): and c/2 in place of c. By
simple approximation via restricting the ranges of M, and domains of Lj, may obtain (without
relabelling) M, C (R. + = u(m)) and L, E L ( - U(m); C) that verify Eq. (12).
We
conclude by the flag structure of U(L (2) U(3)
and u(l) u(2) u(3) C.... D
15

---

B Path Characteristic loss
B.1 Path Characteristic function
Theorem B.1. Let X be X-valued random variable with associated probability measure Px. The path characteristic function $ x uniquely characterises Px.
Proof. Assume that Px, o Xx Then Sig(X1) + Sig(X; by the uniqueness of signature over V--paths (ef. (5).. It is proved in 126 Lemma 2.6] that UM (X;) = .M (Sig(X.)) for any M L (RA a(m)) i E (1,2). Hence Φx, Jx M (Sig(x)) dlxx, (x). But as in the proof of Theorem A.7 A (Rd) separates points over 9 (RR)) (cf. 7 Theorem 4.8]) and the signature of
BV-paths lies G ((Rd). Therefore, there an M E C (IE", u(m)) such that 4x # Dy.
B.2 Distance metric via path characteristic function
Lemma B.2. PCFDM in Eq. b defines a pseudometric the path space X for any N and M E P (C (RR u(m))). In addition, suppose that {M}} is a countable dense subset in P (L RR4. + men u(m)))) Then the following defines metric on X:
min {1, PCFDM, (X, X)
PCFD(X,Y) Y):
(13)
2
In Lemma B.2 above, (R. + u(m)) = (R")) 0x (4mED u(m)) where Øx is the com- pletion of the projective tensor product and DmEN u(m) Banach space under the norm ||T'|| := Emen [ fore, such a sequence [MJ)jeN always exists since P (C (RR +EEN u(m))). being the space of Borel probability measures over Polish space, is itself Polish space. See 33
HS < oo. Here T(m) denotes the mh -poopreectioon of T on u(m). There-
Proof. Non-negativity, symmetry, and that PCFD X) 0 are clear. That PCFD (X. Y) PCFDM Z) + PCFD,M (Z, Y) follows from the triangle inequality the Hilbert-Schmine norm and the linearity of expectation. This shows that PCFD,M is a pseudometric for each M.
In addition, PCFDM(X, Y)
0 implies that
S
dis x (M), By (M)) (PM))PPM(MM)) (M)
=f ((()))(m))
[E
E [M (Y) 2 dl (M) 0.
So, if PM is supported on the whole of L (TRd u u(m)) then xx कy (M) for any M P.M Now, by density of {My))eN in P (Rd + meN u(m))), there exists subsequence ( (Mj(m)) such that M Mj(m) has full support on L (IRd, u(m)) for each m E N. Thus, PCFD(X, Y) = implies that 4x = By on a dense subset of L (Rd u(m)) for every E N. We conclude by the characteristicity Theorem 3.2 and continuity argument.
Lemma B.3 (Lemma 3.5 Let M be an L (RRd, u(m))--vaued -valued random variable. Then for any BV (10, T):
) -valued random variables X and Y, it holds that
PCFDM Y) 2m?
Hilbert space, from the Pythagorean theorem one deduces that Lis (Px(M), (M)) < x (M) + y (M) H Both & xx &y (M) are expectations of valuue random variables, and [HI = Vr(UU*) = /tr(Im) vm for U E U(m..
Proof. As (Cmxm Hs))
Thus das (x(M), (MM)) 22m. We take expectation over P.M to conclude.
16

---

The result below is formulated in terms of the -cheert- norm of matrices in C Any other norm on C is equivalent to that, modulo constant depending on m only. In fact, the strict inequality || I [| for T E C holds. See, e.g. 48 Lemma 3.1.10, p.55].
Lemma B.4. Let L, L |a, b] + Ra be two linear paths, and let M E u(m)d == (Bd u(m)) as before. Denote by || lle the usual Euclidean norm on IR and ||M]]| the operator norm of M (R. (le) + (u(m), (|Hs). Then we have
|oncceee) - eM((())
- <||aM|||| [(t)) (i(t)-((t)]
I(t)l1
for each E [a,
Proof. Let st, = M (1 ((1-s) + +[(() with t [a, and E [0, 1]. This is the linear interpolation between [tt, 0) M o L(t) and [ 1) M L(t). Then we have
1.1/ De
| - MI(0)
(1, ds
IH
Os
Ør 1/-0 [// c(1-r)(((1-8)
Os (t, s)e" seee) dr
HS
thanks identity for differentiation of matrix exponential and the inequality [[T]] | <
[[Ti T2 |us. Here ((1-r))(t1--)) and ((r,s) take values in U (m)) hence of operator norm 1 for any parameters 1, 1,8,7 r. So we infer that
( -e ME(
< $ L.
σr (1,5)
Jo Jo Ōs
dr ds
HS
HS
= =||M M (i(() E(1)
< [IMMI| -M0)
HS
where the first inequality holds for Bochner integrals. See
46 Corollary 1, p.33..
Lemma B.5 (Subadditivity of unitary feature). Let X,Y E BV (00T); R be BV-paths, and Usa be the unitary feature associated with M E L (RRd u(m) = u(m)4. For any 0 < t < T we have - Us (Yo (Xe,r) Un (Y,,r)
(X) (Y) S (Xo,r(
Proof. We apply the multiplicative property of unitary feature Lemma A.3 the triangle inequality, and the unitary invariance of the Hilbert-Schmidt norm to estimate that
[ Uar (Y)
M (Xo,e) Uas (X,,)) Uas Uss (Ye,)
< [1Mss (Y),))) (X,, + [V. Uss (Y,,))
MMs - -Y, | (Xe,)) Um (l
Proposition B.6. For X, Y X, the unitary feature (M with M E u(m)) u(m)d
satisfies
M (X) Us (Y) S [|||I|| Tot. X X -Y- Y).
where Tol. Var.
Y) denotes the total variation over [0, T) of the path X Y.
Proof. Given BV-paths X, Y with the same initial point, there are piecewise linear approximations [X"]. [Y"] with common partition 0 = to < t1 ... < tr = T converging respectively to X and Y in the variaaiion metric; p (1,2). Applying Lemma B.5 recursively, obtain that
(X") - UUr(Y"))l
Unr (Xi....)) - Uar (Y. (Y
17

---

By definition of unitary feature and Lemma B.4
one deduces that
(X (Y S
|I
X
Y
1=0
We may now conclude by taking supremum over all partitions and sending n - oo with the limit. This is a consequence of continuity of the Itó map with respect the driving path in p-variation topology, since the vector field in (4) is Lipschitz ([27) Theorem 1.20)).
Theorem B.7 (Dependence on continuous parameter). Let X and Z be subsets of BV ((0, TI: R..). p). be a metric space, be Borel probability measure Z, and M be a Borel probability measure u(m)". Assume that g θ 2 - X. (0. Z) go (Z) is Lipschitz in A such that Tot. Var. Lgg( ( 9e (Z)] I w ( (0, (²). In addition, suppose that E EMM [[]] < oc and
Ez~q (Z) < cc. Then PCFD (go (Z). X) is Lipschitz in 0.
Moreover, it holds that
JPCFDM (go (Z).X) PCFD, M (go' (Z). S BMMPPM [[]M]]]]] Ez~e [x()]] P (0,
for any 0, 0' E 0 Z 2, X X, and M E P (u(m)").
Proof. As M a pseudometric (Lemma B.2), we have
PCCDD M (ge (Z), X) PCFDM (gg x)) I PCFD.M (go(Z).go (Z)).
We may control the right-hand side follows, using subsequentially the definitions of PCFD and PCF, Proposition B.6 and the assumptions this theorem:
PCFD M (go (Z),9o" Ha (Z))
|| || (o()))(M) $ (z) HS dPM
Jijmaa
Sa Jammyaa 1.fo (llss (go(Z)) Us (go (go)l d(Q(())
2
dPM
HS
dPM } 1
Jacan |1MMI12 V Tot. Var. [go(Z) go (Z)) dQ(2))
{ (Z)) (())))))))))(())
S EM [IIM
This completes the proof.
The unitary feature is universal in the spirit of the Chone-Weierstrass theorem; i.e., continuous functions paths can be uniformly approximated by linear functionals on unitary features.
As PCFD metrises the weak topology on the space of path-valued random variables, emerges as a more sensible distance metric for training time series generations than metrics without this property; e.g., the Jensen-Shannon divergence.
Theorem B.8 (Metrisation of weak star star topology). Let K. C X be a compact subset. Suppose that {Mi) jeN is countable dense subset in P (C (Rdd Dmen u(m))) Then PCFD defined by Eqn. 13
metrises the seak-star topology on P(C). That is, PCFD(X... ~
X, as oo.
where 4 denotes convergence in distribution of random variables.
The metrisability of (P(C) follows from general theorems in functional analysis: K. is compact metric space, hence (KC separable [13 Lemma 3.23]). Then, viewing P(K)) as the unit circle
(C)) via Riesz representation, we infer from [13 Proposition 3.24) that P(K.) is metrisable in the stak-star topology, which is equivalent to the distributional convergence of random variables.
18

---

Proof. The backward direction is straightforward. By the Riesz representation theorem of Radon measures, the distributional convergence equivalent that Jx. fdll'x. Jx. fdlIx for all continu-
ous f E C(C). Thus [| Ma xl fxc Mas dF x is - 0, namely that xx. [M] - o x [M] for each M E & (R4; u(m)). The unitary feature Mas is bounded as it is Umm)-alled for some so
deduce from the dominated convergence theorem that PCFD(X,, X) +
Conversely, suppose that PCFD(X,, X) - Then
Secesudtn 1/e Uns dPx,
{ Un dPx
dF dF (M) =
HS
for any m E N and ME P ( in particular for those with full support. In view of the universality Theorem A.8 proved above, for any fixed e > 0 and any continuous function f E (K.), by approximating f with sum of finitely many L, o MM, (the notations are as in Theorem A.8), one infers that for and m, sufficiently large, it holds that
(())))(m.)) 1/(
f fdPx.
fdFx dPM(MM <E
1x
By considering those measures with spt( ((R),((,)). we deduce that
lim 5 - -Se dFx =0
fdFxx
for any f E (K))
This is tantamount the distributional convergence.
Proof. We first prove the 'if' direction of the statement. By the Portmanteau theorem 21 conver-
gence in distribution X,, 4 X implies, for any bounded continuous map f. we have En-p, [f()]] Egsp [=)]. Therefore, for any M E C(Rd, u(m)), En-P, (x)] + Ez-z [uy (x))))))))) which im- plies Pxx (M) 4x (M) lis + 0 as n + cc. Hence, it follows that, as + 0c,
PCFD X) == EM~PAM xxx, (M) - 2 (M )llss +,
which completes the proof of 'if" direction.
Now proceed with the 'only if' direction. By the universality of the unitary path development from Theorem A.8 for any continuous function f - C and e > 0, there exist M1,- ..,.... MN e
[(Rd, u(m) and Li,.... LN E C (U(m); C) such that
N
Exwe [f(x)]
Σ 1=1
La Eawp MMs (2)) <E
(14)
or equivalently Eawe [f(x)] - EN L, xx Egap as En and IE respectively. Therefore,
< E. For simplicity, we denote EgsP, and
N
N L, o tx (Mi) + =[f((x)] - Σ L. ox (Mi)
E, [f(2)) - [f(+)]] < E, [f(z)] - Σ
7m
(15)
+
Li o ( (0x, (Mi) xx())))
(16)
-=1
< 2e +
Li |xx (M:) 4x (Mi)ll
(17)
where [Top = supzeu((m)\0
[xxx] the operator norm Since PCFD X) = [0] its
EM-PAMl 1 (M) Φ [ + 0 as n + oo and is arbitrary, Ez-z, (f()) + Ez- (x)
for any continuous bounded function ʃ C, which implies X,, 4 X by the Portmanteau theorem (21..
19

---

B.3 Relation with MMD
We now discuss linkages between PCFD and MMD (maximum mean discrepancy) defined over P()), the space of Borel probability measures (equivalently, probability distributions) on A' Definition B.9. Given a kernel function & : X' x X + R, the MMD associated to & is the function MMD,: P(X) x P(x) + R given as follows: for independent random variables X, Y on X, set
MMD (Px: PY) E
X.X xx (xXXX')) E Y (N,YY, Y 2Xx Y. [n(X,) Y)
The PCFD can be interpreted as an MMD on measures of the path space with a specific kernel. Compare with [40] for the case of ld
Proposition B.10 (PCFD as MMD), Given M E P (u(m)²) and X-valued random variables X and Y with induced distributions Px and Py, resp. Then PCFDM(A Y) = MMD, (Px, Py) [tr (Mar (x y))]-
with kernel x(x,y) == RMMP [[] (M)r(y))llnn
EMMPM
Throughout, * designates concatenation of paths and y is the path obtained by running y backwards. The operation y on the path space is analogous to x y on Rd If x, then x y is the null path. See the Appendix for proofs and further discussions.
Remark B.11 (Computational cost complexity). By Proposition B. 10 PCFD is an MMD. However, to compute EPCFD, we may directly calculate the expected distance between the PCFs, without going over the kernel calculations in the MMD approach. Our method is significantly more efficient, especially for large datasets. The computational complexity of EPCED is linear in sample size, whereas the MMD approach is quadratic.
Proof. By definition of PCFD, we have
PCFD v) EM
[ (M()
-EMABR [I + 1 (M) 2 by Hs)
where (M) Ex-julMrr (M)r (X) and f
Bx~ulMM(()) respectively. Using Fubini's theorem and observing that (M), Ty (M)) E (P(,MM) (as 4x (M) and oy (M) are U(m) valued, they indeed lie in 0 (Cmxm; PM) as (m) compact Lie group under the Hilbert- Schmidt metrie), we deduce that
EM x (M), 4y (M) = Ex~
(MM (X), UM (M)))))))))))))))))))(
The first equality then follows from the identification y( EM (x), (V)) and the definition of MMDx.
On the other hand, by Lemma A.5 and the definition of the Hilbert-Schmidt inner product on U(m). one may rewrite the kernel function as follows:
A xx y) RMMPPM (x), (y) ()))))
EM~PM [tr(Uls (x) Und (y))] = EM~PA [tr (Uis (x * F))]
where * denotes the concatenation of paths. The second equality now follows.
B.4
Empirical PCFD
B.4.1 Initialisation of .M
linear map M L (Rd, u(m)) can be canonically represented by d independent anti-Hermitian ma- trices MI,..., Ma C u(m) E Cmxm, To sample empiracal distribution of M E P [ (Rd u(m))] from P.. we propose sampling scheme over u(m). This can also be used as an effective initialisa- tion of model parameters da E dx for the empirical measure of M.
In practice, when working with the Lie algebra u(m), iee, the vector space of m X m complex-valued matrices that are each meri-Herm matrix as an 2m x 2m real matrix via the isomorphism of R-vector spaces p 2mm & cmxm.
(A* + A = O, where A is the transpose conjugate of A), we view
20

---

Under the above identification, we have the decomposition
u(m)= = (m) v-1 (Sym ((m) @ (1)(m,
(18)
where (m) is the Lie algebra of anti-symmetric m x m real matrices, mxm is the space of m x m real symmetric matrices, s(m) consists of m x m real diagonal matrices and mxmm denotes the quotient space of real symmetric matrices by the real diagonal matrices.
The sampling procedure of IP, MM. is given as follows. First, we simulate Rm*m valued and i.i.d random variables A and B, whose elements are i.idd and satisfy the pre-specified distribution in ( We have the decomposition B = D @ E, where D and E are a diagonal random matrix and off-ddagoonl random matrix respectively. Then we construct the anti-symmetric matrix R -22 A)
and matrix in the quotient space Symmxm/ /m()) C 12 (E" + E), and diagonal matrix D.
Correspondingly, we simulate uvm)-valued random variables by virtue of Eq. (18). As the empirical measure of the M can be fully characterised by the model parameters As E ((m))xx we sample d x k i.i.d. samples which take values in u(m).
B.4.2 Hypothesis test
In the following, illustrate the efficacy of the proposed trainable EPCFD metric in the context of the hypothesis test on stochastic processes.
Example B.12 (Hypothesis testing on fractional Brownian motion). Consider the 3-dimensional Brownian motion B == (Bi) and the fraction Brownian motion Bh ;= =BB)) (e(oo)) with the Hurst parameter h. We simulated 5000 sample paths for both B and Bh with 50 discretized time steps. We apply the proposed optimized EPCFD metric to the two-sample testing problem: the null
hypothesis Ho B - Bh against the alternative H1 B Bh. We compare the optimized EPCFD metric with EPCFD metric with the prespecifted distribution (PCF) and the characteristic function functiioiiiiniiiin distance (CF) on the flattened time series (25g. The optimized PCFs are trained on a separale set of 5000 training samples to maximise the PCFD. The details of training can be found at Appendix C.2
We conduct the permutation test compute the power of a test (i.e. the probability of correct rejection of the null Ho) and Type error (i.e. the probability of false acceptances of the null Ho) for varying Note that when h = 0.5, B and Bh have the same distribution and hence are
h E {0.2 + 0.1- i indistinguishable Therefore, the better the test metric is, the test power should be closer to when is close to 0.5, whereas it should be closer depth information on hypothesis testing and permutation test statistics.
when h is away from 0.5. We refer [22] for more
The plot of the test power and Type I error in Figure shows that CF fails in the two sample tests, whilst both EPCFD and optimised EPCFD can distinguish the samples from the stochastic process when h + 0.5. It indicates that the EPCFD captures the distribution of time series much more effectively than the conventional CF metric. Moreover, optimization of EPCFD increases the test power while decreasing the typel error, particularly when h is closer to 0.5.
C Numerical experiments
C.1
Experimental detail on PCF-GAN
C.1.1 General notes
Codes. The code for reproducing all experiments can be found in https://githut: com/ SeeplntoStreeassPccF-O00
Software. We conducted all experiments using PyTorch 1.13.1 34 and performed hyperparameter tuning with Wandb [5]. To ensure reproducibility, we implemented benchmark models based on
open-source code from 45 43 12]
We used the Ksig library [42 to calculate the Sig-MMD metrics.
The codes in 25 were used to compute characteristics function distance in Example B.12
Computing infrastructure. The experiments were performed on computational system running Ubuntu 22.04.2 LTS, comprising three Quadro RTX 8000 and two RTX A6000 GPUs. Each experiment was run independently on single GPU, with the training phase taking between hours to days, depending the dataset and models used.
21

---

10
PCF Opt zed PCT
08
06
0.6
power
typel
04
02
02
0.0
Figure 6: Plots of the test power (Left) and the Type-I error (Right) against the Hurst parameter E [0.2, 0.8] on the two sample tests for the Brownian motion B against Fractional Brownian motions (B") by using three metrics, i.., PCFD, optimized EPCFD and CFD.
Architectures. To ensure a fair comparison, we employed identical network architectures, with two layers of LSTMs having 32 hidden units, for both the generator and discriminator across models. For the generator, the output of the LSTM (full sequence) function and a linear output layer. All generative models take a multi-dimen ditieniiiniinl discretized Brownian motion as the noise distribution, scaling it to ensure values were controlled within the range [-1, The dimension and scaling factor varied based on the dataset and were specified the individual sections below.
passed through Tanh activation
The PCF-GAN uses the development layers on the unitary matrix 26 to calculate the PCFD distance. For all experiments, we fixed the unitary matrix size and coefficient X2 for the regularization loss to 10 and I, respectively. The number unitary linear maps and the coefficient A of the recovery loss were determined via hyper-parameter tuning, which varied depending the dataset (see individual section for details).
Regarding TimeGAN, the following approach described in 45 and employed embedding, supervisor, and recovery modules. Each of these modules had two layers of LSTMs with 32 hidden units. For COT-GAN, we used two separate modules for discriminators, each with two layers of LSTMs with 32 hidden units. Based on the recommendation from COT-GAN 4 and informal hyperparameter tuning, we set À 10 and
for all experiments.
Optimisation & training. We used the ADAM optimizer for all experiments 22), with learning rate of 0.001 for both generators and discriminators. The learning rate for the unitary development network is 0.005. The initial decay rates in the ADAM optimizer are set (3 0, 82 0.9. The discriminator was trained for two iterations per iteration of the generator's 's training. For TimeGAN, we followed the training scheme for each module as suggested in the original paper. The batch size was 64 for all experiments. These hyperparameters do not substantially affect the results.
To improve the training stability of GAN, employed three techniques. Firstly,, applied constant exponential decay rate of 0.97 to the learning rate for every 500 generator training iterations. Secondly, we clipped the norm of gradients in both generator and discriminator to 10. Thirdly, used the Cesaro mean of the generator weights after certain iterations to improve the performance of the final model, suggested by [44] In all cases, we selected the number of training iterations such that all methods could produce stable generative samples. The optimal number of training iterations
and weight averaging scheme varied for each dataset. More details can be found in the respective sections.
Test metries. Discriminative score. The network architecture of the post-hoc classifier consists of two layers of LSTMs with 16 hidden units. The dataset was split into equal proportions of real and generated time series with labels 0 and 1, with 80% 20% train/test split for training and evaluation. The discrininative model was trained for 30 epochs using Adam with a learning rate of 0.001 and batch size of 64. The best classification error the test
reported.
Predictive score. The network architecture of the post-hoc sequence-to-sqquee regressor consists of two layers of LSTMs with 16 hidden units. The model was trained on the generated time series
22

---

and evaluated on the real time series, using the first 80% of the time series to predict the last 20%. The predictive model was trained for 50 epochs using Adam with learning rate of 0.001 and a batch size of 64. The best mean squared error on the test set was reported.
Sig-MMID. We directly computed the Sig-MMD by taking inputs of the real time series samples and generated time series samples. We used the radial basis function kernel applying to the truncated signature feature up to depth 5.
C.2 Time dependent Ornstein- Canstein-Uhlenbeck process
On this dataset, experimented with the basic version of PCF-GAN, which only utilized the EPCFD as the discriminator without the autoencoder structure. The batch size is 256. The model are trained with 20000 generator training iterations and weight averaging on the generator was performed over the final 4000 generator training iterations. We used the 2-dimensional discretized Brownian motion the noise distribution.
C.2.1 Rough volatility model
We followed [31] considering rough stochastic volatility model for an asset price process (St) which satisfies the below stochastic differential equation,
dS; VV,S,dZr.
(19)
V/2 = E(t)ezp nB 2
2,28
(20)
where E(t) denotes the forward variance and B denotes the frational Brownian motion (IBM) given by
BM 3m:=A B [[=[(((ssssd
K(r) :- V2H, H-0.5
where (Z)) (Be)eej0.1) are (possibly correlated) Brownian motions. In our experiments, the synthetic dataset sampled from Equation (19) with E [o. 1]. H 0.25, <(t) (0
= 0.5 and initial condition log(So) ~ (7, 0.05). Each sample path is sampled uniformly from [o, 1] with the time discretization 8t 0.005, which consists of 200 time steps. We train the generators to learn the joint distribution of the log price and log volatility.
All methods trained with 30000 generator training iterations and weight averaging on the generator was performed over the final 5000 generator training iterations. The input noise vectors have dimension and 200 time steps.
For PCF-GAN, the coefficient A for the recovery loss was 50, and the number of unitary linear maps was 6.
C.2.2 Stocks
We selected 10 large market cap stocks, which are Google, Apple, Amazon, Tesla, Meta, Microsoft, Nvidia, JP Morgan, Visa and P&G, from 2013 to 2021. The dataset consists of 5 features, including daily open, close, high, low prices and volume, available on htttps://finanees yahanceeeahhooo. com/ lookup. We truncated the long stock time series into 20 days. The data were normalized with standard Min-Max normalisation on each feature channel. The Stock dataset used in our study is similar to the one employed in [25) but with a broader range of assets. Unlike the previous approach, we avoided sampling the time series using rolling windows with a stride of mitigate the presence of strong dependencies between samples.
All methods trained with 30000 generator training iterations and weight averaging the generator was performed over the final 5000 generator training iterations. The imput noise vectors have 5 feature dimensions and 20 time steps.
For PCF-GAN, the coefficient \ for the recovery loss was 400, and the number of unitary linear maps was 6.
23

---

C.2.3 Beijing Air Quality
We used dataset of the air quality in Beijing from the UCI repository 47 and available on https:// // archive. ics uci. ple is 10-dimensional time series of the SO2, NO2, CO, O3, PM2.5, PMI0 concentrations, temperature, pressure, dew point temperature and wind speed. Each time series is recorded hourly the course of day. The data normalized with standard Min-Max normalisation each feature channel.
/atasets/Beijingg - -t---tter--uulaittyy- Quality-aaaa Each sam-
All methods are trained with 20000 generator training iterations and weight averaging on the generator was performed over the final 4000 generator training iterations. The input noise vectors have dimensions and 24 time steps.
For PCF-GAN, the coefficient A for the recovery loss 50, and the number of unitary linear maps was 6.
C.2.4 EEG
We obtained the EEG eye state dataset from Sttaee The data from one continuous EEG measurement on 14 variables with 14980 time steps. We truncated the long time series into smaller ones with 20 time steps. The data are subtracted by channel-wise mean, divided by three times the channel-wise standard deviation, and then passed through tanh nonlinearity.
ics. uci wew.mm/ddaaaseeeeeeeeeeeeeee
All methods trained with 30000 generator training iterations and weight averaging on the generator was performed over the final 5000 generator training iterations. The input noise vectors have the 8 dimensional and 20 time steps.
For PCF-GAN, the coefficient d for the recovery loss was 50, and the number of unitary linear maps was 8.
D Supplementary results
D.1 Ablation study
An ablation study was conducted on the PCF-GAN model to evaluate the importance of its various components. Specifically, the reconstruction loss and regularization loss were disabled in order to assess their impact model performance across benchmark datasets and various test metries. Table 3 consistently demonstrated that the PCF-GAN model outperformed the ablated versions, confirming the significance of these two losses in the overall model performance.
Table 3: Ablation study of PCF-GAN
Dataset
Test Metrics
PCF-GAN
w/o Lieaovery
w/o Lregularizaion
w/o Lregularizttinn & Leeewery
Discriminative
.010811.066
.0178:1.017
.0152:1.20
.01011.007
RV
Predictive
.0390+0000
.038-+.000
.0390=0003
.0391
Sig- Sig-MMD
.0024+.001
.0037+.001
.0036 +.002
.0027+.001
Discriminative
.07841.028
.09631.0011
.55881052
.08151.001
Stock
Predictive
.0125+.000
.0123+.000
.0177.000
.0126+.001
Sig-MMD
.00171.000
.0062+.002
.0024.0001
.0021+.001
Discriminative
.2326:1.088
.3940-.068
4783+.029
.387541.09
Air
Predictive
.0237=.000
.0239+.000
0283=.001
.0240
Sig-MMD .0126:.005
.01111.003
.232--004
.0163+.004
Discriminative .3660+.025
4942+1.01
5000+0000
.649+.015
EEG
Predictive
.0246:.000
.0299+t0000
.6336.0007
.0248+.000
Sig-MMD
.0180+1.004
.0296+.0008
1.197=.234
.0278=007
Notably, the inclusion of the two additional losses significantly improved model performance on high-dimensional time series datasets, such as Air Quality and FEG, indicating that the proposed auto-encoder architecture effectively learns meaningful low-dimensional sequential embeddings.
24

---

Conversely, the exclusive use of the reconstruction loss led to notable decrease in model performance, suggesting that the 12 samplewise distance might not be suitable for time series data. However, the additional regularization loss helped overcome this issue by ensuring that the sequential embedding space confined to a predetermined noise space, such as the discretized Brownian motion. As result, the regularization loss helped to mitigate the problems that arose when relying solely on the reconstruction loss.
D.2 Generated samples
In this section, we present random samples from the four benchmark datasets generated by PCF- GAN, TimeGAN, RGAN, and COT-GAN. Although interpreting the sample plots of the generated time series poses challenge, our observations reveal that PCF-GAN successfully generates time series that capture the temporal dependencies exhibited in the original time series across all datasets. Conversely, COT-GAN generates trajectories that are relatively smoother compared to the real time series samples, demonstrated on Stock and EEG datasets, by Figure and Figure I0 respectively. Figure 10 shows that TimeGAN occasionally produces samples with higher oscillations than those found in the real samples.
mmd
0.
200
202
Figure 7: Generated samples from all models on Rough volatility dataset
umn
An
m
a mm
Figure 8: Generated samples from all models on Stock dataset
25

---

0.3
Figure 9: Generated samples from all models on Air Quality dataset
-32
Figure 10: Generated samples from all models EEG dataset
D.3
Reconstructed samples
In this section, we present additional reconstructed time series samples generated by PCF-GAN and TimeGAN. Figure TT iillustrates that PCF-GAN consistently outperforms TimeGAN by producing higher-quality reconstructed samples across all datasets.
Real
PCT-GAN
T TmeGAN
Real
POT-GAN
TmeGAN
Real
PCT-GAN
TimeGAN
Av
Stock
m
Ar
BEG
Figure 11: Reconstructed samples from PCF-GAN and TimeGAN on all benchmark datasets.
26

---

D.4 Test metrics on (auto-)correlattoo and marginal distribution
This subsection details the supplementary test metrics terms of fitting the autocorrelation, cross- correlation, and marginal distribution, as presented in Table 4 This table confirms that our proposed PCF-GAN consistently outperforms the benchmarking models across all datasets.
Table 4: Performance comparison of PCF-GAN and baselines auto-correlation, cross-correlation and marginal distribution metrics. Best for each task is shown in bold.
Task
Generation
Dataset
Test Metrics
RGAN
COT-GAN
TimeGAN
PCF-GAN
Auto-cor (lag 1)
03931.001
.060810011
.0031=.001
.0022.1.000
Auto-cor (lag 5)
.01341.002
.1191.002
.00351.002
.00301.002
RV
Cross-cor (lag 0)
.0193=.007
.02344.002
.0187=0011
.0264=.011
Cross-cor (lag 5)
.0222+.007
.1441#.012
.0219=0010
.0158:.011
Marginal Dist
.11+1.113
2117++306
.1636+.223
.1234+.126
Auto-cor (lag 1)
.27+.005
.022.0033
210+.005
.0133+0005
Auto-cor (lag 5)
.199.009
.2671.006
.1411.066
.01871.006
Stock
Cross-cor (lag 0)
.155+.031
.169+.041
5499.034
.1815+058
Cross-cor (lag 5)
.41+.031
45611.053
.7474.038
.51001.062
Marginal Dist
327611004
28261.061
42641.063
.27301.033
Auto-cor (lag 1)
1678+.010
.200.0006
1949=.006
.0927+.003
Auto-cor (lag 5)
3226210016
.2004.028
5349=.034
4739-10023
Air
Cross-cor (lag 0)
2.608+.106
1.942+.059
2.844+.0812
2.687+.149
Cross-cor (lag 5)
3.181#.101
2.176#.116
2.536=.112
2.115+.121
Marginal Dist
55271.523
.11421.600
.52291.955
.56661.572
Auto-cor (lag 1)
5.918+.116
6.202+.111
5.754
5.668+.079
Auto-cor (lag 5)
4.2851.074
5.911
5.265
4.467+.127
EEG
Cross-cor (lag 0)
51.161..08
24.12+.702
26.841.638
22.27+.550
Cross-cor (lag 5)
47.97
31.31
25.95=.466
19.43+.412
Marginal Dist
15.18=21.94
8.518+13.6
13.35=21.7
10.09+
27


Reviews:
[{"review_id": "u8sd2BgLfp", "reviewer": "Reviewer_cYLd", "summary": "The paper presents a new metric for the distributions on the path space via PCF and provides theoretical proofs for analytic properties of the proposed loss metric which benefits GAN training. It introduces a novel PCF-GAN to generate & reconstruct time series simultaneously. It compares the proposed method with prior work and shows improved performance.  ", "questions": "The authors need to clearly distinguish their work with existing literature and provide comparisons with prior work (e.g. [A, B]). ", "limitations": "The authors have adequately discussed limitations of their work. ", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is relatively well-written and the proposed approach is described clearly. \n\n- The authors provide theoretical grounding for their work. They prove PCF's characteristicity, boundedness, differentiability with respect to generator parameters, and its weak continuity which ensures the stability and feasibility of training the PCF-GAN. \n\n- Experimental results demonstrate improved performance compared to several prior works. The authors provide comparisons with TimeGAN, CotGAN and RGAN on RV, Stock, Air and EEG datasets using discriminative and predictive metrics.  ", "weaknesses": "- There are concerns regarding missing comparisons with related work. [A, B] also propose generative models for time series generation. [A] combines the adversarial training of GANs and the exact maximum likelihood training of CTFPs into a single framework. It designs an invertible generator, and adopt an autoencoder, on whose hidden space of the GAN performs the adversarial training. [B] uses Deep Euler representation and Wasserstein distances to propose three generative methods for times series. Two generative method EWGAN and EDGAN demonstrate an accuracy similar to state-of-the-art GAN generators and show better performance for capturing temporal dynamic metrics of the time series. The third method CEGEN is based on a loss metric computed on the conditional distributions of the time series. These papers are not mentioned and no comparisons are provided with them.   \n\n- There are metrics such as FID in prior work, e.g. [B], which are not used in the experiments. \n\n[A] GT-GAN: General Purpose Time Series Synthesis with Generative Adversarial Networks; Jeon et al.; \n \n[B] Conditional Loss and Deep Euler Scheme for Time Series Generation; Remlinger et al.;", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We address all the questions in detail as follows.\n\n**Comparisons with related work** \n\nWe first refer to the comparison to the existing literature in *Reply to all reviewers*. Then we further elaborate on the comparison between our method and [A, B].\n\n[A] proposed a GAN framework to generate time series that is robust to missing data, making extensive use of continuous-time process modules therein. In particular, their framework incorporates CFTP as generator, NCDE as the bottleneck architecture for auto-encoder and critic function, and average cross-entropy loss over time as objective function. In contrast, our work aims to improve GAN's training by introducing and incorporating  PCFD, a novel discriminator. Thus, in view of the essential differences in the architectures of generator and critic, we think it is not suitable to directly benchmark our model against the framework in [A]. We do acknowledge, however, that it is a very interesting direction to incorporate PCFD into the GAN framework in [A]. For instance, one may replace the discriminator in [A] by our proposed PCFD. As PCF relies on the unitary feature arising from controlled differential equations (*e.g.*, NCDE; see [C]), we conjecture that this operation will enhance the model's robustness to missing data, hence improving the performance compared to the traditional cross-entropy loss used in [A].\n\n[B] used the Euler discretization scheme of SDEs as a generator and the Wasserstein metrics as an objective function. For a fair comparison, one should compare PCFD in our work to the Wasserstein metrics in [B]. The use of Wasserstein metrics typically requires a 1-Lipschitz critic function, which entails further constraints on the gradient of the critic function. In contrast, in our work, further constraints are not needed at all, thanks to the boundedness of PCFD.  It would be very interesting to numerically benchmark our method to [B] and analyze the performance of the integrated model based on SDE Euler discretization and PCFD. But as their codes are not publicly available and re-implementation of their model from scratch within a week appears unfeasible, at the moment this is beyond our scope. One minor point: by [B], two generative methods EWGAN and EDGAN (besides CEGEN) consistently underperform against COT-GAN. \n \nAn important innovation of CEGAN is the use of conditional loss, which leads to superior performance ([B]). It would be interesting to substitute the $W_2$ metric by PCFD therein to match the distribution of $\\mathbb{P}[X_{t+1} | X_{t}]$. More importantly, this enables us to extend the loss to match the conditional law of Step $1$ and Step $q$, *i.e.*, $\\mathbb{P}[X_{t+1: t+q} | X_{t}]$. We will briefly mention it in the future work section.\n\n**Additional evaluation metrics**\n\nThe FID score, commonly used in image generation, requires a pre-trained model to map each sample to a high-dimensional vector. For time series data, the choice of appropriate outputs or labels for pre-trained models encompasses ambiguity. In addition, the FID score requires the assumption of normality to avoid bias ([D]). Meanwhile, in our humble opinion, FID is not widely used in time series generation literature (*cf.* works on GT-GAN [A], TimeGAN, and COT-GAN). Taking the above into consideration, together with the fact that [B] did not provide implementation details or code for the FID score, we decide not to include this metric in our experiments.\n\nInstead, we consider three additional metrics: auto-correlation, cross-correlation and marginal distribution metrics. They are intended to assess the temporal dependency, spatial dependency, and marginal fitting via classical statistics, respectively. See the discussion on the evaluation section in *Reply to all reviewers* and the attached supplementary PDF. \n\n[A] Jeon, Jinsung, Jeonghak Kim, Haryong Song, Seunghyeon Cho, and Noseong Park. \"GT-GAN: General Purpose Time Series Synthesis with Generative Adversarial Networks.\" Advances in Neural Information Processing Systems 35 (2022): 36999-37010.\n\n[B] Remlinger, Carl, Joseph Mikael, and Romuald Elie. \"Conditional loss and deep Euler scheme for time series generation.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 36. No. 7. 2022.\n\n[C] Lou, Hang, Siran Li, and Hao Ni. \"Path Development Network with Finite-dimensional Lie Group Representation.\" arXiv preprint arXiv:2204.00740 (2022).\n \n[D] Heusel, Martin, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. \"GANs trained by a two time-scale update rule converge to a local Nash equilibrium.\" Advances in Neural Information Processing Systems 30 (2017)."}, {"review_id": "zCtblKj3sS", "reviewer": "Reviewer_BgyV", "summary": "This paper proposes a path characteristic function GAN, named PCF-GAN, for learning to generate time-series data. More specifically, the authors mainly employ the rough path theory to build the PCF distance, such that the temporal cues can be encoded by unitary features, enabling the PCF to learn sequential data. The PCF distance is then proved to be complete and favorable for minimizing the difference between two stochastic processes. Even though, an encoder-decoder-based embedding is proposed to compare two random processes in the latent space of reduced dimensions, catering for the ease of training in practice. Experimental results have also verified the effectiveness of the proposed PCF-GAN.", "questions": "1. In Equation (4), why x_0=I_m. Wouldn't it be y_0=I_m for the differential equation?\n\n2. In Line 254, what does it mean by IGM? Should it be IPM?\n\n3. Why for the regularization loss, this paper employs a different \\theta'_M? How \\theta_M and \\theta'_M were implemented in this paper?", "limitations": "Please see my weakness.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The established theory on PCF is novel and complete, by far my understanding. The authors also propose a practical way of minimizing the PCF distance, by using an encoder-like embedding that is able to reconstruct signals. ", "weaknesses": "1. I am appreciated the theory behind the PCF. However, the usage of path theory needs to be clarified, when aiming at improving characteristic functions to represent stochastic process. The authors claim that CF-metric fails to capture temporal dependency of sequential data, which is true. However, why using path theory on characteristic functions can well address the temporal dependency, given the fact that there are other representations of the characteristic function of a stochastic integral?\n\n2. The authors claim that the embedding operation does not have any gradient constraints. However, when adding calculating the EPCFD after the embedding layers, the continuous and differential properties may not hold, without the restriction on the embedding layers. I am not sure whether the reconstruction regularization can compensate strictly. The authors are encouraged to elaborate more on this.\n\n3. Another weakness is regarding the experimental validations, which were performed based on low-dimensional time-series datasets. The PCF-GAN is shown to beat the COT-GAN on these datasets. However, COT-GAN is also able to generate short video clips, in which the scenarios are much complicated. Would the PCF-GAN scale well to the higher-dimensional generation tasks, such as short video generation? Also the comparing baselines are not the state-of-the-art. It is the fact that many recent methods have been reported to surpass COT-GAN and TimeGAN, for example, [1] and [2].\n\n[1] Seyfi, Ali, Jean-Francois Rajotte, and Raymond Ng. \"Generating multivariate time series with COmmon Source CoordInated GAN (COSCI-GAN).\" Advances in Neural Information Processing Systems 35 (2022): 32777-32788.\n[2] Jarrett, Daniel, Ioana Bica, and Mihaela van der Schaar. \"Time-series generation by contrastive imitation.\" Advances in Neural Information Processing Systems 34 (2021): 28968-28982.\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We address all the questions in detail as follows.\n\n**1. Temporal dependency**\n\nThe path characteristic function (PCF) is proven to characterize the law of the stochastic processes, or time series. Hence, PCF completely determines the temporal dependency, *e.g.*, statistics like auto-correlation. The fundamental mechanism lies in the non-commutativity of matrix multiplication --- the multiplication in the Lie groups used to define PCF --- which echoes the non-interchangeability of the temporal order of events in time series. Note that the PCF deals with the measures on the *$\\infty$-dimensional* space of paths, while the usual characteristic function (CF) is defined for $\\mathbb{R}^d$-valued random variables. The latter cannot be directly extended to the $\\infty$-dimensional setting mainly because the range of the exponential map used in CF is $i \\mathbb{R}$, which is commutative. \n\nThere might be other representations characterizing the law on the path space; however, to the best of our knowledge, the PCF is the only such representation that has been proven characteristic, uniformly bounded, and differentiable.   \n\n\n**2. Embedding layers**\n\nWe thank the reviewer for pointing out the potential restrictions caused by embedding layers. Some clarifications and further explanations are as follows:\n\n1. We concur with the reviewer on the need for the differentiability assumption of the embedding layer. As long as the embedding map is differentiable, the continuity and differentiability properties (Theorem 3.6) hold after applying the embedding layers. In fact, it has been taken for granted that most of neural networks used for the embedding map are differentiable, so as to ensure the validity of gradient descent algorithms. We shall clarify this point in the revised manuscript.\n\n2. Indeed, gradient constraints are required by various GANs models, such as Wasserstein GANs, in order to impose additional regularity or add gradient norm upper bound on the critic function/embedding layer. In these GAN models, the distance may become unbounded in absence of further constraints. However, the PCFD distance between any two distributions on the path space is *uniformly* bounded by $2m^2$, as proved in Lemma 3.5. Therefore, when applying EPCFD on the embedded distributions, the uniform boundedness assumption holds automatically.  \n\n3. In our work, the reconstruction and regularization losses are not introduced to address the issues of gradient constraints,  continuity, or differentiability --- these have been taken care of naturally by the aforementioned mathematical properties of PCFD. Instead, the reconstruction and regularization losses aim at imposing injectivity on the embedding function $f$, so that  $X \\stackrel{d}{=} Y$  whenever $f(X) \\stackrel{d}{=} f(Y) $ for any path-valued random variables $X$ and $Y$ (here $\\stackrel{d}{=}$ means equality in distributions). The following trivial example validates the assumption on the injectivity on $f$: if $f$ is simply a constant function, then no matter whether distributions of $X$ and $Y$ agree or not, one has $\\mathrm{PCFD}(f(X),f(Y)) = 0$.  \n\n**3. Experimental validations**\n\nTo address this question, we first refer to the discussions on comparison with existing literature and choice of datasets in *Reply to all reviewers*. Besides, in what follows let us elaborate on our method in the context of Refs. [2, 3].\n\nThe main objective of this work is to propose a novel distance metric to improve the quality of GANs for time series generation. In contrast, both [2] and [3] proposed new frameworks for generating sequential data, which may not be suitable to benchmark directly with our method. But it would be interesting to incorporate our methods within their framework. Specifically, [2] proposed a framework to tackle multivariate time series by comparing each channel of the real and fake times series and then comparing across all channels globally. Within this framework, PCFD can be used as a distance measure --- in place of the average cross-entropy loss across time --- for both channel-wise and global comparisons. Similarly, for [3] we may replace the average cross-entropy loss by PCFD in their min-max objective function (see Equation (11) in [3]).    \n\nQuestions: \n\n**1.** Yes, this is a typo. We will fix it accordingly.\n\n\n**2.** IGM stands for Implicit Generative Model. It was our mistake for omission on its clarification. We will modify this part accordingly. \n\n**3.** The generator loss is the EPCFD distance between the reconstructed noise $F_{\\theta_f}(G_{\\theta_g}(Z))$ and $F_{\\theta_f}(X)$. In contrast, the regularization loss is the EPCFD distance between the noise $Z$ and $F_{\\theta_f}(X)$. Although the reconstructed noise converges to the noise distribution eventually, there is no guarantee that they are the same throughout the optimization process. Thus these two losses may have distinctive trainable parameters.\n\nWe initialize $\\theta_M$ and $\\theta_M'$ following the method described in the supplementary material, Section B.4.1 in the original submission. The optimization for $\\theta_M$ follows the procedure in [4]. Note that the parameter $\\theta'_M$ has been initialized and optimized independently of $\\theta_M$.      \n\n[1] Chevyrev, Ilya, and Terry Lyons. \"Characteristic functions of measures on geometric rough paths.\" (2016): 4049-4082.;\n\n\n[2] Seyfi, Ali, Jean-Francois Rajotte, and Raymond Ng. \"Generating multivariate time series with COmmon Source CoordInated GAN (COSCI-GAN).\" Advances in Neural Information Processing Systems 35 (2022): 32777-32788.\n\n[3] Jarrett, Daniel, Ioana Bica, and Mihaela van der Schaar. \"Time-series generation by contrastive imitation.\" Advances in Neural Information Processing Systems 34 (2021): 28968-28982.\n\n[4] Lou, Hang, Siran Li, and Hao Ni. \"Path Development Network with Finite-dimensional Lie Group Representation.\" arXiv preprint arXiv:2204.00740 (2022).\n"}, {"review_id": "Qg4wuG9JWS", "reviewer": "Reviewer_EJ4R", "summary": "The paper proposes an approach to improve time-series modeling in Generative Adversarial Network framework by using path characteristic function as the embedding of the time series sample. It explains the feasibility of PCF distance and how to integrates it as a distance measure between two time series data in order to help the discriminator learn better generative features. The goal and benefit of the proposed method is to empirically improve the generation quality of time series data. \n", "questions": "N/A", "limitations": "As mentioned in the weakness section, having more ablations and experiments for comparison would help the reader better understand the efficacy of the proposed approach.", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper explores using a novel loss function in Generative Adversarial Network framework. It computes the proposed empirical path characteristic function distance between generated and given sample to learn better time series generative features. The proposed method outperforms other generative baselines on the given datasets. The paper is well written ( and includes the codebase in appendix for reproducibility).\n", "weaknesses": "It would be great to have to following experiments in order to compare fairly with the existing baselines:\n\n1. Comparison with existing baselines like COT-GAN on metrics like \n(a) the sum of the absolute difference of the correlation coefficients between channels avg over time \n(b) absolute difference between the correlation coefficients of real and generated samples.\n\n2. Comparison on high dimensional dataset (like Sprites and human action sequences with FID, KID scores )\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We address all the questions in detail as follows.\n\n>Comparison with existing baselines like COT-GAN on metrics like (a) the sum of the absolute difference of the correlation coefficients between channels avg over time (b) absolute difference between the correlation coefficients of real and generated samples.\n\n\nWe have included the two metrics on (a) auto-correlation and (b) cross-correlation with different lags as additional test metrics to evaluate the quality of the generative models. Please see the *test metrics* section in 'Reply to all reviewers'.\n\n\n>Comparison on high dimensional dataset (like Sprites and human action sequences with FID, KID scores)\n\nWe refer to the discussion on the *choice of datasets* section in *Reply to all reviewers*.\n\n*Ablation study*\n\nRegarding the comments in the Limitations section, the ablation results have been provided in Section D.1 of supplementary materials in the original submission. Table 3 shows that by incorporating the regularisation loss $L_{{\\rm regularisation}}$ and the reconstruction loss $L_{{\\rm recovery}}$, the model performance has been improved --- in terms of all the three different test metrics --- consistently across empirical data. Moreover, the additional losses are proven to be more effective when applied to high-dimensional time series."}, {"review_id": "6eFJ9y635a", "reviewer": "Reviewer_EqQ9", "summary": "The paper looks at generative models for times series data. It proposes a new GAN method, based on a novel discriminator. The path characteristic function (PFC) is used as a representation of the time series distribution. Using this, a distance between two distributions is defined (PCFD) as well as a way to approximate it (EPCFD). Afterwards, it is shown how EPCFD can be used as a discriminator in a GAN scenario (resulting in the PCF-GAN). To scale it to larger input dimensions, the authors follow previous literature and introduce a parameterised dimensionality reduction of the inputs and introduce different losses in an effort to ensure that it behaves as expected. Finally, they conduct a number of experiments, showing that PCF-GAN compares favourably to competitive time-series GANs.", "questions": "In Eq. 9, do you expect training only on “ -L_recovery - L_regularisation”  to perform much worse?", "limitations": "The method has computational limitations which limits the number of dimensions which it can be applied to. As mentioned before, an embedding functions is trained to reduce the dimensionality of the inputs, however, if the effective dimensionality is high, the method would fail.\n", "rating": 7, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The method and its different components was well motivated and presented.\nThe paper presents a compelling theoretical motivation for the method, giving evidence on why it might be preferable to alternatives s.a. WGAN.\nThe experimental results are well analysed and presented.\n", "weaknesses": "The reconstruction functionality is not motivated, nor explained. It can be unclear to the unfamiliar reader why one would want it, given that we can just generate images.\n\nRelated work - similar work is only lightly touched upon and the advantages/disadvantages are not outlined, making it harder to compare.\n\nThe loss function in Eq. 9 is not arrived at in a principled way and has two hyper-parameters which might be difficult to tune?\n\nIn the experiments, the dimensionality of the inputs, as well as the length of the sequences appears rather low. This suggests potential scalability issues?\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We address all the questions in detail as follows.\n\n>The reconstruction functionality is not motivated, nor explained. It can be unclear to the unfamiliar reader why one would want it, given that we can just generate images.\n\nTime series reconstruction potentially has broad applications in privacy preservation [1] and in extracting semantical representation for downstream tasks via latent embedding \n[2]. The reconstruction functionality allows us to share the trained generative model and the embedding of real data as the surrogate of actual data. The separation of model and embedding reduces the risk associated with data sharing while still permitting high-quality reconstruction. We will add this to the introduction in the revised version.\n\n>Related work - similar work is only lightly touched upon and the advantages/disadvantages are not outlined, making it harder to compare. \n\nRefer to the *Reply to all reviewers*.\n        \n>The loss function in Eq. 9 is not arrived at in a principled way and has two hyper-parameters which might be difficult to tune?\n\nThe loss function in Eq. (9) incorporates two additional terms $L_{{\\rm recovery}}$ and $L_{{\\rm regularisation}}$ as penalty components. They are used to encourage the constraint $L_{{\\rm recovery}}=0$ and $L_{{\\rm regularisation}}=0$ for preserving injectity of the embedding $F_{\\theta_f}$. The inclusion of penalty terms to transform constrained optimisations into unconstrained ones is a standard technique, exemplified in Lasso regression and Ridge regression. Empirically, the values of two hyperparameters $\\lambda_1$ and $\\lambda_2$ can affect the training process and should be carefully chosen via grid search. Heuristically, the optimal $\\lambda_1$, $\\lambda_2$ are chosen when the magnitude of three losses are comparable. \n\n\n>In the experiments, the dimensionality of the inputs, as well as the length of the sequences appears rather low. This suggests potential scalability issues?\n\nIn our experiments, the time dimension of RV dataset could be as high as 200, whereas the path dimension of EGG is 14 (at least moderate or high). More importantly, the computational complexity of the PCFD used in our PCF-GAN is linear in  time and dimensions, so the scalability issue is not expected for high-dimensional time series. Specifically, let $\\{T,d\\}$ and $m$ be  $\\{time, path\\}$ dimension of time series and matrix order of EPCFD, resp. Computation of PCFD loss $\\sim \\mathcal{O}(Tdm^2)$, which is linear in both time and storage, hence is scalable from the computational perspective. The choice of hyperparameter $m$ may not be directly related to $T$ and $d$. With the embedding layer incorporated, the PCFD loss computation is linear in output dimension of the embedding layer. Please see our reply to **Limitations** section for additional comments.\n\nQuestions:\n\n>In Eq. 9, do you expect training only on “ -L\\_recovery - L\\_regularisation” to perform much worse? \n\nYes, training solely on $-L_{\\rm recovery} - L_{\\rm regularisation}$ leads to significantly worse performance. These two components only ensure the injectivity of the embedding $F$, but are unable to match $F(G(Z))$ and $F(X)$ without including $L_{\\rm generator}$. We ran a simple experiment with only $-L_{\\rm recovery} - L_{\\rm regularisation}$; the training completely failed from the beginning.\n\nLimitations:\n\n>The method has computational limitations which limits the number of dimensions which it can be applied to. As mentioned before, an embedding functions is trained to reduce the dimensionality of the inputs, however, if the effective dimensionality is high, the method would fail.\n\nOur model is scalable with respect to the dimension of embedding output. However, in the case that a large matrix size $m$ is essential for the training of some high-fidelity generative model, the current PCF-GAN might experience computational bottlenecks. In this regard, incorporating the embedding layer effectively reduces the need for large $m$ (see numerical examples in our paper). It is also worthwhile to investigate low-rank approximation of relevant matrices or combine other advanced GAN model architectures to enhance the scalability of our PCF-GAN. We will add a brief discussion on it to \"Limitation and Future work\" section. \n\n\n[1] Rastogi, Vibhor, and Suman Nath. \"Differentially private aggregation of distributed time-series with transformation and encryption.\" In Proceedings of the 2010 ACM SIGMOD International Conference on Management of data, pp. 735-746. 2010.\n\n[2] Cho, Kyunghyun, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. \"Learning phrase representations using RNN encoder-decoder for statistical machine translation.\" arXiv preprint arXiv:1406.1078 (2014)."}, {"review_id": "15SLXz6226", "reviewer": "Reviewer_9Utd", "summary": "This paper proposes  PCF-GAN, which aims to improve the effectiveness of the discriminator in differentiating the time series distributions by utilizing the path characteristic function (PCF) as a principled representation of the time series distribution within the discriminator. The authors also give the theoretical foundation of the PCF distance, proving its properties that ensure stable and feasible training of PCF-GAN. Extensive experiments show the effectiveness of the proposed method.", "questions": "Please see Weaknesses", "limitations": "Please see Weaknesses", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The newly proposed metric in the discriminator for handling sequential data is interesting. Also, the author provides theoretical evidence for the analytical properties of the proposed loss metric.\n", "weaknesses": "- The presentation of this work is kind of confusing, making it difficult to fully comprehend the content. Specifically, in Figure 3, it is unclear what the generator loss represents if it is computed on the embedding vector. Additionally, it is unclear why the generator and regularization loss can enhance the discriminative power in Equation 10. Clarifying these help audience better understand. \n- The related work section consists of only one paragraph, which may overlook some relevant research in the field.\n- The ablation study of every component is required to demonstrate the effectiveness individually. \n- Regarding the sequential data, I am wondering how does this method work well on more challenging domains like video data since videos are also sequential in nature. \n\n\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We address all the questions in detail as follows.\n\n>The presentation of this work is kind of confusing, making it difficult to fully comprehend the content. Specifically, in Figure 3, it is unclear what the generator loss represents if it is computed on the embedding vector. Additionally, it is unclear why the generator and regularization loss can enhance the discriminative power in Equation 10. Clarifying these help audience better understand.\n\nWe extend our gratitude to the reviewer for constructive suggestions on the presentation of our work, based on which careful clarifications will be made in the revised manuscript. We summarise below the key points.\n\n(1) When the embedding map $F$ is injective, it is proved that $F(X) \\neq F(Y)$ if and only if $X \\neq Y$ (both $\\neq$ understood in terms of distributions). Thus the generator loss, defined as the EPCFD between the embedding vectors, can be regarded as a distance function on the original time series distributions via the injective embedding map, hence preserving the discriminative power.\n\n(2) The regularization loss, defined as the EPCFD between the noise and the embedding of true time series, is proposed to enforce the injectivity of the embedding map $F_{\\theta_f}$. When the embedding map and the generator $G_{\\theta_g}$ are pseudo-inverses of each other and true & fake distributions coincide, the regularization loss attains zero. This justifies the use of regularization loss in Equation (8). \n\n(3) The improved discriminative power is manifested via the increased distance score between any pair of given distributions. This is ensured by maximizing the EPCFD used in generative loss and regularization loss through optimizing the model parameters $\\theta_M$ and $\\theta_{M}'$, respectively (see equation (10)). A close analogy in the literature is the optimization of model parameters of the test function (critic) by maximizing the Wasserstein distance in WGANs. \n\n(4) In addition, numerical evidence has been provided in Section B.4.2. Figure 6 demonstrates that the optimized $\\theta_M$ increases the discriminative power of EPCFD on the stochastic process. We will include further modification in Section 3.3 in the revised manuscript. \n\n>The related work section consists of only one paragraph, which may overlook some relevant research in the field.\n\nPlease refer to the *Related work* section in *Reply to all reviewers*.\n\n >The ablation study of every component is required to demonstrate the effectiveness individually.\n\n We provided the ablation results in Section D.1, supplementary materials in the original submission. Table 3 shows that by incorporating the regularisation loss $L_{\\text{regularisation}}$ and the reconstruction loss $L_{\\text{recovery}}$, the model performance has been improved ---  in terms of all three different test metrics --- consistently across empirical data. Moreover, the additional losses are proven to be more effective when applied to high-dimensional time series.\n\n \n>Regarding the sequential data, I am wondering how does this method work well on more challenging domains like video data since videos are also sequential in nature.\n\nSee the *Choice of dataset* section in *Reply to all reviewers*. "}]

Author rebuttal:
We thank all the reviewers for their insightful comments and constructive suggestions. We are pleased that all the reviewers find our work novel, sound, and theoretically motivated. We also acknowledge the shared questions from the reviewers on related work and numerical evaluation. 

**Comparison to existing literature**: 

 - Related work: We will extend the related work section in the revised version by adding discussions on other state-of-the-art time series methods, including the comparison with our model.  We plan to incorporate additional GAN models with autoencoders (*e.g.*, GT-GAN). We shall also delve into strategies for improving GAN training for time series, highlighting innovative approaches such as additional conditional loss and SDE-based neural networks (Deep Euler representation [2]).  

-  Baselines: A main contribution of our PCF-GAN lies in the innovation of the discriminator for time series generation,  built upon the principled mathematical representation PCF. To validate its effectiveness, we focus on those state-of-the-art models with discriminators tailored for time series distribution. We thus choose RCGAN and TimeGAN (which use the average cross-entropy loss over time as the discriminator) as well as COTGAN (whose discriminator loss based on causal optimal transport). For a fair comparison, we use the same generator across all GAN models in our numerical experiments. 
  
    We acknowledge that several other time series generative models (GT-GAN, COSCI-GAN and EWGAN) pointed out by the reviewers may have superior performance, on certain benchmarking datasets, than the baselines adopted in our paper. Nevertheless, these models focus mainly on network framework and generator architecture, while our work emphasises the innovation of a discriminator. Thus, benchmarking our method with [1, 2, 3] as suggested by **cYLd** and **BgyV** appears not directly applicable.

- Future work. Although our numerical experiments do not include the comparison with the additional baselines, it is certainly of great interest to explore how PCFD could be incorporated into these more recent and state-of-the-art GANs for time series generation as future work.  Indeed, as a distance metric on time series, PCFD can be flexibly incorporated with other advanced generators of time series GAN models, hence may further improve the performance. For example, one can replace the average cross-entropy loss used in [1, 3] and the Wasserstein distance in [2] by PCFD, with some simple modifications on the discriminators. Such considerations will be elaborated in the future work section,  highlighting the flexibility of our proposed PCF-GAN. 

**Evaluation**:

- Test metrics. In response to the reviewers' suggestions, we have included several additional commonly-used test metrics to evaluate the quality of the generative models from different perspectives. (1) Fitting of temporal dependency. We adopt auto-correlation metrics used in COT-GAN and cross-correlation metrics used in Quant-GAN, as suggested by Reviewer EJ2R. These are classical statistical tools to measure the temporal dependency of time series within and across channels. (2) Fitting of marginal distribution. Here we adopt the distributional metric in Quant GANs [4]. We benchmark our method against RGAN, COT-GAN, and TimeGAN. Results in the supplementary PDF show that PCF-GAN significantly outperforms baselines. We will include the additional results of these test metrics in the appendix of the revised manuscript.

- Choice of datasets. To demonstrate the effectiveness of our proposed method, we have benchmarked several sequential datasets from different domains with various characteristics, as summarised in Table 1.  We acknowledge that it is of interest to validate our methods on more complex sequential data, such as video and human action. Nonetheless, training generative models on these tasks often requires tailored network architecture (*e.g.*, using deep convolution modules to learn spatial dependency within each frame of video) or more advanced frameworks, which lies out of the main focus of this paper. Furthermore, to our knowledge, these datasets are not commonly used for benchmarking generative models for sequential data (with the exception of COT-GAN, which included a short clip of video data). That said, it is certainly an intriguing direction to incorporate PCFD into complex model architectures/frameworks to tackle more intricate sequential data. This will be addressed in the future work section of the revised manuscript.

[1] Jeon, Jinsung, Jeonghak Kim, Haryong Song, Seunghyeon Cho, and Noseong Park. "GT-GAN: General Purpose Time Series Synthesis with Generative Adversarial Networks." Advances in Neural Information Processing Systems 35 (2022): 36999-37010.

[2] Remlinger, Carl, Joseph Mikael, and Romuald Elie. "Conditional loss and deep Euler scheme for time series generation." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 36. No. 7. 2022.

[3] Seyfi, Ali, Jean-Francois Rajotte, and Raymond Ng. "Generating multivariate time series with COmmon Source CoordInated GAN (COSCI-GAN)." Advances in Neural Information Processing Systems 35 (2022): 32777-32788.

[4] Wiese, Magnus, Robert Knobloch, Ralf Korn, and Peter Kretschmer. "Quant GANs: deep generation of financial time series." Quantitative Finance 20, no. 9 (2020): 1419-1440.

Meta-review: This work addresses generative models of time series, proposing a new metric on time series, PCFD, which they successfully apply to the discriminator of GANs. The presentation is good, well justified and motivated, and the empirical analysis well-validates the approach. There were some concerns about scaling, but the author's seem to have been able to argue successfully that the evaluations are sufficient for the story. Overall, the discussions had a strong positive influence on the final form (e.g., see author notes below), so I recommend this paper is accepted as a poster.
Decision: Accept (poster)