Title: PCF-GAN: generating sequential data via the characteristic function of measures on the path space
Venue: NeurIPS 2023
Accepted: yes
Keywords: Generative adversarial networks

time series generation

rough path theory

Lie group
Abstract: Generating high-fidelity time series data using generative adversarial networks (GANs) remains a challenging task, as it is difficult to capture the temporal dependence of joint probability distributions induced by time-series data. Towards this goal, a key step is the development of an effective discriminator to distinguish between time series distributions. We propose the so-called PCF-GAN, a novel GAN that incorporates the path characteristic function (PCF) as the principled representation of time series distribution into the discriminator to enhance its generative performance.  On the one hand, we establish theoretical foundations of the PCF distance by proving its characteristicity, boundedness, differentiability with respect to generator parameters, and weak continuity, which ensure the stability and feasibility of training the PCF-GAN. On the other hand, we design efficient initialisation and optimisation schemes for PCFs to strengthen the discriminative power and accelerate training efficiency. To further boost the capabilities of complex time series generation, we integrate the auto-encoder structure via sequential embedding into the PCF-GAN, which provides additional reconstruction functionality. Extensive numerical experiments on various datasets demonstrate the consistently superior performance of PCF-GAN over state-of-the-art baselines, in both generation and reconstruction quality.

PCF-GAN: generating sequential data via the characteristic function of measures on the path space

Abstract

Generating high-fidelity time series data using generative adversarial networks (GANs) remains a challenging task, as it is difficult to capture the temporal dependence of joint probability distributions induced by time-series data. Towards this goal, a key step is the development of an effective discriminator to distinguish between time series distributions. We propose the so-called PCF-GAN, a novel GAN that incorporates the path characteristic function (PCF) as the principled representation of time series distribution into the discriminator to enhance its generative performance. On the one hand, we establish theoretical foundations of the PCF distance by proving its characteristicity, boundedness, differentiability with respect to generator parameters, and weak continuity, which ensure the stability and feasibility of training the PCF-GAN. On the other hand, we design efficient initialisation and optimisation schemes for PCFs to strengthen the discriminative power and accelerate training efficiency. To further boost the capabilities of complex time series generation, we integrate the auto-encoder structure via sequential embedding into the PCF-GAN, which provides additional reconstruction functionality. Extensive numerical experiments on various datasets demonstrate the consistently superior performance of PCF-GAN over state-of-the-art baselines, in both generation and reconstruction quality.

1 Introduction

Generative Adversarial Networks (GANs) have been a powerful tool for generating complex data distributions, e.g. , image data. The original GAN suffers from optimisation instability and mode collapse, partially remedied later by an alternative training scheme using integral probability metric (IPM) in lieu of Jensen-Shannon divergence. The IPMs, e.g. , metrics based on Wasserstein distances or Maximum Mean Discrepancy (MMD), consistently yield good measures between generated and real data distributions, thus resulting in more powerful GANs on empirical data ([14, 2, 24]).

More recently, [1] proposed an IPM based on the characteristic function (CF) of measures on R d , which has the characteristic property, boundedness, and differentiability. Such properties enable the GAN constructed using this IPM as discriminator ('CF-GAN') to stabilise training and improve generative performance. However, ineffective in capturing the temporal dependency of sequential data, such CF-metric fails to address high-frequency cases due to the curse of dimensionality. To tackle this issue, we take the continuous time perspective of time series and lift discrete time series to the path space ([28, 29, 23]). This allows us to treat time series of variable length, unequal sampling, and high frequency in a unified approach. We propose a path characteristic function (PCF) distance to characterise distributions on the path space, and propose the corresponding PCF distance as a novel IPM to quantify the distance between measures on the path space.

Built on top of the unitary feature of paths ([26]), our proposed PCF has theoretical foundations deeply rooted in the rough path theory ([7]), which exploits the non-commutativity and the group structure of the unitary feature to encode information on order of paths. The CF may be regarded as the special case of PCF with linear random path and 1 × 1 unitary matrix. We show that the PCF distance (PCFD) possesses favourable analytic properties, including boundedness and differentiability in model parameters, and we establish the linkages between PCFD and MMD. These results vastly generalise classical theorems on measures on R d ([1]), with much more technically involved proofs due to the infinite-dimensionality of path space.

On the numerical side, we design an efficient algorithm which, by optimising the trainable parameters of PCFD, maximises the discriminative power and improves the stability and efficiency of GAN training. Inspired by [25, 41], we integrate the proposed PCF into the IPM-GAN framework, utilising an auto-encoder architecture specifically tailored to sequential data. This model design enables our algorithm to generate and reconstruct realistic time series simultaneously, which has advantages in diverse applications, including privacy preservation ([35]) and semantic representation extraction for downstream tasks ([10]). To assess the efficacy of our PCF-GAN, we conduct extensive numerical experiments on several standard time series benchmarking datasets for both generation and reconstruction tasks.

We summarize key contributions of this work below:

proposing a new metric for the distributions on the path space via PCF;

providing theoretical proofs for analytic properties of the proposed loss metric which benefit GAN training;

introducing a novel PCF-GAN to generate & reconstruct time series simultaneously; and

reporting substantial empirical results validating the out-performance of our approach, compared with several state-of-the-art GANs with different loss functions on various time series generation and reconstruction tasks.

Related work . Given the wide practical use of, and challenges for, realistic time series synthesis ([3, 4]), various approaches are proposed to improve the quality of GANs for synthetic time series generation. Several works, e.g. , [43, 45, 36], are devoted to improving the discriminator of GANs to be better suited to distributions induced by time series. Among them, COT-GAN in [43] shares a similar philosophy with PCF-GAN by introducing a novel discriminator based on causal optimal transport (which can be seen as an improved variant of the Sinkhorn divergence tailored to sequential data), while TimeGAN ([45]) shares a similar auto-encoder structure, which improves the generator's quality and enables time series reconstruction. Unlike PCF-GAN, the reconstruction and generation modules of TimeGAN are separated, whereas it has additional stepwise supervised loss and discriminative loss. In a different vein, CEGEN[36], GT-GAN [17], COSCI-GAN [39], and EWGAN[37] focus primarily on the design of network framework and generator architecture, which achieve state-of-the-art results on several benchmarking datasets.

2 Preliminaries

The characteristic function of a measure on R d , namely that the Fourier transform, plays a central role in probability theory and analysis. The path characteristic function (PCF) is a natural extension of the characteristic function to the path space.

2.1 Characteristic function distance (CFD) between random variables in R d

Let X be an R d -valued random variable with the law µ = P ◦ X -1 . The characteristic function of X , denoted as Φ X : R d → C , maps each λ ∈ R d to the expectation of its complex unitary transform: Φ X : λ ↦-→ E X ∼ µ [ e i ⟨ λ,X ⟩ ] . Here U λ : R d → C , x ↦→ e i ⟨ λ,x ⟩ is the solution to the linear controlled differential equation:

where 0 is the zero vector in R d and ⟨· , ·⟩ is the Euclidean inner product on R d .

References [11, 16] studied the squared characteristic function distance (CFD) between two R d -valued random variables X and Y with respect to another probability distribution Λ on R d :

It is proved in [25, 1] that if the support of Λ is R d , then CFD Λ is a distance metric, so that CFD 2 Λ ( X,Y ) = 0 if and only if X and Y have the same distribution. This justifies the usage of CFD 2 Λ as a discriminator for GAN training to learn finite-dimensional random variables from data.

2.2 Unitary feature of a path

Let BV ( [0 , T ]; R d ) be the space of R d -valued paths of bounded variation over [0 , T ] . Consider

For a discrete time series x = ( t i , x i ) N i =0 , where 0 = t 0 < t 1 < · · · < t N = T and x i ∈ R d ( i ∈ { 0 , · · · , N } ), we can embed it into some x ∈ X whose evaluation at ( t i ) N i =1 coincides with x . This is well suited for sequence-valued data in the high-frequency limit with finer time-discretisation and is often robust in practice ([27, 26]). Such embeddings are not unique. In this work, we adopt the linear interpolation for embedding, following [23, 18, 32].

Let C m × m := { m × m complex matrices } , I m be the identity matrix, and ∗ be conjugate transpose. Write U ( m ) and u ( m ) for the Lie group of m × m unitary matrices and its Lie algebra, resp.:

Definition 2.1. Let x ∈ BV ( [0 , T ]; R d ) be a continuous path and M : R d → u ( m ) be a linear map. The unitary feature of x under M is the solution y : [0 , T ] → U ( m ) to the following equation:

We write U M ( x ) := y T , i.e. , the endpoint of the solution path.

̸

By a slight abuse of notations, U M ( x ) is also called the unitary feature of x under M . Unitary feature is a special case of the Cartan/path development , for which one may consider paths taking values in any Lie group G . We take only G = U ( m ) here; m = d in general ([6, 30]).

Example 2.2. For M ∈ L ( R d , u ( m ) ) and x ∈ BV ( [0 , T ]; R d ) linear , U M ( X ) = e M ( x T -x 0 ) . In particular, when m = 1 , u (1) is reduced to i R and M ( y ) = i ⟨ λ M , y ⟩ for some λ M ∈ R d .

Motivated by the universality and characteristic property of unitary features ([7], see Appendix A.3), we constructed a unitary layer which transforms any d -dimensional time series x = ( x 0 , · · · , x N ) to the unitary feature of its piecewise linear interpolation X . It is a special case of the path development layer [26], when Lie algebra is chosen as u ( m ) . In fact, the explicit formula holds: U M ( X ) = ∏ N +1 i =1 exp( M (∆ x i )) , where ∆ x i := x i -x i -1 and exp is the matrix exponential.

Convention 2.3. The space L ( R d , u ( m ) ) in which M of Eq. (4) resides is isomorphic to u ( m ) d , where u ( m ) is Lie algebra isomorphic to R m ( m -1) 2 . For each θ ∈ u ( m ) d given by anti-Hermitian matrices { θ ( i ) } d i =1 , a linear map M is uniquely induced: M ( x ) = ∑ d i =1 θ ( i ) ⟨ x, e i ⟩ , ∀ x ∈ R d .

3 Path characteristic function loss

3.1 Path characteristic function (PCF)

The unitary feature of a path x ∈ X plays a role similar to that played by e i ⟨ x,λ ⟩ to an R d -valued random variable. Thus, for a random path X , the expected unitary feature can be viewed as the characteristic function for measures on the path space ([7]).

Definition 3.1. Let X be an X -valued random variable and P X be its measure. The path characteristic function (PCF) of X of order m ∈ N is the map Φ ( m ) X : L ( R d , u ( m ) ) → C m × m given by

The path characteristic function (PCF) Φ X : ⊕ ∞ m =0 L ( R d , u ( m ) ) → ⊕ ∞ m =0 C m × m is defined by the natural grading: Φ X ∣ ∣ L ( R d , u ( m )) = Φ ( m ) X for each m ∈ N .

In the above, U M ( x ) ∈ U ( m ) is the unitary feature of the path x under M . See Definition 2.1.

Similarly to the characteristic function of R d -valued random variables, the PCF always exists. Moreover, we have the following important result, whose proof is presented in Appendix A.

Theorem 3.2 (Characteristicity) . Let X and Y be X -valued random variables. They have the same distribution (denoted as X d = Y ) if and only if Φ X = Φ Y .

3.2 A new distance measure via PCF

We now introduce a novel and natural distance metric, which measures the discrepancy between distributions on the path space via comparing their PCFs. Throughout, d HS denotes the metric associated with the Hilbert-Schmidt norm ∥ · ∥ HS on C m × m :

Definition 3.3. Let X , Y : [0 , T ] → R d be stochastic processes and P M be a probability distribution on u ( m ) d := L ( R d , u ( m ) ) (recall Convention 2.3). Define the squared PCF-based distance (PCFD) between X and Y with respect to P M as

We shall not distinguish between M and P M for simplicity.

PCFD exhibits several mathematical properties, which provide the theoretical justification for its efficacy as the discriminator on the space of measures on the path space, leading to empirical performance boost. First, PCFD has the characteristic property.

̸

Lemma 3.4 (Separation of points) . Let X , Y ∈ P ( X ) and X = Y . Then there exists m ∈ N , such that if M is a u ( m ) d -valued random variable with full support, then PCFD M ( X , Y ) = 0 .

̸

Furthermore, PCFD M has a simple uniform upper bound for any fixed m ∈ N :

Lemma 3.5. Let M be a u ( m ) d -valued random variable. Then, for any BV ( [0 , T ]; R d ) -valued random variables X and Y , it holds that PCFD 2 M ( X , Y ) ≤ 2 m 2 .

Under mild conditions, PCFD is a.e. differentiable with respect to a continuous parameter, thus ensuring the feasibility of gradient descent in training.

Theorem 3.6 (Lipschitz dependence on continuous parameter) . Let X and Z be subsets of BV ( [0 , T ]; R d ) , (Θ , ρ ) be a metric space, Q be a Borel probability measure on Z , and M be a Borel probability measure on u ( m ) d . Assume that g : Θ ×Z → X , ( θ, Z ) ↦→ g θ ( Z ) is Lipschitz in θ such that Tot . Var . [ g θ ( Z ) -g θ ′ ( Z )] ≤ ω ( Z ) ρ ( θ, θ ′ ) . In addition, suppose that E M ∼ P M [ |∥ M |∥ 2 ] < ∞ and E Z ∼ Q [ ω ( Z )] < ∞ . Then PCFD M ( g θ ( Z ) , X ) is Lipschitz in θ . Moreover, it holds that

| PCFD M ( g θ ( Z ) , X ) -PCFD M ( g θ ′ ( Z ) , X ) | ≤ √ E M ∼ P M [ |∥ M |∥ 2 ] E Z ∼ Q [ ω ( Z )] ρ ( θ, θ ′ ) for any θ, θ ′ ∈ Θ , Z ∈ Z , X ∈ X , and M∈P ( u ( m ) d ) .

Remark 3.7 . The parameter space (Θ , ρ ) is usually taken to be R ¯ d for some ¯ d ∈ N . In this case, by Rademacher's theorem PCFD M ( g θ ( Z ) , X ) is a.e. differentiable in θ .

Similarly to metrics on measures over R d ( cf . [2, 24]), we construct a metric based on PCFD, denoted as ˜ PCFD , on the space P ( X ) of Borel probability measures over the path space, and we prove that it metrises the weak-star topology on P ( X ) . Throughout, d → denotes the convergence in law.

Theorem 3.8 (Informal, convergence in law) . Let { X n } n ∈ N and X be X -valued random variables with measures supported in a compact subset of X . Then ˜ PCFD( X n , X ) → 0 ⇐⇒ X n d → X .

The formal statement and proof can be found in Lemma B.2 and Theorem B.8 in the Appendix.

Similar to [40] for R d , we prove that PCFD can be interpreted as an MMD with a specific kernel κ (see Appendix B.3). Example B.12 illustrates that the PCFD has the superior test power for hypothesis testing on stochastic processes compared with CF distance on the flattened time series.

3.3 Computing PCFD under empirical measures

Now, we shall illustrate how to compute the PCFD on the path space.

Let ¯ X := { x i } n i =1 and ¯ Y := { y i } n ′ i =1 be i.i.d. drawn respectively from X -valued random variables X and Y . First, for any linear map M ∈ u ( m ) d , the empirical estimator of Φ X ( M ) is the average of unitary features of all observations ¯ X = { x i } n i =1 , i.e. , Φ ¯ X ( M ) = 1 n ∑ n i =1 U M ( x i ) . We then parameterise the u ( m ) d -valued random variable M via the empirical measure M θ M , i.e., M θ M = ∑ k i =1 δ M i , where θ M := { M i } k i =1 ∈ u ( m ) d × k are the trainable model parameters. Finally, define the corresponding empirical path characteristic function distance (EPCFD) as

Our approach to approximating M via the empirical distribution differs from that in [25], where M is parameterised by mixture of Gaussian distributions. In §4.1 and §5, it is shown that, by op- timising the empirical distribu- tion, a moderately sized k is suf- ficient for achieving superior per- formance, in contrast to a larger sample size required by [25].

xi

x1

X²

xn-1

Xn

(path in IRd)

dUt=U·M(dXt）),Uo=Im

θ(d matrices ofsize

Im(u(xi))

m × m)- trainable

parameters

Avg

Φx(M)=

15

7L

u(xi)

4 PCF-GAN for time series generation

4.1 Training of the EPCFD

In this subsection, we apply the EPCFD to GAN training for time series generation as the discriminator. We train the generator to minimise the EPCFD between true and synthetic data distribution, whereas the empirical distribution of M characterised by θ M ∈ u ( m ) d × k is optimised by maximising EPCFD.

By an abuse of notation, let X := R d × n T ( Z := R e × n T , resp.) denote the data (noise, resp.) space, composed of R d ( R e , resp.) time series of length n T . As discussed in §2.2, X and Z can be viewed as path spaces via linear interpolation. Like the standard GANs, our model is comprised of a generator G θ g : Z → R d × n T and the discriminator EPCFD θ M : P ( X ) × P ( X ) → R + , where θ M ∈ u ( m ) k × d is the model parameter of the discriminator, which fully characterises the empirical measure of M . The pre-specified noise random variable Z = ( Z t i ) n T -1 i =0 is the discretised Brownian motion on [0 , 1] with time mesh 1 n T . The induced distribution of the fake data is given by G θ g ( Z ) . Hence, the min-max objective of our basic version PCF-GAN is

We apply mini-batch gradient descent to optimise the model parameters of the generator and discriminator in an alternative manner. In particular, to compute gradients of the discriminator parameter θ M , we use the efficient backpropagation algorithm through time introduced in [26], which effectively leverages the Lie group-valued outputs and the recurrence structure of the unitary feature. The initialisation of θ M for the optimisation is outlined in the Appendix B.4.1.

Learning time-dependent Ornstein-Uhlenbeck process Following [19], we apply the proposed PCF-GAN to the toy example of learning the distribution of synthetic time series data simulated via the time-dependent Ornstein-Uhlenbeck (OU) process. Let ( X t ) t ∈ [0 ,T ] be an R -valued stochastic process described by the SDE, i.e., d X t = ( µt -θ X t ) dt + σd B t with X 0 ∼ N ( 0 , 1 ) , where ( B t ) t ∈ [ 0 , T ] is 1D Brownian motion and N (0 , 1) is the standard normal distribution. We set µ = 0 . 01 , θ = 0 . 02 , σ = 0 . 4 and time discretisation δt = 0 . 1 . We generate 10000 samples from t = 0 to t = 63 , down-sampled at each integer time point. Figure 2 shows that the synthetic data generated by our GAN model, which uses the EPCFD discriminator, is visually indistinguishable from true data. Also, our model accurately captures the marginal distribution at various time points.

35

30

Real

Generated

0.4

0.3

0.5

Step 10

Real

Generated

0.40

0.35

OE°0

0.25

Step 20

Real

Generated

0.25

0.20

Step 30

Real

Generated

0.20

0.15

25

0.2

0.15

0.10

0Z

0.1

0.10

0.05

0.05

0.0

2

0.00

O

2

4

0.00

口

5

10

15

10

0.200

0.175

0.150

Step 40

Real

Generated

0.10

0.12

Step 50

Real

Generated

0.10

0.08

Step 60

Real

Generated

0.100

0.125

0.08

0.06

专

0.06

0.075

0.050

0.025

0.04

0.02

0.04

0.02

01

20

30

0.000

5

10

1.5

20

0.00

10

20

0.00

10

20

30

40

4.2 PCF-GAN: learning with PCFD and sequential embedding

In order to effectively learn the distribution of high-dimensional or complex time series, using solely the EPCF loss as the GAN discriminator fails to be the best approach, due to the computational limitations imposed by the sample size k and the order m of EPCFD. To overcome this issue, we adopt the approach [41, 25], and train a generator that matches the distribution of the embedding of time series via the auto-encoder structure. Figure 3 illustrates the mechanics of our model.

To proceed, let us first recall the generator G θ g : Z → X and introduce the embedding layer F θ f , which maps X to Z (the noise space). Here θ f is the model parameters of the embedding layer and will be learned from data. To this end, it is natural to optimize the model parameters θ g of the generator by minimising the generative loss L generator, which is the EPCFD distance of the embedding between true distribution X and synthetic distribution G θ g ( Z ) ; in formula,

Encoder ( F θ f ) -decoder ( G θ g ) structure : The motivation to consider the auto-encoder structure is based on the observation that the embedding might be degenerated when optimizing L generator. For example, no matter whether true and synthetic distribution agrees or not, F θ f could be simply a constant function to achieve the perfect generator loss 0. Such a degeneracy can be prohibited if F θ f is injective. In heuristic terms, the "good" embedding should capture essential

Reconstruction

Real time-series

Fake time-series

81:T

X1:T

YI:T

G0g

G0g

Fef

Latent embedding

Noise

Z1:T

noise Z1:T

Reconstructed

RegularizationLoss

ReconstructionLoss

Lregularization

Lrecovery

GeneratorLoss

information about real time series of X and allows the reconstruction of time series X from its embedding F θ f ( X ) . This motivates us to train the embedding F θ f such that F θ f ◦ G θ g is close to the identity map. If this condition is satisfied, it implies that F θ f and G θ g are pseudo-inverses of each other, thereby ensuring the desired injectivity. In this way, F θ f and G θ g serve as the encoder and decoder of raw data, respectively.

To impose the injectivity of F θ f , we consider two additional loss functions for training θ f as follows:

Reconstruction loss L recovery : It is defined as the l 2 samplewise distance between the original and reconstructed noise by F θ f ◦ G θ g , i.e., L recovery = E [ | Z -F θ f ( G θ g ( Z )) | 2 ] . Note that L recovery = 0 implies that F θ f ( G θ g ( z )) = z , for any sample z in the support of Z almost surely.

Regularization loss L regularization : It is proposed to match the distribution of the original noise variable Z and embedding of true distribution X . It is motivated by the observation that if the perfect generator G θ ( Z ) = X and F θ f ◦ G θ g is the identity map, then Z = F θ f ( X ) . Specifically,

where we distinguish θ ′ M from θ M in L generator . The regularization loss effectively stabilises the training and resolves the mode collapse [41] due to the lack of infectivity of the embedding.

Training the embedding parameters θ f : The embedding layer F θ f aims to not only discriminate the real and fake data distributions as a critic, but also preserve injectivity. Hence we optimise the

embedding parameter θ f by the following hybrid loss function:

where λ 1 and λ 2 are hyper-parameters that balance the three losses.

Training the EPCFD parameters ( θ M , θ ′ M ) : Note that L generator and L regularization have trainable parameters of EPCFD, i.e., θ M and θ ′ M . Similar to the basic PCF-GAN, we optimize θ M and θ ′ M by maximising the EPCFD to improve the discriminative power.

By doing so, we enhance the discriminative power of EPCFD θ M and EPCFD θ ′ M . Consequently, this facilitates the training of the generator such that the embedding of the true data aligns with both the noise distribution and the reconstructed noise distribution.

Differentiability of EPCFD with respect to parameters of the embedding layer and generators are guaranteed by Theorem 3.6, as long as F θ f ◦ G θ g satisfies the Lipschitz condition thereof. Let us also stress on two key advantages of our proposed PCF-GAN. First, it possesses the ability to generate synthetic time series with reconstruction functionality, thanks to the auto-encoder structure in PCF-GAN. Second, by virtue of the uniform boundedness of PCFD shown in Lemma 3.5, our PCF-GAN does not require any additional gradient constraints of the embedding layer and EPCFD parameters, in contrast to other MMD-based GANs and Wasserstein-GAN. It helps with the training efficiency and alleviates the vanishing gradient problem in training sequential networks like RNNs.

We provide the pseudo-code for the proposed PCF-GAN in Algorithm 1.

Algorithm 1 PCF-GAN.

1: Input: P d (real time series distribution), P z (noise distribution), θ M , θ ′ M , θ f , θ g (model parameters for EPCFD, critic F and generator G ), λ 1 , λ 2 ∈ R + (penalty weights), b (batch size), η ∈ R (learning rate), n c the iteration number of discriminator per generator update, .

2: while θ , θ ′ , θ , θ , θ not converge do

19: end while

5 Numerical Experiments

To validate its efficacy, we apply our proposed PCF-GAN to a broad range of time series data and benchmark with state-of-the-art GANs for time series generation using various test metrics. Full details on numerics (dataset, evaluation metrics, and hyperparameter choices) are in Appendix C. Additional ablation studies and visualisations of generated samples are reported in Appendix D.

Baselines : We take Recurrent GAN (RGAN)[12], TimeGAN [45], and COT-GAN [43] as benchmarking models. These are representatives of GANs exhibiting strong empirical performance for time series generation. For fairness, we compare our model to the baselines while fixing the generators and embedding/discriminator to be the common sequential neural network (2 layers of LSTMs).

Dataset : We benchmark our model on four different time series datasets with various characteristics: dimensions, sample frequency, periodicity, noise level, and correlation. (1) Rough Volatility : Highfrequency synthetic time series data with low noise-to-signal. (2) Stock : The daily historical data on ten publicly traded stocks from 2013 to 2021, including as features the volume and high, low, opening, closing, and adjusted closing prices. (3) Beijing Air Quality [47]: An UCI multivariate time series on hourly air pollutants data from different monitoring sites. (4) EEG Eye State [38]: An UCI dataset of a high frequency and continuous EEG eye measurement. We summarise the key statistics of the datasets in Table 1.

Evaluation metrics : The following three metrics are used to assess the quality of generative models. For time series generation/reconstruction, we compare the true and fake/reconstructed distribution by G θ g ◦ F θ f via the below test metrics. (1) Discriminative score [45]: We train a post-hoc classifier to distinguish true and fake data. We report the classification error on the test data. The better generative model yields a lower classification error, as it means that the classifier struggles to differentiate between true and fake data. (2) Predictive score [45, 12]: We train a post-hoc sequence-to-sequence regression model to predict the latter part of a time series given the first part from the generated data. We then evaluate and report the mean square error (MSE) on the true time series data. The lower MSE indicates better the generated data can be used to train a predictive model. (3) Sig-MMD [9, 42]: We use MMD with the signature feature as a generic metric on time series distribution. Smaller the values, indicating closer the distributions, are better. To compute three evaluation metrics, we randomly generated 10,000 samples of true and synthetic (reconstructed) distribution resp. The mean and standard deviation of each metric based on 10 repeated random sampling are reported.

5.1 Time series generation

Table 2 indicates that PCF-GAN consistently outperforms the other baselines across all datasets, as demonstrated by all three test metrics. Specifically, in terms of the discriminative score, PCFGAN achieves a remarkable performance with values of 0 . 0108 and 0 . 0784 on the Rough volatility and Stock datasets, respectively. These values are 61% and 39% lower than those achieved by the second-best model. Regarding the predictive score, PCF-GAN achieves the best result across all four datasets. While COT-GAN surpasses PCF-GAN in terms of the Sig-MMD metric on the EEG dataset, PCF-GAN consistently outperforms the other models in the remaining three datasets. Additionally, to assess the fitting on auto-correlation, cross-correlation and marginal distribution, we include the corresponding numerical results in Table 4 in Appendix D.4. For a qualitative analysis of generative quality, we provide the visualizations of generated samples for all models and datasets in Appendix D without selective bias. Furthermore, to showcase the effectiveness of our auto-encoder architecture for the generation task, we present an ablation study in Appendix D.

5.2 Time series reconstruction

As TimeGAN is the only baseline model incorporating reconstruction capability, for reconstruction tasks we only compare with TimeGAN. The reconstructed examples of time series using both PCF-GAN and TimeGAN are shown in Figure 4; see Appendix D for more samples.

Visually, the PCF-GAN achieves better reconstruction results than TimeGAN by producing more accurate reconstructed time series samples. Notably, the reconstructed samples from PCF-GAN preserve the temporal dependency of original time series for all four datasets, while some reconstructed samples from TimeGAN in EEG and Stock datasets are completely mismatched. This is further quantified in Table 2 on the reconstruction task, where the reconstructed samples from PCF-GAN consistently outperform those from TimeGAN in terms of all test metrics.

Real

PCF-GAN

TimeGAN

5.3 Training stability and efficiency

Figure 5 demonstrates the training progress of the PCF-GAN on RV dataset. Compared to the fluctuating generator loss typically observed in traditional GANs, the PCF-GAN yields better convergence by leveraging the autoencoder structure. This is achieved by minimising reconstruction and regularisation losses, which ensures the injectivity of F θ f and enables production of a semantic embedding throughout the training process. The decay of generator loss in the embedding space directly reflects the improvement in the quality of the generated time series. This is particularly useful for debugging and conducting hyperparameter searches. Furthermore, decay in both recovery and regularisation loss signifies the enhanced performance of the autoencoder.

16

generatorloss

14

recoveryloss

regularization loss

12

10

8

6

4

2

0

0

5000

10000

15000

20000

25000

30000

iterations

By leveraging the effective critic F θ f , we achieve enhanced performance with a moderate increase in parameters (ranging from 1200 to 6400) within θ M of EPCFD. The training of these additional parameters is highly efficient in PCF-GAN, while still outperforming all baseline models. Specifically, our algorithm is approximately twice as fast as TimeGAN (using three extra critic modules) and three times as fast as COT-GAN (with one additional critic module and the Sinkhorn algorithm). However, it takes 1.5 times as long as RGAN due to the extra training required on θ M .

6 Conclusion & Broader impact

Conclusion We introduce a novel, principled and efficient PCF-GAN model based on PCF for generating high-fidelity sequential data. With theoretical support, it achieves state-of-the-art generative performance with additional reconstruction functionality in various tasks of time series generation.

Limitation and future work In this work, we use LSTM-based networks for the autoencoder and do not explore other sequential models (e.g., transformers). The suitable choice of network architecture for the autoencoder may further improve the efficacy of the proposed PCF-GAN on more complicated data, e.g. , video and skeletal human action sequence, which merits further investigation. As a distance metric on time series, PCFD can be flexibly incorporated with other advanced generators of time

series GAN models, hence may further improve the performance. For example, one can replace the average cross-entropy loss used in [17, 39] and the Wasserstein distance in [36] by PCFD, with some simple modifications on the discriminators. Furthermore, although we establish the link between PCFD and MMD, it is interesting to design efficient algorithms to compute the kernel specified in Appendix B.3.

Broader impact Like other GAN models, this model has the potential to aid data-hungry algorithms by augmenting small datasets. Additionally, it can enable data sharing in domains such as finance and healthcare, where sensitive time series data is plentiful. However, it is important to acknowledge that the generation of synthetic data also carries the risk of potential misuse ( e.g. generating fake news).

Acknowledgments and Disclosure of Funding

The research of SL is supported by NSFC (National Natural Science Foundation of China) Grant No. 12201399, and the Shanghai Frontiers Science Center of Modern Analysis. This research project is also supported by SL's visiting scholarship at New York University-Shanghai. HN is supported by the EPSRC under the program grant EP/S026347/1 and The Alan Turing Institute under the EPSRC grant EP/N510129/1. LH is supported by University College London and the China Scholarship Council under the UCL-CSC scholarship (No. 201908060002). SL and HN are supported by the SJTU-UCL joint seed fund WH610160507/067. HN and HL are supported by the Ecosystem Leadership Award under the EPSRC Grant OobfJ22 / 100020 and The Alan Turing Institute in part. HN is grateful to Jiajie Tao and Zijiu Lyu for proofreading the paper.

References

Abdul Fatir Ansari, Jonathan Scarlett, and Harold Soh. A characteristic function approach to deep implicit generative modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 7478-7487, 2020.

Martin Arjovsky and Léon Bottou. Towards principled methods for training generative adversarial networks. arXiv preprint arXiv:1701.04862 , 2017.

Samuel A Assefa, Danial Dervovic, Mahmoud Mahfouz, Robert E Tillman, Prashant Reddy, and Manuela Veloso. Generating synthetic data in finance: opportunities, challenges and pitfalls. In Proceedings of the First ACM International Conference on AI in Finance , pages 1-8, 2020.

Steven M Bellovin, Preetam K Dutta, and Nathan Reitinger. Privacy and synthetic datasets. Stan. Tech. L. Rev. , 22:1, 2019.

Lukas Biewald. Experiment tracking with weights and biases, 2020. Software available from wandb.com.

Horatio Boedihardjo and Xi Geng. Sl_2 (r)-developments and signature asymptotics for planar paths with bounded variation. arXiv preprint arXiv:2009.13082 , 2020.

Ilya Chevyrev, Terry Lyons, et al. Characteristic functions of measures on geometric rough paths. The Annals of Probability , 44(6):4049-4082, 2016.

Ilya Chevyrev, Vidit Nanda, and Harald Oberhauser. Persistence paths and signature features in topological data analysis. IEEE transactions on pattern analysis and machine intelligence , 42(1):192-202, 2018.

Ilya Chevyrev and Harald Oberhauser. Signature moments to characterize laws of stochastic processes. Journal of Machine Learning Research , 23(176):1-42, 2022.

Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN encoderdecoder for statistical machine translation. arXiv preprint arXiv:1406.1078 , 2014.

Kacper P Chwialkowski, Aaditya Ramdas, Dino Sejdinovic, and Arthur Gretton. Fast twosample testing with analytic representations of probability measures. Advances in Neural Information Processing Systems , 28:1981-1989, 2015.

Cristóbal Esteban, Stephanie L Hyland, and Gunnar Rätsch. Real-valued (medical) time series generation with recurrent conditional gans. arXiv preprint arXiv:1706.02633 , 2017.

Marián Fabian, Petr Habala, Petr Hájek, Vicente Montesinos Santalucía, Jan Pelant, and Václav Zizler. Functional analysis and infinite-dimensional geometry , volume 8 of CMS Books in Mathematics/Ouvrages de Mathématiques de la SMC . Springer-Verlag, New York, 2001.

Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Improved training of Wasserstein GANs. Advances in Neural Information Processing Systems , 30:5769-5779, 2017.

Ben Hambly and Terry Lyons. Uniqueness for the signature of a path of bounded variation and the reduced path group. Annals of Mathematics , pages 109-167, 2010.

Christopher R Heathcote. The integrated squared error estimation of parameters. Biometrika , 64(2):255-264, 1977.

Jinsung Jeon, Jeonghak Kim, Haryong Song, Seunghyeon Cho, and Noseong Park. GT-GAN: General purpose time series synthesis with generative adversarial networks. Advances in Neural Information Processing Systems , 35:36999-37010, 2022.

Patrick Kidger, Patric Bonnier, Imanol Perez Arribas, Cristopher Salvi, and Terry Lyons. Deep signature transforms. Advances in Neural Information Processing Systems , 32:3082-3092, 2019.

Patrick Kidger, James Foster, Xuechen Li, and Terry J Lyons. Neural SDEs as infinitedimensional GANs. In International Conference on Machine Learning , pages 5453-5463. PMLR, 2021.

Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014.

Achim Klenke. Probability theory: a comprehensive course . Springer Science & Business Media, 2013.

Erich Leo Lehmann, Joseph P Romano, and George Casella. Testing statistical hypotheses , volume 3. Springer, 2005.

Daniel Levin, Terry Lyons, and Hao Ni. Learning from the past, predicting the statistics for the future, learning an evolving system. arXiv preprint arXiv:1309.0260 , 2013.

Chun-Liang Li, Wei-Cheng Chang, Yu Cheng, Yiming Yang, and Barnabás Póczos. MMD GAN: Towards deeper understanding of moment matching network. Advances in Neural Information Processing Systems , 30:2200-2210, 2017.

Shengxi Li, Zeyang Yu, Min Xiang, and Danilo Mandic. Reciprocal adversarial learning via characteristic functions. Advances in Neural Information Processing Systems , 33:217-228, 2020.

Hang Lou, Siran Li, and Hao Ni. Path development network with finite-dimensional Lie group representation. arXiv preprint arXiv:2204.00740 , 2022.

Terry Lyons. Rough paths, signatures and the modelling of functions on streams. arXiv preprint arXiv:1405.4537 , 2014.

Terry J Lyons. Differential equations driven by rough signals. Revista Matemática Iberoamericana , 14(2):215-310, 1998.

Terry J Lyons, Michael Caruana, and Thierry Lévy. Differential equations driven by rough paths . Springer, 2007.

Terry J Lyons and Weijun Xu. Hyperbolic development and inversion of signature. Journal of Functional Analysis , 272(7):2933-2955, 2017.

Hao Ni, Lukasz Szpruch, Marc Sabate-Vidales, Baoren Xiao, Magnus Wiese, and Shujian Liao. Sig-Wasserstein GANs for time series generation. In Proceedings of the Second ACM International Conference on AI in Finance , pages 1-8, 2021.

Hao Ni, Lukasz Szpruch, Magnus Wiese, Shujian Liao, and Baoren Xiao. Conditional sigwasserstein gans for time series generation. arXiv preprint arXiv:2006.05421 , 2020.

Kalyanapuram R. Parthasarathy. Probability measures on metric spaces , volume 3 of Probability and Mathematical Statistics . Academic Press, Inc., New York-London, 1967.

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32 , pages 8024-8035. Curran Associates, Inc., 2019.

Vibhor Rastogi and Suman Nath. Differentially private aggregation of distributed time-series with transformation and encryption. In Proceedings of the 2010 ACM SIGMOD International Conference on Management of data , pages 735-746, 2010.

Carl Remlinger, Joseph Mikael, and Romuald Elie. Conditional loss and deep euler scheme for time series generation. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 8098-8105, 2022.

Jinfu Ren, Yang Liu, and Jiming Liu. EWGAN: Entropy-based Wasserstein GAN for imbalanced learning. In Proceedings of the AAAI conference on artificial intelligence , volume 33, pages 10011-10012, 2019.

Oliver Roesler, Lucas Bader, Jan Forster, Yoshikatsu Hayashi, Stefan Heßler, and David Suendermann-Oeft. Comparison of eeg devices for eye state classification. Proc. of the AIHLS , 2014.

Ali Seyfi, Jean-Francois Rajotte, and Raymond Ng. Generating multivariate time series with COmmon Source CoordInated GAN (COSCI-GAN). Advances in Neural Information Processing Systems , 35:32777-32788, 2022.

Bharath K Sriperumbudur, Arthur Gretton, Kenji Fukumizu, Bernhard Schölkopf, and Gert RG Lanckriet. Hilbert space embeddings and metrics on probability measures. The Journal of Machine Learning Research , 11:1517-1561, 2010.

Akash Srivastava, Lazar Valkov, Chris Russell, Michael U Gutmann, and Charles Sutton. VEEGAN: Reducing mode collapse in GANs using implicit variational learning. Advances in Neural Information Processing Systems , 30:3310-3320, 2017.

Csaba Toth and Harald Oberhauser. Bayesian learning from sequential data using Gaussian processes with signature covariances. In International Conference on Machine Learning , pages 9548-9560. PMLR, 2020.

Tianlin Xu, Li Kevin Wenliang, Michael Munn, and Beatrice Acciaio. COT-GAN: Generating sequential data via causal optimal transport. Advances in Neural Information Processing Systems , 33:8798-8809, 2020.

Yasin Yazıcı, Chuan-Sheng Foo, Stefan Winkler, Kim-Hui Yap, Georgios Piliouras, and Vijay Chandrasekhar. The unusual effectiveness of averaging in GAN training. In International Conference on Learning Representations , 2019.

Jinsung Yoon, Daniel Jarrett, and Mihaela Van der Schaar. Time-series generative adversarial networks. Advances in Neural Information Processing Systems , 32:5509-5519, 2019.

Kôsaku Yosida. Functional analysis. Sixth edition , volume 123 of Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences] . Springer-Verlag, Berlin-New York, 1980.

Shuyi Zhang, Bin Guo, Anlan Dong, Jing He, Ziping Xu, and Song Xi Chen. Cautionary tales on air-quality improvement in beijing. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences , 473(2205):20170457, 2017.

Robert J Zimmer. Essential results of functional analysis . University of Chicago Press, 1990.

In Appendix A, we collect some notations and properties for paths and unitary feature of a path. Appendix B gives a thorough introduction to the distance function via the path characteristics function. Detailed proofs for the theoretical results on PCFD are provided. Appendix C discusses experimental details and Appendix D presents supplementary numerical results.

A Preliminaries

A.1 Paths with bounded variation

Definition A.1. Let X : [0 , T ] → R d be a continuous path. The total variation of X on the interval [0 , T ] is defined by

where the supremum is taken over all finite partitions D = { t ℓ } N ℓ =0 of [0 , T ] . When Tot . Var . ( X ) is finite, say that X is a path of bounded variation (BV-path) on [0 , T ] and denote X ∈ BV ( [0 , T ]; R d ) .

BV-paths can be defined without the continuity assumption, but we shall not seek for greater generality in this work. It is well-known that

defines a norm (the BV-norm). There is a more general notion of paths of finite p -variation for p ≥ 1 (see [29]), where the case p = 1 corresponds to BV-paths discussed above. We restrict ourselves to p = 1 , as this is sufficient for the study of sequential data in practice as piecewise linear approximations of continuous paths.

Definition A.2. (Concatenation of paths) Let X : [0 , s ] → R d and Y : [ s, t ] → R d be two continuous paths. Their concatenation denoted as the path X ⋆ Y : [0 , t ] → R d is defined by

Definition A.3 (Tree-like equivalence) . A continuous path X : [0 , T ] → R d is called tree-like if there is an R -tree T , a continuous function ϕ : [0 , T ] →T , and a function ψ : T → R d such that ϕ (0) = ϕ ( T ) and X = ψ ◦ ϕ .

Let ← -X : [0 , T ] → R d denote the time-reversal of continuous path X , namely that ← -X ( t ) = X ( T -t ) . We say that X and Y are in tree-like equivalence (denoted as X ∼ τ Y ) if X ⋆ ← -Y is tree-like.

An important example is when path X is a time re-parameterisation of Y . That is, for X ∈ BV ( [0 , T ]; R d ) , take a nondecreasing surjection λ : [0 , T ] → [ T 1 , T 2 ] , and take X ( t ) = Y ( λ ( t )) .

A.2 Matrix groups and algebras

The unitary group and symplectic group are subsets of the space of m × m matrices:

where J m := ( 0 I m -I m 0 ) and I m ∈ C m × m is the identity. Their corresponding Lie algebras are

The unitary group is compact and is a group of isometries of matrix multiplication with respect to the Hilbert-Schmidt norm. Such properties are crucial for establishing theorems and properties related to the path characteristic function (PCF), as discussed in subsequent sections.

The compact symplectic group Sp( m ) is the simply-connected maximal compact real Lie subgroup of Sp(2 m, C ) . It is the real form of Sp(2 n, C ) , and satisfies

Note that U ( m ) and Sp( m ) are both real Lie groups, albeit they have complex entries in general.

A.3 Unitary feature of a path

Recall Definition 2.1 for the unitary feature, reproduced below:

Definition A.4. Let M : R d → u ( m ) be a linear map and let X ∈ BV ( [0 , T ]; R d ) be a BV-path. The unitary feature [ a.k.a. the path development on the unitary group U ( m ) ] of x under M is the solution to the equation

We write U M ( x ) := y T .

Definition 2.1 is motivated by [7, §4]. Consider M ∈ L ( R d , u ( H fd ) ) with H fd ranging over all finite-dimensional complex Hilbert spaces. Extend M by naturality to the tensor algebra over R d ; that is, define ˜ M : T (( R d )) ≡ ⊕ ∞ k =0 ( R d ) ⊗ k → u ( H fd ) by linearity and the following rule:

Then denote by A ( R d ) the totality of such ˜ M . Any element in A ( R d ) is a unitary representation of the Lie group G (( R d )) := { group-like elements in T (( R d ))} . See [7, p.4059].

The following two lemmas are contained in [26].

Lemma A.5. [Multiplicativity] Let X ∈ BV ( [0 , s ] , R d ) and Y ∈ BV ( [ s, t ] , R d ) . Denote by X ∗ Y their concatenation: ( X ∗ Y )( v ) = X ( v ) for v ∈ [0 , s ] and Y ( v ) -Y ( s ) + X ( s ) for v ∈ [ s, t ] . Then U M ( X ∗ Y ) = U M ( X ) · U M ( Y ) for all M ∈ L ( R d , u ( m ) ) .

We shall compute by Lemma A.5 and Example 2.2 the unitary feature of piecewise linear paths.

Lemma A.6 (Invariance under time-reparametrisation) . Let X ∈ BV([0 , T ] , R d ) and let λ : t ↦→ λ t be a non-decreasing C 1 -diffeomorphism from [0 , T ] onto [0 , S ] . Define X λ t := X λ t for t ∈ [0 , T ] . Then, for all M ∈ L ( R d , u ( m ) ) and for every s, t ∈ [0 , T ] , it holds that U M ( X λ s ,λ t ) = U M ( X λ s,t ) .

A key property of the unitary feature is that it completely determines the law of random paths:

̸

Theorem A.7 (Uniqueness of unitary feature) . For any two paths X 1 = X 2 in X , there exists an M ∈ L ( R d , u ( m ) ) with some m ∈ N such that U M ( X 1 ) = U M ( X 2 ) .

̸

̸

̸

Proof. For X 1 = X 2 in X , by uniqueness of signature over BV-paths ( cf. [15]) one has Sig( X 1 ) = Sig( X 2 ) in G (( R d )) . Here we use the fact that the signatures of BV-paths are group-like elements in the tensor algebra. Then, as A ( R d ) separates points over G (( R d )) ( cf. [7, Theorem 4.8]), there is M ∈ L ( R d , u ( m ) ) such that ˜ M [Sig( X 1 )] = ˜ M [Sig( X 2 )] ; hence M ( X 1 ) = M ( X 2 ) . Therefore, by considering the U ( m ) -valued equation d Y t = Y t · M (d X t ) with Y 0 = I m , we conclude that U M ( X 1 ) = U M ( X 2 ) .

̸

Theorem A.8 (Universality of unitary feature) . Let K ⊂ BV ( [0 , T ]; R d ) be a compact subset. For any continuous function f : K → C and any ϵ > 0 , there exists an m ⋆ ∈ N and finitely many M 1 , · · · , M N ∈ L ( R d , u ( m ⋆ ) ) as well as L 1 , . . . , L N ∈ L ( U ( m ⋆ ); C ) , such that

Proof. It follows from [26, Theorem A.4] and the universality of signature in [8] that Eq. (12) holds with M j ∈ L ( R d , ⊕ m ∈ N u ( m ) ) and L j ∈ L (⊕ m ∈ N U ( m ); C ) and ϵ/ 2 in place of ϵ . By a simple approximation via restricting the ranges of M j and domains of L j , we may obtain (without relabelling) M j ∈ L ( R d , ⊕ m ⋆ m =0 u ( m ) ) and L j ∈ L ( ⊕ m ⋆ m =0 U ( m ); C ) that verify Eq. (12). We conclude by the flag structure of U (1) ⊂ U (2) ⊂ U (3) ⊂ . . . and u (1) ⊂ u (2) ⊂ u (3) ⊂ . . . .

̸

̸

B Path Characteristic loss

B.1 Path Characteristic function

Theorem B.1. Let X be a X -valued random variable with associated probability measure P X . The path characteristic function Φ X uniquely characterises P X .

̸

Proof. Assume that P X 1 = P X 2 . Then Sig( X 1 ) = Sig( X 2 ) by the uniqueness of signature over BV-paths ( cf. [15]). It is proved in [26, Lemma 2.6] that U M ( X i ) = ˜ M (Sig( X i )) for any M ∈ L ( R d , u ( m ) ) ; i ∈ { 1 , 2 } . Hence Φ X i = ∫ X ˜ M (Sig( x )) d P X i ( x ) . But as in the proof of Theorem A.7, A ( R d ) separates points over G (( R d )) ( cf. [7, Theorem 4.8]) and the signature of BV-paths lies in G (( R d )) . Therefore, there is an M ∈ L ( R d , u ( m ) ) such that Φ X = Φ Y .

̸

B.2 Distance metric via path characteristic function

Lemma B.2. PCFD M in Eq. (5) defines a pseudometric on the path space X for any m ∈ N and M∈P ( L ( R d , u ( m ) )) . In addition, suppose that {M j } j ∈ N is a countable dense subset in P ( L ( R d , ⊕ m ∈ N u ( m ) )) . Then the following defines a metric on X :

In Lemma B.2 above, L ( R d , ⊕ m ∈ N u ( m ) ) ∼ = ( R d ) ∗ ̂ ⊗ π (⊕ m ∈ N u ( m ) ) where ̂ ⊗ π is the completion of the projective tensor product and ⊕ m ∈ N u ( m ) is a Banach space under the norm ∥ T ∥ := ∑ m ∈ N ∥ ∥ T ( m ) ∥ ∥ HS < ∞ . Here T ( m ) denotes the m th -projection of T on u ( m ) . Therefore, such a sequence {M j } j ∈ N always exists since P ( L ( R d , ⊕ m ∈ N u ( m ) )) , being the space of Borel probability measures over a Polish space, is itself a Polish space. See [33].

Proof. Non-negativity, symmetry, and that PCFD M ( X , X ) = 0 are clear. That PCFD M ( X , Y ) ≤ PCFD M ( X , Z )+PCFD M ( Z , Y ) follows from the triangle inequality of the Hilbert-Schmidt norm and the linearity of expectation. This shows that PCFD M is a pseudometric for each M .

In addition, PCFD M ( X , Y ) = 0 implies that

So, if P M is supported on the whole of L ( R d , u ( m ) ) , then Φ X ( M ) = Φ Y ( M ) for any M ∼ P M .

Now, by density of {M j } j ∈ N in P ( L ( R d , ⊕ m ∈ N u ( m ) )) , there exists a subsequence { M j ( m ) } such that M j ( m ) has full support on L ( R d , u ( m ) ) for each m ∈ N . Thus, ˜ PCFD( X , Y ) = 0 implies that Φ X = Φ Y on a dense subset of L ( R d , u ( m ) ) for every m ∈ N . We conclude by the characteristicity Theorem 3.2 and a continuity argument.

Lemma B.3 (Lemma 3.5) . Let M be an L ( R d , u ( m ) ) -valued random variable. Then for any BV ( [0 , T ]; R d ) -valued random variables X and Y , it holds that

Proof. As ( C m × m , ∥ · ∥ HS ) is a Hilbert space, from the Pythagorean theorem one deduces that d 2 HS ( Φ X ( M ) , Φ Y ( M )) ≤ ∥ Φ X ( M ) ∥ 2 HS + ∥ Φ Y ( M ) ∥ 2 HS . Both Φ X ( M ) , Φ Y ( M ) are expectations of U ( m ) -valued random variables, and ∥ U ∥ HS := √ tr( UU ∗ ) = √ tr( I m ) = √ m for U ∈ U ( m ) . Thus d HS ( Φ X ( M ) , Φ Y ( M )) ≤ √ 2 m . We take expectation over P M to conclude.

̸

The result below is formulated in terms of the Hilbert-Schmidt norm of matrices in C m × m . Any other norm on C m × m is equivalent to that, modulo a constant depending on m only. In fact, the strict inequality ∥ T ∥ op ≤ ∥ T ∥ HS for T ∈ C m × m holds. See, e.g. , [48, Lemma 3.1.10, p.55].

Lemma B.4. Let L, ˜ L : [ a, b ] → R d be two linear paths, and let M ∈ u ( m ) d := L ( R d , u ( m ) ) as before. Denote by ∥ · ∥ e the usual Euclidean norm on R d and ∥| M ∥| the operator norm of M : ( R d , ∥ · ∥ e ) → ( u ( m ) , ∥ · ∥ HS ) . Then we have

Proof. Let Γ( t, s ) := M ( (1 -s ) L ( t ) + s ˜ L ( t ) ) with t ∈ [ a, b ] and s ∈ [0 , 1] . This is the linear interpolation between Γ( t, 0) = M ◦ L ( t ) and Γ( t, 1) = M ◦ ˜ L ( t ) . Then we have

thanks to an identity for differentiation of matrix exponential and the inequality ∥ T 1 T 2 ∥ HS ≤ ∥ T 1 ∥ op ∥ T 2 ∥ HS . Here e (1 -r )Γ( t, 1 -s ) and e r Γ( t,s ) take values in U ( m ) , hence of operator norm 1 for any parameters t, s, r . So we infer that

,

where the first inequality holds for Bochner integrals. See [46, Corollary 1, p.133].

Lemma B.5 (Subadditivity of unitary feature) . Let X,Y ∈ BV ( [0 , T ]; R d ) be BV-paths, and U M be the unitary feature associated with M ∈ L ( R d , u ( m ) ) = u ( m ) d . For any 0 < t < T we have

Proof. We apply the multiplicative property of unitary feature in Lemma A.5, the triangle inequality, and the unitary invariance of the Hilbert-Schmidt norm to estimate that

Proposition B.6. For X , Y ∈ X , the unitary feature U M ( X ) with M ∈ L ( R d , u ( m ) ) = u ( m ) d satisfies

where Tot . Var . [ X -Y ] denotes the total variation over [0 , T ] of the path X -Y .

Proof. Given BV-paths X , Y with the same initial point, there are piecewise linear approximations { X n } , { Y n } with common partition 0 = t 0 < t 1 < · · · < t n = T converging respectively to X and Y in the p -variation metric; p ∈ (1 , 2) . Applying Lemma B.5 recursively, we obtain that

By definition of unitary feature and Lemma B.4, one deduces that

We may now conclude by taking supremum over all partitions and sending n →∞ with the limit. This is a consequence of continuity of the Itô map with respect to the driving path in p -variation topology, since the vector field in (4) is Lipschitz ([27, Theorem 1.20]).

Theorem B.7 (Dependence on continuous parameter) . Let X and Z be subsets of BV ( [0 , T ]; R d ) , (Θ , ρ ) be a metric space, Q be a Borel probability measure on Z , and M be a Borel probability measure on u ( m ) d . Assume that g : Θ × Z → X , ( θ, Z ) ↦→ g θ ( Z ) is Lipschitz in θ such that Tot . Var . [ g θ ( Z ) -g θ ′ ( Z )] ≤ ω ( Z ) ρ ( θ, θ ′ ) . In addition, suppose that E M ∼ P M [ |∥ M |∥ 2 ] < ∞ and E Z ∼ Q [ ω ( Z )] < ∞ . Then PCFD M ( g θ ( Z ) , X ) is Lipschitz in θ .

Moreover, it holds that

for any θ, θ ∈ Θ , Z ∈ Z , X ∈ X , and M∈P u ( m ) .

Proof. As PCFD M is a pseudometric (Lemma B.2), we have

We may control the right-hand side as follows, using subsequentially the definitions of PCFD and PCF, Proposition B.6, and the assumptions in this theorem:

This completes the proof.

The unitary feature is universal in the spirit of the Stone-Weierstrass theorem; i.e. , continuous functions on paths can be uniformly approximated by linear functionals on unitary features.

As ˜ PCFD metrises the weak topology on the space of path-valued random variables, it emerges as a more sensible distance metric for training time series generations than metrics without this property; e.g. , the Jensen-Shannon divergence.

Theorem B.8 (Metrisation of weak-star topology) . Let K ⊂ X be a compact subset. Suppose that {M j } j ∈ N is a countable dense subset in P ( L ( R d , ⊕ m ∈ N u ( m ) )) . Then ˜ PCFD defined by Eqn. 13 metrises the weak-star topology on P ( K ) . That is, ˜ PCFD( X n , X ) → 0 ⇐⇒ X n d → X as n →∞ , where d → denotes convergence in distribution of random variables.

The metrisability of P ( K ) follows from general theorems in functional analysis: K is a compact metric space, hence C 0 ( K ) is separable ([13, Lemma 3.23]). Then, viewing P ( K ) as the unit circle in [ C 0 ( K ) ] ∗ via Riesz representation, we infer from [13, Proposition 3.24] that P ( K ) is metrisable in the weak-star topology, which is equivalent to the distributional convergence of random variables.

Proof. The backward direction is straightforward. By the Riesz representation theorem of Radon measures, the distributional convergence is equivalent to that ∫ K f d P X n → ∫ K f d P X for all continuous f ∈ C ( K ) . Thus ∥ ∥ ∫ K U M d P X n -∫ K U M d P X ∥ ∥ HS → 0 , namely that Φ X n [ M ] → Φ X [ M ] for each M ∈ L ( R d ; u ( m ) ) . The unitary feature U M is bounded as it is U ( m ) -valued for some m , so we deduce from the dominated convergence theorem that ˜ PCFD( X n , X ) → 0 .

Conversely, suppose that ˜ PCFD( X n , X ) → 0 . Then

for any m ∈ N and M∈P ( L ( R d ; u ( m ) )) , in particular for those with full support. In view of the universality Theorem A.8 proved above, for any fixed ϵ > 0 and any continuous function f ∈ C 0 ( K ) , by approximating f with sum of finitely many L i ◦ U M i (the notations are as in Theorem A.8), one infers that for n and m ⋆ sufficiently large, it holds that

By considering those measures with spt( M ) = L ( R d ; u ( m ⋆ ) ) , we deduce that

This is tantamount to the distributional convergence.

Proof. We first prove the 'if' direction of the statement. By the Portmanteau theorem [21], convergence in distribution X n d → X implies, for any bounded continuous map f , we have E x ∼ P n [ f ( x )] → E x ∼ P [ f ( x )] . Therefore, for any M ∈ L ( R d , u ( m )) , E x ∼ P n [ U M ( x )] → E x ∼ P [ U M ( x )] , which implies || Φ X n ( M ) -Φ X ( M ) || 2 HS → 0 as n →∞ . Hence, it follows that, as n →∞ ,

which completes the proof of 'if' direction.

Now we proceed with the 'only if' direction. By the universality of the unitary path development from Theorem A.8, for any continuous function f : K → C and ϵ > 0 , there exist M 1 , · · · , M N ∈ L ( R d , u ( m )) and L 1 , . . . , L N ∈ L ( U ( m ); C ) such that

or equivalently ∣ ∣ ∣ E x ∼ P [ f ( x )] -∑ N i =1 L i ◦ Φ X ( M i ) ∣ ∣ ∣ < ϵ . For simplicity, we denote E x ∼ P n and E x ∼ P as E n and E respectively. Therefore,

where | L | op := sup x ∈U ( m ) \ 0 | L ( x ) | || x || 2 HS the operator norm. Since PCFD ( X n , X ) := E M ∼ P M || Φ X n ( M ) -Φ X ( M ) || 2 HS → 0 as n →∞ and ϵ is arbitrary, E x ∼ P n [ f ( x )] → E x ∼ P [ f ( x )] for any continuous bounded function f : K → C , which implies X n d → X by the Portmanteau theorem [21].

B.3 Relation with MMD

We now discuss linkages between PCFD and MMD (maximum mean discrepancy) defined over P ( X ) , the space of Borel probability measures (equivalently, probability distributions) on X .

Definition B.9. Given a kernel function κ : X × X → R , the MMD associated to κ is the function MMD κ : P ( X ) ×P ( X ) → R + given as follows: for independent random variables X , Y on X , set

The PCFD can be interpreted as an MMD on measures of the path space with a specific kernel. Compare with [40] for the case of R d .

Proposition B.10 (PCFD as MMD) . Given M∈P ( u ( m ) d ) and X -valued random variables X and Y with induced distributions P X and P Y , resp. Then PCFD M ( X , Y ) = MMD κ ( P X , P Y ) with kernel κ ( x , y ) := E M ∼ P M [∥ ∥ U M ( x ) -U M ( y ) ∥ ∥ HS ] = E M ∼ P M [ tr ( U M ( x ⋆ ← -y ))] .

Throughout, ⋆ designates concatenation of paths and ← -y is the path obtained by running y backwards. The operation x ⋆ ← -y on the path space is analogous to x -y on R d . If y = x , then x ⋆ ← -y is the null path. See the Appendix for proofs and further discussions.

Remark B.11 (Computational cost complexity) . By Proposition B.10, PCFD is an MMD. However, to compute EPCFD, we may directly calculate the expected distance between the PCFs, without going over the kernel calculations in the MMD approach. Our method is significantly more efficient, especially for large datasets. The computational complexity of EPCFD is linear in sample size, whereas the MMD approach is quadratic.

Proof. By definition of PCFD, we have

where Φ X ( M ) = E X ∼ µ [ U M ( X )] and Φ Y ( M ) = E Y ∼ µ [ U M ( Y )] , respectively. Using Fubini's theorem and observing that ⟨ Φ X ( M ) , Φ Y ( M ) ⟩ HS ∈ L 2 ( P M ) (as Φ X ( M ) and Φ Y ( M ) are U ( m ) -valued, they indeed lie in L ∞ ( C m × m ; P M ) as U ( m ) is a compact Lie group under the HilbertSchmidt metric), we deduce that

The first equality then follows from the identification κ ( x , y ) = E M ∼ P M [ ⟨U M ( x ) , U M ( y ) ⟩ HS ] and the definition of MMD κ .

On the other hand, by Lemma A.5 and the definition of the Hilbert-Schmidt inner product on U ( m ) , one may rewrite the kernel function as follows:

where ⋆ denotes the concatenation of paths. The second equality now follows.

B.4 Empirical PCFD

B.4.1 Initialisation of M

Alinear map M ∈ L ( R d , u ( m ) ) can be canonically represented by d independent anti-Hermitian matrices M 1 , . . . , M d ⊂ u ( m ) ∈ C m × m . To sample empiracal distribution of M∈P [ L ( R d , u ( m ) )] from P M , we propose a sampling scheme over u ( m ) . This can also be used as an effective initialisation of model parameters θ M ∈ u ( m ) d × k for the empirical measure of M .

In practice, when working with the Lie algebra u ( m ) , i.e. , the vector space of m × m complex-valued matrices that are anti-Hermitian ( A ∗ + A = 0 , where A ∗ is the transpose conjugate of A ), we view each anti-Hermitian matrix as an 2 m × 2 m real matrix via the isomorphism of R -vector spaces R 2 m × 2 m ∼ = C m × m .

Under the above identification, we have the decomposition

where o ( m ) is the Lie algebra of anti-symmetric m × m real matrices, Sym m × m is the space of m × m real symmetric matrices, z ( m ) consists of m × m real diagonal matrices and Sym m × m / z ( m ) denotes the quotient space of real symmetric matrices by the real diagonal matrices.

The sampling procedure of P M , is given as follows. First, we simulate R m × m valued and i.i.d random variables A and B , whose elements are i.i.d and satisfy the pre-specified distribution in P ( R ) . We have the decomposition B = D ⊕ E , where D and E are a diagonal random matrix and a off-diagonal random matrix respectively. Then we construct the anti-symmetric matrix R = 1 √ 2 ( A T -A ) and matrix in the quotient space Sym m × m / z ( m ) , C = 1 √ 2 ( E T + E ) , and diagonal matrix D . Correspondingly, we simulate u ( m ) -valued random variables by virtue of Eq. (18). As the empirical measure of the M can be fully characterised by the model parameters θ M ∈ u ( m ) d × k , we sample d × k i.i.d. samples which take values in u ( m ) .

B.4.2 Hypothesis test

In the following, we illustrate the efficacy of the proposed trainable EPCFD metric in the context of the hypothesis test on stochastic processes.

Example B.12 (Hypothesis testing on fractional Brownian motion) . Consider the 3-dimensional Brownian motion B := ( B t ) t ∈ [0 ,T ] and the fraction Brownian motion B h := ( B h t ) t ∈ [0 ,T ] with the Hurst parameter h . We simulated 5000 sample paths for both B and B h with 50 discretized time steps. We apply the proposed optimized EPCFD metric to the two-sample testing problem: the null

d

hypothesis H 0 : B d = B h against the alternative H 1 : B = B h . We compare the optimized EPCFD metric with EPCFD metric with the prespecified distribution (PCF) and the characteristic function distance (CF) on the flattened time series [25]. The optimized PCFs are trained on a separate set of 5000 training samples to maximise the PCFD. The details of training can be found at Appendix C.2.

̸

We conduct the permutation test to compute the power of a test (i.e. the probability of correct rejection of the null H 0 ) and Type I error (i.e. the probability of false acceptances of the null H 0 ) for varying h ∈ { 0 . 2 + 0 . 1 · i } 6 i =0 . Note that when h = 0 . 5 , B and B h have the same distribution and hence are indistinguishable. Therefore, the better the test metric is, the test power should be closer to 0 when h is close to 0.5, whereas it should be closer to 1 when h is away from 0 . 5 . We refer to [22] for more in-depth information on hypothesis testing and permutation test statistics.

̸

The plot of the test power and Type 1 error in Figure 6 shows that CF fails in the two sample tests, whilst both EPCFD and optimised EPCFD can distinguish the samples from the stochastic process when h = 0 . 5 . It indicates that the EPCFD captures the distribution of time series much more effectively than the conventional CF metric. Moreover, optimization of EPCFD increases the test power while decreasing the type1 error, particularly when h is closer to 0 . 5 .

C Numerical experiments

C.1 Experimental detail on PCF-GAN

C.1.1 General notes

Codes. The code for reproducing all experiments can be found in https://github.com/ DeepIntoStreams/PCF-GAN .

Software. We conducted all experiments using PyTorch 1.13.1 [34] and performed hyperparameter tuning with Wandb [5]. To ensure reproducibility, we implemented benchmark models based on open-source code from [45, 43, 12]. We used the Ksig library [42] to calculate the Sig-MMD metrics. The codes in [25] were used to compute characteristics function distance in Example B.12.

Computing infrastructure. The experiments were performed on a computational system running Ubuntu 22.04.2 LTS, comprising three Quadro RTX 8000 and two RTX A6000 GPUs. Each experiment was run independently on a single GPU, with the training phase taking between 6 hours to 3 days, depending on the dataset and models used.

1.0

1.0

PCF

Optimized PCF

0.8

0.8

0.6

type1 error

0.6

power

0.4

0.4

0.2

0.2

0.0

0.0

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.2

0.3

0.4

0.5

0.6

0.7

0.8

h

h

Architectures. To ensure a fair comparison, we employed identical network architectures, with two layers of LSTMs having 32 hidden units, for both the generator and discriminator across all models. For the generator, the output of the LSTM (full sequence) was passed through a Tanh activation function and a linear output layer. All generative models take a multi-dimensional discretized Brownian motion as the noise distribution, scaling it to ensure values were controlled within the range [ -1 , 1] . The dimension and scaling factor varied based on the dataset and were specified in the individual sections as below.

The PCF-GAN uses the development layers on the unitary matrix [26] to calculate the PCFD distance. For all experiments, we fixed the unitary matrix size and coefficient λ 2 for the regularization loss to 10 and 1, respectively. The number of unitary linear maps and the coefficient λ 1 of the recovery loss were determined via hyper-parameter tuning, which varied depending on the dataset (see individual section for details).

Regarding TimeGAN, the following approach described in [45] and employed embedding, supervisor, and recovery modules. Each of these modules had two layers of LSTMs with 32 hidden units. For COT-GAN, we used two separate modules for discriminators, each with two layers of LSTMs with 32 hidden units. Based on the recommendation from COT-GAN [43] and informal hyperparameter tuning, we set λ = 10 and ϵ = 1 for all experiments.

Optimisation & training. We used the ADAM optimizer for all experiments [20], with a learning rate of 0.001 for both generators and discriminators. The learning rate for the unitary development network is 0.005. The initial decay rates in the ADAM optimizer are set β 1 = 0 , β 2 = 0 . 9 . The discriminator was trained for two iterations per iteration of the generator's training. For TimeGAN, we followed the training scheme for each module as suggested in the original paper. The batch size was 64 for all experiments. These hyperparameters do not substantially affect the results.

To improve the training stability of GAN, we employed three techniques. Firstly, we applied a constant exponential decay rate of 0.97 to the learning rate for every 500 generator training iterations. Secondly, we clipped the norm of gradients in both generator and discriminator to 10. Thirdly, we used the Cesaro mean of the generator weights after certain iterations to improve the performance of the final model, as suggested by [44]. In all cases, we selected the number of training iterations such that all methods could produce stable generative samples. The optimal number of training iterations and weight averaging scheme varied for each dataset. More details can be found in the respective sections.

Test metrics. Discriminative score. The network architecture of the post-hoc classifier consists of two layers of LSTMs with 16 hidden units. The dataset was split into equal proportions of real and generated time series with labels 0 and 1, with an 80% / 20% train/test split for training and evaluation. The discriminative model was trained for 30 epochs using Adam with a learning rate of 0.001 and a batch size of 64. The best classification error on the test set was reported.

Predictive score. The network architecture of the post-hoc sequence-to-sequence regressor consists of two layers of LSTMs with 16 hidden units. The model was trained on the generated time series

and evaluated on the real time series, using the first 80% of the time series to predict the last 20% . The predictive model was trained for 50 epochs using Adam with a learning rate of 0.001 and a batch size of 64. The best mean squared error on the test set was reported.

Sig-MMD. We directly computed the Sig-MMD by taking inputs of the real time series samples and generated time series samples. We used the radial basis function kernel applying to the truncated signature feature up to depth 5 .

C.2 Time dependent Ornstein-Uhlenbeck process

On this dataset, we experimented with the basic version of PCF-GAN, which only utilized the EPCFD as the discriminator without the autoencoder structure. The batch size is 256. The model are trained with 20000 generator training iterations and weight averaging on the generator was performed over the final 4000 generator training iterations. We used the 2-dimensional discretized Brownian motion as the noise distribution.

C.2.1 Rough volatility model

We followed [31] considering a rough stochastic volatility model for an asset price process ( S t ) t ∈ [0 , 1] , which satisfies the below stochastic differential equation,

where ξ ( t ) denotes the forward variance and B H t denotes the frational Brownian motion (fBM) given by

where ( Z t ) t ∈ [0 , 1] , ( B t ) t ∈ [0 , 1] are (possibly correlated) Brownian motions. In our experiments, the synthetic dataset is sampled from Equation (19) with t ∈ [0 , 1] , H = 0 . 25 , ξ ( t ) ∼ N (0 . 1 , 0 . 01) , η = 0 . 5 and initial condition log( S 0 ) ∼ N (0 , 0 . 05) . Each sample path is sampled uniformly from [0 , 1] with the time discretization δt = 0 . 005 , which consists of 200 time steps. We train the generators to learn the joint distribution of the log price and log volatility.

All methods are trained with 30000 generator training iterations and weight averaging on the generator was performed over the final 5000 generator training iterations. The input noise vectors have 5 dimension and 200 time steps.

For PCF-GAN, the coefficient λ 1 for the recovery loss was 50, and the number of unitary linear maps was 6.

C.2.2 Stocks

We selected 10 large market cap stocks, which are Google, Apple, Amazon, Tesla, Meta, Microsoft, Nvidia, JP Morgan, Visa and P&G, from 2013 to 2021. The dataset consists of 5 features, including daily open, close, high, low prices and volume, available on https://finance.yahoo.com/ lookup . We truncated the long stock time series into 20 days. The data were normalized with standard Min-Max normalisation on each feature channel. The Stock dataset used in our study is similar to the one employed in [25] but with a broader range of assets. Unlike the previous approach, we avoided sampling the time series using rolling windows with a stride of 1 to mitigate the presence of strong dependencies between samples.

All methods are trained with 30000 generator training iterations and weight averaging on the generator was performed over the final 5000 generator training iterations. The input noise vectors have 5 feature dimensions and 20 time steps.

For PCF-GAN, the coefficient λ 1 for the recovery loss was 400, and the number of unitary linear maps was 6.

C.2.3 Beijing Air Quality

Weused a dataset of the air quality in Beijing from the UCI repository [47] and available on https:// archive.ics.uci.edu/ml/datasets/Beijing+Multi-Site+Air-Quality+Data . Each sample is a 10-dimensional time series of the SO2, NO2, CO, O3, PM2.5, PM10 concentrations, temperature, pressure, dew point temperature and wind speed. Each time series is recorded hourly over the course of a day. The data were normalized with standard Min-Max normalisation on each feature channel.

All methods are trained with 20000 generator training iterations and weight averaging on the generator was performed over the final 4000 generator training iterations. The input noise vectors have 5 dimensions and 24 time steps.

For PCF-GAN, the coefficient λ 1 for the recovery loss was 50, and the number of unitary linear maps was 6.

C.2.4 EEG

We obtained the EEG eye state dataset from https://archive.ics.uci.edu/ml/datasets/ EEG+Eye+State . The data is from one continuous EEG measurement on 14 variables with 14980 time steps. We truncated the long time series into smaller ones with 20 time steps. The data are subtracted by channel-wise mean, divided by three times the channel-wise standard deviation, and then passed through a tanh nonlinearity.

All methods are trained with 30000 generator training iterations and weight averaging on the generator was performed over the final 5000 generator training iterations. The input noise vectors have the 8 dimensional and 20 time steps.

For PCF-GAN, the coefficient λ 1 for the recovery loss was 50, and the number of unitary linear maps was 8.

D Supplementary results

D.1 Ablation study

An ablation study was conducted on the PCF-GAN model to evaluate the importance of its various components. Specifically, the reconstruction loss and regularization loss were disabled in order to assess their impact on model performance across benchmark datasets and various test metrics. Table 3 consistently demonstrated that the PCF-GAN model outperformed the ablated versions, confirming the significance of these two losses in the overall model performance.

'

Notably, the inclusion of the two additional losses significantly improved model performance on high-dimensional time series datasets, such as Air Quality and EEG, indicating that the proposed auto-encoder architecture effectively learns meaningful low-dimensional sequential embeddings.

Conversely, the exclusive use of the reconstruction loss led to a notable decrease in model performance, suggesting that the l 2 samplewise distance might not be suitable for time series data. However, the additional regularization loss helped overcome this issue by ensuring that the sequential embedding space is confined to a predetermined noise space, such as the discretized Brownian motion. As a result, the regularization loss helped to mitigate the problems that arose when relying solely on the reconstruction loss.

D.2 Generated samples

In this section, we present random samples from the four benchmark datasets generated by PCFGAN, TimeGAN, RGAN, and COT-GAN. Although interpreting the sample plots of the generated time series poses a challenge, our observations reveal that PCF-GAN successfully generates time series that capture the temporal dependencies exhibited in the original time series across all datasets. Conversely, COT-GAN generates trajectories that are relatively smoother compared to the real time series samples, demonstrated on Stock and EEG datasets, by Figure 8 and Figure 10 respectively. Figure 10 shows that TimeGAN occasionally produces samples with higher oscillations than those found in the real samples.

1.0

0.5

0.5

1.0

0.5

0.0

0.5

0.0

U

20

20①

20①

200

20①

20①

20

200

0.4

0.2

0.0

0.0

0.6

0.2

WMM

0.0

0.0

0.6

0.2

0.0

10

10

10

10

10

10

0

10

0

10

0.75

0.25

0.00

PCF.

0.0

0.0

0.0

0.0

200

200

200

200

200

200

200

20

0.5

Real

0.0

0.2

W

A

0.0

0.2

AWV

0.0

0.2

AAN

0.0

0.2

0.4

0.0

0.2

10

10

10

10

10

10

10

0

10

D.3 Reconstructed samples

In this section, we present additional reconstructed time series samples generated by PCF-GAN and TimeGAN. Figure 11 illustrates that PCF-GAN consistently outperforms TimeGAN by producing higher-quality reconstructed samples across all datasets.

Real

PCF-GAN

TimeGAN

Real

PCF-GAN

TimeGAN

Real

PCF-GAN

TimeGAN

M

D.4 Test metrics on (auto-)correlation and marginal distribution

This subsection details the supplementary test metrics in terms of fitting the autocorrelation, crosscorrelation, and marginal distribution, as presented in Table 4. This table confirms that our proposed PCF-GAN consistently outperforms the benchmarking models across all datasets.

Peer reviews:
comments: SUMMARY:
This paper proposes  PCF-GAN, which aims to improve the effectiveness of the discriminator in differentiating the time series distributions by utilizing the path characteristic function (PCF) as a principled representation of the time series distribution within the discriminator. The authors also give the theoretical foundation of the PCF distance, proving its properties that ensure stable and feasible training of PCF-GAN. Extensive experiments show the effectiveness of the proposed method.

STRENGTHS:
The newly proposed metric in the discriminator for handling sequential data is interesting. Also, the author provides theoretical evidence for the analytical properties of the proposed loss metric.

WEAKNESSES:
- The presentation of this work is kind of confusing, making it difficult to fully comprehend the content. Specifically, in Figure 3, it is unclear what the generator loss represents if it is computed on the embedding vector. Additionally, it is unclear why the generator and regularization loss can enhance the discriminative power in Equation 10. Clarifying these help audience better understand. 
- The related work section consists of only one paragraph, which may overlook some relevant research in the field.
- The ablation study of every component is required to demonstrate the effectiveness individually. 
- Regarding the sequential data, I am wondering how does this method work well on more challenging domains like video data since videos are also sequential in nature.

QUESTIONS:
Please see Weaknesses

LIMITATIONS:
Please see Weaknesses
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2023-07-09

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper looks at generative models for times series data. It proposes a new GAN method, based on a novel discriminator. The path characteristic function (PFC) is used as a representation of the time series distribution. Using this, a distance between two distributions is defined (PCFD) as well as a way to approximate it (EPCFD). Afterwards, it is shown how EPCFD can be used as a discriminator in a GAN scenario (resulting in the PCF-GAN). To scale it to larger input dimensions, the authors follow previous literature and introduce a parameterised dimensionality reduction of the inputs and introduce different losses in an effort to ensure that it behaves as expected. Finally, they conduct a number of experiments, showing that PCF-GAN compares favourably to competitive time-series GANs.\n\nSTRENGTHS:\nThe method and its different components was well motivated and presented.\nThe paper presents a compelling theoretical motivation for the method, giving evidence on why it might be preferable to alternatives s.a. WGAN.\nThe experimental results are well analysed and presented.\n\nWEAKNESSES:\nThe reconstruction functionality is not motivated, nor explained. It can be unclear to the unfamiliar reader why one would want it, given that we can just generate images.\n\nRelated work - similar work is only lightly touched upon and the advantages/disadvantages are not outlined, making it harder to compare.\n\nThe loss function in Eq. 9 is not arrived at in a principled way and has two hyper-parameters which might be difficult to tune?\n\nIn the experiments, the dimensionality of the inputs, as well as the length of the sequences appears rather low. This suggests potential scalability issues?\n\nQUESTIONS:\nIn Eq. 9, do you expect training only on “ -L_recovery - L_regularisation”  to perform much worse?\n\nLIMITATIONS:\nThe method has computational limitations which limits the number of dimensions which it can be applied to. As mentioned before, an embedding functions is trained to reduce the dimensionality of the inputs, however, if the effective dimensionality is high, the method would fail.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-10"}

comments: SUMMARY:
The paper proposes an approach to improve time-series modeling in Generative Adversarial Network framework by using path characteristic function as the embedding of the time series sample. It explains the feasibility of PCF distance and how to integrates it as a distance measure between two time series data in order to help the discriminator learn better generative features. The goal and benefit of the proposed method is to empirically improve the generation quality of time series data.

STRENGTHS:
The paper explores using a novel loss function in Generative Adversarial Network framework. It computes the proposed empirical path characteristic function distance between generated and given sample to learn better time series generative features. The proposed method outperforms other generative baselines on the given datasets. The paper is well written ( and includes the codebase in appendix for reproducibility).

WEAKNESSES:
It would be great to have to following experiments in order to compare fairly with the existing baselines:

1. Comparison with existing baselines like COT-GAN on metrics like 
(a) the sum of the absolute difference of the correlation coefficients between channels avg over time 
(b) absolute difference between the correlation coefficients of real and generated samples.

2. Comparison on high dimensional dataset (like Sprites and human action sequences with FID, KID scores )

QUESTIONS:
N/A

LIMITATIONS:
As mentioned in the weakness section, having more ablations and experiments for comparison would help the reader better understand the efficacy of the proposed approach.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-10

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a path characteristic function GAN, named PCF-GAN, for learning to generate time-series data. More specifically, the authors mainly employ the rough path theory to build the PCF distance, such that the temporal cues can be encoded by unitary features, enabling the PCF to learn sequential data. The PCF distance is then proved to be complete and favorable for minimizing the difference between two stochastic processes. Even though, an encoder-decoder-based embedding is proposed to compare two random processes in the latent space of reduced dimensions, catering for the ease of training in practice. Experimental results have also verified the effectiveness of the proposed PCF-GAN.\n\nSTRENGTHS:\nThe established theory on PCF is novel and complete, by far my understanding. The authors also propose a practical way of minimizing the PCF distance, by using an encoder-like embedding that is able to reconstruct signals.\n\nWEAKNESSES:\n1. I am appreciated the theory behind the PCF. However, the usage of path theory needs to be clarified, when aiming at improving characteristic functions to represent stochastic process. The authors claim that CF-metric fails to capture temporal dependency of sequential data, which is true. However, why using path theory on characteristic functions can well address the temporal dependency, given the fact that there are other representations of the characteristic function of a stochastic integral?\n\n2. The authors claim that the embedding operation does not have any gradient constraints. However, when adding calculating the EPCFD after the embedding layers, the continuous and differential properties may not hold, without the restriction on the embedding layers. I am not sure whether the reconstruction regularization can compensate strictly. The authors are encouraged to elaborate more on this.\n\n3. Another weakness is regarding the experimental validations, which were performed based on low-dimensional time-series datasets. The PCF-GAN is shown to beat the COT-GAN on these datasets. However, COT-GAN is also able to generate short video clips, in which the scenarios are much complicated. Would the PCF-GAN scale well to the higher-dimensional generation tasks, such as short video generation? Also the comparing baselines are not the state-of-the-art. It is the fact that many recent methods have been reported to surpass COT-GAN and TimeGAN, for example, [1] and [2].\n\n[1] Seyfi, Ali, Jean-Francois Rajotte, and Raymond Ng. \"Generating multivariate time series with COmmon Source CoordInated GAN (COSCI-GAN).\" Advances in Neural Information Processing Systems 35 (2022): 32777-32788.\n[2] Jarrett, Daniel, Ioana Bica, and Mihaela van der Schaar. \"Time-series generation by contrastive imitation.\" Advances in Neural Information Processing Systems 34 (2021): 28968-28982.\n\nQUESTIONS:\n1. In Equation (4), why x_0=I_m. Wouldn't it be y_0=I_m for the differential equation?\n\n2. In Line 254, what does it mean by IGM? Should it be IPM?\n\n3. Why for the regularization loss, this paper employs a different \\theta'_M? How \\theta_M and \\theta'_M were implemented in this paper?\n\nLIMITATIONS:\nPlease see my weakness.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a new metric for the distributions on the path space via PCF and provides theoretical proofs for analytic properties of the proposed loss metric which benefits GAN training. It introduces a novel PCF-GAN to generate & reconstruct time series simultaneously. It compares the proposed method with prior work and shows improved performance.\n\nSTRENGTHS:\n- The paper is relatively well-written and the proposed approach is described clearly. \n\n- The authors provide theoretical grounding for their work. They prove PCF's characteristicity, boundedness, differentiability with respect to generator parameters, and its weak continuity which ensures the stability and feasibility of training the PCF-GAN. \n\n- Experimental results demonstrate improved performance compared to several prior works. The authors provide comparisons with TimeGAN, CotGAN and RGAN on RV, Stock, Air and EEG datasets using discriminative and predictive metrics.\n\nWEAKNESSES:\n- There are concerns regarding missing comparisons with related work. [A, B] also propose generative models for time series generation. [A] combines the adversarial training of GANs and the exact maximum likelihood training of CTFPs into a single framework. It designs an invertible generator, and adopt an autoencoder, on whose hidden space of the GAN performs the adversarial training. [B] uses Deep Euler representation and Wasserstein distances to propose three generative methods for times series. Two generative method EWGAN and EDGAN demonstrate an accuracy similar to state-of-the-art GAN generators and show better performance for capturing temporal dynamic metrics of the time series. The third method CEGEN is based on a loss metric computed on the conditional distributions of the time series. These papers are not mentioned and no comparisons are provided with them.   \n\n- There are metrics such as FID in prior work, e.g. [B], which are not used in the experiments. \n\n[A] GT-GAN: General Purpose Time Series Synthesis with Generative Adversarial Networks; Jeon et al.; \n \n[B] Conditional Loss and Deep Euler Scheme for Time Series Generation; Remlinger et al.;\n\nQUESTIONS:\nThe authors need to clearly distinguish their work with existing literature and provide comparisons with prior work (e.g. [A, B]).\n\nLIMITATIONS:\nThe authors have adequately discussed limitations of their work.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-27"}