SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\int_ {\mathcal {L} \left(\mathbb {R} ^ {d}; \mathrm {u} (m)\right)} \left\| \int_ {\mathcal {K}} \mathcal {U} _ {M} \mathrm {d} \mathbb {P} _ {\mathbf {X} _ {n}} - \int_ {\mathcal {K}} \mathcal {U} _ {M} \mathrm {d} \mathbb {P} _ {\mathbf {X}} \right\| _ {\mathrm {H S}} ^ {2} \mathrm {d} \mathbb {P} _ {\mathcal {M}} (M) = 0
$$
for any  $m \in \mathbb{N}$  and  $\mathcal{M} \in \mathcal{P}\left(\mathcal{L}\left(\mathbb{R}^d; \mathfrak{u}(m)\right)\right)$ , in particular for those with full support. In view of the universality Theorem A.8 proved above, for any fixed  $\epsilon > 0$  and any continuous function  $f \in C^0(\mathcal{K})$ , by approximating  $f$  with sum of finitely many  $L_i \circ \mathcal{U}_{M_i}$  (the notations are as in Theorem A.8), one infers that for  $n$  and  $m_\star$  sufficiently large, it holds that
$$
\int_ {\mathcal {L} \left(\mathbb {R} ^ {d}; \mathrm {u} \left(m _ {\star}\right)\right)} \left| \int_ {\mathcal {K}} f \mathrm {d} \mathbb {P} _ {\mathbf {X} _ {n}} - \int_ {\mathcal {K}} f \mathrm {d} \mathbb {P} _ {\mathbf {X}} \right| ^ {2} \mathrm {d} \mathbb {P} _ {\mathcal {M}} (M) <   \epsilon .
$$
By considering those measures with  $\operatorname{spt}(M) = \mathcal{L}\left(\mathbb{R}^d;\mathfrak{u}(m_\star)\right)$ , we deduce that
$$
\lim  _ {n \rightarrow \infty} \left| \int_ {\mathcal {K}} f \mathrm {d} \mathbb {P} _ {\mathbf {X} _ {n}} - \int_ {\mathcal {K}} f \mathrm {d} \mathbb {P} _ {\mathbf {X}} \right| = 0 \quad \text {f o r a n y} f \in C ^ {0} (\mathcal {K}).
$$
This is tantamount to the distributional convergence.
Proof. We first prove the 'if' direction of the statement. By the Portmanteau theorem [21], convergence in distribution  $X_{n} \xrightarrow{\mathrm{d}} X$  implies, for any bounded continuous map  $f$ , we have  $\mathbb{E}_{x \sim \mathbb{P}_n}[f(x)] \to \mathbb{E}_{x \sim \mathbb{P}}[f(x)]$ . Therefore, for any  $M \in \mathcal{L}(\mathbb{R}^d, \mathfrak{u}(m))$ ,  $\mathbb{E}_{x \sim \mathbb{P}_n}[\mathcal{U}_M(x)] \to \mathbb{E}_{x \sim \mathbb{P}}[\mathcal{U}_M(x)]$ , which implies  $||\Phi_{X_n}(M) - \Phi_X(M)||_{HS}^2 \to 0$  as  $n \to \infty$ . Hence, it follows that, as  $n \to \infty$ ,
$$
P C F D \left(X _ {n}, X\right) := \mathbb {E} _ {M \sim \mathbb {P} _ {\mathcal {M}}} | | \Phi_ {X _ {n}} (M) - \Phi_ {X} (M) | | _ {H S} ^ {2} \rightarrow 0,
$$
which completes the proof of 'if' direction.
Now we proceed with the 'only if' direction. By the universality of the unitary path development from Theorem A.8, for any continuous function  $f: \mathcal{K} \to \mathbb{C}$  and  $\epsilon > 0$ , there exist  $M_1, \dots, M_N \in \mathcal{L}(\mathbb{R}^d, \mathfrak{u}(m))$  and  $L_1, \ldots, L_N \in \mathcal{L}(\mathcal{U}(m); \mathbb{C})$  such that
$$
\left| \mathbb {E} _ {x \sim \mathbb {P}} [ f (x) ] - \sum_ {i = 1} ^ {N} L _ {i} \circ \mathbb {E} _ {x \sim \mathbb {P}} [ \mathcal {U} _ {M _ {i}} (x) ] \right| <   \epsilon . \tag {14}
$$
or equivalently  $\left|\mathbb{E}_{x\sim \mathbb{P}}[f(x)] - \sum_{i = 1}^{N}L_{i}\circ \Phi_{X}(M_{i})\right| < \epsilon$ . For simplicity, we denote  $\mathbb{E}_{x\sim \mathbb{P}_n}$  and  $\mathbb{E}_{x\sim \mathbb{P}}$  as  $\mathbb{E}_n$  and  $\mathbb{E}$  respectively. Therefore,
$$
\begin{array}{l} \left| \mathbb {E} _ {n} [ f (x) ] - \mathbb {E} [ f (x) ] \right| \leq \left| \mathbb {E} _ {n} [ f (x) ] - \sum_ {i = 1} ^ {N} L _ {i} \circ \Phi_ {X _ {n}} \left(M _ {i}\right) \right| + \left| \mathbb {E} [ f (x) ] - \sum_ {i = 1} ^ {N} L _ {i} \circ \Phi_ {X} \left(M _ {i}\right) \right| (15) \\ + \left| \sum_ {i = 1} ^ {N} L _ {i} \circ \left(\Phi_ {X _ {n}} \left(M _ {i}\right) - \Phi_ {X} \left(M _ {i}\right)\right) \right| (16) \\ \leq 2 \epsilon + \sum_ {i = 1} ^ {N} \left| L _ {i} \right| _ {o p} \| \Phi_ {X _ {n}} (M _ {i}) - \Phi_ {X} (M _ {i}) \| _ {H S} ^ {2} (17) \\ \end{array}
$$
where  $|L|_{op} \coloneqq \sup_{x \in \mathcal{U}(m) \setminus 0} \frac{|L(x)|}{||x||_{HS}^2}$  the operator norm. Since  $PCFD(X_n, X) \coloneqq \mathbb{E}_{M \sim \mathbb{P}_M} \| \Phi_{X_n}(M) - \Phi_X(M) \|_{HS}^2 \to 0$  as  $n \to \infty$  and  $\epsilon$  is arbitrary,  $\mathbb{E}_{x \sim \mathbb{P}_n}[f(x)] \to \mathbb{E}_{x \sim \mathbb{P}}[f(x)]$  for any continuous bounded function  $f: \mathcal{K} \to \mathbb{C}$ , which implies  $X_n \stackrel{d}{\to} X$  by the Portmanteau theorem [21].
B.3 Relation with MMD
We now discuss linkages between PCFD and MMD (maximum mean discrepancy) defined over  $\mathcal{P}(\mathcal{X})$ , the space of Borel probability measures (equivalently, probability distributions) on  $\mathcal{X}$ .
Definition B.9. Given a kernel function  $\kappa : \mathcal{X} \times \mathcal{X} \to \mathbb{R}$ , the MMD associated to  $\kappa$  is the function  $\mathrm{MMD}_{\kappa} : \mathcal{P}(\mathcal{X}) \times \mathcal{P}(\mathcal{X}) \to \mathbb{R}^{+}$  given as follows: for independent random variables  $\mathbf{X}$ ,  $\mathbf{Y}$  on  $\mathcal{X}$ , set
$$
\mathrm {M M D} _ {\kappa} ^ {2} (\mathbb {P} _ {\mathbf {X}}, \mathbb {P} _ {\mathbf {Y}}) = \mathbb {E} _ {\mathbf {X}, \mathbf {X} ^ {\prime} \stackrel {\mathrm {i d}} {\sim} \mathbb {P} _ {\mathbf {X}}} [ \kappa (\mathbf {X}, \mathbf {X} ^ {\prime}) ] + \mathbb {E} _ {\mathbf {Y}, \mathbf {Y} ^ {\prime} \stackrel {\mathrm {i d}} {\sim} \mathbb {P} _ {\mathbf {Y}}} [ \kappa (\mathbf {Y}, \mathbf {Y} ^ {\prime}) ] - 2 \mathbb {E} _ {\mathbf {X} \sim \mathbb {P} _ {\mathbf {X}}, \mathbf {Y} \sim \mathbb {P} _ {\mathbf {Y}}} [ \kappa (\mathbf {X}, \mathbf {Y}) ].
$$
The PCFD can be interpreted as an MMD on measures of the path space with a specific kernel. Compare with [40] for the case of  $\mathbb{R}^d$ .
Proposition B.10 (PCFD as MMD). Given  $\mathcal{M} \in \mathcal{P}\left(\mathfrak{u}(m)^d\right)$  and  $\mathcal{X}$ -valued random variables  $\mathbf{X}$  and  $\mathbf{Y}$  with induced distributions  $\mathbb{P}_{\mathbf{X}}$  and  $\mathbb{P}_{\mathbf{Y}}$ , resp. Then  $\mathrm{PCFD}_{\mathcal{M}}(\mathbf{X},\mathbf{Y}) = \mathrm{MMD}_{\kappa}(\mathbb{P}_{\mathbf{X}},\mathbb{P}_{\mathbf{Y}})$  with kernel  $\kappa (\mathbf{x},\mathbf{y}) \coloneqq \mathbb{E}_{M\sim \mathbb{P}_{\mathcal{M}}}\left[\left\| \mathcal{U}_M(\mathbf{x}) - \mathcal{U}_M(\mathbf{y})\right\|_{\mathrm{HS}}\right] = \mathbb{E}_{M\sim \mathbb{P}_{\mathcal{M}}}\left[\operatorname {tr}\left(\mathcal{U}_M\left(\mathbf{x}\star \overleftarrow{\mathbf{y}}\right)\right)\right]$ .
Throughout,  $\star$  designates concatenation of paths and  $\overleftarrow{\mathbf{y}}$  is the path obtained by running  $\mathbf{y}$  backwards. The operation  $\mathbf{x} \star \overleftarrow{\mathbf{y}}$  on the path space is analogous to  $\mathbf{x} - \mathbf{y}$  on  $\mathbb{R}^d$ . If  $\mathbf{y} = \mathbf{x}$ , then  $\mathbf{x} \star \overleftarrow{\mathbf{y}}$  is the null path. See the Appendix for proofs and further discussions.
Remark B.11 (Computational cost complexity). By Proposition B.10, PCFD is an MMD. However, to compute EPCFD, we may directly calculate the expected distance between the PCFs, without going over the kernel calculations in the MMD approach. Our method is significantly more efficient, especially for large datasets. The computational complexity of EPCFD is linear in sample size, whereas the MMD approach is quadratic.
Proof. By definition of PCFD, we have
$$
\begin{array}{l} \operatorname {P C F D} _ {\mathcal {M}} ^ {2} (\mu , \nu) = \mathbb {E} _ {M \sim \mathbb {P} _ {\mathcal {M}}} \left[ \| \Phi_ {\mathbf {X}} (M) - \Phi_ {\mathbf {Y}} (M) \| _ {\mathrm {H S}} ^ {2} \right] \\ = \mathbb {E} _ {M \sim \mathbb {P} _ {\mathcal {M}}} \left[ \| \Phi_ {\mathbf {X}} (M) \| _ {\mathrm {H S}} ^ {2} + \| \Phi_ {\mathbf {Y}} (M) \| _ {\mathrm {H S}} ^ {2} - 2 \langle \Phi_ {\mathbf {X}} (M), \Phi_ {\mathbf {Y}} (M) \rangle_ {\mathrm {H S}} \right] \\ \end{array}
$$
where  $\Phi_{\mathbf{X}}(M) = \mathbb{E}_{\mathbf{X}\sim \mu}[\mathcal{U}_M(\mathbf{X})]$  and  $\Phi_{\mathbf{Y}}(M) = \mathbb{E}_{\mathbf{Y}\sim \mu}[\mathcal{U}_M(\mathbf{Y})]$ , respectively. Using Fubini's theorem and observing that  $\langle \Phi_{\mathbf{X}}(M),\Phi_{\mathbf{Y}}(M)\rangle_{\mathrm{HS}}\in L^2 (\mathbb{P}_{\mathcal{M}})$  (as  $\Phi_{\mathbf{X}}(M)$  and  $\Phi_{\mathbf{Y}}(M)$  are  $U(m)$ -valued, they indeed lie in  $L^{\infty}(\mathbb{C}^{m\times m};\mathbb{P}_{\mathcal{M}})$  as  $U(m)$  is a compact Lie group under the Hilbert-Schmidt metric), we deduce that
$$
\mathbb {E} _ {M \sim \mathbb {P} _ {\mathcal {M}}} [ \langle \Phi_ {\mathbf {X}} (M), \Phi_ {\mathbf {Y}} (M) \rangle_ {\mathrm {H S}} ] = \mathbb {E} _ {\mathbf {X} \sim \mu} [ \mathbb {E} _ {\mathbf {Y} \sim \nu} [ \mathbb {E} _ {M \sim \mathbb {P} _ {\mathcal {M}}} [ \langle \mathcal {U} _ {M} (\mathbf {X}), \mathcal {U} _ {M} (\mathbf {Y}) \rangle_ {\mathrm {H S}} ] ] ].
$$
The first equality then follows from the identification  $\kappa (\mathbf{x},\mathbf{y}) = \mathbb{E}_{M\sim \mathbb{P}_M}[\langle \mathcal{U}_M(\mathbf{x}),\mathcal{U}_M(\mathbf{y})\rangle_{\mathrm{HS}}]$  and the definition of  $\mathrm{MMD}_{\kappa}$ .
On the other hand, by Lemma A.5 and the definition of the Hilbert-Schmidt inner product on  $U(m)$ , one may rewrite the kernel function as follows:
$$
\begin{array}{l} \kappa (\mathbf {x}, \mathbf {y}) = \mathbb {E} _ {M \sim \mathbb {P} _ {\mathcal {M}}} [ \langle \mathcal {U} _ {M} (\mathbf {x}), \mathcal {U} _ {M} (\mathbf {y}) \rangle_ {\mathrm {H S}} ] \\ = \mathbb {E} _ {M \sim \mathbb {P} _ {\mathcal {M}}} \left[ \mathrm {t r} (\mathcal {U} _ {M} (\mathbf {x}) \cdot \mathcal {U} _ {M} ^ {- 1} (\mathbf {y})) \right] = \mathbb {E} _ {M \sim \mathbb {P} _ {\mathcal {M}}} \left[ \mathrm {t r} \left(\mathcal {U} _ {M} \left(\mathbf {x} \star \mathbf {\check {\Sigma}} _ {\mathbf {y}}\right)\right) \right], \\ \end{array}
$$
where  $\star$  denotes the concatenation of paths. The second equality now follows.
B.4 Empirical PCFD
B.4.1 Initialisation of  $\mathcal{M}$
A linear map  $M \in \mathcal{L}\left(\mathbb{R}^d, \mathfrak{u}(m)\right)$  can be canonically represented by  $d$  independent anti-Hermitian matrices  $M_1, \ldots, M_d \subset \mathfrak{u}(m) \in \mathbb{C}^{m \times m}$ . To sample empirical distribution of  $\mathcal{M} \in \mathcal{P}\left[\mathcal{L}\left(\mathbb{R}^d, \mathfrak{u}(m)\right)\right]$  from  $\mathbb{P}_{\mathcal{M}}$ , we propose a sampling scheme over  $\mathfrak{u}(m)$ . This can also be used as an effective initialisation of model parameters  $\theta_M \in \mathfrak{u}(m)^{d \times k}$  for the empirical measure of  $\mathcal{M}$ .
In practice, when working with the Lie algebra  $\mathfrak{u}(m)$ , i.e., the vector space of  $m\times m$  complex-valued matrices that are anti-Hermitian ( $A^{*} + A = 0$ , where  $A^{*}$  is the transpose conjugate of  $A$ ), we view each anti-Hermitian matrix as an  $2m\times 2m$  real matrix via the isomorphism of  $\mathbb{R}$ -vector spaces  $\mathbb{R}^{2m\times 2m}\cong \mathbb{C}^{m\times m}$ .
Under the above identification, we have the decomposition
$$
\mathfrak {u} (m) \cong \mathfrak {o} (m) \oplus \sqrt {- 1} \left(\operatorname {S y m} _ {m \times m} / \mathfrak {z} (m)\right) \oplus \sqrt {- 1} \mathfrak {z} (m), \tag {18}
$$
where  $\mathfrak{o}(m)$  is the Lie algebra of anti-symmetric  $m\times m$  real matrices,  $\mathrm{Sym}_{m\times m}$  is the space of  $m\times m$  real symmetric matrices,  $\mathfrak{z}(m)$  consists of  $m\times m$  real diagonal matrices and  $\mathrm{Sym}_{m\times m} / \mathfrak{z}(m)$  denotes the quotient space of real symmetric matrices by the real diagonal matrices.
The sampling procedure of  $\mathbb{P}_{\mathcal{M}}$ , is given as follows. First, we simulate  $\mathbb{R}^{m\times m}$  valued and i.i.d random variables  $A$  and  $B$ , whose elements are i.i.d and satisfy the pre-specified distribution in  $\mathcal{P}(\mathbb{R})$ . We have the decomposition  $B = D\oplus E$ , where  $\mathrm{D}$  and  $\mathrm{E}$  are a diagonal random matrix and a off-diagonal random matrix respectively. Then we construct the anti-symmetric matrix  $R = \frac{1}{\sqrt{2}} (A^T -A)$  and matrix in the quotient space  $\operatorname{Sym}_{m\times m} / \mathfrak{z}(m)$ ,  $C = \frac{1}{\sqrt{2}} (E^T +E)$ , and diagonal matrix  $D$ . Correspondingly, we simulate  $\mathfrak{u}(m)$ -valued random variables by virtue of Eq. (18). As the empirical measure of the  $\mathcal{M}$  can be fully characterised by the model parameters  $\theta_{M}\in \mathfrak{u}(m)^{d\times k}$ , we sample  $d\times k$  i.i.d. samples which take values in  $u(m)$ .
B.4.2 Hypothesis test
In the following, we illustrate the efficacy of the proposed trainable EPCFD metric in the context of the hypothesis test on stochastic processes.
Example B.12 (Hypothesis testing on fractional Brownian motion). Consider the 3-dimensional Brownian motion  $\mathbf{B} \coloneqq (B_t)_{t \in [0, T]}$  and the fraction Brownian motion  $\mathbf{B}^h \coloneqq (B_t^h)_{t \in [0, T]}$  with the Hurst parameter  $h$ . We simulated 5000 sample paths for both  $\mathbf{B}$  and  $\mathbf{B}^h$  with 50 discretized time steps. We apply the proposed optimized EPCFD metric to the two-sample testing problem: the null hypothesis  $H_0: \mathbf{B} \stackrel{d}{=} \mathbf{B}^h$  against the alternative  $H_1: \mathbf{B} \neq \mathbf{B}^h$ . We compare the optimized EPCFD metric with EPCFD metric with the prespecified distribution (PCF) and the characteristic function distance (CF) on the flattened time series [25]. The optimized PCFs are trained on a separate set of 5000 training samples to maximise the PCFD. The details of training can be found at Appendix C.2.
We conduct the permutation test to compute the power of a test (i.e. the probability of correct rejection of the null  $H_0$ ) and Type I error (i.e. the probability of false acceptances of the null  $H_0$ ) for varying  $h \in \{0.2 + 0.1 \cdot i\}_{i=0}^6$ . Note that when  $h = 0.5$ ,  $\mathbf{B}$  and  $\mathbf{B}^h$  have the same distribution and hence are indistinguishable. Therefore, the better the test metric is, the test power should be closer to 0 when  $h$  is close to 0.5, whereas it should be closer to 1 when  $h$  is away from 0.5. We refer to [22] for more in-depth information on hypothesis testing and permutation test statistics.
The plot of the test power and Type 1 error in Figure 6 shows that CF fails in the two sample tests, whilst both EPCFD and optimised EPCFD can distinguish the samples from the stochastic process when  $h \neq 0.5$ . It indicates that the EPCFD captures the distribution of time series much more effectively than the conventional CF metric. Moreover, optimization of EPCFD increases the test power while decreasing the type1 error, particularly when  $h$  is closer to 0.5.
C.1 Experimental detail on PCF-GAN
C.1.1 General notes
Codes. The code for reproducing all experiments can be found in https://github.com/DeepIntoStreams/PCF-GAN.
Software. We conducted all experiments using PyTorch 1.13.1 [34] and performed hyperparameter tuning with Wandb [5]. To ensure reproducibility, we implemented benchmark models based on open-source code from [45, 43, 12]. We used the Ksig library [42] to calculate the Sig-MMD metrics. The codes in [25] were used to compute characteristics function distance in Example B.12.
Computing infrastructure. The experiments were performed on a computational system running Ubuntu 22.04.2 LTS, comprising three Quadro RTX 8000 and two RTX A6000 GPUs. Each experiment was run independently on a single GPU, with the training phase taking between 6 hours to 3 days, depending on the dataset and models used.
Figure 6: Plots of the test power (Left) and the Type-I error (Right) against the Hurst parameter  $h \in [0.2, 0.8]$  on the two sample tests for the Brownian motion  $\mathbf{B}$  against Fractional Brownian motions  $(\mathbf{B}^h)$  by using three metrics, i.e., PCFD, optimized EPCFD and CFD.
Architectures. To ensure a fair comparison, we employed identical network architectures, with two layers of LSTMs having 32 hidden units, for both the generator and discriminator across all models. For the generator, the output of the LSTM (full sequence) was passed through a Tanh activation function and a linear output layer. All generative models take a multi-dimensional discretized Brownian motion as the noise distribution, scaling it to ensure values were controlled within the range  $[-1, 1]$ . The dimension and scaling factor varied based on the dataset and were specified in the individual sections as below.
The PCF-GAN uses the development layers on the unitary matrix [26] to calculate the PCFD distance. For all experiments, we fixed the unitary matrix size and coefficient  $\lambda_{2}$  for the regularization loss to 10 and 1, respectively. The number of unitary linear maps and the coefficient  $\lambda_{1}$  of the recovery loss were determined via hyper-parameter tuning, which varied depending on the dataset (see individual section for details).
Regarding TimeGAN, the following approach described in [45] and employed embedding, supervisor, and recovery modules. Each of these modules had two layers of LSTMs with 32 hidden units. For COT-GAN, we used two separate modules for discriminators, each with two layers of LSTMs with 32 hidden units. Based on the recommendation from COT-GAN [43] and informal hyperparameter tuning, we set  $\lambda = 10$  and  $\epsilon = 1$  for all experiments.
Optimisation & training. We used the ADAM optimizer for all experiments [20], with a learning rate of 0.001 for both generators and discriminators. The learning rate for the unitary development network is 0.005. The initial decay rates in the ADAM optimizer are set  $\beta_{1} = 0$ ,  $\beta_{2} = 0.9$ . The discriminator was trained for two iterations per iteration of the generator's training. For TimeGAN, we followed the training scheme for each module as suggested in the original paper. The batch size was 64 for all experiments. These hyperparameters do not substantially affect the results.
To improve the training stability of GAN, we employed three techniques. Firstly, we applied a constant exponential decay rate of 0.97 to the learning rate for every 500 generator training iterations. Secondly, we clipped the norm of gradients in both generator and discriminator to 10. Thirdly, we used the Cesaro mean of the generator weights after certain iterations to improve the performance of the final model, as suggested by [44]. In all cases, we selected the number of training iterations such that all methods could produce stable generative samples. The optimal number of training iterations and weight averaging scheme varied for each dataset. More details can be found in the respective sections.
Test metrics. Discriminative score. The network architecture of the post-hoc classifier consists of two layers of LSTMs with 16 hidden units. The dataset was split into equal proportions of real and generated time series with labels 0 and 1, with an  $80\% / 20\%$  train/test split for training and evaluation. The discriminative model was trained for 30 epochs using Adam with a learning rate of 0.001 and a batch size of 64. The best classification error on the test set was reported.
Predictive score. The network architecture of the post-hoc sequence-to-sequence regressor consists of two layers of LSTMs with 16 hidden units. The model was trained on the generated time series
and evaluated on the real time series, using the first  $80\%$  of the time series to predict the last  $20\%$ . The predictive model was trained for 50 epochs using Adam with a learning rate of 0.001 and a batch size of 64. The best mean squared error on the test set was reported.
Sig-MMD. We directly computed the Sig-MMD by taking inputs of the real time series samples and generated time series samples. We used the radial basis function kernel applying to the truncated signature feature up to depth 5.
C.2 Time dependent Ornstein-Uhlenbeck process
On this dataset, we experimented with the basic version of PCF-GAN, which only utilized the EPCFD as the discriminator without the autoencoder structure. The batch size is 256. The model are trained with 20000 generator training iterations and weight averaging on the generator was performed over the final 4000 generator training iterations. We used the 2-dimensional discretized Brownian motion as the noise distribution.
C.2.1 Rough volatility model
We followed [31] considering a rough stochastic volatility model for an asset price process  $(S_{t})_{t\in [0,1]}$  which satisfies the below stochastic differential equation,
$$
d S _ {t} = \sqrt {V _ {t}} S _ {t} d Z _ {t}, \tag {19}
$$
$$
V _ {t} := \xi (t) \exp \left(\eta B _ {t} ^ {H} - \frac {1}{2} \eta^ {2} t ^ {2 H}\right), \tag {20}
$$
where  $\xi(t)$  denotes the forward variance and  $B_t^H$  denotes the fractional Brownian motion (fBM) given by
$$
B _ {t} ^ {H} := \int_ {0} ^ {t} K (t - s) d B _ {s}, \quad K (r) := \sqrt {2 H} r ^ {H - 0. 5}
$$
where  $(Z_{t})_{t\in [0,1]}$ ,  $(B_{t})_{t\in [0,1]}$  are (possibly correlated) Brownian motions. In our experiments, the synthetic dataset is sampled from Equation (19) with  $t\in [0,1]$ ,  $H = 0.25$ ,  $\xi(t)\sim \mathcal{N}(0.1,0.01)$ ,  $\eta = 0.5$  and initial condition  $\log(S_0)\sim \mathcal{N}(0,0.05)$ . Each sample path is sampled uniformly from  $[0,1]$  with the time discretization  $\delta t = 0.005$ , which consists of 200 time steps. We train the generators to learn the joint distribution of the log price and log volatility.
All methods are trained with 30000 generator training iterations and weight averaging on the generator was performed over the final 5000 generator training iterations. The input noise vectors have 5 dimension and 200 time steps.
For PCF-GAN, the coefficient  $\lambda_{1}$  for the recovery loss was 50, and the number of unitary linear maps was 6.
C.2.2 Stocks
We selected 10 large market cap stocks, which are Google, Apple, Amazon, Tesla, Meta, Microsoft, Nvidia, JP Morgan, Visa and P&G, from 2013 to 2021. The dataset consists of 5 features, including daily open, close, high, low prices and volume, available on https://finance.yahoo.com/lookup. We truncated the long stock time series into 20 days. The data were normalized with standard Min-Max normalisation on each feature channel. The Stock dataset used in our study is similar to the one employed in [25] but with a broader range of assets. Unlike the previous approach, we avoided sampling the time series using rolling windows with a stride of 1 to mitigate the presence of strong dependencies between samples.
All methods are trained with 30000 generator training iterations and weight averaging on the generator was performed over the final 5000 generator training iterations. The input noise vectors have 5 feature dimensions and 20 time steps.
For PCF-GAN, the coefficient  $\lambda_{1}$  for the recovery loss was 400, and the number of unitary linear maps was 6.
C.2.3 Beijing Air Quality
We used a dataset of the air quality in Beijing from the UCI repository [47] and available on https://archive.ics.uci.edu/ml/datasets/Beijing+Multi-Site+Air-Quality+Data. Each sample is a 10-dimensional time series of the SO2, NO2, CO, O3, PM2.5, PM10 concentrations, temperature, pressure, dew point temperature and wind speed. Each time series is recorded hourly over the course of a day. The data were normalized with standard Min-Max normalisation on each feature channel.
All methods are trained with 20000 generator training iterations and weight averaging on the generator was performed over the final 4000 generator training iterations. The input noise vectors have 5 dimensions and 24 time steps.
C.2.4 EEG
We obtained the EEG eye state dataset from https://archive.ics.uci.edu/ml/datasets/ EEG+Eye+State. The data is from one continuous EEG measurement on 14 variables with 14980 time steps. We truncated the long time series into smaller ones with 20 time steps. The data are subtracted by channel-wise mean, divided by three times the channel-wise standard deviation, and then passed through a tanh nonlinearity.
All methods are trained with 30000 generator training iterations and weight averaging on the generator was performed over the final 5000 generator training iterations. The input noise vectors have the 8 dimensional and 20 time steps.