## 3 Related Work
### Architectural Improvements  
Improvements in the GAN research literature that are relevant for designing features of the RSQGAN model include:  
1. Temporal convolutional networks (TCNs) [12; 46]-- an improvement in neural architecture for learning time series representations;
2. Conditional GANs (cGANs) [18]--an improvement to ensure deliberate generation from distinct class modes;
3. \"z-skipping' and 'z-clipping' [21]--improvements to class conditional synthetic quality, while giving user control to synthetic data size and the ability to explicitly trade off synthetic fidelity to real training data against synthetic variety, respectively. **Temporal convolutional networks** (TCNs) [46] are effective in modelling long-range dependencies in sequential data compared to other recurrent network architectures. Design features of the TCN include _dilated causal convolutions_, which allow the network to learn time-lag-dependent features at different dilation frequencies, but only caused from prior inputs in the sequence; _layered embeddings_, which allow representations to be learned at different levels of time scale resolution (i.e., larger receptive fields); and _residual connections_[47], which improve convergence for deep layered TCN networks. See Figure 1.  
The first known use of TCNs to generate synthetic FTS data was the Quant GAN architecture [12] as described in Section 4 below and in more detail in Appendices A.2 and A.3. **Conditional GANs**[18] aim to explicitly address the issue of mode collapse by directly inducing the generation of class-conditional data. Rather than unconditional generations of \(G(z;\theta_{g})\) from adversarial training against an unconditional discriminator/critic network \(D(x;\theta_{d})\), both generator and discriminator/critic networks can be conditioned against classes \(\mathcal{C}_{j}\) to learn \(G(z|\mathcal{C}_{j};\theta_{g})\) and \(D(x|\mathcal{C}_{j};\theta_{d})\). The conditional vanilla GAN minimax discriminator loss can thus be expressed as:  
\[\min_{G}\max_{D}\mathbb{E}_{x\sim\mathbb{P}_{r}(x)}[\log D(x|\mathcal{C}_{j}) ]+\mathbb{E}_{z\sim\mathbb{P}_{Z}(z)}[\log(1-D(G(z|\mathcal{C}_{j})))]. \tag{7}\]  
The training procedure shown below accepts conditional class \(\mathcal{C}_{j}\) as an additional input tensor in training the cGAN network. Another advantage of this approach is that the user can specify the form of the conditions \(\mathcal{C}_{j}\) in semi-supervised data generation or apply representations of \(\mathcal{C}_{j}\) from an unsupervised learning process. **'skip-z layers' and 'z-clipping'** are two methods applicable to cGANs as described in the single paper by [PERSON] et al. [21]. It explored the effect of larger minibatch sizes in training larger-image cGAN models. Rather than concatenating an encoding of the conditional vector \(C_{j}\) to a noise prior \(z\) at the input layer of a network (as seen in Figure 2 above), the noise and class encoding ('skip-z layers') are instead introduced at multiple hidden layers. In this way, the network can learn class-conditional feature representations at different receptive field sizes. In addition, this innovation allows for an arbitrarily large noise prior \(z\) to be used for the generator input layer, as this no longer requires the topology of the joint noise and class embedding layer to be defined in advance. Further, [PERSON] et al. [21] test a variety of noise prior distributions other than the standard multinomial uniform and Gaussian distributions. The authors discover that sampling from a truncated normal distribution, i.e., re-sampling the noise prior \(z\) if \(|z|\geq z^{clip}\) for some clipping hyperparameter \(z^{clip}\geq 0\), would allow users control, indirectly trading off improved fidelity to the density of the training data \(P_{\mathcal{X}|C_{j}}\) at the cost of intra-mode variety, which could be measured by the lower-class conditional entropy of \(\hat{X}|C_{j}\).