## 2 Background  
### Generative Adversarial Networks (GANs)  
Within the family of generative modelling approaches, generative adversarial networks (GANs) [6] are referred to as an example of a likelihood-free inference approach [22] to generative modelling. The learned density of the _real_ DGP \(X\sim\mathbb{P}_{\mathrm{r}}(x)\) is implicitly known if samples can be generated that are consistent with empirical densities. In contrast, explicit approaches aim to learn parameters of latent variable densities. However, due to their intractability, approximations are needed, which can negatively impact performance [23]. Since the introduction of GANs in 2014 as a novel approach to creating synthetic images, rapid progress has been made in the image domain [21; 24; 25] and has proliferated across data domains such as audio [26], text [27; 28], medical data [19; 29], and, more recently, videos [30; 31]. The idea behind the original 'vanilla' GAN algorithm [6] is the adversarial training of a generator network \(G:(z;\theta_{g})\mapsto\tilde{x}\) against a discriminator network \(D:(x;\theta_{d})\mapsto[0,1]\) that aims to correctly discriminate between generated samples \(\tilde{x}\) and real instances \(x\). \(z\) is drawn from a random noise prior distribution \(p_{Z}(z)\), e.g., multinomial standard Gaussian \(Z\sim\mathcal{N}(\mu=0,\Sigma=I)\). Adversarial training is viewed as a minimax game between \(G\) and \(D\), intuitively, when the joint minimax loss function in Equation (1) converges:  
\[\min_{G}\max_{D}\mathbb{E}_{x\sim\mathbb{P}_{r}(x)}[\log D(x)]+\mathbb{E}_{z\sim p _{Z}(z)}[\log(1-D(G(z)))]. \tag{1}\]  
It was shown by [PERSON] et al. [6] that a unique solution at \(\mathbb{P}_{r}=\mathbb{P}_{g}\) exists when \(D^{*}(x)=\frac{1}{2}\), as optimising Equation (1) under the optimal generator \(D^{*}(x)\) is equivalent to minimising the cost function:  
\[-\log 4+2\cdot JSD(\mathbb{P}_{r}\|\mathbb{P}_{g}) \tag{2}\]  
where \(JSD(\mathbb{P}_{r}\|\mathbb{P}_{g})\) is the Jensen-Shannon divergence (\(JSD\) ) between \(\mathbb{P}_{r}\) and \(\mathbb{P}_{g}\):  
\[JSD(\mathbb{P}_{r}\|\mathbb{P}_{g})=KL(\mathbb{P}_{r}\|\frac{\mathbb{P}_{r}+ \mathbb{P}_{g}}{2})+KL(\mathbb{P}_{g}\|\frac{\mathbb{P}_{r}+\mathbb{P}_{g}}{2}) \tag{3}\]  
and \(KL(\mathbb{P}_{X}\|\mathbb{P}_{Y})\) is the Kullback-Leibler (\(KL\)) divergence between \(\mathbb{P}_{X}\) and \(\mathbb{P}_{Y}\) given by  
\[KL(\mathbb{P}_{X}\|\mathbb{P}_{Y})=\int_{\mathcal{X}}P_{X}(x)\log\frac{P_{X}( x)}{\mathbb{P}_{Y}(x)}dx \tag{4}\]  
over the joint support for \(P_{X,Y}\) denoted by \(\mathcal{X}\). However, the training of vanilla GANs has been problematic in practice. Issues include: (1) a lack of variation in simulated outputs ('mode collapse'), (2) non-convergence during training, and (3) training losses being a poor metric to evaluate GAN output quality. 1. **Mode collapse. ** The true data distribution \(\mathbb{P}_{r}(x)\) is likely to be high-dimensional and multi-modal. Mode collapse can occur if, during training, the discriminator network overfits in its ability to identify generated fakes. The generator network then responds by restricting fake samples to modes that are less likely to be classified as fakes. The progressive overfitting of the discriminator within this subset causes the density of generated samples to concentrate into a shrinking support space. Despite this, observed generator and discriminator losses continue to shrink but generated samples become invariant. Conversely, another cause of mode collapse could be from vanishing discriminator gradients, which result in stalled learning for the generator network. Methods for dealing with discriminator overfitting include weight regularisation [32], regularisation by discriminator learning from stochastically corrupted inputs [33], using alternative loss functions such as Wasserstein loss (WGAN) [34] and applying gradient penalties in the discriminator learning process (WGAN-GP) [35]. 2. **Training non-convergence.