## 2 Materials and Methods
### Generative Adversarial Networks
\begin{table}
\begin{tabular}{c c} \hline \hline
**Hyperparameter** & **Tested Settings** \\ \hline Number of Layers & – \\ Node Density & – \\ Activation Functions & ReLU, Leaky ReLU, Tanh, Swish, ELU, Sigmoid, Linear \\ Kernel Initializer & None, Uniform, Normal and their Random, Truncated or Glorot variants. \\ Dropout & None, Present with thresholds between 0.01 and 0.9 \\ Weight Regularizer & None, I2 with thresholds between 0.01 and 0.0001 \\ Weight Constraint & UnitNorm, MaxNorm, MinMaxNorm \\ Batch Normalization & Present, Absent \\ Training Epochs & Between 100 and 2000 \\ Batch Size & 4, 8, 16, 32 \\ Optimizers & Adam, RMSprop, Stochastic Gradient Descent. Adagrad \\ Learning Rate & Between 0.1 and 0.00001 \\ Decay & Between 0.9 and 0.0001 \\ Momentum & Between 0.99 and 0.1 \\ Loss & Binary cross-entropy, Mean Squared Error, Least Squares, Wasserstein Loss, \\  & Wasserstein Gradient Penalty Loss. \\ \hline \hline \end{tabular}
\end{table} 
Table 1: List of hyperparameters and settings tested during optimization of GAN model architectures. This results in a greater penalization of larger errors (E) which forces the model to update weights more frequently, therefore avoiding vanishing gradients [55]. _WGAN_, on the other hand, is based on the theory of _Earth-Mover's distance_[52], calculating the distance between the two probability distributions so that one distribution can be converted into another (Equations (3) and (4)):  
\[L_{D}^{WGAN}=-\mathrm{E}_{x\sim p_{d}}[D(x)]+\mathrm{E}_{\hat{x}\sim p_{g}}[D( \hat{x})] \tag{3}\]  
\[L_{G}^{\mathrm{SGAN}}=-\mathrm{E}_{\hat{x}\sim p_{g}}[D(\hat{x})] \tag{4}\]  
_WGAN_ additionally uses weight constraints (hypercube of [\(-\)0.01, 0.01]) to ensure that the discriminator lies within a 1-Lipschitz function. In certain cases, however, this has been reported to produce some undesired effects [53]. As an alternative, a proposed adaptation, in the form of _gradient penalty WGAN_ (_WGAN-GP_), includes the same loss for the generator (Equation (4)) but a modified discriminator (eg., 5) with no weight constraints [53; 56]:  
\[L_{D}^{WGANGP}=L_{D}^{WGAN}+\lambda\mathrm{E}_{\hat{x}\sim p_{g}}\Big{[}(\| \
abla D(\alpha x+(1-\alpha\hat{x}))\|_{2}-1)^{2}\Big{]} \tag{5}\]  
For both loss functions to work, the output of \(D\) requires a linear activation function. Finally, optimization tests were performed using Adam (\(\alpha\) = 0.0002, \(\beta_{1}\) = 0.5) and RMSprop (\(\alpha\) = 0.00005) [50; 53; 57; 58]. More details on the mathematical components of each GAN loss function can be consulted in the corresponding references [25; 50; 52; 53; 55]. GANs were trained on scaled PCA feature spaces with 64-bit values ranging between 1 and \(-\)1. This scaling procedure was performed to boost neural network performance and optimization by helping reduce the size of weight updates [24]. For these experiments, GANs were trained on all data within the dataset, regardless of label. This approach was chosen to directly observe how GANs handle this type of input data before considering more complex applications, including sample labels. (See Section 2.4)  
All experiments were performed in the Python programming language ([[https://www.python.org/](https://www.python.org/)]([https://www.python.org/](https://www.python.org/)), v.3.7 64-bit) using TensorFlow ([[https://www.tensorflow.org/](https://www.tensorflow.org/)]([https://www.tensorflow.org/](https://www.tensorflow.org/)), v.2.0). Neural networks were compiled and trained on the CPU of an ASUS X550 VX laptop (Intel(r) CoreTM i5 6300 HQ).