## 2 Materials and Methods
### Neural Networks and VAEs  
Neural networks are algorithms that mimic the human brain structure and operation [23]. They consist of layers of stacked neurons that are interconnected with synapses [24; 25]. A vector \(x=[x_{1},\ x_{2},\ x_{3},\ \ldots,\ x_{n}]\) is inputted to the neural network. The input vector is multiplied with a vector of weights \(w_{i}^{j}=[w_{i1^{\prime}}^{j},\ w_{i2^{\prime}}^{j},\ w_{i3^{\prime}}^{j} \ \ldots,\ w_{in}^{j}\) and by adding a bias term (i.e., \(b_{i}^{j}\)). The linearly combined weight vector and input vector are then transformed  
Figure 1: A graphical illustration of the general idea for using VAEs as a tool for data augmentation in bioequivalence studies with high-variability drugs. Instead of requiring a large sample size (termed “original”), only a subgroup of this (i.e., “subsampled”) is needed. VAE is then applied to the subsampled dataset in order to synthesize the generated datasets. The latter can exhibit either the same size (1\(\times\)), double the size (2\(\times\)), or triple the size (3\(\times\)) of the original dataset.  
by a function (i.e., \(f\)) and are later used as input by the \(i\)th neuron at the \(j\)th layer. This procedure is described in Equation (1) and is usually referred to as \"forward propagation\".  
\[z_{i}=f\Big{(}w_{i}^{j}\cdot x+b_{i}^{j}\Big{)} \tag{1}\]  
where \(\cdot\) denotes the dot (scalar) product and \(f\) is called the activation function. The most common activation functions are the sigmoid function (2), the hyperbolic tangent (3), the rectified linear unit (i.e., ReLU) (4), and softplus (5):  
\[f(z)=\frac{1}{1+e^{-z}} \tag{2}\]  
\[f(z)=\frac{e^{z}-e^{-z}}{e^{z}+e^{-z}} \tag{3}\]  
\[f(z)=max(0,z) \tag{4}\]  
\[f(z)=\ln(1+e^{z}) \tag{5}\]  
This process is repeated across all layers and neurons in a left-to-right manner. The final output of the network \(\hat{y}\) is compared with the actual value \(y\) and the weights are eventually iteratively optimized with respect to a cost function that is suitable to the problem (backward propagation) [26]. The forward propagation calculations for the first neuron of the first layer are visually presented in Figure 2.  
VAEs are a special case of neural networks that can be used to generate new data based on existing data. They are composed of two parts: the \"encoder\" and its \"mirrored\" image, the decoder [22; 27]. The encoder and the decoder are linked with the \"latent space\". The input data are passed through the encoder and are mapped to a probability distribution in the latent space [28; 29]. Datapoints from the latent space are then passed through the decoder, leading to the output. The differences between the traditional autoencoder and the VAE is that the loss function of the latter is composed of two parts, the reconstruction loss and the Kullback-Leibler loss. Moreover, random sampling can be performed from the probability distribution in the latent space. The sampled datapoints can be used as input from the decoder to generate novel datapoints. An extensive description of VAEs and autoencoders was presented in our previous studies [22; 27].  
A more in-depth explanation of variational autoencoders is provided in Appendix A and Figure A1.  
Figure 2: Visual representation of the calculations for the first neuron of the first layer during forward propagation.