## 2 Materials and Methods
### Tuning of Hyperparameters  
In a neural network, such as a VAE, the most important hyperparameters refer to the number of hidden layers, number of neurons per hidden layer, number of epochs, activation function, optimization function, weight initialization, dropout rate, and regularization. In this study, the optimization of hyperparameters was performed using a grid search. In particular, several sets of values were predefined, and the performance of the VAE model was exhaustively evaluated through an iterative process of trial and error [11]. Various values were tested for epochs, such as 100, 500, 1000, 5000, and 10,000. In relation to the activation function, the experiments were conducted utilizing both \"softplus\" and \"linear\" activation functions for both the hidden and output layers. The \"softplus\" activation function is a smooth, nonlinear activation function that converts the input (logit) into a positive range. The \"softplus\" function is defined by Equation (1):  
\[\text{``Softplus\" activation function}=\log(1+\mathrm{e}^{X}) \tag{1}\]  
The \"softplus\" function is widely used since it is continuous and differentiable everywhere. Also, its smoothness helps in training neural networks more effectively through gradient-based optimization algorithms, such as gradient descent. Different methods were used to assess which hyperparameters better fit our application. This included the percentage of convergence, degree of similarity between the generated data, and original bell-shaped source data.  
The KL component and the reconstruction component of the loss function were equally weighted, and the dimension of the latent space was chosen. Finally, with regard to the number of hidden layers and the number of neurons in each hidden layer, various configurations were explored consisting of 2, 3, and 4 hidden layers for both the encoder and decoder. The encoder consisted of 128, 64, 32, and 16 neurons, while the decoder consisted of 16, 32, 64, and 128 neurons, respectively. Table 1 displays all of the aforementioned factors tested during the development of the VAE.  
All possible combinations of the factors listed on the left-hand side of Table 1 were investigated. The experimental runs detailed in Table 1 utilized the TensorFlow 2.10.0 Python  
\begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{2}{*}{\begin{tabular}{c} **Number of** \\ **Epochs** \\ \end{tabular} } & \multirow{2}{*}{\begin{tabular}{c} **Activation Function** \\ **Actions** \\ \end{tabular} } & \multirow{2}{*}{\
\begin{tabular}{c} **Weights of Loss Function** \\ \end{tabular} } & \multicolumn{3}{c}{
\begin{tabular}{c} **Number of** \\ **Hidden Layers** \\ **(from Left to Right)** \\ \end{tabular} } \\ \cline{2-9}  & & & & & & & **Encoder** & **Decoder** & **Decoder** \\ \hline
100 & softplus & softplus & 1 & 1 & 2 & 2 & 32-16 & 16-32 \\ \hline
500 & linear & linear & & & 3 & 3 & 64-32-16 & 16-32-64 \\ \hline
1000 & & & & & 4 & 4 & 128-64-32-16 & 16-32-64-128 \\ \hline
5000 & & & & & & & & \\ \hline
10,000 & & & & & & & & \\ \hline \hline \end{tabular}  
* Key: KL, Kullback–Leibler difference.  
\end{table}
Table 1: Hyperparameter tuning during the development of the variational autoencoders. In all cases, the latent space dimension was equal to 1.  
package and Python version 3.7, with execution taking place within a \"Jupyter notebook\" environment. Figure 1 depicts the general architecture of a variational autoencoder.