## 3 Methodology
### The Proposed VAE Framework
Thus, the KL divergence term enables latent variables to exhibit distributions similar to the prior distribution. This enables the structure and continuity within the latent space, consequently ensuring stability in generative objectives. Thus, the VAE has a goal to optimize two loss functions. The first of them is the'reconstruction' loss function. It is the same as it is the case with standard AEs. Thus, it uses Mean Squared Error (MSE) metrics to reduce the difference between the original and reconstructed inputs by decoder as it is possible to see in Equation (1):  
\[L_{MSE}(\theta,\phi)=\frac{1}{N}\sum_{i=1}^{N}\bigl{(}x_{i}-f_{\theta}\bigl{(} g_{\phi}(x_{i})\bigr{)}\bigr{)}^{2}, \tag{1}\]  
where \(\theta\) and \(\phi\) are the trainable parameters of the decoder and encoder parts of VAEs, respectively. The variable \(N\) denotes the number of samples in the learning dataset, \(x_{i}\) is the original input sequence, while \(f_{\theta}\) and \(g_{\phi}\) are decoder and encoder functions, respectively. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Hyperparameter** & Batch size & Learning rate & Regularization & Learning & Epoch & Learning/ \\  & & & & algorithm & & Validation dataset \\ \hline
**Value** & 128 & 0.001 & Batch Normalization & ADAM & 50 & 80:20 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Hyperparameters for all investigated AE-based models. Thus, Equation (1) is used to compute mean squared differences between each original input sequence \(x_{i}\) and corresponding reconstructions \(f_{\theta}\big{(}g_{\phi}(x_{i})\big{)}\) over \(N\) samples. The second loss that must be optimized with the VAE framework is the KL divergence term. Its main goal is to reduce the difference between values within latent space and learned standard normal distribution as the described regularisation approach. The KL divergence loss can be seen in Equation (2):  
\[L_{KL}\big{[}G\big{(}Z_{\mu},Z_{\sigma}\big{)}|\mathcal{N}(0,1)\big{]}=-0.5 \sum_{i=1}^{N}1+log(Z_{\sigma_{i}^{2}})-Z_{\mu_{i}}^{2}-Z_{\sigma_{i}}^{2}, \tag{2}\]  
where \(G\big{(}Z_{\mu},Z_{\sigma}\big{)}\) is the _Gaussian_ distribution generated by the encoder's outputs \(Z_{\mu}\) as the mean and \(Z_{\sigma}\) as standard deviation. The \(\mathcal{N}(0,1)\) denoted the standard normal distribution. Thus, the final VAE loss is computed as the weighted sum of KL divergence and reconstruction loss as can be seen in Equation (3):  
\[\mathcal{L}_{VAE}=L_{MSE}(\theta,\phi)+L_{KL}\big{[}G\big{(}Z_{\mu},Z_{\sigma }\big{)}|\mathcal{N}(0,1)\big{]}. \tag{3}\]  
Furthermore, the sampling operation from probabilistic distributions generates stochasticity in encoder outputs. Hence, introduced stochasticity destabilises the learning process since training algorithms use gradients of deterministic operations to minimise back-propagation learning error. The reparameterization approach is used in VAE frameworks to split the sampling operation into two parts. The first part is deterministic while the other is stochastic. Hence, a random variable \(\epsilon\) is sampled from a simple probabilistic distribution outside the learning loop. In this study, the sampling is performed by standard normal distribution \(N(0,1)\). The outputs of the encoder model in the form of latent space variables \(z[0]\) and \(z[1]\) in function \(z\) are computed based on the deterministic and differentiable Equation (4):  
\[z=\eta+\sigma\cdot\epsilon. \tag{4}\]  
This approach prevents direct sampling from the parameterised distribution \(N\big{(}\eta,\sigma^{2}\big{)}\) learned by the encoder model. Thus, a learning algorithm based on the back-propagation approach can be implemented through it.