## 2 Related Deep Learning Algorithms
### Variational Autoencoders--VAE  
Variational Autoencoder [23] (VAE) is an extension of an autoencoder. Just like an autoencoder, it contains encoder and decoder network components, but there are significant differences in its problem-learning structure. Contrary to the learning of mapping based on input data to a constant bottleneck vector (point estimation), VAE learns to map from input data to the distribution and reconstruct original data via sampling from this distribution using a hidden code.  
The encoder network learns the distribution parameters (of a medium distribution and a variation of the distribution), which introduces a hidden code vector considering the input data. In other words, it is possible to draw samples of the bottleneck vector that \"correspond\" to the samples from the input data. The nature of this distribution may differ depending on the nature of the data. On the other hand, the decoder learns the distribution that introduces the original input data point, considering the hidden code samples (probability). Usually, this reconstruction space is modeled using an isotropic Gaussian distribution.  
The VAE model is trained by minimization of the difference between the estimated distribution produced by the model and the actual data distribution. This difference is estimated using the Kullback-Leibler divergence, which determines the distance between two distributions by measuring the amount of lost information when a single distribution represents the other. Similarly to autoencoders, the VAE is applied in such cases as an unattended extraction of characteristics, reduction in dimensions, coloring of images, denoising of images, etc. Moreover, considering that they use one distribution model, they may be used to generate samples in a controlled manner.  
The probabilistic components introduced in VAE provide several advantages. First and foremost, VAE enables Bayesian inference; essentially, it is now possible to sample from a learned distribution of the encoder and decode samples that do not explicitly exist in the original dataset but belong to the same data distribution. Secondly, VAE learns a disjoint representation of the data distribution, i.e., a single unit in a hidden code is susceptible to only one generative factor. This enables specific interpretation of output data from VAE since we can change units in the hidden code to allow the controlled generation of samples. Thirdly, VAE ensures accurate probability measures, which offer a fundamental approach to quantifying uncertainty, such as the one used in practice.  
Just like in the case of an autoencoder, one should start by training a VAE on samples of normal data. During the test, the anomaly result may be calculated using two methods. Firstly, it is possible to draw a hidden code sample from the encoder, considering input data and a sample of values reconstructed using the hidden code from the decoder, and then calculate the mean reconstruction error. Anomalies are flagged based on a certain threshold of the reconstruction error.  
An alternative method is to derive an average and a variation parameter from the decoder and calculate the probability of a new data point belonging to the normal distribution on which the model is trained. If the data point lies in a region of low density (below a certain threshold), it will be flagged as an anomaly.  
VAEs demonstrate a robust capability to generalize to unseen data by modeling normal network behavior and identifying anomalies, proving effective even when specific anomalies were not part of the training dataset.