# Variational Autoencoders-Based Algorithm for Multi-Criteria Recommendation Systems
## 2 Background and Related Work
### Variational Autoencoders  
A variational autoencoder (VAE) is a deep learning generative model that merges the principles of autoencoders with probabilistic modeling, particularly variational inference, allowing for the generation of new data points by learning the underlying data distribution. It is commonly used for dimensionality reduction, data generation, and latent space exploration. The main goal of a VAE is to learn a compressed representation, called the latent representation, of the input data that captures its underlying structure. The VAE achieves this by training an encoder and a decoder network sequentially. As illustrated in Figure 1, in a VAE, the architecture and parameters for each layer are designed to achieve a balance between efficient data compression and accurate reconstruction. The VAE consists of two main components: the encoder and the decoder. The encoder reduces the input data to a low-dimensional latent representation. It begins with an input layer, with a number of neurons corresponding to the feature count. This is followed by several hidden layers that iteratively decrease the number of neurons. The latent layer, which is the final layer of the encoder, outputs latent variables \(z\) by estimating two parameters (\(\mu\) mean and \(\sigma\) standard deviation) of a multivariate Gaussian distribution. This allows for sampling from the latent space during training, introducing a stochastic element to the model. The latent variable \(z\) is computed as shown in the following Equation (1):  
\[z=\mu+\sigma\odot\epsilon \tag{1}\]  
where \(\epsilon\sim\mathcal{N}(0,I)\) is sampled from a standard normal distribution, where 0 represents the mean and \(I\) is the identity matrix, ensuring that \(z\) follows a multivariate Gaussian distribution with zero mean and unit variance. \(\odot\) denotes element-wise multiplication. The decoder mirrors the encoder's structure, expanding the latent space back to the original input dimensions. It starts with a latent input layer that takes the latent variables as input. The decoder includes hidden layers that progressively increase the number of neurons, reflecting the encoder's structure in reverse. Finally, the output layer reconstructs the original input dimensions to closely approximate the input data based on the compressed latent representation. Each layer is fully connected to the previous adjacent layer and employs chosen neuron counts and activation functions to ensure generalizability and mitigate the risk of overfitting. During training, the VAE aims to optimize two objectives simultaneously: the reconstruction loss and the KL divergence loss. The reconstruction loss measures how well the decoded output matches the original input by calculating the difference between the original input and its reconstruction, encouraging the decoder to generate accurate outputs. It is computed using the binary cross-entropy BCE or MSE between the original input \(x\) and its reconstruction \(\hat{x}\), as given in the following Equation (2):  
\[\text{Reconstruction Loss}=\|x-\hat{x}\|^{2} \tag{2}\]  
The KL divergence loss ensures that the learned latent space distribution aligns with a prior distribution, usually a standard Gaussian. This regularization term encourages the latent space to be smooth and encourages the model to learn a useful and well-structured representation. This regularizes the learned latent space distribution by reducing the Kullback-Leibler (KL) distance between the approximate posterior distribution \(q(z|x)\) (parametrized by \(\mu\) and \(\sigma\)) and the prior distribution \(p(z)\), which is usually a standard normal distribution \(\mathcal{N}(0,I)\).