## 2 Methodologies
### Variational Autoencoders Model
Furthermore, it is assumed maximizing the marginal likelihood learning of parameters, expressed as [49]:  
\[logp_{\phi}(\mathbf{x}^{{}^{\prime}})=D_{KL}\big{[}q_{\theta}(\mathbf{z}| \mathbf{x})\big{\|}p_{\phi}(\mathbf{x})\big{]}+\mathcal{L}(\theta,\phi; \mathbf{x}), \tag{1}\]  
where \(D_{KL}[. ]\) denotes the Kulback-Leibler divergence, and \(\mathcal{L}\) refers to the likelihood of the parameters of encoder and decoder (i.e., \(\theta\) and \(\phi\)). Hence, the loss can be expressed as  
\[\mathcal{L}(\theta,\phi)=\underbrace{\mathbb{E}_{\mathbf{z}\sim\text{q}_{ \theta}(\mathbf{z}|\mathbf{x})}(logp(\mathbf{x}^{{}^{\prime}}|\mathbf{z}))}_{ \text{Reconstruction term}}-\underbrace{D_{KL}\big{(}\text{q}_{\theta}(\mathbf{z}| \mathbf{x})||\text{p}_{\phi}(\mathbf{z})\big{)}}_{\text{Regularization term}}. \tag{2}\]  
The VAE's loss function is composed of two parts: the reconstruction loss and a regularizer. Reconstruction loss tries to get an efficient encoding-decoding procedure. In contrast, a regularizer part permits the regularization of the latent space construction to approximate the distributions out of the encoder as near as feasible to a prefixed distribution (e.g., Normal distribution). Figure 2 schematically summarizes the procedure for computing the loss function. The term (2) permits reinforcing the decoder capacity to learn data reconstruction. Higher values of the reconstruction loss mean that the performed reconstruction is not suitable, while lower values mean that the model is converging. The regularizer is reported using the Kulback-Leibler (KL) divergence separating the distribution of the encoder function (\(\text{q}_{\theta}(\mathbf{z}|\mathbf{x})\)) and of the latent variable prior (\(\mathbf{z}\), \(|\text{p}_{\phi}(\mathbf{z})\)). Indeed, KL is employed to compute the distance that separates two given probability distributions. The gradient descent method is used to minimize the loss function with respect to the encoder's parameters and decoder in the training phase. Overall, we minimize the loss function to ensure getting a regular latent space,\(z\), and adequate sampling of new observation using \(\mathbf{z}\sim\text{p}_{\phi}(\mathbf{z})\)[50]. Let assume that \(p_{\phi}(\mathbf{z})=\mathcal{N}(z;0,I)\), we can write \(q_{\theta}(\mathbf{z}|\mathbf{x})\) in the following form:  
\[logq_{\theta}(\mathbf{z}|\mathbf{x})=log\mathcal{N}(\mathbf{z};\mu,\sigma^{2}I). \tag{3}\]  
The mean and standard deviation of the approximate posterior are denoted by \((\mu,\sigma)\), respectively. Note here that a layer is dedicated to both of them. Moreover, the latent space \(z\) is constructed using a deterministic function \(g\) parameterized by \(\phi\) and an auxiliary noise variable \(\varepsilon\sim p(\varepsilon)\) or more specifically \(\varepsilon\sim\mathcal{N}(0,I)\). \[z=g_{\phi}(x,\varepsilon)=\mu+\sigma\odot\varepsilon. \tag{4}\]  
The reconstruction error term can be expressed in the following form:  
\[\mathcal{L}(\theta,\phi,\mathbf{x})=\frac{1}{2}\sum_{i}\left(1+log((\sigma_{i })^{2})-(\mu_{i})^{2}-(\sigma_{i})^{2}\right)+\frac{1}{L}\sum_{I=1}^{L}log(p_{ \phi}(\mathbf{x}|\mathbf{z}^{(I)})), \tag{5}\]  
where the \(\odot\) denotes the element-wise product. Overall, the encoder and decoder's parameters are obtained by minimizing the loss function, \(\mathcal{L}(\theta,\phi)\), using the training observations. The VAE is trained using the procedure tabulated Algorithm 1. Figure 2: Reconstruction loss and Kulback-Leibler (KL) divergence to train VAE.