## 2 Methodologies
### Variational Autoencoders Model  
VAEs are an essential class of generative-based techniques that are efficient to automatically extract information from data in an unsupervised manner [20, 47]. One desirable characteristic of VAEs is their ability for reducing the input dimensionality enabling them to compress large dimensional data into a compressed representation. Moreover, they are very effective for approximating complex data distributions using stochastic gradient descent [47]. There are two major advantages of VAEs compared the conventional autoencoders, one is they are efficient to solve the overfitting problem in the conventional autoencoders by using a regulation mechanism in the training phase, and the second advantage is that they have proved effective when handling various kinds of complex data in different applications, including handwritten digits, and urban networks modeling [48]. Here, VAE is adopted for solar PV production forecasting. Figure 1 shows a schematic diagram of the construction of a VAE. Basically, the VAE, as a variant of autoencoders, contains two neural networks an encoder and a decoder, where the encoder mission is to encode a given observed set, **X** into a latent space \(Z\) as distribution, \(\mathbf{q}\left(\mathbf{z}|\mathbf{x}\right)\). The latent (termed hidden) space dimension is decreased in comparison to the dimension of the observed set. Indeed, the encoder is built to compress the observed set  
\begin{table}
\begin{tabular}{p{142.3 pt} p{142.3 pt} p{142.3 pt}} \hline \hline
**Model** & **Description** & **Key Points** \\ \hline \hline \multirow{4}{*}{**Supervised Autoencoder (SAE)**} & \multirow{4}{*}{**Autoencoders are neural networks that aim to create a compact representation of a given input x like images or any type of data [44]. **} & \multirow{4}{*}{** \begin{tabular}{} \end{tabular} **} \\  & & \\ \cline{1-1}  
toward this reduced dimensional space efficiently. Then, a sample is generated via, \(z\sim\text{q}(\mathbf{z}|\mathbf{x})\),using the learned probability distribution. On the other hand, the key purpose of the decoder, \(\mathbf{p}\left(\mathbf{x}|\mathbf{z}\right)\), consists in generating the observation \(x\) based on the input \(z\). It should be emphasized that the reconstruction of data using the decoder results in some deviation of reconstruction, which is calculated and backpropagated through the network. This error is minimized in the training phase of the VAE model by the minimization of the deviation between the observed set and the encoded-decoded set. To summarize, the VAE encoder is gotten via an approximate posterior \(\mathbf{q}_{\theta}\left(\mathbf{z}|\mathbf{x}\right)\), and the decoder is obtained by a likelihood \(\mathbf{p}_{\phi}\left(\mathbf{x}|\mathbf{z}\right)\), where \(\theta\) and \(\phi\) refers respectively to the parameters of encoder and decoder. Here a neural network is constructed for learning \(\theta\) and \(\phi\). Essentially, the VAE encoder's role is learning latent variable \(\mathbf{z}\) based on gathered sensor data, and the decoder employs the learned latent variable \(\mathbf{z}\) for recovering the input data. The deviation between the reconstructed data and the input data should be close to zero as possible. Notably, the learned latent variable \(\mathbf{z}\) from the encoder is used for feature extraction based on the input data. Usually, the dimension of the output of the encoder is smaller than that of the original data, which leads to the dimensionality reduction of input data. Note that the encoder is trained by training the entire VAE comprising encoder and decoder. It is worth pointing out that the loss function has an essential effect on feature extraction for training VAE. Assume that \(\mathbf{X}_{t}=\left[x_{11},x_{2t},\ldots,x_{Nt}\right]\) is the input data points of VAE at time point \(t\), and \(\mathbf{X}_{t}^{{}^{\prime}}\) is the reconstructed data using the VAE model.