## 3 Methods
### Variational Autoencoder  
Variational autoencoders represent one of the most effective and proficient classes of deep generative methods [41]. Recently, VAE-based models have been shown good performance in different applications, including forecasting of photovoltaic solar power [42], desertification detection [43], overcrowding forecasting [44], air pollution forecasting [45], and COVID-19 time series forecasting [46; 47]. The primary components of the VAE architecture are two neural networks: An encoder and a decoder (Figure 2). The encoder maps the data into a latent representation to get more compacted and informative data with a reduced dimension compared to the input data. Crucially, the decoder's principal mission is to learn a data distribution (i.e., the distribution parameters) over the latent variables. In other words, the decoder attempts to rebuild the input data based on the sampled data provided by the encoder. The encoder is usually computed via a posterior approximation of \(\mathbf{q}_{\theta}(\mathbf{h}|\mathbf{y})\), whereas the decoder is derived via a likelihood \(\mathbf{p}_{\phi}(\mathbf{y}|\mathbf{h})\), where \(\theta\) and \(\phi\) denote, respectively, the parameters of the VAE encoder and decoder. The VAE tries to find the appropriate assignments of latent variables \(\mathbf{h}\) that would have resulted in input data \(\mathbf{y}\). More specifically, \(\mathbf{h}\) is considered following a prior distri  
Figure 2: Variational autoencoder architecture. bution \(p_{\theta}(\mathbf{h})\), usually Gaussian distribution \(\mathcal{N}(\mathbf{0},\mathbf{I})\); the VAE encoder tries to estimate the parameters of this distribution. Analytically, the purpose is to determine  
\[p_{\theta}(\mathbf{h}|\mathbf{y})=\frac{p_{\theta}(\mathbf{y},\mathbf{h})}{p_{ \theta}(\mathbf{y})} \tag{8}\]  
It should be noted that this is challenging to compute because the left-hand side in (8) includes \(p_{\theta}(\mathbf{y})\), which is intractable. Precisely, it could be calculated through the marginalization out of the latent variables:  
\[p(\mathbf{y})=\int p(\mathbf{y}|\mathbf{h})p(\mathbf{h})dh. \tag{9}\]  
Regrettably, this integral is not easy to calculate. To bypass this difficulty, VAE treats it as an optimization problem [48]. More specifically, we can solve this based on the variational inference procedure by finding an approximation posterior \(q_{\phi}(\mathbf{h}|\mathbf{y})\)[48, 49]  
\[q_{\phi}(\mathbf{h}|\mathbf{y})=\textbf{ }\textbf  
First, the input (data sequence in our case) is processed via a non-linear transformation using a dense layer. Next, a self-attention layer is applied to dense layer output to highlight interactions between sequence data points by computing the context vector (i.e., a weighted sum of features). The regularization procedure ensures the diversification of the weighted sum; we applied regularization optimization methods to weights normalization based on the kernel-regularizer: and bias attenuation via bias-regularizer: L1 [52]. Moreover, regularization aims to avoid over-fitting during traffic flow training. The third step consists of feeding the LSTM using the self-attention output, which starts extracting the long-term dependencies learning and capturing the temporal sequential dependency embedded in traffic flow input (time-series). LSTM output is obtained after several non-linear transformations supported by a complex gating mechanism; this output serves as input to regularize the covariance matrix and the mean of the distributions returned by LSTM. More specifically, the regularization is realized by imposing that the distributions be similar to a standard Gaussian distribution and enforcing the covariance matrix close to the identity. Next, the latent space is obtained after double self-attention of the regularized mean and variance; both are concatenated to form an enhanced input for the encoder output layer. Data points are sampled from the latent space to be reconstructed using the decoder model; only generative models can generate new data.