## 4 Model Testing and Results Analysis
### Results Analysis and Comparison
This could be attributed to its capacity to capture relevant information and dynamics from traffic flow time series. \begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Model** & **Dataset** & **RMSE** & **MAE** & \(R^{2}\) & **EV** \\ \hline LR & 1 & 17.568 & 15.698 & 0.295 & 0.571 \\  & 2 & 10.053 & 7.764 & 0.602 & 0.626 \\  & 3 & 10.332 & 8.17 & \(-\)0.364 & 0.019 \\  & 4 & 10.94  
To further assess the performance of the GAHD-VAE, we investigate the impact of the attention mechanism setting used in the proposed GAHD-VAE model on the forecasting accuracy. An important point to highlight is that the activation function changes how data is transformed (or processed) at the layer unit level and significantly impacts the neural network's overall performance. Mainly, we evaluate the impact of the used activation function in the attention mechanism on the proposed approach's forecasting performance. Table 6 shows the forecasting results obtained through different configurations of the activation function used on each attention layer: Rectified Linear Unit (ReLU), Hyperbolic Tangent (tanh), and Logistic Sigmoid. We also evaluate the impact of the attention type, namely multiplicative and additive, on the forecasting accuracy. Moreover, these experiments are based on four traffic flow datasets for the proposed approach with a self-attention mechanism (Table 6). Note here that the highlighted rows in Table 6 represent the results obtained with default attention configuration (activation function: Tanh; attention type: Additive), while the results in bold are the enhanced forecasting metrics. The term 'None' in Table 6 represents the case where the multiplicative self-attention is based only on matrix multiplications without the activation function. Figure 5: Measured and forecasted, (**a**) Data Set 1, (**b**) Data Set 2, (**c**) Data Set 3, (**d**) Data Set 4, (**e**) Data Set 5 and (**f**) Data Set 6. Results in Table 6 show that the GAHD-VAE model with the Sigmoid activation function, when applied to dataset 1, provides the best results for both attention types (i.e., multiplicative and additive). Specifically, it achieves the lowest RMSE and MAE values (i.e., 1.812 and 1.224, respectively) and describes 98.8% of the traffic flow variance. We also observe that adjusting the attention layer significantly improves the forecasting quality by reducing RMSE from 3.336 to 1.812 and MAE from 2.81 to 1.224 and improving \(R^{2}\) to more than 0.98. Moreover, Table 6 shows that the multiplicative type with Tanh and additive with Sigmoid offers the most favorable result for the traffic data set 4 (i.e., RMSE = 1.18, MAE = 0.761, and \(R^{2}\) = 0.986). The best forecasting accuracy when applying GAHD-VAE to Data Set 5 is obtained by using multiplicative type with Sigmoid activation function, where RMSE was reduced from 5.641 to 2.743 and MAE from 4.853 to 1.969, compared to the additive type with Tanh (i.e., default configuration). From Table 6, we also observe that there is no improvement on Dataset 6; the default configuration scored the best results. Overall, it is not obvious to automatically decide the best attention configuration for any dataset. On average, the use of GAHD-VAE with the Sigmoid activation function provides suitable forecasting performance. Figure 6: Boxplot of forecasting errors obtained by the seven considered methods based on the six datasets: (**a**) Data Set 1, (**b**) Data Set 2, (**c**) Data Set 3, (**d**) Data Set 4, (**e**) Data Set 5 and (**f**) Data Set 6.