## 5 Experimental Analysis
### Ablation Study  
In this section, an ablation study conducted on STREED-Net is presented, in which variations in the input structure and in the network architecture are analyzed. The study, for reasons of space, refers only to BikeNYC case study and does not involve the full combinatorics of all possible variants of the proposed model but aims to assess the impact on performance metrics of some parameters (namely, the number of input time points \(n\)) and specific architectural choices (viz., long skip connection, attention blocks, and external \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **RMSE** & **MAPE** & **APE** \\ \hline HA & 164.31 & 27.19 & 7.94 \(\cdot 10^{5}\) \\ ST-ResNet* & \(\mathbf{35.87\pm 0.60}\) & \(22.52\pm 3.43\) & \(6.57\cdot 10^{5}\pm 1.00\cdot 10^{5}\) \\ MST3D & \(48.91\pm 1.98\) & \(23.98\pm 1.30\) & \(6.98\cdot 10^{5}\pm 1.34\cdot 10^{4}\) \\
3D-CLoST & \(48.17\pm 3.16\) & \(22.18\pm 1.05\) & \(6.48\cdot 10^{5}\pm 3.08\cdot 10^{4}\) \\ PredCNN & \(40.91\pm 0.51\) & \(25.65\pm 2.16\) & \(7.49\cdot 10^{5}\pm 6.32\cdot 10^{4}\) \\ ST-3 DNet & \(41.62\pm 3.44\) & \(25.75\pm 6.11\) & \(7.52\cdot 10^{5}\pm 1.78\cdot 10^{5}\) \\ STAR* & \(36.44\pm 0.88\) & \(25.36\pm 5.24\) & \(7.41\cdot 10^{5}\pm 1.53\cdot 10^{5}\) \\ ACFM & \(36.75\pm 0.94\) & \(\mathbf{19.10\pm 1.08}\) & \(\mathbf{5.58\cdot 10^{5}\pm 2.21\cdot 10^{4}}\) \\ \hline STREED-Net & \(36.22\pm 0.72\) & \(20.29\pm 1.48\) & \(5.93\cdot 10^{5}\pm 4.31\cdot 10^{4}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Results obtained for the Taxi New York data set. factors input branch), while maintaining all other conditions. More precisely, in what follows, STREED-Net is compared against the 5 different variations described below:  
* **STREED-Net_N3**. Same architecture as STREED-Net, but input volumes with 3 frames (\([X_{t-3},X_{t-2},X_{t-1}]\)). * **STREED-Net_N5**. Same architecture as STREED-Net, but input volumes with 5 frames (\([X_{t-5},X_{t-4},X_{t-3},X_{t-2},X_{t-1}]\)). * **STREED-Net_NoLSC**. STREED-Net by removing the _long skip connection_ between encoder and decoder. * **STREED-Net_NoAtt**. STREED-Net without the _attention_ blocks. * **STREED-Net_NoExt**. STREED-Net without the external factors. Notice that the study does not consider the variations with \(n=1\) and \(n=2\) as such values would not allow the network to capture meaningful temporal patterns between traffic flows. Table 4 reports the results of the ablation study conducted. Each data point in the table has been obtained performing 10 times the training procedure for each model variation changing the random seed, and evaluating the resulting network on the test set. The mean and standard deviation are reported. The results show that regarding the time horizon, for the BikeNYC case study, \(n=4\) allows the model to obtain better results. This means that, considering the particular setup, for the city of New York 4 hours of data allow to predict more accurately the dynamics of bicycle mobility whereas considering a greater amount of information (\(n=5\)) would reduce the accuracy of the network. It is plausible to believe that considering a larger number of temporal instants would lead the network to grow in the number of parameters to be trained and thus require a larger amount of data to identify possible longer-term patterns. From the architectural point of view, the two components attention block and long skip connection, confirm their importance in improving the performance of the proposed model, accounting for a 2.36% and 3.64% increase in RMSE, respectively. In particular, as regards the attention block, not only STREED-Net reaches lower average error values, but also the standard deviation is reduced, proving that the attention blocks are effective in helping the network single out the most meaningful information and in making the training process more stable.