## 5 Practical Example and Experimental Evaluation
### Experimental Evaluation  
The traffic flow prediction accuracy results of the different prediction methods by means of Root Mean Square Error (RMSE) are shown in Figure 4. Figure 5 shows the average RMSE and Root Mean Square Percentage Error (RMSPE). The average of Figures 4 and 5 is the average RMSE, RMSPE values of roads R7-3, R7-2, R3-3, and R3-4 per the prediction models. The definition of RMSE and RMSPE are shown below.  
\[\text{RMSE}=\sqrt{\frac{\sum_{i=1}^{n}(x_{i}-\hat{x}_{i})^{2}}{n}} \tag{23}\]  
\[\text{RMSPE}=\frac{\frac{\sum_{i=1}^{n}(x_{i}-\hat{x}_{i})^{2}}{n}}{\frac{ \sum_{i=1}^{n}x_{i}}{n}}\times 100\% \tag{24}\]  
where \(n\) is the prediction interval number, \(x_{i}\) is the actual value, and \(\hat{x}_{i}\) is the prediction value.  
Figure 4: Prediction accuracy comparison of different models with: (**a**) 5-min prediction interval; (**b**) 15-min prediction interval; and (**c**) 30-min prediction interval.  
The average RMSE and RMSPE values of all road sections are shown in Figure 5.  
From Figure 4, the results showed that on most roads, the prediction accuracies of the different prediction models were similar for all intervals. The predication accuracy from low to high was Shift, AR, Seasonal MA, STAR, LSTAR, and LSTARIMA, with some exceptions on R3-4 and R3-3. From Figure 5, we can see that the RMSE increased as the prediction intervals increased for all prediction methods, while the RMSPE decreased when the prediction intervals increased. This indicates that although the absolute error increased as the prediction intervals increased, the actual prediction accuracy increased with larger prediction intervals as the percentage form errors decreased. Figure 5 also shows that, regardless of the prediction interval, the average prediction accuracy of LSTAR was always better than Shift, AR, Seasonal MA, and STAR. Moreover, LSTARIMA always had a little higher accuracy when compared to LSTAR in all prediction intervals.  
According to [PERSON] [43], only comparing values such as RMSE is not sufficient to declare that one prediction model is better than another without a statistics significance check. There are many hypothesis tests designed for prediction accuracy comparison and the Diebold-Mariano (DM) test [43] is the most popular one. To further evaluate the prediction performance of LSTAR, the DM test was used to check if LSTAR was better than other statistically significant prediction models. The forecast package of R [44] was used to conduct the DM test for the prediction results of road R7-3. As the DM test can only compare the prediction accuracy of two models, we did the DM test for LSTAR and the other models one by one. The DM test hypothesis was that LSTAR had better performance than all of the other methods subjected to the test. The \(p\)-values of each DM test are shown in Table 3.  
Figure 5: Average prediction accuracy comparison of different models: (**a**) Root Mean Square Error (RMSE); and (**b**) Root Mean Square Percentage Error (RMSPE).  
The DM test results showed that the LSTAR prediction accuracy was significantly better than Shift at \(p\)-value \(<\) 1%, better than AR at \(p\)-value \(<\) 5%, and almost better than Seasonal MA and STAR at \(p\)-value \(<\) 10% (one \(p\)-value of STAR \(>\) 10%). LSTARIMA was not significantly better than LSTAR at \(p\)-value \(<\) 10%, as no \(p\)-value was greater than 90% with the hypothesis that LSTAR is better than LSTARIMA.