# Research on Safety Prediction of Sector Traffic Operation Based on a Long Short Term Memory Model
## 4 Case Study
### Time Series Forecasting with Traffic Parameters  
The results in Section 4.1 show that the LSTM model does have comparatively higher prediction accuracy for conflict prediction. In order to achieve the potential improvements for model performance and issue a comprehensive analysis of the relative predictive power of traffic parameter, traffic volume, density, and velocity are added into prediction model as input variables for further observation. The experiment process is shown in Figure 9.  
For the LSTM model with new input variables, its training loss and verification loss in 10 trials are shown in Figure 10, which includes the value of loss in 10 trials and the average value of loss. At the beginning of training, the value of loss dropped significantly, and convergence was met within 20 generations. Then, the value of loss gradually became stabilized. In the end, the gap between the verification loss and training loss was pretty narrow and the task of model training was completed.  
\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{2}{c}{**Case 1**} & \multicolumn{2}{c}{**Case 2**} & \multicolumn{2}{c}{**Case 3**} \\  & **MAE** & **RMSE** & **MAE** & **RMSE** & **MAE** & **RMSE** \\ \hline SVR & 1.7500 & 2.5814 & 0.7050 & 1.7709 & 0.0500 & 0.2328 \\ LSTM & 1.9062 & 3.0783 & 0.3901 & 0.8037 & 0.0493 & 0.2252 \\ RR & 1.7877 & 3.0092 & 0.6259 & 1.5346 & 0.0500 & 0.2328 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Average error of models.  
Figure 9: LSTM model with traffic parameters.  
As shown in Figure 11, the values of performance indicators in 10 repeated experiments are in comparison with the results in Section 4.1, where the solid line represents prediction error of each trial, the dashed line represents average prediction error, and the box chart shows the distributions of prediction error.  
From the results, we observe that the LSTM model with all traffic parameters has presented better performance in predicting flight conflicts with error reduced by 3%.  
Figure 11: Performance of LSTM models with all traffic parameters and without traffic parameters.  
Figure 10: The training and verification loss.  
Moreover, the performance of prediction model without traffic parameters is more unstable and fluctuating greatly, which could be seen from the box chart. The results shows that the time series of traffic parameters do hold information related to the flight conflicts, which is of positive significance for conflict forecasting.  
Next, we establish the model with single traffic parameters to analyze the predictive power of each parameter in predicting flight conflicts, which could give insight into the relationship between traffic flow parameters and conflict. In a similar way, we compare the performance of the LSTM model with single parameter. The RMSE and MAE between predicted values and actual values are used as an evaluation metric to compare model performance, as shown in Figure 12, where the solid line represents prediction error of each trial, and the dashed line represents average prediction error.  
It is clear that the model with traffic volume shows a much better prediction accuracy, i.e., the smallest average RMSE and MAE, as the blue dashed line shows. The order ranking from high to low according to prediction accuracy is the model with volume, the model with density, and the model with velocity. As the box chart shows, which describes the values of average error as they spread across the entire range, we observe that the model with density is of best stability, the model with volume comes second and the model with velocity takes the final place in the aspect of model stability. We believe that volume is the key factor that could give help in improving the prediction accuracy among all traffic parameters, followed by density and velocity, while the improvement of model stability which is brought by density should be given adequate attention.