# HP-LSTM: Hawkes Process-LSTM-Based Detection of DDoS Attack for In-Vehicle Network
## 5 Experiments
### Experimental Results
#### 5.2.2 Model Evaluation  
The bar chart in Figure 7 provides a comprehensive evaluation of the original LSTM and four variants of the HP-LSTM. We evaluated the performances of the five models based on four metrics: accuracy, precision, recall, and F1 score. Through Figure 7, it is visually evident that the HP-LSTM demonstrates superior capabilities compared to the other models. This is attributed to several factors. Firstly, the parameter \(\beta\) in the Hawkes coefficient undergoes exponential operations, enabling a more accurate depiction of the dynamics described by the Hawkes process, thereby facilitating model learning. Secondly, when there is a multiplicative relationship between the Hawkes coefficient and the cell state of HP-LSTM, it better constrains the outputs of the forget gate and input gate, allowing the excitatory and decay characteristics of the Hawkes process to more effectively assist the neural network in learning patterns within the message window.  
Table 7 elucidates the performances of the five models under study. The table is structured to highlight the highest metric values for each category using bold typeface. Although, numerically, the differences among the five models are minimal, the performances of the four variants of the HP-LSTM are generally superior to that of the LSTM, with the HP-LSTM model exhibiting the most pronounced superiority.  
Figure 8 presents the confusion matrices of the five models. The analysis of these matrices revealed that all five models demonstrated high detection rates and low false positive rates. This occurrence primarily stems from the fact that all models are based on the HP-LSTM (or the original LSTM), which inherently possesses the capability to effectively capture and analyze the temporal dynamics inherent in DDoS attacks. The results indicate that all five models exhibit strong feature extraction capabilities in terms of time, with HP-LSTM and HP-LSTM hted-chm demonstrating superior performances in terms of detection rate.  
Figure 7: Accuracy, precision, recall and F1 score comparison of LSTM and four variants of HP-LSTM.  
Figure 9 shows the ROC curves and AUC values for the original LSTM and four variants of the HP-LSTM. It can be observed that the AUC values of all models reached 1. This indicates that all models are capable of perfectly distinguishing between positive and negative samples at all possible thresholds, suggesting their excellent performance. In fact, as illustrated in Table 7, the detection accuracy of the five models fluctuate around 0.99, demonstrating their outstanding ability to detect DDoS attacks.  
Table 8 compares parameters and time costs among the LSTM and four variants of the HP-LSTM. It can be observed that due to the introduction of parameters from the Hawkes process, the HP-LSTM variants generally have more parameters and time costs compared to the traditional LSTM. Moreover, HP-LSTM hted-chm has more parameters and a slower detection speed than the other variants due to the introduction of an additional bias term D. In fact, all models are capable of meeting the traffic and real-time requirements defined in IEEE 802.1 DG [14] (<1 ms), demonstrating the ability of the HP-LSTM to achieve real-time detection in practical vehicular environments.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Class** & **Accuracy** & **Precision** & **Recall** & **F1-Score** \\ \hline HP-LSTM & **0.993** & **0.993** & **0.993** & **0.993** \\ \hline LSTM & 0.985 & 0.985 & 0.985 & 0.985 \\ \hline HP-LSTM hted-chm & 0.992 & 0.992 & 0.992 & 0.992 \\ \hline HP-LSTM ht-chm & 0.986 & 0.986 & 0.986 & 0.986 \\ \hline HP-LSTM hte-chp & 0.990 & 0.990 & 0.990 & 0.990 \\ \hline \hline \end{tabular}
\end{table}
Table 7: Comprehensive evaluation of our models.  
Figure 8: Confusion matrices for our five models on testing dataset (AN: Actual Negative; AP: Actual Positive; PN: Predicted Negative; PP: Predicted Positive).