## 4 Evaluation
### Influence of Packet2 vec-AE Hyper-Parameters  
Different penalty values \(\eta\) in the similarity relationship of network packets have different effects on the performance of IDS. In addition, other parameters have different effects on the performance of IDS, such as optimizer of auto-encoder, the pruning threshold, the length of random walk. Therefore, these parameters are also adjustable. Table 8 shows the performance of the auto-encoder (AE) in packet2 vec-AE with different optimizers. From this table, we have found that _adadelta_ as the optimizer for AE has the ability to obtain better performance of IDS. Table 9 shows the effect of different random walk lengths of packet2 vec on the performance of IDS. From Table 9, we conclude that the packet2 vec-AE algorithm has the ability to achieve optimal performance of IDS when the length of the random walk is 60 or 80. When the random walk length is 10 or 100, the performance of the packet2 vec-AE algorithm is relatively poor. One possible explanation is that random walks only sample ten proximity packets of the source network packet when the random walk length is 10. The number of network packets sampled by random walks is small, so the features obtained by random walk do not have the ability to accurately describe the local proximity of the source network packet. Therefore, when the random walk length is 10, the performance of the IDS is not good. When the random walk length is 100, the proximity features obtained by random walk sampling is too redundant. Therefore, when the random walk length is 60 or 80, the features of the local proximity of the source network packet can be accurately characterized. Table 10 shows the architectural parameters of the auto-encoder in the packet2 vec-AE algorithm. Table 11 shows a list of parameters for packet2 vec in packet2 vec-AE. Table 12 shows a list of auto-encoder hyper-parameters and its optimizer in packet2 vec-AE. The parameter values given in Tables 11 and 12 are the optimal results obtained by experiments. The hardware used in the experiments are presented below. The configuration of the experimental environment is shown in Table 13. In addition, we used a 12 GHz NVIDIA Tesla K40m GPU as an accelerator.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Weight in \(G^{\prime}\)** & **Value** \\ \hline Mean & 0.93 \\ Max & 0.99 \\ Min & 0.86 \\ \hline \hline \end{tabular}
\end{table}
Table 6: Statistical value of the weight in the \(G^{\prime}\).  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Penalty \(\eta\)** & **ACC** & **DR** & **Precision** & \(F_{1}\) \\ \hline \(-\)0.01 & 89.8 & 87.9 & 88.0 & 87.9 \\ \(-\)0.03 & **94.7** & **90.9** & **94.3** & **92.6** \\ \(-\)0.05 & 93.8 & 89.2 & 92.6 & 90.9 \\ \(-\)0.1 & 90.2 & 87.3 & 88.7 & 88.0 \\ \hline \hline \end{tabular}
\end{table}
Table 7: Compare the impact of different penalty values on the performance of Packet2 vec-AE (%).  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Optimizer** & **ACC** & **DR** & **Precision** & \(F_{1}\) \\ \hline _adam_ & 90.5 & 85.9 & 88.9 & 87.4 \\ _adadelta_ & **94.7** & **90.9** & **94.3** & **92.6** \\ _SGD_ & 92.9 & 88.3 & 92.6 & 90.4 \\ \hline \hline \end{tabular}
\end{table}
Table 8: Compare the impact of optimizer on the performance of Packet2 vec-AE (%).