# Improving Deep Learning-Based Recommendation Attack Detection Using Harris Hawks Optimization
## 4 Experiments and Analysis
### Experimental Results and Analysis
#### 4.5.2 Comparison with Representative Methods  
The performance comparison of multiple detection methods when detecting the test sets injected with random, average, and bandwagon attack on Movielens 10 M dataset are shown in Figures 9-11, respectively. As shown in Figures 9-11, the PCA-VarSelect has low performance when detecting the bandwagon attack. The possible reason is that there are many common ratings, which correspond to the popular items, between attack profiles and genuine profiles. The principal component analysis technology determines some of the attack profiles as principal components. Therefore, the performance of PCA-VarSelect decreases. The CNN-LSTM method has poor detection performance for small filler sizes. The reason for this phenomenon might be that attack profiles with small filler sizes contain a few ratings. The detection method CNN-LSTM cannot effectively capture enough knowledge for recognizing the patterns of attack profiles. The SSADR-CoF, CNN-SAD-HHO, and DL-DRA-HHO have good detection performance for most of the test sets. The experimental results fully show the effectiveness of the proposed approach. The detection results of PCA-VarSelect, SSADR-CoF, CNN-LSTM, CNN-SAD-HHO, and DL-DRA-HHO on the test set of Amazon dataset are shown in Table 8. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Metrics** & **CNN-SAD** & **DL-DRA** &
\begin{tabular}{c} **CNN-SAD-** \\ **HHO** \\ \end{tabular} & **DL-DRA-HHO** \\ \hline Recall & 0.795 & 0.931 & 0.941 & 0.951 \\ Precisioin & 0.819 & 0.797 & 0.812 & 0.839 \\ AUC & 0.872 & 0.932 & 0.939 & 0.949 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Performance comparison of the four detection methods when detecting the test set on Amazon dataset. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Dataset** & **CNN-SAD** & **DL-DRA** &
\begin{tabular}{c} **CNN-SAD-** \\ **HHO** \\ \end{tabular} & **DL-DRA-HHO** \\ \hline Movielens 10 M & 217.5 & 10.5 & 72.5 & 54 \\ Amazon & 210 & 2.9 & 84 & 20 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Training time (measured in minutes) of the multiple detection methods with the key hyperparameter optimization process. Figure 10: Performance comparison of the five detection methods when detecting the test sets injected with average attack on Movielens 10 M dataset. \begin{table}
\begin{tabular}{c c c c c c} \hline
**Metrics** & **PCA-VarSelect** & **SSADR-CoF** & **CNN-LSTM** & **CNN-SAD-HHO** & **DL-DRA-HHO** \\ \hline Recall & 0.207 & 0.961 & 0.951 & 0.941 & 0.951 \\ Precision & 0.207 & 0.795 & 0.605 & 0.812 & 0.839 \\ AUC & 0.490 & 0.945 & 0.886 & 0.939 & 0.949 \\ \hline \end{tabular}
\end{table} 
Table 8: Performance comparison of the five detection methods when detecting the test set on Amazon dataset. Figure 9: Performance comparison of the five detection methods when detecting the test sets injected with random attack on Movielens 10 M dataset. As shown in Table 8, the performance of PCA-VarSelect decreases greatly on the real dataset. The reason for this phenomenon might be that the real dataset usually contains multiple groups of attack profiles. The groups of attack profiles interfere with the principal component analysis and reduce the detection performance. The methods SSADR-CoF, CNN-LSTM, CNN-SAD-HHO, and DL-DRA-HHO have certain detection performance on the real dataset. Benefit from ensemble learning, the SSADR-CoF method has the largest recall. The improved method DL-DRA-HHO has the largest precision and AUC. Note that, the recall of DL-DRA-HHO is only 0.01 less than that of SSADR-CoF. The comparison results fully illustrate the success of the proposed approach. As a nonparametric test method, Friedman test uses rank to determine whether there is significant difference between multiple population distributions [42]. In our experiments, we use this method to measure the stability of the proposed detection method. We independently run the DL-DRA-HHO method with the Amazon dataset in the real scene for 30 times.