# A Universal Malicious Documents Static Detection Framework Based on Feature Generalization
## 6 Experiments and Results
### Experimental Results
#### 6.3.2 Comparison with Antivirus Software  
The proposed method was compared with commonly used antivirus software, including Kaspersky, NOD32, and McAfee. The virus database was updated in October 2021. As shown in Table 9, compared with Kaspersky, NOD32, and McAfee, the proposed static detection framework for malicious documents achieved superior accuracy and recall rates for the detection of various types of documents; however, the FPR of benign samples was slightly increased. The main reasons for this are as follows:  
1. To reduce the detection time, the number of features selected in the experiment was small. For example, seven important features can be selected as regular features for \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Type** & **Method** & **ACC (\%)** & **TPR (\%)** & **FPR (\%)** \\ \hline \multirow{2}{*}{PDF} & Decision Tree & 99.11 & 99.43 & 1.83 \\  & Random Forest & 99.37 & 99.40 & 0.78 \\ \hline \multirow{2}{*}{DOC} & Decision Tree & 97.16 & 97.11 & 2.40 \\  & Random Forest & 97.40 & 97.23 & 0.82 \\ \hline \multirow{2}{*}{DOCX} & Decision Tree & 98.28 & 95.71 & 0.67 \\  & Random Forest & 99.04 & 97.36 & 0.27 \\ \hline \multirow{2}{*}{XLS} & Decision Tree & 93.89 & 92.91 & 2.12 \\  & Random Forest & 94.66 & 93.64 & 1.17 \\ \hline \multirow{2}{*}{GIF} & Decision Tree & 99.50 & 99.58 & 0.73 \\  & Random Forest & 99.88 & 99.89 & 0.17 \\ \hline \multirow{2}{*}{JPEG} & Decision Tree & 96.57 & 94.12 & 1.39 \\  & Random Forest & 96.76 & 94.54 & 1.34 \\ \hline \multirow{2}{*}{PNG} & Decision Tree & 98.63 & 98.63 & 1.37 \\  & Random Forest & 98.50 & 97.64 & 0.615 \\ \hline \multirow{2}{*}{RTF} & Decision Tree & 97.81 & 95.93 & 0.43 \\  & Random Forest & 98.68 & 97.59 & 0.28 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: The experimental results of various documents under the proposed static detection framework for malicious documents based on feature generalization. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Type** & **ACC (\%)** & **TPR (\%)** & **FPR (\%)** & **F1-Score** \\ \hline PDF & 99.2 & 98.4 & 0 & 0.9919 \\ DOC/DOCX & 98 & 99 & 0.03 & 0.9802 \\ XLS/XLSX & 97.25 & 97 & 0.025 & 0.9724 \\ GIF & 100 & 100 & 0 & 1 \\ JPEG & 97.69 & 95.42 & 0 & 0.9766 \\ PNG & 99.47 & 99 & 0 & 0.995 \\ RTF & 99.25 & 98.5 & 0 & 0.9924 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: The detection accuracies of different documents. PDF document detection [10], but only four important features were selected in this experiment;
2. The antivirus software has a huge virus database, and the sample size of this system is not sufficiently large; this led to a slight increase in the FPR. As shown in Table 9, the FPRs of Kaspersky, NOD32, and McAfee may have been zero. The main reasons for this situation are as follows:  
1. If the antivirus software mistakenly reports a benign document as a malicious document, this will cause serious consequences; the antivirus software will try to reduce the FPR;
2. Antivirus software may not be perfect for this type of document detection. Although the FPR was 0, the ACC was also low;
3. The number of samples in the comparison experiment may not have been sufficient, and there were no samples that could have caused the antivirus software to falsely report; and
4. Some types of documents have fewer vulnerabilities that can be used for virus embedding, and antivirus software is characterized by more complete detection for this type of document; thus, there also occurred the situation in which the FPR was 0 and the accuracy rate was also high.