# [[https://github.com/opscq/exploit-CVE-2016-10033](https://github.com/opscq/exploit-CVE-2016-10033)]([https://github.com/opscq/exploit-CVE-2016-10033](https://github.com/opscq/exploit-CVE-2016-10033))
## 5 Conclusions  
It is very important to predict whether a disclosed vulnerability will be exploited because it would be helpful to judge which vulnerabilities should be removed first. To this end, existing studies have \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Algorithm** & **Num. of** & **Reference Host** & **Accuracy** & **Precision** & **Recall** & **F1-Score** \\ \hline \multirow{3}{*}{Random} & All host & 9226 & 82.96\% & 55.73\% & 79.96\% & 65.68\% \\  & PoC host & 1712 & 83.07\% & 56.03\% & 78.98\% & 65.55\% \\ Forest & No-PoC host & 4868 & 82.41\% & 54.89\% & 77.19\% & 64.16\% \\  & No host & 0 & 82.44\% & 54.99\% & 76.54\% & 64.00\% \\ \hline \multirow{3}{*}{Linear} & All host & 9226 & 81.45\% & 52.92\% & 81.91\% & 64.30\% \\  & PoC host & 1712 & 81.05\% & 52.27\% & 81.54\% & 63.71\% \\ SVC & No-PoC host & 4868 & 80.27\% & 51.03\% & 80.85\% & 62.57\% \\  & No host & 0 & 80.03\% & 50.66\% & 80.19\% & 62.09\% \\ \hline \multirow{3}{*}{Logistic} & All host & 9226 & 81.48\% & 52.97\% & 82.18\% & 64.42\% \\  & PoC host & 1712 & 81.20\% & 52.52\% & 81.86\% & 63.98\% \\ Regression & No-PoC host & 4868 & 80.41\% & 51.26\% & 80.52\% & 62.64\% \\  & No host & 0 & 80.23\% & 50.97\% & 80.26\% & 62.35\% \\ \hline \multirow{3}{*}{Decision} & All host & 9226 & 77.30\% & 46.60\% & 77.25\% & 58.13\% \\  & PoC host & 1712 & 76.90\% & 46.04\% & 77.10\% & 57.65\% \\ Tree & No-PoC host & 4868 & 75.47\% & 44.07\% & 75.39\% & 55.63\% \\ \cline{1-1}  & No host & 0 & 75.28\% & 43.80\% & 74.88\% & 55.27\% \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Results of the exploit prediction experiment based on Reference Feature. used various information about CVEs provided by NVD as features of machine learning algorithms. Among them, Reference, which provides URLs related to the CVE, has been used as the main feature. However, many references might have PoC codes. This would mean that the input data include answer information. We discovered that at least 46,202 references among a total of 308,427 references included PoC codes. We also figured out which hosts contained PoC codes, called PoC hosts. After applying a machine learning algorithm, we found that PoC hosts had a rather crucial influence on the vulnerability exploitation prediction. Moreover, the No PoC host model did not show better prediction performance than the No host model where any hosts were not used at all. Therefore, we conclude that it is better not to use Reference host as a feature. As a future study, we will continue to investigate into the reason why the prediction performance difference between PoC host and No-PoC host models are not huge, and try to find other features that might falsely affect prediction results like PoC hosts. In addition, we will perform more specific prediction research on how exploitation prediction can change according to specific vendors or application fields like embedded systems since their exploitability prediction can be completely different from the prediction of the entire CVEs. [PERSON] and [PERSON] proposed the ideas and designed exploitability prediction model. [PERSON] collected and classified data, and [PERSON] experimented machine learning test. [PERSON] wrote the paper, [PERSON] and [PERSON] reviewed the paper. All authors have read and agreed to the published version of the manuscript. This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government(MSIT) (NRF-2018R1A4A1025632)  
The authors declare no conflict of interest.