# CVE2 ATT&CK: BERT-Based Mapping of CVEs to MITRE ATT&CK Techniques
## 3 Results
Moreover, the difference between models is minimal (close to 0) for the techniques where the initial model obtains a better F1-score. \begin{table}
\begin{tabular}{l c c c c} \hline \hline  
Figure 7: Comparison of training loss for the multi-output BERT architecture. (**a**) Without data augmentation; (**b**) With data augmentation. The added gain of the multi-label SciBERT model trained on the augmented dataset resides in its ability to maximize the F1-score for techniques where the initial model performed poorly. One such example is _Forge Web Credentials_. The initial model obtained an F1-score of 0% since both recall and precision were 0%. However, the improved version of the model obtained an F1-score of 66.66%, with a recall of 50% and precision of 100% after data augmentation; similarly, data augmentation tuned the model to predict the _Forge Web Credentials_ technique with 100% precision. Overall, the number of techniques with which the model had difficulty in learning has decreased substantially. Figure 9 shows the correlation between the CVE distribution and the F1-score obtained for the SciBERT models, both using the initial dataset and the one trained after augmentation. The techniques are displayed on both graphs in the same order to indicate how the CVE distribution changed after performing the process of data augmentation and how the adjustments in CVE distribution impacted the F1-score. We observe that not only the techniques initially associated with a small number of CVEs benefited from the augmentation method, but also the techniques associated with a high distribution of samples--for example, the F1-score for the _Command and Scripting Interpreter_ technique increased from the initial 58.92% to 64.12%. Figure 8: Comparing F1-score per technique between SciBERT model trained on initial and augmented dataset. Figure 9: Comparing the F1-score over the CVE distribution for the SciBERT model. (**a**) Without augmentation; (**b**) With augmentation. \begin{table}
\begin{tabular}{l c c c} \hline \hline
**Technique** & **Weighted P** & **Weighted R** & **Weighted** \\ \hline Endpoint Denial of Service & 77.58\% & 83.33\% & 80.35\% \\ Forge Web Credentials & 100.00\% & 50.00\% & 66.66\% \\ Unsecured Credentials & 60.00\% & 75.00\% & 66.66\% \\ Command and Scripting Interpreter & 60.00\% & 68.85\% & 64.12\% \\ Exploitation for Privilege Escalation & 56.45\% & 70.00\% & 62.50\% \\ Adversary-in-the-Middle & 80.00\% & 50.00\% & 61.53\% \\ Brute Force & 100.00\% & 42.85\% & 60.00\% \\ Exploitation for Client Execution & 50.87\% & 50.81\% & 58.49\% \\ User Execution & 58.33\% & 46.67\% & 51.85\% \\ Drive-by Compromise & 64.70\% & 40.74\% & 50.00\% \\ Data Manipulation & 44.44\% & 47.05\% & 45.71\% \\ Exploit Public-Facing Application & 48.27\% & 43.07\% & 45.52\% \\ Hijack Execution Flow & 50.00\% & 37.03\% & 42.55\% \\ Valid Accounts & 41.37\% & 36.36\% & 38.70\% \\ Data from Local System & 41.66\% & 34.48\% & 37.73\% \\ Browser Session Hijacking & 42.85\% & 27.27\% & 33.33\% \\ Phishing & 42.85\% & 27.27\% & 33.33\% \\ Archive Collected Data & 50.00\% & 20.00\% & 28.57\% \\ File and Directory Discovery & 40.00\% & 22.22\% & 28.57\% \\ Server Software Component & 50.00\% & 16.66\% & 25.00\% \\ External Remote Services & 50.00\% & 8.33\% & 14.28\% \\ Process Injection & 25.00\% & 8.33\% & 12.50\% \\ _Exploitation for Defense Evasion (26)_ & 0.00\% & 0.00\% & 0.00\% \\ _Create Account (19)_ & 0.00\% & 0.00\% & 0.00\% \\ _Access Token Manipulation(18)_ & 0.00\% & 0.00\% & 0.00\% \\ _Exploitation of Remote Services (22)_ & 0.00\% & 0.00\% & 0.00\% \\ _Stage Capabilities (22)_ & 0.00\% & 0.00\% & 0.00\% \\ _Abuse Elevation Control Mechanism (44)_ & 0.00\% & 0.00\% & 0.00\% \\ _Exploitation for Credential Access (17)_ & 0.00\% & 0.00\% & 0.00\% \\ _Data Destruction (34)_ & 0.00\% & 0.00\% & 0.00\% \\ _Network Sniffing (18)_ & 0.00\% & 0.00\% & 0.00\% \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Precision, Recall and F1-Scores for the best model.