# CVE2 ATT&CK: BERT-Based Mapping of CVEs to MITRE ATT&CK Techniques
## 4 Discussion
### Error Analysis
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**\#** & **CVE Text** & **True Techniques** & **Predicted Techniques** \\ \hline
1 & Jenkins Publish stores password unencrypted in its global configuration file on the Jenkins controller where it can be viewed by users with access to the Jenkins controller file system. & Unsecured Credentials \\
2 & Due to improper input validation in InfraBox, logs can be modified by an authenticated user. & Valid Accounts, Exploitation for Client Execution & Valid Accounts \\
3 & In Django 2.2 MultiPartParser, UploadFile, and FieldFile allowed directory traversal via uploaded files with suitably crafted file name & File and Directory Discovery, Command and Scripting Interpreter & File and Directory Discovery, Exploit Public-Facing Application \\
4 & Whale browser for iOS before 1.14.0 has an inconsistent user interface issue that allows an attacker to obfuscate the address bar which may lead to address bar spoofing. & Exploitation for Client Execution & Endpoint Denial of Service \\
5 & isula-build before 0.9.5-6 can cause a program crash, when building container images, part of the functions for processing external data do not remove spaces when processing data. & Exploitation for Client Execution & Endpoint Denial of Service \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparing predictions with the true values for the best model. Figure 10: Comparison of word mappings for each technique corresponding to CVE #2 from Table 4. (**a**) Mapping Valid Accounts; (**b**) Mapping Exploitation for Client Execution. Error #4 is analyzed in detail in Figure 11 to observe insights on how the model associates the features. The word _browser_ is highlighted for both the predicted and the correct label. However, the difference resides in the relevance percentage associated with the word for each label, namely 0.45 for _User Execution_ and 0.03 for _Browser Session Hijacking_. While the word _browser_ is recognized as being relevant for both labels, the label with the higher percentage is selected. This finding can be associated with the discrepancy between training examples--240 for _User Execution_, while _Browser Session Hijacking_ has only 102. Thus, the class imbalance affects the model's capability to recognize the real correlation between features and techniques, and leads the model to a biased decision. The model extracts a correct technique for error #5 in Table 4, although it was not among the true labels. As Figure 12 shows, the CVE text description indicates the _Endpoint Denial of Service_ technique, since the word _crash_ is present and the relevance of the word for the _Endpoint Denial of Service_ technique is 0.93. Figure 12 also suggests that the word _crash_ is the only word that has a high impact on the model's decision to label the CVE as _Endpoint Denial of Service_. Two observations can be made based on Figure 12. One is that the model successfully captures a technique overlooked by the reviewer. The technique labeling process is error-prone due to the ambiguity of the CVE text description and also the complexity of the labeling processing given the wide range of available techniques. Second, the model assigns a higher relevance to features that suggest _Endpoint Denial of Service_ even though key features for the _Exploitation for Client Execution_ are identified (i.e., _program_ and _functions_). Table 5 presents the most relevant words when performing feature extraction for each technique. More than 50% of the techniques have the same most relevant feature in common with other techniques in the MITRE ATT&CK Enterprise Matrix. For example, _Exploitation for Privilege Escalation_, _Data from Local System_, _Data Destruction_, _Browser Session Hijacking_, _Archive Collected Data_, and _Create Account_ are all mapped to the same feature. Having the same most relevant extracted feature implies a strong intersection between techniques. This further emphasizes that the separation between labels is fuzzy.