Comprehensive threat analysis and systematic mapping of CVE s to MITRE framework (2024)

Abstract: This research addresses the significance of threat intelligence by presenting a practical approach to generate a labeled dataset for mapping CVEs to MITRE. By linking Common Vulnerabilities and Exposures (CVEs) with the MITRE ATT&CK framework, the paper outlines a scheme that integrates the extensive CVE database with the techniques and tactics of the ATT&CK knowledge base. The core contribution lies in a detailed methodology designed to map CVEs onto corresponding ATT&CK techniques and, in turn, to tactics through a data-driven perspective, centering specifically on the labeling provided by NIST. This procedure enhances our understanding of cybersecurity threats and yields a structured, labeled dataset essential for practical threat analysis. It facilitates and improves the recognition and categorization of cybersecurity threats. Furthermore, the paper analyses the dataset in the context of cyber-threat intelligence. It highlights how vulnerability understanding and awareness have improved over the years through the continuous effort to place vulnerabilities in the context of an attack by linking it to abstract techniques. The dataset allows for a comprehensive cyber attack stage and kill-chain analysis. It serves as a training resource for algorithm development in various use cases, such as threat detection and large language model fine-tuning.

## introduction over 25 years, from 1999 to 2023, the national vulnerability database (nvd) foot_0 maintained by the national institute of standards and technology (nist), has been a critical repository for cybersecurity information. during this extended period, the nvd has played a key role in documenting data on vulnerabilities across various systems, software, and technologies. a new cve is generated each time a security flaw is identified in software or hardware and subsequently reported to the organization. despite their importance to the cybersecurity community, cves often lack specific guidance on countering identified vulnerabilities. this information gap becomes particularly crucial when considering the role of vulnerabilities in unlocking particular attack patterns. as pointed out by @xcite , the timely identification of relevant threats before the attackers exploit is fundamental for proactive defense approaches. sequences of adversarial actions that may evolve into attacks can be identified through multi-step attacks, which can be modeled using the kill-chain concept. this vision consists of ordered phases describing the attacker's progress in achieving objectives @xcite . natural language processing (nlp) and artificial intelligence (ai) can clarify the relationships between entities and events mentioned in text data. by contextualizing this information, these technologies help build a more comprehensive view of cyber threats and the actors behind them @xcite . recently, indications of generative ai in cyber-threat intelligence have emerged @xcite . however, these applications require high-quality and substantial data for effective training to build their knowledge base. the paper aims to establish a reliable foundation for correlating vulnerabilities with techniques and tactics by implementing a well-defined and structured pipeline. the main contributions of this paper are: the resulting dataset extends to threat intelligence, where it aids analysts in identifying potential risks, while also enabling better comprehension of kill chains and the identification of techniques used by adversaries to more effectively defend against attacks. ## background and taxonomy understanding and addressing vulnerabilities is essential to strengthen applications effectively. threat identification uses multiple risk factors to prioritize threats according to their severity by using the multiple risk factors and calculating the threat prioritization value, which represents the severity level of the threat @xcite . however, protecting digital assets from potential threats and attacks is a constant challenge that demands expertise and a comprehensive understanding of the company's environment. as shown in @xcite , it is possible to go from a cve to the related techniques and tactics following the path of cve-cwe-capec-att&ck. before explaining this framework in more detail, we describe each pipeline component. ## dataset creation the main contribution of this paper is the creation of a dataset that links cves to mitre techniques and tactics. the knowledge deriving from cves, cwes, capec and att&ck is fragmented, and the available data are disconnected. it seems that all these organizations are working in sealed environments, missing the bigger picture, to which a vulnerability can be useful to trace back a specific step in the cyber kill chain. for example, cves serve as unique identifiers for publicly recognized cybersecurity vulnerabilities, whereas cwes aim to abstract and categorize cves. although both frameworks have distinct objectives, combining their knowledge allows us to comprehend the wider context. to this purpose, we designed a pipeline to systematically retrieve the tactics and techniques associated with any known cve. this leads to the largest dataset where cves are linked with tactics. other works are proposing ways to achieve the same task as described in section 5, but our approach poses some constraints over the linking from cves to tactics to avoid an exploding surface: we got a ground truth dataset that can be used as a baseline for multiple purposes by relying on entities, e.g., mitre, nist, etc. the final dataset is available online @xcite . the implementation stages are depicted in fig. 1 and can be summarized in the following subsections. ## dataset analysis in addition to making the dataset accessible, we perform an analysis utilizing the insights gathered. initially, we visualize the yearly distribution of disclosed vulnerabilities, as depicted in fig. 2 , totaling 236,071 cves across all years. the visual representation shows an upward trajectory in the annual number of vulnerabilities. this escalating trend implies a continual growth in the overall number of security vulnerabilities discovered and reported over the years. as vulnerabilities proliferate, they contribute to broadening the attack surface, highlighting an increasing array of potential points through which adversaries can exploit weaknesses in systems or applications. this expanding attack surface poses challenges for cybersecurity measures, requiring organizations to adapt and enhance their defenses to address the evolving threat landscape effectively. furthermore, we conduct an in-depth analysis of the mapping from cves to cwes. we visualize the data by plotting the top three cwes identified for each year, as shown in fig. 3 . the findings paint a familiar picture, portraying recurrent patterns in the prevalent vulnerabilities. this observation reveals that certain weaknesses consistently appear to be the leading contributors to security concerns across different time frames. such insights into the recurring cwes aid in understanding persistent challenges, guiding efforts toward targeted mitigation strategies, and reinforcing cybersecurity measures against well-established vulnerabilities. observing the graph, it is clear that cwe-79 (improper neutralization of input during web page generation) has consistently maintained its status as the most prevalent vulnerability over the last six years. this vulnerability manifests when the application fails to neutralize or incorrectly neutralizes user-controllable input before incorporating it into output, which subsequently serves as a web page to other users. an example of cwe-79 is presented as follows: <@xmath0;?> </@xmath1 the web application takes a user-supplied input parameter name from the query string and directly echoes it back into the html response without validation or sanitization. this creates a vulnerability because if an attacker crafts a url such as: http://example.com/welcome.php ? @xmath2, the script tag will be executed when the page is loaded in a victim's browser, leading to a cross-site scripting attack. the other relevant during the timeframe taken into account are: digging deeper into our analysis, we extended our investigation by establishing a mapping between the cwes and the corresponding techniques documented in the capec mapping. this complex mapping allowed us to connect vulnerabilities with specific attack techniques, providing a more comprehensive understanding of the potential exploits associated with each weakness. by bridging the gap between cwes and techniques, we gained valuable insights into how adversaries may leverage identified weaknesses to carry out sophisticated cyber attacks. after analyzing attack techniques (fig. 4 ), we consistently find that "hijack execution flow" emerges as the most frequent technique that attackers can employ. additionally, in the connection between techniques and tactics, where "defense evasion" notably stands out as the most prominent tactic that malicious actors may utilize (fig. 5 ). we want to emphasize that these findings do not represent what malicious actors employ daily to perform attacks. instead, they reveal the potential exploits enabled by vulnerabilities that adversaries may leverage. for visualization purposes, we con-strained the timeframe from 2015 to 2023 inclusive. ## related work comprehensive threat intelligence datasets, especially those focused on vulnerabilities, are crucial for cybersecurity research. understanding and analyzing vulnerabilities are key for fortifying digital systems. quality data is essential for training machine learning models, enabling them to capture intricate patterns in real-world cybersecurity scenarios @xcite . our dataset establishes a foundation that does not depend on external experts for mapping cves to mitre, unlike previous approaches such as the one proposed by @xcite . this aims to provide a more unbiased and objective basis for threat intelligence analysis. this section provides an overview of existing research on threat intelligence datasets. vulnerabilities have been thoroughly examined in previous research, ozment @xcite conducted an in-depth study and analysis of the national vulnerability database (nvd), highlighting various limitations. more recently also, @xcite focuses on a basic analysis of vulnerabilities and scores from the nvd. data sources about vulnerabilities are widespread, and the most used for threat identification mostly come from two datasets, one from enisa (2019) and the other that can be extracted from bron @xcite . ## conclusion as highlighted by @xcite , the labeling process of reports with vulnerability identifiers has thus far been performed manually and has, therefore, suffered from scalability issues due to the shortage of security experts. the versatility of the proposed dataset makes it invaluable for a wide range of applications, showcasing its adaptability and utility across various domains. its applicability extends to threat intelligence, where analysts can leverage the data to enhance their understanding of potential risks and vulnerabilities. the dataset's rich content and diverse sources provide a comprehensive view of the threat landscape, aiding in the identification and mitigation of potential cyber threats. moreover, the dataset is well-suited for killchain concatenation, enabling the mapping and analysis of different stages in a cyber attack. this facilitates a more holistic approach to cybersecurity, allowing practitioners to identify patterns, vulnerabilities, and attack vectors throughout the entire kill chain. this insight is crucial for developing effective defense strategies and proactive measures against evolving cyber threats. as highlighted by @xcite , understanding the attacker's choice of vulnerability for a particular attack stage is a hard problem. in machine learning and artificial intelligence, the dataset is a valuable resource for training models. its extensive nature allows for the development of robust machine-learning algorithms capable of recognizing and predicting patterns within complex data. researchers and developers can refine and enhance the models' language understanding capabilities by exposing language models to a broad range of scenarios and contexts present in the dataset.