Constructing a Knowledge Graph from Textual Descriptions of Software Vulnerabilities in the National Vulnerability Database (2023)

Abstract: Knowledge graphs have shown promise for several cybersecurity tasks, such as vulnerability assessment and threat analysis. In this work, we present a new method for constructing a vulnerability knowledge graph from information in the National Vulnerability Database (NVD). Our approach combines named entity recognition (NER), relation extraction (RE), and entity prediction using a combination of neural models, heuristic rules, and knowledge graph embeddings. We demonstrate how our method helps to fix missing entities in knowledge graphs used for cybersecurity and evaluate the performance.

## introduction an increasing number of services are moving to digital platforms. the software used on these digital platforms is, unfortunately, not without flaws. some of these flaws can be categorized as security vulnerabilities that an attacker can exploit, potentially leading to financial damage or loss of sensitive data for the affected victims. the national vulnerability database (nvd) foot_0 is a database of known vulnerabilities which, as of january 2023, contains more than 200 000 vulnerability records. the common vulnerability and exposures (cve) program foot_1 catalogs publicly disclosed vulnerabilities with an id number, vulnerability description, and links to advisories. nvd fetches the data from cve and provides additional metadata such as weakness type (cwe) and products (cpe). cwes are classes of vulnerabilities (cves), for example, cwe-862: missing authorization contains all cves related to users accessing resources without proper authorization. a cpe is a uri string specifying the product and its version, for example, cpe:2.3:a:limesurvey:limesurvey:5.4.15 is the cpe for the survey app limesurvey with version 5.4.15. keeping the information in the database up to date is important to patch vulnerabilities in a timely manner. unfortunately, patching becomes increasingly difficult as the yearly number of published vulnerabilities increases. 3 to automatically extract relevant information from vulnerability descriptions, named entity recognition (ner) and relation extraction (re) can be applied as shown in fig. 1 . the extracted information can be stored as triples in a knowledge graph (kg). as the extracted triples might be incorrect or missing, knowledge graph embeddings (kge) can be used to learn the latent structures of the graph and predict missing entities or relations. the work described in this paper is based on the master thesis by the first author. we investigate how nlp and kgs can be applied to vulnerability records to predict missing software entities. more specifically, we address the following research question: rq: can our knowledge graph predict vulnerability weakness types and vulnerable products? the contributions of this paper include: (1) an approach for extracting and assessing vulnerability data from nvd; (2) a vulnerability ontology for knowledge graph construction; ## related work we distinguish the ensuing areas of related work: labeling: labeled data may not always be available to train supervised learning models for tasks including ner and re. to address this problem, distant supervision aims at proposing a set of labeling functions for the automatic labeling of data. @xcite applied distant supervision using a cybersecurity corpus. their ap- an arbitrary file upload vulnerability in the plugin manager of v5.4.15 version execute arbitrary code relevant term limesurvey application allows attackers to via a crafted php file. figure 1 : example of a cve with labels proach includes database matching using the cpe vector, regular expressions to identify common phrases related to versioning, for example, "before 2.5", and gazetteers, which are dictionaries of vulnerability-relevant terms, such as "execute arbitrary code". after manual validation of the labeled entities, @xcite report a precision of 0.99 and a recall of 0.78. named entity recognition: training ner models on labeled data are useful as distant supervision depends on assumptions about the input data, which does not always hold. for example, in the case of nvd, if the new data is missing cpe information. machine learning models are not dependent on such metadata, and, as a consequence can generalize better to new situations. @xcite propose ner based on the averaged perceptron (ap). the conventional perceptron updates its weights for every prediction, which can over-weight the final example. the averaged perception keeps a running weighted sum of the obtained feature weights through all training examples and iterations. the final weights are obtained by dividing the weighted sum by the number of iterations. @xcite propose another ner model based on a long short-term memory (lstm) architecture. the authors argue that it can be more useful when the data set has more variation, as the lstm model does not require time-consuming feature engineering. however, their results show it is not able to reach the same level of performance as @xcite . secbert foot_3 is a pre-trained encoder trained on a large corpus of cybersecurity texts. it is based on the bert architecture @xcite and uses a vocabulary specialized for cybersecurity. secbert can be fine-tuned for specific tasks such as ner. another pre-trained encoder similar to secbert is securebert, proposed by @xcite . securebert leverages a customized tokenizer and an approach to alter pre-trained weights. by altering pre-trained weights, secure-bert aims to increase understanding of cyber security texts while reducing the emphasis on general english. relation extraction: relations between named entities can be discovered with re. @xcite propose three re models for vulnerability descriptions from nvd based on lstms. their best-performing model achieves a precision score of 0.92. for labeling the relations, @xcite , applies distant supervision @xcite . @xcite does not manually evaluate their labels before using them in the lstm models; however, the approach is based on @xcite , which indicates 0.82 in precision score after manual validation. both ner and re are important components for constructing knowledge graphs from textual descriptions. we explore several knowledge graphs related to cybersecurity in the next section. ## methods our approach is shown in fig. 2 and gives an overview of the construction of the vulnerability knowledge graph from cve records. we discuss the different steps below. for replication, we share details about the hyperparameter tuning of various models in the appendices. data: our dataset is downloaded in json format from nvd, and the pipeline consists of multiple steps before predicting missing or incorrect labels as the final step. the data set consists of all cve records from 2003 to 2022, which contains approximately 175 000 cves. the cve records are labeled using the distant supervision approach proposed by @xcite . named entity recognition: we train two architectures: averaged perceptron and secbert. averaged perceptron (ap): ap is a featureengineered model, and we use the same features as @xcite due to computational constraints in the ap model, we restricted our training data to 4000 cves. we first replicate their approach and separately trained and evaluated two ap models, one for iob-labeling and one for domain-labeling, using the distant supervision-generated labels. in practice, when a new cve is published, we only have access to the textual description. since the iob labels are input features to the domain model, those must be predicted first. thus, in our second experiment, we again train two ap models, but use the predicted iob labels as input to the domain labeling, instead of the generated labels. secbert: in addition to ap, we use the pretrained secbert model for ner. a significant difference from ap is that secbert jointly extracts iob and domain labels. moreover, as secbert is significantly faster than ap, there is no need to restrict the dataset. we split our data into 60/20/20 for training, evaluation, and testing. relation extraction: for relation extracting, we use an ontology illustrated in fig. 3 , to guide their creation: when two entities of type a and b are detected in a cve, a relation between the two is created if the ontology has an edge between types a and b. note that entities are connected to their corresponding cve-id and cwe-id, and we concatenate multi-word entities based on their iob labels. the vulnerability descriptions are generally written so that vendors are followed by their products which are then followed by their versions. thus, we can derive relations between vendor, product, and version by looking at the word order. we also make relations from relevant terms to the corresponding cve id entity, and through the cve-id the relevant terms are connected to the corresponding vendors, products, and versions. ## results and discussion our empirical evaluation uses the cve dataset discussed in section 3. for replication, the parameters of the best-performing models are in the appendices. ner: ner results are presented in we compare our reproduction results with the results reported by @xcite in tato the best of our knowledge, we can not analytically combine the iob model and domain model results reported by bridges et al.. as such, we rely on our own experimental results, which show that the performance of the fine-tuned secbert model outperforms the ap model.