CTIKG: LLM-Powered Knowledge Graph Construction from Cyber Threat Intelligence

Authors: liangyi  huang
xusheng  xiao
Venue: colmweb_org_COLM_2024_Conference
Published: 2024-03-14 06:59:50

Abstract: To gain visibility into evolving threat landscape, knowledge of cyber threats has been aggressively collected across organizations and is often shared through Cyber Threat Intelligence (CTI). While knowledge of CTI can be shared via structured format such as Indicators of Compromise (IOC), articles in technical blogs and posts in forums (referred to as CTI articles) provide more comprehensive descriptions of the observed real-world at- tacks. However, existing works can only analyze standard texts from mainstream cyber threat knowledge bases such as CVE and NVD, and lack of the capability to link multiple CTI articles to uncover the relationships among security-related entities such as vulnerabilities. In this paper, we propose a novel approach, CTIKG, that utilizes prompt engineering to efficiently build a security-oriented knowledge graph from CTI articles based on LLMs. To mitigate the challenges of LLMs in randomness, hallucinations and tokens limitation, CTIKG divides an article into segments and employs multiple LLM agents with dual memory design to (1) process each text segment separately and (2) summarize the results of the text segments to generate more accurate results. We evaluate CTIKG on two representative benchmarks built from real world CTI articles, and the results show that CTIKG achieves 86.88% precision in building security-oriented knowledge graphs, achieving at least 30% improvements over the state-of-the-art techniques. We also demonstrate that the retry mechanism makes open source language models outperform GPT4 for building knowledge graphs.

Main paper: 
 introduction: 
 In recent years, advanced cyber attacks often exploits multiple vulnerabilities and progressively penetrates the enterprise network to compromise security and confidentiality, such as the I2CE3 ransomware attack chain (Keshavarzi & Ghaffary, 2020) . To effectively counter these attacks, knowledge of cyber threats has been aggressively collected across organizations and is often shared through Cyber Threat Intelligence (CTI), which models cyber threats including their actors, tactics and techniques McMillan (2013) ; Wagner et al. (2019) . CTI has been primarily collected in the form of Indicators of Compromise (IOCs) Obrst et al. (2012) ; Liao et al. (2016) , which are forensic artifacts of an intrusion such as malicious file/process names and IPs/domains of botnets. For example, a variety of security websites have published their own IOC reports of the Log4J vulnerability (Kim, 2022; ElastZris, 2022; Azure, 2023) . Automated security detection solutions can also benefit from IOCs. For instance, the Microsoft Malicious Software Removal Tool (MSERT) employed relevant IOCs following the cyber attack on the Chilean bank regulator (Onyegbula, 2023) . Additionally, CrowdStrike Falcon, Microsoft Sentinel and Microsoft Defender XDR integrate IOC as a component of their detection systems (CrowdStrike, 2023; Microsoft, 2023) .Besides structured formats like IOCs, recent studies Liao et al. (2016) ; Dong et al. (2019) show that the information of cyber threat delivered by traditional blacklists Group; MX is rather thin, which covers only a limited set of IOC classes (URL, domain, IP and MD5) and cannot reveal the relations between IOCs. Instead, descriptions of cyber threats, such as articles in technical blogs and posts in forums, referred to as CTI articles, are more favorable for security practitioners since they provide more comprehensive descriptions of the observed real-world attacks. For example, Figure 1 shows three CTI articles that describe the malware Bedep. Based on these articles, Bedep has relationship with Angler and Magnitude, and performs click-fraud, which can not be found in professional databases. Thus, existing techniques Obrst et al. (2012) ; Liao et al. (2016) ; Catakoglu et al. (2016) ; Dong et al. (2019) that focus on extracting IoCs is insufficient to represent the cyber threat knowledge in CTI since they overlook the IOCs' behaviors and their relationships. To enhance the representation of the knowledge accumulated in CTI, a specialized knowledge graph that depicts domainspecific knowledge through entity-entity relationships, which has been widely used among large enterprises for data management (Yan et al., 2018) and Natural Language Processing (NLP) tasks (Liu et al., 2021) , can be employed. Constructing a knowledge graph out of CTI articles entails automatic techniques for extracting triples from the sentences within these articles, and linking these triples to form a graph. Although existing works based on pre-trained language models show promising results in extracting triples from general purpose texts, they show limited capabilities in dealing with the complex domain-specific sentence in these CTI articles (Satvat et al., 2021; Rossiello et al., 2023) . Fortunately, large language models (LLM) have recently been shown to perform much better in NLP tasks such as text comprehension, triple extraction, coreference resolution, and context reading (OpenAI, 2023a; Brown et al., 2020; 01.AI, 2023; 01-ai, 2023; Achiam et al., 2023) , and thus becomes a potential solution to analyze the complex entities and relationships from CTI articles.Goal. Recognizing the importance of CTI and potential of LLMs, we aim to build a novel LLM approach, CTIKG, that effectively analyzes descriptions of security-related entities in CTI articles to build a knowledge graph and reveals relationships and behaviors among security-related entities across CTI articles. In particular, the nodes in the knowledge graph represent securityrelated entities and the edges in the graph represent the relationships among these securityrelated entities across different CTI articles.Challenges. We encounter several key challenges due to the limitation, hallucinations and inaccurate comprehension of LLMs:• Token Limitation: Since some CTI articles have long contexts, and LLM prompts need to include chain of thoughts with examples, LLMs cannot process an entire CTI article at once due to the token limitation, and may miss important CTI information. • Incorrect Triple Format: Even explicitly specified in the prompts, LLMs may not follow the required formats, and will produce extracted information as tuples containing more than two elements. • Incorrect Output: LLMs may extract inaccurate information from the given text, such as the information from the given example but not the given text or wrong triple information due to misunderstanding the text. • Misunderstood Task: LLMs may misunderstand the task as the task for generating subsequent text of the input CTI article and do not produce triples.To address these challenges, CTIKG performs text segmentation on CTI articles and employs multiple LLM agents with dual memory design to process text segments separately (results stored in short-term memory) and summarize results to produce final results (results refined in the long-term memory). In this way, CTIKG can effectively address the token limitation and use long chain of thoughts to guide the LLM agents to extract triples from CTI articles in every text segment. Moreover, CTIKG employs multiple LLM agents to Evaluations. We evaluate CTIKG on three benchmarks constructed from 500+ real world CTI articles. For triple extraction from 255 sentence cross 13 cyber attack tactics defined by ATT&CK knowledge base (Corporation, 2022), CTIKG achieves 91.89% precision and 89.39% recall, which is better than Extractor (Satvat et al., 2021) , REBEL (Cabot & Navigli, 2021) and KnowGL (Rossiello et al., 2023) by more than 10%. For knowledge graph construction from 30 CTI articles, CTIKG achieves 86.88% precision and 70.86% recall, which is at least 30% better than these three approaches in both precision and recall. We also evaluate CTIKG on 478 correlated CTI articles that describe 15 CVE entities, and show that CTIKG can averagely find 39.2 entity behaviors for each CVE entity by connecting a CTI article to 14.6 correlated CTI articles, compared to 8.6 entity behaviors that can be found by analyzing a single CTI article. We also demonstrate that local LLM models (01-ai, 2023 ) deployed on single GPU can outperform GPT-4 through prompt engineering, LLM agents cooperation, and chain of thoughts. Our implementation of CTIKG and the evaluation datasets are available at our project website (CTIKG, 2024) . We also provide fine-tuned models at our project website (Research, 2024a) . The latest model is currently based on Qwen1.5-70B (Research, 2024b) . 
overview: 
 Figure 2 shows the architecture of CTIKG, consisting of three phases:• Short-term Memory Construction. CTIKG deploys multiple LLM agents to extract the knowledge within a CTI article segment in the form of triples. This phase employs four types of LLM agents: worker, integrator, refiner, and checker. The worker agents extract the knowledge from the natural language text to form the triples. The integrator agent combines the multiple results on the same text from multiple workers. The refiner agent achieves the normalization and resolution tasks on the combined triples. The checker agent inspects the triples and instructs the other agents to re-work when it detects common errors of the LLMs. • Long-term Memory Construction. As CTIKG uses segment processing to overcome LLMs' token limitation, the merger agent is responsible for assembling the triples from different article segments into one long-term memory.• Knowledge Graph Construction. CTIKG builds a knowledge graph using multiple articles.To ensure consistency across different articles for the same entity (e.g., a specific malware), CTIKG employs another merger agent to achieve a unified entity description. 
design of triple extraction: 
 CTIKG extracts knowledge from article as triples in the form of (subject, relation, object) , where subject and object are entities and relation is their relation (e.g., (Linux, is, Operating System)).Inspired by existing LLM studies (Park et al., 2023) , CTIKG divides the article into segments and extracts the triples from each segment as the short-term memory. Meanwhile, CTIKG maintains a list of all extracted triples from different segments as the long-term memory. Figure 3 shows the algorithm for building a knowledge graph with these two memories.Text Segmentation. To segment a CTI article, CTIKG first divides it into paragraphs at line breaks and merges these paragraphs into segments. For paragraphs exceeding 600 characters, CTIKG uses NLTK (Loper & Bird, 2002) to further split the paragraphs into sentences and reassemble the sentences to form segments. The rationale behind such text segmentation is shown in the Appendix A.Entity Types and Relationships. Unlike the ontology-based knowledge graph, which can only extract predefined relationships and entity types, CTIKG is designed to dynamically capture all types of relationships related to any cyber threat in CTI articles. As shown in the Appendix G.3, the similarity between CVE-2012-0158 and CVE-2017-1188 was discovered based on the entity relationship "for" and "memory corruption vulnerability". Our further investigation revealed that Magniber and Cerber have a "mutual ransomware payload" relationship, and Emotet and Trickbot have a "switch to" relationship. These entity relationships are determined by the author's writing style and cannot be anticipated in advance and defined in the ontology.To construct the short-term memory of a text segment, CTIKG leverages four types of LLM agents: worker, integrator, refiner, and checker. The prompts for these agents can be found at Appendix B.Worker Agent. Worker agents, told as experts in information extraction and computing, are tasked with extracting security-related data from texts (Lines 4-7). In particular, to instruct LLMs to focus on security-related data, we specify "Only extract triples that are related to cyber attacks. If a sentence does not contain any triple about cyber attacks, skip the sentence and do not include it in your output" and "Focus on malware, Trojan horses, CVEs, or hacking organizations as the subjects of the triples" in the prompts of worker agents, and provide multiple security-related examples. CTIKG deploys three worker agents with the same prompt and different temperature settings (1, 0.5, and 0.2). If at least two worker agents identify no cyber security-related triples in a segment, CTIKG skips the other agents and returns an empty result for that segment directly (Line 11). Specifically, the prompt for the worker agents contain the following parts:• Background: An introduction that defines the task of triple extraction with examples.• Rules: An instruction that asks the worker agent to extract only entity relations relevant to cyber attacks, and provides output format requirements. • Few-shots: Three complete chat conversations, showing example inputs and responses.• Input: An instruction that asks the worker agent to perform triple extraction on a given text.Integrator Agent. The integrator agent extracts a final triple from the worker agents' results (Line 8). Due to the randomness of LLM (Lee et al., 2022) , multiple workers may use Specifically, the prompt for the integrator agent contains the following parts:• Background: An introduction that defines the task and provides examples to illustrate which triples are assumed to have the identical meanings. • Rules: An instruction that asks the integrator agent to integrate the triples with identical meanings, and provides expected input and output format. • Few-shots: A description that consists of three complete chat conversations, showing how an integrator successfully does its job with the right outputs. • Input: An instruction that provides the outputs from the three worker agents as the input for the integrator agent to process.Refiner Agent. The refiner performs the normalization and resolution tasks (Line 9) (Satvat et al., 2021) , and leverages chain of thoughts (Wei et al., 2022) for complex task. The refiner agent takes the triples produced by the integrator agent, and performs the following tasks:• Simplification: Some triple's subject and object parts have modifiers or unnecessary prefixes. • Splitting Complex Triples: Some parts of the triples may contain conjunctions.• IOC Conversion: Some phases may contain IOCs and should be highlighted in the triple.• Coreference Resolution: The refiner agent will replace the implicit references in a triple with the actual name according to the context. • Normalization: The refiner agent will standardize the relation in triples.Checker Agent. Although prompts emphasize requirement several times, LLMs may still output the incorrect results. Through empirical observations of the incorrect outputs, we summarize some common patterns. The checker agent of CTIKG checks whether the outputs from other agents based on the given rules and determines whether the output is correct based on the summarized patterns of incorrect outputs (Line 12). If the output of an LLM agent is incorrect, CTIKG then instructs the LLM agent to re-perform the task. Due to resource constraints, we limit such retry to be up to three times.Figure 4 : Algorithm for multiple CTI articles CTIKG uses the merger agent to further refine short-term memory triples to fit the long-term memory (Line 16). After refinement and check, CTIKG directly adds the triples into the long-term memory (Line 18). The merger agent determines whether the triples' subjects or objects in short-term and longterm memory refer to the same entities. It then modifies the corresponding subject or object in the short-term memory triple to unify representations of identical entities. Since these tasks require complex reasoning, the merger also leverages chain of thoughts to improve effectiveness (Wei et al., 2022) . Specifically, the merger agent performs the following tasks:• Unifying Terms: Subjects and objects that contain partially identical terms and refer to the same entities are unified by removing prefixes, suffixes, or modifiers. • Modifier Removal: Specific names, such as those of malware or hacker organizations, should have all unnecessary modifiers removed. • Hallucination Restriction: The creation of triples through LLM's imagination is strictly prohibited.Based on the long-term memory of each CTI article, CTIKG builds a knowledge graph using the subjects and the objects as nodes and the relations as edges. Figure 4 shows the algorithm for knowledge graph construction. Specifically, CTIKG use triples from first article to build the a basic graph, then gradually incorporates triples from other CTI articles.To maintain consistency of entity names across articles, CTIKG leverages another merger agent. CTIKG first uses the RoBERTa model function (Liu et al., 2019) to achieving the text embedding of subject and object (Line 5). For each triple to be added, CTIKG searches five most similar nodes in the knowledge graph based on the embedding vector similarity (Line 6). These five node names are used by the merger agent to refine the triple (Line 7). Finally, CTIKG connects the refined triple to the knowledge graph (Line 8). 
evaluation: 
 In the evaluations, we aim to answer the following research questions:• RQ1: How effectively can CTIKG extract triples from CTI articles?• RQ2: How effectively can CTIKG generate knowledge graphs for CTI articles?• RQ3: How effectively can CTIKG reveal relationships of security-related entities from correlated CTI articles?Implementation. We implement CTIKG ( 20K characters prompts) on a Ubuntu 20.04.6 server with AMD 5955WX CPU, RTX 6000 GPU and 128GB memory upon VLLM, OpenAI, Networkx, Stanford NLP tool suite, and NLTK (Kwon et al., 2023; OpenAI, 2023b; developers, 2023; Lab, 2003; Loper & Bird, 2002) . We choose Yi-34B-4bits model as the core LLM model. We provide more details on LLM selection in Appendix D Evaluation Subject. We curate a large dataset with 72, 538 CTI articles from security-related websites and cyber threat knowledge base, and construct three benchmarks:• RQ1 Sentence Benchmark: For each enterprise tactic defined by the Mitre ATT&CK (Corporation, 2022), we select over 20 representative sentences from the CTI articles, resulting in 255 CTI sentences with 699 manually extracted triples. • RQ2 CTI Article Benchmark: We category CTI articles into three types: standalone (highlighting a single cyber threat), chain (introducing multiple interlinked cyber threats), and overview (introducing multiple cyber threats that are not directly related) categories. Ten articles were selected from each category for manual analysis, resulting in 30 knowledge graphs with a total of 315 edges. We provide more details of the dataset in Appendix E.To evaluate CTIKG's triple extraction, we compare CTIKG with three baseline approaches and two variants of CTIKG powered by GPT-4 (Achiam et al., 2023) and GPT-3.5 (OpenAI, 2023a) . The evaluation is conducted on the triple extraction benchmark constructed from the CTI article dataset.Baseline Approaches. We compare CTIKG with two state-of-the-art entity relation extraction approaches: REBEL (Cabot & Navigli, 2021) and KnowGL (Rossiello et al., 2023) , both of which are transformer-based seq2seq models for generic text. We also compare CTIKG with a secure text-oriented approach, Extractor (Satvat et al., 2021) , which are based on BERT (Devlin et al., 2018) and BiLSTM (Hameed & Garcia-Zapirain, 2020) . We also use two variants of CTIKG (CTIKG based on GPT-4 and GPT-3.5) to evaluate the impact of the underlying LLM models. These two variants of CTIKG use the same prompts, except for the checker agent and the corresponding retry mechanism. This is due to the fact that OpenAI only provides the paid API, and applying the retry mechanism will result in significant expenses.Table 1 shows the precision and the recall of CTIKG and the baseline approaches. Overall, CTIKG with the YI model achieves the best performance with 91.89% precision and 89.39% recall, and CTIKG with GPT-4 achieves the second best performance: 84.26% precision and 83.66% recall. The precision and the recall of CTIKG with GPT-3.5 are only about 77.88%. Meanwhile, two approaches for general texts, REBEL and KnowGL, completely fail the task of triple extraction on the security-related sentences, achieving both precision and recall below 21%. Extractor achieves a slightly worse performance with 82.56% precision and 79.35% recall. For specific cyber attack tactics, CTIKG achieves the best performance (> 94%) for the tactics of Persistence, Discovery, and Lateral Movement, and worst for the tactics of Impact (about 84%). The second best approach, CTIKG with GPT-4, has a larger variant for different tactics, achieving only 71.59% for the tactics of Initial Access.While GPT-4 is a better-trained model with top MMLU scores, thus CTIKG with GPT-4 suffers from the randomness and hallucination due to lacking of checker agent and retry mechanism. CTIKG with GPT-3.5 has the lowest MMLU score model, and lacking the checker agent and retry mechanism, achieves the worst results among these three versions of CTIKG. While Extractor is competitive with CTIKG with GPT-4 in terms of accuracy due to its well-designed internal components and multi-step pipeline, its text comprehension capability is less powerful than that of LLMs. When processing sentences like "The first thread is responsible for finding the CHD in the process, writing the results to a file, and preparing the files for exfiltration", Extractor cannot find the relationship between the first thread and the file. This makes its recall lower than CTIKG with YI and GPT-4. Error Analysis. Since LLMs are essentially text generation models, the next word generated is dominated by probability (OpenAI, 2023a). We summarize some LLM-specific errors in triple extraction as follows:• Hallucination: Hallucination is the most common problem. For example, for sentence "Both apps upload users phone book to remote server and use it for SMS spam", GPT-4 generates new non-existent sentence and incorrect triple ⟨ AdwareY, is linked to, GhostNet⟩. • Misunderstood Task: Another common error is misunderstood task. For example, for the sentence "This image file exists on the third page of the document, so the user would have to scroll down in the document to this third page to get the SWF file to run", CTIKG with GPT-4 may outputs the incorrect triple ⟨ Formbook, is, malware⟩ based on the few-shot example. 13.42% and 6.95% of the outputs of CTIKG with GPT-3.5 and GPT-4, respectively, incorrectly use few-shot sample text due to lack of checker agent. • Text Comprehension Ability: The level of text comprehension ability has a great impact on the outputs. CTIKG with GPT-3.5 often misunderstands the content, while CTIKG with the YI model and GPT-4 are much less likely to do so. For the sentence "Octopus uses wmic for local discovery information.", only CTIKG with GPT-3.5 produced the incorrect result ⟨ Octopus, uses, information⟩.Recent studies show that evaluations using LLMs can match or surpass the accuracy of manual inspection (McAleese et al., 2024; Zheng et al., 2024; Kenton et al., 2024) . In addition to manual inspection, we use an automatic evaluator based on GPT-4, which has an accuracy rate of 93%, to evaluate the correctness of triples and categorize their error types using a few-shot learning and debating mechanism. The results demonstrate that CTIKG, when utilizing the YI model and checker agent, extracts fewer hallucinated triples compared to CTIKG based on GPT-4 and GPT-3.5. This finding demonstrates that the checker agent effectively filters out hallucinated triples. Additionally, the number of triples with incorrect relationships is lower than that produced by CTIKG using the other two LLM models. Hallucinations often cause other triples within the same sentence to be incorrect. Consequently, the checker, despite not being specifically designed to address such errors, can reduce the number of incorrect triples by filtering out hallucinated triples. The detailed figure is shown in Appendix F.2.Impacts of Sentence Complexity. We measure the complexity of a sentence using the number of ground truth triples and word count. As shown in Figure 6 , with the complexity increases, the precision of CTIKG with the YI model improves while its recall decreases, and thus the overall effectiveness does not change much. It is mainly due to the check agent that leverages retry to minimize errors. CTIKG with GPT-4 achieves similar performance with the complexity increases but has a larger variance due to the randomness of GPT-4. Both Extractor and CTIKG with GPT-3.5 achieve worse results when the complexity increases.Although the syntax of these sentences is relatively simple, it requires Extractor to be able to detect a relationship between two entities far apart. For example, Extractor fails to extract triples from the sentence "The overall purpose of Cannon is to use several email accounts to send system data (system information and screenshot) to the threat actors and to ultimately obtain a payload from an email from the actors". CTIKG with the YI model and CTIKG with GPT-4 have better context reading capabilities, which allows them to find these relationships correctly. Figure 7 shows a similar trend for Extractor: with the increase of the words in a sentence, Extractor performs worse. Overall, CTIKG with the YI model and CTIKG with GPT-4 achieve the most robust results for sentences of different complexity. We notice that the performance of CTIKG with GPT-4 does not degrade as the number of triples or the number of words within a sentence increases. This is mainly due to GPT-4's larger token limit, which is 131, 072 tokens, larger than GPT-3.5 and the YI model's default 4, 096 tokens.Entity Types and Relationships. We have manually 699 cyber security-related triples of the RQ1 sentence benchmark, and fount out that 51.96% of nodes cannot be categorized into common entity types such as malware names, and 54.93% of edges cannot be categorized into computer science-related relationship types. These results show that most of the entities and their relationships cannot be easily anticipated in advance and defined in the ontology.To evaluate CTIKG's knowledge graph construction, we compare CTIKG with the three baselines: Extractor (Satvat et al., 2021) , REBEL (Cabot & Navigli, 2021), and KnowGL (Rossiello et al., 2023) on the knowledge graph benchmark. Since we demonstrated that CTIKG performs better than CTIKG with GPT-4 and GPT-3, we no longer include them in RQ2.As shown in Table 2 , CTIKG achieves the best precision (86.88%) and recall (70.86%), which is at least 73% better than the second best approach, Extractor, in both precision and recall. Extractor achieves a precision of 55.93% and a worse recall of 31.32%. Both REBEL and KnowGL perform poorly on these security-related CTI articles, with precision and recall below 35%. This is consistent with their triple extraction effectiveness since inaccurate triple extraction will cause the built knowledge graph to be even more inaccurate.Across article types, CTIKG performs best with standalone articles and least effectively with chain articles. This variance is due to the complexity of chain articles, which detail multiple interrelated threat entities, making them more difficult to analyze than standalone articles, which focus on a single cyber threat and maintain cohesion on a topic. Despite these challenges, CTIKG's precision and recall significantly outperform baseline methods, with improvements of at least 24.8%. We provide more details on the impacts of article type in Appendix C.Error Analysis. By inspecting the final results of CTIKG, and the intermediate results of the short-term memories and the long-term memories, we found that besides those common errors observed in triple extraction, CTIKG also suffered from the following problems:• Length limit: Although the LLM context length limit is extended to 16, 384 tokens, CTIKG may still need more space during the chain of thoughts process (Wei et al., 2022) . If there are too many triples in the short-term memory, the merger agent will reach the length limit when performing the chain of thoughts and output unfinished results. This limitation cannot be easily addressed by performing retry. • Incorrect Coreference Resolution: If an article segment uses a demonstrative pronoun instead of a specific name throughout the text, the merge agent may incorrectly replace those pronouns with another specific name from the long-term memory. This error can be mitigated by recording the original sentence for each triple, but this method will make the problem of length limit worse.In this RQ, we compare the knowledge graph constructed from correlated CTI articles with the knowledge graph constructed from a single CTI article, and use the number of edges connected to a specific node in the knowledge graph as a metric to represent the number of that entity's behaviors described within the graph. Appendix G shows the detailed effectiveness data for each CVE, and a corresponding case study. 
discussion: 
 Retrieval Augmented Generation. We used Retrieval Augmented Generation (RAG) (Jeong, 2023) to dynamically generate examples for each prompt. However, since LLM sometimes uses these examples as part of the input, we cannot predict the wrong output that may occur when hallucinations occur when using RAG. Currently, we use fixed examples to eliminate the hallucination phenomenon.The current CTIKG based on YI-32B processes an article in 5 minutes on dual RTX A6000. The newly released model, LLaMA3-8B, has shown better performance with a smaller model size according to LMSYS Chatbot Arena Leaderboard (Chiang et al., 2024) . Since CTIKG can seamlessly switch to any LLM model as the backend, tool with LLaMA3-8B takes only 50 seconds to process an article. In the long run, the CTIKG with improved LLMs will achieve better performance with more advanced GPUs.Other Domains While it is not within our current focused scope, in future work, we plan to verify the effectiveness of CTIKG with adaptation to other domain knowledge in other domains like SciERC (Luan et al., 2018) . 
related work: 
 Knowledge Graph Construction. Nidhi et al. Rastogi et al. (2020) propose an ontology to generate knowledge graphs for malwares. Aritran et al. Piplai et al. (2020)  
conclusion: 
 We have proposed CTIKG, which employs multiple LLM agents and dual memory design to build a knowledge graph from CTI articles with high effectiveness. CTIKG divides a long CTI article into text segments and processes each text segment separately using multiple LLM aganets with different temperature settings for mitigating the randomness of LLMs. CTIKG then summarize the results of the text segments to generate more accurate results. Our evaluations on two representative benchmarks derived from real world CTI articles demonstrate the superiority of CTIKG over the state-of-the-art approaches. 
 
 Appendix: We used a data-driven method to set the text segment size, analyzing 13, 385 CTI-related sentences with an average length of 141 characters. Open source LLMs, such as the YI model (01-ai, 2023) , accurately handle up to 600 characters or about 4 sentences. Hence, we limit segments to 600 characters. User: "You are responsible for combining the three different entity extraction results from three different assistants extracting from the same sentence into one. The final results is a python list format. When you encounter multiple triples with the same meaning but slightly different words, choose the one that expresses it most succinctly and do not add the others to your final result. Just answer me the final python dictionary with triple format without any other words."User: "You play the role of an entity extraction expert and modify/simplify/split the text (extracted multiple triples) in the entity extraction result I gave you. according to the following rules. A triple is a basic data structure used to represent knowledge graphs, which are structured semantic knowledge bases that describe concepts and their relationships in the physical world. Here are all rules you must follow."Rule 1: "If the subject or object in a triple contains pronouns such as it, they, malware, trojan, attacker, ransomware, or group, replace them with a specific name as much as possible according to the context."Rule 2: "Focus on malware, Trojan, CVE, or hacking organization or as the subject of the triples, if a subject with malware or Trojan or CVE or hacking organization is found and has additional suffixes, remove the suffixes."Rule 3: "Split a complex triple into multiple simpler forms. For example, [Formbook and XLoader, are, malware] should be split into [Formbook, is, malware] and [XLoader,is, malware] Explanation: The words the Formbook and Formbook sample refer to the same entity, so they are unified to use the exact same subject Formbook for consistency.Rule 2: "Be especially careful that when you meet specific names of malware,CVE, Trojans, hacker organizations, etc., always use their specific names and remove the prefixes/suffixes/modifiers."Before rule: [Malware Formbook, is, malware] After rule: [Formbook, is, malware] Explanation: The word Formbook is a specific name of malware, so it should be used as the subject of the triple and the prefix Malware should be removed. -The end of the new short-term memory area-"User: "Good. Now, lets switch to another article.-The start of the long-term memory area-[Input Text]-The end of the long-term memory area--The start of the short-term memory area--The end of the short-term memory area-Now, follow the rules. Write down how you use the rule to modify the triples in short-term memory. Then, write down new short-term memory which must be started with -The start of the new short-term memory area-and ended with -The end of the new short-term memory area-" We evaluate the performance of CTIKG on different article types. In general, the standalone type is the simplest, where the author describes only one security threat, such as a malware or a CVE, in an article. As a result CTIKG maintains a high recall. The reason for the lower recall is that some authors tend to mention the specific name of the threat only at the beginning of the article, and use the indicative pronouns to refer to it in all other parts of the article. For example, the article "Unix-privesc-check -Unix/Linux User Privilege Escalation Scanner" Darknet (2015) mentions the name Unix-privesc-check only twice in the entire article. For CTIKG, even if the specific name of it is not in the short-term memory, it still obtains its name and completes the knowledge graph by retrieving the name from the long-term memory. As for Extractor, even though it also performs coreference resolution, quite a number of edges in the knowledge graph still link to "it" node.The overview article tends to describe different cyber threats in different parts. For example, the article "A Brief History of Malware: Part Three (1993-1999)" Crawley ( 2016) describes different malware in different paragraphs. For this type of article, CTIKG completes the construction of the whole article's knowledge graph through the direct stacking of the short-term memories from article segments.For both CTIKG and Extractor, the chain type is definitely the most challenging to process. This type of articles often describe how a malicious actor or team utilizes multiple tools to reach an intrusion and complete the final task, often stealing data or ransomware. For example, the article "IR Team Investigations Uncover eCrime Use of Nation-State Attack Methods" Etheridge (2018) describes how an attacker can start an intrusion at the beginning and maintain the persistence mechanism to eventually install ransomware on the target computer by using multiple attack methods. For each attack method, the article provides some information that can be extracted as triples. When the long-term memory contains a large number of information about different cyber threats, the merger agent of CTIKG is more likely to produce inaccurate results, making the final result inferior to the result of the other two types.We prefer open source LLMs that can be deployed on our own server. We finally chose the YI 34B model (01-ai, 2023 ) that achieved top scores on the Alpaca Eval Leaderboard (Li et al.) and the MMLU benchmark (Hendrycks et al., 2020) . We also attempted to use LLaMa 2 (Touvron et al., 2023) . However, in early experiments, even the largest LLaMa 2 70B model cannot strictly follow CTIKG's prompt instructions, and result in an accuracy of merely 30%. This phenomenon is consistent with its lower score on the MMLU benchmark, and thus CTIKG uses only the YI 34B model for CTIKG.We collect 88, 131 CTI articles from 67 websites and 672 articles from the ATT&CK knowledge base (Corporation, 2022) . Since some articles contain very short descriptions, such as just listing URLs or file addresses, we further apply NLP tools to filter out them and obtain 72, 538 meaningful articles. These articles have average 52 sentences, where each sentence has an average of 15 words or 100 characters (Figure 5 ). With CTIKG capacity to process 4 sentences at a time, most articles require processing in 10 segments. 10 show the accuracy of the models for different text complexity. In the graph, there are 6301 nodes related to cyber security topics, categorized under "Attack Vector And Techniques", "Malware Name" and "CVE". Additionally, there are 4361 nodes related to other computer science topics, which include "Software/Tools", "File Type", "Domain Names", "Hashes", "IP Addresses" and "Email Addresses". The remaining 16365 nodes are classified under "Other", "Organization Name" and "Geographic Region". Overall, 39.45% of the nodes are related to computer science topics.. As for the edges, the graph includes 562 edges related to cyber security topics, 1355 edges related to other computer science topics, and a total of 1774 general edges. Overall, 51.94% of the edges pertain to computer science topics. Table 3 shows the effectiveness of cyber threat revelation specifically for each CVE.  


Decision: The paper is a solid investigation of the application of LLMs to a realistic information extraction setup, dealing with issues such as hallucinations and context limitations. The evaluation is thorough. The reviewers agreed on these points and the authors addressed their questions in their rebuttals adequately.

Reviews: review: paper_summary: paper_summary: The paper presents an approach using LLMs to automate the process of constructing a knowledge graph of cyber-security-related entities from text documents related to cyber threat intelligence. The approach is evaluated on a new, manually curated dataset of articles extracted from security-related websites and knowledge bases.
strength_weakness: strength_weakness: - the scientific novelty of the approach is limited to the use of existing techniques for the use of LLMs in information extraction.
- as the paper focuses on the construction of knowledge graphs for the field of cyber-threat intelligence, it is unfortunate that the authors never actually describe in detail the types of entities and relations that they seek to extract.This should, in my view, feature much more prominently in the paper. 
- the paper should include more information about the dataset of CTI articles employed for the evaluation. How was the annotation performed? If several annotators were involved, did you compute their inter-annotator agreement?
questions: questions: I'm not sure I understood correctly the reason for not testing parameter-efficient fine-tuning methods such as LoRA. On page 9, the authors mention the inability of loading multiple LoRA in VLLM. But why would you need to load multiple LoRA simultaneously? Couldn't you (at least for the purpose of your experiments) perform each task one at a time, each time storing intermediate results ?
paper_summary: paper_summary: The paper introduces a novel approach, CTIKG, for constructing a security-oriented knowledge graph from Cyber Threat Intelligence (CTI) articles. The authors address the limitations of existing work and propose the use of LLMs to overcome these challenges. CTIKG employs prompt engineering and multiple LLM agents with a dual memory design to efficiently extract triples from the articles and generate results. The effectiveness of CTIKG is evaluated on three benchmarks constructed from real-world CTI articles, demonstrating significant improvements over state-of-the-art techniques.
strength_weakness: strength_weakness: Limited discussion on generalizability. The authors focus solely on cybersecurity-related text analysis using LLMs, and it would be beneficial to discuss potential applications or extensions of their approach beyond this specific domain.
questions: questions: How scalable is your approach when working with larger-scale datasets or real-time applications? Did you encounter any performance issues during evaluation?
paper_summary: paper_summary: In this paper, the authors proposed a system that utilizes multiple LLM agents and a dual memory design to efficiently construct a knowledge graph from CTI articles. This advanced system breaks down lengthy CTI articles into smaller text segments, which are then processed individually by various LLM agents operating at different temperature settings. This approach serves to minimize the inherent randomness of LLMs. Following this segmented processing, the CTIKG summarizes the outcomes to produce more precise results. Evaluations conducted on three benchmarks, derived from actual CTI articles, have shown that CTIKG outperforms other cutting-edge methods.

Strengths:

1.The proposed multi-agent collaborative method for constructing knowledge graphs, as outlined in the paper, possesses practical value, particularly in the realm of knowledge representation and reasoning;

2.The paper is very well written with clarity, easy-to-follow logic, and adequate discussions;

3.After conducting extensive computational experiments, the results clearly demonstrated the effectiveness of the method.

Weaknesses:

1.The depth and breadth of the research conducted on knowledge graph construction(section 6) is disappointingly insufficient, lacking substantial content and comprehensive analysis;
2.In the process of collaboratively constructing the knowledge graph, for intelligent agents playing different roles, what are their contribution rates? Analysis results of this part of the content also need to be given.
strength_weakness: strength_weakness: 1.The depth and breadth of the research conducted on knowledge graph construction (section 6) is disappointingly insufficient, lacking substantial content and comprehensive analysis;
2.In the process of collaboratively constructing the knowledge graph, for intelligent agents playing different roles, what are their contribution rates? Analysis results of this part of the content also need to be given.
questions: questions: In the process of collaboratively constructing the knowledge graph, for intelligent agents playing different roles, what are their contribution rates?
paper_summary: paper_summary: This paper presents an LLM-based system to extract triples and resolve them into a knowledge graph for a specific domain, namely cyber threat intelligence. The system involves a short-term memory constructor with multiple steps to create initial triples, a long-term memory constructor to merge and modify those triples, and finally the KG constructor.   In addition to the system itself, the paper presents a gold standard set of extracted triples in order to measure performance.  Several variants of the proposed system as well as several existing systems (both general KG construction and CTI specific) are compared against existing benchmarks, with the proposed system performing much better than any of the other solutions.
strength_weakness: strength_weakness: - The English is off, making the paper difficult to read due to distracting ungrammaticalities and odd phrasing.  This can be fixed by having the final paper proof-read by a native speaker of English.
- Needs to add statistical significance of results (given the major improvements, it is highly likely that these are statistically significant)
questions: questions: In order of the paper.

Please have the paper proofread by a native speaker of English. There are issues with determiner placement, verb agreement and word usage.

Will the datasets and code (e.g. for evaluation) be released?  If so, state that and include under what license.

Sec 2: Add a sentence about how the system targets which entities and which relations. This becomes clearer later, but at this point it is confusing whether all subj-verb-obj triples are extracted and then later filtered out if not relevant.
Also add roughly how many unique nodes there are (e.g. 100, 1000, 100000) and how many unique relations (e.g. 10, 100, 1000).

Sec 3.2: What is IOC?
Checker Agent: Can you provide some rough indicator of how often the LLMs output incorrect results?  Is this 1%, 10% or 50% of the time?  Later in the paper with the error analysis (which is very helpful), if you have a rough estimate of how often each error type occurs, that would be interesting to include.

Sec 3.4: How does the system decide whether the node is a new one or should be merged with an existing one?  Often thresholding on similarity is hard (although if the node names are relatively the same length, this becomes easier) with embeddings and so some additional factor is needed.

sec 4.1: "sta" looks odd in the citation; use the name of the lab (e.g. Stanford NLP Lab or whatever the official name is)

Table 1: Include statistical significance of best result against next best.
Perhaps bold face the best results.
It would be great to know the number of triples of each type (row); could these be provided in parentheses after the name of the tactic?
Why does the GPT4 variant do better than CtiKG for Command & Control and Impact? Is there something different about these two compared to all the others?

Sec 4.3: In the long run, systems will have to overcome the coreference resolution issue if they are to work over textual documents. Any suggestions for future work for how to deal with this?

Table 2: Add statistical significance of best system over next best

Sec 6: Just merge this into the intro.  It will be more helpful there and this will save space for other analysis.

Comments: title: Rebuttal
comment: We truly appreciate your feedback and comments.

## **Security-oriented Knowledge Graph**
Regarding the security-oriented knowledge graph, we have implemented the following designs to ensure the security orientation. We specified "Only extract triples that are related to cyber attacks. If a sentence does not contain any triple about cyber attacks, skip the sentence and do not include it in your output" and "Focus on malware, Trojan horses, CVEs, or hacking organizations as the subjects of the triples'' in the prompts of LLM agents, and provided multiple security-related examples, such as "[Leafminer, attempts to infiltrate, target networks]" in the few-shot learning section of LLM agents. According to the evaluation results, CTIKG with these designs can be guided to focus on security-related entities and their relationships.

## **Related IE Approach**
Regarding related IE approaches, we compared CTIKG with Extractor, the SOTA non-LLM based IE approach specifically designed for cyber threat report text. In the evaluation, CTIKG outperforms it in both sentence triple extraction and article knowledge graph construction. For sentence triple extraction, CTIKG outperforms Extractor by about 9% in both precision and recall. For article knowledge graph construction, CTIKG outperforms Extractor by about 30% in both precision and recall.

## **Type of  Entity Relationship**
Regarding the ontology of knowledge graph, first, the article sources are from different websites and authors. These authors have very different writing styles, and pose challenges for a fixed ontology to represent the entities and their relationships. To address this challenge, CTIKG adopts multiple agents to unify different writing styles and achieve better coreference resolution (such as the refiner on the triple level and the merger agent on the segment and article levels). Extractor, on the other hand, suffers from low precision due to this challenge. Second, non-security-related relationships between security-related entities also play an indispensable role for connecting correlated entities, such as two security-related entities like CVEs described in different sentences but connected via a non-security-related entity. In our Appendix case study, the similarity between CVE-2012-0158 and CVE-2017-1188 was discovered based on the entity relationship of “for” and “memory corruption vulnerability”. Our further investigation revealed that Magniber and Cerber have a "mutual ransomware payload" relationship, and Emotet and Trickbot have a "switch to" relationship.
These entity relationships are determined by the author's writing style and cannot be anticipated in advance and defined in the ontology. We will include the effectiveness analysis of non-security-related relationships in the Appendix of the camera-ready version.
We are glad that our answers addressed your concerns and appreciate your feedback. Thank you again for your review and comments.
Thank you for your answer. I agree that the article presents some useful technical solutions to extract triples from those CTI articles, notably to overcome token length limitations (by storing intermediate results) and ensure the formatting is correct. But those solutions are variants of techniques that are already commonly used by LLM practictioners.
I still believe that the article does not pay enough attention to the cyber-security aspects of their work. As stated in the abstract, the goal is to build a *security-oriented knowledge graph". There already exists dozens of such security-oriented KGs, many extrracted automatically through IE techniques (although not all LLM-based). Their purpose is often to allow for some automated reasoning and pattern discovery. To this end, it is often useful to define a priori the types of entities and relations one seeks to extract. In other words, for many applications of such KGs, restricting oneself to an ontology is often an advantage, not a limitation. I believe the article would benefit from a more thorough analysis of whether the extracted KG actually contain entities and relations that are useful and relevant from a cyber-security perspective.
Thank you for your answers that have solved my problem. I have no other questions.
Thank you for taking time to evaluate the paper and your constructive feedback! Your feedback is greatly appreciated.
Thank you for the clarifications. As mentioned in the review, this is very interesting research and it is great to know that the authors will be updating to fix some minor issues to make it easier to read.