Enhancing Fact Verification with Causal Knowledge Graphs and Transformer-Based Retrieval for Deductive Reasoning (2024)

Abstract: The ability to extract and verify factual information from free-form text is critical in an era where vast amounts of unstructured data are available, yet unreliable sources abound. This paper focuses on enhancing causal deductive reasoning, a key component of factual verification, through the lens of accident investigation, where determining the probable causes of events is paramount. Deductive reasoning refers to the task of drawing conclusions based on a premise. While some deductive reasoning benchmarks exist, none focus on causal deductive reasoning and are from real-world applications. Recently, large language models (LLMs) used with prompt engineering techniques like retrieval-augmented generation (RAG) have demonstrated remarkable performance across various natural language processing benchmarks. However, adapting these techniques to handle scenarios with no knowledge bases and to different data structures, such as graphs, remains an ongoing challenge. In our study, we introduce a novel framework leveraging LLMs’ decent ability to detect and infer causal relations to construct a causal Knowledge Graph (KG) which represents knowledge that the LLM recognizes. Additionally, we propose a RoBERTa-based Transformer Graph Neural Network (RoTG) specifically designed to select relevant nodes within this KG. Integrating RoTG-retrieved causal chains into prompts effectively enhances LLM performance, demonstrating usefulness of our approach in advancing LLMs’ causal deductive reasoning capabilities.

## introduction large language models (llms) have shown impressive performance on some language tasks, however, their ability to plan and reason on complex tasks remains an ongoing challenge @xcite . in psychology, the standard test for deductive reasoning consists of giving people premises and asking them to draw conclusions @xcite @xcite . in natural language processing (nlp), ruletaker @xcite and proofwriter @xcite are datasets that challenge models to assign true or false labels to statements about a probable implication. however, there are no nlp benchmarks on causal deductive reasoning, where the premise are facts about an outcome and the statement is about a probable cause. furthermore, @xcite ; @xcite find that current benchmarks do not truly investigate the reasoning capabilities of llms, because the tasks are not meaningfully applied in the real-world. researchers have proposed prompt engineering techniques to improve few-shot and zero-shot task performance @xcite , like using role-play @xcite , in-context learning @xcite , and retrieval-augmented generation (rag) @xcite . recent work has explored using llms to retrieve a task-relevant knowledge sub-graph to support reasoning @xcite . however, extending these techniques to handle cases where no explicit knowledge base is available, or and how to best use knowledge graphs (kgs) in a rag-based llm system remains an open area for research. this paper focuses on the causal deductive reasoning task performed by accident investigators. when an accident occurs, investigators conduct thorough investigations, and come up with a probable cause for the accident. our main contributions can be summarized as follows: ## causal deductive reasoning given an input context c, the goal is to identify the likelihood of a statement s i being a probable cause of accident a. this likelihood is represented by y i 2 (0, 1), where y @xmath0. the task is to determine p (y i |c) for each potential cause s i within a report context c. since we have multiple reports in our dataset, the objective extends to calculating p (y it |c t ), where t denotes the report id. we define g @xmath1. the function f extract (.) extracts causal relations from the context. the aggregated set of all extracted relations from the dataset is denoted as g, representing the repository of causal relations of our dataset. each relation in g t is represented by a cause and effect pair, denoted as (s i , s j ). if a causal chain x @xmath2, s j1 ), (s j1 , s j2 ), ..., (j k , k) / 2 g t , then y @xmath3. however, if x it 2 g t , the rank of y it relative to other potential causes y jt must be considered. only the top z rank of most important causes can be the probable cause of an accident a. in the case where we only consider the top cause (@xmath4, then the probability of p (y it ) can be reformulated into: since the task is a binary classification task, every example s it is not aware of the other possible s jt for the same report t. therefore, s jt are causes the model implicit tracks and has to rank against for the current task. our causal deductive task can be re-framed into two sub-challenges: (1) extracting x it and identifying x it 2 g t , and (2) implicitly ranking p (y @xmath5. hypothesis 1: generalizing causal chain to outof-context in the first challenge, extracting x it and identifying x it 2 g t , restricting the knowledge source to a report results in a high chance for there to be gaps in the causal chain. all else fixed, p (y it |x it / 2 g t ) will be overestimated (i.e., model predicts more 0s than 1s). if are willing to relax our criteria to check if s i 2 c t and x it 2 g, then we are allowing our model to generalize to its own knowledge base, to recognize more valid causal chains, and therefore, increase the probability of predicting p (y @xmath6. when working with llms, therefore, we could inject causal relations outside of g t but semantically part of x it to improve prediction. hypothesis 2: ranking importance of cause within context if the llm is exposed to too many relevant causal relations in the prompt, it would hallucinate and start to always view s i as the most important probable cause (over other possible options in c t ). however, we do not know z. in some reports, there are a few probable causes. one approach is to explicitly expose the llm to the available causes in the report, so that we re-ground the response, and in some way, a ranking based on context is encouraged. ## dataset & task creation we wish to investigate the llms' ability to perform a real-world causal deductive reasoning task. given an accident description (<@xmath7, the model must determine if a sentence about the probable cause of the accident (<@xmath8. to facilitate our research, we leverage on reasoning-rich investigation reports from the national transportation safety board (ntsb) 1 . ntsb publishes accident reports that provides details about an accident, analysis of the factual data, conclusions and the probable cause of the accident, and the related safety recommendations. there can be one or multiple probable cause(s). we downloaded reports published after year 2000, across all reported categories (aviation, hazardous materials, highway, marine, pipeline and railroad). report pre-processing pre-processing was done to convert the pdf reports to json, and subsequently, we removed information like headers, page numbers, and table of contents. we identified the probable cause of the accident by searching for the title "probable cause". we discarded reports where this match was impossible. any text before this section is defined as @xmath9. in our experiments, we constrained our coverage to 157 reports where the context length is  2, 000 words. extracting true statements trailing descriptions in the probable cause were removed. 2 we used anthropic's claude 2.1 3 to convert the para-1 https://www.ntsb.gov/investigations/ accidentreports/pages/reports.aspx 2 e.g. descriptions unrelated to the cause (e.g. "the national transportation safety board determines that the") were removed. 3 we intentionally used an llm different from mistral when creating our dataset to avoid cases where the llm rec-graphs into a list of probable causes. prompt 1 in appendix outlines the one-shot prompt template that we used. we manually annotated four examples to measure the extraction performance, of which we found rougel score of 87.46 and bleu4 score of 75.02. when evaluating by semantic match 4 with a threshold of 0.7 as a match, claude 2.1 scored 100% for recall, 72.92% for precision, and 84.34% for f1. to summarize, the high scores for the evaluated sample provides us with the confidence to reliably use the extracted probable causes as true instances for our main causal deductive task. ## causal kg rag with llm we mentioned in section 2 that we wish to help the llm recognize generalized (j a , j b ) 2 d by injecting relevant causal relations outside of g t . however, we do not have a knowledge base for g. we also do not have any annotations for the intermediate causal chains that might be relevant given a probable cause i and accident a. to work around these problems, we constructed our knowledge base using the llm itself. afterwhich, we designed a novel graph-based retriever model, trained on the auxiliary binary classification task, to select relevant nodes. ## experimental findings this paper focuses the investigation on the mistral-instruct 7b llm @xcite . we used mistral to extract and infer causal relations for our kg as described in section 4.1, then trained rotg over this kg as described in section 4.2. finally, we tested mistral on the causal deductive reasoning task as described by section 4.3. ## related work our dataset and task is most relevant to the deductive reasoning nlp literature, like efforts by rule-taker @xcite and proofwriter @xcite . different from them, our dataset is a real-world deductive reasoning task about accident investigations, and dive deep into the causal aspect. huang and chang (2023); @xcite stated that current reasoning benchmarks are not meaningfully applied in the real-world. thus, we hope that our dataset and work allievates this gap in the literature. our methodology is relevant to literature on rag for llms @xcite . however, due to the nature of causal relations in our task, we focus on retrieval techniques over a graph. thus, we were also inspired by prior research on retrieval on kgs @xcite and on node classification @xcite . since encoding graph structured data for llms is also an ongoing research @xcite , more investigations on how to best present the causal chains in the prompts are needed. different from previous works, we investigate how to leverage on knowledge already present in the dataset (extract) and within the llms (infer) to improve performance, instead of relying on external databases that many rag methodologies focus on. ## conclusion our study addresses the challenging task of causal deductive reasoning, particularly within the context of real-world accident investigation reports. firstly, we introduced a framework that constructs a causal kg based on what llms' can extract and infer. secondly, we proposed rotg, trained to select relevant nodes, utilizing deductive reasoning labels as an auxiliary task. our experiments demonstrate that incorporating rotg relations into the prompt enhances the performance of llms (from 70.36% (90.30%) to 73.19% (91.65%) macro (micro) f1), highlighting the effectiveness of integrating graph-based retrieved relations in improving llms' causal deductive reasoning abilities. lastly, our dataset will be released and will be a valuable resource for researchers. overall, our study advances the understanding and application of deductive reasoning tasks in nlp, specifically in the domain of kg-based rag for llms.