Is C hat GPT a Good Causal Reasoner? A Comprehensive Evaluation (2023)

Abstract: Causal reasoning ability is crucial for numerous NLP applications. Despite the impressive emerging ability of ChatGPT in various NLP tasks, it is unclear how well ChatGPT performs in causal reasoning. In this paper, we conduct the first comprehensive evaluation of the ChatGPT’s causal reasoning capabilities. Experiments show that ChatGPT is not a good causal reasoner, but a good causal interpreter. Besides, ChatGPT has a serious hallucination on causal reasoning, possibly due to the reporting biases between causal and non-causal relationships in natural language, as well as ChatGPT’s upgrading processes, such as RLHF. The In-Context Learning (ICL) and Chain-of-Thought (COT) techniques can further exacerbate such causal hallucination. Additionally, the causal reasoning ability of ChatGPT is sensitive to the words used to express the causal concept in prompts, and close-ended prompts perform better than open-ended prompts. For events in sentences, ChatGPT excels at capturing explicit causality rather than implicit causality, and performs better in sentences with lower event density and smaller lexical distance between events.

## introduction causal reasoning ability is crucial for numerous nlp applications. the recent causal reasoning systems are mainly based on fine-tuned pre-trained language models (plms) such as bert @xcite and roberta @xcite . however, their causal reasoning abilities rely on supervised training using large amounts of annotated data. most recently, chatgpt has achieved remarkable performance in various nlp tasks without the need for supervised training. however, there is currently no work that comprehensively evaluates chatgpt's ability in causal reasoning. figure 1 : the forms of three causal reasoning tasks and the prompts we use. the content that requires chatgpt to reply is marked in red. the multiple-choice cd task also involves samples that ask for selecting the result of the input event. for such samples, we modify the "cause" in the question to "result". in this paper, we conduct a comprehensive evaluation to demonstrate chatgpt's causal reasoning capabilities, involving four state-of-the-art (sota) versions of chatgpt: text-davinci-002, text-davinci-003, gpt-3.5-turbo and gpt-4. firstly, we utilize the event causality identification (eci) task as a comprehensive causal reasoning benchmark. as shown in figure 1 , the eci task aims to determine whether there is a causal relationship between two events in a sentence. this requires the chatgpt to not only use extensive commonsense knowledge but also understand the complex context composed of multiple entities and events. finally, chatgpt must combine all information to identify causal relationships. secondly, we employ causal discovery (cd) task for evaluation, which requires chatgpt to possess a broader and more specialized knowledge, yet does not necessitate consideration of complex contexts. as shown in figure 1 , two cd task formats are used: 1) multiple-choice, aims to select the cause or effect of the input event from two options; 2) binary classification, aims to determine whether there is a causal relationship between the two input events. for binary classification, we convert each multiple-choice example into two binaryclassification examples by pairing the input event with each of the two options. our experiment indicates that binary classification is a more reliable evaluation method for chatgpt. furthermore, as shown in figure 1 , we conduct causal explanation generation (ceg) task to test whether chatgpt can generate explanations for causal relations between events. this is typically used to test whether machines truly understand the principles behind causality, which is crucial for building a reliable causal reasoning system. ## conclusion in this paper, we conduct a comprehensive evaluation of chatgpt's causal reasoning capabilities. experiments show that: 1) chatgpt is not a good causal reasoner, but is good at causal explanation generation; 2) chatgpt has a serious causal hallucination, possibly due to the causal reporting biases and chatgpt's upgrading processes 3) the icl and cot techniques can further exacerbate such causal hallucination; 4) the chatgpt is sensitive to the words used to express the causal concept in prompts, and open-ended causal reasoning prompts is not suitable for chatgpt; 5) for events in sentences, chatgpt excels at capturing explicit causality, and performs better in sentences with lower event density and smaller event distances. although there may be more delicate prompts that can further surpass our reported results, we believe that relying solely on prompts cannot fundamentally solve the issues that chatgpt faces in causal reasoning. we hope that this study can inspire future works, such as addressing the causal hallucination issue of chatgpt or further evaluating chatgpt in scenarios involving multi-factor and multi-modal causal reasoning.