Can C hat GPT Understand Causal Language in Science Claims? (2023)

Abstract: This study evaluated ChatGPT’s ability to understand causal language in science papers and news by testing its accuracy in a task of labeling the strength of a claim as causal, conditional causal, correlational, or no relationship. The results show that ChatGPT is still behind the existing fine-tuned BERT models by a large margin. ChatGPT also had difficulty understanding conditional causal claims mitigated by hedges. However, its weakness may be utilized to improve the clarity of human annotation guideline. Chain-of-Thoughts were faithful and helpful for improving prompt performance, but finding the optimal prompt is difficult with inconsistent results and the lack of effective method to establish cause-effect between prompts and outcomes, suggesting caution when generalizing prompt engineering results across tasks or models.

## introduction finding causal relationship is an important goal in scientific research. however, choosing appropriate causal language that accurately reflects the strength of evidence is a non-trivial task when describing research findings. subjectivity and bias may affect how authors interpret the results. for example, some researchers argued that observational studies can not illuminate causal claims and thus causal language should not be used (e.g., @xcite , while others called for more confidence in causal inference with improved methods and guidelines, (e.g., @xcite . on the other hand, average human readers reported difficulty in judging the strength of causal claims mitigated with hedges such as "may" or ambiguous terms like "linked to" @xcite . manual factchecking of causal claims in academic publications, news and social media posts also demonstrated evidence of prevalent exaggeration when reporting causal findings @xcite @xcite . prior studies have also looked into computational approaches for identifying claim strengths and exaggerated claims. the core component is a text classification task that categorizes research findings by their strengths. the original task definition was based on a manual content analysis @xcite , which defined seven certainty levels: no statement, explicit statement of no relation, correlational (e.g. "drinking wine is associated with increased cancer rates"), ambiguous (e.g. "drinking wine linked to cancer risk"), conditional causal (e.g. "drinking wine might increase cancer risk"), can cause (e.g, "drinking wine can increase cancer risk"), and unconditionally causal (e.g. "drinking wine increases cancer risk"). however, adams et al. @xcite found that average human readers can distinguish three categories of relationship only: direct cause statements (e.g. "makes"), can cause statements (e.g. "can make"); and moderate cause statements (e.g. "might cause", "linked", "associated with"), and they encountered difficulty in distinguishing the conditional causal statements and correlational statements in the last group. in light of these observations, the subsequent computational modeling studies simplified the task to classify four categories: direct causal, conditional causal, correlational, or no relationship @xcite @xcite . these specialized models used various techniques to achieve high accuracy, such as fine-tuning pre-trained bert models @xcite , or through causal augmentation @xcite . these models also have limitations, such as mistaking a no-relationship sentence as causal or correlational when confounding cues exist. they also rely on thousands of humanannotated training examples. recently, the debut of large language models (llms) such as chatgpt shifted the nlp research paradigm toward the direction of "pre-train, prompt, and predict", where downstream tasks are reformulated into textual prompts on zero-shot or few-shot settings @xcite . llms trained on sufficiently large and diverse datasets demonstrate promising performance on reasoning tasks without additional task specific training @xcite . the promising results inspired hope for eliminating the need for specialized models and expensive human annotations @xcite . a question rises then -can chatgpt "understand" causal language in science writing? more specifically, can chatgpt label the strength of causal claims correctly? furthermore, since chatgpt was trained with a variety of textual data, did it inherit the confusion that human readers have regarding conditional causal claims? in this study, we evaluate chatgpt's ability to understand causal claims in science papers and news. we tested gpt3.5 (text-davinci-003) and chatgpt (gpt-3.5-turbo) performance in classifying causal claim strength using the annotated corpora provided by @xcite and @xcite . specifically, we ask the following research questions: our paper is organized as follows: section 2 summarizes studies on prompt engineering and various classification tasks evaluated on chatgpt. in the methods section, we introduce the different prompt designs we experimented (section 3.1), explain how we evaluated the model's performance (section 3.2), provide a summary of the data we used (section 3.3) and present the api details for the experiment (section 3.4). we then report experiment results that address rq1 and rq2 (section 4.1) as well as rq3 and rq4 (section 4.2). based on the experiment we test the entire dataset (section 4.3) and also evaluate the result of applying ensemble models (section 4.4). finally, we discuss our limitations and present our conclusion with discussions in section 5. ## related work since prompts provide crucial information for llms such as chatgpt, a number of studies have explored prompt engineering strategies @xcite . here we summarize several common prompt design approaches with a focus on text classification tasks, which are most relevant to our study. zero-shot, one-shot, and few-shot learning are three types of prompting designs used to guide llms. zero-shot prompting provides task descriptions or instructions without explicit examples. one-shot prompting uses a single example for the desired task. few-shot prompting is similar to the one-shot design, but it involves providing the model with a small number of examples instead of just one for the model to learn from and generate task-aligned responses @xcite . prompts can be generated either manually or automatically @xcite @xcite . while manual, intuitive approach is straightforward, it can be timeconsuming to identify the most effective prompt and there is no guarantee to find one @xcite . researchers then sought automatic approaches @xcite @xcite , or even asking chatgpt itself to generate prompts @xcite . however, since llms sometimes do not follow instructions, their answers may be ill-formatted or even invalid. when that happens, human intervention is needed, which increases the time cost for post-processing llm results @xcite . therefore, in this study, we focused on manually-generated prompts. text classification tasks often use instruction prompts to explicitly tell llms what to do. for instance, @xcite 's prompt starts with an instruction of task description: "for each snippet of text, label the sentiment of the text as positive or negative. the answer should be exact 'positive' or 'negative' ", followed with the text to be labeled. @xcite formulate their prompt as "definition: ... input: ... answer: ..." where an example of definition can be determine the speaker of the dialogue, agent or customer. more context information about the task may be added to the instruction prompt, such as providing the definition of genre for genre classification @xcite . another commonly-used instructional element is the chain-of-thought (cot), which has been found to improve llms' performance on certain arithmetic, commonsense, and symbolic reasoning tasks @xcite . while cot was initially an instance of few-shot prompts, a decent zero-shot performance in reasoning tasks and classification tasks was demonstrated by adding a simple cot prompt such as "let's think step by step" at the end of a question @xcite . chatgpt also provides a unique feature, system messages, which can be used as part of the prompt to guide the model's behavior @xcite , such as "you are a helpful assistant that can classify sentences as either causal or correlational research findings" which specifies the model to behave as a professional for our task. previous studies have evaluated chatgpt on various classification tasks @xcite @xcite , using various prompt designs. the results indicate promise and limitations. in the case of sentiment classification, chatgpt was found to have difficulty in understanding neutral sentiment, or give unbalanced predictions on negative vs. positive sentiment, raising questions on the extent to which chatgpt really "understands" sentiment as a linguistic concept @xcite . it is also difficult to directly compare the results due to different sample sets and prompts. the sample sizes were usually small since most studies were conducted before openai made the api available. ## conclusion and discussion causal language is an important rhetorical device in science communication. however, subjectivity in causal language use and understanding is a challenge for science writing and reading. since chat-gpt captures latent social information to some extent, this study evaluated its ability to understand causal language in science papers and news by testing their accuracy in a task of claim strength classification. the results show that (1) chatgpt is still behind the existing fine-tuned bert models by a large margin; (2) chatgpt seems to have inherited the confusion observed among average human readers when judging the strength of conditional causal claims that were mitigated by hedges; while we were conducting our experiment, another study posted to arxiv @xcite reported their chatgpt evaluation on the pubmed data set. both studies shared the findings that prompt engineering required significant investment and a slight difference in prompts could lead to substantial change in performance. both studies on the same task found that cot helped performance. while both zero-shot and few-shot settings were tested in @xcite , we tested the zero-shot setting only. our study has better performance under the zero-shot setting. comparing the prompts, we hypothesize that explicitly asking the causal relationship in the prompt may have helped. however, a systematic method is still lacking to infer causality between word choices in prompts and the performance. despite our effort for a systematic review of chatgpt's understanding of causal claims, our study design has some limitations as the evaluation methodology for prompt engineering is still under development in the nlp community. we arbitrarily decided on the size of the development set. our study focused on zero-shot setting with the purpose of evaluating the latent understanding on causal claims within chatgpt. further exploration could be conducted to investigate the impact of few-shot settings by carefully selecting examples based on recent progress in few-shots prompting methods @xcite . we conclude that chatgpt has a promising but still limited ability in understanding causal language in science writing. cots improved prompt performance, but finding the optimal prompt is difficult with inconsistent results and the lack of effective methods to establish cause-effect between prompts and outcomes. following instruction is an important prerequisite for using chatgpt as a text classification tool, to avoid high labor cost for post-processing its answers. however, chat-gpt provides a new, simulation-style approach for designing and evaluating human annotation guidelines. bei yu, yingya li, and jun wang. 2019. detecting causal language use in science findings. in proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (emnlp-ijcnlp), pages 4664-4674. bei yu, jun wang, lu guo, and yingya li. 2020. measuring correlation-to-causation exaggeration in press releases. in proceedings of the 28th international conference on computational linguistics, pages 4860-4872. qihuang zhong, liang ding, juhua liu, bo du, and dacheng tao. 2023. can chatgpt understand too? a comparative study on chatgpt and fine-tuned bert. arxiv preprint arxiv:2302.10198.