CLOMO : Counterfactual Logical Modification with Large Language Models (2024)

Abstract: In this study, we delve into the realm of counterfactual reasoning capabilities of large language models (LLMs). Our primary objective is to cultivate the counterfactual thought processes within LLMs and rigorously assess these processes for their validity. Specifically, we introduce a novel task, Counterfactual Logical Modification (CLOMO), and a high-quality human-annotated benchmark. In this task, LLMs must adeptly alter a given argumentative text to uphold a predetermined logical relationship. To effectively evaluate a generation model’s counterfactual capabilities, we propose an innovative evaluation metric, the decomposed Self-Evaluation Score (SES) to directly evaluate the natural language output of LLMs instead of modeling the task as a multiple-choice problem. Analysis shows that the proposed automatic metric aligns well with human preference. Our experimental results show that while LLMs demonstrate a notable capacity for logical counterfactual thinking, there remains a discernible gap between their current abilities and human performance. Code and data are available at https://github.com/Eleanor-H/CLOMO.

## introduction despite large language models @xcite perform strikingly in plenty of reasoning benchmarks @xcite , late studies observe an internal inconsistency in their reasoning processes @xcite . the inconsistency is attributed to misunderstanding and misapplication of logical relations. however, logical relations in complex language reasoning are not yet properly quantified and evaluated. current studies on evaluating model reasoning are limited in both form and content. on the one hand, benchmarking complex reasoning is generally applying discrimination tasks such as multiplechoice questions @xcite @xcite , where accuracy and pass rate serve as the main evaluation metric. however, such evaluations oversimplify the goal of uncovering essential and subtle pitfalls in complex reasoning. for example, the reasoning processes could contain misconceptions in logical relations but give correct answers due to the data distribution @xcite . therefore, evaluating the generated content would provide a more realistic measurement of model reasoning. on the other hand, unlike widely studied reasoning tasks such as math reasoning @xcite and standard exams @xcite , counterfactual reasoning @xcite as a fundamental evaluation of logical relations is less explored in the context of large language models. previous literature studies counterfactual reasoning either in a multiple-choice manner @xcite or applying labored human study to evaluate counterfactual generation @xcite , leaving an effective evaluation of counterfactual generation unexplored. in our study, we delve into the realm of evaluating large language models' (llms) ability to generate counterfactually coherent thoughts. figure 1 demonstrates the paradigm. specifically, we proposed an innovative evaluation system that quantitatively measures the evolution of information in statement pairs, ensuring that they adhere to a specified logical relationship. our approach includes designing a specialized task where models are presented with mismatched argument-premise pairs bound by a specific logical relation. the objective for these models is to adeptly modify the argument text until the specified logical relation is satisfactorily established. in conjunction with this task, we have created the first dataset of its kind, compris- r m counterfactual logical modification logical relations • provides a necessary assumption to • provides a sufficient assumption to • strengthen • weaken argument premise 1 premise 2 r m r state 1 state 2 argument: statement1: after the second world war, the charter of the newly formed united nations established an eleven-member security council and charged it with taking collective action in response to threats to world peace. statement2: the charter further provided that the five nations that were then the major powers would permanently have sole authority to cast vetoes. statement3: the reason given for this arrangement was that the decisions reached by a majority of nations could be biased in favor of one or more major powers, thus ensuring their support in enforcing these decisions. in the following, you will see an argument and 2 premises, where premise 1 provides a sufficient assumption to the argument. please modify the statements in the argument until premise 2 provides a sufficient assumption to the argument instead, while premise 1 fails to provides a sufficient assumption to the argument. note that no additional statement should be added. argument: statement1: after the second world war, the charter of the newly formed united nations established an eleven-member security council and charged it with taking collective action in response to threats to world peace. statement2: the charter further provided that the five nations that were then the major powers would permanently have sole authority to cast vetoes. statement3: the reason given for this arrangement was that the burden of maintaining world peace would rest on the world's major powers, and no nation should be required to assume the burden of enforcing a decision it found repugnant. premise1: no nation that was not among the major powers at the end of the second world war would become a major power. premise2: decisions reached by a majority of nations in response to threats to world peace would be biased in favor of one or more major powers. please write the modified argument below: ing dual argument-premise pairs, each annotated with a defined logical relation. this dataset is vital for facilitating logically restricted counterfactual modifications, and we have enriched it with humanwritten modifications to serve as a benchmark for evaluation. our experimental investigations encompass a range of large language models, including the latest gpt-4o foot_0 , gpt-4 (openai, 2023), and gpt-3.5-turbo (openai, 2022), as well as smaller models from the llama @xcite and llama 2 @xcite families. through these experiments, we have discerned that the task of clomo poses a significant challenge. it becomes evident that these models' current counterfactual logical reasoning capabilities fall short of the desired proficiency. this observation underscores the need for further advancements in enhancing the counterfactual reasoning abilities of existing language models, paving the way for more sophisticated and logically coherent ai systems. the contributions of this paper are three-fold: ## related works from complex reasoning to counterfactual reasoning complex reasoning has been highly concerned as a significant yet challenging task for inspecting advanced artificial intelligence. for example, for solving commonsense reasoning problems @xcite @xcite @xcite , the models @xcite @xcite are required to reasonable applying commonsense knowledge to conduct the reasoning process for the final answer. furthermore, multi-step reasoning needs the models to perform multiple reasoning steps while maintaining consistency and faithfulness. to achieve this, synthetic compositional reasoning tasks @xcite @xcite @xcite incorporate first-order logic to inspect and improve models logical consistency @xcite ; sanyal ## ses: self-evaluation scores we aim to use the proven complex reasoning capabilities of the large language model itself to perform faster and more efficient reasoning evaluations of complex reasoning tasks that do not ## conclusion in this paper, we study large language models' counterfactual reasoning capability under the constraint of proper logical relations. to this end, we introduce a novel task, counterfactual logical modification, that requires the llms to conduct counterfactual modification with logical restriction, where llms need to appropriately modify an argument text so that a specified logical relation stands. to ensure a comprehensive evaluation, we then construct a benchmark dataset clomo. more-over, we propose a self-evaluation score (ses) that decomposes the evaluation into several llms basic discrimination tasks, which is demonstrated aligned with human evaluations. we further evaluate smaller language models in inference-only and fine-tuning manners. the fine-tuned models' performances are significantly improved, but there is still a large gap with human performance. our findings thus underscore the need for further development in llms' counterfactual reasoning.