L ogic A sker: Evaluating and Improving the Logical Reasoning Ability of Large Language Models (2024)

Abstract: We introduce LogicAsker, a novel approach for evaluating and enhancing the logical reasoning capabilities of large language models (LLMs) such as ChatGPT and GPT-4. Despite LLMs’ prowess in tasks like writing assistance, code generation, and machine translation, assessing their ability to reason has been challenging. Traditional evaluations often prioritize accuracy on downstream tasks over direct assessments of reasoning processes. LogicAsker addresses this gap by employing a set of atomic reasoning skills grounded in propositional and predicate logic to systematically examine and improve the reasoning prowess of LLMs. Our methodology reveals significant gaps in LLMs’ learning of logical rules, with identified reasoning failures ranging from 29% to 90% across different models. Moreover, we leverage these findings to construct targeted demonstration examples and fine-tune data, notably enhancing logical reasoning in models like GPT-4o by up to 5%. To our knowledge, this is the first effort to utilize test case outcomes to effectively refine LLMs’ formal reasoning capabilities. We make our code, data, and results publicly available(https://github.com/yxwan123/LogicAsker) to facilitate further research and replication of our findings.

## introduction large language models (llms), such as openai's gpt series have significantly impacted natural language processing, excelling in a variety of tasks including text generation, machine translation, and code generation @xcite @xcite . reasoning, defined as the cognitive process of using logic to draw conclusions from given facts @xcite , is crucial for complex interactions that go beyond text generation. accurately assessing this ability in llms is essen-tial, yet challenging, as models may correctly perform tasks merely relying on shortcuts such as pattern recognition without truly engaging in logical reasoning @xcite @xcite . consider the following inference example: either it is raining, or tom will play football; if it rains, then the floor will be wet; the floor is dry; therefore, tom will play football. we may encounter the following challenges: 1) it's unclear if a correct llm response is due to reasoning or simple heuristics like word correlations (e.g., "dry floor" is more likely to correlate with "playing football"). 2) if an llm fails, pinpointing the specific breakdown in reasoning is difficult (i.e., inferring not raining from the floor being dry or inferring playing football from not raining). 3) current systems lack comprehensive test cases that encompass various formal reasoning types beyond implication, such as logical equivalence (e.g., a and b are true; therefore, b and a are true. 4) evaluating an llm's reasoning on such cases offers limited insight into enhancing its reasoning capabilities. to better handle these challenges, a wellperforming testing framework should be able to define a set of skills that a) directly correspond to the reasoning process, b) cannot be further divided, c) cover all formal logical reasoning scenarios, and d) can identify llms' weaknesses and facilitate improving llms' performance. property a) ensures that the task cannot be accomplished by other approaches, such as inferring from the correlations of words, and the evaluation result directly reflects the model's reasoning ability. property b) and c) ensure that the set of skills is fundamental and comprehensive, which can provide helpful insights to accomplish property d). we introduce logicasker, an automatic framework designed to evaluate and enhance llms' formal reasoning skills using minimum functionality tests (mfts) @xcite , akin to software engineering's unit tests, which utilize straightforward examples to assess specific behaviors. these tests help identify when models rely on shortcuts rather than genuinely mastering a skill @xcite . specifically, logicasker builds a set of atomic skills from foundational principles of propositional and predicate logic, two fundamental systems used to formalize reasoning procedures @xcite , together with common logical fallacies (hurley and watson, 2020). based on the skill set, logicasker generates reasoning questions by translating standard logic expressions into natural language, assesses llms' accuracy per skill, pinpoints weaknesses, and creates in-context-learning (brown et al., 2020) examples and fine-tuning data to bolster reasoning abilities. in addition, for each skill, logicasker uses diverse vocabulary to frame various natural language queries, computing average performance to minimize biases from word correlations. we evaluated logicasker's performance through extensive testing on six state-of-the-art (sota) llms (hugging face, 2024), including four closed-source llms (gpt-4o, gpt-4, chatgpt, and gemini-1.5) and two open-source llms (llama3 and mixtral). our findings reveal that logicasker's test cases effectively pinpoint logical reasoning failures across these models, with error rates (i.e., 1 -accuracy) between 29% and 90%. these test cases also facilitate the creation of in-context learning examples and fine-tuning data, thereby enhancing logical reasoning capabilities. for instance, applying logicasker's cases to gpt-4o improved its reasoning accuracy from 92% to 97%. all resources are released for reproduction and further research foot_0 . we summarize the main contributions of this work as follows: we demonstrate that the test results by logicasker can be used to effectively evaluate and improve the performance of llms. ## logicasker in this section, we introduce the design and implementation of logicasker, a novel tool to trigger logical reasoning failures in large language models. figure 1 overviews the workflow of logicasker, which consists of three main modules: test case generation, weakness identification and in-context learning (icl) demonstration. in particular, the test case generation module utilizes atomic skills defined on the two formal logic systems and an inference synthesis approach to generate questions as test cases. then, the generated cases are fed into the llms to reveal weaknesses and provide insights into the llms by the weakness identification process. finally, logicasker utilizes these insights to construct icl demonstrations to improve the reasoning abilities of the llms. ## discussion a surprising result from our experiments is the lower performance of cot prompting in some llms, such as gpt-4, despite the widespread belief in cot's ability to enhance logical reasoning. one possible explanation lies in the fundamental difference between how cot operates and the structure of logicasker's test cases. cot functions by breaking down complex reasoning tasks into intermediate, manageable steps, guiding the model toward a final conclusion through a series of natural language reasoning steps. however, logicasker tests focus on assessing llms' mastery of the most basic, indivisible reasoning skills. these fundamental reasoning skills are the building blocks for more complex reasoning processes like cot. since these basic skills cannot be further subdivided into intermediate steps, the benefits of cot are limited when applied to logicasker's simpler test cases. in contrast, improving an llm's understanding of these core skills can enhance its performance on more complex tasks where cot is applicable, as demonstrated by chatgpt's improved results on the logiqa dataset (see q: consider the following premises: if linda were sad, then it is sunny, and if it is sunny, then linda is sad. can we infer the following from them? answer yes or no: it is not overcast. ans (zero-shot): from these premises, we can infer that linda is sad and it is sunny are equivalent; ... this means that linda is sad if and only if it is sunny. however, without additional information about whether linda is actually sad or not, we cannot definitively conclude the state of the weather (overcast or not). therefore, based on the information given and without knowing whether linda is sad, the answer is no. ans (cot): ... the statement "it is sunny" directly contradicts the possibility of it being overcast at the same time, assuming a typical understanding of weather where sunny conditions and overcast conditions are mutually exclusive. ... therefore, if either is true, it must be sunny, and thus it cannot be overcast. ... since the premises lead us to conclude that it is sunny, we can infer that it is not overcast. final answer: yes. this observation underscores the effectiveness of logicasker in revealing situations where models default to heuristics or memorized knowledge rather than true logical reasoning. while cot has been shown to improve logical reasoning in general tasks, logicasker's framework exposes when models fail to genuinely reason and instead fall back on familiar or remembered patterns. this insight suggests that strengthening llms' mastery of basic reasoning skills is a necessary foundation for improving performance on tasks that benefit from cot strategies. ## related work significant advancements in nlp reasoning have been achieved through methods such as chain-of-thoughts (cot) prompting @xcite , which enables models to generate reasoning steps with minimal training. enhancing this, the program of thoughts (pot) prompting (chen et al., 2022) leverages external interpreters like python to handle complex mathematical problems. further augmenting reasoning validity, the logic agent framework transforms llms into logic agents that can dynamically apply propositional logic rules to convert natural language inputs into structured logic forms @xcite . recent studies have focused on evaluating the reasoning capabilities of large language models (llms) by measuring their performance across various reasoning tasks. these include arithmetic @xcite @xcite @xcite @xcite , commonsense @xcite @xcite , and table reasoning @xcite , as well as understanding words, dates, and causal relationships @xcite , and generalization (lake and @xcite . despite these efforts, it remains uncertain whether llms truly reason or rely on simple heuristics, since most assessments focus only on accuracy and do not thoroughly evaluate the reasoning processes. efforts to develop metrics for more formal reasoning analysis in llms include creating datasets with first-order logic problems @xcite , generating test cases using a single predicate inference rule (saparov and he, 2022), building instruction-tuning dataset designed for cot reasoning with @xcite , and employing propositional logic with randomized methods @xcite . these methods, however, often lack generalizability or focus on limited deduction rules. @xcite introduced a comprehensive approach by using all deduction rules in propositional logic to assess llms' deductive reasoning across complex proofs. our research expands further, incorporating all rules and equivalent laws in both propositional and predicate logic, aiming to enhance understanding of each rule's impact on llm performance and using these insights for improvement. ## conclusion in this paper, we present logicasker, an automated tool designed to comprehensively evaluate and improve the formal reasoning abilities of llms under a set of atomic skills. our research demonstrated the efficacy of logi-casker in identifying logical reasoning failures in a diverse set of widely deployed llms, we achieved a substantial error detection rate in revealing reasoning flaws in these models, ranging from 29% to 90%. additionally, we utilized the test cases from logicasker to design in-context learning demonstrations, which effectively enhance the logical reasoning capabilities of llms, e.g., improving from 92% to 97% for gpt-4o. by providing insights into the strengths and weaknesses of llms in reasoning, we are able to improve the reliability and trustworthiness of these models. the release of all the code and data aims to facilitate replication and encourage further research in this crucial area.