L ogic B ench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Models (2024)

Abstract: Recently developed large language models (LLMs) have been shown to perform remarkably well on a wide range of language understanding tasks. But, can they really “reason” over the natural language? This question has been receiving significant research attention and many reasoning skills such as commonsense, numerical, and qualitative have been studied. However, the crucial skill pertaining to ‘logical reasoning’ has remained underexplored. Existing work investigating this reasoning ability of LLMs has focused only on a couple of inference rules (such as modus ponens and modus tollens) of propositional and first-order logic. Addressing the above limitation, we comprehensively evaluate the logical reasoning ability of LLMs on 25 different reasoning patterns spanning over propositional, first-order, and non-monotonic logics. To enable systematic evaluation, we introduce LogicBench, a natural language question-answering dataset focusing on the use of a single inference rule. We conduct detailed analysis with a range of LLMs such as GPT-4, ChatGPT, Gemini, Llama-2, and Mistral using chain-of-thought prompting. Experimental results show that existing LLMs do not fare well on LogicBench; especially, they struggle with instances involving complex reasoning and negations. Furthermore, they sometimes tend to prioritize parametric knowledge over contextual information and overlook the correct reasoning chain. We believe that our work and findings facilitate future research for evaluating and enhancing the logical reasoning ability of LLMs.

## introduction large language models such as @xcite , and mistral @xcite have made remarkable progress in nlp research enabling machines figure 1 : comprehensive representation of different inference rules and reasoning patterns covered by propositional, first-order, and non-monotonic logics. exp. indicates expectation to perform a variety of language tasks that were previously thought to be exclusive to humans @xcite @xcite . however, the ability of these llms to reason "logically" over natural language text remains under-explored, even though logical reasoning is a fundamental aspect of intelligence and a crucial requirement for many practical applications, such as questionanswering systems @xcite and conversational agents @xcite . although several datasets have been proposed @xcite @xcite to evaluate the logical reasoning capabilities of llms, these datasets are limited in their scope by (1) not evaluating logical reasoning independently of other forms of reasoning such as logiqa @xcite and reclor @xcite ; and (2) evaluating only a single type of logic and covering only few logical inference rules as done in folio @xcite and @xcite . thus, our aim in this work is to address the lacuna of having a more comprehensive set of inference rules for evaluating the logical reasoning ability of llms. to this end, we introduce a systematically created question-answering dataset for the evaluation of logical reasoning ability using a single inference rule, called logicbench. besides evaluating the logical reasoning ability of llms, by evaluating models on single inference rules, we can also gain insights into the frequency of text sequences corresponding to these rules in the pre-training data and their impact on model performance. as illustrated in figure 1 , logicbench includes a total of 25 reasoning patterns across 'propositional, first-order, and non-monotonic' logics. to the best of the authors' knowledge, this is the first work to study non-monotonic reasoning, as well as various inference rules in propositional and first-order logics including hypothetical and disjunctive syllogism; and bidirectional, constructive, and destructive dilemmas in nlp domain. to evaluate llms using log-icbench, we formulate two different tasks: (i) a binary question-answering (bqa) task in which the context comprises logical statements and the models have to determine whether a conclusion given in the question is logically entailed by the context, and (ii) a multiple-choice questions-answering (mcqa) task where models must select the most appropriate logical conclusion from four distinct options, based on the provided context. the rationale behind having bqa and mcqa tasks is that it provides systematic standard metric-based evaluation (i.e., direct comparison of llms' performance in terms of accuracy), which could be more challenging with open-ended question-answer formats. examples instances of various reasoning patterns are presented in to construct logicbench, we use a three-stage procedure (refer to §3). in the first stage, we prompt gpt-3.5 to generate a variety of coherent natural language sentences having different 'ontologies' (i.e., a collection of concepts as car, person, and animals) and their corresponding negations (refer to §3.2.1). in the second stage, we generate (context, question) pairs where the context represents a natural language narrative consisting of logical statements, and the question is formulated to exhibit the logical conclusion derived from the context. in the third stage, we generate task-specific data instances (i.e., (context, question, answer) triplets). we conduct a comprehensive evaluation with a range of llms on logicbench including gpt-4, chatgpt (gpt-3.5-turbo), gemini-pro, llama-2-7b-chat, and mistral-7b-instruct using chainof-thought @xcite . in particular, we measure the accuracy of llms predictions on both bqa and mcqa tasks. our experiments result in several interesting findings such as llms often struggle to reason over complex logical contexts that involve only a single inference rule and encounter difficulties with inference rules involving negations. experimental results reveal that these models struggle with respect to many of the inference rules and patterns, suggesting significant room for improvement in their logical reasoning abilities. to further demonstrate the use of log-icbench, we synthetically augment it and fine-tune t5-large. our preliminary results (app. i) show that this improves the logical reasoning ability of existing models leading to performance improvement on other logic datasets, logicnli, and fo-lio (∼ 2% on an average), and shows competitive performance on logiqa and reclor. ## related work as llms continue to evolve rapidly, it becomes increasingly crucial to evaluate their diverse reasoning capabilities, as well as those of forthcoming llms. logiqa @xcite and reclor @xcite have made notable contributions by compiling multichoice questions from standardized examinations that demand diverse forms of logical reasoning. in contrast to logicbench, these datasets involve mixed forms of reasoning and do not focus on assessing logical reasoning in isolation. a few past attempts have been made to evaluate only logical reasoning while excluding other forms of reasoning. for example, clutter @xcite covers inductive reasoning, @xcite covers temporal logic, and ruletaker @xcite evaluates whether a transformerbased model emulates deductive reasoning over synthetically generated statements in a limited setting. @xcite introduced a diagnostic benchmark for fol reasoning, with the dataset constructed by automatically generating logic expressions and replacing the entity and attribute placeholders. our proposed dataset is similar (in terms of task formulation) to @xcite , ## conclusions in this work, we evaluated the logical reasoning ability of llms on 25 distinct inference rules and reasoning patterns covering pl, fol, and nm logics. to this end, we introduced logicbench, a natural language question-answering dataset focusing on evaluating a single inference rule. we devised two tasks using logicbench: (i) bqa, and (ii) mcqa. we evaluated a range of llms including gpt-4, chatgpt, gemini-pro, llama-2, and mistral on both tasks. experimental results showed that llms do not perform well on logicbench, even though they require the application of only a single inference rule. furthermore, we also augmented logicbench to logicbench(aug), which can be utilized for training purposes. using log-icbench(aug), we demonstrated that llms trained using it showcase an improved understanding of logical reasoning, resulting in a better performance on existing logic datasets.