Exploring Reasoning Biases in Large Language Models Through Syllogism: Insights from the N eu BAROCO Dataset (2024)

Abstract: This paper explores the question of how accurately current large language models can perform logical reasoning in natural language, with an emphasis on whether these models exhibit reasoning biases similar to humans. Specifically, our study focuses on syllogistic reasoning, a form of deductive reasoning extensively studied in cognitive science as a natural form of human reasoning. We present a syllogism dataset called NeuBAROCO, which consists of syllogistic reasoning problems in English and Japanese. This dataset was originally designed for psychological experiments to assess human reasoning capabilities using various forms of syllogisms. Our experiments with leading large language models indicate that these models exhibit reasoning biases similar to humans, along with other error tendencies. Notably, there is significant room for improvement in reasoning problems where the relationship between premises and hypotheses is neither entailment nor contradiction. We also present experimental results and in-depth analysis using a new Chain-of-Thought prompting method, which asks LLMs to translate syllogisms into abstract logical expressions and then explain their reasoning process. Our analysis using this method suggests that the primary limitations of LLMs lie in the reasoning process itself rather than the interpretation of syllogisms.

## introduction the ability to perform accurate deductive reasoning in natural language, once pursued by classical symbolic ai, has recently become a vital issue in the development and evaluation of large language models (llms) @xcite . regarding humans, empirical research in cognitive science has demonstrated that humans often exhibit various errors and tendencies in reasoning, known as reasoning biases @xcite . among various forms of reasoning, syllogistic reasoning is one of the basic forms of deductive reasoning and has been studied extensively @xcite @xcite . however, the evaluation of llms and the construction of datasets incorporating these experimental methodologies has not been systematically pursued. in this paper, we present the evaluation of llms with neubaroco, a manually constructed syllogism dataset with diverse properties and annotations designed to evaluate the reasoning abilities and biases of llms in syllogistic reasoning tasks. neubaroco is a bilingual (japanese and english) dataset and includes detailed annotations for the types of reasoning biases associated with each problem. the dataset is based on a problem set used for a series of psychological experiments assessing human reasoning ability with syllogisms @xcite . a subset of the problems in the dataset is aligned with human performance metrics from these experiments. building on the work of @xcite , we have adapted this problem set to evaluate whether language models exhibit reasoning biases similar to those of humans. the main contributions in this paper are summarized as follows. 1 first, we constructed a dataset containing 95 and 790 syllogistic reasoning problems in the format of multiple-choice and natural language inference (nli) tasks, respectively. this dataset design facilitates comparison with results and insights from human psychological experiments and preliminary studies on machine learning models. second, we systematically investigated various reasoning biases observed in llms in relation to the form and content of syllogisms, including belief biases, conversion errors, and atmosphere effects, along with other tendencies, across multiple tasks and in both english and japanese. finally, to more precisely identify the reasoning errors in llms, we introduce a new task called translate-and-explain. this task requires llms to first translate the syllogism into a logical expression (translate) and then explain the reasoning process (explain), by leveraging multi-step chainof-thought prompting. our findings indicate two distinct error patterns: one in the interpretation of the syllogism demonstrated in translation, and the other in the explained reasoning process. however, the primary limitations of these models lie in the reasoning process itself, rather than in the interpretation of syllogisms. we first provide an overview of syllogistic reasoning in section 2, followed by a detailed presentation of the dataset in section 3. subsequent sections present the evaluation tasks (section 4) and the experimental setup along with an analysis of the results (section 5). additionally, section 6 reviews related work in the field. ## background: syllogistic reasoning in this study, we primarily focus on the logical inference task that classifies inferences into three labels: entailment, contradiction, and neutral (neither entailment nor contradiction). a syllogism is an inference that consists of two premises and one conclusion, where the premises and the conclusion are composed of four basic types of quantified sentences: all, no, some, and some-not, as shown in p1: all b are c p2: all a are b c: all a are c syllogisms are relatively simple logical inferences that can be represented in monadic predicate logic @xcite , a fragment of first-order logic where each predicate can take only one argument. despite being logically simple, syllogisms are known to be challenging inferences for humans. which types of syllogisms are prone to causing errors in human reasoning, or in other words, involve reasoning biases, is a topic widely studied in the field of cognitive science of human reasoning @xcite @xcite @xcite . our choice to focus on syllogistic reasoning is deliberate and aims to facilitate comparisons with insights from the extensive research on biases and reasoning in cognitive science. some typical biases of syllogism will be introduced in section 3.2. focusing on the possibility of a detailed classification of such human reasoning biases, this study uses syllogistic reasoning in natural language as a testbed to evaluate the logical reasoning capabilities of llms. ## the neubaroco dataset the neubaroco dataset is based on a syllogism problem set called the baroco test @xcite , originally designed for large-scale research on human reasoning abilities. baroco includes the so-called belief-bias tasks, which are typical examples involving human reasoning biases (see section 3.2.1). in addition to linguistic tasks, it also includes euler diagram tasks to test spatial cognition. these formats of reasoning were used to investigate the correlation and the contributions of genetic and environmental factors through twin studies @xcite . furthermore, studies combining these tasks with experimental tasks in behavioral economics have also been conducted @xcite . @xcite provides a preliminary study preceding this research, aiming to apply the baroco problem set for evaluating llms by re- ## evaluation tasks we introduce three types of tasks for evaluating llms using the neubaroco dataset: multiple-choice, nli, and translate-and-explain. multiple-choice the multiple-choice task is a format widely used in cognitive psychology. in this task, models are presented with two premises and asked to choose the correct conclusion from five options. these options include the all, some, no, and some-not sentences, as well as the "none of them" choice. ## related work recent years have seen active research on machine learning and deep learning models that focus on logical reasoning in natural language, including syllogistic reasoning. @xcite investigated the abilities of nli models in handling various logical inferences involving boolean operators, quantifiers, conditionals, and negation using synthetically generated data. @xcite studied monotonicity inferences. monotonicity inferences are simpler than syllogistic inferences in that they only have single premises, whereas syllogisms involve multiple premises with challenging combinations of quantifiers and negation. @xcite conducted an empirical study to explore the detection of formally valid inferences within con-trolled fragments of natural language, designed to increase satisfiability problem complexity. these studies combine pre-training and fine-tuning with relatively large datasets for logical reasoning and are not aimed at evaluating current llms based on in-context learning. as datasets for learning and evaluating syllogistic reasoning, @xcite and @xcite have constructed datasets using linguistic resources such as wordnet and their own lists of words, employing template-based automatic generation methods based on types of syllogisms. the dataset of @xcite includes threeclass labels (entailment, contradiction, and neutral) but does not include additional information related to reasoning biases. avicenna @xcite is a crowdsourced dataset containing binary labels that indicate whether the conclusion of a syllogism follows from two given sentences (and if so, the conclusion sentence is also provided), but it does not contain information about reasoning biases. syllobase @xcite contains five types of syllogisms. it consists 50,000 syllogism samples automatically generated from existing knowledge bases, with 1,000 of these samples manually annotated as a test set. experiments were conducted in zero-shot and few-shot settings, covering both generation and selection tasks. all of the datasets above are in english. as a framework for generating deduction datasets, fld @xcite and its japanese version, jfld @xcite , have been proposed. @xcite empirically verifies that language models trained with fld demonstrate enhanced generalizable deductive reasoning abilities. @xcite and @xcite are the works closely related to ours. @xcite focused on belief biases in syllogistic reasoning and showed that the alignment of conclusions with human beliefs affects the performance of llms. in their studies, scenarios were classified based on whether the content of the conclusion contradicts our beliefs, does not contradict, or consists of meaningless words. llms were then tasked with determining whether a combination of two premises and one conclusion constitutes valid or invalid syllogistic reasoning in a binary choice format. this approach differs from the three-class classification used in our work. our research aims to further explore reasoning biases in llms by examining various types of biases, particularly those related not only to the content of reasoning, as highlighted by belief bias, but also to the form of reasoning. additionally, we investigated these biases in experimental settings, including the multiple-choice and translate-and-explain tasks. @xcite , focusing on syllogisms, investigates how language models perform logical reasoning compared to humans. experiments were conducted using the palm 2 models, and the results indicate that larger models perform more accurately than smaller models and humans. it was also confirmed that even the largest models tend to make errors that reflect human reasoning biases in certain types of syllogisms. ando et al. ( 2023 ) is a preliminary work that precedes our research, reporting only the zero-shot performance of gpt-3.5 across 375 nli tasks. in contrast, our study reports results on 790 nli tasks, 80 multiple-choice tasks using chain-of-thought prompts, and 90 translate-and-explain tasks. furthermore, we evaluated a broader range of models, including gpt-4, llama-2 (13b and 70b), and swallow (13b and 70b), in both zero-shot and fewshot settings. the findings related to the multiple-choice and translate-and-explain tasks represent entirely new contributions of this study. ## summary and conclusion in this paper, we evaluated the logical reasoning ability of llms by developing the neubaroco dataset, which consists of syllogisms in both japanese and english, annotated with information on reasoning biases and results from large-scale human evaluation data. the results of experiments using the nli task demonstrated that some llms, particularly gpt-4, achieved high accuracy in both english and japanese, especially in the few-shot setting. however, significant room for improvement remains in problems where the entailment relation is neutral (neither entailment nor contradiction). the results suggest that, particularly in problems labeled as conversion and atmosphere, there is a tendency to exhibit the same reasoning biases as humans. moreover, in the experiments using the multiple-choice task, a method commonly used in psychological studies of syllogisms, some models achieved accuracies surpassing those of human participants in large-scale experiments. it is important to note that the methodology for comparing human accuracy with llms is not yet fully established, and a detailed comparison with human performance remains a future challenge. finally, in the translate-and-explain task, which requires providing explanations of reasoning along with translations of syllogisms into logical expressions, many models showed improved accuracy, with translation accuracy nearly reaching 100%. however, reasoning errors remain, suggesting that the source of these errors is not the misinterpretation of the premise sentences but rather the reasoning process itself. while syllogisms represent one of the basic forms of logical reasoning used in psychology, expanding our evaluation to include more diverse and complex natural language inferences that induce reasoning biases remains an essential challenge. this includes boolean propositional inferences @xcite , if-then conditionals @xcite , and spatial inferences involving polyadic relations @xcite . the biases we addressed are not specific to syllogistic reasoning and, therefore, may potentially be generalized to tasks beyond syllogistic reasoning. addressing these issues in future research is crucial for advancing our understanding of the reasoning capabilities of llms.