A Systematic Comparison of Syllogistic Reasoning in Humans and Language Models (2024)

Abstract: A central component of rational behavior is logical inference: the process of determining which conclusions follow from a set of premises. Psychologists have documented several ways in which humans’ inferences deviate from the rules of logic. Do language models, which are trained on text generated by humans, replicate such human biases, or are they able to overcome them? Focusing on the case of syllogisms—inferences from two simple premises—we show that, within the PaLM 2 family of transformer language models, larger models are more logical than smaller ones, and also more logical than humans. At the same time, even the largest models make systematic errors, some of which mirror human reasoning biases: they show sensitivity to the (irrelevant) ordering of the variables in the syllogism, and draw confident but incorrect inferences from particular syllogisms (syllogistic fallacies). Overall, we find that language models often mimic the human biases included in their training data, but are able to overcome them in some cases.

## introduction the capacity to reason deductively-that is, to determine which inferences, if any, follow from a given set of premises-is central to rational thought @xcite @xcite . despite the importance of this capacity, human reasoning often displays systematic biases (gigerenzer and † work done when te was a student researcher at google research. * svs and tl are joint senior authors. author contributions: te, svs, and tl co-led the project. te conducted the experiments and analysis. te, mht, id, fs, svs, and tl helped with project framing, analysis and suggesting experiments. id and svs offered technical guidance and help with engineering. te wrote the paper with help from mht, id, fs, svs, and tl. correspondence: eisape@mit.edu, svansteenkiste@google.com, linzen@google.com. @xcite @xcite . in recent years, language models (lms) trained with self-supervised objectives have been reported to display a range of capabilities, including the ability to reason @xcite @xcite . does lms' logical reasoning follow the rules of logic to a greater extent than humans'? to the extent that lms' reasoning deviates from normative logic, are their biases similar to humans' @xcite ? in this work, we address these questions with a detailed study of a particularly simple caseinferences from pairs of premises, or syllogisms, such as the following: if all bakers are artists, and some bakers are chemists, then: some artists are chemists. in a syllogism, each premise relates two terms with one of four quantifiers (traditionally known as "moods"): all, some, none and some are not. only one term is shared between the premises (bakers in the example above). inference is required to determine if there is a necessary relationship between the two remaining terms (here, artists and chemists) when the premises in question are true. when human participants in experiments are asked to make syllogistic inferences, their responses often deviate from the rules of logic; in fact, for some syllogisms the vast majority of participants draw incorrect inferences @xcite . this could pose a challenge to language models (lms), as they learn from corpora consisting primarily of human-generated texts-texts which, in turn, reflect human beliefs and inferences. is there sufficient signal in the training corpus to steer lms away from ( often incorrect we address this question in a detailed comparison between the palm 2 family of transformer lms @xcite and studies from cognitive psychology, as well as a replication with the llama 2 family of transformer lms @xcite . we report the following results: overall, we find that palm 2 lms replicate many of the human biases discovered in psychology studies, consistent with the fact that lms are trained on human-generated text. for some syllogisms, however, sufficiently large models overcome those biases and achieve dramatically better accuracy than humans, although their overall accuracy is still far from the perfect logical reasoner. ## interpreting language models using mental models theory we next analyze the behavior of palm 2 models using the mental models theory of human logical reasoning @xcite ), which has been developed over decades to account for human data. the theory takes humans to be resource-limited and simulation-based reasoners @xcite @xcite , with a potentially high degree of variability across individuals. the implementation we use-mreasoner foot_1 (khemlani and johnson-laird, 2022)-captures these aspects of human reasoning with a small set of interpretable hyperparameters that enable it to construct, refine, and draw conclusions from internal mental models of the situations described in a syllogism. mental models consist of sets of entities instantiating the premises, where an entity is represented by a conjunction of logical properties. for example, figure 7 illustrates a mental model corresponding to the syllogism some artists are bakers, some bakers are chemists. this model consists of just three entities, the first of whom is an artist who is also a baker and a chemist, the second is an artist and a baker who may or may not be a chemist (this uncertainty is represented in the figure with a blank space), and so on. the reasoner constructs and maintains its mental model with a set of actions parameterized by four hyperparameters: figure 7 illustrates how mreasoner might process the syllogism some artists are bakers, some bakers are chemists. first, it constructs a mental model, with length governed by len and content governed by broad, consisting of the entities mentioned above: an artist-baker-chemist, an artistbaker, and an artist. the conclusion some artists are chemists is consistent with this particular model (i.e., the first entity is both an artist and a chemist). this conclusion is not true in every model that is consistent with the premises, and as such it is not logically valid; however, if the reasoner does not trigger a system 2 process, it will (incorrectly) take this conclusion as valid and return it. alternatively, with probability systm2 mreasoner will scrutinize the conclusion by amending its model in an attempt to find a counterexample. in this case, mreasoner successfully finds a counterexample by breaking the first entity into two entities that are still consistent with the premises but are not consistent with some artists are chemists; consequently, mreasoner corrects its answer to "nothing follows". mapping lm predictions onto cognitively meaningful dimensions. syllogistic reasoning behavior is high-dimensional; in the set of syllogisms and conclusions we consider, there are 27 syllogisms and eight possible responses to each, for a total of 216 dimensions. we instantiate 1296 mreasoner models, one for each point in a parameter grid, and analyze the 923 of them that finished simulations before timing out (for the details of the parameter grid, see characterizing the space of reasoning behaviors described by mreasoner. although mreasoner is characterized by four parameters, we find a single principal component (pc 1) that captures 77% of the variance in the model's behavior. this component loads heavily on systm2 and, to a lesser degree, on weaken (figure 8 ). following the terminology of @xcite , we view this dimension as representing deliberative reasoning. similarly, pc 2 loads heavily on broad. this dimension, however, describes much less of the behavioral variance of mreasoner. ## discussion human-like reasoning or accurate reasoning? because of humans' systematic reasoning errors, syllogistic reasoning is a particularly clear demonstration of the tension between the two central aims of artificial intelligence: human-likeness and accuracy. we hypothesize that for most applications, accuracy is more important than human-likeness; one notable exception is cognitive modeling, where the goal is to better understand human reasoning by developing models that reason like humans. we consider this application to be an important direction for future work. why are lms more accurate than humans? lms learn from human-generated text, which is likely to reflect human beliefs and biases; it is natural to hypothesize that the language modeling objective would incentivize lms to replicate those biases. we find only partial support for this hypothesis. while the largest model's responses are indeed slightly more correlated with human responses than the smaller ones, for some syllogisms where humans reason very poorly, the models overcome human biases and reason correctly. one possible explanation for this finding is that the data that palm 2 models were trained on includes not only natural language text, but also source code @xcite , which may teach models to reason more effectively. the effect of the composition of the lm's training corpus can be tested in a controlled comparison in the future. cognitive science for lm interpretation. we have used cognitive science to shed light on lm reasoning in two ways. first, we used the biases documented in the cognitive psychology literature as hypotheses for the biases that lms might acquire. this approach is motivated by the hypothesis that because lms are trained on human-generated texts, which reflect human biases and beliefs, they will be incentivized to replicate those biases to improve perplexity. we found partial support for this hypothesis: larger lms were more calibrated to human responses in some cases, in particular in our analysis of the correlation between accuracy and entropy (section 4.2). the second and more novel way in which we use cognitive science is in interpreting lm behavior using a computational cognitive model developed to explain human reasoning. under the assumption that lm reasoning follows the same heuristic strategies as humans do (section 5)-an assumption which, again, is informed by the fact that lms learn from text generated by humans-we concluded from this analysis that lms become more deliberative as their size increases. ## conclusion do lms learn to reason correctly from selfsupervised learning alone, even though much of their training data was produced by humans, whose reasoning often deviates from normative logic? we have addressed this question through a detailed examination of the syllogistic reasoning behavior of the palm 2 family of lms. we find that the largest lms make significantly fewer mistakes than humans but still display systematic errors (section 4.1), and that while their mistakes are only partly aligned with human errors, lms are susceptible to several qualitative reasoning biases shown by humans (section 4.2). eliciting lm reasoning. the space of possible ways to evaluate lms on paradigms from human experiments is fairly large. one can generate from the model @xcite , as we did; elicit meta-level judgements @xcite ; or simply compare the probabilities assigned by the lm to possible continuations @xcite . evaluations can be done in a zero-shot way, as we did, or in a few-shot way, which may better approximate the training phase used in some human reasoning experiments, such as @xcite ; for discussion, see @xcite . finally, generative approaches can rely on a large set of possible prompts, and can be used with or without "chain-of-thought" statements encouraging the model to reveal its reasoning process @xcite . following preliminary experiments (appendix b), we focused on zero-shot chain-of-thought; a more systematic evaluation of the different elicitation approaches would be an important direction for future work. the focus on mental models theory. in section 5, we used a particular cognitive model, the mental models theory, to interpret lm reasoning behavior. this is not the only possible mechanism that might underlie lm reasoning. other accounts of human reasoning have argued that people do, in fact, apply normative logic rules @xcite , perform probabilistic inference with constrained resources @xcite , or combine probabilistic, heuristic and pragmatic reasoning @xcite ; and it is possible that lms reason in a way that does not match any of these theories. we leave a systematic comparison of the fit of each of these theories to lm reasoning for future work. figure 10: valid conclusions for each syllogism. conclusion identifiers show the conclusion mood (see