Reasoning Circuits in Language Models: A Mechanistic Interpretation of Syllogistic Inference (2025)

Abstract: Recent studies on reasoning in language models (LMs) have sparked a debate on whether they can learn systematic inferential principles or merely exploit superficial patterns in the training data. To understand and uncover the mechanisms adopted for formal reasoning in LMs, this paper presents a mechanistic interpretation of syllogistic inference. Specifically, we present a methodology for circuit discovery aimed at interpreting content-independent and formal reasoning mechanisms. Through two distinct intervention methods, we uncover a sufficient and necessary circuit involving middle-term suppression that elucidates how LMs transfer information to derive valid conclusions from premises. Furthermore, we investigate how belief biases manifest in syllogistic inference, finding evidence of partial contamination from additional attention heads responsible for encoding commonsense and contextualized knowledge. Finally, we explore the generalization of the discovered mechanisms across various syllogistic schemes, model sizes and architectures. The identified circuit is sufficient and necessary for syllogistic schemes on which the models achieve high accuracy ( \geq 60%), with compatible activation patterns across models of different families. Overall, our findings suggest that LMs learn transferable content-independent reasoning mechanisms, but that, at the same time, such mechanisms do not involve generalizable and abstract logical primitives, being susceptible to contamination by the same world knowledge acquired during pre-training.

## introduction language models (lms) have led to remarkable results across various natural language processing tasks @xcite @xcite @xcite . this success has catalyzed research interest in systematically exploring the reasoning capabilities emerging during pre-training (clark et al. @xcite . recent findings suggest that logical and formal reasoning abilities may emerge in largescale models @xcite @xcite or through transfer learning on specialized datasets @xcite . however, ongoing debates question whether these models apply systematic inference rules or reuse superficial patterns learned during pre-training @xcite @xcite . this controversy underscores the need for a deeper understanding of the low-level logical inference mechanisms in lms @xcite @xcite . to improve our understanding of the internal mechanisms, this paper focuses on mechanistic interpretability @xcite , aiming to discover the core circuit responsible for syllogistic reasoning. in particular, we focus on categorical syllogisms with universal affirmative quantifiers (i.e., aaa-1, barbara) motivated by two key factors. first, as observed in natural logic studies @xcite , this form of syllogistic reasoning is prevalent in everyday language. therefore, it is likely that lms are exposed to such reasoning schema during pre-training. second, aaa-1 is a form of unconditionally valid syllogism independent of the premises' truth condition @xcite . this characteristic offers a deterministic and scalable task design, other than allowing us to investigate the disentanglement between reasoning and knowledge representation @xcite @xcite . through mechanistic interpretability techniques such as activation patching @xcite and embedding space analysis (i.e., logit lens) @xcite @xcite , we investigate the following main research questions: rq1: how is the content-independent syllogistic reasoning mechanism internalized in lms during pre-training?; rq2: are contentindependent mechanisms disentangled from specific world knowledge and belief biases?; rq3: does the core reasoning mechanism generalize across syllogistic schemes, different model sizes, and architectures? to answer these questions, we present a methodology that consists of 3 main stages. first, we define a syllogistic completion task designed to assess the model's ability to predict valid conclusions from premises and facilitate the construction of test sets for circuit analysis. second, we implement a circuit discovery pipeline on the syllogistic schema instantiated only with symbolic variables ( third, we investigate the generalization of the identified circuit on concrete schemes instantiated with commonsense knowledge to identify potential belief biases and explore how the internal behavior varies with different schemes and model sizes. we present the following overall conclusions: 1. the circuit analysis reveals that lms develop specific inference mechanisms during pre-training, finding evidence supporting a three-stage mechanism for syllogistic reasoning: (1) naive recitation of the first premise; (2) suppression of duplicated middle-term information; and (3) mediation towards the correct output through the interplay of mover attention heads (see figure 1 ). our study is conducted using transformer-lens @xcite on an nvidia a100 gpu with 80gb of memory. the dataset and code to reproduce our experiments are available online to encourage future work in the field foot_0 . ## methodology our main research objective is to discover and interpret the core mechanisms adopted by autopremises (p1, p2) conclusion (c) symbolic all a are b. all b are c. therefore, all a are c. ## mechanistic interpretation empirical setup. we select the lm for circuit analysis based on a trade-off between model size and performance. to this end, we measure the accuracy of gpt-2 @xcite at various sizes (117m, 345m, 762m, and 1.5b) on the syllogism completion task, aiming to identify potential phase transition points where performance changes significantly @xcite . we observe a marked transition from small to medium sizes, where average performance across three datasets increases by 439.06% (see figure 6(a) , appendix e). the shift is even more pronounced under the logit difference metric (see section 4.4 and figure 6(b) , appendix e). based on these findings, we select gpt-2 medium-whose architecture is summarized in appendix f-for circuit discovery, and subsequently evaluate generalization across model sizes and architectures (sections 4.3 and 4.4). ## circuit evaluation the circuit is sufficient and necessary for symbolic arguments. to evaluate the comprehensive correctness of the circuit, we assess necessity and sufficiency via the ablation method described in section 2. the ablation study in figure 4(a) shows that the identified circuit is both necessary and sufficient, demonstrating a consistent performance degradation when removing circuit components and revealing a complete restoration of the original model's performance when considering only the circuit's subcomponents. the circuit is robust to superficial variations. we further verify the robustness of the symbolic circuit to superficial and semantic-preserving perturbations. in particular, we modify the letters into numbers (figure 4(b) ) and adopt semantically equivalent quantifiers and related verbs (e.g., "all (b) (a) accumulated ablated (added) heads logit difference figure 5: circuit ablation results for evaluating transferability on non-symbolic datasets: (a) the beliefconsistent and (b) the belief-inconsistent. the dashed line represents the baseline logit difference without knockouts. in both cases, the symbolic circuit is necessary, as ablating it reduces performance. however, the circuit is only sufficient for belief-consistent data, as recovery fails in belief-inconsistent settings, suggesting interference from belief-bias. ## related work mechanistic circuit analysis has emerged as a promising approach to interpreting the internal mechanisms of transformers @xcite @xcite @xcite . existing approaches investigating internal reasoning mechanisms mainly focus on mathrelated tasks, elucidating the information flow for answering mathematical questions @xcite and examining arithmetic operations @xcite . most pertinent to our work, @xcite provides a mechanistic interpretation of multiple-choice question answering, investigating attention head-level patterns. in general, our mechanistic analysis complements recent work investigating the challenges in processing reasoning arguments that contradict established beliefs and whether the reasoning in lms stems from internalized rules or memorized content @xcite @xcite @xcite @xcite @xcite @xcite . to address this question, different mechanistic approaches have been adopted to localizing factual associations @xcite @xcite or assessing conditions for generalization @xcite . however, to the best of our knowledge, we are the first to investigate mechanisms for syllogistic reasoning. ## conclusion in this study, we presented a comprehensive mechanistic interpretation of syllogistic reasoning in language models. by combining activation patching, embedding space analysis, and circuit ablation, we uncovered a structured error-correction mechanism characterized by the suppression of duplicated middle terms and the propagation of relevant information via mover heads. our findings show that this circuit is necessary and sufficient for symbolic syllogistic reasoning. moreover, we demonstrated that, while this reasoning mechanism generalizes across syllogistic schemes, model architectures and sizes, it remains susceptible to contamination from belief biases when instantiated with natural language inputs. overall, our findings suggest that lms learn transferable reasoning mechanisms but that, at the same time, such mechanisms might be contaminated and suppressed by the same world knowledge acquired during pre-training. further studies will be required to understand how the discovered symbolic circuit can be effectively disentangled from world knowledge to enable systematic generalization in lms' reasoning.