Theorem-of-Thought: A Multi-Agent Framework for Abductive, Deductive, and Inductive Reasoning in Language Models (2025)

Abstract: Large language models (LLMs) have shown strong performance across natural language reasoning tasks, yet their reasoning processes remain brittle and difficult to interpret. Prompting techniques like Chain-of-Thought (CoT) enhance reliability by eliciting intermediate reasoning steps or aggregating multiple outputs. However, they lack mechanisms for enforcing logical structure and assessing internal coherence. We introduce Theorem-of-Thought (ToTh), a novel framework that models reasoning as collaboration among three parallel agents, each simulating a distinct mode of inference: abductive, deductive, and inductive. Each agent produces a reasoning trace, which is structured into a formal reasoning graph. To evaluate consistency, we apply Bayesian belief propagation guided by natural language inference (NLI), assigning confidence scores to each step. The most coherent graph is selected to derive the final answer. Experiments on symbolic (WebOfLies) and numerical (MultiArith) reasoning benchmarks show that ToTh consistently outperforms CoT, Self-Consistency, and CoT-Decoding across multiple LLMs, while producing interpretable and logically grounded reasoning chains. Our findings suggest a promising direction for building more robust and cognitively inspired LLM reasoning. The implementation is available at https://github.com/KurbanIntelligenceLab/theorem-of-thought.

## introduction large language models (llms) have achieved impressive performance across a wide range of natural language understanding and generation tasks @xcite , enabled by advances in in-context learning @xcite , instruction tuning @xcite , and chain-of-thought * corresponding author: hkurban@hbku.edu.qa (cot) prompting @xcite . these methods have extended llms' capabilities to handle complex forms of reasoning, including mathematical, logical, and commonsense inference. despite these advances, llm reasoning remains shallow and unreliable. existing approaches often rely on single-shot or sampling-based decoding along linear reasoning paths, making them susceptible to hallucinations @xcite , logical inconsistencies (uceda @xcite , and weak generalization @xcite . methods such as cot and self-consistency @xcite encourage intermediate steps and majority voting across sampled outputs, but lack mechanisms to verify internal coherence and model the logical structure of reasoning. as a result, outputs may appear fluent and plausible while remaining logically unsound. this brittleness contrasts sharply with human reasoning, which is inherently multifaceted. drawing on insights from cognitive science @xcite , we observe that human inference typically blends three complementary modes-abduction, deduction, and induction-that support explanation, derivation, and generalization. however, llms typically conflate these distinct processes into a single, undifferentiated flow, limiting both interpretability and reliability. to address this gap, we propose theorem-of-thought (toth), a framework that models diverse reasoning strategies through structured, verifiable interactions. toth employs three specialized agents, each emulating a distinct cognitive mode: each agent independently generates a reasoning trace, which is transformed into a formal reasoning graph (frg)-a directed graph where nodes represent intermediate conclusions and edges capture logical dependencies. we evaluate the internal consistency of each frg using bayesian belief propagation, with edge confidence scores calibrated via a natural language inference (nli) model. a composite score balancing average belief and logical entropy is used to select the most coherent graph, from which the final answer is extracted. ## related work prompt-based reasoning in llms. a growing body of work explores prompting strategies to enhance the reasoning capabilities of llms. cot prompting @xcite encourages models to decompose problems into intermediate steps, guiding reasoning along a linear path. building on this, auto-cot @xcite automates prompt generation by sampling diverse questions and producing corresponding reasoning traces, reducing manual effort. beyond prompt generation, several works focus on optimizing prompt selection strategies. activeprompt @xcite identifies high-uncertainty instances for annotation, improving data efficiency and reasoning robustness through active learning. more recent approaches introduce explicit structure into the reasoning process. tree-of-thought (tot) @xcite enables multi-path exploration with internal evaluation, while graph-of-thought (got) @xcite structures reasoning as a graph to better model dependencies between steps. instruction tuning for reasoning. instruction tuning and knowledge distillation offer alternative approaches to eliciting reasoning in llms without relying on explicit prompts @xcite @xcite . while effective, these methods typically require computationally intensive fine-tuning on large-scale datasets annotated with reasoning traces and cot examples, which are often costly and domain-specific. recent work has explored more indirect supervision strategies. for instance, @xcite introduce proxy tuning, which leverages auxiliary models to contrast a base llm with its adapted variant. although this approach reduces the need for direct supervision, it still assumes access to cot-like outputs and pre-aligned reasoning benchmarks. ## methodology toth is a graph-based reasoning framework designed to enhance the accuracy, interpretability, and generalization capabilities of llms on complex tasks. it decomposes reasoning into three modular agents, each simulating a classical inference paradigm-abduction, deduction, and induction. each agent produces a structured reasoning trace, which is composed into a frg. final answers are derived via nli-calibrated bayesian belief propagation and composite graph scoring. the full pipeline is depicted in fig. 1 . toth differs from prior reasoning paradigms along three axes: architecture, supervision, and verification. prompt-based methods (e.g., cot, tot, got) elicit reasoning via linear or loosely structured traces, yet lack mechanisms for enforcing logical consistency. instruction-tuned models embed reasoning behavior through fine-tuning on annotated traces, often requiring large datasets and figure 1 : overview of the theorem-of-thought (toth) reasoning pipeline. a question is independently processed by three agents, each using a distinct reasoning style: abductive (type 1), deductive (type 2), and inductive (type 3). each agent produces a structured reasoning graph, which is scored via bayesian confidence propagation. abduction infers the best hypothesis h given observations o and knowledge k (i.e., arg max h p (h | o, k)); deduction derives a conclusion c from premises {p 1 , . . . , p n } (i.e., {p i } ⊢ c); induction generalizes from examples {x 1 , . . . , x n } to a rule r (i.e., {x i } ⇒ r). the highest-scoring graph contributes its final node as the answer. ✓ and ✗ indicate whether a given agent's output was selected. ## experiments data. toth was evaluated on two representative reasoning benchmarks. multiarith @xcite targets compositional numerical inference through multi-step arithmetic word problems. we-boflies @xcite , part of the big-bench-hard suite, involves determining truth values among logically entangled symbolic statements. these datasets are known to challenge llms under direct prompting @xcite , making them suitable for testing structured reasoning capabilities. ## conclusion and future work this work presents theorem-of-thought (toth), a graph-based reasoning framework that integrates abductive, deductive, and inductive inference through a modular multi-agent design. each agent generates structured reasoning traces, which are composed into formal graphs and verified using nli-calibrated bayesian confidence propagation. this approach supports both accurate prediction and interpretable, logically grounded reasoning. empirical evaluations on symbolic and numerical benchmarks demonstrate that toth consistently outperforms strong prompting and decoding baselines, particularly in scenarios requiring structured logical inference. toth introduces a new paradigm in reasoning with language models by treating inference as a verifiable, compositional process, rather than a monolithic generation task. future research will explore dynamic agent routing based on input characteristics, inter-agent collaboration protocols, and adaptive trust estimation via fine-tuned and ensemblebased nli models. extending the framework to scientific hypothesis validation, law and policy reasoning, and multimodal domains such as visual question answering represents a promising direction for advancing general-purpose, verifiable reasoning in large language models.