MAD-Logic: Multi-Agent Debate Enhances Symbolic Translation and Reasoning
Authors: Haocheng Yang
Fengxiang Cheng
Tianjun Yao
Mengyue Yang
Jiajun Chai
Xiaohan Wang
Guojun Yin
Wei Lin
Soummya Kar
Fenrong Liu
Haoxuan Li
Yisen Wang
Venue: ICLR 2026 Poster (2026)
Keywords: Logical Reasoning
Symbolic AI
Multi-agent System
Large Language Models
Primary area: neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)

Abstract: Large language models (LLMs) struggle with complex logical reasoning. Previous methods can be briefly summarized into two pipelines: (1) translating natural language (NL) to symbolic language (SL) then reasoning via external solvers, and (2) adopting LLMs to reason directly in NL based on prompting or fine-tuning.  However, we point out that on the one hand, the translation relying on a specific SL often fails to capture different important features of raw NL, leading to information loss or translation errors. On the other hand, both two pipelines have unignorable limitations. For example, the former (SL-based) methods are highly sensitive to imperfect translation, and the latter (NL-based) methods are prone to hallucinations.   Motivated by this, we are the first to propose a multi-agent debate framework to leverage the strengths of different SLs and reasoning methods, achieving better performance in both translation and reasoning stages. Specifically, in the translation stage, multiple agents translate the NL into different SL and refine translations through debate. In the reasoning stage, multiple agents based on SL (obtained by the corresponding solver) and NL debate multiple rounds, with the final answer determined by majority vote. In addition, to address the inefficiency of multi-agent debates, we introduce an adaptive sparse communication strategy that prunes unnecessary interactions based on agent confidence and information gains. Extensive experiments on three datasets show that our method enhances logical QA performance while reducing computational cost.

Published as a conference paper at ICLR 2026
MAD-LOGIC: MULTI-AGENT DEBATE ENHANCES SYMBOLIC TRANSLATION AND REASONING
Haocheng Yang!. 1,2.3 Fengxiang Cheng
Tianjun Yao
Mengyue Yang" Jiajun Chai
Xiaohan Wang' Guojun Yin?
Wei Lin
Soummya Kar" Fenrong Liu
Haoxuan Li 1 Yisen Wang
'Peking University "NUS 'Meituan *UvA .MBZUAI UUniversity of Bristol
"CMU TTinghuu University 'University of Oxford IInsiitute for Decentralized AI
haocheng_yaaae yang@u. nus edu, irroonggtsinnguua edu.eddu.cm cn
hxl i iisst...pkke edu. cn, wang@.ka edu.cn
ABSTRACT
Large language models (LLMs) struggle with complex logical reasoning. Previ- ous methods can be briefly summarized into two pipelines: (1) translating natural language (NL) to symbolic language (SL) then reasoning via external solvers, and (2) adopting LLMs to reason directly in NL based on prompting or fine-tuning. However, we point
that the one hand, the translation relying on spe- cific SL often fails to capture different important features of raw NL, leading information loss or translation errors. On the other hand, both two pipelines have unignorable limitations. For example, the former (SL-based) methods are highly sensitive to imperfect translation, and the latter (NI-based) methods are prone to hallucinations. Motivated by this, we are the first to propose a multi- agent debate framework to leverage the strengths different SLs and reasoning methods, achieving better performance in both translation and reasoning stages. Specifically, in the translation stage, multiple agents translate the NL into differ- ent SL and refine translations through debate. In the reasoning stage, multiple agents based on SL (obtained by the corresponding solver) and NL debate mul- tiple rounds, with the final answer determined by majority vote. In addition, address the inefficiency of multi-agent debates, we introduce an adaptive sparse communication strategy that prunes unnecessary interactions based on agent con- fidence and information gains. Extensive experiments on three datasets show that our method enhances logical QA performance while reducing computational cost.
INTRODUCTION
Large language models (LLMs) have demonstrated exceptional capabilities across a wide range of tasks. However, they still face significant challenges when performing complex logical reasoning. limiting their applicability in real-world scenarios (Cheng et al.] 2025). There are two categories of existing methods for logical question-answering (QA). One type of methods translate natural language (NL) problems into symbolic language (SL), such as logic programming (LP), first-order logic (FOL), or Boolean satisfiability (SAT) format, and then perform reasoning via external logi- cal solver using these symbolic representations, i.e., reasoning in the translated SL (Ye et al.) 2023) Olausson al.) 2023). An alternate type of methods use prompting or fine-tuning strategies (Yao) et al. 2023 Besta et al.) 2024, Zhang et al. 2024) to enable LLMSs to answer logical questions in NL directly, i.e., reasoning in NL. More details about related work is provided in Appendix [A]
Previous methods generally include two stages: the symbolic translation stage and the reasoning stage. In the translation stage, existing works usually translate problem in NL into a single, pre- defined SL (LP, FOL or SAT, etc.). However, each SL varies in expressivity, and only relying on specific SL often fails to capture different important features of raw NL, leading information loss or translation errors (Pan et al.) 2023: Ryu et al. 2025). In the reasoning stage, prior works perform reasoning either via SL solver via LLMs, which leads trade-off. For SL solvers, though enabling rigorous reasoning, they may fail to return a valid output when translations are imperfect (Feng et al.
"Fenrong Liu, Haoxuan Li, and Yisen Wang are the corresponding authors.

---

Published as a conference paper at ICLR 2026
2024
2025a). i.e., strong reasoning, weak robustness. For direct reasoning via LLM, it can tolerate inaccurate translation, but is prone to hallucinations or logical inconsistencies in LLM itself (Liu et al. 2023a; Xu et al. 2024 2025a i.c., strong robustness, weak reasoning. Consequently, existing methods in single-agent struggle to simultaneously achieve strong logical reasoning and robustness to translation errors.
Callewaert al. 2025 Liu et al.
To address this issue, we are the first to propose an extended multi-agent framework to leverage the strengths of different symbolic languages and reasoning methods, achieving better performance in both translation and reasoning stages. Specifically, in the translation stage, we employ multiple agents, with each agent responsible for translating NL to specific SL, and then correct the trans- lation errors through mutual refinement, ultimately enhancing the accuracy of the translation. In the reasoning stage, assign multiple agents perform reasoning process by using SL via solvers and NL via LLMs, and prompt them to debate multiple rounds, where they can benefit each other reasoning to achieve optimal reasoning performance.
Moreover, deploying a multi-an agent debate framework suffers computational overhead and token consumption (Du et al. (2023). particularly when debates involve repetitive exchanges or redundant information sharing. To address this inefficiency, we propose an adaptive sparse communication strategy that prunes unnecessary communication by assessing the agent's confidence and informa- tion gains, allowing each agent to selectively retain only the most valuable outputs from others.
The main contribution of this paper can be summarized as follows:
We analyze the complementarity between SL and NL reasoning paradigms, as well as the com- plementarity within various systems and NI reasoning approaches.
We are the first to propose multi-agent approach with an adaptive sparse communication mech- anism for logical reasoning, which not only enables the absorption of advantages from multiple reasoning methods through debate but also optimizes computational efficiency and cost. Extensive experiments on three datasets demonstrate our method can improve the performance of logical QA while reducing the computational cost.
LOGICAL QUESTION ANSWERING PROBLEM SETUP
The task of logical question answering requires determining if conclusion be validly inferred from a provided set of facts and rules. For this type of problem, the model's objective is to classify the statement as true, false, or unknown. This challenge illustrated by the example below, taken from the ProofWriter dataset (Tafjord et al. 2021)
Premises:
The bear chases the squirrel. The bear not cold. The bear visits the cat. The bear visits the lion. The cat needs the squirrel. The lion needs the cat. The squirrel needs the lion. If something visits the lion then visits the squirrel. something chases the cat then the cat visits the lion. Rules:
If something visits the squirrel and it needs the lion then the lion does not chase the bear.
If something is round and visits the lion then the lion is not cold.
If something visits the squirrel then it chases the cat.
If the cat does not chase the bear then the cat visits the bear.
If something visits the squirrel then it is not nice.
If the bear big then the bear visits the squirrel.
Question: Based on the above information, is the following statement true, false, or unknown? The squirrel does not need the lion. Options: A) True B) False
C) Unknown
Answer: B
Even with recent advancements, LLMs continue to face significant difficulties with logical reason- ing, evidenced by their limited performance. For instance, prior work achieved only approximately 80% accuracy on ProofWriter (Xu et al. 2025a)
2

---

Published as a conference paper at ICLR 2026
Example Question
Solve with SL translated via Debate
Premises:
Monn Wyse)
Hamma
Answer:
1.All meniraals are verietratee.
Marmma W00 VertabesseO))
Marmea Veretriees
Reason
knswnn sulic
2.1 an animal a vertebrate, has bones. he cal is mammell
Query
Ouery
Pyka
Maammnd for Truel bock
Presieml
Preszess
Answer:
ve (etrwookoo HHodo vog
va (Pendisat0)) Hoodees0()
Reason broomraa))) (nounpliing
Question:
The cat have bones?
FOL
Condicaion
Condiaaion
Prover9
Options:
Clesee Narmma (20)
Desses:
Answer: A
True
Recson: sepp unea comns cathe Bodison varlb
B. False
Unknown
SAT
Sonee cot
Blxes
Query:
mad oa)
Dabata for Symbolic Translation
Symbolic Language
Solve with NL
Improve Accuracy via Reasoning Debate
Anseeer: Ressontuat's Mp by day
C
LLM
Round 1
Round N Cii
Final Anewer: A. True
Answee:
Debate
Majority Vole
Recsonnrkun wwtornottttottt
ven ceemtus
A
LLM
Figure 1: Overview of our sparse multi-agent debate framework for logical reasoning.
PROPOSED METHOD
3.1 OVERVIEW
To address the limitations of existing single-agent logical reasoning methods based on SL or NL, we propose a multi-agent debate framework. Specifically, as shown in Figure [ we first translate NL logical questions into multiple SI., such logic programming (L.P), first-order logic (FOI.), and Boolean satisfiability (SAT). Agents debate to refine their translations, ensuring translation accuracy for subsequent SL-based solving with solvers such as Pyke, Prover9, and Z3. Mcanwhile, we adopt LLMs to directly solve the NL logical question based on the Chain-of-though and Plan-and-Solve techniques. Finally, agents based on results from SL and NL perform debates in multiple rounds to absorb the strengths of various methods, and majority vote among agents is used to determine the final answer. Additionally, sparse communication mechanism is proposed to optimize the efficiency and cost of multi-agent interactions.
3.2 DEBATE FOR SYMBOLIC TRANSLATION OF LOGICAL QA
To perform logical reasoning in structured and unambiguous format, we begin by converting the raw natural language question into a formal symbolic expression. As illustrated in the top of Fig- ure this process first translates logical reasoning question into three distinct symbolic languages (LP, FOL, and SAT) in parallel, then leverages multi-agent debate to refine the final translations to improve the translation accuracy. In the following, we briefly introduce LP, FOL and SAT with their mutually different advantages and shortcomings, which motivates us to use them simultaneously.
Logic Programming (LP). Logic programming is tailored for rule-based deduction, providing systematic framework for forward or backward inference chains. For example, a rule could be represented has jas_parent(xx, y) haas_parent(y,2 > has mas__gaannprret(x,, z). While LP excels in its brief and efficient deduction, its expressiveness is constrained to rule-based problems.
First-Order Logic (FOL). First-Order Logic provides highly expressive framework of representing complex relations and universal quantifiers. A typical expression might be VrVVyLovves(xx)) - -Hales(x,y)) FOL's power lies in its ability to model intricate logical struc- tures, but limited to the computational complexity for large-scale problems.
Boolean Satisfiability (SAT). SAT formalizes problem constraints, solvable by highly optimized solvers. An example = Write(Cat), B = Write(Deer), Black(Cat),, (A v B) (-A V C). This approach is extremely efficient for
set of Boolean variables and
3

---

Published as a conference paper at ICLR 2026
constraint- problems, though its limited expressiveness makes it unsuitable for complex, Boolean logical relationships.
3.3 DEBATE FOR REASONING SYMBOLIC LANGUAGE AND NATURAL LANGUAGE
Reasoning via Corresponding Logical Solvers. Given the translated symbolic languages such as IP, FOI., or SAT, solver-based reasoning methods use external logical solvers to perform logical reasoning. Despite the strong symbolic reasoning capabilities of these solvers, their effectiveness is highly sensitive to the accuracy of translation from natural to symbolic language, even minor errors can distort solver outputs (Li et al. 2024at Liu et al., 2025b), and information loss during translation often prevents execution, rendering the problem unsolvable (Fegg et al. 2024).
LP example Example: LP Reasoning Extracted from Pyke Solver We have following known rules from the context:
rulel: Sees(Sx, cat, True) Green($x, False) > Sees(Sx, cow, True)
rule2: Kind(rabbit, True) Sees(rabbit, squirrel, True) Needs(squirrel, rabbit, True)
Now begin reasoning to obtain all implied facts:
Use rulel: Sees($x, cat, True) Green(Sx, False) > Sees(Sx, cow, True)
Use rule2:
(ind(ra True) Sees(rabbit, squirrel, True). Needs(squirrel, rabbit, True)
All newly implied Facts: Cold('cat', True), Cold( cow', True), Eats (aauirru,,,,"",, "cow", True), Rough('ca'', True), Round( cat', False), Round('cows False), Round( Rquirr'' False), Sees("cat', 'rabbit', True), Sees("oow' 'rabbit', True), Sees(" ("euuu'l,,"""" 'rabbit', True)
Reasoning Pipelines in Natural Language. Prompt-based reasoning methods guide I.I.s to ex- plicitly construct logical chains during question answering, thereby producing step-by-step natural language reasoning (Wei et al. 2022] Yao et al. 2023 Zhang et al. 2023 2024). By reasoning di- rectly in natural language, these methods avoid rigid failures caused by symbolic translation errors, thus exhibiting high robustness. However, their reasoning ability is limited by the intrinsic capac- ity of LLMs, making them prone errors complex tasks, while repeated multi-step calls to the model incur substantial computational costs (Yang et al., 2023).
Multi-agents' Debate to Improve the Accuracy of Reasoning, Solver-based methods, which exhibit strong reasoning ability but low robustness, and prompt-based methods, which are highly robust but weaker in reasoning, are inherently complementary. This motivates our proposal of multi-agent debate strategy for mutual benefit between these two paradigms, ultimately enhancing reasoning accuracy. Specifically, our approach begins by generating set of initial natural language reasoning narratives. For solver-based method, symbolic reasoning process, encompassing the rules, steps, and conclusions, is visualized comprehensive natural language description, exemplified by a Logic Programming (LP) reasoning text from a Pyke solver. Concurrently, the prompt-based method directly outputs narrative documenting its thought process. Subsequently, the process enters an iterative refinement loop driven by LLM. In each round, the LLM is prompted to rewrite each reasoning narrative, using all other narratives and guide its revision. This procedure is repeated for N rounds (a predefined hyper-parameter), to facilitate deep interaction and mutual calibration. The final answer is then determined by a majority vote on the conclusions from all refined narratives.
the provided context to inform
3.4 IMPROVING EFFICIENCY VIA SPARSE DEBATE FRAMEWORK
To reduce the computational cost, we further introduce a sparse communication strategy, in which communication between agents is dynamically pruned based a preference score. This metric assesses the potential benefit of an interaction between two L.MS in each turn by jointly considering the relative confidence of the agents and the information gains from the opponents.
3.4.1 MULTI-TURN DYNAMIC INTERACTION PREFERENCE BETWEEN LLMS
We establish sparse communication topology to improve the efficiency in multi-turn interactions by dynamic pruning mechanism. which allows source agent to communicate its output to the receiving agent j at round d. Specifically, we propose preference score quantifying the potential
4

---

Published as a conference paper at ICLR 2026
utility of the information in the communication, which is defined as:
C% cj
Pref
X(1 - cos cos( (41
This score comprises two key components. The first / representing the ratio of con- fidence scores between the source agent i and the receiving agent j at round d. The second is 1 cos(A4, A measuring the difference of two outputs, regarded as information gain. The con- fidence score is generated by each LLM agent in the same response turn as its predicted answer. Concretely, every agent is prompted to output: (i) its predicted label, (ii) the reasoning trace, and (iii) scalar confidence value in [0,1] during communication.
To guarantee efficiency, we propose a dynamic strategy to determine which agent should be commu- nicated with. Specifically, in round d, we use this average preference score Pre the adaptive threshold, we define binary communication gate Of-y- Communication from to j is permitted only if the current preference score is greater than equal the historical average, indicating that the current interaction is at least as beneficial as the average past interaction between this pair. The indicator of whether agent i benefits agent at round d is formally defined as:
{
1, Pre
d-I Pre
0
1 0, Pre
Pre d-1
3.4.2 SELECTIVE MEMORY UPDATING VIA SPARSE COMMUNICATION
The sparse communication mechanism directly informs how each agent updates its internal state or memory across debate rounds. Each agent maintains personalized memory that aggregates valuable insights from others. At the beginning of the first round (d = 1). all agents start with an for all pairs). From the
empty memory M < Ø and communication is fully connected (O second round, the sparse communication gate OrL activated. At the end of each round d, every agent updates its memory for the next round M by selectively incorporating the outputs A from only those agents i for which the communication channel was open (i.e., O = 1.. After the memory updated, agent generates its output for the next round A by querying the symbolic question and i's newly updated, personalized memory. After D rounds of debate, the final outputs from all agents AP+1... . A,++1, are aggregated via a majority vote to determine the final answer. The complete sparse communication Algorithm I detailed in Appendix
THEORETICAL ANALYSIS
We formulate Logical QA task multiclass classification problem. Denote the input space X and output space y (21,02, Cx. (k 2 2), where y/ E y denotes the ground-truth label. We have collection of agents H. = {h1, fug ..., }. For any agent h4, assume it is better than random guess, i.c., the overall accuracy p = (h)((x) = y) > 1/k: For simplicity, assume uniform error answer distribution (relaxable non-uniform with minor adjustments). For any two distinct agents h;, hy (i + j), we define the average pairwise class-wise correlation E [0, 1):
Σ
Pab:
ISacbsk
where Pab = :Cv(( Zz,eb)/ /ar(22 at Var (2j,o))) and =()
b).
The correlation p captures how often two learners agree on answer pairs. The majority vote yields an ensemble learner H (z) = arg maxesy I (x) = c), and we take random selection in case of a tie. We can show the accuracy lower bound follows (see Appendix For proofs).
Theorem (Aceuracy Lower Bound for Majority Vote Ensemble). Under the above setting, let 5 p- and note that 8 > O. For any incorrect class + y, define T; I(h+(x) -)) and assume Var(T,) o² and Cov(T; T₃) = poz for i + j, where p is the average pairwise class- wise correlation defined above. Then the accuracy of the majority vote ensemble satisfies:
c)
a² [1 + (m - 1)p]
P(H(x) v) 2 1 (k- 11).
2
In particular, we have
5

---

Published as a conference paper at ICLR 2026
Table 1: Performance comparison across three synthetic benchmarks with Temperature set as O.
GPT4
Claude 3.7 Sonnet
DeepSeck V3
Methods
ProntoQA
ProufWiiter
LogiDeduct
ProntoQA
ProofWriter
LogiDeduct
ProntoQA
Proo/Write
LogiDeduct
Direct
75.40%
53.50%
59.00%
77.00%
70.50%
76.67%
79.20%
68.33%
85.33%
1-shot COT
81.20%
67.17%
69.67%
87.20%
81.50%
82.33%
85.00%
71.83%
83,00%
L.IC
90.40%
80.67%
82.33%
91. 11.20%
83.83%
87.67%
91. 31.00%
84.33%
84.00%
LogicLM
93.40%
17%
87 87.00%
91.80%
76.17%
94.00%
83.20%
80.50%
93.33%
Aristoile
95.80%
87.00%
65.67%
98. 20%%
89.67%
75.33%
94.40%
85.17%
72.33%
SymbCOT
96.00%
82.33%
86.33%
97.20%
34.00%
98
85.83%
94.00%
CR
93.20%
71.67%
80.33%
96.80%
82.83%
86.67%
95.40%
80.33%
83.67%
DetermLR
97.80%
77.33%
85,00%
98.00 98.00%
84.33%
88.33%
96.80%
82.17%
88.33%
MAD
99,80%
89.50%
88.67%
99 80%
92.83%
99.83%
98.00%
92.50%
95.33%
DebtexDebate
99.60%
90.83%
92.33%
99 80%
96.17%
99.67%
99.80%
93.00%
99.67%
Ours (w/ sparse)
99.40%
90.17%
94.00%
100.00%
97.00 %
99. 60.67%
99 80%
92.83%
100,00 %
(w/ sparse)
100.00%
92.00%
94.33%
100.00%
100.00%
100.00
93.33%
100.00%
Table 2: Performance comparison across three real-world benchmarks with Temperature set as O.
GPT-4
DeepSeek-Vi V3
Methods
AR-LSAT
FOLIO
Chinese LogiQA-V2
AR-LSAT
FOLIO Chinese LogiQA-V2
Direct Answer
32.90%
65.20%
62.27%
36.80%
66.18%
74.33%
CoT LogicLM
35.06%
70.59%
65.22%
45.45%
76.96%
77.97%
40.86%
76.96%
25.99%
43.72%
78.92%
28.63%
SymbCoT
42.86%
80.39%
70.57%
47.02%
81.37%
81.98%
CobtexDebate
51.08%
84.80%
74.13%
74.03%
88.73%
83.04%
Ours (w/o sparse)
50.42%
84.31%
74.01%
73.62%
89.22%
85.68%
Ours (w/ sparse)
53.25%
86.27% %
74.76%
75.76%
90.67
86.93%
Table 3: Performance comparison on smaller models using Dxven2.5-7B-Instruct.
Method
ProofWriter ProntoQA LogiDeduct
AR-LSAT FOLIO Chinese LogiQA-V2
Direct Answer
40.17%
55.20%
42.67%
24.24%
32.35%
62.96%
CoT
40.50%
75.60%
40.33%
29.87%
53.24%
59.32%
LogicLM
61. 11.6%%
73.80%
63.00%
14.29%
59.61%
25.33%
SymbCoT
70.17%
80.60%
60.00%
32.03%
57.39%
65.44%
CortexDebate (w/o NL-SL)
47.50%
78.20%
39.67%
19.91%
53.92%
62.96%
CortexDebate (w/ NL- SL)
75.83%
84.00%
67.00%
35.93%
63.73%
66.92%
Ours (w/o sparse)
74.67%
85.20%
68.33%
35.06%
62.24%
67.98%
Ours (w/ sparse)
76.50%
86.40 %
67.67%
37.20%
65.68
68.11% 1
pf 0, then limm-toc g))
If 0, then
0o, the accuracy lower bound converges to 1 1 (k 1) po?
For any e > 0, if (h-1) then there exists me such that for all m > mo. ((H(() = y)
c
This illustrates (i) errors are independent م 0, the lower bound goes to as m - 00, and (ii)
if errors are positively but moderately correlated > 0, the bound converges to (k p) / as m + oc, demonstrating that the majority vote remains well-behaved unless agents are highly correlated, supporting that the heterogeneous SL/NL agents have sufficiently low error correlation for the majority vote to remain well-behaved and avoid the failure mode of spurious agreement.
5 EXPERIMENTS
5.1 EXPERIMENTAL SETUP
Datasets. We evaluate our method on three synthetic benchmarks (ProntoQA (Saparov He
2022), ProofWriter (Tafjord et al. 202I) and LogicalDeduction (Srivastava et al. (023) and on
three real-world benchmarks (AR-LSAT (Zhong et al. 2021). FOLIO (Han et al. 2022), and
Chinese LogiQA-V2 (Liu al. 2023b)). More details are presented in Appendix B
Baselines. We compare against nine representative methods that span different approaches, includ- ing solver-based, prompt-based and multi-agent methods. To isolate the effect of debate topology,

---

Published as a conference paper at ICLR 2026
Table Performance comparison with error bars using prompt paraphrases incorporate ran- domness, indicates statistical significance using pairwise t-test (p < 0.05).
Model
Method
ProofWriter
ProutoQA
LeglcaDDeduction
AR-LSAT
FOLIO
Chinese LogiQA-V2
wlo sparse
89.78 = 0.35
99.20 + 0.20
93.78 0.19
50.29 = 0.12
84,47 = 0.28
74.11 = 0.19
GPT-4
CortexDebate
90.78
99.67
85.13
73.95 0.22
w/ sparse
91.83 + 0.17
99.87 + 0.12
94.61 = 0.26
53.17 = 0.14
86.60
0.43
74.66 + 0.14
w/o sparse
92.61 0,38
99
0.20
99.11 0.51
73.31 0.27
89.22
85.75
V3
Cebte
92.83 = 0.17
99.80 = 0,20
99.67 0.33
73,88 0.25
89.18 = 0.44
83.04 = 0.27
sparse
0.14
99.93 0.12
0.19
75.76 0.44
0.28
86.55 0.48
Table 5: Impact of different debate components on performance
GPT-4
Claude 3.7 Sonnet
DeepSeck-V3 LogiDeduct
Method
ProntoQA
ProofWriter
LogiDeduct
ProntoQA ProofWriter LogiDeduct ProntoQA
ProofWriter
w/o MA Trans.
99.40%
89.17%
90.00%
100.00%
96.00%
97.33%
99.60%
92.67%
97.33%
w/o MA Ren. via ST.
60%
79.33%
84.67%.
98.00%
83.33%
91.00%
96.0nS:
86.17%
93.00%
w/o MA Rea. via NL
99. 20%
90.67%
94.00%
100.00% %
96.67%
100.00% %
99 99.20%
90.00%%
98. 98.00%
100.00%
92.00%
94.33%
100.00%
96.83%
100.00%
100.00%
93.33%
100.00%
Solver
FOL Solver
SAT Solver
ProofWriter
LogicalDeduction
ProntoQA
1.0
1.0
1.0
Execution Rate
Execution Rate
Execution Rate
0.9
0.8
0.8
0.6
0.8
0.7
0.4
0.7
0.6
T-1
T-2
T-3
T-4
T-5
T-1
T-2
T-3
T-4 T-5
T-1
T-2
T-3
T-4
T-5
(T)
Number Turns (T)
Number of (T)
Figure 2: Relation between debate rounds and solver execution rate (GPT-4). Execution rate peaks
2-3 rounds then declines.
Accuracy
Token Saving Rate
ProntoQA
ProofWriter
LogicalDeduction
1.000
Token Saving Rate
0.920
0.945
Taken Saving Rate
0.3
0.2
Accuracy
0.992
0.2
Accuracy
0.910
0.2
Token Saving
Accuracy
0.930
0.984
0.2
0.900
0.2
0.915
0.890
0.1
0.976
0.1
0.1
0.900
0.880
0 0.4
0.8
1.2
1.6
0.0
0.4
0.8
1.2
1.6
0.0
0.4 0.8 1.2 1.6
0.0
Interaction Threshold
Interaction Threshold a
Interaction Threshold a
Figure 3: Effect of communication gating threshold on accuracy and token saving rate on GPT-4.
the reported results for multi-agent baseline methods adopt our proposed NL-SL hybrid reasoning stage; they differ from our method only in the debate topology. For real-world setting, we include the strongest symbolic reasoning (SymbCoT and LogicLM) and the strongest multi-agent baseline (CortexDebate). The details of baseline methods and implementations are presented in Appendix B
Evaluation Metrics. We report Accuracy, the percentage of correctly answered logical questions.
5.2
MAIN RESULTS
We set temperature as enabling deterministic reproducibility. Table presents results across three logical reasoning benchmarks, and Table presents results across three real-world reasoning bench- marks. experiments on Qwen2.5-7B-Innn -B--truuttttt is in Table To assess the robustness our method, conduct controlled variance study by generating three semantically equivalent prompt para- phrases (using GPT-5) for each experiment. We report mean and standard deviation across the three paraphrases in Table 4 Improvements marked with & statistically significant compared to strongest competing baseline. Results for gpt-4o-mini follow the same experimental protocol and reported in Appendix
7

---

Published as a conference paper at ICLR 2026
Table 6: Effect of agent diversity and composition
GPT-4
Claude Sonnet
DeepSeek-V
SI. reasoning
NL. reasoning
ProntoQA
ProofWiiter
LogiDeduct
ProsioQA
ProofWriter
LogiDeduct
PrenioQA
ProofWriter
LogiDeduct
FOL
COT
97.00%
35.50%
81,67%
99.20%
93.83%
97.67%
98. 08.00%
91. 91.00%%
92.00%
SAT-FOL
COT
97.20%
17%
93.00%
99.60%
94 00%
99.67%
98. 98.40%
92.50%
99.67%
SAT-FOL+LP
100.00%
100.00%
96%
100.00%
99.60%
92.83%
SAT-FOL+LP COT+P&S
100.00%
92.00%
94.33%
100.00%
96.83%
100.00%
100.00%
93.33%
100,00%
Table 7: Sensitivity to À in the sparse gate. "Tok" denotes token saving rate (%).
GPT-4
D3epSock-V3
ProofWriter
ProntoQA
LogicalDeduction
ProofWriter
ProntoQA
LegicalDeduction
A
Acc
Tok
Acc
Tok
Acc
Tok
Acc
Tok
Acc
Tok
Acc
Tok
0 90.17%
8.52%
99.20%
11.45%
92.67%
5.63%
92.17%
3.88%
99. 99.60%
18.01%
98.33%
17.29%
0.5
92.17%
13.62%
99.60%
18.31%
94.00%
10.29%
93.00%
6.02%
99.80%
28.82%
98.67%
27.66%
1.0
92.00%
17.03%
100.00%
22.89%
94.33%
12.35%
93.33%
13.15%
100.00%
36.02%
100.00%
34.57%
1.5
91.50%
18.73%
99. 80%
25.18%
93.67%
13.02%
93.67%
15.15%
99.60%
36.53%
99.33%
38. 38.44%
2.0
91.50%
19.59%
100.00%
26.32%
93.33%
13.31%
93.33%
15.34%
100.00%
36.87%
99.67%
39.31%
Overall Performance. Our method with sparse debate consistently outperforms all baselines across benchmarks and models. Compared to single-agent methods, we achieve substantial improvements over LogieLM, LINC, Aristotle, SymbCOT, CR. and DetermLR. Against multi-agent baselines, we surpass both SparseMAD and CertexDebate while maintaining computational efficiency (detailed token cost comparisons are provided in Appendix
Sparse Full Communication. Notably, our sparse variant consistently outperforms the fully- connected version, indicating that selective communication filtering not only reduces computational costs but also mitigates noise from redundant agent interactions, leading more effective debates. The improvements across diverse base models demonstrate the robustness of our approach.
5.3 ABLATION STUDY
To understand the contribution of each component in our framework, we conduct comprehensive ablation studies on both the debate stages and the agent composition.
Impact of Debate Stages. Table ablates three debate components: (1) translation debate during NL-to-SL conversion, (2) symbolic reasoning agents, and (3) natural language reasoning agents. Removing symbolic reasoning causes the largest performance drop, followed by translation debate, confirming that formal logical reasoning is most critical while accurate symbolic translation and nat- ural language reasoning provide complementary
our multi saati--tage debate design.
Impact of Agent Diversity, Table examines how different combinations of reasoning agents affect performance. We progressively add agents: starting from single FOL agent with COT reasoning. we incrementally incorporate SAT, LP, and Plan&Solve agents. The results reveal improvements with each addition, demonstrating that both symbolic reasoning diversity (FOL, SAT, LP) and natu- ral language reasoning diversity (COT, Plan&Solve) are essential for robust logical reasoning.
5.4 HYPERPARAMETER ANALYSIS
Translation Debate. Figure shows executable rates of translated symbolic expressions peak at 2-3 debate rounds before degrading pattern consistent across all models (see Appendices (aaeees/KJnndd aa other models). This degradation beyond round 3 indicates excessive debate introduces noise through over-correction of initially accurate translations. The finding validates our choice of D = 3 rounds. We further conducted a consslationquullle study in Appendix 5.5) where we quantify symbolic translation eтror rates and evaluate our FOL translations against gold formulas, confirming that the translation-debate stage reliably improves NI.-SI. quality.
shows
Cecurac-Commuuication Sparsity Trade-off. We investigate the impact of the communica- tion threshold a on both accuracy and computational efficiency, measured as token saving rate: /atp///////aaeooo/s//////ss Tokensso sparse- Higher values enforee stricter communication filtering, resulting in sparser interaction graphs. Figure illustrates this trade-off for GPT-4 (see Appendices K and L for other models). A notable pattern emerges: increases, accuracy of-
spasee (Tokensw sparse Tokensw/ spanse)
8

---

Published as a conference paper at ICLR 2026
Table 8: Translation common error rate T-CER, for three SL agents (LP/FOL/SAT), Values are probabilities (lower is better).
#SL agents
GPT-4
DeepSeek-Vi V3
Writer
ProntoQA
LogicDeduct
ProofWriter
ProntoQA
LogiDeduct
agent
18.50%
10.33%
44.56%
17.61%
14.53%
53.56% 13.78% 2.87%
2 agents
2.50%
1.20%
19.33%
1.44%
3 agents 0.33%
0.20%
4.00%
0.17%
1.00%
5.67%
Table FOL translation quality on FOLIO, Numbers are LLM-judged semantic correctness of the FOL translations (%).
Model
w/o translation debate
w/ translation debate
GPT-4o-mini
68.14 14%
75.49%
GPT-4
76.47%
84,80%
DeepSeek-V. V3
75.98%
88.24%
ten improves while simultaneously reducing token costs by 10-30%. This suggests that moderate sparsity filters out redundant inter-agent communications that can harm reasoning quality.
Reasoning Debate. Figure shows accuracy saturates after 2-3 debate rounds across three bench- marks, then plateauing or slightly degrading. This pattern suggests agents quickly reach consensus on logical problems, with further rounds introducing noise through overthinking or redundant argu- ments. The consistent 3-round optimum across datasets validates choice of D 4, balancing reasoning quality with computational efficiency.
examine the effect of the sparsity coefficient À in communication gate (Table 7. Results show that the accuracy of our method remains high and stable once À 2 0.5 (fluctuations within sz pp).
Sensitivity to the Sparsity Hyperparameter
5.5 TRANSLATION QUALITY ANALYSIS
To assess the quality of our NL - SL translations, provide two complementary analyses:
Translation Common Error Rate. For each of the three SL agents and each question, we mark the translated program as correct or incorrect. For any subset of n SL agents, we define T-CER, as the probability that all agents are wrong on the same question. Table g reports T-CER, on the three main benchmarks. In all cases, T-CERa is very small, showing that all three symbolic translators rarely fail simultaneously, which supports combining multiple heterogeneous SL. agents.
Direct Validation on FOLIO. We further evaluate on FOLIO, which is one the few datasets that provide human-annotated FOL formulas aligned with natural-language premises and hypothe- ses. For each example, we compare our translated FOI. formula with the gold one and use an LLM judge to decide semantic equivalence, reporting the percentage of translations judged correct. As shown in Table ? across all three base models the translation-debate stage consistently improves FOL translation accuracy, confirming that debate enhances NL. SL translation quality.
5.6 SOLVER TIMING ANALYSIS
Tablle 10 shows, for each dataset and solver (Pyke for LP, Prover9 for FOL, Z3 for SAT), the average solving time on executable instances and the timeout rate under a fixed threshold. Overall, symbolic solving rarely times out and does not dominate the computational cost of our method.
5.7
CASE STUDIES: MULTI-AGENT DEBATE DYNAMICS
To illustrate how our multi-agent debate framework achieves consensus through collaborative rea- soning debate, we present case study from the ProofWriter dataset shown in Table 11 The prob- lem requires determining whether "The lion visits the lion" is true (A), false (B), or unknown (C) based given logical rules and facts, with ground truth answer being (A). The debate showcases effective peer correction: agents with incorrect initial answers recognize their logical oversights through examining others' reasoning chains and converge the correct solution, validating multi-

---

Published as a conference paper at ICLR 2026
Table 10: Average solving time (seconds) and timeout rate (%) of symbolic solvers (measured on executable samples only).
indicates that the solver is not used on that dataset.
Dataset
Pyke avg time
Pyke timcout
Prover9 avg time
Prover9 timcout Z3 avg time
Z3 timeout
ProntoQA
0.031
0.00%
0.097
0.00%
0.065
0.00%
ProofWriter
0.028
0.00%
0.102
0.00%
0.065
0.00%
LogicalDeduction
0.031
0.00%
3.762
2.23%
0.072
0,00%
FOLIO
0,568
0,49%
0.063
0.00%
AR-LSAT
4,419
3,40%
0,061
0.00%
Chinese LogiQA-V2
1.487
0.00%
0.063
0.00%
Table 11: Case Study of agent reasoning across debate rounds. In this case, A is the correct answer, C the incorrect answer.
Round
Agent
Answer
Key Reasoning
Needa ILion, Beax) and Rule 2: Needs (Sx, Boax) -Nedda (8x, Llon], we deduce
Needs (Lion, Lion). Rule Needs (foe, Lion) wiisits (8x, Lion: gives us Visits (Lion, Lion).
FOL
Both proving and denying Viaits (ILon, Liont yield contradiction. Neither onnclusively established as true false. Problem is undentetined.
SAT
c
23 sulver finds no explici fact that implics Vieito (Lion, Lion). Rules du not desive it. Stalus: Unknows/indeermiisie
P&S
Lion needs bear (Fact 6). By Rule 2: lion needs lion. By Rule 3: lion visits lioe. Clear logical chain establishes truth.
nile or starement catublishes whether visits No
reasoning leads to conclusioe.
FOL's claim of "underdelined" incorrect. 2-+3 are deterministic, "underdefined""
FOL
Still claiming Unknown despite clear logical chain shown by others. Incisting problem
yields definitive resolution.
SAT
After reviewing Soave' argument, agroe. The dedietiio chain is valid. Changing from Unknown to True.
P8S
A
Standing by dedaction. Facts and rales suffice to establish truth. The logical steps are valid and complete.
cor
A
I lacknw ledging my oversight before. Plav&S Rule 2+3 choin valid. Changing answer to True.
LP
Chain reasening sirtight. Therefore, lian visits itself.
After renssessing, concur, Rule Rale 3 chain valid and well-supported. Changing from Unknown to Trme.
SAT
Maintaiming revised conclusion.
P&S
Reasoning follows structured appenach: Fact 6 Rule 2 Rule definitive Thee.
cor
Agreement confirmed. Chain of using Fact 6, Rule and Rule is sound.
All agents reach consensus: Answer A "The lion visits the lion" is conclusively true
ProntoQA
ProotWriter
-
LogicalDeduction
GPT-4
DeepSeek
Claude
100%
100%
100%
Accuraty
95%
Aczuracy
Accuracy
96%
95%
90%
92%
90%
85%
80%
2
3
4
5
6
88%
2
3
5
6
7
85%
2
3
4
5
6
Number Turns (T)
Turns (T)
Number Turns (T)
Figure 4: Relation between turns and final accuracy.
agent debate's error-correction capability. Full question and dialogues for this case be found in Appendix Q.3)
Case study for translation debate is presented in Appendix Q.2
6
CONCLUSION
This paper mitigates the important limitations of large language models (LLMs) in complex logical reasoning. To our best knowledge, we are the first to propose a multi-agent approach, which enables the absorption of advantages from multiple reasoning methods through debate. Additionally, we propose a sparse communication mechanism optimize the efficiency and cost of these multi-agent interactions. Extensive experiments on three datasets show that our method enhances logical QA performance while reducing computational cost. A limitation of this work, also serves as a future research direction, motivates from observation that the LLM's logical reasoning performance drops significantly when handling newly released and out-of-distribution datasets (Liu et al. 2023a), thus it is crucial to extend approach to accommodate out-of distributtoonno scenarios.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
Yisen Wang supported by Beijing Natural Science Foundation (L257007), Beijing Major Sci- ence and Technology Project (Z251 100008425006), National Natural Science Foundation of China (92370129, 62376010), Beijing Nova Program (20230484344, 20240484642), and State Key Lab- oratory of General Artificial Intelligence. Fenrong Liu is supported by Beijing Natural Science Foundation (L257007) and Tsinghua University's Initiative for Advancing First-Class and World- Leading Disciplines in the Humanities and Social Sciences. Haoxuan Li is supported by the Institute for Decentralized AI, project of the Cosmos Institute funded by the Al Safety Fund.
REFERENCES
Anthropic. Claude 3.7 sonnet system card, 2025. Accessed 2025-09-25.
Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gian- inazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, et al. Graph of thoughts: Solving elaborate problems with large language models. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pp. 17682-17690, 2024.
Benjamin Callewaert, Simon Vandevelde, and Joost Vennekens. Verus-LM: a versatile framework for combining LLMs with symbolic reasoning. arXiv preprint arXiv:2501. 14540, 2025.
Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. Chateval: Towards better Ilm-based evaluators through multi-agent dehate. In The Twelfth International Conference on Learning Representations, 2023.
Justin Chen, Swarnadeep Saha, and Mohit Bansal. Reconcile: Round-table conference improves reasoning via consensus among diverse 1lms. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 7066-7085, 2024.
Fengxiang Cheng, Haoxuan Li, Fenrong Liu, Robert Rooij, Kun Zhang, and Zhouchen Lin. Em- powering Iims with logical reasoning: A comprehensive survey. International Joint Conference on Artificial Intelligence, Survey Track, 2025.
Yilun Du Shuang Li, Antonio Torralba, Joshua B Tenenbaum, and Igor Mordatch. Improving fac- tuality and reasoning in language models through multiagent debate. In Forty-first International Conference on Machine Learning, 2023.
Andrew Estornll and Yang Liu. Multi-llm debate: Framework, principals, and interventions. Ad- vances in Neural Information Processing Systems, 37:28938-28899 2024.
Jiazhan Feng, Ruochen Xu, Junheng Hao, Hiteshi Sharma, Yelong Shen, Dongyan Zhao, and Weizhu Chen. Language models can be deductive solvers. In Findings of the Association for Computational Linguistics: NAACL 2024, pp. 4026-4042, 2024.
Simeng Han, Hailey Schoelkopf, Yilun Zhao, al. Folio: Natural language reasoning with first- order logic. In EMNLP, 2022.
Akbir Khan, John Hughes, Dan Valentine, Laura Ruis, Kshitij Sachan, Ansh Radhakrishnan, Ed- ward Grefenstette, Samuel R Bowman, Tim Rocktäschel, and Ethan Perez. Debating with more persuasive Ims leads to more truthful answers. Proceedings of Machine Learning Research, 235: 23662-23733, 2024.
Guohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Com- municative agents for" mind" exploration of large language model society. Advances in Neural Information Processing Systems, 36:5 36:51191-520000 2023.
Qingchuan Li, Jiatong Li, Tongxuan Liu, Yuting Zeng, Mingyue Cheng, Weizhe Huang, and Qi Liu. Leveraging Ilms for hypothetical deduction logical inference: A neuro-symbolic approach. arXiv preprint arXiv:2410.21177 2024a.
Yunxuan Li, Yibing Du, Jiageng Zhang, Le Hou, Peter Grabowski, Yeqing Li, and Eugene Ie. Im- proving multi-agent debate with sparse communication topology. In Findings of the Association for Computational Linguistics: EMNIP 2024, pp. 7281-7294, 2024b.

---

Published as a conference paper at ICLR 2026
Tian Liang, Zhiwei He. Wenxiang Jiao, Xing Wang. Yan Wang, Rui Wang, Yujiu Yang, Shuming Shi, and Zhaopeng Tu. Encouraging divergent thinking in large language models through multi- agent debate. In EMNLP, 2024.
Hanmeng Liu, Ruoxi Ning, Zhiyang Teng, Jian Liu, Qiji Zhou, and Yue Zhang. Evaluating the logical reasoning abilitt of chatgpt and gpt-4. arXiv preprint arXiv:2304.00 254::33340343,, 2023a.
Hanmeng Liu, Zhizhang Fu, Mengru Ding, Ruoxi Ning, Chaoli Zhang, Xiaozhang Liu, and Yue Zhang. Logical reasoning in large language models: A survey. arXiv preprint arXiv:2502.09100, 2025a.
Hanmeng Liu et al. Logiqa 2.0: An improved dataset for logical reasoning in natural language understanding. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2023b.
Tongxuan Liu, Wenjiang Xu, Weizhe Huang, Yuting Zeng, Jiaxing Wang, Xingyu Wang, Hailong Yang, and Jing Li. Logic-of-though: Injecting logic into contexts for full reasoning large language models. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies, 2025b.
Qing Lyu, Shreya Havaldar, Adam Stein, Li Zhang, Delip Rao, Eric Wong, Marianna Apidianaki, and Chris Callison- Callison-Burch. Faithful chain-of-thought reasoning. In The 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia- Pac--accfiic Chapter of the Association for Computational Linguistics (IJCNLP-AACE 2023), 2023.
Terufumi Morishita, Gaku Morio, Atsuki Yamaguchi, and Yasuhiro Sogawa. Enhancing reason- ing capabilities of Ilms via principled synthetic logic corpus. Advances in Neural Information Processing Systems, 37:73572-73604, 2024.
Theo Olausson, Alex Gu, Ben Lipkin. Cedegao Zhang, Armando Solar-Lezama, Joshua Tenenbaum, and Roger Levy. Line: A neurosymbolic approach for logical reasoning by combining language models with first-order logic provers. In Proceedings of the 2023 Conference on Empirical Meth- ods in Natural Language Processing, pp. 5153-5176, 2023.
OpenAl. Gpt-4 technical report. 2023.
OpenAl. Gpt-4o and gpt-4o-mini: Openai omni models, 2024. Model documentation available at hutps://platform.opees
Liangming Pan. Alon Albalak, Xinyi Wang, and William Wang. Logic-Im:
Empowering large lan- guage models with symbolic solvers for faithful logical reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 3806-3824, 2023.
Joon Sung Park, Joseph O' O'Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology, pp. 1-22, 2023.
Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert- networks. Proceedings of the 2019 Conference on Empirical Methods Natural Language Processing and the 9th International Joint Joint Conference on Natural Language Processing (EMNLP-
IJCNLP), pp. 3982-3992, 2019.
Hyun Ryu, Gyeongman Kim, Hyemin S. Lee, and Eunho Yang. Divide and translate: Compositional first-order logic translation and verification for complex logical reasoning. In ICLR, 2025.
Abulhair Saparov and He He. Language models are greedy reasoners: systematic formal analysis of thain-of-thougee arXiv preprint arXiv:2210.011 2022.
Aarohi Srivastava, Abhinay Rastogi, Abhishek Rao, Abu Awal Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adri Garriga-Alonso et al. Beyond the imita- tion game: Quantifying and extrapolating the capabilities of language models. Transactions on machine learning research, 2023.
12

---

Published as a conference paper at ICLR 2026
Hongda Sun, Weikai Xu, Wei Liu, Jian Luan, Bin Wang, Shuo Shang, Ji- Ji-Rong Wen, and Rui Yan. DetermLR: Augmenting LLM-based logical reasoning from indeterminacy to determinacy. In Lun- Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18853/ 12055/////022...looogg.3.33
Yiliu Sun, Zicheng Zhao, Sheng Wan, and Chen Gong. Cortexdebate: Debating sparsely and equally for multi-agent debate. In Findings of the Association for Computational Linguistics: ACL 2025, pp. 9503-9523, 2025.
Oyvind Tafjord, Bhavana Dalvi, and Peter Clark. Proofwriter: Generating implications, proofs, and abductive statements over natural language. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 3621-3634 2021.
Qwen Team. Qwen2.5: A scalable family large language models, 2025.
Yuxuan Wan, Wenxuan Wang, Yiliu Yang. Youliang Yuan, Jen-tse Huang, Pinjia He, Wenxiang Jiao, and Michael R Lyu. Logicasker: Evaluating and improving the logical reasoning ability of large language models. In EMNLP, 2024.
Junlin Wang, WANG Jue, Ben Athiwaratkun, Ce Zhang, and James Zou. Mixture-of- en- hances large language model capabilities. In The Thirleenth International Conference on Learn- ing Representations, 2024a.
Zhongsheng Wang, Jiamou Liu, Qiming Bao, Hongfei Rong, and Jingfeng Zhang. Chatlogic: Inte- grating logic programming with large language models for multi-step reasoning. In 2024 Inter- national Joint Conference on Neural Networks (IJCNN), pp. 1-8. IEEE, 2024b.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi. Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824 24837, 2022.
Wu, Z. Ren, Z. Sha, Z. Fu, Z. Xu, Huang. Zhang, Z. Xie, Z. Hao, Z. Gou, Z. Ma, Z. Yan, Z. Shao, Z. Xu, Z. Wu, Z. Zhang, Z. Li, Z. Gu, Z. Liu, Z. Li, Z. Xie, Z. Song, Z. Gao, Z. Pan, et al. Deepseek-y -33 technical report. 2024.
Jundong Xu, Hao Fei, Liangming Pan, Qian Liu, Mong-Li Lee, and Wynne Hsu. Faithful logical reasoning via symbolic tf-thugh... In The 62nd Annual Meeting of the Association for Computational Linguistics, 2024.
Jundong Xu, Hao Fei, Meng Luo, Qian Liu, Liangming Pan, William Yang Wang, Preslav Nakov, Mong-Li Lee, and Wynne Hsu. Aristotle: Mastering logical reasoning with a logic-complete framework. In Proceedings of the 62nd Annual Meeting of the Associ- ation for Computational Linguistics, 2025a.
decomppsee-eearc--reees
Jundong Xu, Hao Fei, Yuhui Zhang, Liangming Pan, Qijun Huang, Qian Liu, Preslav Nakov, Min- Yen Kan, William Yang Wang, Mong-Li Lee, and Wynne Hsu. Muslr: Multimodal symbolic logical reasoning. In Proceedings of the Annual Conference on Neural Information Processing Systems, 2025b.
Jundong Xu, Hao Fei, Huichi Zhou, Xin Quan, Qijun Huang, Shengqiong Wu, William Yang Wang, Mong-Li Lee, and Wynne Hsu. Logicreward: Incentivizing Ilm reasoning via step-wise logical supervision. In Proceedings of the International Conference on Learning Representations, 2026.
Sen Yang, Xin Li. Leyang Cui, Lidong Bing, and Wai Lam. Neuro-symbolic integration brings causal and reliable reasoning proofs. arXiv preprint arXiv:231 2023.
Shunyu Yao. Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Ad- vances in neural information processing systems, 36:1 1809-11822, 2023.
Xi Ye, Qiaochu Chen, Isil Dillig, and Greg Durrett. Satlm: Satisfiabilit--iiee language models using declarative prompting. Advances in Neural Information Processing Systems, 36:45548- 45580, 2023.
13

---

Published as a conference paper at ICLR 2026
Yifan Zhang, Jingqin Yang. Yang Yuan, and Andrew Chi-Chih Yao. Cumulative reasoning with large language models. arXiv preprint arXiv:2308. 04371, 2023.
Yifan Zhang, Yang Yuan, and Andrew Chi-Chih Yao. On the diagram of thought. arXiv preprint urXiv:2409.10033 2024.
Wanjun Zhong. Siyuan Wang, Duyu Tang, Zenan Xu, Daya Guo, Jiahai Wang, Jian Yin, Ming Zhou, and Nan Duan. Ar-lsat: Investigating analytical reasoning of text. In Findings of ACL, 2021.
14

---

Published as a conference paper at ICLR 2026
USAGE OF AI
In this work, made limited use of LLMs an assistive writing tool. Specifically, we used LLMs to replace synonyms, restructure sentences, and brainstorm alternative ways of expressing ideas within paragraphs. All conceptual contributions, research design, experiments, analyses, and final writing decisions were made by the authors. The authors take full responsibility for the accuracy and originality of the content.
A RELATED WORK
Logical Question Answering. The field of logical question answering seeks to enhance the reason- ing capabilities of large language models and is generally pursued through three main approaches: solver-based, fine-tuning, and prompt-based methods (Cheng et al. (025). Solver-based methods operate by converting natural language queries into formal symbolic expressions before utilizing specialized solvers for inference (Lyu et al. [023 Olausson et al. [023]] Ye et al., 2023 Ryu et al., 2025]. Fine-tuning techniques employ a dual strategy of creating synthetic datasets with explicit reasoning processes and augmenting training corpora with structured logical knowledge to embed
reasoning abilities directly within model parameters (Feng et al. 2024) Morishita al. 2024 Wan
et al., 2024). Prompt-based based methods explore variety of strategies, with some generating explicit
reasoning chains to guide inference (Wei al.] 2022] Yao al.) 2023; |Bstaa al.] 2024 ZZhng
eet al.,
2024). while others direct models to produce symbolic forms for stepwise verification (Li
jet al.
2024a: Wang et al.] 2024bb Xu et al. 2024] Liu et al., 2025b] Xu et al.] 2025a). Recent
work has also explored multimodal symbolic logical reasoning and step-we wwie logical supervision for reasoning improvement (Xu et al.) 2025b] 2026). While existing research has predominantly
focused on single-agent systems, our work introduces multi-agent dehate framework to synergize the complementary advantages of both SI. and NI. reasoning.
Multi-Agent Debate in LLMs. Within this domain, multi-agent debate (MAD) (Du et al. 2023) )
strategy where agents engage in iterative rounds of discussion to improve their final responses through a process of collective refinement. Research on agent roles has explored distinct reasoning modes and functional assignments, such as a proposer, a critic, planner, and an executor, to in- crease diversity and reliability (Li et al.] 2023 Park et al.) 2023 Liang et al.) 2024). The inclusion of an independent judge has been shown to enhance the factual accuracy and stability of results across tasks (Chan et al. 2023 Du et al. 2023 Estornell & Liu 2024 Khan et al. 2024). Addition- ally, collaboration among heterogeneous models aims for a more robust consensus through opinion aggregation, with methods like Reconcile adding confidence-weighted voting to integrate varying viewpoints (Chen et al. 2024| Wang et al. (024a)). To address the inherent cost of these frame- works, some methods, such SparseMAD, reduce communication by pruning the topology static sparse graph where agents read from fixed neighbors (Li et al. 20245), while CortexDebatt constructs a sparse debate graph with equal participation and learns edge weights using the McKin- sey Trust Formula (Sun et al. 2025). Our work builds on these efforts by proposing a multi-agent debate framework that combines both symbolic and natural language reasoning, and we introduce a novel adaptive sparse communication mechanism to significantly enhance efficiency.
B IMPLEMENTATION DETAILS.
Backbone models. Our main experiments three widely adopted and highly capable GPM---GP-44 (OpenAT|20233 Claude 3.7 Sonnet (Anthropic) 2025). and DeepSeek-V V3 (Wu et al.) 2024]. To further assess the scalability and applicability of our method smaller and ordinary mod- els, we also include Qwen2. 5-7B-Instruct
(Team 2025) and GPT-4o-mini (OpenAl 2024).
Datasets. We evaluate our method on three synthetic benchmarks: (1) ProntoQA (Saparov & He
2022), a dataset for testing deductive reasoning over ontological knowledge with 500 test examples; (2) ProofWriter (Tarjord et al., (2021). We use the test set following (Pan et al. (023). which is a set of randomly sampled 600 examples from the most challenging depth-5 subset; and (3) Log- icalDeduction (Srivastava al. 2023), dataset from BIG-Bench focusing complex deductive reasoning with ordering constraints, containing 300 test examples. These benchmarks assess dif- ferent aspects of logical reasoning, from basic syllogistic inference to complex multi-hop deduction and constraint- canttraint-based reasoning. In addition, we further evaluate our framework on three real-world benchmarks: (4) AR-ISAT (Zhong et al. 2021), consisting of reasoning questions from official
15

---

Published as a conference paper at ICLR 2026
Table 12:
Per-dataset cost-effecttveness comparison. Tokens are prefill tokens per question (L),
accuracy is in % (1).
Model
Method
ProntoQA
ProofWriter
LogicalDeduct
Tokens
Acc
Tokens
Acc (%)
Tokens
Ace (%)
SparseMAD
37.784
99.80
41,678
89.50
43,635
88.67
Cebaee
35,973
99.60
37,554
90.83
45,487
92.33
GPT-4
Ours (w/o sparse)
46,502
99.40
51,358
90.17
54,857
94.00
Ours (w/ sparse)
35,854
100.00
42,617
92.00
54,171
94.33
SparseMAD
79,456
99.80
48,190
92.83
44,245
99.83
CebtexDebate
68,023
99.80
41,897
96.17
45,962
99.67
Claude 3.7
Ours (w/o sparse)
106,015
100.00
63,105
97.00
68,636
99.67
Ours (w/ sparse)
52,923
100.00
62,204
96.83
47,121
100.00
SparseMAD CehtexDehate
40,200
98.00
18,527
92.50
53,257
95.33
35,381
99.80
19,349
93.00
47,388
99.67
DeepSeek-VV
Ours (w/o sparse)
57,366
99.80
25,059
92.83
70,464
100.00
Ours (w/ sparse)
36,702
100.00
24,115
93.33
46,107
100.00
LSAT examinations; (5) FOLIO (Han et al. 2022). human-authored natural language dataset an- notated with first-order-logic formulas: and (6) Chinese LogiQA-V2 (Liu (202b)) Chinese
logical reasoning benchmark adapted from real civil-service examination questions.
Baselines. We compare against nine representative methods that span different approaches: (1) Solver-based methods: LogicLM (Pan et al. 2023) and LINC (Olausson et al. (023), which trans- based methods: one-shot COT (Wei al. (2022), Aristotle (Xu al. 2025a), SymbCOT (Xu al. (024), CR (Cu- mulative Reasoning) (Zhang et al.] 2023), and DetermLR (Sun et al. 2024] (3) Mulli-agent meth-
late natural language into symbolic forms for external solver processing: (2)
ods: SparseMAD (Li et al. (022b), and CebtexDebate (Sun al.) 2025). To isolate the effect of debate topology, the reported results for SparseMAD and CortexDebate adopt our proposed NL-SL hybrid reasoning stage; they differ from our method only in the debate topology. For real-world setting, we include the strongest symbolic reasoning (SymbCoT and LogicLM) and the strongest multi-agen debate baseline (CortexDebatee. The details of implementations are presented in Ap- pendix B
Our framework employs five agents in the reasoning debate stage (three symbolic reasoning agents using LP, FOL, and SAT solvers respectively, plus natural language reasoning agents using COT and Plan-and-Solve prompting). We set the debate rounds D for translation and D = 4 for rea- soning stages based our parameter analysis (Sections 5.4). The hyperparameter for balancing confidence and information gain is set to 1.0. When symbolic solvers fail to execute, we employ the "Simulate" strategy (detailed Appendix [) where agents fall back to LLM reasoning while main- taining their symbolic perspective. The complete prompt used is detailed in the Appendix We use Sentence-Bil BERT (Reimers & Gurevych] 2019) to encode agent outputs into dense embeddings for computing cosine similarity.
C
ANALYSIS
We evaluate the cost-effeettveenes of our sparse communication approach by measuring token con- sumption and accuracy across three LLMs and three benchmarks. Following evaluation proto- col, we report prefill tokens per question as a reproducible cost proxy and accuracy as efféctivvness:; lower tokens are better (4), higher accuracy is better (f). We do not report wall-clock time due to API jitter; tokens serve as a stable, reproducible proxy for runtime and dollar cost.
In our experiments. Ours (w/o sparse) approximates a fully-connected debate topology where all agents communicate in each round, while Ours (w/ sparse) uses our adaptive sparse communication gate to selectively prune interactions based on confidence and information gains.
Our adaptive sparse gate achieves the highest accuracy while keeping token costs comparable to strong baselines. As shown Table [2] and Table [3] our sparse method consistently outperforms the fully-connected baseline all three models, achieving both higher accuracy (+0.92pp on GPT-
16

---

Published as a conference paper at ICLR 2026
Table 13: Aggregate performance across three benchmarks. Token Saving and AAce are relative to
Ours (w/o sparse).
Model
Method
Avg. Acc
Avg. Tokens
Token Saving
AAce (pp) 1
(%)
+
(%)
SparseMAD
92.66
41,032
19.40
-1.87
CortexDebate
94.39
39,671
22.07
-0.14
GPT-4
Ours (w/o sparse)
94.52
50,906
0.00
+0.00
Ours (w/ sparse)
95.44
44,214
13.15
+0.92
SparseMAD
97.49
57,297
27.70
-1.40
Cebaee
98.61
51,961
34.44
-0.8
Claude 3.7
Ours (w/o sparse)
98.89
79,252
0.00
+0.00
Ours (w/ sparse)
98.94
54,082
31.76
+0.05
MparseMAD CortexDebate
95.28
37,328
26.75
-2227
97.49
34,039
33.21
-0.05
DeepSeck- V3
Ours sparse)
97.54
50,963
0.00
+0.00
Ours (w/ sparse)
97.78
35,641
30.06
+0.24
Table 14: Average token cost and accuracy GPT-4.
Method
ProntoQA
ProfWriter
LogicalDeduution
AR-LSAT
FOLIO
Chinese LogiQA
Tokens
Acc
Tokens
Acc
Tokens
Acc
Tokens
Tokens
Acc
Tokens
Acc
Direct Answer
252
75.40%
315
53.50%
286
59.00%
144
32.90%
100
65.20%
355
62.27%
CoT
303
81.20%
498
67.17%
261
69.67%
747
35.06%
177
70.59%
534
22%
sparse)
99.40%
51,358
90.17%
54,857
94.00%
35,012
50.42%
23,167
84.31%
74.01%
Ours (w/ sparse)
35,854
100.00%
%
42,617
92.00%
43,264
94.67%
28,044
53.25%
19,023
86.27%
14,733
74.76%
+0.05pp Claude 3.7, +0.24pp on DeepSeek-V35 V3) and substantial token savings (13-36%). Re- markably, it also surpasses existing multi-agent baselines (SparseMAD and CortexDebate) in accu- racy while maintaining competitive token efficiency. This demonstrates that our confidence-based pruning mechanism not only reduces computational overhead but also improves reasoning quality by filtering redundant inter-agent communications.
Token and accuracy comparison against Direct Reasoning and COT is provided in Table 14 and
Table 15
D HANDLING SYMBOLIC SOLVER FAILURES
During the symbolic reasoning stage, solvers may occasionally fail to execute the translated logi- cal expressions due syntax errors, incompatible formula structures, computational timeouts. Since our multi-agent framework relies on symbolic solvers (Pyke, Prover9, and Z3) to provide formal reasoning, handling these execution failures appropriately is erucial for maintaining system robustness.
Table T6 presents the impact different failure handling strategies on final accuracy across three benchmarks using GPT-4. We evaluate three strategies:
Random: When solver fails, the agent randomly selects answer from the available options. This serves as a baseline strategy.
Discard: Failed solver agents are excluded from the debate, and only successfully executed
agents participate in subsequent rounds and final voting.
Simulate: When solver fails, prompt the corresponding agent to simulate the solver's rea- soning process using the LLM's inherent logical capabilities, effectively falling back to natural language reasoning while maintaining the agent's role in the debate.
The results demonstrate that the Simulate strategy consistently achieves the best performance across all benchmarks. This approach leverages the LLM's ability to approximate symbolic reasoning when formal execution fails, maintaining full agent participation while providing reasonable fallback rea- soning. The Discard strategy performs better than random selection but loses valuable perspectives from failed agents. These findings suggest that maintaining agent diversity through simulation is
17

---

Published as a conference paper at ICLR 2026
Table 15: Average token cost and accuracy on DeepSeek-Vi
Method
ProntoQA
ProofWriter
LegicalDeduction
AR LSAT
FOLIO
Chinese LogiQA V2
Tokens
Acc
Tokens
Acc
Tokens
Acc
Tokens
Acc
Tokens
Acc
Tokens
Acc
Direct
79.20%
68.33%
2,324
74.33%
CaT
257
85.00%
296
71.83%
497.55
83.00%
2,453
45.45%
651
76.96%
488
77.97%
Ours (w/ sparse)
57,366
99.80%
25,059
92.83%
70.464
100.00%:
38,973
76.79%
24,410
89.22%%
19,334
65.68%
Ours sparse)
36,702
100.0%
24.115
93.33%
46,107
100.00%
30,152
79.65%
19,832
90.67%
15,547
86.93%
Table 16: Final accuracy (%) under different handling strategies when symbolic solver fails (GPT-
4).
Strategy
ProntoQA
ProofWriter
LogicalDeduction
Random
99.20%
89.83%
91.33%
Discard
99.80%
91.33%
93.67%
Simulate
100.00 %
92.00%
94.33%
more beneficial than excluding agents, even when their primary symbolic reasoning mechanism fails.
E THEORETICAL ANALYSIS OF MAJORITY VOTE
Problem Setup and Assumptions
Setting: We focus on Logical QA, which is multiclass classification task. For simplicity, we denote the input space as X' and output space y = {c1, 02, ..., cx) (k: 2 2), where y denotes the ground-truth label.
We have collection of agents H {h1, h2 hm For any agent h;, assume it is better than random guess, i.c. the overall accuracy p = (2) = y) > 1/k. For simplicity, assume uniform error answer distribution (relaxable to non-uniform with minor adjustments).
For any two distinct agents h; hy (i + f), we define the average pairwise class-wise correlation p E [0,1); 1):
Pahr
Isa<bsk
where Pab (Cov(Zi.a Zj.ab// / Var( Zi ) Var (jr()), and Ziab = (x) = a) I(h;(z)) b) (binary indicator for answering vs. for learner h2). This captures how often two learners agree answer pairs.
The majority vote yields an ensemble learner
(xx)
arg max eEV >
[(h((z) =
In case of tic, random selection is applied.
Theorem (Aceuracy Lower Bound for Majority Vote Ensemble). Under the above setting, let 5
p- ! and note that 8 > For any incorrect class e + y, define T; I(h; y) - I[h, (2) = c)
and assume Var(T;) o² and Cov(T; Ti) = pa for + j. where is the average pairwise class- wise correlation defined above. Then the accuracy of the majority
ensemble satisfies:
o2 + (m 1)p]
P(H(x:) = v) 2 (A: 1).
mI²
In particular:
1. If p = 0, then limm-+40
y) v)
p > then as m + oo, the accuracy lower bound converges (k- -k-1) P02
Jr.
3. For any 0, if p
(k-1))2, then there exists mg such that for all m mg. P(H(x) v)
8²
1
18

---

Published as a conference paper at ICLR 2026
Proof.
Let:
- (x) = y): number of agents predicting the correct class. For each c # y, Se Ex= [((h(()) = c): number of agents predicting class c.
The ensemble H predicts correctly if and only if' S > S. for all + y.
We compute expectations:
[B[S] mpp. Due to uniform error distribution, E.S.] = m. t for each + y.
Define = F Since p 1/ki we have:
1 h
1-p
- kp >
=
8800 0.
k 1-1
Therefore, for each c + y:
ElS Scl md >
For a fixed e + y, define T; = [1) = y) - T(h)(x)) = c). so S - S. = Di Tg
Compute the statistics of T;:
[[[T]] = p 1== = 6. 1-p
E[T?] = p- 12 + k-1 (-1)² + (1 1 (1-p- 1-p) 0 1-p
Var(T,) E[T?] (E(T,)) p+ 4-1 -- = 02.
1-p
By assumption, for + j. Cov (T T;) = po?
Therefore, the variance of S S is:
Var(S - -ce)
Var(T:) +
Σ
Cov(T; Tj) mo² + mm(m 1)po² = = (m 1)p]
ifj
Using Chebyshev's inequality:
P(S < Sc) = PPS 0) PP(S (c) - m8 < -m8)
< SPSS S. md) 2 md) < Var(S - Sc)
0² + (m 1)p]
(md)²
md²
The ensemble if there exists some * y such that 5 < Sc. By the union bound:
((H(s)) v) - Σ P(S (k - 1) o² [1 + (m 1)pl
md2
Thus:
(()(x) v) 21 -k- 02 + (m 1)p)
mnd2
We then analyze the asymptotic properties.
Case p 0.
o²
P(H(x) = y) 1 (k 11-11
1).
as m + cc.
m
Therefore, limm-rze P(H(2) = 1.
19

---

Published as a conference paper at ICLR 2026
2. Case p 0.
0 (m 1)p]
(()(2) y) 2 (k: (6--). 1).
mn22
(k: 1) po²
(k: - 1) 02
6²
md²
As m - oo, the lower bound converges
1 - 1) po²
Arbitrary accuracy guarantee. For any > 0, if <
(A (k-1) then:
6²
1- 1) po²
0,
and there exists my such that for all > mo:
o² (m 1)pl
1 (k: -
1 -
m8²
Hence, (x) = y) 1
For completeness, we provide an explicit expression for o²:
1 p 8² -pt
1 2 k:
o² p
p
k-1
This expression can be further simplified but is not essential for the theorem statement proof.
Interpretation and Corollaries The above result yields the following important insights, which we highlight for clarity:
(i) If errors are independent 0, the lower bound goes to 1 as + co.
(ii) If errors are positively but moderately correlated p > 0, the bound converges to
1) pa² 2
as m + 00;
demonstrating that the majority vote remains rell-behaved unless agents are highly correlated. This formalizes key intuition in our system: since our agents come from distinct SL/NL reasoning paradigms, their error correlation substantially below regime that leads to the failure mode of high spurious agreement.
F CONSENSUS ANALYSIS VIA VOTE ENTROPY
Since introduced preference sparse pruning affects the level consensus reached by the agents.
prune communication edges, here assess whether
For each question q, let ) be the set of answer options, the number of agents in the debate, and cy the number of agents voting for option y. We define the normalized vote entropy
H (g) log
Cy
log
Cy m
[0,
|
DEV
where 0 corresponds to perfect agreement (all agents choose the same option) and 1 corresponds to maximally split votes. We report the average Hn over all questions at the final debate round.
Table 17 compares the average normalized vote entropy between a fully connected debate graph (w/o sparse) and our sparse communication graph (w/ sparse) for GPT-4 and DeepSeek-V2 V3. Sparse prun- ing does not much change vote entropy across datasets, and in several cases even slightly reduces it, indicating that our sparse topology preserves the consensus behavior of the debate in practice.
20

---

Published as a conference paper at ICLR 2026
Table 17: Average normalized vote entropy Horm [0, 1] across questions (lower stronger con- sensus), measured at the final debate round.
Model
Method
ProofWriter
ProntoQA
LogicalDeduution
AR-LSAT
FOLIO
Chinese LogiQA-V2
GPT-4
w/o sparse
0.0224
0.0014
0.0055
0.0881
0.0655
0.0710
GPT-4
w/ sparse
0.0243
0.0014
0.0054
0.0893
0.0687
0.0733
DeepSeek-VV V3
w/o sparse
0.2359 0.2334
0.0540
0.0034
0.0346
0,2832
0.0142
DccpSock-V3
w/ sparse
0.0555
0.0034
0.0340
0.2732
0.0150
Table 18: Sensitivity aggregation rules (Accuracy %)
GPT-4o-mini
DeepSeck-33
Dataset
Majority
Conf- WeighWe LLMM-asJuuge
Majority
Conf-Weighted
LLM-as-Judge
Majority
Conf Weif-Weigtee
LLMa-as-ddge
ProofWriter
76.33%
75.50%
76.00%
92.00%
91.60%
92.00%
93.33%
93.50%
93.50%
ProntoQQ
60%
88.00
90.40%
100.00%
100.00%
100,00%
100.00%
100.90%
LogiDeduct
82.35%
82.67%
83.33%
94.33%
93.67%
95.00%
100.00%
99.80%
100.00%
G SENSITIVITY TO AGGREGATION RULES
To show that our results are not due to a specific voting rule, we added sensitivity study on three vote (iii) "LLM-as-judge""
aggregation rules (Table 18 (i) Majority vote (īi) Contidence-weighhee (agents debate, and an independent LLM reads all rationales and produces final prediction). Across 3 base LLMs x 3 datasets, the gap between all three methods is within pp. This indicates that:
Our improvements are not due to a particular voting method.
The gains primarily come from the SL multi-agent debate and sparse communication, while the final aggregator is easily changeable.
H ADDITIONAL EXPERIMENTS ON SMALL / ORDINARY MODELS
To evaluate the generalization of our framework to smaller and more accessible LLMss, duct experiments on two compact models: Dwven..5-7B-Inntruu and GPT-4o-mini. Despite their significantly lower parameter counts, sparse multi-agent debate framework continues yield consistent gains across all six benchmarks on both models as shown in Table and Table 19 demon- strating that our approach is not limited large frontier LLMs. CortexDebate (w/ NL-SL) reuses our translation stage, solver stage, and agent roles thlu differing from our method only in the com- munication graph topology. CortexDebate (w/o NL-SL) corresponds to the original pure-NL version as used in its original work.
I ABLATION ON SL-NL
AND SPARSE COMMUNICATION
To disentangle the contributions of the SL-NL cross- passspar design and the sparse communication topology, we provide comprehensive ablation study. The variants are grouped into two families: (A) SL-NL paraddimm ablations that manipulate symbolic vs. natural language reasoning com- ponents, and (B) wateeeeoomccccooooooolooooooo ablations that vary the debate graph while keeping the number of agents fixed.
(A) SL-NL cross-paradigm ablations.
We consider the following variants:
COT + P&S only (NI-only).. Remove all symbolic translators and solvers. Only two NL. agents (Chain-of-Thought and Plan-and-Sollv) participate in the debate.
LP FOL + SAT only (SL-onlyy. Remove all NL agents. Keep only the three solver-based agents (LP/Pyke, FOL/Prover9, SAT/Z3).
No SL-NL interaction in debate. Keep all 5 agents, but force SL agents debate only with SL agents and NL agents only with NL agents (two disjoint debates).
Translation dehate rounds 0. Disable the translation-stage debate (Dirans 0). SL transla- tions are generated once and used as-is by solvers.
5-agent direct vote (no debate). All agents (SL and NL) answer once independently; the final answer is decided by a single majority vote without any iterative debate.
(B) Sparse topology communication ablations. We next investigate different communication topologies:
21

---

Published as a conference paper at ICLR 2026
Table 19: Performance of different methods on GPT-4o-mini.
Method
ProofWriter ProntoQA
LogiDeduct
AR-LSAT
FOLIO
Chinese LogiQA-V2
Direct Answer
53,50%
62.60%
56.00%
19.91%
59. 80%
57.31%
CoT
43.67%
75.00%
70.33%
19.04%
61.76%
54.30%
LogicLM
58.67%
76.00%
73.00%
22.94%
34.80%
25.00%
SymbCoT
70.33%
82.80%
75.33%
26.00%
69.10%
58.87%
Cebtex (w/o NL-SL)
60.87%
80.40%
70.67%
21.21%
62.75%
62.02%
CortexDebate (w/ NL-SL)
75.17%
89.00%
82.33%
34.20%
75.00%
64.03%
Ours (w/o sparse)
74.00%
89.40%
82.33%
33.81%
74.02%
65.91%
Ours (w/ sparse)
76.33%
90.60%
84.67%
34.20%
76.47%
67.29% %
Table 20: Ablations on (A) SL-NL cross-paradigm reasoning and (B) sparse communication strate-
gies. Accuracy (%). Columns correspond GPT-4 DeepSeek-V. V3 on ProofWriter (PW), ProntoQA (PQA), and LogicalDeduction (LD).
Setting/ Variant
GPT4-PW GPT4-PQA GPT4-LD
DS-PW
DS-PQA
DS-LDD
(A) SL.NLL Cross-Prradign Ablations
COT PRS only (NI-only)
79.33%
95.60%
84.67%
86.17%
96.00%
93.00%
LP FOL SAT only (SL-only)
90.67%
99.20%
94.00%
90.00%
99.20%
98.00%
No SL.-N NI. interaction in dehate
90.85%
99.20%
93.00%
90.17%
99.20%
97.33%
Translation debate rounds
89. 17%
99.40%
90.00%
92.67%
99.60%
97.33%
5 agent direct vote (no dehate)
86.50%
97.00%
82.67%
90.00%
97.60%
91.67%
(B) Sparse Topology/ Communication Ablations (S-agent)
Pure S-agent chat, folneccen Pure NL S-agent chat SparseMAD Pure 5 chat + CortexDebatee our sparse gate
73.00%
91.40%
84.00%
82.83%
93.00%
88.33%
72.83%
91.00%
85.00%
87.1
93.20%
87.67%
73.50%
89.00%
84.33%
83.17%
94.00%
90.00%
5-agent
75.67%
85.33%
84,33
94.00%
91.33%
Replace our
w/ SparseMAD
89.50%
99.80%
88.67%
92.50%
98.00%
95.33%
Replace our sparse gate w/ Dehate
90.83%
99 60%
92.333%
93.005
99.805
98. 33%
Ours (full SL+NL, full debate, our gate)
92.00% %
100.00% %
94.33%
93.33%
100.00%
100.00%
Pure NL 5-agent chat, fully-connected. Remove all SL agents. Use 5 identical NL agents; every agent reads all others (fully connected graph).
Pure NI 5-agent chat + SparseMAD. Same pure-NL setup, but replace the communication
graph with SparseMAD's static neighbor topology.
Pure NL 5-agent chat + CortexDebate. Same pure-NL setup, but use CortexDebate's trust-
weighted sparse graph.
Pure NL 5-agent chat + sparse gate. Same pure-NL setup, but apply confidence information-gal gain based sparse gate.
Replace sparse gate with SparseMAD (full SL+NL pipeline). Use full SL+NL pipeline (translators, solvers, NI. agents), but replace our gate with the SparseMAD topology. Replace sparse gate with CortexDebate (full SL+NL pipeline). Same full SL+NL
pipeline, but use CortexDebate's learned trust graph.
Ours (full SL+NL, full debate, our gate). The complete proposed method.
Table 20 summarizes the accuracy of all ablations under GPT-4 and DeepSeek-V3 across the three benchmarks. The results show that: symbolic and natural language reasoning are cewenntarrrr-mmiinng.ee.g removing either side or their interaction harms accuracy; and (ii) within both pure- NI. and full SI.NNI. pipelines, our adaptive sparse gate outperforms static sparse topologies such as SparseMAD and CertexDebate.
complementary-
J ABLATION ON THE CONFIDENCE TERM IN THE SPARSE GATE
Our design of confidence scores follows variety of multi-agent works, where LLMSs generate an explicit confidence score that then used for confidence- weighted voting or debate control. In- stead of using absolute probabilities, sparse gate uses self-reported confidence only relative ranking signal, via ratios such as C,/C,.
To test the necessity and robustness of the self-reported confidence scores in sparse communica- tion gate, we perform an ablation where we disable the confidence term in sparse- gaasse.gtiin...... Table 21 compares accuracy with and without the confidence term Qwen2.5-7B Instrruct and GPT-4 over three datasets (ProofWriter, FOLIO, Chinese LogiQA-Ve V2). Using confidence is consistently better, suggesting that self-reported confidence always provides useful additional signal for sparse gating.
22

---

Published as a conference paper at ICLR 2026
Table 21: Accuracy (%) with and without the confidence term in the sparse gate.
Model
Variant
ProofWriter
FOLIO
Chinese LogiQA-V2
Swen2.5-7B-Instruus w/o conf
75.33%
64.23%
67. 67.92%
Qwen2.5-7B-Instee -nnstructtt w/ conf
76.50%
65.68%
68.11%
GPT-4
w/o conf w/ conf 92.00%
90.87%
84.80%
74.26%
GPT-4
86.27%
74.76%
K ADDITIONAL EXPERIMENTAL RESULTS ON DEEPSEEK- V
This section presents additional experimental results for DeepSeek-Vi V3 that show similar patterns to the GPT-4 results discussed in the main paper.
K.1 COMMUNICATION THRESHOLD ANALYSIS
Figure 3 shows the effect of communication gating threshold on accuracy and token saving rate for DeepSeek-V3. V3, The results demonstrate patterns consistent with GPT-4, achieving token reduction while maintaining high accuracy.
Accuracy
Token Saving Rate
ProntoQA
ProofWriter
LogicalDeduction
1.000
0.5
Token Saving Rate
0.928
0.4
Token Saving Rate
1.000
0.5
Token Saving Rate
Accuracy
0.996
Accuracy
0.3
Accuracy
0.996
0.920
0.3
0.992
0.3
0.2
0.992
0.912
0.988
0.2
0.1
0.988
0.2
0.904
0.0
0.0
0.0
0 0.4 0.8 1.2 1.6
0
0.4
0.8 1.2 1.6
0
0.4
0.8 1.2 1.6
Interaction Threshold a
Interaction Threshold a
Interaction Threshold a
Figure 5: Effect of communication gating threshold accuracy and token saving DeepSeek- V3.
K.2 TRANSLATION QUALITY ANALYSIS
Figureo.shin jsoows the relationship between debate rounds and solver execution rates for DeepSeek V3. Consistent with our GPT-4 findings, the execution rate increases during the first 1-2 rounds and then shows diminishing returns.
Solver
FOL Solver
SAT Solver
ProofWriter
LogicalDeduction
ProntoQA
Execution Rate
1.0
Execution Rate
Execution Rate
1.0
0.9
0.9
0.6
0.8
0.8
0.4
0.7
0.7
0.2
0.5
T=1
T=2
T=3
T=4 T=5
T=1
T
T=3
T=4 T=5
T=1
T=2
T=3
T=4
T=5
Number of Turns (T)
Number of Turns (T)
Number of Turns (T)
Figure 6: Relation between debate rounds and solver execution rate for DeepSeek-V3.
L ADDITIONAL EXPERIMENTAL RESULTS ON CLAUDE 3.7 SONNET
This section provides supplementary experimental results for Claude 3.7 Sonnet.
23

---

Published as a conference paper at ICLR 2026
L.1 COMMUNICATION THRESHOLD ANALYSIS
Figure 7 presents the accuracy eccuracy-effiiiency trade-off for Claude 3.7 Sonnet. Similar to GPT-4 and DeepSeek- V3, Claude 3.7 maintains high accuracy while achieving significant token savings through sparse communication.
Accuracy
Taken Saving Rate
ProntoQA
ProofWriter
LogicalDeduction
Accuracy
0.9998
1.0000
0.5
Token Saving Rate
Accuracy
0.968
W
0.4
Token Rate
Accuracy
0.999
1.000
0.5
Token Saving Rate
0.964
0.3
0.998
0.3
0.9995
0.2
0.2
0.960
0.997
0.2
0.9992
0.1
0.956
0.996
0.0
0.0
0.0
0.8 1.2
1.6
0 0.4 0.8 1.2 1.6
0.4 0.8 1.2 1.6
Interaction Threshold
Interaction Threshold
Interaction Threshold
Figure 7: Effect of communication gating threshold on accuracy and token saving rate on Claude
3.7 Sonnet.
L.2 TRANSLATION QUALITY ANALYSIS
Figure illustrates the translation quality dynamics for Claude 3.7 Sonnet. The pattern is consis- tent with other models: execution rates improve significantly within the first 2-3 debate rounds, validating our multi-agent debate approach for translation refinement.
LP Solver
FOL Solver
SAT Solver
ProofWriter
LogicalDeduction
ProntoQA
Execution Rate
1.0
Execution Rate
1.0
Execution Rate
1.0
0.9
0.8
0.9
0.9
0.7
0.8
0.6
0.8
0.5
0.7
T=1 T=2
T=3
T=4
T=5
T=1
T=2
T=3
T=4 T=5
T=1
T=2 T=3 T=4 T=5
Number of Turns (T)
Number of Turns (T)
Number of Turns (T)
Figure 8: Relation between debate rounds and solver execution rate for Claude 3.7 Sonnet.
M EMPIRICAL QUANTIFICATION OF SHARED MISTAKES AMONG AGENTS
To measure the probability of agents sharing similar mistakes in our system, we introduce the Com- mon Error Rate CER,-- For any subset S of n agents, let the answer given by agent a be Ya.qz the indicator of correctness of answer Ya.q be Ca.g E {0,1}.
1 Φ 1(q Q: Va E S, ca.q O and Vo.g identical)/-
CER,
That is, it measures the fraction of questions on which all agents in subset S pick the same wrong option. then average over (a) subsets to obtain CER,,
As show in Table 22
CER,, drops rapidly as n increases. Our sparse gating further largely reduces these correlated mistakes.
This shows that "many agents choosing the same wrong label" already rare, and our sparse gating mechanism further reduces such shared mistakes.
24

---

Published as a conference paper at ICLR 2026
Table 22: Common Error Rate (CER,) across agent subsets (lower is better).
GPT-4
DeepSeek V3
#Agents
Variant
ProofWriter
ProntoQA
LogicalDeduction
ProofWriter
ProntoQA
LogicalDeduction
w/o sparse
0.1017
0.0024
0.0700
0.2593
0.0016
0,0060
w/ sparse
0.1060
0.0016
0.0855
0.2475
0.0008
0.0047
w/o sparse
0.0900
0.0020
0.0555
0.0955
0.0002
0.0033
2
sparse
0.0503
0.0002
0.0446
0.0643
0.0000
0.0003
w/o sparse
0.0653
0.0020
0.0426
0.0537
0.0000
0,0033
w/ sparse
0.0183
0.0000
0.0212
0.0207
0.0000
0,0000
w/o sparse
0.0425
0,0020
0.0402
0.0367
0.0000
0.0033
4
w/ sparse
0.0117
0.0000
0.0101
0.0125
0.0000
0.0000
5 w/o sparse
0,0200
0.0020
0.0383
0.0283
0.0000
0.0033
sparse
0.0067
0.0000
0.0097
0.0095
0.0000
0.0000
N MULTI-TURN INTERACTION ALGORITHM FOR SPARSE COMMUNICATION
N.1 MULTI-TURN DYNAMIC INTERACTION PREFERENCE BETWEEN LLMs
We establish sparse communication topology to improve the efficiency in multi-turn interactions through a dynamic pruming mechanism. which allows source agent to communicate its output to the receiving agent j at round d. Specifically, we propose preference score quantifying the potential utility of the information in the communication, which defined as:
C₁ Cj.
Pre
A(1 cos(A 491144)
This score comprises two key components. The first is C /C/. representing the ratio of con- fidence scores between the source agent and the receiving agent j at round d. The second is 1 (A Aç)), measuring the difference between the two outputs, regarded as information gain.
To guarantee efficiency, we propose a dynamic strategy to determine with which agent to communi- cate. Specifically, in round d, use this average preference score Pre d-I the adaptive threshold. We define a binary communication gate O 0 Communication from to j is permitted only if the current preference score is greater than or equal to the historical average, indicating that the current interaction is at least as beneficial as the average past interaction between this pair. The indicator of whether agent i benefits agent at round d is formally defined as:
{
1, Pre 2 Pref-1 d-1
0
0, Pre < Pre
d-1
N.2 MUL.TI-URN INTERACTION ALGORITHM FOR ENHANCING LLMS' REASONING
The sparse communication mechanism directly informs how each agent updates its internal state memory across debate rounds. Each agent maintains personalized memory that aggregates valuable insights from others. At the beginning of the first round (d = 1), all agents start with an for all pairs). From the
empty memory M + Ø and communication fully connected (O second round, the sparse communication gate o is activated. At the end of each round d, every agent s updates its memory for the next round Md+1 by selectively incorporating the outputs A from only those agents i for which the communication channel was open (i.e., 0 = ). After the memory is updated, agent generates its output for the next round A by querying the symbolic question and i's newly updated, personalized memory. After D rounds of debate, the final outputs from all agents A P+1 AD+ are aggregated via majority vote to determine the final answer.
O SENSITIVITY TO THE SIMILARITY METRIC
Our sparse gate uses a similarity measure between agent rationales to estimate information gain. In the main experiments ilarity metrics are also possible. To test robustness, we compare cosine similarity against ROUGE-L the similarity metric inside the gate.
cosine similarity over Sentence- BERT embeddings, but other text sim-
25

---

Published as a conference paper at ICLR 2026
Algorithm 1: Multi-Turn Interaction Algorithm for Enhancing LLMs' Logical Reasoning
Input: Communication rounds D. Agent number n, hyper-parameter A;
Translate logical question Q symbolic expression Sym(Q);
M ................................ Md=1 2; for d f1....,, } do
0r = 1 for all i.j E n):
Compute Pre.
Cd
+ A(1
cos( A!)) for all i i j
Cd
Compute Pre (Pre
(d 1)+ Tे. + X(1 CA]-,,osAA,.40 A())) for all * j:
Pre + < <a
Pre و-j then
L 0 O²-0=00 =0;
for 8 E nn.....n.oo do
// Memory update of the s-th agent round d
Md+1 + Md u i (l....,n] n], 0 1};
10
// Output of the s-th agent at round d using personalized memory
A + LLM, (Sym(Q)))
11
Majority vote among the n agents Ap+1
AD+1;
12
Table 23: Sensitivity the similarity metric. Accuracies (%) for cosine similarity vs. ROUGE-L.
Metric
Model
PW Acc
PQA Ace
LD Ace
Cosine
GPT-4
92.00%
100.00%
94.33% 93.33%
ROUGE-L
GPT-4
91.00%
97.80%
Cosine
DeepSeek-V3
93.33%
100.00%
100.00%
ROUGE-L
DeepSeck-V3
92.00%
98.40%
98.33%
Table 23 reports accuracies when using cosine vs. ROUGE-L for GPT-4 and DeepSeck-V V3 across the three main benchmarks. The differences within 1-2 percentage points, indicating that our framework is not sensitive to the specific choice of similarity metric.
P PROMPT TEMPLATES
P.I TRANSLATION DEBATE
Translation Prompt
Task. You are given a logic problem in natural language including context and question as follows:
Context: $ (context]
Question: {question)
Discussion Rules
Syntax Verification: Carefully review previous discussions to understand others' translations. While maintaining your own symbolic language system, check and correct any syntax errors in your transla- tion (c.g., unclosed parentheses, malformed expressions).
2. Completeness Check: Review others' translations to understand their interpretation of the natural lan- guage problem. While keeping your symbolic language system, verify and correct the information completeness of your translation (no missing/extra facts, rules, predicates, statements from the orig- inal problem).
3. Language Independence:
When referencing others' translations. you must maintain your sym- bolic language system. Do not adopt symbols or syntax from other languages.
Discussion history
chat__iistory
26

---

Published as a conference paper at ICLR 2026
Role-specific description
$ grooe_descrin
Now it's your turn to speak. Please speak as concisely and clearly as possible
Role-specific description - LP translator
Your task is translate the logic problem in natural language into LP logic formulas:
1. define all the predicates in the problem
2. parse the problem into logic rules based on the defined predicates
3. write all the facts mentioned in the problem
4. parse the question into the logic form (Use & to represent AND, and you cannot use NOT or other negations in LP)
Example
Context: : Zach jompus is fruity.
(... more context here ...) Rompuses are zumpuses. Alex a tumpus.
Question: True or false: Alex is not shy.
Predicates:
Jompus (9x, bool) ::: Does x belong to Jompus?
(... more predicates here ...)
Zumpus (Sx, bool) 1:: Does belong to Zumpuses?
Facts:
Tumpuses (Alex, True)
Rules:
Jompus (9x, True) Fruity (9x, True)
(... more rules here ...)
Dumpus (5x, True) >>> Rompus (5x, True)
Query:
Shy (Alex, False)
Role-specific description - FOL translator
Your task is translate the logic problem in natural language into first-order logic formulas. The grammar of first-order logic is defined as follows:
logical conjunction:
expr, A expr,
logical disjunction:
expr, V expr,
logical exclusive disjunction:
expr, @ expr, eexpr,
logical negation:
expri implies expra:
expr, + expr, expr, < expr,
expri iff expr2:
logical universal quantification:
Væ
logical existential quantification: 3x
Output format: logic form description
Example
Context:
All people regularly drink coffee are dependent on caffeinc.
(... more context here ...) If Rina is not person dependent on caffeine and student, then Rina is either person dependent on caffeine and a student, or a person dependent on caffeine nor a student, or neither a person. dependent on caffeine nor student.
27

---

Published as a conference paper at ICLR 2026
question: Based on the above Information, is the following statement true, false, or uncertain? Rina is either a person who jokes about being addicted to caffeine or unaware that caffeine is a drug.
Predicates:
Dependent (x) ::: is person dependent on caffeine (... more predicates here ...)
Student (x) ::: x is a student
Premises:
A\forall x$ (Drinks (x) Skrightarrows Dependent (x)) ::: All people who rogularly drink coffee are dependent on caffeino.
(... more premises here ...)
$Nforall (Jokes (x) strightarrow sAnegSUnaware (x)) No who jokes
about being addicted to caffeine is unaware that caffeine is a drug.
Conclusion:
Jokes (rina) Unaware (rina) ::: Rina either a person who jokes about being addicted to caffeine or is unaware that caffeine is a drug.
Role-specific description -AT translator
Your task is to parse the logic problem in natural language as SAT problem using Z3 syntax, defining declarations, constraints, and options.
1. Always include all three section headers in order: Opt ions
Declarations, + Constraints, f
Declarations must follow exact patterns:
name EnumSort ( [items, ...]) for non-numeric items
IntSert [[number,, ... for numeric items
name = Function [types] -> [return_type]]
3. Constraints support:
Direct expressions with 1=, <=, >=, <, >, Implics (), And(), (r
Not ()
ForAll (Ivar:type, .. expr) and Exists (Ivar:type, ...l, expr)
Count ([var:type]] condition)
Distinct (1 ([vart (lvar:type), expr)
4. Options must use predefined functions:
is sis_validl (), is sat is unsat
5. Add explanation with :::
6. Avoid:
Add & any other places apart from three section headers
Add any other unnecessary comment dashes
Example
Context: Bob is cold. Bob is quiet. Bob is red. Bob is smart. Charlie is kind. Charlie is quict. Charlic is red. Charlic is rough. Dave cold, Dave kind. Dave is smart. Fiona is quiet.
something is quiet and cold then it is smart. Red, cold things are round. If something is kind and rough then it is red. All quiet things are rough. Cold, smart things are red. If something is rough then cold. All red things are rough. Dave is smart and Dave kind then Dave is quiet.
Question: True or false: Charlie is kind.
28

---

Charlie kind is Published as a conference paper at ICLR 2026
I Declarations objects = EnurSort [Bob, Charlie, Dave, Fiona])
attributes EnumSort ( [coid, quiet, red, smart, kind, rough, round]) has_attribute Function ([objects, attributes] [bool])
# Constraints
has_attribute (Bob, cold) = True ::: Bob is cold.
has attribute (Bob, quiet) True ::: Bob is quiet.
has_attribute (Hob, red) True :::
red.
has_attribute smart) True ::: Bob is smart.
@es_attributte kind) -- True ::: Charlie is kind.
has @as_attzibotte quiet) == True 1:: Charlie is quiet.
has has_attriibtte red) True
Charlie is red.
has pas_attziibtteeChrzii,, rough)
True ::: Charlie is rough.
(Da_ cold) = True ::: Dave is cold.
(Dav,,,, kind) = True ::: Dave is kind.
(De_atttribttteeave,, smart)
True
Dave is smart.
(as_attributeeeionn,, quiet)
True ::: Fiona is quiet.
ForA1l ((x:ajesst) Implies (And (has_attributee (x, quiet) True,
has_attribute cold) =- True), has_attribute (x, smart) -- True))
::: If something is quiet and cold then it is smart.
ForAll (1x:ojestt))t) Implies (And (has_attribute red) True, has (ttriburt colo)
True), has attribute (x, round) True))
::: Red, cold things are round.
ForAll ([x:ojeccs)))) Implies (And (has_attribute kind) -- True, True), tas_attribute red) True))
has_attribute rough)
::: If something is kind and rough then it is red.
([::bjett Implies (has_attribute(x, quiet) True, has_attrriuute rough) == True) ::: All quiet things are rough. P[x:o]((x: Implies (And (has_attributee cold) == True, has_attribute (x, smart)
True), has_attribute red) True))
::: Cold, smart things are red.
(( Implies mas_attribuute rough) == True, has_attribute cold) -- True)) ::: If something is rough then it is cold.
ForAll ([x:objects), Implies (has_attribute
red) True, True)) 1:: All red things are rough.
has attribute (x, rough) Implies (has_aat attribute (Dave, smart) == True, has_att___ attribute (Dave, kind) == True), has attribute (Dave, quiet) == True) 11: If Dave is smart and Dave is kind then Dave is quiet.
F Options
is_valid (has_attributee (Charlie,, kind) == True) :: Charlie is kind is True (A)
is_unsat thas_attribute (Charlie, kind) True)
False (B).
P.2 REASONING DEBATE
Final Debate Prompt
are given logic problem that contains a context, a question, options:
Context: $ (context]]
Question: $ (questioo)
Options: $ ((options)
Role description: $ (Role-specific description)
Your initial answer $ (predict],
Your initial reasoning is: $ ng))).
You are now in collaborative debate with other reasoning agents. Your goal is to reach the correct answer through discussion.
Important Debate Rules
29

---

Published as a conference paper at ICLR 2026
Review other agents' arguments in discussion history first.
2. Identify specific points of agreement disagreement.
3. Challenge weak reasoning with concrete counterexamples.
4. No need to repeat your whole reasoning if your argument remains unchanged.
5. Acknowledge other arguments when you find them correct, even if they contradict your initial position.
6. If you change your answer, always explain why you changed. willing to change your answer convinced by other arguments.
8. Reference specific agents and their arguments when responding.
9. Be interactive and engaging with other agents!
Discussion history
$ %/chathhistory)
Turn-specific instruction
$
Role-specific description - LP supporter
are supporter Logic Programming (LP) approach. Strengths:
Systematic rule-based reasoning with clear steps
Handle complex relations via predicates and rules
Transparent reasoning process verifiable step by step
Strong foundation formal logic theorem proving the debate, you should:
Emphasize rigor and reliability of LP reasoning
Highlight systematic application of logical rules
Defend transparency and verifiability
Challenge others when lacking formal logical foundation
Role-specific description - FOL supporter
You are supporter of First-Order Logic (FOL). Strengths:
Mathematical precision with quantifiers and operators
Express complex relationships precisely
Sound theoretical foundation
Handle nested structures and implications
In the debate, you should:
Emphasize rigor and expressiveness of FOL
Highlight formal completeness and soundness
Defend against criticisms of complexity
Challenge others when lacking precision
Role-specific description SAT supporter
You are supporter of the SAT/SMT (Z3) approach. Strengths:
Formal representation using Z3 syntax
Complete and sound reasoning
Ability to prove validity/satisfiabbiity
Handle quantifiers and complex formulas
In the debate, you should:
Emphasize formal correctness and completeness
Highlight Z3's power constraint solving
Defend clarity declarative specification
Challenge others on missed edge cases
Role-specific description - Plan-and-Solvee supporter
You are supporter of the Plan-and-Solve approach. Strengths:
First devise plan, then execute it
Structured reasoning framework
30

---

Published as a conference paper at ICLR 2026
the debate, you should:
Defend the reasoning process and solution steps
Justify the plan structure and accuracy
Role-specific description COT Solver supporter
You are the Chain-of- Thought (CoT) supporter. Strengths:
Step-by-step reasoning from facts to conclusions
In the debate, you should:
Defend solutions generated via CoT prompting
Validate each inference in the reasoning chain
Argue for soundness of the conclusion
Q CASE STUDIES
Q.1 REASONINGS EXTRACTED FROM SYMBOLIC SOLVERS
Case Study of Reasonings Extracted
Context. The cat eats the squirrel. The cat sees the squirrel. The eats the squirel.. The COW sees the cat. The rabbit is round. The rabbit sees the cat. The squirrel eats the rabbit. The squirrel cold. The squirrel needs the rabbit. The squirrel sees the cat. If someone sees the cat and they are not green then they see the cow. If the rabbit is kind and the rabbit sees the squirrel then the squirrel needs the rabbit. Rough people are cold. If someone sees the rabbit then they are not round. If someone sees the squirrel and they are not green then they need the squirrel. If someone eats the cow then they see the rabbit. If someone eats the squirrel then they are rough. If someone is cold then they eat the cow.
Question. Based on the above information, is the following statement true, false, or un- known? The cat is not round.
Options. A) True
B) False
C) Unknown
LP Reasoning Extracted from Pyke Solver:
We first deflne Iollowing predicales and cozresponding natural language explanatiens: Rats (8x, Sy, boel) ::: Does est y? y?
(8x, $y,
111
Round (8x, bool) ::: Is x round?
Cold;3x, bcol) 1:2
Neads (8x, sy, bool) ::: Does x need y?
Green (8x, bool) 1:1
green?
Kind beol)
Is kind?
Rough (9x, bool) 1:: 1s rough?
have following known [acis Rats (cat, squisrel, True)
Seas teat, squirrel, True)
Rat.a toow, aqui>rel, True]
Seea (oow, cat, Tzue)
Round (rabbit, True) Sees trabbit, cat, True]
Rata (aguirrel, rabbit, True) cold (aquirrel, True)
(squirrel, rabbit, Trie)
Seea tacuirrel, eat, True)
We have follewing known rules from the context:
rulel: (Sx,
Green ralse) (Sx, True)
rule2: (ranbbi,,,,, Truc) 66 Sees (rabbit, aquirrel, Truc) >>> Keeda (squireel, rabbit, Truet
rulel: (ox, Trus) >>> cold:5x, True)
ruled: SSx, rabbit, True)
Round (3x, False)
rule5: Sexs equirrel, True! (Sxen False) >o9 Weeds(5x, squicrel, True)
rule6: Bats(Sx, cox, Trus! a5s Sees(5x, rabbit, True!
rulel: (ax, squirrel, Trus!
Reugh True)
rule8: cold Trual >>> (ax, cow, Truel implied facts: Use rulol: (Sx, eat, True) 66 Green (8x, False) >>> Seea com, True) Bind 'cow"
begin reasoning
Unbind 8x
31

---

Published as a conference paper at ICLR 2026
Bind 9x to 'rabbit' Unbind 8x Bind 5x to 'aquirrel'
Unbind
Finish implied with culel
rule2: (sabbit,,,,, True) (sabbii,,,, squirrel, True) (squirrel,
rabbit, True)
Finish implied with rule2
rulo3: 13sgh True) >>> (6x, True) Finish implted with rule3
ruled: Sees rabbit, Truel RoundiSx, Talse) finish implied with ruled
ruleš: (9x, squirral, True) (Sx,n False) >>> Needs 18x, squirrel, True) Bind 'cat'
Unbind sx
Finish implied with rule5
Use ruleć: (5x, cow, True) >>> Sees rabbit, Tzue) Finish implied with zule6
Use rule7: (Sx, squirrel, True) >>> sough True]
Obtain
implied fact: Rough True)
Reusa zulel: (Sx, Truat >>> Cold True] Bino 'eat'
Obtala new implied fact: Cold('cal', Izue)
rulež: (old(Sx, True) Baxa cow, True)
Finich implied with rule8
Finish implied
Reuse sulel: (Sx, Trus) cold(fx, True) Bind fx to 'cow"
Ootain
implied fact: Cold''oow', True)
Reuse sule8: Cold(fx, Tzue) a>s Bate cow, Irue) rinish implied with rule8 Finish implied with rule3 Finish implied with rule7 Reusa suled: cold(3x, True) >>> Ects Bind 5x to 'aqui>rel'
True)
('squirzel', "cow', Reuse sules; fats соM, True) sas Sees (3x, rabbit, True] sind
implied fact:
'squizrel'
Obtain new implied fact: Sees ( ('squi rrel', 'rabbit', Truel
Reuse zulel: (ex, cat, Tzuel be Green (3x, False] 522 Bees (8x, cow, Tzue)
Bind 3x 'squizrel'
Finish implled with rulel
zule2: Kindfzebbit, True) (cabbit, squirrel, True) (squircel, rabbit, lrue)
Finish implied with rule2
Reuse rule4; (ex, rabolt, Truel >>> Roxnd False) Bind 9x
Obtain 'squisrel'
implied fact: Round ('quirzee,," False)
Finish implled with rulef
Reuse zuleš: (ex, squirrel, True) Bind $x to 'aquisrel' implfed with
(8x, Palse) Needs (8x, squirrel,
Finish implled with rule6
Reuse sule7: (sxs squizrel, True) s55 Rough (8x, Tzue) 'squisrel'
Bind
Finish implied with rule?
Seuse zuleš: lats соm, True) Sees (8x, rabbit, True)
Bind $x to 'cat'
Ootain new implied fact: sees l'eat, 'rabbit', True)
Reuse zulel: Sex, cat, Tzue] Green (8x, Palsel so0 Sees (8x, CoM, Tzue) Bind 9x 'cat"
Finish implied with rulel
Beuse sule2: Kindizabbit, True) 45 Sees (rabbit, squirrel, True) sas Needs (squircel, rabbit, True)
Finish impliad with rule2
Reuse zule4: Sex, rabbit, Truel 352 (ox, False)
Bind 3x 'cat"
Obtain new implied fact: ('cad's False)
Finish implied with rule4
Beuse zuleš: (exs squirrel, True) s& Green (4x, False) >>> Needs (8x, squirrel, True) Bind
Finish implied with rules Finish implied with zule5
Beuse zule7: Eats squirrel, True) as> (Sx, True) Bind $x to 'cat"
Finish implfed with role?
Reuse zules: 19x, соM, True) >>> sees (5x, rabbit', true!
32

---

Published as a conference paper at ICLR 2026
Bind 9x to "cow'
Obtain implied fact: ('cew", 'rabbit', irue)
Reusa rulel: Sees((x, cat, Truel 66 Sreen (3x, Falsel >>> 50es (9x, CoM, True)
Bind 'cow"
Finish implied with rulel
sule2: kindirabbit,, True) 46 (rabbit, squirzel, True) (squirrel,
rabbit, True)
Finish implied with rule2
Reusa rule4: (ex, rabbit, True) >>> (ox, Folse)
Bind 5x to 'oow"
implied fact: Round('cow', False)
finish implied with ruled
Seusa zuleš: (ix, squirrel, True) Green (9x, False) Needa (8x, squireel, True)
Bind 3x 'cow'
Finish implied with rule5 Finish implted with rule6
Reuse sulel; Eats aquirrel, True) >>> (Sx, Tzue]
Bind
sinish implied with rule?
implind
rule8
All nexly implied Facts: cold('cat', Truel, coldi'cow', True), Eats s'qquiizir,''l' crue), Roughl'cat Truc), Round 'aat" False), Round (' cow'', False), Round squirrel', False), ('cat',, 'rabbit', True), (loow', 'rabbit', Truel, ('squiczel', 'rabbit', True]
'com',
Finish reasoning
FOL Reasoning Extracted from Prover9 Solver:
prove
original conclusion:
3 [all
(8augh (x] -> (old(x))) [assumption].
(Sees (x, Rabbit)
-Ro [sssumpt fon]
5 tall (Bate Cow) >> Sees Rabbit))). [asaurption
7 [all (Bats (x, Squirrel) -> (x)))) (assumption).
(Cold(x)
[sssumption].
9 [ound [goal]
-Eats (x, Cox) (abbitt (clmusify(E)).
11 Ears (Cot, (Cat, Squisrel)- (assurption).
-Eata Squirrell Rxugh(x). (clauaity 17)]
15 Cold(x) Bats (x,
[clsusify(B)].
16 -Sees (x, Rabbit: (x)und(x). [clsusify(4).
Round(CaL). [deny (9)].
19 -Cold(x) Sees Rabbitt)
-Rough
(xld(x) (eleusify(3)).
27 -Seea Rabait). [resolve 16,
29 Sees (x, Rabbitt Roughtx) [1
31 (Cat). (14,
33 -Rough (Cat). (29)
Q.2 TRANSLATE DEBATE
We present a case study from LogicalDeduction where agents collaboratively translate a fruit pricing problem into symbolic languages. Table 24 shows how through three debate rounds, agents identify and correct critical eтrоrs: SAT discovers its inverted ranking (treating as most expensive rather than cheapest), LP clarifies ranking ambiguity, and FOL adds missing ordering axioms. This peer review process yields syntactically and semantically accurate translations across all three symbolic languages, demonstrating how multi-agent debate enhances translation quality.
Case Study from logical deduction
fruit stand sells five fruits: mangoes, kiwis, plums, pears, and watermelons. The kiwis are less expensive than the plums. The pears are the third-most expensive. The kiwis are the second-cheapest. The watermelons are the most expensive.
Context:
Question:
Which of the following is true? A) The mangoes are the third-most expensive.
B) The kiwis are the third-most expensive.
C) The plums are the third-most expensive.
D) The pears are the third-most expensive.
E) The watermelons are the third- most expensive.
33

---

Published as a conference paper at ICLR 2026
Table 24: Case Study of Translation Debate: Agents collaboratively refine their NL-to-SL transla- tions through debate, / indicates correct translation, X indicates incorrect translation.
Round
Agent Trandaatonn
Kay Printe during Dehate
LP Prodiosic Fruit 19x), Moreeiapencion on 10x, Sy), SessExpenss fex, Syn, roMost Expenss
imncition with milliple prolicates for comprrion. Uses separate predicatos for each rinking postion.
Facts: SesoExpenskye (KiMS, plun, Truct, This seleddeeeteeppeeeee (pear, Trus)
FOL, &s Bank (fruls whare E (onessoo
Uses proclinne rarking with onumuual expersine.
Sheaper oo) Premisen: Rank pseat erme onel.
Faccdda ondering refetionahis hetwern
Rank (peaxs, threel. Bank 1kikl tourt...
Eruit amcoundoct (1- (I...] prior_ranke for (14, 2, possFonee" 11 frrritt (priio rankil
Muscc prices integer postioon
Uses condmsints like [KI so (PI (Plons).
Constrinns pos (Kiv1 2.. pos (Peaze) - 3,pos pos cnntrrteoonn
LP Medified Rank 6x, Sn, bocl) where
"I need precise about the ranking system." "
Sa: Swmest expensive, I-leas expensive
Realizes ambiguity in ranking direction needs clarification.
Adled for deriving complese oodering
nivsing desivatios nulss complete codering.
Po Same predicates, but added ondering ations:
need add the ordering relationships between ranle."
XXVY (Baik one) Bank twe) Cheaper
ordering ations specify rank relations.
Plus compleeeees each Eut must lave some nak
but crttical Bedlics nork should be chespot, ust mos eqpasive
made critical
ranking system."
Edentifies that rasking direction was intered.
LP Pinal version with both Expenssive NordExpenssvve
Maintains symbolic while incorporating insights
Complese rules for rutk based comparisors
Final trasolttion is sy synactically coreect and complete.
(5x,
19y, Srue)
Ieselippension
vo Complete with all 10 wedning axions
Fisal vensios include all necessrry constrintss. logical comistrncy oedering
Asymmary
VXVY [Cheager 1X, "Cheaper (1,313
Each fruit and mink uniqueness constmints maintained
Inchespest, st net (' ffrtuiis), pos 1-) for uniqu ranks All constmints propetly onemted: pos (Kivia) spos (Flume]
Conectal
expersive
Successfully corrected the ranking direction.
Final traralatinn sligns with prilien semomiiss. gmax requirenees.
Answer: D
Round
L. translator:
I'll translate this logic problem about fruit prices into LP format:
Predicates:
Fruit(Sx, bool) Sx is one of the five fruits
MoreExxensive(Ssx, $y. bool) ::: Fruit Sx is more expensive than fruit $y
LessExpensive(Sx, Sy. bool) ::: Fruit Sx less expensive than fruit Sy MostExpensivee($x, bool) ::: Fruit $x is the most expensive
eccondMosEExpensiveees bool) ::: Fruit Sx is the second-mon most expensive
ThirdMostExpensiveeSs bool) ::: Fruit Sx is the third-most expensive
FourthMosdExpeesivv(SS,, bool) ::: Fruit Sx is the fourth-most expensive (second-cheapest)
LeastExpensive(Sx, bool) Fruit $x is the least expensive (cheapest)
Facts:
Fruit(mango, True)
Fruit(kiwi, True)
Fruit(plum, True)
Fruit(pear, True)
Fruit( True)
LestExpensive(kiiii, plum, True) Kiwis are less expensive than plums
ThirdMostExpensiieeppaa. True) ::: Pears are the third-most expensive
FourthMostExpeessieekiiii True) Kiwis are the second-cheapest (fourth-most
expensive)
MostExxeensveeeaaaeeeeeo.. True) Watermelons are the most expensive
Rules:
LessExpensive(Sxx, Sy. True) MoreExpensiveeSS,, $x, True)
MorelExpensive(Sx, $y, True) >> LessHExpensiveeSSy, $x, True)
Query:
ThirdNMssEXxpessiiemmaa... True) ::: Option A
ThirdMossExppnnsieekkiii True) ::: Option B
ThirdMossExxpensieelllmm True) Option C
34

---

Published as a conference paper at ICLR 2026
ThircMostExxpaniiveeppnn True) ::: Option D
ThickMossExpeesseeeWdeemmeeee True) Option E
FOL translator:
I'll translate this logical deduction problem into first-order logic.
Predicates:
Rank(frrit,, pos) ::: fruit has price position pos, where pos E {one, two, three, four, five): one most expensive, five cheapest Cheaper(x, y) ::: is cheaper than y
Premises:
Rank(watermelonn, one) ::: Watermelons are the most expensive
Rank(pears, three) ::: Pears are the third-most expensive
Rank(kiwis, four) ::: Kiwis are the second- cheapett (fourth-mw moot expensive)
Cheaper(kiwis, plums) ::: Kiwis are less expensive than plums
AFAPYQ (Rank(F, F, P) A Rank(F, Q)) - (P Q)) ::: Each fruit has exactly one rank
VPAFAG (Rank(F, (F, P) A Rank(G, P)) + (F' = G))
:: Each rank has exactly one fruit
Conclusion:
(mangeas, three) Option A Rank(kiwis, three) ::: Option
Rank(plums, three) Option C
Rank(pears, three) ::: Option
Rank wattermelon, three) Option
SAT translator:
I'll formulate this problem by modeling the fruit prices positions in ordering.
# Declarations
fruits EnumSorr([Mangoes Kiwis, Plums, Pears, Watermelons] price_ Bacee_ran---0 = IntSort([1, 2, 3, 4, 5])
pos Funutionn(fruits] -> (price_rank])
# Constraints
Distinct([:fruitss,, pos(()) ::: Each fruit has unique price rank (1=cheapest, 5=most expensive)
pos(Kiwis) pos(Plums) :::
The kiwis are less expensive than the plums
pos( cos(Pears) == 3 ::: The pears are the third-most expensive
pos(Kiwis) = ::: The kiwis are the second-cheapest (rank 2)
pos Watermelons)) = 5 ::: The watermelons are the most expensive (rank 5)
# Options
Maaid(psssMangocs)
3) A) The mangoes the third-most expensive
is_ (Kiiiss = 3) ::: B) The kiwis are the third-most expensive
(alids)ss(Plumms) == 3) C) The plums are the third-most expensive
is_ (PPars) = 3) ::: D) The pears are the third-most expensive
is_ valid( (Waterreiiaatriiiosss)
::: E) The watermelons the third-most expensive
Round 2
L. translator:
Looking at the other translations, I notice some inconsistencies in approach that need
correction:
1. Price ranking clarification: I need to be more precise about the ranking system. In my translation, I should clarify that rank is least expensive and rank 5 most expensive (or vice versa) to avoid ambiguity.
Missing derivation rules: should add rules that allow us derive the complete ordering of fruits based on the given information.
Let me correct my translation: Predicates:
Fruit(Sx, bool) ::: $x is one of the five fruits
Rank(Sx, $n, bool) ::: Fruit Sx has price rank Sn (5=most expensive, 1=least expensive) LessExxensive(Sx, Sy, bool) ::: Fruit Sx less expensive than fruit Sy
Facts:
35

---

Published as a conference paper at ICLR 2026
Fruit(mango, True)
Fruit(kiwi, True)
Fruit(plum, True)
Fruit(pear, True)
wattrmmatem,l,o,, True)
LessExpensive(kiwi, plum, True) ::: Kiwis are less expensive than plums
Rank 3, True) ::: Pears are the third-most expensive
Rank(kiwi, 2, True) ::: Kiwis are the second-cheapest (rank 2)
Rank(watermelon, 5, True) Watermelons the most expensive (rank 5)
Rules:
LessExpensive(Sx, Sy. True) && Rank($y, Sn, True) >> Rank(Sx, Sm, True) && Sm < Sn ::: If $x is less expensive than Sy, then Sx's rank is lower than Sy's rank Rank(Sx, $n, True) && Rank(Sy, Sm, True) && Sn $m Lessxxpensive(Sx, Sy, True) If Sx's rank is lower than $y's, then $x less expensive than Sy
:::
Query:
Rank(mango, 3. True) ::: Option A
Rank(kiwi, 3, True) ::: Option B
Rank(plum, 3, True) ::: Option C
Rank(pear, 3, True) ::! Option D
(aatewatemmeeoon 3, True) ::: Option
FOL translator
I'll translate this problem into first-order logic, focusing on the ordering of fruits by price. Predicates:
Rank(fruit, pos) ::: fruit has price position pos, where pos {one, two, three, four, five); one most expensive, five = cheapest
Cheaper(x, y) ::: is cheaper (less expensive) than y
Premises:
Rank one) Watermelons the most expensive
Rank(pears, three) ::: Pears are the third-most expensive
Rank(, four) Kiwis are the second-cheapest
Cheaper(kiwis, plums) ::: Kiwis are cheaper than plums
VF
(Rank(F, VF
Rank(E, Q)) + (P = Q)) ::: One rank per fruit
YPAFAG ((onkk [F P) A Rank(G, P)) + (F= ::: One fruit per rank Rank(mangoes, one) V Rank(mangoes, two) V Rannkmangges,, three) V Rank(mangoes, four) V Rank(mangoes, five) ::: Mangoes must have some rank Rank(plums, one) V Rank(plums, two) V Rank(plums, three) V Rank(plums, four) Rank(plums, five) ::: Plums must have some rank
I need to add the ordering relationships between ranks: VX AY (Rank(XX one) A Rank(Y, (Y, two) + Cheaper(Y, (Y) ::: "higher rank -> more expensive"
AX Rank X, two) A RY, three) - Cheaper (Y.)))
AY Rank X, three) Rank four) - Cheaper (Y, X))
AX
Rank X, four) A Rank(Y five) + Cheaper (Y, (X)))
YX AY CChapee (X, Y) Cheaper (Y, X)) ::: "cheaper" is asymmetric
Conclusion:
Rank(mmngoes, three) ::: Option A Rank(kiwis, three) Option B
Rank(plums, three) ::: Option
[parr, three) Option D
Rank(watemmelon, three) ::: Option E
SAT translator
need to correct my SAT translation. Looking at the problem more carefully: # Declarations
fruits = (MaumSSoo([Maoooo) Kiwis, Plums, Pears, Watermelons])
panckrrak--o IntSort([1, 2. 3, 5])
pos Function((fruits] -> (price_rank])
36

---

Published as a conference paper at ICLR 2026
# Constraints
Distintt([ffrrits), pos(f)) :.. Each fruit has unique price rank
pos( (ossks pos(Plums) ::: The kiwis are less expensive than the plums
pos(Pears) == ::: The pears are the third-most expensive
pos(Kiwis) = 2 ::: The kiwis are the second-cheapest
pos( Watermelons) 5 ::: The watermelons are the most expensive
# Options
is_ validipos == ::: A) The mangoes are the third-mos expensive
is valid(pos(Kiwis) = ::: B) The kiwis are the third-most expensive
is_ valii((pss(Plums) == 3) ::: C) The plums are the third-most expensive
(Pears)) = ::: D) The pears are the third-most expensive
is_ watiinlosssatteeeeloooo = 3) ::: E) The watermelons are the third-most expensive I made critical error in my ranking system. In the problem, rank I should be cheapest and rank 5 most expensive, not the other way around. Let me fix this.
Round
LP translator:
Predicates:
Fruit(Sx, bool) ::: Sx is one of the five fruits Rank(Sx, Šn, bool) ::: Fruit $x has price rank Šn (5=most expensive, 1=least expensive) LessExpensive(Sx, Sy, bool) ::: Fruit Sx is less expensive than fruit Sy
MoreExpensive(Sx, Sy, bool) ::: Fruit Sx more expensive than fruit $y
Facts:
Fruit(mango, True)
Fruit(kiwi, True)
Fruit(plum, True)
Fruit(pear, True)
Fruit(watermeloo, True)
LessExpensive(kiwi, plum, True) ::: Kiwis are less expensive than plums
Rank(pear, 3, True) :: Pears are the third-most expensive
Rank(kiwi, 2, True) ::: Kiwis are the second-cheapest (rank 2)
Rank(watermelon, 5, True) ::: Watermelons are the most expensive (rank 5)
Rules:
LessExpensive(Sx, Sy, True) >> MoreEExpesivveSS,, $x, True) ::: If $x is less expensive than Sy, then Sy more expensive than MoreExpensive(Sx, $y, Truc) >> LessExpensive(Sy, $x, True) ::: If Sx is more expensive than Sy. then Sy is less expensive than Sx Rank(Sx. $n, True) && Rank(Sy, Sm, True) && Sn 5m LessExpensive(Sxx Sy, True) ::: If Sx's rank is lower than $y's, then Sx less expensive than Sy
LessExpensive(Sx, Sy, True) && Rank(Sy, Sn, True) && Sn 1 >> Rank(Sx, Sm, True) && Sm < Sn :: If $x is less expensive than $y and Sy has rank greater than 1, then $x has lower rank than $y
Query:
Rank(mango. 3. True) ::: Option
Rank 3, True) ::: Option B
Rank(plum, 3, True) ::: Option C
Rank(pear, 3, True) ::: Option D
Rant(waten,eloon 3, True) ::: Option
FOL translator:
Predicates:
Rank(fruit, pos) ::: fruit has price position pos, where pos E {one, two, three, five, most expensive, five cheapest
Cheaper(x, y) ::: x is cheaper (less expensive) than Premises:
Rank (atermmnnr,,,oo, one) ::: Watermelons are the most expensive
(pars, three) ::: Pears the third-most expensive
Rank(kiwis, four) ::: Kiwis are the second-cheaped
37

---

Published as a conference paper at ICLR 2026
Cheaper(kiwis, plums) ::: Kiwis are cheaper than plums
SFAPYQ((Ran)) P) Rank( F, Q)) (P Q)) One rank per fruit AP (Rank(( F.
P) 2 Rank(G, P)) (F= ::. One fruit per rank Rank(mangoes,, one) V (ankoes,,e two) V Ranngoss,,ss, three) V Rank(mangoes, four) (mangoes, five) ::: each still-unknown fruit occupies some rank Rank(plums, onc) V Rank(plums, two) V Rank(plums, three) V Rank(plums, four) Rank(plums, five) ::: each still-unknown fruit occupies some rank
VX vY (Rank(X. onc) A Rank(Y, two) Cheaper (Y, X)) "higher rank more expensive" (10 ordered pairs)
VX VY (Rank(X (X, one) A Rank(Y, (tree)) Cheaper (, X)) "higher rank
expensive" (10 ordered pairs)
XX vY (Rank(X X, one) A Rank(Yy four) Cheaper (Y, X)) ::: "higher rank more
expensive" (10 ordered pairs)
VX VY (Rank(A (X, one) A Rank(Y, five) Cheaaer (Y))) "higher rank more
expensive" (10 ordered pairs)
YX AY (Rank( X, two) A Rank(YY three) - Cheaape Y X)) ::: "higher rank more
expensive" (10 ordered pairs)
VX VY (Rank( XX two) A Rankk(Y, four) - Cheaper (Y, X)) ::: "higher rank + more
expensive" (10 ordered pairs)
YX AY (Rank(X [X, two) A Rank(Y, five) Cheaper (Y, X)) ::: "higher rank + more
expensive" (10 ordered pairs)
VX AY (Rank( X, three) A Rank(Yy four) + Cheaper (Y, X)) ::: "higher rank + more expensive" (10 ordered pairs)
VX vY (Rank(X [X, three) A Rank(Y, five) + Cheaper X)) ::: "higher rank + more
expensive" (10 ordered pairs)
vX V (Rank(X X, four) A Rank(Y, fice) - Cheaper XY) "higher rank - more expensive" (10 ordered pairs)
AY (Cheuper (X, Y) - -Cheaper (Y, X)) "cheaper" is asymmetric
Conclusion:
(mangoes,, three) ::: Option A
Rank(kiwis, three) ::: Option
Rank(plums, three) Option C
Rank(pears, three) ::: Option
Rank three) Option E
SAT translator:
# Declarations
fruits = FnumSort([Mangoe,, Kiwis, Plums, Pears, Watermelons)) price rank IntSort([1, 2, 4, 5])
pos Function((fruits]] -> (ppice_rank]]
# Constraints
Distinct([fffruits] pfs( ::: Each fruit has unique price rank (1=cheapest, 5=most
expensive)
pos( (owii) pos(msl ::: The kiwis are less expensive than the plums
pos(Pears) == ::: The pears are third-most expensive
== ::: The kiwis the second-cheapest
pos( Watermelons) 5 ::: The watermelons are the most expensive
# Options
is_ valid(ppos(Mangoes)) == 3) ::: A) The mangoes are the third-most expensive ::: B) The kiwis are the third-most expensive
is valid(pos(Kiwis)
is validd(ossPllmms) == 3) ::: C) The plums are the third-most expensive
(Pears)ss(Pears) = =3) ::: D) The pears are the third-most expensive
is valid(pos(Waatrt (Watrrmloos))))) = 3) ::: E) The watermelons are the third-mor most expensive
38

---

Published as a conference paper at ICLR 2026
Q.3 REASONING DEBATE
Case Study from ProofWriter Dataset
The bear is round. The bear likes the cow. The bear visits the lion. The cow is big. The cow is rough. The lion needs the bear. The mouse likes the lion. If someone visits the lion then they are round. If someone needs the bear then they need the lion. If someone needs the lion then they visit the lion. If someone visits the lion and the lion is round then they like the lion. If someone is rough then they need the lion. someone likes the lion and they need the lion then the lion visits the mouse. If someone is rough then they visit the lion. someone likes the lion and the lion is big then they are kind.
Context.
Question. Based on the above information, is the following statement true, false, or un- known? The lion visits the lion.
Options. A) True
B) False
C) Unknown
Round
LP supporter:
After reviewing the reasoning and deductions, I conclude that the statement "The lion vis- its the lion" is True. From rule 2 (Needs ($x, Bear) the given fact Needs (Lion, Bear), we deduce Needs (Lion, Lion). Applying rule 3 (Needs ($x, Lien)
Needs ($x, Lion)) and
Visits ($x, Lion) ), this implies Visits (Lion, Lion). Therefore, the statement holds true. Key Points: Needs (Lion, Bear) is given fact.
2. Rule 2 extends this to Needs (Lion, Lion).
Rule 3 further concludes Visits (Lion, Lion).
Thus, my answer remains True, and my confidence in this answer is 1.0, based on the rigorous application logical rules. Confidence: 1.0
FOL supporter:
Based on my analysis using First-Order Logic (FOL), stand by my initial conclusion of Unknown. Here's 's why:
Logic Consistency: Both attempting to prove Visits (Lion, Lion) and its nega- tion resulted in no contradiction from the given predicates. This means neither can be conclusively established as true or false.
2. Expressiveness of FOL: FOL captured all the nuanced dependencies and conditions in the problem (such as implications involving Needs, Visits, and Likes). However, the information provided still leaves the truth of Visits (Lion, Lion) indetermi- nate.
3. Evidence Limitations: Although key facts like Rough (Cow) Visits (Cow, Lion) and conditions involving Needs and Likes were derived, none directly es- tablish the behavior of Vis Visits (Lion, Lion), nor does denying Visits (Lion, Lion) provide a definitive contradiction.
summary, this clear example of the limitations of the provided context, not of FOL itself. The problem is underdefined based on the assumptions and given predicates, leading the result of Unknown.
Confidence: 1.0
SAT supporter:
To analyze whether "The lion visits the lion" is true, false, or unknown, model the logical context using Z3. Here's formal representation of the problem with the relevant predicates:
1. Individuals: (bear, cow, lion, mouse)
2. Predicates:
Round (x)
Big g (x)
Rough (x)
Likes (x, y)
Visits (x, y)
Needs (x, y)
find(x)
39

---

Published as a conference paper at ICLR 2026
Facts encoded: Round (bear), Likes (bear, cow), Visits (bear, lion), Big (cow), Rough (cow), Needs (lion, bear), Likes (mouse, lion)
Rules encoded: (Visits (x, lion) + Round (x)), (Needs (x, bear)
+ Needs (x, lion)), )
(Needs (x, lion) + Visits lion)), (Visits (x, lion) A Round (lion) + Likes (x, lion)), (Rough (x) (Likes (x, lion) A Needs (x, lion) 4
+ Needs (x, lion)),
Visits (lion, mouse)), (Rough (x) Visits (x, lion)), (Likes (x, lion) A (1ion) - kind(x))
To verify "The lion visits the lion", encode: ?Visits (lion, lion).
Solving this using Z3 for satisfiability under the constraints:
No explicit fact implies Visits (lion, lion).
Rules do not derive Visits (lion, lion).
Unknown/indeterminaa due to lack of evidence.
Thus, the statement remains UNKNOWN. Confidence: 1.0
Plan-and-Solve supporter:
The Plan-and-Solee approach confidently concludes that "The lion visits the lion" true. This conclusion hinges on two key deductions:
The lion needs the bear (Fact 6).
Using Rule 2 ("If someone needs the bear, then they need the lion"), we deduce that the lion also needs the lion.
2. The lion needs the lion.
Applying Rule 3 ('If someone needs the lion, then they visit the lion"), , follows directly that the lion visits the lion.
This reasoning is logically consistent and follows the facts and rules provided. My confi- dence in this conclusion remains high unless another agent proves inconsistency over- looked factor.
Confidence: 1.0
COT Solver supporter:
I conclude that the answer is C (Unknown) because there is no information in the context that establishes whether the lion visits itself. None of the given rules or statements necessar- ily imply that the lion visits itself. While others visiting the lion or liking/needing the lion are explained, no direct condition or chain of reasoning leads to the conclusion about the lion visiting itself. Therefore, this remains undetermined based on the given information. Confidence: 0.9
Round 2
LP supporter:
After reviewing FOL supporter's argument, I must directly address the critical error in their reasoming.
FOL state that Needs (Lion, Lion) "could" imply Visits (Lion, Lion) via Rule 3. There is no "could" here Rule 3 is universal implication: Needs ($x, Lion) > Visits ($x, Lion). This is deterministic, not probabilistic.
2. FOL's claim of "no explicit derived fact ensures Visits (Lion, Lion) conclu- sively" ignores that we do derive this fact through the logical chain: Needs (Lion, Bear) + Needs (Lien, Lion) - Visits (Lion, Lion). The statement "The lion visits the lion" is conclusively True. Confidence: 1.0
FOL supporter:
I maintain that the answer is C (Unknown). My reasoning is as follows:
Logical Consistency: Attempts to both prove Visita t.s (Lion, Lion) and its nega- tion resulted in no contradictions, which indicates that the problem is underdefined with the current rules and facts.
2. Dependence: While Needs (Lien, Bear) implies Needs (Lion, Lion) (via Rule 2), and Needs (Lion, Lion) could imply Visits (Lion, Lion) (via Rule
40

---

Published as a conference paper at ICLR 2026
3), this chain of reasoning isn't fully supported because no explicit or derived fact ensures Visits (Lion, Lion) conclusively.
3. No Contradiction Found: The problem's 's structure does not yield a definitive resolution the truth of the statement.
Thus, the status remains Unknown based on the provided information and logical frame- work.
Confidence: 1.0
SAT supporter:
After reviewing the arguments provided by others, I align with Plan-and-Solve Supporter's reasoning. The chain deductions from Rule 2 (Needs (Lion, Bear) Needs (Lion, Lion)) and then Rule (Needs (Lion, Lion) + Visits (Lion, Lion)) unequivocally establishes that the lion visits itself. This rea- soning directly satisfies the logical derivation that would also construct using Z3.
I now change my answer from Unknown to True, as the required logical steps are valid and complete.
Confidence: 1.0
Plan-and-Solve supporter:
I stand by my original answer that "The lion visits the lion" is True, supported by valid logical deductions from the provided rules and facts:
1. Fact 6: "The lion needs the bear."
Applying Rule ("If someone needs the bear, then they need the lion"), we deduce that the lion needs the lion.
Rule 3: "If someone needs the lion, then they visit the lion."
Since the lion needs the lion, it follows that the lion visits the lion.
This argument addresses the claim that we lack sufficient information. The facts and rules provided suffice to logically establish the truth of the statement. Confidence: 1.0
COT Solver supporter:
After reviewing the ongoing arguments, I acknowledge that the argument provided by Plan- and-Solve supporter is valid: "The lion needs the bear," and by applying Rule 2 and Rule 3, it follows logically that "The lion visits the lion." This was an oversight in my earlier evaluation.
Thus, I change my answer to A (True) as the reasoning is clear and follows directly from the rules and the given facts. Confidence: 1.0
Round 3
LP supporter:
I remain confident that the statement "The lion visits the lion" is True. The reasoning provided earlier, and supported by others, systematically deduces this conclusion:
From Fact 6, Needs on, Bear) is given.
2. Using Rule 2, this implies Needs (Lien, Lion).
By Rule 3, Needs (Lion, Lion) directly leads Visits (Lion, Lion). This chain of reasoning is airtight and rigorously validated. Therefore, the lion indeed visits itself.
Confidence: 1.0
supporer::
After carefully reassessing the arguments, now concur with Plan-and-Solve Supporter and others who assert that "The lion visits the lion" is True. The reasoning that:
1. Fact 6: "The lion needs the bear" leads to "The lion needs the lion" via Rule 2.
The lion needing the lion implies "The lion visits the lion" through Rule 3. This chain of reasoning is both valid and well-supported by the premises. I change my initial conclusion from Unknown to True. Confidence: 1.0
41

---

Published as a conference paper at ICLR 2026
SAT' supporter:
I maintain my revised conclusion with confidence: "The lion visits the lion" is True. Fol-
lowing the reasoning provided:
Fact: Needs (Lion, Bear) is given.
2. Rule 2:
Needs (Lion, Bear) + Needs (Lion, Lion), so Needs (Lion, Lion) is deduced.
3. Rule 3: Needs (Lion, Lion) + Visits (Lion, Lion), so Visits (Lion, Lion) is established.
This reasoning aligns perfectly with the constraints I would also encode in Z3. Therefore, the statement is logically valid and proven to be True.
Confidence: 1.0
Plan-and-Soue supporter:
Thank you. My position remains that the statement "The lion visits the lion" is True. The reasoning follows directly from the structured Plan-and-3 and-Solve approach:
Fact 6: The lion needs the bear.
2. Rule 2: someone needs the bear, they need the lion.
From Fact this gives Needs(Lion, Lion).
3. Rule 3: If someone needs the lion, they visit the lion. Using Needs(Lion, Lion), this directly implies Visits(Lion, Lion). Thus, the statement is logically derived as True based on a clear application of the given rules. No conflicting interpretations invalidate this deduction. Confidence: 1.0
COT Solver supporter:
After reviewing the provided arguments, I now agree with the reasoning by Plan-and-Solve supporter and others who concluded that the statement True. The chain of logic using Fact 6, Rule 2, and Rule 3 is sound, showing that "The lion visits the lion" must hold. I initially missed this chain of reasoning.
Revised Answer: A (True)
Confidence: 1.0
Round 4 (Final answers)
LP supporter: <answer>A</answer> FOL supporter: <answer>A</answer>
SAT supporter: <answer>A</answerz
Plan-and-Soler supporter: <answer>A</answat
COT Solver supporter: <answer>><<
Case Study from LogicalDeduution Dataset
The following paragraphs each describe a set of five objects arranged in a fixed order. The statements are logically consistent within each paragraph.
Context.
In an antique car show, there are five vehicles: a minivan, a limousine, sedan, tractor, and a hatchback. The tractor is the second-newest. The limousine newer than the hatchback. The limousine is older than the sedan. The minivan is newer than the sedan.
Question. Which of the following true? Options. A) The minivan is the second-oldest. C) The sedan is the second-oldest. back is the second-oldest
B) The limousine is the second-oldest.
D) The tractor the second-oldest. E) The hatch-
Round
LP supporter:
Using logic programming (LP), I argue that the correct option is A) the minivan is the second-oldest. My reasoning is as follows. First, the tractor is explicitly stated to be the second-newest, so exactly one vehicle is newer than the tractor. Next, from the pairwise constraints we have S L > H (i.e., the sedan is newer than the limousine, which is
42

---

Published as a conference paper at ICLR 2026
newer than the hatchback), and we also have > S (the minivan newer than the sedan). Intuitively, this pushes the trio (H, L, toward the older side in that order while keeping the minivan ahead of the sedan; with the tractor fixed near the new end (second-newest), the remaining older-side slot that best satisfies all relations is occupied by the minivan, making it the second-oldest Therefore, I select A. Confidence: 0.8
FOL supporter:
Let's s formalize the problem using First-Order Logic (FOL) for precision:
Definitions:
Let Newer (x, y) mean "x is newer than y."
Objects: Minivan (M), Limousine (L), Sedan (S), Tractor (T),
Hatchback (H)
Given information:
1. Newer (S, L) (The sedan is newer than the limousine.)
2. Newer (I,, H) (The limousine is newer than the hatchback.)
3. T is the secend-newest.
4. Newer (M, S) (The minivan newer than the sedan.)
Logical Analysis: We need to determine the ordering from newest to oldest, consistent with the above statements.
Step 1: Identify the newest vehicle. From being second-newest, exactly one vehicle is newer than T. Since M > S and $ > L > Н. neither S, L, nor H can be newest. Thus, minivan is the newest.
Step 2: Order the limousine, sedan, and hatchback. From statement I: Newer (S, L) + S > From statement 2: Newer (L, H) + H. Thus, the order for these three is S> L> H.
Step 3: Position the tractor. From statement 3, the tractor must be the second-newest vehicle. Thus, it follows the minivan: M T.
Final order (newest + oldest): M > T S L>
Evaluation of "second-oldest": The second-oldest is the vehicle fourth in the order. From the final order, this Limousine.
Answer: The limousine is the second-oldest.
Confidence: 1.0
SAT supporter:
I will represent the problem using Z3 to rigorously verify the correct ordering. The five vehicles (minivan, limousine, sedan, tractor, hatchback) are given, and we define their ages using integer variables. Let:
minivan x1
1 limousine x2
sedan = x3
tractor x4
hatchback x5
We impose the following constraints from the context:
1. x4 second newest implies there are three vehicles older than tractor and one
2. x2 > x5: The limousine is newer than the hatchback.
3. x2 x3: The limousine older than the sedan.
4. x1 > x3: The minivan is newer than the sedan.
To solve this:
Define constraints for ages to ensure strict ordering, i.e., no two vehicles have the same age.
Enumerate all possibilities for which vehicle can be "second-oldest" and validate each against the constraints.
Upon solving:
The second-oldest is indeed the limousine, satisfying all conditions. Drvrement/Disagreemeett with Reasoning: No counterarguments provided yet. Conclusion: The correct answer is B) The limousine is the second-oldest.
Confidence: 1.0
43

---

Published as a conference paper at ICLR 2026
Plan-and-Solve supporter:
I have reviewed the arguments provided. The Plan-and-Solve method has clearly analyzed the four clues systematically, arranged the vehicles in order of their relative ages, and iden- tified the second-oldest vehicle as the limousine, All steps were justified and consistent with the given information. No challenges or alternative reasoning have been introduced that con- tradict this deduction. Since the solution is logically sound, supported by verification of all clues, and accurately identifies the second-oldest vehicle, maintain that the answer is B) The limousine the second-oldest. Confidence: 1.0
COT Solver supporter:
stand by my initial answer of C (the sedan is the second-oldest). Here's my reasoning in summary:
The tractor the second-newse newest, so it cannot be the second-oldest.
2. The limousine is older than the sedan and newer than the hatchback, making it older than at least two vehicles, but not the second-oldest.
3. The minivan is newer than the sedan, so it cannot be the second-oldest.
4. The hatchback is the oldest or close to it since the limousine is newer than it, thus elimi- nating it as the second-oldest.
5. The sedan is older than the minivan but not the oldest, making it the second-oldest.
Hence, the sedan fits the criteria for being the second-oldest. My confidence remains high as the logic follows directly from the constraints provided. Confidence: 0.9
Round 2
LP supporter:
After reviewing the arguments in this collaborative debate, I realize critical discrepancy in my initial reasoning. The process of logical deduction and the formal order articulated by the FOL supporter, SAT supporter, and Plan-and-Solve supporter has established the following sequence of ages:
Newest Oldest: M > > S> L>>
This makes the limousine the second-olded vehicle. My earlier labeling of multiple second- oldest vehicles contradicted the constraints and was erroneous. agree that the correct option is B) The limousine is the second-oldest.
Confidence: 1.0
FOL supporter:
After carefully reviewing the arguments from all agents and reapplying First-Order Logic, I disagree with COT SolverSvverrre gaaver_supporter"" deduction the sedan as the second-oldest. Ac- cording to the formal logical constraints derived, the correct ordering from newest to oldest is: M T S > H. Thus, the second-oldest vehicle is clearly Limousine, not the sedan.
This conclusion is consistent with all constraints: the tractor second-newest, and the or- dering ensures all relative relationships hold true.
Final Answer: B) The limousine is the slcond-oldest.
Confidence: 1.0
SAT supporter:
I disagree with COT Si___Sllee_suui supporter claiming the sedan is the second-oldest. Z3's SMT solving clearly resolves all constraints without ambiguity and confirms the order as M > T> > Н. Hence, the limousine is fourth in the sequence, or, equivalently, the "second-oldest." The claim that sedan the second-oldest is incorrect because it explicitly positioned third in the correct sequence (newer than limousine and hatchback but older than tractor and minivan).
Thus, the correct answer remains B) The limousine is the second-oldest. Confidence: 1.0
Plan-and-Solve supporter:
After reviewing the discussion and the constraints again, my initial reasoning and solution re- main valid. The Plan- Snlve approach systematically deduced that the order from oldest
44

---

Published as a conference paper at ICLR 2026
to newest is hatchback limousine > sedan > tractor > minivan. Therefore, the second- oldest vehicle is the limousine, consistent with all clues. The argument for the sedan being the second-oldest is incorrect because it conflicts with the clue stating that the limousine is older than the sedan. This directly eliminates the sedan contender for second-oldest. Final Answer: B) The limousine is the second-oldest. Confidence: 1.0
COT Solver supporter:
now realize that made an error in my earlier reasoning. Upon reviewing the arguments and steps from the other agents, it is evident that the proper ordering is M T > S > L > H (newest to oldest). From this, the limousine is the second-oldest vehicle, not the sedan. Thus, I agree with the corrected logical deductions provided by the other agents. The correct answer is B) The limousine the second-oldest. Confidence: 1.0
Round
LP supporter:
Thank you. After reviewing all the reasoning provided by the agents and scrutinizing my initial position, fully align with the consensus reached by the FOL supporter, SAT supporter, and others. The logical deductions clearly establish that the correct order M > T>> S > L H. making the limousine the second-oldest vehicle. My initial misla- beling has been corrected.
Final Answer: B) The limousine is the second-oldest.
Confidence: 1.0
FOL supporter:
Given the thorough reasoning and agreement across multiple approaches, including clear formal deductions, the correct ordering is M > T Therefore, the second-oldest is the limousine.
SS> T.
> H (newest oldest).
Final Answer: The limousine is the second-oldest. Confidence: 1.0
SAT supporter:
I maintain that the correct answer is B) The limousine is the second-oldest. Using Z3 to represent the constraints ensures complete and rigorous solution, confirming the order: M > T $ > T. H. No contradictions remain in this conclusion, and all logical relationships are satisfied.
Confidence: 1.0
Plan-and-Solve supporter:
The correct answer remains B) The limousine is the second-oldest. The Plan-and-Solve ap- proach has systematically deduced the correct ordering and verified it against all constraints. Other agents have corroborated this conclusion, and alternative claims have been refuted. The reasoning is consistent, logical, and validated.
Confidence: 1.0
COT Solver supporter:
After reviewing and assessing the arguments presented by other agents, I acknowledge my earlier error. The systematic deductions using constraints demonstrate that the correct order M T> $ L H. From this order, the limousine is undoubtedly the second-oldest vehicle. Final Answer: B) The limousine is the second-oldest.
Confidence: 1.0
Round (Final answers)
LP supporter: <answer>B</answer>
FOL supporter: <answer>R</ansver>
SAT supporter: <answer>B</answer>
Plan-and-Solve supporter: <answerB</answer
COT Solver supporter: <answer>B</answer>
45


Reviews:
[{"review_id": "Fb4bwNDg3B", "reviewer": "Reviewer_DhnF", "summary": "The paper proposes a two-stage, sparse multi-agent debate framework to improve complex logical reasoning with LLMs by explicitly combining symbolic-language (SL) and natural-language (NL) paradigms: multiple agents first translate an NL problem into diverse SL formalisms (LP, FOL, SAT) and refine these translations via debate; then SL-solver agents (Pyke/Prover9/Z3 results verbalized) and NL-reasoning agents (e.g., CoT, Plan-and-Solve) debate for several rounds before a majority vote determines the answer. To curb the high cost of multi-agent discussion, the authors introduce an adaptive sparse communication mechanism that gates interactions using a preference score mixing agent confidence and information gain (cosine dissimilarity), coupled with selective memory updates. Experiments on ProntoQA, ProofWriter (depth-5 subset), and BIG-Bench LogicalDeduction across GPT-4, Claude 3.7 Sonnet, and DeepSeek-V3 report new SOTA accuracies while reducing token usage compared to strong single- and multi-agent baselines, with ablations showing gains from both the translation-stage debate and heterogeneous SL+NL agent composition.", "questions": "Please concretely differentiate your contributions from recent multi-agent debate and sparse/topology papers (methodologically, not just empirically). Could you add an ablation that (a) removes the SL-NL cross-paradigm stage and (b) replaces your sparse gating with a strongest prior baseline, to quantify each idea’s standalone lift?\n\nHow often do translation mistakes occur, and how does the system recover when all SL agents share the same systematic error?\n\nPlease include sensitivity analyses for λ and the similarity metric, and compare majority vote against confidence-weighted or learned adjudicators.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Novel combination of symbolic and natural-language reasoning in a multi-agent debate.\n- Sparse-communication mechanism effectively reduces token cost with minimal accuracy loss.\n- Strong empirical performance and ablation studies across several reasoning benchmarks.", "weaknesses": "- The paper’s two main ideas extend well-known patterns (multi-agent debate; sparse topologies like SparseMAD/CortexDebate) rather than fundamentally rethinking them; the conceptual gap from past methods looks narrow.\n- No theoretical framework proving that the proposed agents can perform valid NL to FOL/LP/SAT translation.\n- With heterogeneous agents, simple majority voting can entrench shared biases or spurious agreement.\n- Critical components (e.g., $C_i^d$, Algorithm 1) are under-defined or inconsistent.\n- Lack of significance testing and limited evaluation scope on real-world reasoning tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xZnAUgjyLn", "reviewer": "Reviewer_UAB7", "summary": "The authors introduce a multi-agentic debate system to elicit logical reasoning in LLMs, by integrating symbolic and natural language reasoning. They do this through translating natural language to symbolic language using multiple agents, with the refinement process done through agentic debate. For reasoning, the symbolic output from the translation stage and natural language are used to determine an output through majority voting. To enhance computational efficiency, the authors use a sparse communication strategy within the debate framework.", "questions": "See above.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The use of general purpose (i.e. GPT) and reasoning-enhanced (i.e. deepseek) LLMs helps the reader understand the generalizability of the proposed approach. \n- The results are impressive, as they achieve 100% across several set-ups. \n- The number of baselines used to compare to the proposed approach is sufficient, showcasing the true performance capability of the framework. \n- The use of an adaptive sparse communication strategy to enhance computational efficiency is a very important advantage of this framework.", "weaknesses": "- The whole pipeline depends on the translation quality in the first stage. There is some missing validation of the symbolic translations to enhance the validity of the pipeline. \n- The majority-vote process could highlight systematic biases if multiple agents share similar pretraining biases or translation tendencies.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "x94K6aqD94", "reviewer": "Reviewer_awiG", "summary": "This paper proposes a sparse multi-agent debate framework  to improve the performance of large language models in complex symbolic logic reasoning tasks. This approach combines the advantages of symbolic language (SL) and natural language (NL) reasoning: multiple agents translate the question into different formal logics (LP, FOL, SAT) and optimize the translation through debate. The SL and NL agents then debate and vote on the final answer over multiple rounds. To improve efficiency, the authors design an adaptive sparse communication mechanism, which reduces token consumption while improving accuracy.\n\nMy opinion may be negative at this stage, if the author can address my concerns, I will increase my score.", "questions": "1. Is there a more challenging dataset now? This is necessary to verify the effectiveness of the method in this paper.\n\n2. How is the confidence score obtained in sparse communication?", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. Achieved very good performance on multiple data sets, with many indicators reaching 100%.\n\n2. The experiments are very substantial, and a large number of experiments are done to verify the performance and efficiency of the proposed method.", "weaknesses": "1. The paper repeatedly states that LLMs struggle with complex logical reasoning. However, the metrics shown in Table 1 are all very high, which appears inconsistent with this claim. Moreover, I am curious about the results of directly testing the LLMs—i.e., having the LLM answer the questions without any additional framework—as such baseline results might better support the authors’ argument.\n\n2. While most of the metrics in Table 1 are high, the proposed method does not achieve significant improvements over the baseline. The numerical metrics indicate that all three datasets appear simple, with 100% accuracy achieved multiple times, making the experiment less than convincing.\n\n3. The paper uses the latest powerful baseline models. Does this mean that the method proposed in this paper has high requirements for the base model? Is it applicable to those ordinary models or smaller models such as qwen 2.5 7B?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BLicwMGZYt", "reviewer": "Reviewer_Q5K8", "summary": "The manuscript proposes a sparse multi-agent debate framework that first translates natural-language logical questions into three symbolic languages (LP, FOL, SAT) via agent debate, then lets symbolic and natural-language agents debate the answers, finally voting on the result. An adaptive communication gate prunes low-value inter-agent messages. Experiments on three logical-QA benchmarks report accuracy improvements over a leaderboard that has already reached a degree of saturation.", "questions": "Suggestions:\nRun pipeline on FOLIO multilingual, Chinese LogiQA-V2 and LogiEval-Hard; report delta and failure rates.\nRepeat main experiments with three prompt seeds; report mean and std-dev in Table 1.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 1, "presentation": 2, "contribution": 2, "strengths": "Table 1 shows +0.92 pp average gain over the fully-connected multi-agent baseline on GPT-4, demonstrating that merging symbolic and NL reasoning is more accurate than either alone.\n\nTable 3 ablation adding each solver (LP→SAT→FOL) steadily lifts accuracy (88.06 % → 95.44 %), confirming that diversity of symbolic systems matters.\n\nAlgorithm 1 (Appendix E) gives reproducible pseudo-code, enhancing technical soundness.", "weaknesses": "The experiments are confined to three English synthetic datasets—all of which have already reached saturation. Their small size further undermines the significance of the findings, and no evidence is provided for performance on natural text, whether from specialized domains (e.g., legal, medical) or multilingual contexts.\n\nTable 1 presents performance percentages without accompanying variance estimates (e.g., standard deviation, confidence intervals). Given the dataset’s size, small discrepancies in reported performance could easily be attributed to random noise rather than meaningful differences.\n\nNo token cost comparison is provided against two critical baselines: human-only reasoning, or single-agent Chain-of-Thought (CoT) operating within the same budget. This makes the \"token saving\" claim questionable, as the reference point for comparison remains unspecified.\n\nNo timing analysis: how often do Pyke/Prover9/Z3 timeout?\n\nEq. (1) introduces a preference score but no lemma guarantees that pruning preserves consensus or convergence. No ablation on λ beyond a single value (1.0).\n\nI'm concerned with the prompt sensitivity. Appendix F shows one fixed prompt per role; no robustness check with alternative prompt wordings.\n\nGPT-4 and Claude may have seen ProofWriter during pre-training; no sanitisation check is mentioned.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)