Multi-Agent Debate with Memory Masking
Authors: Hongduan Tian
Xiao Feng
Ziyuan ZHAO
XiangyuZhu
Rolan Yan
Bo Han
Venue: ICLR 2026 Poster (2026)
Keywords: multi-agent debate
memory selection
robustness
Primary area: foundation or frontier models, including LLMs

Abstract: Large language models (LLMs) have recently demonstrated impressive capabilities in reasoning tasks. Currently, mainstream LLM reasoning frameworks predominantly focus on scaling up inference-time sampling to enhance performance. In particular, among all LLM reasoning frameworks, *multi-agent debate* (MAD), which employs multiple LLMs as agents to perform reasoning in the way of multi-round debate, has emerged as a powerful reasoning paradigm since it allows agents to access previous memories to alleviate fallacious content and refine their reasoning iteratively in each debate round. However, although MAD significantly improves the reasoning capabilities of LLMs, in this paper, we observe that there remain erroneous memories, and LLM agents are vulnerable to these erroneous memories. To explore this phenomenon, we provide a theoretical insight that the performance of MAD is highly dependent on the quality of memories derived from the previous debate, indicating that the existence of erroneous memories poses a threat to the performance of MAD. To address this problem, we introduce a simple yet effective multi-agent debate framework, *multi-agent debate with memory masking* (MAD-M$^2$), to improve the robustness of MAD by allowing LLM agents to mask erroneous memories from the previous debate round at the beginning of each debate round. In this way, MAD-M$^2$ can polish the contextual information before each debate round by preserving informative and meaningful memories while discarding the erroneous memories. Extensive experiments and analyses on mainstream mathematical and logical reasoning benchmarks demonstrate that MAD-M$^2$ can identify the erroneous memories and achieve better performance in reasoning than MAD.

Published as a conference paper at ICLR 2026
MULTI-AGENT DEBATE WITH MEMORY MASKING
Hongduan Tian' Xiao Feng' Ziyuan Zhao Xiangyu Zhu?
Bo Han' 'TMLR Group, Hong Kong Baptist University Search Algorithm Group, WeChat, Tencent (cshdtian, xiaofeng, bhanml } @comp. hkbu. edu. hk (joshuazhao, xiangyuzhu, rolanyan, ) @tencent.com com
Rolan Yan
ABSTRACT
Large language models (LLMs) have recently demonstrated impressive capabilities in reasoning tasks. Currently, mainstream LLM reasoning frameworks predomi- nantly focus on scaling up inference-time sampling to enhance performance. In par- ticular, among all L.LM reasoning frameworks, multi-agent debate (MAD), which employs multiple L.MSs as agents to perform reasoning in the way of multi-round debate, has emerged as a powerful reasoning paradigm since it allows agents access previous memories to alleviate fallacious content and refine their reasoning iteratively in each debate round. However, although MAD significantly improves the reasoning capabilities of LLMs, in this paper, we observe that there remain erroneous memories, and LLM agents are vulnerable these erroneous memories. To explore this phenomenon, we provide theoretical insight that the performance of MAD highly dependent on the quality of memories derived from the pre- vious debate, indicating that the existence of erroneous memories poses threat to the performance of MAD. To address this problem, we introduce a simple yet effective multi-agent debate framework, mulli--geet debate with memory masking (MAD-M²), to improve the robustness of MAD by allowing LLM agents to mask erroneous memories from the previous debate round at the beginning of each debate round. In this way. MAD-M can polish the contextual information before each debate round by preserving informative and meaningful memories while discarding the erroneous memories, Extensive experiments on mainstream math reasoning and language understanding benehmarks demonstrate that MAD-M² can identify the erroneous memories and achieve better performance in reasoning than MAD. The code of MAD-M² is available at: http://ggtuubb.co//mmmgrrroo//// groorr//-uMMMM....
INTRODUCTION
Large language models (LLMs) have shown impressive reasoning capabilities in a range of tasks and have attracted inereasing attention in this field. Typically, in LLMs, provided with elabo- rately designed instructions (Ouyang et al., 2022), LLMs can complete the tasks with exceptional ability. Moreover, it has also been demonstrated that the reasoning capability can be further enhanced through fine-graid tne-grained reasoning tricks (e.g., stepwise reasoning (Wei et al., 2022b; Kojima et al., 2022; Zhou et al., 2023) and reflection (Madaan et al., 2023)) with abundant demonstrations (Brown et al., 2020). Despite such an impressive capability, a challenge in L.M reasoning is invalid facts and fallacious content. Although some previous works propose to alleviate the problem via multiple sampling (Wang et al., 2023; Taubenfeld et al., 2025), due to the insufficient generation diversity issue (Si et al., 2025; Hayati et al., 2023), LLMs may be stuck in their own biases, thereby limiting the efficacy of the test-time scaling strategies (Snell et al., 2024; Brown et al., 2024).
instruction-followiii
Inspired by The Society of Mind (Minsky, 1986), Multi-agent debate (MAD (Du et al., 2023)), )), which employs multiple LLMs as agents to perform reasoning via multi-round debate, has recently emerged as an effective approach to overcome the aforementioned two limitations. Specifically, in each debate round (except for the first round), agents required generate answers based on their critical evaluation of all memories from the previous round. In this case, MAD improves the reasoning capability of LLMs mainly from two perspectives. On the one hand, in MAD, agents have access to all previous memories and generate new answers based on their critical evaluation of memories. Thus, the invalid and fallacious content can be identified and corrected. On the other
1 Corresponneece Bo Han (bhanml (bhanml@comm wtt.c/oo....b.........e

---

Published as a conference paper at ICLR 2026
Gerry goes to sleep at 11 pm each night. But he doesn't sleep very well, awakens at random time between 1:00 and 3:59 am, with each minute equally likely (including 1:00 and 3:59). He then reads the hour and minute shown on his clock probability that that number will be divisible by 7?"
three digit number, 2:56 would be 256. What the
Debate Round
Agentl
Agent
Gerry can wake up anytime between 1:00 and 3:59 180 possible minutes. Each minute is written as a three-digit number (e.g. 1:00
Gerry wake up anytime from
1:00 3:59 180 possible minutes.
Each time written three-digit number (HMM).
100).
Count how many of these are divisible by 7:
Find numbers divisible by between 100 and 359.
1:00-1:59 - 8 numbers,
2:00-2259 numbers,
Smallest is 105, largest is 357
33:00-3559 - 9 numbers.
total 37 valid times.
Total favorable 26.
Probability 37/180.
-Probability = 26/180 13/90.
Take both responses as references
Debate Round
Agent
Agent!
Gerry can wake between 1:00-3:59
Gerry can wake between 1:00 180 possible minutes.
180 total minutes (180 possible
numbers from 100-359).
Each time written three-digit number (100-359). Numbers divisible by 7 this range go from 105 to 357
Numbers divisible by 7 in this range
are from 105 to 357.
valid cases.
X
There are 37 such valid times.
Probability 37/180.
Probability = 37/180.
Figure I: An illustration of the effects of erroneous memories. The example is real case picked from the MATH dataset. In MAD, all memories in the previous debate round are considered in the next debate round. However, the memories from the previous round may include erroneous reasoning responses (cf. the responses of debate round 1 in the figure). The erroneous memory may misguide the agent, which was correct, and result in the wrong final answer (cf. Agent in the debate round 2).
hand, by employing previous memories as in-context information for the next debate round, agents may get rid of their inherent bias (Panickssery et al., 2024) and obtain better answers regarding the query. Nevertheless, a concern about MAD remains: Are L.M agents robust to erroneous memories? For example, as shown in Fig. 1, memories derived from the previous debate round may be incorrect (e.g., Agent 2 in the debate round 1), and the agent, which takes all memories into consideration, will be misguided and generate the incorrect answers (e.g., Agent 1 in the debate round 2), thereby undermining the reasoning performance. As far as we know, this concern remains underexplored.
To achieve a comprehensive understanding of MAD, we provide an analysis from the perspective of the probability that MAD can correctly infer the answer given query. The results indicate that the multi-agent debate framework, under mild assumptions, is vulnerable to erroneous memories. Specifically, the performance of LLM agents in debate round closely depends on the number of wrong memories in the previous debate round. When the number of wrong memories from the previous debate round increases, the reasoning capability of LLMs degrades correspondingly, According to our theoretical analyses, in both easy and hard reasoning scenarios, enhancing the robustness of LLM agents to erroneous memories consistently improves the performance of MAD.
Motivated by the empirical observation and theoretical analysis, we propose simple yet effective method, multi-agent debate with memory masking (MAD-M2), to enhance the robustness of MAD to erroneous memories by allowing LLMs to mask the incorrect memories in the previous debate round depending on the evaluations on those memories. Specifically, at each debate round (except for the first debate round), the memories derived from the previous debate round will be critically evaluated, Then, a binary mask vector will be generated according to the evaluations to filter the memories. Then, LLM agents required to generate final answer will be selected from a set of responses by performing majority voting in the final round. Intuitively, the reasoning in each round resembles in-context learning process (Wei al., 2022a).
responses based the preserved memories. The
2

---

Published as a conference paper at ICLR 2026
In this case, erroneous memories are equivalent to poor-quality demonstrations that may distract LLMs (Shi et al., 2023), thereby leading to poor performance. Thus, MAD-M improves MAD's capability by filtering out fallacious content to improve contextual information (Mei et al., 2025).
To validate the effectiveness of MAD-M', mainstream mathematical reasoning (c.g., GSM8K (Cobbe et al., 2021), MATH (Hendrycks et al., 2021), and AIME24&25) and language understanding (e.g.. MMLU-Pro (Wang et al., 2024)) benchmarks are adopted for evaluation. Empirically, MAD-M2 generally achieves better performance than naive MAD (Du et al., 2023) on various reasoning tasks.
Overall, our contribution in this paper can be primarily summarized as the following four aspects:
We find that LLM agents in the conventional MAD framework are vulnerable to erroneous memories derived from the last debate round and may be misled to incorrect reasoning responses (c.f. Fig. 1). In Section 2. we demonstrate the vulnerability of the MAD framework from mathematical perspective. Theoretically, compared to simply increasing the number of sampling/agents, filtering out erroneous memories in the last debate consistently improves the performance of MAD in all cases. As far as we know, ours is the first work to discuss this phenomenon in multi-agent debate. In Section 3, to alleviate the detriment above, we propose a simple yet effective multi-agent debate framework, multi-agent debate with memory masking (MAD-M²), to enhance the capability of MAD by critically evaluating memories in the last debate and masking those incorrect memories. In Section 4, we empirically evaluate MAD-M² on both mathematical and language understanding benchmarks and demonstrate that MAD-M2 generally achieves better performance than MAD.
2 AN OVERVIEW OF MULTI-AGENT DEBATE FRAMEWORK
In this section, we first introduce an overview and formulate the workflow of the conventional multi-agent debate framework (Du et al., 2023). Then, we theoretically analyze the probability of MAD successfully inferring the answer to given query. The results further demonstrate that the naive MAD framework is vulnerable to erroneous memories derived from the last round of debate.
2.1 PROBLEM FORMULATION OF MULTI-AGENT DEBATE
Multi agent debate (MAD) (Du et al., 2023) is a powerful reasoning paradigm that infers the answers to questions through multiple rounds of interactions among multiple LLM agents. Specifically, in the typical multi-agent debate framework, L.MS are first instructed to perform reasoning on the queries independently at the initial round. Then, from the second round, agents are required to critically evaluate the memories of all agents in the last debate round to generate new responses. After a multi-round debate, the final answer is obtained by performing majority voting
set of candidates.
Consider a typical multi-agent debate framework (Du et al., 2023) composed of a total of Nround debate rounds and a set of N₂ LLM agents A ={Ae, A0,, ...; Asx, } that are parameterized with parameters 0,, respectively. Consider a query prompt test E X' that includes both task instructions and the question, where x'= {#frstt .322 ..... x: test } is discrete set of N₁ query prompts. Let M, = [Ao, taetest 1,,-1), A denote the memory set derived from the r-th debate round. Then, for each LLM agent Au, E A, given the memories M, and prompt x test the response generation in the next round is formulated as:
test
N.
1), Asw, testt :M,-1)], where 1 S r S Nround, Mo = 0.
Ae, eest (M) L1.z, where ên arg max P(zl test, MM. 0z).
(1)
z
where è, denotes the 1-th word prediction of the response, रх denotes the previous word predictions of ět, and L denotes the length of the response. The response R1LL a sequence with the length of L.
2.2 MAD IS VULNERABLE TO ERRONEOUS MEMORIES
In the typical multi-agent debate framework (cf. Eq. (1)), LLM agents are required to generate new responses based on the memories derived from the previous debate round. Thus, intuitively, erroneous memories in the previous debate round may potentially undermine the performance of MAD (cf. Fig. 1). However, as far we know, few works have discussed this issue in detail. Thus, in this section, we provide discussion to explore the effect of erroneous memories on the MAD framework.
2.2.1 A THEORETICAL PERSPECTIVE FOR UNDERSTANDING OF MAD
In this section, we first provide a theoretical understanding of MAD. As a comparison, we here adopt CoT-SC (Wang et al., 2023) as the counterpart of MAD. From the perspective of the reasoning
3

---

Published as a conference paper at ICLR 2026
paradigm, both CoT-SC and MAD can be seen as the test-time scaling reasoning framework (Snell et al., 2024; Brown et al., 2024), which improves the robustness and the performance of reasoning by sampling multiple responses for single query. The main difference between the two reasoning paradigms is that CoT--C performs reasoning once and obtains the answer by voting for the majority among a set of responses, while MAD is formulated as the multi-round reasoning among multiple agents, where the final answer is voted from responses generated by agents in the final debate round.
Assumption 2.1. Assume that the probability that an agent independently generates the correct answer only with the given query is p [0, 1]. while the probability that the agent generates correct answers to the query based on the previous memories is assumed to be e e-oNe, where N₂ denotes the number of erroneous memories and a E IR is a constant coefficient that indicates the robustness of agents to erroneous memories. The reasoning is deemed as successful when the number of correct answers Neor in the final round satisfies Neor > 2n,, where Nres denotes the number of responses.
Remark 1. In Assumption 2.1, we formulate the probability that a single LLM agent generates the correct answer to the given query based on the previous memories as e e-N, where N₂ E {0, 1,2,...,, N& denoles the number of erroneous memories. Two aspects are considered here. On the one hand, in an ideal case where the number of erroneous memories is O, the probability of the agent generating the correct reasoning the next debate round will be 1. On the other hand, in the case where all memories are erroneous (i.e., No = NN), the probability of the agent generating correct response to the query will degrade to e e > 0. This aligns with the situation that agents can retain a non-zero probability of generating correct reasoning even if all memories are erroneous.
Proposition 2.2 (CoT--C), Consider total number of Nuc independent responses generated in the way of CoT-SC With Assumption 2.1, the probability that the final answer is correct bounded by:
P(Neor Noc S exp
1
-2NNcl
p << 2'
2
P(Ncor > Nuc 2 exp
1 2 -p)²)
1
-2Nocl
p
The corresponding lower bound and upper bound of cases
< and 2 3 are 0 and 1, respectively. Proposition 2.3 (MAD). Consider a 2-round MAD reasoning, where N₂ agents are involved in each debute round. With Assumption 2.1, the probubility that the final answer is correct is bounded by:
P(N() oot
No Wy exp
-2N
(1 - -
Σ
1 wys e Ni)- < 2'
2
1=0
j=
P(N
>
N₂
2
Σ
W/
1- exe
-2NN (2- (aal
-
Nai)-No) 2 1
2
4=
where w,- = (1- No-1, 5" = |Na ln2] J. For simplicity, the corresponding lower and a upper bounds of situations Na) < 2 and cald-Na) 2 2 are trivial bounds 0 and 1, respectively.
Remark 2. The proofs of Propositions 2.2 and 2.3 are available in Appendix B. In Propositions 2.2 and 2.3, we investigate the capability boundary of CoT-SC and MAD in reasoning. In the case of CoT- SC, we can observe that the performance is mainly determined by the number of sampled responses (i.e., Noc) and the capability of LLM agents in reasoning (i.e., the probability of L.LM agents correctly answering the query p). However, in the case of MAD, the performance is explicitly determined by the number of agents (i.e., Na) and the probability of LLM agents generating correct answers based Since LLMs are vulnerable to erroneous
on the memories in the previous round (i.e., e c memories, except for the initial debate round, the probability e -N(NN fluctuates according to the number of erroneous memories in the previous round. Meanwhile, the performance is also influenced by the probability distribution of the memories in the previous debate round. In the 2-round MAD
case, the distribution is formulated as a binomial distribution: P(X j = -N) "²''p"(( (1 - -p)
According to Propositions 2.2 and 2.3, both results indicate that CoT-SC and MAD rely heavily on the capabilities of agents (i.e., or e -a(j-N)), Specifically, the performance is determined by whether LLM agents powerful enough to correctly perform reasoning on given tasks. Different from C-T-SC, where the capability is inherently determined, the capability of MAD is conditioned on the quality of memories from the last debate round, thereby being particularly vulnerable to erroneous
4

---

Published as a conference paper at ICLR 2026
memories. Consider a case that Na Noc. In the case that LLM agents can hardly generate correct reasoning trajectories (i.e., e-alj-N.. < 2) the bound of MAD will become tighter (i.e., lower probability of correctly performing reasoning) compared to CoT-SC. Moreover, even for the case that agents can probably generate correct reasoning responses (i.e., e e=a(j-Na) Z 2), as wy is smaller than 1, with the mild assumption that e-a()-Na) does not deviate much from (since MAD can easily infer the answers in this case), the lower bound of MAD is also tighter than that of CoT-SC. All discussions above imply that CoT-SC is an ideal case of MAD. These observations, to some extent, further explain why MAD generally fails to outperform CoT-SC in reasoning (Huang et al., 2023).
2.2.2 FURTHER DISCUSSIONS ABOUT THE THEORETICAL RESULTS
According to the propositions above, in both CoT-SC and MAD, the bounds of the probability that the answer to the given query is correctly inferred are discussed in two different cases. Specifically, when p < 1 or ..N. < 2. the probability is upper bounded, while the probability is lower bounded 2
when p 2 or 6 2 3. The two cases implicitly correspond to two types of reasoning cases: hard problem reasoning and easy problem reasoming. Detailed discussions are provided follows.
Hard Problem Reasoning. In the context of hard problem reasoning (HPR) setting (i.e., p< < 2/2 or
e-aNe < 22), it assumes that LLM agents can hardly infer the correct answer to the query due to the difficulty of the problem and many erroneous memories in the previous debate round. In CoT-SC, when the capability of L.M agents (i.e., the probability p) is fixed, increasing the number of responses deteriorates the performance exponentially. In contrast, when the number of responses (i.c., Noc) is fixed, employing more powerful LLM agents (p - 5) tends to improve the reasoning performance. As comparison, in MAD, the performance is mainly determined by the number of agents N, and erroneous memories No. In the case of a 2-round MAD, on the one hand, when the number of agents increases (with N₂ fixed), the performance deteriorates exponentially; on the other hand, with Na fixed, when the number of erroneous memories decreases (i.e., e-a. + 1)), the performance of MAD is improved and approaches to the performance of COT-SC. Thus, the performance of CoT-SC can be viewed as an upper bound of that of MAD HPR. Both aspects above indicate that increasing the number of samples (i.c., Nac and N₂)) in the HPR setting tends to result in performance collapse.
Easy Problem Reasoning, Different from the hard problem reasoning setting, LLM agents can easily infer the answer of the query in the context of the easy problem reasoning setting (EPR, i.e., p 2 or e eaN 2 -)) In this case, for both CoT-SC and MAD frameworks, on the one hand, increasing the number of sampled answers or agents (i.c., Nucc or N₂a) helps improve the performance. On the other hand, employing powerful LLM agents (i.e., p 1) reducing erroncous memories (i.c., Na)) + 1) also contributes to improving the performance of the two paradigms in reasoning.
Thus, with the discussions above, we can summarize two insights. On the one hand, simply increasing sampling numbers (i.e., N.) does not improve the performance in all cases, while reducing erro- neous memories (i.e., No) consistently improves the performance of MAD. On the other hand, the performance of MAD is also constrained by the reasoning performance in the previous debate round (i.e.. Wy). thereby being thcoretically bounded by CoT-SC. With these two observations taken into consideration, we propose improve the performance of MAD by removing the erroneous memories to remove the constraint above and guarantee the performance of both HPR and EPR being improved.
3 MAD-M2: MULTI-AGENT DEBATE WITH MEMORY MASKING
In this section, propose simple yet effective multi-agent framework, multi-agent debate with memory masking (M)D-M2), to critically evaluate memories from the last debate round and mask erroneous memories so that reasoning can be performed with correct memories in the next debate.
3.1 DETAILED IMPLEMENTATION OF MAD-M²
An overview of our proposed 2 framework is visualized in Fig. 2. In general, the core idea of our proposed MAD-M² framework is inserting critical evaluation and masking operations between two debate rounds identify and remove the potential erroneous memories for the next debate round.
Step 1. Initial Debate Round. In the M framework, at the initial debate round, all Na LLM agents are required to independently generate responses to the given query test E X' without any contextual information. Specifically, a total of N₂ responses are independently generated from LLM agents Ao, A, respectively, and formulate the memory vector of the initial round: Mi
5

---

Published as a conference paper at ICLR 2026
While staying in 15-story hotel, Polya plays the following game. She enters alevstor the 6-th floor. She flips a coin five times determine her next five stops. Each time she flips heads, she goes up one floor. Each time she flips tails, she goes down one floor. What is the probability that each of her next five stops the T-th Roor higher? Express your
common fraction.
Answerl Reasoning
Answer
Answer
Reasoning: Ballot/Dyck path
Reasoning
-Find 4 valld sequences out of 16, probability (1% (14)*
*Find the same valid sequences out of probability
-Finn 6 valid sequences sacisfy the condition
total (%) x (6/16) 3/16. 3/16.
Round
see... I...
Answer 3...
Emn... Answer is the same mine, mine wrong. Okay,, take Answer 3.
Let me check... both Answer & undercounted valid paths... Mina correct. Okay, take Answer 3.
Ops, my answer seems wrong, and Anywer is also wrong. would like Answer
ReferencesAnswer
Reference:Answer
Reference:Answer
Evaluation and Masking
flip be move to the 7th floor. -Then checks all possible four-flip sequences to see which keep her 27.
first flip the 7th floor.
-Begin Polya moving to
-Analyzes the problem with Catalan ballot path analogy but notices miematch.
-Systematically lists sequences with or 2 tails across the next moves. Confirms exactly valid sequences remain
after one H.
-Explicitly enumerates
explicit enumeration of 4-flip sequences.
FFllls
finds valid sequences out of 16. -Probability (16) x (6/16) 3/16.
-Identifies again valld sequences out of
-Computes probability (1)) (6/16) 3/16.
PPrbabilltyy = (16) x (6/16) 3/16.
Reason
Memories
An illustration of MAD-M² framework. In general, MAD-M² mainly includes three steps. (i) the initial debate round, LLM agents independently generate responses based on the given query. (ii) The responses generated in the previous round are treated as memories. All memories will be critically evaluated and the potential erroneous memories will be masked for the reasoning the next debate round. (iii) With the preserved memories, agents perform reasoning in the next debate.
Figure 2:
[Ao, fzt ,0), Al8, tottst 0), ..... Agx, (z= .0], where A'e, (xrest, (0) denotes the wrong reasoning responses generated by the i-th LLM agent while Ag, (2test, 0) denotes correct reasoning responses.
Step 2. Evaluation and Masking. After a round of debate, a set of memories regarding the given query is obtained. Here, we consider case where both correct and incorrect reasoning responses are contained. Specifically, the memories can be completely correct, partially correct, or completely wrong. According our previous demonstration, in the case where erroneous memories included, the capability agents generating correct responses in the next debate round will be undermined. To solve this problem, in MAD-M2, we propose to first critically evaluate the memories from the previous debate and then generate a binary vector M = {0, J to mask the potential erroneous memories identified by the agent. Then, we obtain set memories with erroneous ones removed:
M = (() O My, where M Ao.
map (Mr):
(2)
map where 9Ag, M, + {0, N. denotes operation that maps the evaluation of the agent Ae, on the
r-th debate memory M, into a binary mask vector, Mr denotes the new set memories selected by the agent from the memory, and @ denotes the element-wise multiplication. Since all agents are initialized from the same model, the masking operation can be simply performed once for all agents.
Subjective Masking Strategy. In the subjective masking strategy, memories masked according to the subjective evaluations of LLM agents. Specifically, in MAD-M-. LLM agents are required to evaluate memories with the flags "YES". "NO", and "NOT SURE". Depending the strictness of the predefined filtering rule, the "NOT SURE" is treated as either " "YES" or "NO" correspondingly.
Objective Masking Strategy. Inspired by previous work (Fu et al., 2025), also leverage the perplexity of LLMS to perform memory masking. Since high perplexity usually implies that LLMS are not confident of the generated content, answers with high perplexity may probably contain erroneous

---

Published as a conference paper at ICLR 2026
information (e.g., fallacious content and hallucinations). Thus, as objective masking strategy, the perplexity of responses is measured, and only the response with the lowest perplexity is preserved.
Step 3. Reasoning with Masked Memories. With the new set of retained memories M obtained as contextual information through Eq. (2), in the + 1-th debate round, a set of new reasoning responses M,+1 = [Ag, (x MM.)) ), Ao, (atest )..,,
(xtest, .M (o,%)),, [], where both correct and incorrect responses are included, generated from agents with the query and the refined memories.
In our proposed M framework, steps 2 and 3 are conducted iteratively until the end of the debate. At the final round of debate, majority voting performed to obtain the answer to the query.
3.2 MORE DISCUSSIONS
In the previous section, we introduce a simple yet effective MAD-M² method to improve the robustness of the conventional multi-agent debate framework by removing the potential incorrect memories derived in the last debate round. In this section, we propose to provide more analyses and discussions about the MAD-M² framework to have a comprehensive understanding of its property.
Token Consumption Analysis. Due to the multi multi-round interactions among agents in the multi- debate paradigm, the consumption of tokens of reasoning paradigms has become a significant concern in recent works (Liu et al., 2024; Zeng et al., 2025). In our proposed subjective masking strategy, the introduction of the self-evaluation step tends to result more token consumptions. Thus, in order to quantitatively compare naive MAD and our proposed MAD-M frameworks, we follow Liu et al. (2024) to conduct a detailed analysis on the token consumption for both naive MAD and MAD-M2.
Let's denote the token of the query weest as 7 the token of the output of the agent Ag, at the r-th debate round as Tyr Then, we formulate the token consumption of both MAD and MAD-MPas:
N
Nevund N,
Neound N.
Ntoken N
MAD = NaNrond T + Na
+
T.
r=2
Neaund
Nraund
MAD-ME M N. Nound T + 2N
TH
According to the results above, compared to the conventional MAD framework, our proposed MAD- M2 consumes more tokens. Specifically, even in the worst case, where all memories are preserved, No
MAD-M² consumes Na DyNouuud DE TP more input tokens than the naive MAD framework.
Comparison to Existing Works. In literature, many works have explored memory selection in the multi-agent debate framework (Liu et al., 2024; Li et al., 2024; Zeng et al., 2025). Specifically, Li et al. (2024) proposes Sparse MAD (S- (S-MAD) to organize agents in MAD as a graph. One agent can access the response of the other only if the two agents connected. Moreover, Zeng et al. (2025) proposes Selective Sparse MAD (S-MMDD to reduce the exchange of trivial memories and unproductive discussions among agents. Among these works, the most related work to our proposed MAD-M is S-MAD, However, S-MAD only considers static (predefined) topologies for sparse communications among agents. Although dynamic topology is also mentioned in the paper. the memories are selected in random way. However, in our proposed 2 framework, the memories selected from the last debate round and the sparsity of communications are dynamically determined by LLM agents with cost of more token consumption or the internal states (e.g., perplexity) regarding the memories.
EXPERIMENTS
In this section, evaluate our proposed MAD-M on both mathematical reasoning and language understanding benchmarks. We first briefly introduce the experimental settings. Then, we provide detailed quantitative results and further analyses to validate the efficacy and obtain a comprehensive understanding of MAD-M². Complete implementations and settings are available in Appendix D.
4.1 EXPERIMENTAL SETUPS
Models. In our experiments, we mainly consider evaluating MAD-M2 equipped with four mainstream open-source large language models: Qwen2.5-7B Instruct
-n-tsruttt (Yang et al., 2024a), 0wen2.5-Math-7 Instruct (Yang et al., 2024b), DeepSeek-Matte Math-7B-Insttn IB-Inntruct (Shao et al., 2024), and QwQ-32B (Team, 2025). Compared to Qwen2.5-7B. Breen.5-7B-Innttreee Instruct, other models more powerful in math reasoning tasks.
7

---

Published as a conference paper at ICLR 2026
Table 1: Empirical results of accuracy (with standard deviation) and token consumption (T.). We evaluate four mainstream open-source LLMs on both mathematical reasoning and language under- standing benchmarks. We highlight the best performance in bold, and the second-best performance in underline. For fairness, all results are the average of five trials on different seeds (i.e., 41-45).
AIME24
AIME25
MMLU_PP Pro
MATH
GSM8K
Methods
Acc. (%) T.
Acc. (%) T.
Acc. (%) T.
Acc. (%) T.
Acc. (%) T.
Qwen2. 5- 7B- Instruct
CoT
3.3
x0.07
3.3
x0.08
36.0.16.3
x0.08
49.2.13.8
x0.07
64.8.15.44
x0.09
CoT-SC
10.0
x0.43
10.0
x0.45
39.2.14.0
x0.51
58.0
x0.45
83.6.13.9
x0.51
MAD
13.3
x1.00
6.7
x1.00
43.0.12.2
x1.00
55.6113.7
x1.00
91.8
x1.00
MAD-M*(S)
13.3
x x1.13
3.3
x1.21
43.6+2.3
x1.17
56.8+2.1
x1.20
89.0+4.0
x x1.25
MAD-M
6.7
x0.68
6.7
x0.65
42.4+5.3
x0.69
54.2+2.6
x0.67
89.0+2.0
x0.72
Owen2 5 Mat.- 7B ruct
CoT
13.3
x0.08
10.0
x0.08
39.6+0,9
x0.08
77.8+5,2
x0.08
9.62
x0.08
CC
23.3
x0.53
10.0
x0.48
41.4+5.1
x0.47
82.01:4.7
x0.44
96.41.77
x0.45
MAD
6.7
1.00
6.7
x1.00
34.24:2.9
x1.00
71.24:3.3
x1.00
95.2+
x1.00
MAD-M
6.7
x 11.7
6.7
x1.37
35.0::2.2
x1.36
71.24:3.3
x1.41
95.2
x1.44
MAD-M'(O)
13.3
x0.67
13.3
x0.62
37.04:2.
x0.62
80.243.8
x0.62
95.411.7
x0.60
DeepSeck
CoT
0.0
x0.07
0.0
x0.09
27.8+77.9
x0.17
34.2::4.5
x0.08
79.0
x0.09
CoT-SC
3.3
x0.44
0.0
x0.46
32.2+4433
x0.99
44.4
x0.47
88.8
x0.52
MAD
0.0
x1.00
x1.00
31.24:5.
x1.00
38.612.6
x1.00
81.212.7
x1.00
MAD-M (S)
0,0
x1.32
0.0
x1.30
30.815.2
x1.66
37.015.1
x1.31
80.8
x1.33
(OD-M
0.0
x0.67
0.0
x0.67
30.8+66.4
x0.75
39.8+3.6
X0.68
82.2+-444
x0.71
QHQ 328
COT
80.0
x0.13
56.7
x0.14
75.2414.9
x0.11
80.8116
x0.10
97.41..3
x0.08
CoT-SC
80.0
x0.85
80.0
x0.85
6.84+
x0.63
81.6
x0.61
97.442.3
x0.44
MAD
76.7
x 1.00
73.3
x1.00
75.4+4.2
x1.00
79.24228
x1.00
97.242.3
x1.00
MAD-M((S)
76.7
x 1.28
73.3
X x1.25
6..8+66.
x
x1.22
79.6+2.33
x1.27
97.8+1.99
x1.32
MO)-M
80.0
x0.67
76.7
x0.90
75.2415.
x0.69
75.0
x0.67
96.611.8
x0.56
714
59.6
6.6
K7
0.0
90.2
55.4
15.4
15.5
0.8
95.0
70.4
6.8
5.5
67
95.8 76-4
11.8 15.3 10.0
80.2
12.4
12.4
67
6.7
77.4
71.
16.8
13.3
13.3
71.6
71. 26.8
15.3
05
CSMEX
AIN
ESMIKK
AINS
CSMAX MATO KIMER
(a) Qwen2.5-7B (S)
(b) Qwen2.5-7B (L) (c) Qwen2.5 1-atn2.55Math--- 7B (S) (d) Owen2.5-Math (L)
Figure 3: Visualization of erroneous memory identification of different LLMs. We here examine the erroneous memory identification capability of different LLMs. "S" denotes the strict rule, and "L" denotes the loose rule. According to the results, the objective masking strategy generally works better on the powerful LLMs, while the subjective masking works better on relatively weak LLMs. Benchmarks. In experiments, we evaluate 2 both mathematical reasoning benchmarks (i.e., GSM8K (Cobbe et al., 2021), MATH (Hendrycks et al., 2021), AIME:24255 and language understanding benchmarks (i.e.. MMLU Pro (Wang et al., 2024)). In this case, GSM8K, MATH and MMLU Pro are treated as easy reasoning tasks, while AIME24& are the hard reasoning tasks.
Baselines. In our experiments, the following frameworks are adopted as baselines: (1) Chain-of- Thought (CoT) (Wei et al., 2022b); (2) Self- Seff-Consistency Mfain-of-Thougeee (CoT-SC) (Wang et al., 2023) with 6 independent reasoning paths; and (3) Multi-Agen Debate (MAD) (Du al., 2023). For all MAD frameworks above, the number of agents and debate rounds are set to 3 and 2, respectively.
4.2 MAIN RESULTS
We evaluate 2 with both subjective (MAD-M and objective (MAD-M²(O)) masking strategies on both mathematical reasoning and language understanding benchmarks. The quantitative results, including accuracy (with standard deviation) and token consumption, reported in Table 1.
8

---

Published as a conference paper at ICLR 2026
94.5
19.9
13.3
594
AccurT
91.5
$7.7
43.4
13.3
50.0-
42.7
10.0
$6.4
140
Male M4
6
10
8 16
8 5
6
10
8 10
Num of Agents
Num of Agents
Num Agants
Num of Agents
Num of Ag ents
(a) GSM8K
(b) MATH
(c) MMLU_PPP Pro
(d) AIME24
(c) AIME25
Figure 4: Effect of scaling the number of agents in the case of Qwen2.5-7B-Instruus The number
of agents is increased from 3 to 10. According the figures, both frameworks benefit from the increase of the number of agents and MSD-MP(S) tends to achieve better performance in most cases.
33
855.8
33.6
34.6
2.5
225
84.0-
28.7
1.6
1.6
31.5
0.8
0.8
20.5
R TD
R
B 10
0.0
7
9 10
7
8 10
Num of Agents
Num of Agents
Num of Agents
Num of Agents
Num Agents
(a) GSM8K
(b) MATH
(c) MMLU Pro
(d) AIME24
(e) AIME25
Figure 5: Effect of scaling the number of agents in the case of DeepSeek-Math- -eee-----Innsttt The
number of agents is increased from 10. According to the figures, both frameworks act differently
when the number of agents increases and MAD-M achieves better performance in most cases.
Observation 1. MAD-M'outpprroou MAD in most cases, As shown in Table 1, we can observe that MAD-M achieves better performance than MAD in most cases with different LLMss. Specif- ically, in the case of Owen2.55-7B-Instrucc MAD-MP(S) achieves 0.6% and 1.2% improvements on MMLU Pro and MATH benchmarks, respectively. Moreover, in the case of Qwen2.5-Math- 7B-Instruct, MAD-M²(O) achieves 2.8%, 9.0%, and 0.2% on MMLU_ Prr, MATH, and GSM8K benchmarks. Meanwhile, MAD-M²(O) also achieves 6.6% improvements on both AIME24&25.
Observation 2. MAD with powerful LLMs tends to achieve better performance on reasoning tasks. In Table 1. MAD with weak LLMs, such as Qwen2.5-7B- --nestst-B-n achieves better perfor- mance on easier tasks (c.g.. MATH and MMLU (MM))_rro.. while MAD with powerful LLMs. such as Qwen2.5-Math-7B and QwQ-32B, achieves better performance on both easy and hard benchmarks simultaneously. This phenomenon aligns with our theoretical analyses that MAD benefits from the powerful reasoning capability. Masking erroneous memories helps improve the reasoning capability.
Observation 3.
The efficacy of masking strategies is highly dependent on the intrinsic capability of LLMs. As shown in Table 1, in the case of MAD with weak LLMs (e.g., Swee225-7B-IInsrrun MAD-M' 2M generally achieves better performance than MAD-M (O)) However, in the case of MAD with powerful LLMs (e.g., Qwen2. @wwn2.55Matth 5-a55- - Instructt & QwQ-32B), MAD-M²(O) demonstrates its advantage more challenging AIME benchmarks. Specifically, MAD-M² achieves significantly better performance than MAD-M2(S) with about 50% tokens. This indicates that the perplexity- -bssed masking strategy provides more precise signal for erroneous memory identification in reasoning.
4.3
MORE ANALYSES
Token Consumption. We here take the conventional MAD framework the baseline and compare the token consumptions of all reasoning frameworks in Table 1. According to the results, multi-agent reasoning paradigms typically consume more tokens than single-agent paradigms (i.e.. CoT & CoT- SC) due to multi-round interactions (i.e., inputs and outputs) and memories in prompts. Consistent with the analysis results in Section 3.2, subjective masking consumes more tokens (13%~<1 (%) due to the extra subjective evaluation phase. In contrast, when integrating objective masking with MAD-M2. the token consumption decreases by about 30% almost all cases due fewer memories in prompts.
Erroneous Memory Identification. To figure out whether MAD-M² can correctly identify those erroneous memories, we consider two different rules here: (1) Striet rule: All erroneous memories are identified. (2) Loose rule: Correct memories are dominant in the retained memories. The results are visualized Fig. 3. Note that the strict rule does not perfectly fit objective masking since only one answer is preserved when performing objective masking. According to the results, the one hand, we can observe that MAD-M consistently achieves good performance on simple reasoning tasks (e.g., GSM8K & MATH). On the other hand, subjective masking generally achieves better performance with the loose rule, where correct memories take a dominant role in preserved memories. In addition, 2 with weak L.MMs prefers subjective masking (ef. Figs. 3(a) & 3(b)) while
9

---

Published as a conference paper at ICLR 2026
$3.0
132
Accuray
42.5
accuraty
56.0
Accurnay
00.0
$5.0
62.4
C
$10
0
6.
Num Rounde
Num Rounds
Mum of Rounds
Num Rounds
Num of Rounds
(a) GSM8K
(b)
MATH
(c) MMLU Pro
(d) AIME24
(e)
AIME25
Figure 6: Effect of scaling the number of debate rounds in the case of Qwen2.5-7B- Instruct. The
number of debate rounds increases from 2 to 10. According to the figures, both frameworks basically
benefit from the increase in debate rounds. In most cases, MAD-M (S) outperforms MAD-M' (O).
Accuney
Acourscy
12.0
10
20.9
20.0
1.0
26.3
(0)
10
10
10
(a) GSM8K
(b) MATH
(c) MMLU Pro
(d) AIME24
(e) AIME25
Figure 7: Effect of scaling the number of debate rounds w.r.t DeepSeek-Math-7B--. Innstrutt.
The number of debate rounds increases from 2 to 10. According to the figures, the performance of different cases varies. In most cases, (OD-M²(O) achieves better performance than MSD-M²(S).
MAD-M2 with powerful LLMs prefers objective masking, in particular on hard reasoning tasks (cf. Figs. 3(C) & 3(d). Generally, objective masking is more robust as the difficulty of tasks increases.
Scaling of Number of Agents & Debate Rounds. We investigate the scaling capability of MAD-M². As comparison, we take MAD the baseline. In the case of scaling the number of agents, we fix the number of debate rounds as and increase the number of agents to 4, 5, 6, 8, and 10, respectively. The results of both MAD and MAD-M equipped with two different LLMs (Qwen2.5-7B and DeepSeek- Math-7B) are visualized in Figs. 4 and 5. Empirically, both MAD and MAD-M² benefit from the increase in the number of agents. From the perspective of masking strategy, subjective masking performs better than objective masking on MAD-M² with weak LLMs (e.g.. Jowen2.5-7B-Instruue while objective masking works better on powerful LLMs (e.g..
1aaea-.5-Maa--7--Intttre the case of scaling the number of debate rounds, we fix the number of agents to 3 and increase the number of debate rounds to 4, 5, 6, and 10, respectively. The results with respect to two LLMs are visualized in Figs, 6 and 7. Increasing the number of debate rounds does not consistently improve performance in all cases. Specifically, increasing the number of debate rounds benefits However, in the case where the powerful models (e.g.,
In
the frameworks with Qwen2.5-7B-Instrve DeepSeek-Maiie Math-7B-Inntrucch are equipped, with the increase of debate rounds, the performance even drops. In addition, consistent with the case of scaling the number of agents, subjective masking performs better than objective masking on MAD-M² with weak LLMs while objective masking works better MAD-M with powerful LLMs. More analysis results are available in Appendix E.
between 2 and Sparse MAD. We here compare the performance between our proposed 2AD-M² and Sparse MAD, which is related work of our method. Here, we consider an MAD with 6 agents and 2 debate rounds. For Sparse MAD, each agent only obtains memories from the two agents it connects to. The results performance than Sparse MAD in most cases, especially on those hard reasoning tasks (e.g., AIME24 & 25). For example, in the case of Qwen2.5-7BB Swen2..-7BBInstrueet. MAD-M2 outperforms Sparse MAD. Although Sparse MAD excels MAD-M² on MATH in the case of Qwen2.5 ----2.h--------- Matt-77BIInstruct its catastrophic drop in MMLU Pro suggests a lack of robust generalization compared our proposed M MAD-M Thus, MAD-M can generally achieve more balanced performance on the reasoning tasks.
Comparison
reported in Table 3. Generally, 2AD-M² achieves better
CONCLUSION
Our work mainly focuses on the robustness of the conventional multi-agent debate framework. In this paper, we find that MAD is vulnerable to erroneous memories from the last debate round. The erroneous memories may mislead agents from the original correct reasoning to erroncous ones. To understand this phenomenon, we theoretically demonstrate that the reasoning capability of LLM agents in the next debate round is closely related to the number of erroneous memories in the previous round. Inspired by this observation, we propose simple yet effective framework, multi-agent debate with memory masking, to enhance the reasoning capability of MAD by masking potential erroneous memories. Extensive experiments and analyses demonstrate the efficacy of our proposed method.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENT
HDT, XF and BH were supported by RGC Young Collaborative Research Grant No. C2005-24Y, Tencent WeChat Faculty Research Award, and HKBU CSD Departmental Incentive Scheme.
ETHICS STATEMENT
We believe that our work raises no significant ethical concerns.
REPRODUCIBILITY STATEMENT
In this paper, both theoretical and empirical results are included. For theoretical results, the necessary assumptions and the completed proofs have been provided in the main paper and the appendix, respectively. For the reproducibility of our experiments, we have provided detailed explanations about our experimental settings and the necessary introduction to the datasets in our paper.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et Gpt-4 technical report. arXiv preprint 2:Xxx:2303.08744 2023.
Pranjal Aggarwal and Sean Welleck. L1: Controlling how long reasoning model thinks with reinforcement learning. In Second Conference on Language Modeling, 2025.
Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. Self-RAG: Learning to retrieve, generate, and critique through self-reflcction. ICLR, 2024.
Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, K et al. Graph of thoughts: Solving elaborate problems with large language models. In AAAI, 2024.
Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V Le, Christopher Ré, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling. arXiv preprint arXiv:400..21787, 2024.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. NeurIPS, 2020.
Hyeong Kyu Choi, Xiaojin Zhu, and Sharon Li. Debate or vote: Which yields better decisions in multi-agent large language models? In NeurIPS, 2025.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:21 10. 14168, 2021.
Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenenbaum, and Igor Mordatch. Improving factuality and reasoning in language models through multiagent debate. In ICML, 2023.
Siqi Fan, Peng Han, Shuo Shang, Yequan Wang, and Aixin Sun. Cothink: Token-efficient reasoning via instruct models guiding reasoning models. arXiv preprint arXiv:2505.2200 2025.
Wenqi Fan, Yujuan Ding, Liangbo Ning. Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. A survey on rag meeting lmms: Towards aarmenel- large language models. In Proceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining, pp. 6491-6501, 2024.
Yichao Fu, Xuewei Wang, Yuandong Tian, and Jiawei Zhao. Deep think with confidence. arXiv preprint arXiv:2508. /5260, 2025.
11

---

Published as a conference paper at ICLR 2026
Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yixin Dai, Jiawei Sun, Haofen Wang, and Haofen Wang. Retrieval-augmented generation for large language models: A survey, arXiv preprint arXiv:2312.10997, 2(1), 2023.
Shirley Anugrah Hayati, Minhwa Lee, Dheeraj Rajagopal, and Dongyeop Kang. How far can we extract diverse perspectives from large language models? arXiv preprini urXiv:2311.09799 2023.
Dan Hendrycks, Collin Burn,, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.0333 03300, 2020.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021.
Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. Large language models cannot self-correct reasoning yet. arXiv preprint arXiv:2310.01990 2023.
Ziqi Jin and Wei Lu. Tab-cot: Zero-shot tabular chain of thought. arXiv preprint 17X11 2023.
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. In NeurlPS, 2022.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, al. Retrieval-augmented genera- tion for knowledge-intensive nlp tasks. NeurIPS, 2020.
Yunxuan Li, Yibing Du, Jiageng Zhang, Le Hou, Peter Grabowski, Yeqing Li, and Eugene Ic. Improv- ing multi-agent debate with sparse communication topology. arXiv preprint 2477:24006.1776, 2024.
Jinqing Lian, Xinyi Liu, Yingxia Shao, Yang Dong, Ming Wang, Zhang Wei, Tianqi Wan, Ming Dong, and Hailin Yan. Chatbi: Towards natural language to complex business intelligence sql. arXiv preprint arXiv:2405.00522 2024.
Tongxuan Liu, Xingyu Wang, Weizhe Huang, Wenjiang Xu, Yuting Zeng, Lei Jiang, Hailong Yang, and Jing Li. Groupdebate: Enhaneing the efficiency of multi-agent debate using group discussion. arXiv preprint arXiv:2409. 1405), 2024.
Haotian Luo, Li Shen, Haiying He. Yibo Wang, Shiwei Liu, Wei Li, Naiqiang Tan, Xiaochun Cao, and Dacheng Tao. O1-pruner: cength-harmonizing fine-tuning for ol-like reasoning pruning. arXiv preprint arXiv: arXi::2001.12570, 2025.
LINHAO LUO, Yuan-Fang Fang Li, Gholamreza Haffari, and Shirui Pan. Reasoning on graphs: Faithful and interpretable large language model reasoning. In ICLR, 2024.
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang. Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterative refinement with self-feedback. In NeurlPS, 2023.
Lingrui Mei, Jiayu Yao, Yuyao Ge, Yiwei Wang, Baolong Bi, Yujun Cai, Jiazhi Liu, Mingyu Li, Zhong- Zhi Li, Duzhen Zhang, et al. A survey of context engineering for large language models. arXiv preprint arXiv:2507. 13334 2025.
Marvin Minsky. The society of mind. Simon & Schuster, Inc., USA, 1986. ISBN 067 1607405.
Alexander Novikov, Ngân Vü, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang. Adam Zsolt Wag- ner, Sergey Shirobokov, Borislav Kozlovskii, Francisco JR Ruiz, Abbas Mehrabian, et al. Alphae- volve: A coding agent for scientific and algorithmic discovery. arXiv preprint arXiv:2506.1311 2025.
12

---

Published as a conference paper at ICLR 2026
Long Ouyang, Jeffrey Wu, Xu Jiang. Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. NeurIPS, 2022.
Arjun Panickssery, Samuel Bowman, and Shi Feng. Llm evaluators recognize and favor their own generations. NeurlPS, 2024.
Parth Sarthi, Salman Abdullah, Aditi Tuli, Shubh Khanna, Anna Goldie, and Christopher D Manning. Raptor: Recursive abstractive processing for tree-organized retrieval. In ICLR, 2024.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepscekmath: Pushing the limits of mathemat- ical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024.
Yi Shen, Jian Zhang, Jieyun Huang, Shuming Shi, Wenjing Zhang, Jiangze Yan, Ning Wang, Kai Wang, Zhaoxiang Liu, and Shiguo Lian. Dast: Difficult--daaptiv slow-thinking for large reasoning 04442, 2025.
models. arXiv preprint
Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H Chi, Nathanael Schärli, and Denny Zhou. Large language models can be easily distracted by irrelevant context. In ICML, 2023.
Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik R Narasimhan, and Shunyu Yao. Reflexion: language agents with verbal reinforcement learning. In NeurIPS, 2023.
KaShun Shum, Shizhe Diao, and Tong Zhang. Automatic prompt augmentation and selection with chain-of-thougle thouuht from labeled data. arXiv preprint arXiv:2302./2282) 2023.
Chenglei Si, Diyi Yang, and Tatsunori Hashimoto. Can LLMSs generate novel research ideas? large- scale human study with 100+ NLP researchers. In ICLR, 2025. URL https: //tppreei//peeeeieee net forum? i M2 3dTGWcZy.
Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling IIm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.033144 2024.
Jiashuo Sun, Chengjin Xu, Lumingyuan Tang, Saizhuo Wang. Chen Lin, Yeyun Gong, Lionel Ni, Heung-Yeeu Yeung Shum, and Jian Guo. Think-on-graph: Deep and responsible reasoning of large language model on knowledge graph. In ICLR, 2024.
Amir Taubenfeld, Tom Sheffer, Eran Ofek, Amir Feder, Ariel Goldstein. Zorik Gekhman, and Gal Yona. Confidence improves self-consistency in llms. arXiv preprint arXiv:2502.06233, 2025.
Qwen Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025. URL
https: /ttp:////wwwmmmmmm wtt.g/wwwl.....haaa. ///llg///////oo/oooooooooloool// holll//gg-2/0 qwq- 32b/.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe Le, Ed Chi, Sharan Narang, Aakanksha Chowdh- ery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In ICLR, 2023.
Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. NeurlPS, 2024.
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. Emergent abilities of large language models. arXiv preprint (tXXi:2206607682.. 2022a.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. In NeurIPS, 2022b.
Yuchen Yan, Yongliang Shen, Yang Liu, Jin Jiang, Mengdi Zhang, Jian Shao, and Yueting Zhuang. Inftythink: Breaking the length limits of long-context reasoning in large language models. arXiv preprint 0tl::2533.06662 2025.
13

---

Published as a conference paper at ICLR 2026
An Yang. Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang. Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115. 2024a.
Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, Keming Lu, Mingfeng Xue, Runji Lin, Tianyu Liu, Xingzhang Ren, and Zhenru Zhang. Qwen2.5-math technical report: Toward mathematical expert model via self-improvement. arXiv preprint MxXiv:2409.12122 2024b.
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. In NeurlPS, 2023.
Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. Star: Bootstrapping reasoning with reasoning. NeurIPS, 2022.
Yuting Zeng, Weizhe Huang, Lei Jiang, Tongxuan Liu, Xitai Jin, Chen Tianying Tiana, Jing Li, and Xiaohua Xu. S²-mad: Breaking the token barrier to enhance multi-agent debate efficiency. In NAACL, 2025.
Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc V Le, and Ed H. Chi. Least-to-most prompting enables complex reasoning in large language models. In ICLR, 2023.
14

---

Published as a conference paper at ICLR 2026
A DETAILED RELATED WORK
Recently, more and more attention has been attracted to large language models (Achiam et al., 2023; Yang et al., 2024a), which are parameterized with millions of weights and trained on numerous corpora. Along with the impressive improvements in performance on conventional natural language tasks, one important emergent ability of these large language models is performing reasoning tasks.
LLM Reasoning, As the most impressive emergent capability of large language models, reasoning has received more and more attention from the community. Such a capability enables LLMs to address complex questions in logical way as human beings based on their prior knowledge without further training. So far, numerous research works have been conducted to improve the reasoning capability of L.MMs. One typical framework to enhance the reasoning capability of LLMs is Chain- of-Thought (CoT) (Wei et al., 2022b). In COT, LLMs are instructed to break the tasks into sequential steps and solve the problem step by step. Based on CoT, some other variants are proposed to further improve the reasoning capability of LLMs (Zelikman et al., 2022; Wang et al., 2023; Shum et al., 2023). Among these works, a famous work is Self- Self-Connsstency Chain-of- Chain-of-Thought (CoT-SC) (Wang et al,, 2023). CoT-SC improves the reasoning capability of LLMS significantly by simply sampling multiple reasoning trajectories and performing majority voting on these trajectories for the final answer. In addition to modeling the reasoning trajectories into the chain of thoughts, other works also explore other logical structures. For example, Jin & Lu (2023) (Tab-CoT) models the reasoning trajectories in a highly structured way. Yao et al. (2023) models the reasoning trajectories into a Thought, ToT). This further facilitates evaluating multiple reasoning paths and self- assessesssmm However, both chain- based and tree-based reasoning paradigms are constrained to address linear reasoning tasks. To solve this problem, Graph-of-Thought (Besta et al., 2024) is proposed to model the reasoning trajectory into a flexible graph to solve non-linear tasks.
tree-like path (a.k.a. Tree-of-Thought,
Multi-agent Debate. Multi-agent debate (MAD) (Du et al., 2023) is equivalent to scaling of conventional reasoning paradigms. By comparing the answers derived from a set of LLMs, the fallacious content can be removed, and the correct answers can be achieved. The main goal of MAD is to perform reasoning with multiple LLMs in the way of debate as human beings do. Specifically, in this case, the reasoning is performed by allowing multiple LLMs to debate their previous responses in a multi-round way. The final answer is obtained via performing majority voting on the set of responses in the last debate round. By considering previous responses, the answers generated in the new round can be further refined. Compared to the conventional reasoning paradigms, such as COT, MAD consumes more resources (e.g.. tokens and time) (Du al., 2023; Li al., 2024; Liu et al., 2024; Zeng et al., 2025). Moreover, due to the randomness of response generation, multi-agent debate also suffers from the noisy responses (Zeng et al., 2025). Thus, a series of works is proposed to reduce the resource consumption and improve the performance of MAD by polishing the memories. Specifically, Sparse MAD (S-MAD) (Li al., 20244 proposes to formulate the MAD framework as graph and sparsify the topology of the graph, which is able to simultaneously reduce resource consumption and improve the performance. Moreover, Liu et al. (2024) proposes Group Debate (GD) to divide all agents into several debate groups and only allow agents to debate in their own group. Further, Selective Sparse MAD MAAD (Zeng ct al., 2025)) is proposed to reduce the redundant content and unproductive discussions in debate to improve the efficiency and performance of MAD. Recently, Choi et al. (2025) conducted a theoretical analysis on the MAD framework, demonstrating that majority voting plays an important role in MAD achieving good performance in reasoning tasks.
Context Engineering, Context Engineering methods (Mei et al., 2025) manipulate unstructured streaming LLM reasoning to construct structured context systems, enabling sophisticated LLM--rivee MAD systems and agentic systems by managing scaling context length, where numerous tokens are irrelevant or confusing for answers. Recent advancements in context engineering are two-fold: context augmentation and context compression. Context augmentation methods aim at extending critical information in context to optimize reasoning performance. methods (Lewis et al., 2020; Gao et al., 2023; Fan et al., 2024) provide access to external information sources, including databases (Lian et al., 2024), knowledge graphs (Sun et al., 2024; LUO et al., 2024), and textual document collections (Asai et al., 2024; Sarthi et al., 2024), enabling access to relevant knowledge. Additionally, comprehensive feedback approaches, such as Self-Refine (Madaan et al., 2023), employ LLMs to evaluate their prior reasoning for self-improvement, while Reflexion (Shinn et al., 2023), STaR (Zelikman et al., 2022), and AlphaEvolve (Novikov et al., 2025) introduce reliable external environment feedback for consistent performance improvement. Different from
Generation (RAG)
15

---

Published as a conference paper at ICLR 2026
context augmentation, context compression methods aim at addressing the overlong context issue that challenges the finite context window size by dynamically reducing the contextual information. For example, INFTYTHINK (Yan et al., 2025) iteratively summarizes prior reasoning contexts for short-context reasoning, while CoThhink (Fan et al., 2025) employs short-response LLMs to orchestrate reasoning strategy and dynamically control context length. Additionally, O1-Pruner (Luo et al., 2025), LI (Aggarwal & Welleck, 2025), and DAST (Shen et al., 2025) consider the reasoning length during the training phase encourage LLMs address the given queries with fewer tokens.
B PROOFS
B.1 PROOF OF PROPOSITION 2.2
Proof. Since CoT-SC performs reasoning on the given query by voting the majority on set of independently generated multiple responses, given Assumption 2.1, the number of the correct responses conforms to a binomial distribution Neor B(Nse: P):
P(Nenr = =k) (Noo p0 p) Noc
where B denotes the binomial distribution. Thus, the case that CoT-SC correctly answers the question can formulated as the event: Neoo > Nis. Then, we further have the probability that CoT-SC correctly infers the answer to the query
Noc 2
Noc
P(Nenr >
P(Noo = k:)
k=
J+1
Nac p (1 p) Nac Noo
Am Ngn J+11
where [.] the flooring operator, and (Nxe = EI(NNe-K] is the binomial coefficient with Nac and
= 1) = p and P(X, = 0) = 1 Let Noor Enn X,, we then have E|X,] and E[Ncor] = NNcP. With the Hoeffding Inequality, we have
Consider set of independent random variables (X, Ne,, Ny, where X; E 10, P(X;=1) Nec
2t
P(Ncor - ElNoor] 2 S exp
Noc
If p 3. then E[Noor] Noe No. Lett Ns) 0:
Noc -2N -
P(Neor
S exp
2
Else, if 2 1. then E[Noor] 2 No. Thus, we consider the complementary case: P(Ncor E[Neor] S
Nxx). Then, let = (p- Noc > we can get:
Nor < exp
-2 1 -p) )
P(Ncor I
2
Thus, for the original case P(Nen > Nye) we have
P(Noor Nac exp
(1- ")
-2Nsc
The proof is completed.
16

---

Published as a conference paper at ICLR 2026
B.2 PROOF OF PROPOSITION 2.3
Proof. In a 2-round multi-agent debate case, where N, agents are involved in each debate round, we first consider the initial debate round, where the responses are only conditioned on the query prompt. Thus, the probability of MAD achieving Nood E {0, 2..., Na. correct reasoning responses is:
P(NC)) cor =) -0)
Na p² (1 p) where j E {0, 2,...
For the second round, assume that the probability LLM agents correctly inferring the answer is modified e~o,, where N, Na Noor, depending on the number of correct memories in the previous debate round. Note that new responses are generated independently in the second round among agents, although the generated answers are conditioned on memories, and the number of cor- rect responses in the second round also conforms to a binomial distribution N(2) ~ B(Na, Na)) 0. In this case, the probability that the answer is correctly inferred is:
N₂
No
P(N 0o0
P(N cor = 1/ N/ = P) =)
2
j=U
k=
J+1
N,
Nu
Nu
P(NO
=}
Σ
oor
j=0
k= No +1
where P(N(0) -) = (5)) p² (1 p) Na-1
Then, similar to Proposition 2.2, if ca() - Na)) 2 2 (i.e.. j 2 No - _ In 2). we have the lower bound:
N
N₂ Σ
Na
p0(1 - p) No-j 1 exp -2N, 2 co(i-M))
P(N(2) cor
2
[N
1n2
Otherwise, if (aa
No.) < 1/2 (i.e.. < No - In 2), we have:
(N.-112)
N₂.
NNa)
(No) Cakls Malls-Ma - cabs-Na)) No-k
Na
P(N())
p² (1 - Ns-
2
1=0
k= N +11
N
N₂
+
Na (G)
Dz (1 ((1-p) No
Na
Cakl)-Nal(1 )
k
j
N In2
+1
Since the case j 2 N₂- 1 2n is not considered here, we simply bound the probability P P(N >
( = =] by 1. Thus, we then have:
IN_ h=
(( cor
N₂
<
Σ
N
D² (1 - ND)
Naa k
- cali-
j=0
k= 1+1
N,
Na p² p(1 - (1-p) Na-
+
j=IN--
1u21
N
[N
ln2
NNa p² (1 - Na-eex exp (-2Na (1
[ - 2
P(N(2) Cor
<
2
j
1=
No
Na)
p0(1 - -p) Na-j.
i
J=
lna
The proof is completed.
17

---

Published as a conference paper at ICLR 2026
C DETAILED ANALYSIS ON TOKEN CONSUMPTION
Denote the token of the query x xtest as 74, the token of the output of the agent Ag, at the r-th debate round as Th Then, we formulate the token consumption of both MAD and MAD-M as follows.
For MAD, at the initial round, the query zesst is fed into Na LLM agents and then N₂ responses are generated from these LLM agents. Thus, the consumption of MAD at the initial debate round is:
N
N Mtoken
7
T
Then, from the second debate round, each agent will take all N, outputs in the last debate round and output new reasoning response with the query. Thus, the consumption of tokens for MAD at the r-th debate round is:
N
N
Ntoken = Na
T
T-
T
1,
Thus, the total consumption tokens of MAD can be formulated
Noonnd
MAD
Nroken
N
Nround No
Nround
Ng
NaNround T + Na
T
To.
i=1
For MAD-M2, at the initial round, the consumption of tokens the same as MAD):
Ntoken
N₂T
T
From the second round, the agents will first evaluate the previous memories and mask the potentially incorrect memories. In this step, memories are fed into each agent, and the agent will output "yes", "no", or "unsure". Here, we ignore the tokens of outputs and the instructions. We then formulate the consumption of tokens at this step as:
Neval taken
N
T T
Then, based the selected memories M 7-11 the input tokens for each agent in the r-th debate round should be no more than the sum of the query tokens and the tokens of all previous tokens. Thus, considering the output tokens, we have:
Ntoken S No
T9
To
1
To
Thus, the total consumption of MAD-M can be formulated as:
Nround
MAD-MA M Nrokenn
(Neoallcooon taken + Attken)
r=2
Neound
N₂ T
N,
S N
To
N
T
No
T
To
1
+
To
Nround N.
Nronne Ni
NN Noondd
7 + 2N
To 1,5
+
1=1
i=
18

---

Qwen2.5- Published as a conference paper at ICLR 2026
D COMPLETE EXPERIMENTAL SETTINGS
In this section, we provide more detailed experimental settings adopted in this paper to ensure that the results reported in this paper are reproducible. The content here refers to the details of adopted LLMs, evaluation benchmarks, hyperparameters, hardware, and reasoning/ddbatt/mmmskgg prompts.
D.1 DETAILS PROMPTS
In this section, we provide comprehensive details regarding the prompts adopted in the experiments of our proposed MAD-M framework. Specifically, we provide the prompts adopted in the CoT reasoning, the multi agent debate process, and the subjective evaluation operation on memories.
CoT Prompt
<QUESTION>
Please solve the problem step by step.
### Response format (MUST be strictly followed) (DO NOT include any other formats except for the given XML format): <think> THINKING HERE</think>
<answer> YOUR FINAL ANSWER ONLY, NO OTHER TEXT< 1/XT</answer>.
Debate Prompt
These are the potential solutions to the problem: <CONTEXT>
Use the potential solutions additional information for the following question. Question:<QUESTIONN
Please think step step and solve the problem.
### Response format (MUST be strictly followed) (DO NOT include any other formats except for the given XMI format):
<think>YOUR YOUR THINKING HERE</think>
<answer> YOUR FINAL ANSWER ONLY, NO OTHER TEXT</answer>.
Self-Evaluation Prompt
Evaluate the given solutions based on the question. * Your response MUST end with the following format: clabe)>YES<<labee> or <labe)>NO</labl>> or <label>NOT SURE</label>. ** Return YES if the solution is completely correct, NO any part of the solution is incorrect, and NOT SURE if you unsure.
Question: <QUESTION>
Solutions: <SOLUTION>
D.2 BASELINES
To validate the effectiveness of our proposed MAD-M², the following reasoning frameworks adopted as baselines in this paper: (1) Chain-of-Thouue (CoT) (Wei et al., 2022b); (2) Self- Consistency Sfain-off-hhuuhhe (CoT-SC) (Wang et al., 2023) with 6 independent reasoning paths; and (3) Multi-Agent Debate (MAD, 3-agent 2-round) (Du et al., 2023). As a default setting, an MAD framework is composed of LLM agents and 2 debate rounds if there are specific clarifications,
D.3 LARGE LANGUAGE MODELS
In this work, to validate the performance of our proposed MAD-M², we mainly consider evaluating baseline reasoning/mullii aagntt debate frameworks and our proposed MAD-M² with open-source large language models with different sizes. Specifically, we adopt Q Binht7B-Instruct (Yang et al., 2024a;b), DeepSeck-Mer Mattk---athBBBB.. B (Shao et al., 2024), and QwQ 32B (Team, 2025) in all our experiments and analyses.
19

---

Published as a conference paper at ICLR 2026
D.4 GENERAL TEST DATA SETTINGS
In experiments of this paper, we evaluate all reasoning frameworks with trials under five different random seeds (i.e., 41 415) and report the average of all five runs as the final results. In each run, for GSM8K, MATH, and MMIJU Pro, we randomly sample 100 samples from a shuffled sequence of the original benchmark due to the computational budgets. Such setting is consistent with that in the previous work (Du et al., 2023). In contrast, for AIME 24 & 25, we adopt all questions for evaluation.
D.5 HYPERPARAMETER SETTINGS
Hyperparameters of LLMs. In this paper, all reasoning responses are performed on LLMs with the temperature set to 1.0 except for Sven2.5-Math-275 - characters when performing reasoning with (wen2.5-Math-77B -n-sntruc with the temperature set to 1.0. Thus, to avoid failure in reasoning. the temperature for Qwen2.5 Srren.5-MMath-755 ---tth-B---nntruuc is set to 0.2. Moreover, the top_p set to 1.0 for all cases. The maximum length of the token set to 24064 for Qwen2.5-7B Inntttruut, 4096 for Qwen2.5-Mathe Boath-7B-Instruee and DeepSeek-Math-77 -I-strruct, and 131072 for QwQ-32B. Since the length of output tokens of QwQ-32B is usually too long, the thinking tokens in outputs will be truncated. Specifically, only the first 4096 tokens are preserved memories.
Instruct, since there are many unrecognized
Fairness of Comparison between CoT-SC and MAD. In the experiments of our paper, the default setting for MAD includes 3 agents and 2 debate rounds. Thus, the LLMs in MAD will consume the resources of 6 reasoning trajectory generations. Thus, for fairesss, in the case of CoT--C,, the final answer is achieved by performing majority voting among 6 responses generated in the way of CoT.
Computational Resources. In this paper, the main experiments and the analyses regarding Qwen2.5- IB-Instruct, Swen2.5-MMth-775 IB-strruct, and Saaippeeee-M-h---- -aat-rB--nsttruet Instruct are conducted with two RTX 3090 GPUs, while the experiments of QwQ-32B are conducted on several A100 and H20 GPUs.
D.6 DETAILS OF EVALUATION BENCHMARKS
In this paper, we adopt 5 mainstream benchmarks, ranging from mathematical reasoning to language understanding tasks, to evaluate all baselines and our proposed MAD-M2. The mathematical reasoning benchmarks include GSM8K, MATH, AIME 24, and AIME 25, while the language understanding benchmark is MMLU_Pro Details about these benchmarks are provided as follows.
AIME 24 & 25: AIME 24 & 25 are challenging competition mathematical question sets on the 2024 and 2025 American Invitational Mathematics Examinations. Each dataset contains 30 hard mathematical problems. These two benchmarks are famous for their difficulty in reasoning.
GSMSK (Cobbe et al., 2021): GSM8K (a.k.a. Grade School Math 8K) a popular mathematical reasoning dataset consisting of about 8500 high-quality, diverse grade school math word problems. In our experiments, we only adopt its test set that is composed of 1. questions for evaluation.
MATH (Hendrycks et al., 2021): MATH is a comprehensive mathematical dataset designed to rigorously challenge the mathematical reasoning abilities of LLMs. The questions in the MATH dataset refer to algebra, geometry, number theory, and combinatorics, thereby thoroughly evaluating the mathematical understanding and problem-solving capabilities of L.Mss. In the experiments of this paper, we only adopt its 5,000 test data evaluate the baselines and our proposed MAD-M
MMLU Pro (Wang al., 2024): MMLU-Pro is an advanced extension of the Massive Multitask Language Understanding (MMLU) dataset (Hendrycks et al., 2020), encompassing 57 diverse tasks across domains such as mathematics, computer science, biology, and physics. Compared to MMLU, MMLU-Pro introduces more complex, reasoning- intensive problems, which are more challenging.
-aasoning-intensvve
E COMPLETE EXPERIMENTAL RESULTS
In this section, we provide more supplementary experimental results of those in Section 4 and more analysis results regarding our proposed M validate its efficacy and explore its properties.
E COMPLETE ANALYSES ON SCALING THE NUMBER OF AGENTS AND DEBATE ROUNDS
In Section 4.3, we have conducted experiments to figure out the scaling capability of our proposed Instruct and DeepSeek-Matt 7B- Instrrccct.. Generally, according to the results in Section 4.3, MAD-M² consistently benefits from the scaling of the number of agents, while the performance fluctuates as the number of debate rounds increases. Moreover, another important observation here is that the subjective masking strategy performs
MAD-M2 equipped with different LLMs (i.e., Qwen2.5-78 -
20

---

Published as a conference paper at ICLR 2026
Accurac
89.0
264.8
81.0
Assuracy
31.0
10.00
13.3
Accurac
13.3
00.1
48.6
23.2
32.4
3.3
3.3
62.3
16.2
7.8g
8 1D
8
6
10
Num of Agents
Num of Agents
Num of Agents
Num of Agents
Num of Agents
(a) GSM8K
(b) MATH
(c) MMLU Pro
(d) AIME24
(e) AIME25
Figure 8: Effect of scaling the number of agents in the case of Qwen2.5-Math- -B-ntrrt-t The
number of agents is increased from 10. According the ligures, both frameworks act differently
when the number of agents increases and MAD-M²(O) achieves better performance in most cases.
Accuracy
78.9
40.3
38..7
15.0
1150
96.0
15.2
12.5
10.0
10.0
72.8
33.0
34.1
7:5
7.5
7 R 10
7
Num of Rounds
Num Rounds
Num of Rounds
Num of Rounds
Nu of Rounds
(a) GSM8K
(b) MATH
(c) MMLU Pro
(d) AIME24
(e) AIME25
Figure 9: Effect of scaling the number of debate rounds w.r.t. Qwen2.5-Math- Math-hB---ssrruettt
The number of debate rounds increases from 2 to 10. According to the figures, the performance of
different cases varies. most cases, MAD-M²(O) achieves better performance than (S).
Time (4)
HAD
MAD-HN (SI
LOD
MAD (C)
Time E5)
to.
15
MAD-a6
MAD-M (0)
Hime
5.0
0.0
GSMOK
MATH
NANUU AIME24 AIMEZ5
GSROK MATH
MILLU AIMEZ4 AINEZ5
GSMBK MATH MILLU AIMEZ4 AIMEZS
(a) Owen2.5-7B-Inna
(b)
(c) DeepSeek-Math-7B-nmntee
Figure 10: Analyses on time consumption of MAD and MAD-M² with different LLMs. Each figure compares the time consumption of MAD, (SD-MP(S), and MAD-M Generally, MAD- O²(O) consumes less time than MAD-M2(S) since it is free of an additional self-evaluation step.
better MAD-M² equipped with weak L.MSs while the objective masking strategy achieves better performance on MAD-M2 equipped with powerful models. Here, we further visualize the analyses of MAD-M² with different masking strategies on Dwen2.5-MMath-BB-mme
(ef. Figs. 8 and 9).
According to the visualization results, we can observe that the objective masking strategy consistently achieves better performance than the subjective masking strategy when scaling the number of agents and debate rounds. This phenomenon aligns with that in Section 4.3. However, different from the case of MAD-M² with Qwen2.5-7B- B--nsruttt and DeepSeek Math-7B-Insii M drops when scaling the number of agents. Compared to (SD-MM(S). MAD-M²(O) is more robust in the performance degradation. According to our observation on the behavior of the LLM, the reason for this phenomenon is the failure of reasoning due to the overlong output sequence.
the performance of
E.2 TIME CONSUMPTION ANALYSIS
Compared to the conventional MAD framework, MAD-M² introduces a self- relf-evaluation and masking phase remove the potential erroneous memories and further improve the performance of multi-agent debate. However, concern raised here is whether the efficiency of MAD is negatively influenced.
To figure out this concern, we compared the time consumption of MAD, MAD-M² and MAD- (O) on the five benchmarks with different LLMs. The quantitative results are visualized in Fig. 10. According to the results in figures, we can observe that MAD-M²(O) consumes comparable time to the baseline, MAD, while much less time than MAD-M* Such gap probably results from the additional self-evaluation operation in the subjective masking strategy. Meanwhile, compared to those simple reasoning tasks (c.g., GSM8K, MATH, and MMLU Pro), more time is consumed on harder reasoning tasks (c.g., AIME). Thus, for the case of MAD-M with powerful LLMs, our proposed MAD-M2(with the objective masking strategy) is effective and efficient in reasoning tasks.
E.3 INVESTIGATION ON STRICTNESS SUBJECTIVE MASKING STRATEGY
In our proposed MAD-M2, an important component the evaluation and masking. In particular, in the subjective masking strategy, the memories are evaluated by agents between rounds of
21

---

Published as a conference paper at ICLR 2026
Table 2: Analysis on strictness of MAD-M Average accuracy with standard deviation is reported. We evaluate three small LLMs with both mathematical reasoning and language understanding benchmarks. For fairness, all results are the average five trials on different seeds (i.e., 41-45).
Method
AIME24
AIME25
MMLU_PP
MATH
GSM8K
Qwen2
(SD-M*(S)
13.3
3.3
43.6.12.3
56.8
89.0.14.0
MAD-M
10.0 (-3.3)
10.0 (+6.7)
43.4+5.3 (-0.2)
54.8=2.8 (-2.0)
87.6#3.4 (-1.4)
Owen2. 5
(SD-M
6.7
6.7
35.0+2.2
71.2=3.3
95.2+1.8
MAD-M'
13.3 (+6.6)
6.7 (+0.0)
40.8+4.4 (45.8)
$14=4.4(+10.)
95.4 (+0.2)
DeepSeek -Bttr
MAD-M*(S)
0.0
0.0
30.8+5.2
37.0=5.1
80.8
MAD-M
0.0 (+0.0)
0.0 (40.0)
26.2+4.7 (4.6)
40.0=5.1 (+3.0)
82.6+2.9 (+1.8)
Table 3: Comparison between and Sparse MAD. Average accuracy with standard devia- tion is reported. We compare MAD-M MAD-M²(O), and Sparse MAD on Qwen2.5- 7----unsttt and Owen2255-Mthh77B-nnstre benchmarks. For fairness, all results are the average of five trials on different seeds (i.e., 41-45).
with both mathematical reasoning and language understanding
Method
AIME24
AIME25
MMLU Pro
MATH
GSM8K
Qwen2 7B rnstttueet
MAD-MP
16.7
13.3
44.6+5..7
60.8+
13.
MAD-M
20.0
0.0
42.244.2
58.8+-5.6
1..
Sparse MAD
13.3
6.7
44.243.4
59.24:2.7
94.2++1.9
Qwen2, 5 5-Math-- 7B Instruct
MAD-M'
16.7
13.3
66.2444.6
62.8+14.3
9.
MAD-M*(O)
13.3
16.7
67.417.0
63.813.0
95.8
Sparse MAD
10.0
10.0
38.6::2.3
76.4
9..
debates. In our MAD-M² framework, when applying the subjective masking strategy, the memories are evaluated with flags "YES", "NO", and "NOT SURE". Typically, the "YES" and "NO" flags are mapped into True and False, respectively. For the "NOT SURE" flag, according to the strictness, mapped into True or False. Thus, to figure out the effect the strictness, we compare the performance of MAD-M with and without the strictness. The results are reported in Table 2.
According to the results, we can observe that the performance of MAD-M varies among LLMs. Specifically, in the case of MAD-M (S) with weak LLMs (e.g.. Qwen2.5-777 1a-.2.57B--nntruuee Instruct), applying strict rules to evaluation deteriorates the performance. For example, the performance on MATH and GSM8K drops by 2.0% and 1.4%, respectively. However, in the case of MAD-M2(S) with powerful LLMs (e.g., Qwen2.5-Math- -B-ntrrt-nst the performance increases by 5.8%. 10.2%, and 0.2%, respectively, MMLU __ro, MATH, and GSM8K. In summary, MAD-M with powerful LLMs benefits from the strict rules, while MAD-M² fails to achieve better performance with strict rules.
22


Reviews:
[{"review_id": "zM3zOV1CFr", "reviewer": "Reviewer_u3Fc", "summary": "This paper addresses the robustness of multi-agent debate frameworks to wrong memories from previous debate rounds. The authors provide a theoretical analysis showing that MAD performance is closely related to the number of wrong memories, then propose MAD-M², which allows agents to evaluate and mask undesirable memories before reasoning in the next debate round. Experiments on MATH, MMLU-Pro, AIME24/25 show MAD-M² improves over standard MAD, particularly on easy tasks with weaker models (Qwen2.5-72B, Mixtral-8x7B), though gains on hard tasks and stronger models (DeepSeek-V3) are limited", "questions": "How often do agents correctly identify wrong memories? What's the precision/recall?\n\n\nFigure 2 shows one cherry-picked example. Could you also provide the failure cases?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper identifies a real issue in MAD: agents can be misled by wrong memories from previous rounds, as illustrated in Figure 1 where Agent 1 initially answers correctly but is misled in Round 2 after seeing Agent 2's incorrect response. This is intuitive and well-demonstrated.\n\nMAD-M² is straightforward to implement: agents evaluate memories, generate binary masks, and reason with filtered memories. The method doesn't require additional models or complex infrastructure, making it practical for real deployment.\n\n\nThe theoretical analysis in Section 2.2 provides useful insights. Propositions 2.2 and 2.3 formalize the performance bounds for CoT-SC and MAD, showing how wrong memories (Ne) exponentially degrade performance through the term e^(-αNe). The distinction between easy problem reasoning (EPR) and hard problem reasoning (HPR) settings offers guidance for when memory masking helps.", "weaknesses": "MAD-M² shows improvement on MATH (0.90→0.92, +2%) and AIME24 (0.37→0.50), but degrades or maintains performance on MMLU-Pro (0.83→0.83) and AIME25 (0.40→0.40). Meanwhile, MAD-M² shows improvement on MATH (0.90→0.92, +2%) and AIME24 (0.37→0.50), but degrades or maintains performance on MMLU-Pro (0.83→0.83) and AIME25 (0.40→0.40)\n\n\nToken consumption analysis in Table 1 shows MAD-M² consumes 50-100% more tokens than standard MAD. The cost-benefit ratio is poor, especially when standard MAD already improves factuality and reasoning significantly over single-agent methods\n\nFor assumption 2.1, the claim that the probability of correct answers given memories follows e^(-αNe) is asserted without justification. Why exponential decay? Why not linear, polynomial, or other forms?\n\nHoeffding requires independent samples, but debate rounds are sequential and dependent - each round conditions on previous memories. The analysis doesn't account for this dependency.\n\n\nFigures 3-4 show that scaling agents help on easy tasks only (consistent with theory), but MAD-M² doesn't consistently outperform MAD even there", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UGIUZPhYWf", "reviewer": "Reviewer_7GkP", "summary": "This paper investigates the robustness issue in the multi-agent debate (MAD) framework for large language model (LLM) reasoning. The authors theoretically demonstrate that the performance of MAD degrades when agents rely on “wrong memories” from prior debate rounds, since each round’s reasoning depends on previous outputs. To mitigate this, they propose MAD-M2 (Multi-Agent Debate with Memory Masking), which allows each agent to evaluate and selectively mask unreliable memories before generating new responses. The method is conceptually simple yet well-motivated. Experiments across four reasoning benchmarks and diverse models show consistent or improved results compared with vanilla MAD.", "questions": "1. How sensitive is MAD-M2 to the rule for generating the binary mask?\n\n2. Did the authors explore partial masking or soft weighting? Could such approaches improve stability?\n\n3. Can the authors provide more comprehensive experiments on more LLM backbones and compare more baseline MAD approaches?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. Clear motivation grounded in a theoretical gap: The authors identify an underexplored limitation in multi-agent debate frameworks — vulnerability to low-quality reasoning memories — and back it up with intuitive examples (Figure 1) and formal analysis.\n\n2. The analysis in Section 2 provides explicit probabilistic bounds linking performance degradation to the number of wrong memories. The inclusion of both hard and easy reasoning settings (HPR/EPR) offers a nuanced interpretation of how memory errors affect debate robustness.", "weaknesses": "1. My biggest concerns are about approach performance. I only see the marginal performance gains in stronger models. While MAD-M2 improves certain benchmarks on easy tasks, the improvements for DeepSeek-V3 are minor or inconsistent. This suggests limited scalability to highly capable LLMs. Could the authors provide more results on diverse and powerful LLM benchmarks?\n\n2. The proposed masking step roughly doubles token consumption in some cases, but the paper provides limited quantitative analysis of efficiency–performance trade-offs. \n\n3. The masking mechanism is described at a high level but not deeply analyzed. For instance, how different scoring heuristics or thresholds affect the results remains unclear. The authors are suggested to add more fine-grained ablations (e.g., varying mask strictness).\n\n4. Although the paper compares conceptually to S-MAD and S2-MAD, it doesn’t empirically benchmark against them.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bsfJH9UskG", "reviewer": "Reviewer_gCX4", "summary": "This paper investigates a fundamental limitation of the Multi-Agent Debate (MAD) framework, demonstrating that LLM agents exhibit vulnerability to erroneous memories from preceding debate rounds, which can propagate incorrect reasoning trajectories. Through rigorous theoretical analysis, the authors establish that MAD performance is intrinsically bounded by e^(-αNₑ), where Nₑ denotes the cardinality of erroneous memories and α characterizes the agent's robustness coefficient. To address this vulnerability, they propose MAD-M² (Multi-Agent Debate with Memory Masking), a framework that incorporates a memory evaluation and masking mechanism, enabling agents to selectively filter low-quality contextual information prior to each reasoning iteration.", "questions": "Refer to the weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. This paper is the first to demonstrate that Multi-Agent Debate (MAD) frameworks are vulnerable to false memories, providing theoretical proof of this phenomenon.\n2. This study offers a thorough analysis of the key characteristics of the proposed MAD-M² framework in relation to existing methods.", "weaknesses": "1. The primary contribution of this paper lies in its theoretical insights rather than methodological innovations.\n\n2. The proposed MAD-M² framework achieves state-of-the-art performance in only a limited subset of experimental scenarios, while underperforming traditional MAD or CoT-SC methods in the majority of cases. Moreover, MAD-M² incurs significantly higher token consumption compared to baseline methods. From a performance-efficiency trade-off perspective, MAD-M² does not demonstrate clear contributions. If additional metrics can substantiate the contributions of MAD-M², please include them in Table 1.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)