Multi-LLM Debate: Framework, Principals, and Interventions

Authors: andrew  estornell
yang  liu
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 00:32:12

Abstract: The flexible and generalized nature of large language models has allowed for their application in a wide array of language-based domains.
Much like their human contemporaries, these models are capable of engaging in discussions and debates as a means of improving answer quality.
We first take a theoretical approach to analyzing debate and provide a framework through which debate can be mathematically examined.
Building on this framework, we provide several theoretical results for multi-agent debate.
In particular, we demonstrate that similar model capabilities, or similar model responses, can result in static debate dynamics where the debate procedure simply converges to the majority opinion. 
When this majority opinion is the result of a common misconception (ingrained in the models through shared training data) debate is likely to converge to answers associated with that common misconception.
Using insights from our theoretical results we then propose three interventions which improve the efficacy of debate. 
For each intervention, we provide theoretical results demonstrating how debate is improved.
We also demonstrate that these interventions result in better performance on four common benchmark tasks.

Main paper: 
 introduction: 
 Large language models (LLMs) have demonstrated a remarkable ability to perform unseen tasks with high efficacy. This behavior, often referred to as emergent, allows LLMs to serve as general-purpose tools for a wide array of language-based functions. One such behavior of particular interest is the ability of LLMs to intake and process opinions from other models (or humans). As shown in several previous works, this ability allows LLMs to collaboratively solve tasks by engaging in debate Chan et al. [2023] , Liang et al. [2023] , Du et al. [2023] . For a given task, multi-agent debate operates by eliciting responses from each model, distributing those responses among the models, and then eliciting updated responses from each model.In this work, we aim to explore the debate procedure, by first providing a theoretical framework through which debate can be better understood. This framework draws inspiration from Bayesian inference and in-context learning, showing that debate can be partially viewed as a special type of in-context learning. Through this framework, we then provide several theoretical insights into the debate procedure. In particular, we demonstrate the susceptibility of multi-agent debate to echochamber effects. These echo-chamber effects are especially consequential when they stem from a shared misconception between a majority of models, which can arise from circumstances such as highly correlated training data between each model.We then leverage results from our theoretical framework to improve the efficacy of the debate procedure. In particular, we propose three interventions (modifications to the debate procedure). First, diversity-pruning which aims to maximize the information entropy in model responses at each round of debate; this intervention has the added benefit of reducing the severity of the echo chamber effect. Second, quality-pruning which aims to maximize the relevance of each model's response. We demonstrate that this intervention improves the likelihood that the debate procedure converges to correct answers. Lastly, misconception-refutation which directly identifies, and attempts to refute misconceptions in model responses. This intervention takes inspiration from works such as Robinson et al. [2022] which demonstrate that LLMs are often more skilled at evaluating answers, compared with directly providing answers. For each of our interventions, we provide theoretical results outlining the way in which each improves debate. We also conduct experiments on four common benchmarks demonstrating that these interventions improve debate efficacy in practice.Our contributions 1) We propose a theoretical framework for multi-LLM debate that draws on connections from in-context learning and Bayesian inference. 2) We provide theoretical insights on several key principles of multi-LLM debate. 3) Using these insights, we design three debate interventions which result in consistent improvement to debate, across four language benchmarks (BoolQ, MMLU, MathQ, TruthfuQA) and three families of models (GPT, Lama, and Mistral). 
related work: 
 Our work is closely related to multi-agent debate, which focuses on iterative collaboration between agents to make a decision Chan et al. [2023] , Liang et al. [2023] , Du et al. [2023] , Khan et al. [2024] , Irving et al. [2018] , Michael et al. [2023] , Rasal [2024] , Pham et al. [2023] , Chang [2024b] . These works often focus on multi-agent debate in the context of question-answering tasks and aim to provide higher quality answers (compared to those of a single model) by engaging multiple models in discussion. The preliminary debate framework, proposed by Du et al. [2023] , facilitates debate by first asking each model a question, and then iteratively re-asking agents that same question contextualized by the responses of all models in the previous round. Different variants of this procedure have been proposed: debate where models have different functionality Liang et al. [2023] , round-robin style debate Chan et al. [2023] , dynamically controlling the level of disagreement between agents in debate Chang [2024a] , or using judges to assess the correctness of debaters Khan et al. [2024] . Other techniques for iteratively improving the quality of answers have also been proposed, e.g., chain-of-thought Wei et al. [2022] , Kojima et al. [2022] , self-consistency Wang et al. [2022] , Singhal et al. [2023] , and self-reflection Ren et al. [2023] . Similar to debate, there have been investigations into the use of different LLMs to engage with one-another Liu et al. [2023] , Abdelnabi et al. [2023] , Zhang et al. [2023] , Li et al. [2023c] , Park et al. [2023a] , explain their rational to others Wang et al. [2023a] , or collaboratively engage in general tasks Li et al. [2023a] , Wang et al. [2023b] , Park et al. [2023b] , Wu et al. [2023] , Hong et al. [2023] , Li et al. [2023d,b] , Tsao and AILAB [2023] . While debate has shown promise in a wide range of domains, several works have also demonstrated that the debate process can be unstable and can lead to worse performance than using just a single model Wang et al. [2024] , Smit et al. [2023] .Our work is also related to in-context learning and Bayesian inference. The former, Brown et al. [2020] , Min et al. [2022 ], Lampinen et al. [2022] demonstrates that LLMs can perform unseen tasks when provided only a few examples of that task. Other works Xie et al. [2021] , Jiang [2023] have shown a connection between in-context learning and Bayesian inference; the additional examples provided to the model can be viewed as updates to the model's posterior distribution over tokens. 
preliminaries: 
 Debate Let x be a given question, with associated answer y, for example x ="What color is the sky?" and y ="Blue". Following the debate procedure proposed by Du et al. [2023] , a collection of n LLMs (also referred to as agents) collaborate to infer the correct answer y by iteratively engaging in discussion over T rounds, as described next:• At round t = 0 each agent i observes task x, then provides response z (0) i .• At rounds t > 0 each agent i observes task x and the outputs of the n agents at the previous timesetep Z (t-1) = (z(t-1) 1 , . . . , z (t-1) n), then provides response z (t)i . • The debate process ends if t = T or if agents reach a consensus.To measure if consensus is reached a function a extracts an answer1 from a given response z. Suppose z ="During the day, the sky is blue", then a(z) = "Blue". At round t, the probability that agent i provides response z (t+1) i, is given byP model z (t+1) i updated model response x task , Z (t) = (z (t) 1 , . . . , z (t) n )all model responses from previous round, ϕ i model parameters (1)where ϕ i captures model hyperparameters (such as training data, architecture, etc.). Both the input (Z (t) , x) as well as the hyperparameters ϕ i , ultimately influence the output z (t+1) i. Note that on each round, all agents observe the same input, namely (Z (t) , x). Thus, differences in agent output z (t+1) i are determined by both the stochastic nature of output generations, and the unique parameters ϕ i of each model. For notational convenience, we drop the subscript in P model when the parameters ϕ i are given, and simply write P(•|ϕ i ).The key distinction between our approach and "vanilla" debate, is that we will leverage latent concepts (discussed next) to modify the responses in Z (t) in between each round of debate.Latent Concepts Central to our investigation is the idea of latent concepts in language generation. As outlined by Xie et al. [2021] , Jiang [2023] latent concepts capture the idea that language is not generated at random (either by humans or by models). Rather, when generating language, we first have an idea or an intention form in our minds; we then select words that we believe will convey that underlying idea or intention. More formally, let Θ be a latent concept space and let θ ∈ Θ be a concept. Following Xie et al. [2021] , tasks x, and their associated answer y are generated by first selecting a vector of latent concept θ ∈ Θ and then sampling (x, y) ∼ D(θ), where D is some distribution mapping concepts to task-answer pairs. Similarly, when providing responses, models will observe x, and infer the latent concept θ, or more generally a distribution over the latent concept space, and then generate a response according to those inferred concepts, i.e., the model's generation probability in Equation 1 can be expressed asEQUATIONNote that the above holds by the law of total probability for any choice of latent concept space.To provide a more concrete example of latent concepts, consider the question-answering task in the BoolQ dataset. One of the questions in this dataset is "Did Abraham Lincoln write the letter in the film Saving Private Ryan?" to which the correct answer is "Yes". The latent concept, in this case, corresponds to the actual scene in the movie where the Bixby letter (written by Lincoln) is read to a group of soldiers. Just as in our case, first a concept θ is drawn, e.g., the film is made; then from the film, a string x is sampled, i.e., the previous question about the film.For another example of latent concepts, we can think of arithmetic calculations such as multiplication.When we wish to express multiplication through language, we may write something like "4 * 4". The latent concepts behind this string are the mechanisms of multiplication (e.g., multiplication is just iterative addition, and addition itself is simply increasing the value of a number by one iteratively). These examples are intended to be easily digestible. However, latent concepts can also be significantly more abstract, such as a vector in some unknown embedding space. 
a theoretical formulation of multi-agent debate: 
 We begin by providing a theoretical formulation of multi-agent debate. This formulation will provide key insights into the inner workings of the debate procedure, which we will use to improve debate.The key behind our framework is to use the idea of latent concepts and expansion of each model's generation probability (Equation 2) in order to better understand debate. Prior to presenting our theoretical framework, we first state an important assumption.Assumption 4.1. For a given latent concept space Θ, the probability of generating response z (t+1) i is conditionally independent of both the responses Z (t) and the task x, given concept θ ∈ Θ and model parameters ϕ i , i.e., P z(t+1) i | θ, x, Z (t) , ϕ i = P z (t+1) iAssumption 4.1 can be interpreted as saying that a model's generation z i , is uniquely determined by the model's parameters ϕ i and the concepts identified by a model, namely θ. In the case of encoder-decoder-based models, one can conceptualize the joint between ϕ and θ as corresponding to the embedding produced by the encoder. With this embedding in hand, the original input (x, Z (t) ) no longer influences the model's output, rather the embedding and model parameters will uniquely determine the model's output.Next, we derive the following lemma which will be useful in examining the way that model responses evolve debate rounds. Lemma 4.2. The generation of model i at time t + 1 can be expressed as, P z (t+1) i | Z (t) , x, ϕ i ∝ θ∈Θ P z (t+1) i |θ, ϕ i Pj=1 P z (t) j | θ, ϕ i .To help conceptualize the role of latent concepts in debate, we discuss the work of Xie et al. [2021] , which uses Bayesian inference over latent concepts to understand in-context learning. There are natural connections between in-context learning and multi-agent debate. In-context learning works as follows: given a task x and a model f , select several examples of taskanswers pairs (x 1 , y 1 ) . . . (x m , y m ) which are similar to x. Then prompt the model f for an answer to task x, given examples (x j , y j ), i.e. z = f x| (x 1 , y 1 ) . . . (x m , y m ) . A key result of Xie et al. [2021] is that latent concepts in the examples (x j , y j ), particularly concepts shared between many examples, influence the model's answer z. For any concept where P θ| (x 1 , y 1 ) . . . (x m , y m ) is large relative to other concepts (i.e., there is a shared concept θ between the examples), the model is more likely to give response z which also share that concept. Model responses at the previous round Z (t) serve a similar function to the examples of in-context learning. The model's updated response at time t + 1, namely z (t) i , is influenced by concepts shared between the responses in Z (t) . The skew term in Lemma 4.2 provides a glimpse of how latent concepts conveyed by Z (t) will influence the generation of z Through this perspective of debate we can more effectively design debate procedures by leveraging the concept space Θ. To do this, we will first formulate debate as an optimization problem where the skew term, described in Lemma 4.2, corresponds to the optimization variables. For a given task x and answer y, each round of debate can be formulated as the following optimization problem. arg maxZ (t) n i=0 P a(z (t+1) i ) = y| Z (t) , ϕ iAt time t we aim to craft responses Z (t) such that they maximize the probability of providing the correct answer at the next time step. Expanding this objective over the latent concept space Θ, yields arg maxEQUATIONThe key challenges with directly optimizing this objective are: 1) the true concept θ * from which x and y originate, as well as the relationship between z (t) j and the underlying concepts, is unknown, 2) the responses in Z (t) ) are natural language. However, will allow us to design several approaches within the concept space to better optimize debate. To motivate these approaches, we first need to make several observations about the debate procedure as a whole. 
debate principals: 
 In this section, we discuss factors that affect the efficacy of LLMs debate. In particular, we look at the role of information diversity, both in terms of the diversity of responses in Z (t) as well as diversity in model capabilities. We see that a lack of diversity in either aspect is detrimental to the debate process. Lastly, we study a particular type of homogeneity in debate, namely shared misconceptions in which a large portion of models all share a similar erroneous belief about the task x.We begin by examining how the debate procedure is affected by both the diversity of model abilities and the diversity of model responses. Homogeneity in either ability or responses will bias the debate procedure towards certain latent concepts.Similar Model Capabilities Suppose the debate process is conducted with only one type of model (in effect n copies of the same model). That is, ϕ i ≡ ϕ for all i ∈ [n]. Then, as the number of agents increases, the debate procedure is more greatly impacted by the echo chamber effect, i.e., the probability that a round of debate results in a change to the most likely concept, perceived by agents, approaches 0. That is, a greater number of similar agents results in static debate dynamics, in essence defeating the purpose of debate. Theorem 5.1. Suppose all n agents have identical configurations (ϕ i ≡ ϕ for all i). For round t > 0 let θ ′(t) = arg max θ∈Θ P θ| x, Z (t) , ϕ and θ ′(t+1) = arg max θ∈Θ P θ| x, Z (t+1) , ϕ , i.e., θ ′(t) and θ ′(t+1) are the concepts most likely to be inferred by a model with parameters ϕ when given task x and responses Z (t) , Z (t+1) respectively. Then P θ′ (t) = θ ′(t+1)) → 1 as n → ∞.We defer a full proof to the Supplement, Section A. Theorem 5.1 implies that when debate is conducted with multiple copies of the same model (or very similar models), increasing the number of models results in debate centering on a single (unchanging) concept, rather than a balanced distribution over multiple concepts.Similar Model Opinions Next, we examine how similar responses impact the collaboration process. At time t suppose that there are n responses Z (t) and at least m of those responses are similar, i.e., there exists some concept θ ′ such that θ ′ = arg max θ∈Θ P θ| z (t) j , ϕ i ) for all j ≤ m. That is, each of the m responses has a shared "most likely" concept when viewed by a model with parameters ϕ i . Theorem 5.2. Suppose that Z (t) contains at least m responses with the property that θ ′ = arg max θ∈Θ P z(t) i | θ, ϕ i . Then, as m → ∞ the model's generation at the next round (t + 1) becomes uniquely determined by a single concept θ ′ i.e. P z (t+1) (i,1) | Z (t) ,x,ϕ i P z (t+1) (i,2) | Z (t) ,x,ϕ i → P z (t+1) (i,1) | θ ′ ,ϕ i ) P z (t+1) (i,2) | θ ′ ,ϕ i ) for all response pairs z (t+1) (i,1) , z (t+1) (i,2) .We defer a full proof to the Supplement Section A. Theorem 5.2 indicates the susceptibility that LLM debate has towards tyranny of the majority. If a large number of models provide similar responses to a task x, then those repeated answers will drown out the single provided by the other models' responses, as well as the task x itself. In Section 7 we demonstrate that this occurs in practice.Next, we study a particular type of homogeneity in model capabilities and responses, namely shared misconceptions. When a common misconception is shared among the models, debate is less effective and is likely to converge to erroneous concepts associated with the shared misconception. We now formalize the notion of misconceptions. Definition 5.3. (Misconception): For a given concept θ * , a model with parameters i is said to have a misconception regarding θ * if there exists another concept θ ′ s.t.,P x∼D(θ * ) P(x| θ ′ , ϕ i ) > P(x| θ * , ϕ i ) > 1 /2That is, for tasks generated from the concept θ * , the model believes that the erroneous concept θ ′ explains more than half of the tasks better than the true concept θ * .There is said to be a shared misconception if m of agents have a misconception and share the same erroneous concept θ ′ . When the models share a common misconception the responses produced by those models are biased towards the erroneous concept θ ′ . Theorem 5.4. Let the true concept be θ * and suppose that m of the n agents have a shared misconception for erroneous concept θ ′ . Then, task and answer (x, y) ∼ D(θ * ) expected average correctness of the debate procedure at the final round T is monotonically decreasing with m, i.e.,1 n n i P a(z (T ) i ) = y is decreasing with m.We defer the full proof to the supplement Section A. It should be noted that the phenomenon of converging to erroneous concepts is unlikely to be mitigated by adding more models. When the misconceptions of one model are formed through training data, it is likely that other models will possess the same misconception unless specifically trained to avoid such errors due to the high correlation in training data between models. 
interventions: 
 In this section, we discuss several modifications to the debate procedure, referred to as interventions. We break our interventions into two categories: Pruning which focuses on choosing which responses to keep in Z (t) , and Modifying which focuses on changing or editing the responses Z (t) .At round t of debate, running interventions work by selecting only a subset of responses Z ′(t) from Z (t) before starting the next round t + 1. When using a pruning intervention, the models at round t + 1 will see only the pruned response set Z ′(t) , rather than the full response set Z (t) .Diversity Pruning Let KL represent Kullback-Leibler divergence. The diversity pruning intervention selects k of the n responses in Z (t) which maximizes information entropy, i.e.,Z ′(t) = argmax Y ⊂Y (t) zi,zj ∈Z KL D(θ| z i ), D(θ| z j ) s.t. |Z| = kQuality Pruning Quality pruning aims to select the k responses in Z (t) with the highest similarity to the task x. Similar to diversity pruning, quality pruning selects k of the n responses at time t.Rather than selecting for diversity, quality pruning aims to select the k highest question responses. This is done by selecting the k responses which maximizeZ ′(t) = argmin Z⊂Z (t) zi,∈Z KL D(θ| x), D(θ| z i ) s.t. |Z| = kIn practice computing KL D(θ|x), D(θ|z i ) or KL D(θ|z i ), D(θ|z j ) is intractable. However, sentence embedding can be used as a proxy for these values. Section C discusses this in further detail.Next, we show that when models have a shared misconception, diversity pruning decreases the likelihood that the debate procedure will converge to the erroneous concept corresponding to the shared misconception. Theorem 6.1. Let the true concept be θ * and suppose that at least n/2 agents have a shared misconception for erroneous concept θ ′ . Then diversity pruning decreases the probability that debate converges to an answer y ′ which is sourced from the erroneous concept θ ′ , i.e. y ′ ∼ D(θ ′ ).We defer the full proof to the Supplement, Section A.Theorem 6.2. For a given task-answer pair (x, y) quality pruning increases the probability that debate converges to the correct answer, i.e. let Z (t) be the set of all responses at time t and Z ′(t) be the result of quality pruning, thenn i=1 P(a(z (t+1) i ) = y|x, Z ′(t) , ϕ i ) > n i=1 P(a(z (t+1) i = y|x, Z (t) , ϕ i ).We defer the full proof to the Supplement, Section A.Remark 6.3. As shown by Theorems 6.1 and 6.2, diversity pruning decreases the probability that debate converges to incorrect answers sourced from a particular concept, while quality pruning increases the probability that debate converges to a correct answer sourced from the true concept.Both interventions can be used simultaneously to guide the debate procedure more effectively away from wrong answers and towards correct answers.Misconception Refutation In addition to selecting which responses in Z (t) will be used in the next round of debate, we can also modify the responses in Z (t) . Misconception refutation aims to do precisely this by updating response z j . The model is first prompted for a list of misconceptions and errors identified in the response. Given the list of misconceptions, the model is asked for both a refutation of the misconception and a corrected version of the response. For more details, see Section C of the Supplement. Theorem 6.4. For task-answer pair (x, y), misconception refutation increases the probability of debate converging to the correct answer, i.e. let Z (t) , Z * (t) be the responses before and after misconception refutation, then n i=1 P(a(z(t+1) i ) = y|x, Z * (t) , ϕ i ) > n i=1 P(a(z (t+1) i = y|x, Z (t) , ϕ i ). 
experiments: 
 Experimental Design We conduct experiments on four common language model benchmarks. BoolQ Clark et al. [2019] , which consists of 3, 270 yes-no questions, MMLU Hendrycks et al. [2020] which consists of 13, 869 multiple-choice questions (we use the 3, 406 high-school-level questions), TruthfulQA Lin et al. [2021] We begin by making several empirical observations about the multi-agent debate process.Tyranny of the Majority First, we examine the susceptibility of models towards agreement with the majority opinion. That is, how likely are models to give a specific answer at round t + 1 when m of the models provided that specific answer at round the previous round (round t)? For example, in BoolQ suppose the specific answer is "Yes", then we want to know: how likely is a model to give a "Yes"-answer at round t + 1 if that model observes m "Yes" answers at round t. To measure this, we first select a random target answer, e.g., "Yes", and then prompt m of the models (out of 11) to provide responses "Yes"-answers2 (while the other 11-m models are prompted to provide a different randomly selected answer). These 11 responses make up Z (t) , we then test each model's likelihood of providing the target at round t + 1 when observing Z (t) before diversity pruning (solid) and after diversity running (hatched).In Figure 1 , we see models are susceptible to echo chamber effects (this phenomenon is predicted by Theorem 5.1). The likelihood of providing the majority answer increases when Z (t) contains more instances of the majority answer (i.e., as m increases). Figure 1 also demonstrates that diversity pruning (with k = 5) reduces this echo chamber effect. See the Supplement for details.Diversity of Opinions Next, we examine the effectiveness of SoM and our method as a function of opinion diversity. Figure 2 shows the average accuracy improvement of SoM (dashed) and our method (solid) over single model performance (i.e., average performance at round t = 0), as a function of the similarity between all responses at round t = 0 of debate (measured via pairwise cosine similarity). We see that for BoolQ, MMLU, and TruthfulQA, SoM is less effective when the similarity between responses increases. This observation is predicted by Theorems 5.1 and 5.2, which show that debate, without interventions, is less effective when model responses are too similar.We see that our method's improvement compared to SoM is greatest when model opinions are more similar (cosine similarity close to 1). Note that the MathQ benchmark, where responses consist primarily of arithmetic, serves as a counter-example to these observations. This is due to the fact that sentence embedding of any two arithmetic expansions will be similar, regardless of their true similarity; as such, the cosine similarity between embedding is less meaningful on this benchmark.Debate Interventions Now, we examine the effectiveness of a combination of our three debate interventions (see Algorithm 1 for full details of how the interventions are combined). We begin with a per-round performance of our method and SoM, as shown in Figure 3 . We see that typically, the advantage of our method over debate arises in the later rounds of debate. Next, in Table 1 , we present a full set of results for single models, SoM, and a combination of our three interventions. In all cases, our method is either competitive with, or superior to, SoM. Figure 3 : Accuracy per round, our method and SoM when combing GPT-3.5 with Llama-3 or Mistral.In addition to providing results for the combination of our interventions, we also investigate the effectiveness of each intervention applied individually (see Table 3 of the Supplement). These results indicate that our method is most successful when applying all three interventions simultaneously. In fact, some interventions can be detrimental to the debate process when applied in isolation. This is expected as each intervention is inherently designed to be complementary. 
limitations: 
 While we aim to address some of the fundamental issues of multi-LLM debate, such as tyranny of the majority, there are several factors that need to be considered when adopting our framework. Firstly, our theoretical results leverage a latent concept space, which may not be accessible in practice, necessitating the use of proxies such as sentence embeddings. Reliance on proxies is particularly consequential for quality and diversity pruning; these interventions are less effective in domains where sentence embeddings are less meaningful, e.g., arithmetic questions. Additionally, our interventions can increase the inference time of the debate procedure. Increased inference time stems primarily from misconception refutation, as this intervention requires re-prompting each debater multiple times. 
conclusion: 
 Multi-agent debate is an effective tool for improving the efficacy of LLM responses. However, debate is naturally susceptible to issues such as tyranny of the majority and shared misconceptions between models. By making use of our theoretical framework for debate, we are able to establish interventions for the debate procedure which help to alleviate these issues and improve the general performance of multi-agent debate. We saw that diversity pruning reduces the influence of similar responses. This is especially helpful in settings where the majority of agents provide incorrect responses that share a common error. A combination of all three interventions consistently leads to better debate.Let θ ′ be the concept which model i, with configuration ϕ i , believes is most likely to have produced response z ′ , i.e., θ ′ = arg maxθ P z ′ | θ, ϕ iThen the above ratio can be rewritten asθ∈Θ P z (t+1) (i,1) | θ, ϕ i )P x| θ, ϕ i P(θ| ϕ i ) k j=1 P z (t) j | θ, ϕ i P z ′(t) | θ, ϕ i m θ∈Θ P z (t+1) (i,2) | θ, ϕ i )P x| θ, ϕ i P(θ| ϕ i ) k j=1 P z (t) j | θ, ϕ i P z ′(t) | θ, ϕ i m 1 P z ′(t) | θ ′ ,ϕ i m 1 P z ′(t) | θ ′ ,ϕ i m = θ∈Θ:θ̸ =θ ′ P z (t+1) (i,1) | θ, ϕ i )P x| θ, ϕ i P(θ| ϕ i ) k j=1 P z (t) j | θ, ϕ i P z ′(t) | θ,ϕ i P z ′(t) | θ ′ ,ϕ i m θ∈Θ:θ̸ =θ ′ P z (t+1) (i,2) | θ, ϕ i )P x| θ, ϕ i P(θ| ϕ i ) k j=1 P z (t) j | θ, ϕ i P z ′(t) | θ,ϕ i P z ′(t) | θ ′ ,ϕ i m • +P z (t+1) (i,2) | θ ′ , ϕ i )P x| θ ′ , ϕ i P(θ ′ | ϕ i ) k j=1 P z (t) j | θ ′ , ϕ i +P z (t+1) (i,2) | θ ′ , ϕ i )P x| θ ′ , ϕ i P(θ ′ | ϕ i ) k j=1 P z (t) j | θ ′ , ϕ iFor each θ ̸ = θ ′ we haveP z ′ | θ,ϕ i P z ′ | θ ′ ,ϕ i < 1. Therefore lim m→∞ P z ′ | θ, ϕ i P z ′ | θ ′ , ϕ i m = 0Note that the only summands which do not have an exponential dependency on m are those associated with concept θ ′ . Therefore, when examining the limit of the above ratio with respect to the number of repeated signals m, we getlim m→∞ θ∈Θ P z (t+1) (i,1) | θ, ϕ i )P x| θ, ϕ i P(θ| ϕ i ) k j=1 P z (t) j | θ, ϕ i P z ′(t) | θ, ϕ i m θ∈Θ P z (t+1) (i,2) | θ, ϕ i )P x| θ, ϕ i P(θ| ϕ i ) k j=1 P z (t) j | θ, ϕ i P z ′(t) | θ, ϕ i m = P z (t+1) (i,1) | θ ′ , ϕ i )P x| θ, ϕ i P(θ ′ | ϕ i ) P z (t+1) (i,2) | θ ′ , ϕ i )P x| θ, ϕ i P(θ ′ | ϕ i ) = P z (t+1) (i,1) | θ ′ , ϕ i ) P z (t+1) (i,2) | θ ′ , ϕ i )Thus the relationship between any two conditional generation probabilities can be uniquely defined by θ ′ .Proof of Theorem 5.4. First, we examine the models' outputs at the first rounds of debate. For a model i which possess the shared misconception, their conditional generation probability on the first round of debate can be expressed as,P z (t+1) i |x, ϕ i ∝ θ P z (t+1) i | θ, ϕ i P x| θ, ϕ iNext, consider the term P x| θ, ϕ i . Let ϕ ′ i be a set of model parameters which does posses the common misconception. Then,P x| θ, ϕ i = P x, x ′ | θ, ϕ ′ iwhere x ′ is a message which conveys the erroneous concept θ ′ . Using this change of model parameters, we can express the condition generation probability as,P z (t+1) i |x, ϕ i ∝ θ P z (t+1) i | θ, ϕ i P x, x ′ | θ, ϕ ′ i = θ P z (t+1) i | θ, ϕ i P x| θ, ϕ ′ i P x ′ | θ, ϕ ′ iWith this formulation of the conditional generation probability, we can the ratio between the true concept θ * and the erroneous concept θ ′ .P z (t+1) i | θ * , ϕ i P x| θ * , ϕ ′ i P x ′ | θ * , ϕ ′ i P z (t+1) i | θ ′ , ϕ i P x| θ ′ , ϕ ′ i P x ′ | θ ′ , ϕ ′ i < P z (t+1) i | θ * , ϕ i P x| θ * , ϕ ′ i P z (t+1) i | θ ′ , ϕ i P x| θ ′ , ϕ ′ i (4) which follows directly from P x ′ | θ ′ , ϕ ′ i > P x ′ | θ * , ϕ ′ i .When models have a shared misconception (left side of Equation 4),P z (t+1) i |x, ϕ i < P z (t+1) i |x, ϕ ′ i for any z i with P z (t+1) i | θ * , ϕ i < P z (t+1) i | θ ′ , ϕ iTherefore, at timestep t = 0, models with the shared misconception are more likely to yield responses which correlate with θ ′ .For rounds t > 0, we can express the conditional probability as,P z (t+1) i | x, Z (t) , ϕ i ∝ θ P z (t+1) i | θ, ϕ i n j=1 P z (t) j | θ, ϕ i P x, x ′ | θ, ϕ ′ i = θ P z (t+1) i | θ, ϕ i n j≤m P z (t) j | θ, ϕ i n j>m P z (t) j | θ, ϕ i P x| θ, ϕ ′ i P x ′ | θ, ϕ ′ iFor a given x, the termsn j≤m P z (t) j | θ, ϕ i x ′ | θ, ϕ ′ iare maximized, in expectation, for θ = θ ′ . Moreover, for the any concept θ ̸ = θ ′ , the ration j≤m P z (t) j | θ ′ , ϕ i x ′ | θ ′ , ϕ ′ i n j≤m P z (t) j | θ, ϕ i x ′ | θ, ϕ ′ iis monotonically increasing as m increases. Therefore, for any round, a model with the shared misconception is more likely to generate answers correlating with θ ′ , than those without the shared misconception. Further, the likelihood of generating such answers increases with m.Proof of Theorem 6.1. As shown in the proof of Theorem 5.4, we can express each model's conditional generation probability asP z (t+1) i | x, Z (t) , ϕ i ∝ θ P z (t+1) i | θ, ϕ i n j=1 P z (t) j | θ, ϕ i P x, x ′ | θ, ϕ ′ i = θ P z (t+1) i | θ, ϕ i n j≤m P z (t) j | θ, ϕ i n j>m P z (t) j | θ, ϕ i P x| θ, ϕ ′ i P x ′ | θ, ϕ ′ iwhere x ′ is a response which conveys the erroneous concept θ ′ . Under this expression, each response z(t)i is generated according to D(x, Z (t) , ϕ i ) where each distribution differs only by the model parameters ϕ i . For i ≤ m, the model parameters ϕ i posses the common misconception, i.e., each distribution D(z(t) j | x, Z (t) , ϕ i ) has a common scaling factor P x ′ | θ, ϕ ′i which is maximized at θ = θ ′ . Since only these models share this scaling factor,E KL D(θ ′ | z i1 ), D(θ ′ | z i2 ) ≥ E[KL D(θ ′ | z i1 ), D(θ ′ | z j ) whenever i 1 , i 2 ≤ m < j.Hence, diversity pruning is more likely to select terms from agents with m < j (i.e., those without the shared misconception), then agents with j ≤ m. Since this holds true on every round of debate, all responses in the debate process place a lower weight on responses associated with θ ′ , i.e., a lower weight is placed on responses which have a higher chance of being incorrect.Proof of Theorem 6.2. At each round t, quality pruning selection a set of k responses,Z ′(t) =argmin Y ⊂Z (t) zi,∈Y KL D(θ| x), D(θ| z i ) s.t. |Y | = kAs such, for any set of responses Z (t)z ′ i ∈Z ′(t) KL D(θ| x), D(θ| z ′ i ) ≤ zi∈Z (t) KL D(θ| x), D(θ| z i )Using the fact that Z ′(t) ⊂ Z (t) , we can write,z ′ i ∈Z ′(t) KL D(θ| x), D(θ| z ′ i ) ≤ z ′ i ∈Z ′(t) KL D(θ| x), D(θ| z ′ i ) + zi∈Z (t) \Z ′(t) KL D(θ| x), D(θ| z i ) =⇒ 0 ≤ zi∈Z (t) \Z ′(t) KL D(θ| x), D(θ| z i )Thus, for a random task and answer pair (x, y) ∼ D(θ * ), the relationship between the correctness of answers in Z (t) and Z ′(t) isE (x,y)∼D(θ * ) 1 |Z ′(t) | zi∈Z ′(t) P a(z i ) = y ≥ E (x,y)∼D(θ * ) 1 |Z (t) \ Z ′(t) | zi∈Z (t) \Z ′(t) P a(z i ) = yThat is, in expectation, the responses which are selected for quality pruning are at least as correct as those which are removed by quality pruning. Therefore, in expectation across all possible generations z (t+1) i with a(z(t) i ) = y, E z (t+1) i θ∈Θ P z (t+1) i | θ, ϕ i P x| θ, ϕ i P θ| ϕ i n z ′ j ∈Z ′(t) P z ′(t) j | θ, ϕ i ≥ E z (t+1) i θ∈Θ P z (t+1) i | θ, ϕ i P x| θ, ϕ i P θ| ϕ i n zj ∈Z (t) P z (t) j | θ, ϕ i =⇒ E z (t+1) i P z (t+1) i | x, Z (t) , ϕ i ≥ E z (t+1) i P z (t+1) i | x, Z ′(t) , ϕ iTherefore, the probability of model i generating a response z (t) i at time t, which has a(z(t)i ) = y is greater when conditioning only on the responses selected by quality pruning, i.e., Z ′(t) .Proof of Theorem 6.4. Let z be a given response and z ′ be a corrected version of that response after misconception refutation, thenKL D(θ| x, y), D(θ| z ′ ) ≤ KL D(θ| x, y), D(θ| z)That is, the distribution over concepts given the corrected response z ′ is more similar to the true distribution over concepts given the task x and answer y compared to the original answer z.Similar to the case of quality pruning, we can then express the KL divergence of all responses before refutation Z (t) , and after refutation Z ′(t) , asz ′ i ∈Z ′(t) KL D(θ| x, y), D(θ| z ′ i ) ≤ zi∈Z (t) KL D(θ| x, y), D(θ| z i ) =⇒ n i=1 KL D(θ| x, y), D(θ| z ′ i ) -KL D(θ| x, y), D(θ| z i ) ≤ 0Thus, in aggregate, misconception refutation results in all responses in Y ( t) inducing a distribution over concepts which is more similar to the distribution over concepts given the task x and answer y.As shown in the case of quality pruning, this relationship implies that at the next step of generation, model i is more likely to generate zEQUATION) = y when conditioning on Z ′(t) compared with Z (t) . 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: We discuss societal impacts.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA]Justification: [NA] Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: All data that we use is publicly available and properly cited.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.• If this information is not available online, the authors are encouraged to reach out to the asset's creators. 13. New Assets Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [NA] Justification: [NA] Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: Latent concepts, and by extension, shared misconceptions, are quite general and may not always be human-interpretable. To better elucidate what is meant by a shared misconception, we provide an example of one possible type of misconception. In Figure 4 we see model responses to a question regarding the song "Take Me Home, Country Roads", which is a well-known song about the state West Virginia3 . While the models identify the connection between the song and West Virginia, they each erroneously equate West Virginia and Virginia, ultimately leading to each model providing the wrong answer. In situations such as this, debate will converge to an incorrect answer due to each model sharing the same false belief.Misconceptions can also be viewed through the lens of hallucinations. As a byproduct of erroneous training, the models in the above example have learned a false connection between two topics (Virginia and West Virginia).Interventions When combining our three interventions together, we find that first applying quality pruning, then diversity pruning, then misconception refutation, results in the best performance. In practice, we consider all previous responses up to the current round t when applying pruning, i.e., Z (0) ∪ . . . ∪ Z (t) results in better performance. When considering all past responses during pruning, there is a potentential misconception refutation, then al that the same set of responses is picked at each round. We simply prevent the methods from selecting the same response during two consecutive rounds to avoid this issue. we use the distance between sentence embeddings of each string, x, z i , and z j , i.e., given sentence embedding model g, we to approximate the above in practice via, Z all .add(z (0) j ) // Get an initial set of responses from each LLM 6: end for 7: for t = 1 . . . T do 8: end for 15: end for 16: Return Z all [-n :] // Each model's response on the last round of debate Models For our experiments we make use of four models: GPT-3.5, Llama-2, Llama-3, and Mistral.EQUATIONZ ′(t) = QualityPrune Z all , k = 1/2|Z all | // Prune half the current responses 9: Z ′(t) = DiversityPrune Z ′(t) , k = n 10: Z ′(t) = MisconceptionRefutation Z ′(t) //For certain tasks such as BoolQ or MMLU, such as only providing "Yes" or "No" to BoolQ questions. Justifications for answers are important for both our method and regular debate, as such we set the minimum token for Flan-T5 to be 10, and we set the repetition penalty to be 1.5.Measuring Accuracy In the BoolQ, MMLU, and Math datasets, we extract the model's answers through regular expression checking and compare these extracted answers to the true answer; models are prompted to provide their final answer in the form "Final Answer: X". In the TrutfhulQA dataset Mistral 7B Instruct v02 huggingface model answers are taken to be their entire response, which is then judged as being correct or incorrect by a GPT-4 judge. This judge is prompted to provide a yes-no answer to the question "Dose the answer {_answer_} accurately answer the question {_question_}?". We allow for models to provide answers of abstention, e.g., responding "I do not know". Abstentions correspond to an accuracy of .5 in BoolQ, .25 in MMLU, 0 in Math, and are directly scored by the LLM judge in TruthfulQA.In Figure 1 we measure the likelihood that a given model will echo a target answer as a function of how many other model select that target answer. Target answers are Yes in BoolQ, option A in MMLU, correctAnswer -30 in Math, and a false answer in TruthfulQA.In our experiment, we elicit 20 answers from each model (GPT-3.5, Llama-2, Flan-T5) and then downsample these answers to ensure that each model receives a specific number of target answers. Resource Details For all experiments, we use one Nvdida Tesla V100 GPU and one Intel 32-core CPU. For inference with non-API models (i.e., Llama-2, Llama-3, and Mistral, we use the VLLM library Kwon et al. [2023] ). Ten rounds of debate with six models and 3,000 questions has a mean completion time of 4 hours for Llama-2, Llama-3, and Mistral. For GPT-3.5, we use the open library, ten rounds of debate with six models and 3,000 questions, has a mean completion time of 12 hours.We provide example prompt templates for the BoolQ dataset. Prompt templates for other datasets are similar but have changes to reflect the different task types (e.g., multiple-choice answers in MMLU compared to yes-no answers in BoolQ).Round We have read the code of ethics and are adhering to it.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).In practice, a can be a regular-expression checker or an LLM-based judge such asLiang et al. [2023]Any model not providing the target answer is re-prompted until the target answer is provided.Models are given a question and a passage (the passage is omitted from Figure4) 


Decision: The paper presents a theoretical framework for studying multi-LLM debate, and further proposes two types interventions to debate processes. The framework establishes principles of debates, including the diversity of model responses and shared misconceptions among the models. The first type of intervention is pruning, which filters responses to be included in the response, and the second type of intervention is to minimize misconception, verifying and editing previously generated answers. 

Lastly, they show empirical results of using their protocol compared to prior work [Du et al 23]. The reviewers appreciate the topic studied (reviewer 6kQa, iBgJ) and the empirical gains of suggested interventions (iBgJ, C8as). The authors should include discussions from the rebuttal period, such as analysis on how their framework increases inference costs and improve clarity of the paper.

Reviews: review: paper_summary: paper_summary: In this paper, the authors establish a theoretical framework for multiagent debate. The authors find that LLMs are susceptible to issues such as leaning toward majority opinion (tyranny of the majority) and error expansion during debates (shared misconceptions between models). Thus, the authors propose three types of interventions, including debate selection and debate modification to mitigate these issues. Results show that LLMs improve on 4 common datasets, BoolQ, MMLU, TruthfullQA, Math.
strength_weakness: strength_weakness: 1. Some assumptions seem too strong. It is assumed that each agent can only see one round of history. However, empirically, the agent can see all his own dialogues. For Asumpotion 4.1, how could the model respond without the reaction of other models? For instance, other models may react with opposite results. 

2. Lacks some details on experiments. How do you force the model to affirm and then prompt m of the models (out of 11) to provide responses (Line 287)? Besides, for Figure 2, what is the dataset used, and is it averaged across all 4 datasets?

3. Since debating is ready API costly, I would expect a time estimation for the proposed algorithm on one sample and how it compares with the original debating.
questions: questions: Some presentation typos:

line 292 $Z^{(0)}$

Figure 3 caption is repeating
limitations: limitations: I encourage the authors to write limitation and broader impacts sections independent of the main paper.
paper_summary: paper_summary: The paper presents a framework for multi-LLM debate, gives a detailed overview on drawbacks within multi-LLM debate and proposes several interventions to improve debates. The authors show the effectiveness of their interventions in several experiments on four benchmarks.
strength_weakness: strength_weakness: The related work section would be more readable, if the author's names where mentioned instead of the reference number. Like in: "[19] focuses on debate when each model..."
KL denotes the Kullback–Leibler divergence, which should be noted, even if it seems obvious.
There are several Mistral models. I would specify the model further (Mistral 7B).
Figure 3 is a bit too small and the two lines are the same color, so it's hard to tell them apart. 
You chose Debate as the main baseline tool. A detailed explanation of the method to understand the differences of the approaches is missing.
- The method and task are both called "debate.", which is confusing.
 
small errors: 
- line 82 should be "that agent i provides response zi(t+1), is given by"..
- no error, just a small suggestion: in line 86 write (1) and (2) instead of 1) and 2) which makes it more readable in my opinion 
- line 94: task x, and its associated answer y (singular) else I would not understand why x denotes multiple tasks?
- line 189: do you mean with the property instead of propriety?
- it seems that Diversity Pruning should maximize the KL-divergence and not the mutual information as stated in lines 232 and 233. In the diversity pruning we are maximizing over the answers Z so it should be Z ⊂ Z(t) instead of Y.
- What is S (z ∈ S) in the formula for misconception refutation?
- line 292: Z(0) instead of Z0) 
- Figure 3: small error underneath the graphic: you wrote pairwise similarity twice
questions: questions: - Maybe I overlooked the explanation in the paper, but what does alpha denote in Formula (3)?
- How does the misconception refutation work exactly? How should a model be able to identify misconceptions in the answer of another model?
limitations: limitations: An explicit sections that mentions limitations of the work is missing.
paper_summary: paper_summary: The main questions, that is tackled in this paper is: How can the debate of LLMs be influenced to ensure the best possible outcome. This question is broken down into three parts.

The first part is a theoretical model of debate between LLMs. The model is very intreaging. From the presentation it is not clear, why it should only be limited to the debate of LLMs. It would be very interesting to see, how the presented model of debate relates to models of debate from other fields, e.g. sociology.

The second part describes three methods of interfering with the outputs of the models to ensure that they do not converge to a suboptimal solution.

The third part shows the results of experiments.
strength_weakness: strength_weakness: There are some typos in the document, that need to be cleaned up.

The introduction to section 5 seems incomplete. Line 166 ends within a sentence. There seems to be a grammar mistake and a ) too much in line 158.
questions: questions: I had a hard time understanding, why Diversity Pruning is referred to as maximizing the mutual information. Intuitively, maximizing mutual information would mean, that by knowing X you know everything or a lot about Y. Here, you are doing to the opposite. You are maximizing the difference of D(Theta|z_i) and D(Theta|z_j), hence, knowing z_i should not give you much information on how z_j influences Theta. I would appreciate, if you could help me understand my misunderstanding.
limitations: limitations: The authors claim that they have addressed the limitations. I was not able to find that section. Please point out, where you discuss the limitation of your models and interventions.
paper_summary: paper_summary: The paper tackles the issue of echo chambers in multi-agent debate (MAD), a critical area that has not been sufficiently explored theoretically in current MAD research as the authors claimed. The authors propose three interventions – diversity pruning, quality pruning, and refuting misconceptions – to address this problem.
strength_weakness: strength_weakness: Major critiques in several areas:

A. Theoretical Issues:

A.1 Underdeveloped theoretical foundation: While the authors propose theorems, many appear trivial or state obvious conclusions (e.g., Theorems 5.1 and 5.2).

A.2 Ambiguous parameter representation: The representation of θ in Assumption 4.1 is unclear, raising questions about how model parameters are represented and changed.

A.3 Shared latent space issues: Lemma 4.2 doesn't adequately address how different LLMs would share their latent space.

B. Methodological Concerns:

B.1 Simplistic multi-agent dialogue setup: The model fails to fully leverage the potential of a multi-LLM ensemble, using it more like traditional ensemble schemes.

B.2 Unclear latent concept model: The origin and nature of latent concepts in Section 2 lack clarity.
Lack of quantifiability: The formulation in Section 4 cannot be quantified, leading to reliance on heuristics.

C. Implementation and Experimental Issues:

C.1 Disconnect between theory and implementation: Section 6 presents questionable formulations, particularly in the use of KL divergence and reliance on sentence embedding as a proxy.

C.2 Outdated model selection: Experiments use older models instead of more recent, capable ones.
Limited experimental insights: Results only show significant improvement in the first round of voting.

C.3 Questionable example: The example in Section 5.2 doesn't effectively illustrate the point about misconceptions.

Conclusion: While the paper addresses an important issue and attempts to provide theoretical grounding, it falls short in several areas. The multi-LLM debate framework presented doesn't demonstrate its full potential for sophisticated decision-making tasks. The intervention strategies and control parameters lack necessary granularity. Despite the paper's accessibility, the gap between theoretical concepts and practical application remains significant, with many proofs being trivial or of limited practical value.

Recommendation for future work: Focus on developing more sophisticated decision-making tasks, refine intervention strategies, and use state-of-the-art language models for experiments. Additionally, strengthen the connection between theoretical concepts and practical implementations to enhance the paper's overall impact and relevance.
questions: questions: 1. Suppose KL represents KL divergence. KL divergence is asymmetric and unbounded. Why not use other metrics?

2. In the experiment section, would you consider adding ablation tests to determine which of your intervention strategies are effective?

3. Can you clarify how you pass the distribution of latent concepts between LLMs?

4. Please find more related papers in multi-LLM debate or dialogue.  You missed a couple of key papers.
limitations: limitations: NO! Limitation section missing.

The authors on the checklist state that they submitted limitations, but they did not.
This paper could have been desk-rejected.

Comments: title: Misunderstanding
Answer
That last comment was not for this paper, but copied from somewhere else
Author Followup
Authors' Response
related work, etc.
comment: I pasted the statement of rejecting my papers.  Of course I have no authority what-so-ever to make that decision.

Good luck!
Dear reviewer, 
Thank you for carefully reading the paper and raising the issue. However, it is not your role to decide whether the paper gets desk-rejected or not. I'll check with SAC about this, but I do not think this is a serious enough issue for desk rejection.
Thanks for the point-by-point in-depth analysis and thoughtful explanations. I appreciate that my concerns on W2 and W3 are mostly addressed. I encourage the authors to include the analysis in the final version. Thanks!
Thank you very much for the rebuttal and addressing my comments. I am looking forward to the final version of the paper.
Thank you for your feedback.

The last comment from reviewer 1QQG is **not** for this paper. We believe it is a paragraph copied from the reviewer either from their end or from somewhere else. 

As we replied to reviewer 1QQG, at the time of submission, we had a different interpretation of the limitation requirement (Does the paper discuss the limitations of the work performed by the authors?). While a separate section is encouraged in the guideline, we thought the paper flows better when we discuss the limitations right in the main text, and labeling the answer as "Yes" simply states that we are aware of our limitations and have discussed them in the main text. Indeed, in the checklist, we had the following justification: "Justification: We mention the limitation of our approach in the conclusion section and experiment section".

Please let me know if there are further questions.
We are sorry to hear the unfortunate desk rejections from your end, and thank you for reminding us of this. In the future, we will pay closer attention to it. 

At the time of submission, we had a different interpretation of the limitation requirement. While a separate section is encouraged, we thought the paper flows better when we discuss the limitations right in the main text, and labeling Yes simply states that we are aware of our limitations and have discussed them in the main text. (in the checklist, we added the justification that "Justification: We mention the limitation of our approach in the conclusion section and experiment section").

That said, we would like to be clear that we fully agree with the reviewer's comments regarding our discussion of limitations; our initial discussion of limitations was insufficient and should be provided in its own section (please see our general response for our proposal).
Thank you for considering all my remarks for adapting your final manuscript.
I will adjust my rating accordingly. I am just confused about the comment from reviewer 1QQG who pointed out that the paper should have been desk rejected because of the missing limitations sections. Is the last comment an official response from the area chairs?
Dear Authors: Thank you for your response.  I will adjust the rating accordingly.

Dear Area Chair,

Please determine whether the omission of the "limitations" section in the submitted checklist is a significant issue. Consistency in review policy is crucial for fairness, especially given that both of our NeurIPS submissions were desk-rejected due to checklist-related issues.

Thank you.

-------------------

Dear Authors,

According to the NeurIPS policies, any violence of formatting instructions (as indicated in the Call for Papers: https://neurips.cc/Conferences/2024/CallForPapers -> Formatting instructions) results in a desk rejection. Your paper was rejected since the NeurIPS paper checklist is part of these formatting instructions (see  https://neurips.cc/Conferences/2024/CallForPapers -> Formatting instructions -> Point 3). Unfortunately, we are unable to allow further submissions or amendments to submissions as the system is now closed.

We are aware that this is not a decision you expected, but we must follow the NeurIPS policies. The decision is final and cannot be reverted.
Thank you for the pointer to this textbook! We were not aware of this book at the time of submission. We agree that this text is highly relevant to any paper investigating multi-LLM debate. We plan to cite this book and make special mention of chapters 5 and 6; specifically, we propose to add the following text to our related work section:

> “An overview of debate techniques and analysis can be found in [Chang 24]. Chapter 5 outlines a debate procedure that dynamically controls the amount of disagreement between debating LLMs. Chapter 6 outlines the way in which LLM responses change during the debate process; distributions over responses tend to converge (approximately) to consensus as debate rounds increase.”

As you point out, chapter 6 offers justification for studying distributions of LLM responses via several metrics including KL. For clarity we would like to mention that we study distributions over latent concepts (see Lines 233, 237 and 260 of our paper) rather than distributions over LLM responses (as is the case in [Chang 24]). Further differentiating our work is the use of these distributions to then optimize the debate procedure.

If we can provide any additional clarification, please let us know.

[1] Edward Y. Chang (2024). The Path to Artificial General Intelligence –- Insights from Adversarial LLM Dialogue.
Thank you for providing clarifications. Because of the tight schedule, let me provide a textbook relevant to your work for consideration and discussion. The book was used in a university class and was known in the NLP community. Its Chapters 4 to 9, and especially chapter 5 and 6 are highly relevant to this paper.  E.g., chapter 6 justifies using cross-entropy, earth-mover, mutual information and other metrics in addition to the KL.

https://www.amazon.com/dp/B0D8NGQZFR