Title: AI Debate Aids Assessment of Controversial Claims
Venue: NeurIPS 2025
Accepted: yes
Keywords: Scalable Oversight

AI Debate

Cognitive Bias

Factuality
Abstract: As AI grows more powerful, it will increasingly shape how we understand the world. But with this influence comes the risk of amplifying misinformation and deepening social divides—especially on consequential topics where factual accuracy directly impacts well-being. Scalable Oversight aims to ensure AI systems remain truthful even when their capabilities exceed those of their evaluators. Yet when humans serve as evaluators, their own beliefs and biases can impair judgment. We study whether AI debate can guide biased judges toward the truth by having two AI systems debate opposing sides of controversial factuality claims on COVID-19 and climate change where people hold strong prior beliefs. We conduct two studies. Study I recruits human judges with either mainstream or skeptical beliefs who evaluate claims through two protocols: debate (interaction with two AI advisors arguing opposing sides) or consultancy (interaction with a single AI advisor). Study II uses AI judges with and without human-like personas to evaluate the same protocols. In Study I, debate consistently improves human judgment accuracy and confidence calibration, outperforming consultancy by 4-10\% across COVID-19 and climate change claims. The improvement is most significant for judges with mainstream beliefs (up to +15.2\% accuracy on COVID-19 claims), though debate also helps skeptical judges who initially misjudge claims move toward accurate views (+4.7\% accuracy). In Study II, AI judges with human-like personas achieve even higher accuracy (78.5\%) than human judges (70.1\%) and default AI judges without personas (69.8\%), suggesting their potential for supervising frontier AI models. These findings highlight AI debate as a promising path toward scalable, bias-resilient oversight in contested domains.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn the context of approaches to scaling oversight of large AI models such as “frontier models performing tasks beyond human capabilities,” the paper aims to gauge approaches that can help ensure that such oversight is reliable, and that human or AI agent judges involved in conducting oversight tasks are not swayed by the models themselves toward incorrect judgements. Through two studies, the research attempts to measure and compare two frameworks for scalable oversight, Debate and Consultancy; One study used human judges and the other used automated judges.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\nThe authors are clear in their goals to study the efficacy of emerging frameworks for scaling oversight of large AI models in the context of problems related to the more widespread technique of reinforcement learning with human feedback. Their work supplements prior work in the area of gauging the Debate protocol introduced in 2018, particularly in comparison to the Consultancy framework. The authors cite related prior work and offer a helpful section detailing how it relates to their research. Their work also attempts to consider and address problems that may arise from manipulation of human and automated judges.\n\nWeaknesses \nThe paper draws on prior work  in important and potentially problematic ways; some key prior work either has not been peer-reviewed or is peer-reviewed but itself based on non-peer reviewed work. The area of scalable oversight does not appear to be researched robustly yet and may benefit from genuine peer review of “foundational” papers before researchers continue to attempt to advance it.\n\nAnother weakness: The authors conclude that their work could have greater implications than the research, which offers a relatively small sample size, may warrant. Despite the fact that their research studied application of the Debate and comparison frameworks only in the context of COVID-19 related information, and in a relatively limited capacity, they conclude that the Debate framework could be “a practical and scalable mechanism for oversight in high-stakes, belief-sensitive domains.”\n\nAlso, the authors also cite a paper used in relation to their dataset discovery [50], which appears to have been withdrawn by its authors (https://openreview.net/forum?id=Jztt1nrjAM). Although an updated version of the paper exists (https://arxiv.org/pdf/2411.05060), they'll need to clarify and possibly update the citation.\n\nQUESTIONS:\n-Have the authors considered adjusting their conclusion to temper their assessment that the work has implications for high-stakes decisions? High stakes decisions could go far beyond public health related information.\n-Have the authors considered providing more detail on why they considered a judge’s background and beliefs? It would be helpful here to more directly state the reasoning behind this, even if it seems obvious.\n\nLIMITATIONS:\nThe authors seem to recognize that their criteria for selecting datasets to conduct the research creates important limitations, noting that \"Few datasets exist satisfying all of these criteria.\" The choice of COVID-19 as the subject area for study presents problems, in part because the topic still remains unsettled in many ways. There do exist fact-checking datasets that include data from earlier years that could be more suitable.\n\nUltimately, the societal impacts of automating decisions regarding accuracy of information related to highly-contested and belief-oriented concepts is not only a challenge, it could have negative impacts on people that are not considered by the authors. While the study is a small way of understanding the caveats, the authors could benefit from further analysis of the very goal of this work.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nWhile scalable oversight methods have typically been conceptualized in the context of training and evaluation of AI models for correctness/truthfulness, the same mechanisms could be considered in the context of improving the information landscape in general. This paper touches on this goal, specifically for evaluating the correctness of misinformation/debunking misinformation. They focus on COVID-19 claims, and ran studies with human judges (sampling from both those who believed the mainstream claim and those who were skeptical of it) as well as LLM judges (prompted with a \"persona\" priming it to have biases matching the human sample) comparing Debate against Consultancy for effectiveness at improving the accuracy of judges' beliefs (i.e. to match the mainstream view).\n\nThey find that:\n- Debate is better than Consultancy at improving judge accuracy.\n- The effect is especially stronger for Judges who already hold the correct belief (i.e. the impact in strengthening mainstream believers' convictions is more than the impact in making skeptics switch beliefs)\n\nSTRENGTHS AND WEAKNESSES:\n*Strengths*\n\n1) The application of Debate to evaluating information in \"the wild\" is novel to my knowledge; while existing works focus on evaluating AI models.\n2) The study accounts for the problem for the judges having preconceived biases (which is a problem unique to this domain; judges would not have preconceived biases over a question-answering task for example) and evaluate the cases of the judge being preconceived toward each side separately.\n3) They also account for this in their experiment with LLM judges, using persona-based LLMs (based on the demographics of the people in their data who hold each belief) to simulate biased judges.\n\n*Weaknesses*\n\n1) The most serious weakness is in the sample of \"misinformation claims\": for example, it is possible that AI debate is good at convincing people against /non-mainstream/ viewpoints rather than /false/ ones (one obvious hypothesis for why this might be: mainstream claims have a larger body of literature backing them than non-mainstream claims), and the chosen sample (COVID-19 claims etc.) simply leads to confounding between these features.\n\n   One robust way to select a better sample: look at controversial claims being traded on prediction markets or metaculus, e.g. claims about ongoing wars, causes for accidents, or back when the LK-99 superconductor story was in the news, or COVID-19 questions where they were yet unresolved, or Biden's health issues --- all of these were controversial questions whose ground truth would eventually be (but were not yet) resolved. Do the experiment with these claims now, then add ground truth labels once the markets resolve. This way you can also evaluate the confounding effect of \"mainstream-ness\" separately.\n\n   The paper consider the problem of the judges having preconceived biases, but not of biases in the dataset.\n\n2) In fact the paper seems to have /only/ looked at COVID-19 claims as their misinformation sample. This severely limits generalization!\n   \n3) Not enough engagement with existing literature applying scalable oversight to misinformation/etc. There is some previous work in this area, e.g. https://www.science.org/doi/10.1126/science.adq1814 which has not been cited. (Incidentally this paper also reinforces the point in 1).\n\n4) The consultancy baseline is quite weak --- qualitatively the result only demonstrates that \"hearing both sides is better than hearing the wrong side 50\\% of the time\", which is quite an expected result. This weakness (even in previous debate experiments) is pointed out in https://arxiv.org/pdf/2504.03731 who propose a more robust benchmark.\n\nQUESTIONS:\n1) What is the explanation for focusing on COVID-19 claims?\n2) To what extent do you think the results are influenced by the relative _prior convictions_ in each belief? --- it seems very likely that people who hold non-mainstream beliefs are less sure of them.\n3) I am interested to know how the educational background of the believers affects their susceptibility to persuasion --- this is also a confounding factor, but perhaps one you could adjust for from your existing data.\n4) I am also interested by your finding that the belief update is greater for those who already hold the mainstream belief --- this would indicate that debate would widen the belief gap. Do you think this might be a factor affecting political polarization in the real world?\n\nLIMITATIONS:\nI am not familiar enough with ethical stuff to say for certain, but the collection of demographic info from the human participants to create \"LLM personas\" of them might be a cause for concern. The authors say:\n\n> This study was performed under approval from the appropriate institutional ethics review board. Full IRB documentation will be made available upon request.\n\nIt might be worth requesting this.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper investigates whether adversarial debate between large language models (LLMs) can improve human and AI judgment on controversial factual claims, specifically in the context of COVID-19 misinformation. It compares a debate protocol, where two AI agents argue opposing sides of a claim, to a consultancy setup involving a single AI advisor who can be \"consulted\". Human judges with either \"mainstream\" or \"skeptical\" beliefs are evaluated pre- and post-intervention, and a second study compares the performance of persona-based LLM judges against human judges.\n\nThe authors find that debate improves factual accuracy more than consultancy. Persona-based LLM judges outperform humans and default LLMs in supervising factual claims. The authors argue that debate promotes better belief updating and confidence calibration, supporting its use as a scalable oversight mechanism.\n\nSTRENGTHS AND WEAKNESSES:\nKindly note that my comments are based solely on the content presented in the main paper. While I appreciate that supplementary material can provide additional context, I believe the main paper should contain the core information necessary to understand the experimental design and interpret the results. In its current form, the paper relies heavily on a lengthy appendix (over 60 pages), which places a significant burden on the reader.\n\n### Strengths\n\n- The paper addresses a interesting and valid issue in AI alignment and safety: how to supervise LLMs when humans are biased or less knowledgeable. Motivation for the study is clear and well presented in the paper.\n- The use of adversarial AI debate as a tool to correct cognitive bias in factual reasoning is very intuitive.\n- The authors conduct two comprehensive studies: one with human judges and another with persona-based LLM judges, covering both human-in-the-loop and AI-only supervision settings. The sample includes both \"mainstream\" and \"skeptical\" belief holders, allowing for better analysis of how AI interaction affects different cognitive baselines.\n\n\n### Weaknesses\n- The baseline (consultancy) is very weak in comparison to the debate method, so the fact that debate performs better than baseline is unsurprising. \n- The study is limited to one domain. While this is a high-stakes domain, it is hard to asses whether results generalize to other polarized or technical topics (e.g., climate change)\n- Most of the questions that come up while reading the paper is actually answered in the appendix. Can you provide a visualization of the study setup? What kind of claims were used? Where legitimate evidences / sources presented by LLM debaters. etc. \n- While this is a promising and empirically grounded paper exploring AI debate for oversight, the strongest contributions are undercut by over-reliance on the appendix, missing qualitative depth, and a somewhat weak baseline comparison.\n\nQUESTIONS:\n- Abstract: It is hard to parse the abstract accurately in isolation without having read the complete paper. There are 2 studies, one with human judges and one with personalized AI judges. And there are two protocols, debate and consultancy. The relationships between the studies, judge types, and protocols are not clearly laid out, making it hard to follow the overall structure and contributions of the work.\n- Introduction: I found it difficult to understand the overall setup of the study just from the text. It would be nice if you can come up with some examples or visualization to make this more clear (move from appendix to main paper). \n- I recommend revising the main paper to include more methodological transparency by moving the content from appendix to main paper.\n\nLIMITATIONS:\nSince the topic is related to deception and persuasiveness, it is important to discuss the impact and potential misuse of such systems.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper explores whether AI-assisted debate, in which two AI systems argue opposing viewpoints, can help human judges more accurately evaluate controversial factual claims related to COVID-19, compared to traditional AI consultancy (single advisor). The authors conduct two studies: (1) a human-judge experiment contrasting debate versus consultancy across different belief systems (mainstream vs. skeptical), and (2) a study using personalized AI judges designed to simulate human biases. Results indicate that debate consistently improves human accuracy and confidence calibration, especially among mainstream believers, while personalized AI judges outperform both humans and non-personalized AI judges. The findings suggest debate could provide scalable, bias-resilient oversight in critical factuality assessment tasks.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n**\\[S1] Practical significance:**\nGiven growing concerns around misinformation, polarization, and LLM safety, this work addresses timely and urgent issues, providing valuable evidence that structured AI debate can meaningfully improve human judgment in controversial domains.\n\n**\\[S2] Careful, transparent experimental design:**\nThe use of real-world, contentious claims (COVID-19), pre-screened participants with diverse beliefs, and methodical protocols (debate vs consultancy) is strong. The statistical reporting is detailed, and key confounds (e.g., prior confidence, judge experience, belief strength) are explicitly measured and modeled.\n\n**\\[S3] Nuanced, actionable findings:**\nThe paper does not claim universal superiority of debate, but rather documents specific scenarios where debate outperforms (especially for mainstream judges or judges with low confidence) and highlights the limitations of consultancy, especially when the consultant reinforces pre-existing (sometimes incorrect) beliefs.\nAdditionally, the detailed analysis of persuasion strategies, calibration curves, and confidence shifts adds considerable nuance, helping explain mechanisms behind observed improvements.\n\n\n### Weaknesses\n\n**\\[W1] Limited Comparison and Benchmarking:**\nThe consultancy condition is notably weak (randomly correct 50% of the time), potentially overstating debate advantages. Stronger baselines, such as single-advisor conditions optimized for user persuasion or calibrated trust could substantially challenge the debate method’s dominance.\nMoreover, the choice of COVID-19 claims, though justified by the authors, remains narrow. It is unclear how findings generalize to less scientific, more nuanced topics (e.g., politics, ethics).\n\n**\\[W2] Persona-based AI judges:**\nWhile personalized AI judges provide interesting findings, limited exploration of ethical implications (e.g., personalization exacerbating biases rather than mitigating them) weakens the practical recommendation for deployment.\n\nQUESTIONS:\n- While personalized judges show higher accuracy, could they also reinforce problematic biases, echo chambers, or manipulation risks? How can we ensure personalized oversight enhances rather than diminishes epistemic integrity?\n- Given debate’s strong performance, how might it integrate into practical workflows (e.g., news fact-checking)? What infrastructure or training would human evaluators require to use AI debate effectively?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}