GAM-Agent: Game-Theoretic and Uncertainty-Aware Collaboration for Complex Visual Reasoning
Authors: Jusheng Zhang
Yijia Fan
Wenjun Lin
Ruiqi Chen
Haoyi Jiang
Wenhao Chai
Jian Wang
Keze Wang
Venue: NeurIPS 2025 poster ()
Keywords: Multi-Agent; Uncertainty; Visual Language Model
Primary area: applications
TL;DR: We propose GAM-Agent, a game-theoretic multi-agent framework where visual and logic agents debate via structured communication and uncertainty control, boosting VLM performance, robustness, and interpretability. It is modular, scalable, and general.

Abstract: We propose **GAM-Agent**, a game-theoretic multi-agent framework for enhancing vision-language reasoning. Unlike prior single-agent or monolithic models, GAM-Agent formulates the reasoning process as a non-zero-sum game between base agents—each specializing in visual perception subtasks—and a critical agent that verifies logic consistency and factual correctness. Agents communicate via structured claims, evidence, and uncertainty estimates. The framework introduces an uncertainty-aware controller to dynamically adjust agent collaboration, triggering multi-round debates when disagreement or ambiguity is detected. This process yields more robust and interpretable predictions. Experiments on four challenging benchmarks—MMMU, MMBench, MVBench, and V*Bench—demonstrate that GAM-Agent significantly improves performance across various VLM backbones. Notably, GAM-Agent boosts the accuracy of small-to-mid scale models (e.g., Qwen2.5-VL-7B, InternVL3-14B) by 5–6\%, and still enhances strong models like GPT-4o by up to 2–3\%. Our approach is modular, scalable, and generalizable, offering a path toward reliable and explainable multi-agent multimodal reasoning.

GAM-Agent: Game-Theoretic and Uncertainty-Aware Collaboration for Complex Visual Reasoning
Jusheng Zhang" Yijia Fan Wenjun Lin', Ruiqi Chen',
Haoyi Jiang', Wenhao Chai Jian Wang Keze Wang
Sun Yat-sen University
PPrinceton University
SSnap Inc.
*Corresponding author: kezewang@@gaill com
Abstract
We propose GAM-Agent, game-theoretic multi-agent framework for enhane- ing vision-language reasoning. Unlike prior single-agent or monolithic models, GAM-Agent formulates the reasoning process non-zero-sum game between base agents each specializing in visual perception subtasks-an and a critical agent that verifies logic consistency and factual correctness. Agents communicate via structured claims, evidence, and uncertainty estimates. The framework introduces an uncertainty-awant controller to dynamically adjust agent collaboration, trigger- ing multi-round debates when disagreement or ambiguity is detected. This process yields more robust and interpretable predictions. Experiments on four challenging benchmarks MMMMhM MMBench, MVBench, and V*Bench demonstrate that GAM-Agent significantly improves performance across various VLM backbones. Notably, GAM-Agent boosts the accuracy of small-to-mid scale models (e.g.. Swen2.55VLL-7H Intern VL33-1B) by 5-6%, and still enhances strong models like GPT-4o by up to 2-3%. Our approach is modular, scalable, and generalizable, offering a path toward reliable and explainable multi-agent multimodal reasoning.
Introduction
has made significant progress in tasks such as complex reasoning, code generation, and knowledge integration, overcoming the limitations of single models through multi-agent collaboration 774 11
Recently, large language models (LLMs) 4 31
19
50
53
67
4 71]. In contrast, although there have been advances in integrating visual perception with language understanding
28
33
73,, the potential of multi-agent collaboration for VLMs remains largely untapped. Most existing VLM approaches rely on single models simple ensemble strategies, which struggle to address challenges like multi- step reasoning and visual ambiguity in complex visual reasoning tasks 17 10] Benefiting from the development of multi-agent systems [8] 55 7. some recent studies have proposed mueti- debate frameworks to compensate for the shortcomings of VLMs and have obtained promising results (56] II 25). However, these methods typically adopt simple mechanisms such as averaging or voting mechanisms and lack visual reasoning-based strategic interactions between agents [55] 18 Hence, they perform poorly in nigh-complexity visual reasoning scenarios (e.g., MMMU 64 Just as human experts reach consensus through strategic game-like interactions when they disagree 47
in the field of vision language models (VLMs) 42 27
10
9
68
70
69
45 58 3]
introducing deep collaborative game-theoretic mechanism into complex visual
reasoning is valuable endeavor [3], which has already seen preliminary applications in some
LLM-based works (33 (2).
However, current collaborative game-theoretic methods are often highly complex and heavily rely on reasoning paths, clues, and the fusion of multiple information, making them difficult to apply directly
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

to visual reasoning [54] 14 12 72 48) To address this issue, this paper explores the underlying
architecture of existing VLMs, effectively utiliizes important intermediate results in the reasoning process, and extracts representations of uncertainty in reasoning outcomes [21]. Specifically, we propose a collaborative on game-theoretic and uncertainty-aware inference. In this way, the complex visual reasoning process can be modeled as a non-zero-sum game involving multiple agents collaborating to reach consensus [63 266.. Specifically, we encourage agents to share their respective assessments of reasoning uncertainty and engage in a progressive interactive game to guide GAM-Agent to evolve step-by-step and ultimately reach consensus.
theorthe framework [37 26
44 30, named GAM-Agent, based
To address these challenges, we introduce GAM-Agent, a novel agent collaboration framework centered around a strategic interplay between two specialized agent cohorts: Base Agents and Critical Agents. The Base Agents are tasked with initial visual interpretation and evidence generation from distinct perspectives, such as object recognition, scene description, and textual analysis from images. Concurrently, Critical Agents, acting as reasoning critique experts, scrutinize the outputs from Base Agents and evaluate factual accuracy, logical coherence, and overall completeness. The core of our GAM-Agent lies in modeling the interaction between these Base and Critical Agents as a non- zero-sum game, fundamentally arbitrated by quantified uncertainty. In this game, agents iteratively share and refine their uncertainty assessments regarding their claims and evidence, engaging in structured debate process aimed at progressively reducing ambiguity and converging toward consensus. This uncertainty-driit driven, game- theoretic collaboration allows for dynamic and strategic integration of diverse insights, leading to more robust and reliable visual reasoning outcomes. Specifically, Base Agents first generate diverse preliminary analyses and identify supporting evidence for their claims. These outputs then processed by Claim Parser module, which deconstructs the unstructured responses into structured information tuples, and an Evidence Mapping module, which links these textual claims specific visual regions in the input image, thereby grounding the reasoning process. Besides, an Uncertainty Quantification mechanism continuously assesses the confidence of each agent's contribution. The entire process orchestrated by Debate Controller & Integrator. This component first evaluates the initial consensus and system uncertainty. If significant discrepancies or high uncertainty are detected, it initiates an iterative debate. During this debate, Base Agents refine their arguments, while Critical Agents provide evaluations, with the Uncertainty Quantification guiding the dynamic weighting and integration of information. This iterative loop continues, progressively refining the collective understanding and reducing uncertainty until a robust consensus is achieved or termination criteria are met. Extensive and comprehensive evaluations on large-scale benchmarks demonstrate the superiority of our GAM-Agent. Experimental results show that GAM-Agent achieves significant performance improvements on multiple complex visual reasoning benchmarks such as MMMU, MMBench, MVBench, and V*Bench. For example, it boosts the accuracy of small-to-mid scale VLMs by 5-6% and enhances the accuracy of top-tier models like GPT-4o by up to 2-3%.
2 Methodology
Our GAM-Agent is defined six-tuple S (EA E 1,2,..., N represents the set of N expert agents, each possessing distinct cognitive perspectives. A= A1, A2 Ag...,, AN defines the set of analysis capability mapping functions, where A; : X Pr + R maps an input image X and a question Pr to agent i's response Rú +b = 1,, 22...., &N is the set of uncertainty assessment functions, where Φ; R [0, 1] quantifies the uncertainty U, associated with response RG. M- = M1, M2,... My represents the set of evidence localization mapping functions, where M; : Ci X + x [0, 1] associates claim with a visual region r and confidence score a. Ci denotes the set of claims made by agent i, and V is the space of visual 1 the claim and evidence parser, converting an unstructured
regions. P R + (c,, oy, response R into a set of structured information tuples, each containing claim Cj. confidence øj. textual evidence ej. and visual region reference rj. D X x Pr x Ri, U,i = 1.v R the debate module aligns conflict and consensus for Riinat-
2.1 Base and Critical Agents
Our GAM-Agent employs two distinct categories of agents: Base Agents and Critical Agents, each with specialized roles in the visual reasoning process. Base Agents[1l] are conceived as specialized
2

---

Uncertaiity
offemsive player
F Wi
Exidencs Mapping
Base Agents
Gapry Irage
Uncertain Weight
x
Critical Agents Debate
Parser
Mapping
...
Ws
canergesoe? Expected Weight
Pare
Is Sipher dery levie
Teat
(a) Base Agents
(b) Critical Agents
Stephen (urryb (blue/ yellow uniform) is drihiling while the appasing player (white/ nary uniform) is Carry pessession, maneovering apponent's stance clearly undicate offengive rale.
The enalyss correctly identifies Curry offerse, supported by the opponent's del Nefansive Carry dribling against blacking defender No corrections needed.
Object Recognition
Foct Checker (Inape) agent
evidence
In on interse indeor arena, Stephen Curry (Golden State Worriers unifertn) is offerswely drobhing and ettempting clearly defensive opponent, movements suggesting offerswe play.
The compreeentinng and accurate ahalysis (Curry offenmie) well supported but could
Scene Description agent
I Completeness Cheeker agent
ID, game specific maves, and crowd reoction for greater depth.
Text on jersyy "Rokuter") and court logo identify the While direct proof, sribbling being guorded strongly impliss offensivee action.
jersey ("GOLDEN STATE"
The reasoning that Curry of offenge logically consistent, aligning his actions (dribóling, opponent's defensive stance nerrative. Minor detais iike defender's reaction could enhonce it.
posture)
Text/OCk Analysis
Logic
egent
Figure 1: Our GAM-Agent pipeline processing a visual query: Initial responses from Base Agents( (R)) undergo Claim Parsing and Evidence Mapping before an Uncertainty Convergence Judgment assesses consensus, If required, the Debate Controller & Integrator orchestrate an iterative loop, engaging both Base Agents for argumentation and Critical Agents for verification, refining the response until convergence or termination to produce the final output.
visual reasoning experts. Their primary function is analyze the visual input and generate initial interpretations or findings related to specific aspects of the scene or query. Examples of Base Agents include: an Object Recognition Agent, tasked with identifying and localizing objects within the visual input; Scene Description Agent, focused on generating textual description of the overall scene, its context, and the relationships between entities; and an OCR Agent, specialized in extracting textual information from images. Critical Agents function specialized reasoning critique experts. Their role to scrutinize the outputs generated by the Base Agents (and potentially other Critical Agents), identify inconsistencies, and assess factual accuracy, logical coherence, and Critical Agents include: a Fact Checker Agent, which verifies
overall completeness. Examples the factual claims made by other agents against known information contextual understanding; Completeness Checker Ágent, which assesses whether the information provided is sufficient and comprehensive in addressing the query or task; and Logic Checker Agent, which evaluates the logical consistency and validity of the reasoning steps and conclusions presented. With these two tiers of expert agents, our GAM-Agent introduces iterative came-theoretic interplay and information refinement process via uncertainty quantification to perform robust and accurate visual reasoning.
2.2 Uncertainty Quantification Function (.)
We propose a dual-level Φ; that models uncertainty from both response generation and overall semantics. Coneration-Proeess Based Uncertainty ( (Pigen)) When the probability distribution Pr, of the underlying VLM during the generation of the token sequence Y, (Vi,1. T, is accessible, we propose an integrated multi-feature uncertainty metric & Pigen+ (R;)
T:
4 Digen+ (Ri)
T. a
H(P..)
+3 13..max001111 - Atop (PPe)
(I)
t=1
Information Finrreyy
Probability Difference Inverno (Lack Comiccion)
(Hlssiation)
where H(Pr.) is the information entropy quantifying the dispersion of the token distribution, reflect- ing the model's hesitation, and Atop PP.)) = (1) (2))(2) (2) is the difference between the top two token probabilities, capturing the model's conviction in its top choice. Parameters and 3 balance

---

these complementary signals. This method captures microscopic uncertainty patterns during token generation, shifting from output evaluation generation process evaluation.
2.2.1 Semantic-Marker Based Uncertainty (Pisem)
When generation probabilities unavailable, GAM-Agent adopts semantic marker-based strategy, Pisem (R.), to assess uncertainty. This strategy systematically identifies and quantifies uncertainty markers within the text: Pisem (R.) asigmoid (k: (p(Rt) offiset). where W repre- sents multi-level lexicon of uncertainty markers. For each marker w E W, weight(w (w) reflects its uncertainty intensity, count(us w. R.) denotes its frequency in response R, and [R&] the length of the response. The parameters k and offset control scaling and bias, respectively, while Osimmoa(") normalizes the score the (0, 1) interval. By statistically analyzing semantic features in the text, this strategy provides a reliable estimate of uncertainty. Moreover, the uncertainty U; for agent i is determined using a priority strategy: if generation probabilities are available, U/ = 4 iigen+ (R,), cap- turing uncertainty inherent to the generation process. Otherwise, U; = Disem ( (Rs) is used, reflecting uncertainty inferred from semantic markers. This enables reliable uncertainty quantification for U, in any setting, supporting response weighting, conflict detection, and iterative debate.
2.3 Initial Integration and Conflict Detection
After obtaining initial responses from expert agents, GAM-Agent introduces an adaptive weight allocation and conflict assessment mechanism. This initial integration process efficiently aggregates diverse viewpoints and precisely identifies cognitive disagreements requiring debate.
Initial Weight Allocation 74 and Response Integration. GAM-Agent dynamically assigns initial
weights wi based on each agent's 's uncertainty assessment U; and generates an integrated response
(0)
R()): w
(0)
R IntegrateJudgee Pr. (R,, (_))) The parameter B
controls the sensitivity of weights to uncertainty. The IntegrateJudge function performs quality assessment, mutual support analysis, and information merging, balancing high-confidence opinions
with diverse perspectives.
Conflict Detection and Debate Triggering. GAM-Agent calculates the weighted average uncertainty to decide whether to initiate the debate
(9) and an inter-expert conflict score process using dual-criterion strategy:
N
U
U
TriggerDebale
(U (Con Oc)
System uncerianrty csccada thcoholi
Signileara inicrenapri conflit
Wrightsd average uncoriiiity
where Con ConflicsScore computed from the consistency of key claims, evidence interpretation, and ic, the system triggers an iterative debate. This
agent logic. If T > Gy or ensures quick consensus on easy cases while invoking deeper collaboration when needed.
2.4 Evidence Mapping (M) and Claim Parsing (P)
GAM-Agent incorporates an Evidence Mapping module (M) and a Claim Parsing module (P) as core components bridging visual and language reasoning. These modules construct a traceable network linking textual claims to specific visual evidence, ensuring precise alignment between reasoning and visual information. The collaborative mechanism of these modules can be formalized as follows: the evidence mapping module M; establishes explicit links between an agent's 's claim E C, and a visual region r in image X (specified by bounding box, mask, or description), with o E [0. 1] quantifying the confidence of this association, that is, M, C, x X - (r, o). This creates direct "claim-vise visual evidence" link. The claim parsing module P transforms an unstructured response R, into a set of structured tuples PRR)) + {c,, Jj. Cy. " where each tuple consists of e (the specific textual claim), σj (the associated confidence), ej (a description of the supporting textual evidence), and r, (a reference to the relevant visual region).
Working Mechanism of Evidence Mapping and Parsing. The evidence mapping process can be represented as a conditional mapping function from the claim space to the visual region space: p(rle, X) X) + (r, o) where p(rlc, X) represents the probability distribution over visual

---

regions r being relevant evidence given claim e and image X, and Ø is the confidence of this mapping. The claim parsing process can be formalized as text decomposition and structural reorganization:
P(R.) M. + (1
Associate({s) ). (5))mm
Assess((o)) + to
Extract( (Ra) - tey (m
Visual Region Magping
Exklesee Assciitie
Coofdrece Assooreees
Clin Extratiie
This decomposition highlights four steps: claim extraction, confidence assessment, evidence asso- ciation, and visual region mapping. collectively transforming R; into structured tuples. This can enhance transparency and evidence traceability in complex visual reasoning by grounding the visual evidence, enabling precise conflict localization, and performing fine-grate fine-grained debate focusing.
2.5 Iterative Debate Process: Dynamics, Termination, and Consensus
When initial integration reveals high uncertainty or critical disagreements, GAM-Agent initiates an erative debate process ("Dehating Uncertainty"). This structured interaction drives the agent collective towards a high-quality consensus through a feedback loop focusing on key disputes, evidence sharing, and dynamic weight adjustments. The iterative debate process is formalized as sequence of state tran- sitions. Each round of debate k involves five key steps forming a complete state evolution cycle: (a-b) Disput focus and expert argumentation: Model identifies key points of contention and guides experts to generate targeted arguments: ((k)
Identify DidentffyDisuuues(((;. Sj, Sj r2)i 1 (( U!
Arg. (M) (X, Pr, R(k-1), debate) The IdentifyDisputes function analyzes structured information from the previous round to select key claims debate with low confidence jj or significant inter- expert conflict. Each expert then generates an argumentation package including urgumentative text exp(-(tt:() exp()C!!))
((k)
(k)
) or Σ exp(>c)))) Parameters and 7 control sensitivity to uncertainty
or confidence changes. This adaptive weighting incentivizes effective argumentation, aligning with game- game-theoretic theoretic principles. (d-e) Iterative Response Generation and Uncertainty Evaluation: we integrate updated expert views to generate the response the current round and assess the consensus
Ri IntegrateJudge
P.
(R:)5-1
(Arg) (
p2 (4)
1 17-1
1 1--1
Cilbatt
(4)
Paos Rocnd Ban
Reyg
Curcel Exper Uplan
Dysanao Madgles Dippute Fiecus
In this process, the IntegrateJudge function fuses the original input Pt), the previous round's answer R((-1), each expert's initial response Rg * j = 1N, and their structured debate updates + 1., along with dynamic weights w * N and the current set of disputed points C' * debate (k) to produce the integrated decision (((), This progressive integration debate while preserving consensus from (()-1), The
Arg, (N) E/A)) (r) c(ttt
focuses on resolving the present disputes Cilk)
overall system uncertainty U- E w/ ((()) quantifies the level of consensus. Through multiple iterations, GAM-Agent aggregates agreement and resolves conflict, steadily converging to a high-confidence, low-uncertainty final answer. Debate Termination Decision Mechanism monitors multiple criteria to decide whether to continue the debate. Termination occurs as follows:
Terminate Debate
(U Ov)
(A: N KmAx)
(AU(
<
(5)
Uncertainty threshold condition
Max Steration limit
Convergence speed condition
where AU(() = (1 U(A-1)) represents the change in system uncertainty between consecutive rounds, and c is convergence threshold. These conditions correspond to reaching satisfactory certainty, resource limits, or debate stagnation. Upon termination at round K, the final response is Riinal RK), This multi-step, multi-round debate enables knowledge sharing and complementarity among agents and forms self- ieproviig cognitive loop for complex visual scenarios.
3 Experiments
3.1 Image and Video Understanding Benchmarks
Experiment Setup To comprehensively evaluate our GAM-Agent on challenging image and video benchmarks, we compare with recent multi-agent methods using various ofate-of-the-art lasion-language models. We select five prominent VLMs: Qwen2.5VL[41 (7B and 72B parameter
5

---

Table 1: The evaluation of image and video understanding. Values in parentheses denote the improvement in accuracy over the respective base model (Ori). The best performance for each base model on each benchmark is bolded. Best is in red and Base (Ori) data is in blue
Base Model
Framework
MMMU
MMBench. VII Test
MVBench, Test
Base (Ori)
53.82
82.61
69.62
DMAD
(+1.31)
85.62 (-3.01)
(+3.616
(+2.45)
86.21
Owen2.5VL-78 (Small)
ChatEval
56.98 (+3.16)
86.93 (-4.32)
74.25 (+4.63)
MAD
55.01 (+1.19)
85.11 (-2.50)
72.17 (+2.55)
DehUne
57.59 (o3.77)
87.65 (-5.04)
14.89 (65.27)
GAM-Agent (Ours)
(+6.53)
(Or)
66:24
70:38
(10.74)
(-1.49)
11.85 (+1.47)
DMPL.
69 (9.13(
90.31 92)
72. 36 980
Quen2.fVL_728 (Large)
ChatFxal
69.81 (+1.57)
90.75 (-2.36)
72.81 (+.2.43)
68.
DehUne
(=1.))))))))
91.12 (-2.73)
73. (+2.81)
GAM-Agent (Ours)
70.98 (+2,74)
91.82 (+3/43)
74.12
Bosc (Ori)
67.09
83.54
76.54
DMAD
68 (-1.74)
16.63 (-3.09)
79.78 (+3.19)
DMPL
69 00.52244 (+2.43)
$7.35 (-3.81)
80.57 (+3.98)
IntenVLL3114 (Small)
ChatEsal
70.27 (43.11)
MAD
68:25 (+1.16)
86.01 (-2.47)
79:13 (+2.54)
DehUie
70.94 (+3.85)
88.86 (-5.32)
82. (15.556
GAM-Agent (Ours)
72.26
90.15 (+6.61)
83.23 (+6.64)
Base (Ori)
72:18
$7.65
78.81
DMAD
72.99
89.21
IntemW11.3-788 args)
ChatFaal
73.88 (+1.700
90.15 (-2.50)
81.44 (+2.60) 602
MAD
72.71 (40.53)
88.83 (-1.18)
80.01 (+1.220 (0)
DehUnc
GAM-Agent (Ours)
91.53
(43.88)
(Ori)
75.18
DMAD
69 69.78 (+0.8)
84.77 1.664)
77.15 (+1.67)
DMPL
(+1.29)
85.29 (-2.16)
77.71 (+2.23)
GPT-1u-0513 (Large)
ChaiEval
70.65 (+1.68)
85.73 (-2.60)
14219 (+2.71)
36.73 (+1.25)
DebUnc
71.03 (+2.06)
86.15 (-3,02)
78.57 (+3.09)
GAM-Agent (Ours)
71.91 (+22944)
87.04 (43.91)
(9.52()44.84)
versions), Intern VL3 (14B and 78B parameter versions), and GPT-4o-0513[). On top of these base models, we implemented our GAM--geet framework and five other multi-agent frameworks: DMAD (25)) ChatEval 5 MAD (Multi-Agent Debate) (2)] and DebUnc 62). The
evaluation conducted on three challenging benchmarks: MMMU (4] (multi-discipline multimodal understanding), MMBench_ Littench_VVi__eess__ Test (visual reasoning and perception). and MVBench B (video temporal reasoning). For all experimental runs involving debate frameworks, we set Each framework utilized 3 expert agents (N = 3) for generating initial responses and argumentation, and critic agents (Nerit = 3) for frameworks requiring critique/verification. such as GAM-Agent. We used greedy decoding for text generation. Open-source models (Qwen2.5VL, Intern VL3) were deployed locally on NVIDIA A100 GPUs, while the closed-souree GPT-40-0513 accessed via the OpenRouter API. Detailed hyperparameter settings and other experimental configurations are provided in the Supplementary Material. On each benchmark, than its corresponding base model, with improvements summarized in Table [ Unless otherwise stated, all experiments in this paper are performed NVIDIA A100 GPU.
maximum of 3 debate rounds meax__dbate_rroure
GAM-Agent achieves higher overall accuracy (Overall ACC, %)
Experiment Results Table summarizes the comparative performance of GAM-Agent and other multi-agent frameworks across all base models and benchmarks. Results show that GAM- Agent consistently achieves the highest overall accuracy. Improvements are especially notable for smaller models (e.g., Qwen2.5VL-7B and InternVL3-14B), with gains of +5.5 5.8% +6.7% 6.7% on MM- Bench V11__eT and MVBench, Test (c.g., +6.61% % on InterVLL-14B//MMBccee
Larger models (Qwen2.5VL-72B, Intern VL3-78B, GPT-4o-0513) also see consistent gains of +2.6% to +4.1% (c.g.. +3.91% on GPT-40-0513/MMMM 13/MMBench), On MMMU, improvements are about 1-1.5 percentage points lower but GAM-Agent still leads, such as +5.11% for Qwen2.5VL-7B. The supplementary material provides more detailed experimental analyses, including cast-effectiveness studies (Appendix A cost dynamics during debates (Appendix [B). a comparative analysis of uncertainty handling ca- pabilities (Appendix C. empirical analysis of debate termination conditions (Appendix C.4). and additional hyperparameter studies (Appendix [F]. GAM-Agent consistently outperforms other multi- agent frameworks (DMAD, DMPL, ChatEval, MAD, DebUne), delivering larger accuracy gains

---

Table Performance on V*Bench (%). AR: Attribute Recognition. SR: Spatial Reasoning. Values in parentheses denote the accuracy improvement over the respective base model (Ori). The best performance for each base model bolded. Best overall with GAM-Agent framework is highlighted
in red Base (Ori) data is in blue
Bas Model
Framewurk
Altribute Rec. (%)
Spatial Reas. (5)
Overall (%)
(Or)
30
DMAD
59.12(+1.75)
69.03 (+1.922
63.28 (+2.02)
DMPL
59.95 (+2.59)
70.15 (+3,04)
64.21 (+2.95)
Owen2.5VL-7B (Small)
ChatEval
60.73 (+3.34)
70.92 (+3.81)
65:03 (+3.77)
DebUnc
61.54 (+4.15)
71,88 (+4.77)
65.87 (+4.61)
GAe-Ageet (Ours)
62.38 (+4.99)
72.53 (+5.42)
66.51 (+5.25)
Base (Or)
64.52
73.08
68.30
DMAD
65.88 (+1
(+1.43)
70.19 (-11.9)
DMPL
66.57 (+2.05)
75.29 (+2.21)
70.93 (+2.13)
Qwen2.5VL-728 (Large)
ChatEival
(+2.75)
MAD
65.31 (e0.79)
74,02 (+0.94)
69.67 (+0.87)
Deblic:
67.93 (+3.41)
76.54 (+3.46)
72.24 (+3,44)
GAM-Agent (Ours)
68.77 (+4.25)
77.39 (+4.31)
72.78 (+3.985
Base (Or)
61.06
69.87
65.55
DMAD
(+1.91)
(+2.2(+2205)
67.50
(+2.94)
(+2.82)
Inser (Small)
ChatEval
64.45 (+33339)
73.55 (+3.68)
69.05 (+5.5
MAD
62.24 (+1.18)
71.25 (+1.36)
66.80 (+1.27)
Debünc
65:21 (+1.15)
74.38 (+4.51)
69.80 (+1.27)
GAM-Agent (Ours)
(+5.5))
(On)
68.90
76.45
72:23
DMPL
70.63 (+1.73)
78.33 (+1.88)
14.48 (+2.25)
LstomVLL3-78B (Large)
ChatEval
71.15 (+2.25)
78.82 (+2.57)
74,99 (+2.76)
MAD
69.55
(+2.99)
(.5.0
GAM-Agent (Ours)
72.63 (+3.73)
80.24 (+3.79)
76.16 (+3.93)
(Or)
DMAD
72.33 (+11.1)
(+1.28)
15.96 (=1.224
DMPL
72.98 (+1.83)
80.29 (+1.99)
76.64 (+1.92)
GPT-46-0513
ChaiEval
(+2.36)
(+2.42)
(60.77)
(+0.75)
(+0:77)
Debtinc
74.24 (+3.009
81 (+3.19)
77.87 (43.15)
GAM-Agent (Ours)
74,98 (1-U83)
82.15 (43.85)
78.32 (43.66)
SEAL (original wook)
74.78
76.31
75.39
over base models. For instance, on Intern VI.3- 4.31MehhhMBBecch vei_Test, GAM-Agent achieves 90.15% (+6.61%) 6),, compared to DebUne's 88.86% (+5.32%) GAM-Agent's game- theoretic design, leveraging uncertainty, evidence, and iterative debate, boosts VLM performance across model sizes and complex visual tasks, especially for smaller models, with consistent gains for larger ones.
3.2 Guided Visual Search Capabilities
Experiment Setup With the release of the o3 model, MILMs have entered new phase of capability high-resolution, information-dense visual scenes, bringing attention to the challenge of fine-grained visual grounding. To evaluate such capabilities, we compare GAM-Agent and the same set comparative multi-agent frameworks (DMAD, DMPL, ChatEval, MAD, and DebUnc) to the V*Bench benchmark. We utilized the identical five base VLMs in Section B.I: Qwen2.5VL (7B and 72B), InternVL3 (14B and 78B), and GPT-4o-0513, For frameworks, including GAM- Agent, maintained the configuration of N base expert agents and Nerit 3 critic agents (when the debate is triggered), with maximum of 3 debate rounds The objective to assess how these collaborative frameworks, particularly GAM-Agent with its incertainty-driver debate, enhance performance on tasks requiring guided visual search capabilities.
(eebate_round=3)
Experiment Results. The performance of GAM-Agent and other multi-agent frameworks on the V*Bench benchmark is presented in Table 2 The table details accuracy for Attribute Recognition (AR), Spatial Reasoning (SR), and Overall scores. Base model (Ori) scores are derived from our initial evaluations, and the SEAL framework's reported performance is included reference.
The results in Table demonstrate that GAM--gent consistently enhances the performance of base VLMs on the V*Bench tasks, which demand precise visual search and reasoning. For instance, GAM-Agent improved the Qwen2.5VL-
model by +5.25% overall, achieving 66.51%. Similar significant gains are observed across other base models, such Intern VL.3-11.111 (+4.80% overall to 70.33%) and GPT-40-0513 (+3.60% overall to 78.32%). While other multi-agent frameworks also yield improvements over the base models, GAM-Agent consistently achieves the highest scores

---

06.54%
Accuracy vs. Max Debate Rounds
90
01.66
Deb. Raunds N
w
257
Accuracy (%g)
86
Acc. (%)
15.03
175
3.94
85
$0.44 9332
84
22.60%
Upper Bound 54%
seteennee Lhe 2..5%
Base
w/o N=1
w/o P&M
w/b
wo
wio
DW
Full
Base
wio N-1
w/o P&M
w/o D
w/o e
wio DW
Full
3/38
Arg Aztual benare fourds Mar beaurs Bounss
Deb. Trgg. (%))
(tokens)
3660
Actual Rounds
2152
2000
1000
1002
Avg. Actuo Rounes UpperBound 3
Bane
w/o Nw
wi PGM
D
win
win DW
Full
Bosie
wo No1
w/o P&M
wio
w/o
wio DW
Full
Max Debate Rounds
Figure 2: Acc. vs. Max Dehate Rounds Figure 3: Ablation study results on MMBench_ TEST_ V11
among them. The gains are generally more pronounced for smaller models, yet remain substantial for larger, more capable base VLMs. Notably, GAM-Agent's performance with models like GPT-40-0513 (78.32%) approaches or surpasses SEAL performance (75.39%) on this challenging benchmark.
GAM-Agent's uncertainty-driven debate mechanism shines on V*Bench enabling VLMs to quantify uncertainty. ground claims in visual evidence, and iteratively resolve disagreements for precise visual understanding. This guided visual search directs agents re-evaluate details and converge on sidence-backeer interpretations, boosting MLLMs' reasoning, and rivaling visual search.
3.3 Ablation Experiments
To assess each component's contribution, we conducted ablation studies by removing or sim- plifying modules. Experiments were run on MMBench as the base model with N for setups involving iterative debate. We report overall accuracy (Acc.), average debate rounds (Deb. Rounds), debate trigger rate (Deb. Trig.), and average LLM inference cost (Cost). Additional parameter ablations are provided in Supplementary Material We define five ablation variants to assess key components of GAM-Agent: w/o Multi-Agent (N=1): Single-agent, without collaboration or debate. w/o Uncertainty (Φ): Removes uncertainty; uses uniform weights and simple heuristics. w/o P&M: Disables claim-evidence grounding; debate operates outputs. w/o Debate (D) No iterative refinement; uses only initial integration. w/o Dynamic Weights: Debate with fixed agent weights. Reported metrics: Accuracy (Acc. %). Avg. Debate Rounds (Deb. Rounds, max 3), Debate Trigger Rate (Deb. Trig. %), and Inference Cost (tokens/instance), Experiment Results Figure shows that the full GAM-Agent achieves the highest accuracy (88.80%) with the fewest debate rounds (1.76), reflecting efficient convergence. All ablations degrade performance: removing Uncertaints or Dynamic We Weights s increases debate length and cost; disabling Iterative Debate cuts cost but reduces accuracy by 4.28%; removing P P&M causes a 4.72% drop and high debate triggers (80%). The w/o Multi -Agent setup performs only slightly above the base model, confirming the necessity of all components.
using IntemnLL3-14P agents (unless noted), and a maximum of 3 debate rounds
3.4 Study on Maximum Debate Rounds
To assess the effect of debate length, we ablated the fea__debate_roun hyperparame- ter using Qwen2.5VL-7B on the MMMBench_V11
The core GAM-Agent compo- nents uncertuncy modeling, claim parsing, and dynamic weighting- kept unchanged. We varied max debate aouundateeuuudllaaa from 0 and recorded overall accuracy and average actual debate rounds. The results presented in Figure 2 show that increasing max_de _eex___dbaeee_eeeee round improves accuracy up to 3 rounds: from 82.97% (0 rounds) to 86.21% (1), 86.35% (2), and 86.53% (3). Beyond this, accuracy plateaus around 86.52% 86.57% converging near 86.59%. Similarly, average actual debate rounds grow with higher max max_debate______aaaaaaaaaabaaee
debate sebbate_round, from 0.00 (0) to 1.00 (1), 2.21 (3), and up to 3.27 (9), showing early termination via internal convergence criteria. These results highlight that 3-4 debate rounds suffice for near-optimal accuracy with limited cost, as GAM-Agent adaptively
halts when uncertainty is resolved.
3.5 Expert Weight Trajectory and System Uncertainty Dynamics in Logical Reasoning Tasks To analyze how GAM-Agent modulates expert influence and manages uncertainty during reasoning, tracked expert contributions across eight base VLMs- SWMMs--wen2.5VL (3B-72B) and Intern VL3
8

---

DCA urcerarry
Moddl 1oums1332
Hode inaenta--0
Modal Insmrt
12 NA4V23 Nau
Oenme Roadd
Dobule Round
found
Figure 4: Evolution of Expert Weights (Relational Reasoning, Scene Description, OCR) and System Uncertainty across three debate rounds for various Qwen2.5VL and Intern VL3 series models in logical reasoning tasks. The four bars for each round represent the weights of the Relation expert, Scene expert, OCR expert, and the overall System Uncertainty, respectively.
(2B-78B). Each setup included three expert agents: Relational Reasoning, Scene Description, and OCR (see Supplementary for prompts). We visualized uncertainty- based weights and system uncertainty over three debate rounds (Figure 4. revealing how GAM-Agent dynamically adjusts expert influence to reduce uncertainty and drive consensus. Figure illustrates GAM-Agent's dynamic expert modulation during reasoning. System uncertainty (purple bars) consistently decreases over three rounds (e.g.. Qwen2.5VL-72B: 0.64++.55; VL3e7nV 0.66-+..55) indicating effective convergence. Meanwhile, OCR expert weights (yellow) typically rise (e.g.. Qwen2.5VLL VLL 72B: 0.35 -0.4),, while Relational and Scene experts adjust downward. OCR provides stable cues; expert-task alignment varies. Larger models reduce uncertainty, showing GAM-Agent's capacity- aware, uncertainty-drinn coordination.
Related Works
Multi-agent Systems (MAS) 75 24 4 8 [5] have gained significant attention in artificial intelli- gence, particularly with the integration of large language models (LLMs). These systems enhance capabilities in areas such complex reasoning, code generation, and knowledge integration by leveraging collaborative agents [TO. For example, Du et al [ demonstrated improved factuality and reasoning in LLMs through multi-agent debates, while Li et al 28 explored communicative agents to simulate language model societies [40 33 60). Despite these advances, the application of multi-agent collaboration in vision- laniuaee models (VLMs) remains largely untapped. Existing multi-agent debate frameworks often lack strategic interactions tailored for visual reasoning.
16 51 is cornerstone of reliable decision-making in machine learning, particularly visual reasoning where ambiguities, such as occlusions or poor lighting. frequently arise. Previous methods, including entropy -based measures and semantic analysis, have been widely adopted (21 30). In VLMs, uncertainty stems from both visual and linguistic sources, complicating accurate assessment [46).
Uncertainty Quantification (30 1 61
Visual Reasoning entails interpreting and inferring from visual inputs and is critical domain for VLMs 36 23 6 10 9.. Current VLM strategies often depend on single models or basic ensemble
methods, which struggle with multi-step reasoning and visual ambiguities [59] 65 35 Unlike basic averaging voting in VLM multi-agent debate, GAM-Agent introduces a game-theoretic collaboration mechanism, leveraging uncertainty and intermediate reasoning for deeper agent synergy.
Conclusion
We presented GAM-Agent, a game-theoretic multi-agent reasoning framework designed to enhance the robustness, interpretability, and accuracy of vision-language models on complex multimodal tasks. GAM--gent facilitates structured claim-evidence interactions guided by uncertainty- aware collaboration by modeling reasoning as a
sum game among specialized base agents and a critical verification agent. This design allows the system to identify ambiguities, trigger multi-roum debates, and dynamically adjust reasoning strategies based on agent confidence and disagreement. Extensive experiments across four challenging benchmarks demonstrate that GAM-Agent consistently improves performance across a wide range of VLM backbones, including both lightweight and strong foundation models. Looking forward, we believe this work opens new directions in self-reflective and

---

multi-agent multimodal reasoning. Future research may explore tighter integration with external tools, extending to dialog agents, or applying GAM-Agent to safety-critical and explainable Al systems.
Acknowledgements
This work was supported in part by the National Natural Science Foundation of China (NSFC) under Grant 62276283, in part by the China Meteorological Administration' 's Science and Technology Project under Grant CMAJBGS202517, in part by Guangdong Basic and Applied Basic Research Foundation under Grant 2023A 151501 2985, in part by Guangdong- -oon Kong-Macao Greater Bay Area Meteorological Technology Collaborative Research Project under Grant GHMA2024Z04, in part by Fundamental Research Funds for the Central Universities. Sun Yat-sen University under Grant 23hytd006, and in part by Guangdong Provincial High-Level Young Talent Program under Grant RL2024-151-2-11.
References
[1] Moloud Abdar, Farhad Pourpanah, Sadiq Hussain, Dana Rezazadegan, Li Liu, Mohammad Ghavamzadeh, Paul Fieguth, Xiaochun Cao, Abbas Khosravi, U. Rajendra Acharya, Vladimir Makarenkov, and Saeid Nahavandi. A review of uncertainty quantification in deep learning: Techniques, applications and challenges. Information Fusion, 76:243 297, December 2021.
[2] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.087744 2023.
[3] David Barrett, Felix Hill, Adam Santoro, Ari Morcos, and Timothy Lillicrap. Measuring abstract reasoning in neural networks. In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning. volume 80 of Proceedings of Machine Learning Research, pages 511 120020. PMLR, (0-15 Jul 2018.
[4] Lucian Busoniu, Radoslav Babuska, and Bart De Schutter. Multi-agent reinforcement learning: survey, 2025-05-15.
hwt.:///oo........oooooo/oooeeee /415019 2008. Accessed:
[5] Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. Chateval: Towards better 1Im-based evaluators through multi-agent debate, 2023.
[6] Fei-Long Chen, Du-Zhen Zhang, Ming-Lum Han, Xiu-Yi Chen, Jing Shi, Shuang Xu, and Bo Xu. Vlp: A survey on vision-language pre-training. Machine Intelligence Research, 20(1):38-56, January 2023.
[7] Abhishek Das, Satwik Kottur, José M. F. Moura, Stefan Lee, and Dhruv Batra. Learning cooperative visual dialog agents with deep reinforcement learning. 2017 IEEE International Conference on Computer Vision (ICCV), pages 2970-2979, 2017.
[8] Ali Dorri, Salil Kanhere, and Raja Jurdak. Multi-agent systems: A survey, IEEE Access, 6:28573-28593, 2018.
[9] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale, 2021.
[10] Yifan Du, Zikang Liu, Junyi Li, and Wayne Xin Zhao. A survey of vision-language pre-trained models, 2022.
[11] Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. Improving factuality and reasoning in language models through multiagent debate, 2023.
[12] Mohammed Elhenawy, Ahmad Abutahoun, Taqwa Alhadidi, Ahmed Jaber, Huthaifa Ashqar, Shadi Jaradat, Ahmed Abdelhay, Sebastien Glaser, and Andry Rakotonirainy. Visual reasoning and multi-agent approach in multimodal large language models (mllms): Solving tsp and mtsp combinatorial challenges, 2024.
10

---

[13] Ghada M. Elshamy, Marco Alfonse, and Mostafa M. Aref. A guesswhat?! game for goal- oriented visual dialog: A survey. In 2021 Tenth International Conference on Intelligent Computing and Information Systems (ICICIS), pages 116-123, 2021.
[14] Seth Frey, Jules Hedges, Joshua Tan, and Philipp Zahn. Composing games into complex institutions, 2023.
[15] Zijian Gao, Kele Xu, Bo Ding, and Huaimin Wang. Knowru: Knowledge reuse via knowledge distillation in multi-agent reinforcement learning. Entropy, 23(8):1043, August 2021.
[16] Jakob Gawlikowski, Cedrique Rovile Njieutcheu Tassi, Mohsin Ali, Jongseok Lee, Matthias Humt, Jianxiang Feng. Anna Kruspe, Rudolph Triebel, Peter Jung, Ribana Roscher, Muhammad Shahzad, Wen Yang. Richard Bamler, and Xiao Xiang Zhu. A survey of uncertainty in deep neural networks, 2022.
[17] Simon Ging, Mohammadreza Zolfaghari, Hamed Pirsiavash, and Thomas Brox. Coot: Coopera- tive hierarchical transformer for video-text representation learning, 2020.
[18] Sven Gronauer and Klaus Diepold. Multi-agent deep reinforcement learning: a survey, Artif. Intell. Rev., 55(2):895-943, February 2022.
[19] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Iims via reinforcement learning. arXiv preprint arXiv:2501. 2025.
[20] Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor, 2018.
[21] Frank Heinemann, Rosemarie Nagel, and Peter Ockenfels. Measuring strategic uncertainty in coordination games. The Review of Economic Studies, (6(1):181-22)) 2009.
[22] Jen-Hao Hsiao and Kwang-Chent Cheng Chen. Communication methodology to control distributed multi-agent system. ICC 2019- 2019 IEEE International Conference on Communications (ICC), pages 1-6, 2019.
[23] Drew A. Hudson and Christopher D. Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6693 2019.
[24] Dom Huh and Prasant Mohapatra. Multi-agent reinforcement learning: A comprehensive survey, 2024.
[25] Akbir Khan, John Hughes, Dan Valentine, Laura Ruis, Kshitij Sachan, Ansh Radhakrishnan, Edward Grefenstette, Samuel R. Bowman, Tim Rocktäschel, and Ethan Perez. Debating with more persuasive Ilms leads to more truthful answers. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. JMLR.org. 2024.
[26] Mathieu Laurière, Sarah Perrin, Julien Pérolat, Sertan Girgin, Paul Muller, Romuald Élie, Matthieu Geist, and Olivier Pietquin. Learning in mean field games: A survey, 2024.
[27] Kenton Lee, Mandar Joshi, Julia Ture, Hexiang Hu, Fangyu Liu, Julian Eisenschlos, Urvashi Khandelwal, Peter Shaw, Ming- Wei Chang, and Kristina Toutanova. Pix2struct: Screenshot parsing as pretraining for visual language understanding, 2023.
[28] Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Communicative agents for "mind" exploration of large language model society, 2023.
[29] Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang. Shuming Shi, and Zhaopeng Tu. Encouraging divergent thinking in large language models through multi-agent debate. Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17889-17904, Miami, Florida, USA, November 2024. Association for Computational Linguistics.
11

---

[30] Zhen Lin, Shubhendu Trivedi, and Jimeng Sun. Generating with confidence: Uncertainty quantification for black-box large language models, 2024.
[31] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19453 19437, 2024.
[32] Yexiang Liu, Jie Cao, Zekun Li, Ran He, and Tieniu Tan. Breaking mental set to improve reasoning through diverse multi-agent debate. In The Thirteenth International Conference on Learning Representations, 2025.
[33] Yikang Liu, Ziyin Zhang, Wanyang Zhang, Shisen Yue, Xiaojing Zhao, Xinyuan Cheng, Yiwen Zhang, and Hai Hu. Argugpt: evaluating, understanding and identifying argumentative essays generated by gpt models, 2023.
[34] Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, Kai Chen, and Dahua Lin. Mmbench: Is your multi-modal model an all-around player?, 2024.
[35] Guiyang Luo, Hui Zhang, Haibo He, Jinglin Li, and Fei- Yue Wang. Multiagent adversarial collaborative learning via mean-field theory. IEEE Transactions on Cybernetics, 51((0):4994- 5007, 2021.
[36] Anand Mishra, Shashank Shekhar, Ajeet Kumar Singh, and Anirban Chakraborty. Ocr- Visual question answering by reading text in images. In 2019 International Conference on Document Analysis and Recognition (ICDAR), pages 947-952, 2019.
[37] John Nash. Equilibrium points in n-person games, 1950.
[38] Masanao Ozawa. Heisenberg's original derivation of the uncertainty principle and its universally valid reformulations, 2020.
[39] Joon Sung Park, Joseph C. O'Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. Generative agents: Interactive simulacra of human behavior, 2023.
[40] Joon Sung Park, Carolyn Q. Zou, Aaron Shaw, Benjamin Mako Hill, Carrie Cai, Meredith Ringel Morris, Robb Willer, Percy Liang, and Michael S. Bernstein. Generative agent simulations of 1,000 people, 2024.
[41] Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang. Jianxin Yang, Jiaxi Yang. Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report, 2025.
[42] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021.
[43] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and llya Sutskever. Language models are unsupervised multitask learners. OpenAl Blog, 188):9, 2019.
[44] Rahul Rahaman and Alexandre H. Thiery. Uncertainty quantification and deep ensembles. In Proceedings of the 35th International Conference on Neural Information Processing Systems, NIPS '21, Red Hook, NY, USA, 2021. Curran Associates Inc.
[45] James K. Rilling and Alan G. Sanfey. Social decision-making insights from game theory and neuroscience. Trends in cognitive sciences, 15(5):216-224, 2011.
[46] Leonard Salewski, Stephan Alaniz, Isabel Rio-Torto, Eric Schulz, and Zeynep Akata. In-context impersonation reveals large language models' strengths and biases. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS '23, Red Hook, NY, USA, 2023. Curran Associates Inc.
12

---

[47] Alan G. Sanfey, James K. Rilling, Jennifer A, Aronson, Leigh E. Nystrom, and Jonathan D. Cohen. The neural basis of economic decision-makee making in the ultimatum game. Science, 3005562))11555-1758 2003.
[48] Jinzhou Tang, Jusheng Zhang, Qinhan Lv, Sidi Liu, Jing Yang. Chengpei Tang, and Keze Wang. Hiva: Self-organized hierarchical variable agent via goal-driven semantic-tooolggien evolution, 2025.
[49] Gemini Team. Gemini: A family of highly capable multimodal models, 2025.
[50] Hugo Touvron and Louis Martin. Llama 2: Open foundation and fine-tuned chat models, 2023.
[51] Khanh-Tung Tran, Dung Dao, Minh-Duong Nguyen, Quoc-Viet Viet Pham, Barry O' Sullivan, and Hoang D. Nguyen. Multi-agent collaboration mechanisms: A survey of Ilms, 2025.
[52] Wen-Kwang Tsao. Multi-agent reasoning with large language models for effective corporate planning. In 2023 International Conference on Computational Science and Computational Intelligence (CSCI), pages 365-370, 2023.
[53] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2023.
[54] Jin Wang, Shichao Dong, Yapeng Zhu, Kelu Yao, Weidong Zhao, Chao Li, and Ping Luo. Diagnosing the compositional knowledge of vision language models from a game-theoretic view, 2024.
[55] Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Jirong Wen. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18(6), March 2024.
[56) Qineng Wang, Zihao Wang, Ying Su, Hanghang Tong, and Yangqiu Song. Rethinking the bounds L.MM reasoning: Are multi-agent discussions the key? Jei-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6106-6131, Bangkok, Thailand, August 2024. Association for Computational Linguistics.
[57] Shenzhi Wang, Chang Liu, Zilong Zheng, Siyuan Qi. Shuo Chen, Qisen Yang, Andrew Zhao, Chaofei Wang, Shiji Song, and Gao Huang. Avalon's 's game of thoughts: Battle against deception through recursive contemplation, 2023.
[58] X. J. Wang, Y. Q. Li, Z. Y. Chen, et al. Neurocomputationall mechanisms involved in adaptation to fluctuating intentions of others. Nature Communications, 15(1):1234, 2024.
[59] Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Seff-consistency improves chain of thought reasoning in language models, 2023.
[60] Y. Wang, J. Gao, H. Li, et al. Large language models empowered agent-based modeling and simulation: survey and perspectives, 2024.
[61] Fanghua Ye, Mingming Yang, Jianhui Pang, Longyue Wang, Derek F. Wong, Emine Yilmaz, Shuming Shi, and Zhaopeng Tu. Benchmarking llms via uncertainty quantification, 2024.
[62] Luke Yoffe, Alfonso Amayuelas, and William Yang Wang. Debunc: Improving large language model agent communication with uncertainty metrics, 2025.
[63] Chao Yu, Xinyi Yang, Jiaxuan Gao, Huazhong Yang, Yu Wang, and Yi Wu. Learning efficient multi-agent cooperative visual exploration, 2022.
[64] Xiang Yue, Yuansheng Ni, Tianyu Zheng, Kai Zhang, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang. Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. 2024 IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 91356-956,,000
13

---

[65] Andy Zeng, Maria Attarian, Brian Ichter, Krzysztof Choromanski, Adrian Wong, Stefan Welker, Federico Tombari, Aveek Purohit, Michael Ryoo, Vikas Sindhwani, Johnny Lee, Vincent Vanhoucke, and Pete Florence. Socratic models: Composing zero-shot multimodal reasoning with language, 2022.
[66] Chongjie Zhang and Victor Lesser. Multi-agent learning with policy prediction. In Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence, AAAI' 10, page 927-934. AAAI Press, 2010.
[67] Jusheng Zhang, Kaitong Cai, Yijia Fan, Ningyuan Liu, and Keze Wang. Mat-agent: Adaptive multi-agent training optimization, 2025.
[68] Jusheng Zhang, Kaitong Cai, Yijia Fan, Jian Wang, and Keze Wang. CFvilmccuntterfactuall vision-language fine-tuning, 2025.
[69] Jusheng Zhang. Kaitong Cai, Xiaoyang Guo, Sidi Liu, Qinhan Lv, Ruiqi Chen, Jing Yang, Yijia Fan, Xiaofei Sun, Jian Wang. Ziliang Chen, Liang Lin, and Keze Wang. Mm-cot:a benchmark for probing visual chain-of-thought reasoning in multimodal models, 2025.
[70] Jusheng Zhang, Kaitong Cai, Jing Yang, and Keze Wang. Learning dynamics of vlm finetuning, 2025.
[71] Jusheng Zhang, Kaitong Cai, Qinglin Zeng, Ningyuan Liu, Stephen Fan, Ziliang Chen, and Keze Wang. Failure-driven workflow refinement, 2025.
[72] Jusheng Zhang, Yijia Fan, Kaitong Cai, Zimeng Huang, Xiaofei Sun, Jian Wang, Chengpei Tang. and Keze Wang. Drdiff: Dynamic routing diffusion with hierarchical attention for breaking the ffficiency-qualit trade-off, 2025.
[73] Jusheng Zhang, Yijia Fan, Kaitong Cai, Jing Yang, Jiawei Yao, Jian Wang, Guanlong Qu, Ziliang Chen, and Keze Wang. Why keep your doubts to yourself? trading visual uncertainties in multi-agent bandit systems, 2026.
[74] Jusheng Zhang, Zimeng Huang, Yijia Fan, Ningyuan Liu, Mingyan Li, Zhuojie Yang. Jiawei Yao, Jian Wang, and Keze Wang. KABB: Knowledge-award bayesian bandits for dynamic expert coordination in multi-agent systems. In Forty-second International Conference on Machine Learning, 2025.
[75] Kaiqing Zhang, Zhuoran Yang, and Tamer Başar. Multi-agent reinforcement learning: A selective overview of theories and algorithms, 2021.
[76] Jinguo Zhu, Weiyun Wang. Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, Zhangwei Gao, Erfei Cui, Xuehui Wang. Yue Cao, Yangzhou Liu, Xingguang Wei, Hongjie Zhang, Haomin Wang, Weiye Xu, Hao Jiahao Wang, Nianchen Deng, Songze Li, Yinan He, Tan Jiang. Jiapeng Luo, Yi Wang. Conghui He, Botian Shi, Xingcheng Zhang, Wenqi Shao, Junjun He, Yingtong Xiong, Wenwen Qu, Peng Sun, Penglong Jiao, Han Lv, Lijun Wu, Kaipeng Zhang, Huipeng Deng, Jiaye Ge, Kai Chen, Limin Wang, Min Dou, Lewei Lu, Xizhou Zhu, Tong Lu, Dahua Lin, Yu Qiao, Jifeng Dai, and Wenhai Wang. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models, 2025.
14

---

Supplementary Material
Supplementary Material Overview
The supplementary material provides detailed experimental setups, results, theoretical analyses, and case studies to support the main paper's claims on the GAM-Agent framework. includes cost-performance analyses, uncertainty handling evaluations, theoretical derivations, hyperparameter studies, prompt configurations, and case analyses to demonstrate the framework's 's robustness and limitations.
Contents
Introduction
2
Methodology
Base and Critical Agents
2.2
Uncertainty Quantification Function (Φ)
2.2.1
Semantic-Marker Based Uncertainty ( isem
2.3
Initial Integration and Conflict Detection)
2.4
Evidence Mapping (M) and Claim Parsing (P)
2.5
Iterative Debate Process: Dynamics, Termination, and Consensus
3
Experiments
Image and Video Understanding Benchmarks
3.2
Guided Visual Search Capabilities
3.3
Ablation Experiments
8
3.4
Study Maximum Debate Rounds
3.5
Expert Weight Trajectory and System Uncertainty Dynamics Logical Reasoning
Tasks
Related Works
Conclusion
Supplementary Material Overview
15
Cost and Performance Balance Analysis
17
A.I
Experiment Setup
17
A.2 Experiment Result
17
Experiment on Cost Dynamics During Round Debates
18
I Experimental Setup
18
B.2
Experimental Results
18
Deepening Comparative Analysis of Multi-Agent Methods: Uncertainty Handling Capa-
bility
19
15

---

Comprehensive Analysis of Uncertainty Handling Capability
19
Experimental Setup
19
Experimental Results Analysis
20
Experimental Analysis
21
C.S
Theoretical Analysis of Termination Threshold U,ter in Multi- Debate
23
Derivation of Uncertainty Theory
23
DJ
Mathematical Formalization of the Non-Zero-Sum Game
24
D.2
Utility Function Design
25
D.3
Exploration of Nash Equilibrium Existence
26
D.4
Cooperative Game Objective and Dynamic Weighting Protoco
28
Thoorem
Fundamental Game Theoretic Framework of GAM-Agent
Theorem II: Comparative Analysis with the DMAD Framework
34
Theorem III: Differences Mathematical Construction between GAM--gent and
Traditional Multi-Agent Methods
35
Hyperparameter Ablation
38
Hyperparameter Setting
39
Settings for Image Understanding Experiments (MMBench and MMMU)
40
[.1.1 Owen2.5VL Series (3B, 7B, 32B, 72B) and Intern VL.3 Series (2B, 8B, 14B,
38B, 78B)
40
.1.2
GPT-4o-0513
40
Settings for Video Understanding Experiments (M Bench)
41
I.2.1
Qwen2.5VL Series (3B, 7B, 32B. 72B), Intern VL3 Series (2B, 8B, 38B).
and Intern Video2.5
I.2.2
GPT-4-00533 (for MVBench)
41
L3
Settings for Comparison with Existing Multi- Methods (MMBench)
41
Settings for Ablation Studies Base Configuration (MMBench. TEST_VIL
42
1.5
Settings for Study on Maximum Debate Rounds (MMBench VII Test)
42
1.6
Settings for Expert Weight Trajectory Analysis (Logical Reasoning Tasks)
42
J
Prompt Setting Statement
43
General Prompts for Expert Roles and the Aggregator
43
J.2
Prompts for VLM Experts in Benchmark Evaluations
43
J.2.1 Base Expert Prompts
44
J.2.2 Critic Expert Prompts
45
K Case Analysis
46
K. Successful Case
46
2
Unsuccessful Case
46
16

---

A Cost and Performance Balance Analysis
A.1 Experiment Setup
To evaluate the cost-performance balance of our proposed approach, we conducted a comparative analysis on the MMbench_ Tisihh__TT_____ V11 benchmark. We measured the "Cost per Instruction on MMbench_TEST. "11"" and "Overall ACC (%)" for several leading Vision-Language Models (VLMs) and configurations derived from our framework (denoted as "Ours"). The commercial models benchmarked include [wen2.5VL[4[] series (7B. 72B via APD), Gemini 2.5 Pro Preview [49], Gemini 2.0 Pro. GPT-44(2). and Claude 3.7, with their respective costs per instruction sourced from OpenRouter API pricing. For our configurations, leveraging models such as Qwen2.5VL (7B, 72B) and Intern VL3 (14B, (76) the cost per instruction was calculated based the inference time on a local NVIDIA A100 GPU. This local GPU cost was estimated using the on-demand price of an AWS p4d.24. 14xlaee instance, which includes 8 A 100 40GB GPUs approximately $32.77 per hour, translating to about $4.10 10 per hour for a single A 100 GPU. This setup allows for a direct comparison of the economic efficiency of deploying VLMs via commercial APIS versus utilizing them within our optimized framework on local hardware.
A.2 Experiment Result
The results of our cost and performance comparison the MMbench TEST V11 benchmark are summarized in Figure 5 The experimental results presented in Figure 5 reveal significant disparities in cost-effectiveness across the evaluated models. Notably, our framework 's configurations demonstrate exceptional performance with compelling cost benefits. VnternVL33788 (Ours) achieved the highest accuracy of 92.2% with competitive cost of approximately $0.00022 per instruction. Closely following, Qwen2.5VL-728 (Ours) (one configuration) recorded 92.0% accuracy at a cost of $0.00020 per instruction. Interestingly, another configuration of Qwen2.5VLL-2BB deployed with our framework (labeled as "Qwen2.5VLL-72" accuracy at an even lower cost of $0.00011 per instruction. This particular configuration not only surpasses the Qwen2.5VLL72B API (88.4% ACC at $0.00015) in accuracy but is also approximately 1.36 times more cost-effective. Other configurations, such as Intern VL3-14B (Ours), delivered 91.4% accuracy at $0.00016 per instruction, while Qwen2.5VL-7B (Ours) achieved a strong 89.0% accuracy at cost of $0.000055 per instruction. This is notable improvement in accuracy over its API counterpart (Qwen2.5VL-7B API, 82.6% ACC at $0.00007), albeit at slightly higher local inference cost.
on the "Ours" trend line in Figure 5) achieved 90.4%
While Gemini 2.5 Pro Preview (API) ex- hibited high accuracy (89.3%), its API cost ($0.00075) is considerably higher than our top-perfore oop-performing local configurations. For in- stance, ILterBVL--78B (Ours) and the higher-
Cost vs Accuracy with flaating Trend Line (Ours Only) -yw.iiiuuu
performing Qwen2.5VL-72B (Ours) achieve su- perior accuracy (92.2% and 92.0%, respectively)
and are approximately 3.4 to 3.75 times less ex-
pensive than Gemini 2.5 Pro Preview. In con- trast, widely used models like GPT-4o (API) and Claude 3.7 (API), despite their capabilities, in- curred substantially higher costs per instruction ($0.015 015 and $0.018, respectively) for accuracies of 84.3% and 81.2% on this benchmark, accu- racies which are surpassed by several of our framework's 's configurations at a fraction of the cost.
OLD SA
Figure 5: Comparison of Cost per Instruction and Overall Accuracy on MMbench TEST_ETT_VI "Ours" denotes models run with our proposed framework and local inference cost calculation.
This analysis underscores the substantial eco- nomic advantages of leveraging optimized local inference strategies for state-of of-the-art open-source VLMs. The results indicate that our approach can achieve or even surpass the accuracy of expensive commercial API-based services at significantly reduced costs. For example, our Intern VI.3- (Ours) configuration not only outperforms Gemini 2.5 Pro Preview in accuracy (92.2% vs. 89.3%) but does so at approximately 1/3.4th of the cost per instruction. Similarly, the Qwen2.5VL-72B
17

---

(Ours) configuration achieving 92.0% accuracy does so at about 1/3.75th of the cost of Gemini 2.5 Pro Preview, while also delivering higher accuracy. This highlights pathway to democratize access to high-performance VLM capabilities, making advanced visual reasoning more accessible and economically viable. The significant cost reductions and performance enhancements observed for models like Qwen2.5VL. and Intern VI.3 when run under our framework suggest that efficient utilization of local compute resources can unlock dramatic improvements in cost-efficiency without compromising.
B Experiment on Cost Dynamics During Multi-Round Debates
To investigate the economic implications of iterative debate, particularly how computational costs evolve across multiple rounds, we conducted a dedicated experiment. This analysis aims to compare the cumulative cost of GAM-Agent framework against a baseline multi-agent debate scenario where original models engage in self-correction or iterative refinement without GAM-Agent's struc- tured uncertainty-driven mechanisms. Understanding these cost dynamics is crucial for assessing the practical viability and efficiency of multi-round debate strategies.
B.1 Experimental Setup
The experiments were performed on the MMBench, VI1 Test benchmark. We selected four promi- nent open-source Vision-Language Models (VLMs) base models: Qwen2.5VL-7B, Qwen2.5VL- 72B, IntemVLL-144B, and VL3-7BB
We compared two primary configurations:
Our proposed GAM-Agent framework was applied using each of the selected base VLMs. The setup involved N expert agents and Nerit = 3 critic agents, consistent with other experiments.
GAM--geent Framework:
2. Baseline Multi Muuti-Round Self-Debate (Base-Debate): For this configuration, we simulated multi-round debate scenario where instances of the original base VLM were prompted to iteratively refine their answers based on their previous outputs or simple peer feedback, without the structured uncertainty guantification, evidence mapping, or dynamic weighting of GAM-Agent. This represents more direct or naive approach to multi-round refinement.
For both configurations, we tracked the cumulative computational cost after 1, 2, and 3 rounds of debate. The "cost" quantified by the average number of tokens processed per instance (input prompt tokens + generated output tokens) at the conclusion each specified round. This metric serves proxy for both local GPU computation time and potential API call expenses. The primary goal of this experiment is to elucidate how GAM-Agent manages cost accumulation over successive debate rounds compared to less structured iterative debate, and to highlight the efficiency benefits derived from its consensus mechanism which aims for early convergence.
B.2 Experimental Results
The average cumulative costs (in tokens per instance) for the GAM-Agent framework and the Baseline Multi-Round Self-Debate (Base-Debate) configuration across 1, 2, and 3 rounds of debate are presented Figure G
Figure 6 illustrates that while costs for GAM-Agent increase with more debate rounds, this is accompanied by significant accuracy improvements that consistently outperform the single-agent baseline. For instance, with Intern VL3-78B as the base, GAM-Agent's 's average cost per instruction after rounds $0.00033, which point it achieves an accuracy of 91.90%. In contrast, the Single-agent (Base-Debate configuration with Intern VL3-78B reaches an accuracy of 90.20% after rounds. GAM-Agent achieves superior accuracy (e.g.. +1.7 percentage points for Intern VL3. 78B after 3 rounds) for a defined number of debate cycles. This trend is consistent across other models. With Qwen2.5VL-72B. GAM-Agent achieves 91.82% accuracy at a cost $0.00045 after rounds, while the single-agent baseline reaches 90.20%. For Qwen2.5VL-7B, GAM-Agent reaches 89.02% accuracy for $0.00009 after 3 rounds, compared the baseline's 86.50% Similarly, for IntervV3--1m GAM-Agent achieves 90.15% for $0.00017 after 3 rounds, surpassing the bascline's
18

---

Melagers
Neltvant Mart
Miltpa
Figure 6: Average cumulative cost (tokens per instance) on MMBench _ei_Test after 1, 2, and rounds of debate for GAM-Agent (utilizing different base models) versus Baseline Multi-Ron mouni Self-Debate (Base-Debate).
88.50%. The GAM-Agent's structured debate, driven by uncertainty and conflict detection, aims to terminate dehates earlier if consensus reached, contributing to an optimized balance between cost and performance. While the Baseline Self-Debate might also incur increasing costs with more rounds, consistently achieves lower accuracy. This suggests that to reach the accuracy levels of GAM- Agent, the baseline would likely require even more rounds, leading to significantly higher cumulative costs, or may not reach such accuracy levels at all within a comparable number of iterations. Thus, only improve accuracy but also offer a more efficient pathway to
GAM-Agent's mechanisms achieving high performance in multi-round visual reasoning by strategically focusing the debate and adaptively managing its length and cost.
C Deepening Comparative Analysis of Multi-Agent Methods: Uncertainty Handling Capability
To comprehensively evaluate the uncertainty handling capabilities of our proposed GAM-Agent framework across various base models, we extended our analysis to include all major models from the Qwen2.5VL series (3B to 72B) and the VLternVL3 series (2B to 78B). This allows us observe the relationship between uncertainty processing capabilities and model scale, as well as the performance of GAM-Agent across different architectures.
C.1 Comprehensive Analysis of Uncertainty Handling Capability
We continue to use three key metrics to evaluate uncertainty-hannling capabilities:
Uncertainty Accuracy (UAf): GAM-Agent's ability to correctly identify instances of high uncertainty (higher is better).
Calibration Error (CE.): The consistency between the model's predicted confidence and its actual accuracy (lower is better).
Dynamic Adaptability (DA1): The ability to adjust the decision-making process based on uncertainty (higher better).
F Qwen2.5 VL series) and (Intern VL3 series) show the performance of different multi-agent methods on these metrics and their MMBench accuracy.
C.2 Experimental Setup
The evaluation of uncertainty handling capabilities (UA, CE, DA) conducted on the MMBench(34] dataset, which is also used for reporting the final accuracy. The primary moti- vation for this extended analysis is to systematically assess how different multi-agent frameworks, including our GAM-Agent, manage and leverage uncertainty, and how these capabilities scale with the underlying VLM's size and architecture.
For each base model the Qwen2.5VL. series (3B, 7B, 32B, 72B) and Intern series (2B, 8B, 38B, 78B), we applied GAM-Agent and the baseline multi-agent methods (DMAD, DMPI, ChatEval, MAD, DebUnc). The baseline methods were adapted to the visual question answering task on MMBench The metries were measured as follows:
19

---

Uncertainty Handling Performance on Qwen2.5VL and InterVLL3 Series
Score
Score
6.9
0.8
6.6
0.5
Creent SW 38
Intemi 180
Intemvt I-te
Greent Sie. 38
Gonent Sie. 78
Intem 8:
Intemit Jap
Intemit D- 700
Crmenz SM.
Ownnt St
Oweet SV
Intemii
Inten
Owinz 3V,
Intertl
Oweet
OF Score (4)
MMBench Accurary (8)
0.15
90
0.10
85
0.05
quent Sva.
coand SVt.
Ownn SV. 320
OwenZ 728
Internit..j .0
Paam ag
Witemii 3
InternLS 770
Qwent Stt-30
Qwent Sit-
Owenz 52L320
Owent SVt 720
Intemit a
Intervit a
Intemit. 380
Intemii3- 760
D0P
M00
AVerEVEN
Figure 7: Comparison of uncertainty handling performance for various methods on the Qwen2.5VL and Intern VL3 model series. UA, DA higher is better; CE lower is better.
Uncertainty Accuracy (UA): This metric was evaluated by defining a threshold for high system-level uncertainty (e.g., based entropy semantic cues from the aggregated response). We then assessed the proportion of instances correctly flagged as uncertain (e.g., where the model's prediction against a ground truth or heuristic for uncertainty.
incorrect or the question was inherently ambiguous)
Calibration Error (CE): We utilized an Expected Calibration Error (ECE) variant. Model responses were binned by their expressed confidence scores (derived from uncertainty metrics like U sys). The ECE was then calculated as the weighted average difference between the mean confidence and observed accuracy within each bin.
This was qualitatively and quantitatively assessed by ob- serving whether the framework modified its reasoning process resource allocation (e.g.. triggering debate rounds in GAM-Agent, or adjusting agent weighting) in response to detected uncertainty, and whether such adaptations led to improved outcomes more cautious predictions in uncertain scenarios. For quantitative proxy, we measured the corre- lation between VLM's internal uncertainty signals and its engagement of deeper reasoning mechanisms or confidence adjustments.
Dynamic Adaptability (DA):
The core idea is to investigate whether sophisticated uncertainty modeling, as implemented in GAM- Agent, translates into more reliable and robust performance, especially as model complexity and task difficulty increase.
C.3
Experimental Results and Analysis
The results presented in Figure 7 offer a detailed view of the uncertainty handling capabilities of different multi-agent frameworks.
GAM-Agent Consistently Excels in Uncertainty Management: Across both the Qwen2.5VI and Intern VL3 series, and for all model sizes, GAM-Agent consistently outperforms the baseline methods (DMAD, DMPL, ChalEval, MAD, DebUne) on all three uncertainty metrics: Uncertainty Accuracy (UA), Calibration Error (CE), and Dynamic Adaptability (DA). For example, with Qwen2.5VI. 72B, GAM-Agent achieves UA of 0.82, CE of 0.04, and DA of 0.76, significantly surpassing other approaches. A similar trend is observed with Intern VL3-78B, where GAM-Agent scores 0.84
20

---

(UA), 0.03 (CE), and 0.78 (DA). This suggests that GAM-Agent's 's explicit uncertainty quantification, evidence mapping, and mncertainty-driver debate mechanisms are highly effective.
Scalability with Model Size: key observation is the positive scaling of GAM-Agent's uncertainty handling capabilities with increasing model size. For the Qwen2.5V1. series, the model size increases from 3B to 72B, GAM-Agent's ' UA improves from 0.72 to 0.82, CE reduces from 0.08 to 0.04, and DA increases from 0.65 to 0.76. Similarly, for the Intern VL3 series (2B to 78B), GAM- Agent's UA rises from 0.70 to 0.84, CE drops from 0.09 to 0.03, and DA improves from 0.63 to 0.78. This indicates that larger base models, when integrated into the GAM-Ageet framework, not only provide better raw capabilities but also become more adept at identifying, calibrating, and adapting to uncertainty. While baseline methods also show some improvement with model size, their gains in uncertainty metrics are generally less pronounced than those achieved by GAM-Agent,
Correlation with MMBench Accuracy: The data strongly suggests positive correlation between superior uncertainty handling and higher overall task performance on MMBench. GAM-Agent, which consistently leads in UA, CE, and DA, also achieves the highest MMBench accuracy for each respective base model. For instance, GAM-Agent with Qwen2.5VL-722 not only has the best uncertainty scores but also the top MMBench accuracy of 90.56% in Figure 7 This underscores the importance of robust uncertainty management for achieving reliable and accurate visual reasoning. Frameworks that can better understand their own limitations and adapt accordingly are more likely to succeed in complex tasks.
Performance of Baseline Methods: The baseline methods exhibit varied performance in uncertainty handling. DehUnc, which also focuses on uncertainty, generally performs second best to GAM-Agem uncertainty metrics but still lags significantly. Other methods like DMAD, DMPL, ChatEval, and MAD show more modest capabilities in these specific uncertainty-related assessments. Their mechanisms for collaboration may not explicitly or effectively propagate and utilize fine-grained uncertainty signals to the same extent as GAM-Agent.
In conclusion, this expanded comparative analysis demonstrates GAM-Agent's superior and scalable cegrtaiint-handdlinn capabilities. Its architecture, designed to explicitly model and debate uncertainty, enables it to achieve better calibration, accurate identification of challenging instances, and more effective adaptation of its reasoning processes. These strengths in managing uncertainty are critical contributors to its leading performance on complex visual reasoning benchmarks like MMBench.
C.4 Experimental Analysis
The experimental results strongly confirm the efffectiveness and robustness of the proposed debate termination conditions. As shown in Figure (your Figure 1, which you indicated is 'boyil. Ipd"". all tested models exhibited a consistent accuracy distribution pattern across the various tasks in MMBench. Specifically. large-scale models (Qwen2.5VL-722) formed small-scale models (Qwen2.5VIL. models of similar scale, the Intern VL.3 series demonstrated a slight performance advantage over the Qwen2.5VL. series. Regarding task types, text understanding tasks typically achieved the highest accuracy, while relational reasoning problems proved to be relatively more challenging.
VLternVL3-780 generally outper- Intern VL.3-14B) in terms of accuracy. When comparing
21

---

Accuracy Comparison Types
100
Model
0men2.5VL-78 SVL-78
Gwenz SVL-228
95
Intervi33 148
InterviL 788
Accuraty (1)
90
85
Figure Performance and convergence dynamics under default termination conditions. This figure shows model accuracy across MMBench tasks, general performance trends, and key debate convergence statistics like average rounds and primary termination reasons.
Key experimental findings are summarized Delow:Natural Convergence and Efficiency: Under the default termination conditions, high proportion of samples, ranging from 88% to 94%, achieved natural convergence before reaching the preset maximum number of debate rounds (Kmax 10). On average, GAM-Agent required only 2.1 to 2.6 debate rounds converge, with large-scale models exhibiting faster convergence. Distribution of Termination Reasons: Triggering of the uncertainty
threshold (i.e., system uncertainty U (,,ter)) was the primary reason for debate termination, accounting for 63% to 72% of cases. The convergence speed threshold (i.c., change in system
uncertainty AU sys to 25% terminations. Only a minority of instances terminated due to reaching the maximum number of debate rounds. Bask-Dependent Convergence Speed: The convergence speed varied across different task types. Text understanding problems typically converged the fastest, requiring an average of only 1.7 2.1 debate rounds. In contrast, relational reasoning problems, owing their complexity, converged relatively slowest, needing an average of 2.6 to 3.2 debate rounds. Figure (your Figure 2) further illustrates the impact of the uncertainty termination threshold fy,term on model accuracy and the average number of debate rounds.
e) the second most common reason, responsible for approximately 22%
Accuracy vs. Uncertainty Threshold
95.0
92.5
90.0
Accuracy (%)
Model
Qwen2.5VL-78
Qwer2.5VL-729
InternVL33114
internVL3-78B
0.10
0.12
0.14
0.18
0.20
Uncortainty Threshold (8. (.rmm)
Figure 9: Sensitivity analysis of the uncertainty termination threshold (fu,term). This figure illustrates the trade-off between prediction accuracy and average debate rounds when varying fu (0.10, 0. 0.15, 0.20), with fytterm 0.15 identified an optimal balance.
22

---

When tu.term was set to 0.10 (a stricter convergence requirement), there was a slight improvement in aceuracy (approximately +0.2% to +0.4%), but the average number of debate rounds increased significantly (by about 25% to 31%) )), leading to higher computational costs. Conversely, when tetterm was set to 0.20 (a looser convergence requirement,, the average number of debate rounds decreased (by about 22% to 26%). but accuracy also declined (by approximately -0.9% to - -1.2%). The experimental data indicate that all tested models achieve an optimal balance between prediction accuracy and computational efficiency at the default setting of dU,trrm 0.15. Notably, large-scale models (72B/78B parameter scale) demonstrated lower sensitivity to variations in the Uuteerm parameter, suggesting greater robustness under different termination conditions. In summary, these experimental results clearly demonstrate that our designed debate termination mechanism can effectively balance prediction accuracy and computational efficiency across vision language models of different scales and architectures. This provides reliable parameter setting guidance and performance expectations for the practical deployment of the GAM-Agent framework.
C,5 Theoretical Analysis of Termination Threshold futerrm in Multi-Agent Debate
The termination criterion in GAM-Agent's multi-agent debate framework plays a pivotal role in balancing system performance and computational efficiency. Drawing on our main experimental observations. this section provides a theoretical dissection of how the core uncertainty threshold parameter, futterm influences the debate process. Three principal conditions govern the debate
termination mechanism (see Section 1 and Equation 5): (1) the system uncertainty U.4 drops below the pre-set threshold Outerrmi (2) the debate reaches the maximum number of allowed iterations
Kmans (3) the change in system uncertainty between consecutive rounds | falls below a convergence threshold c. Among these, BU,term theoretically defines the confidence requirement for terminating deliberation. A deeper investigation into the UU,term parameter reveals its multifaceted role. From Bayesian decision theory perspective, OUtterm establishes a decision boundary, aiming to balance Type I errors (e.g.. prematurely accepting a sub-optimal consensus) against Type II errors (e-g., unnecessarily prolonging the debate). Our primary experimental observations (see Section 4 and Appendix D.4) further expose several key convergence characteristics and sensitivities: First, the system uncertainty U/4y) typically decays in an approximately exponential manner during debate; Second, models with larger parameter scales (e.g.. 72B/78B) exhibit lower sensitivity to changes in Og indicating more robust uncertainty estimation in larger models; Third, more complex tasks such as relational reasoning display higher sensitivity to the setting of fyteerm compared to simpler tasks like text comprehension. Despite the effectiveness of the current termination mechanism, several theoretical limitations [22) remain: (I) adopting a single global Outterm for all task types may not be optimal; (2) the mechanism does not dynamically adjust Ou,term based on instance- specific factors (e.g., problem difficulty); (3) employing a fixed fu,errm throughout the debate does not account for the dynamic evolution of uncertainty. Based on the above theoretical analysis and main experimental findings, we offer the following theoretical insights for setting 0u,errm: (I) Reasonable upper and lower bounds should be established an excessively low fu,eerm may lead to unnecessary debate rounds and diminishing marginal returns, while an excessively high threshold risks premature termination, undermining the benefits of iterative deliberation; (2) Model-adaptive setting in theory, as model capability increases, the optimal fyyterm should decrease accordingly; (3) Task-adapp dasktidaetive setting-thr the ideal implementation should adjust Outterm based on task type, assigning stricter thresholds for more complex tasks
D Derivation of Uncertainty Theory
To provide a rigorous theoretical foundation for the collaborative reasoning process within the GAM- Agent framework, particularly concerning the role of uncertainty and dynamic consensus building (88. we model the interaction among expert agents as consensus-driven non-zero-sum game. This game- theoretic perspective elucidates how agents converge towards final consensus through interaction and dynamic adjustment of uncertainty. Empirically, we observe that the uncertainty of predictions decays exponentially with debate rounds, which we quantify using a decay coefficient X. As shown in Figure X, larger models such as VL.3- exhibit higher À values and faster per-round uncertainty suppression (e.g., x = 0.51 with a 39.99..9 99.9% decay rate), supporting the view that iterative multiagent consensus not only improves performance but also systematically reduces epistemic uncertainty.
23

---

Relationship Between À and Uncertainty Decay Rate
42
Lineaf Ir 1.00)
Uncertainty Decay Rate per Round (9)
40
deen2 5VL-72
38
36
30
28
(wen2.5VL-78
26
0.30
0.40
0.45
A (Decay Coefficient)
Figure 10: Exponential decay of uncertainty with debate rounds across models. Larger models (e.g.. VL3-7VL3378B) exhibit faster uncertainty suppression.
D.1
Mathematical Formalization of the Non-Zero-Sum Game
We consider multi-agent system comprising N expert agents. The system processes input instance J, which consists of an image X and a question P,, The input space is formalized as the Cartesian product of the image space X' and the question space Pr, denoted as I X' x P,, A specific input instance is / = (X, P,) E I. Agent Set (E): The participants in the system form the set of agents, denoted as & framework definition.
N). This corresponds the E' in the GAM-Agent
Agent Function (Agent Mapping A:): Each agent i E & is formalized as function or mapping that takes an input instance and produces an output pair, consisting a response R2 and an associated uncertainty U,. The response space is denoted by R, and the uncertainty space is the interval [0, 1]. The output space for agent i is the Cartesian product O = R [0, 1]. Thus, the function of agent can be represented mapping:
A 14:I~> - O
(6)
For given input the output of agent i is:
(R,(D),
A
Output pair of Agent i
Mapping of Agent applicd inpur I
where (T) E R is the response and U.( E [o, 1] is the uncertainty. This aligns with the analysis capability mapping A, and uncertainty assessment o described in the Method section (A, produces R, and L (R,) yields U.).
Joint Agent Output (Joint Agent Output Mapping A): The collective output of the entire agent set for the same input can be viewed as combined mapping. Its output space is the Cartesian product of all individual agent output spaces: O defined as:
II= O. (R [0, 1)) The joint output mapping
A:I+
(8)
For input I, the joint output is:
(1), (R,
(9)
Joint Agent Ourput
Tuplic individual Agont outputs
System Weight Allocation Protocol (P): The system dynamically assigns weights w
(Wi...., WN) based the joint agent output, particularly their uncertainties U = (UI,..., Un) and
24

---

potentially their responses { The weight vector w belongs to the standard - - dimmesnsiin simplex Á AN-1. We formalize the weight allocation mechanism as a protocol function P that takes the joint agent output as input and produces a weight vector:
P: O A-1
(10)
Thus, for input I, the weights assigned by GAM-Agent are w(1)
P(A())
Weights computed by protocol based on Ageni outputs
The weight vector must satisfy the constraints of the standard simplex:
(T(T)) 00 E 3 and Σ
(T( =
(11)
weight is non-negative
equals 1
Sum of all weights
This protocol corresponds to the dynamic weighting mechanism used in the initial integration and iterative debate phases of GAM-Agent.
System State (σ): After receiving input I, the instantaneous state of GAM-Agent can be fully described by the joint output of the agents and the corresponding weight vector computed by GAM- Agentm. We define the system state as tuple:
(1) ((Ai()), (I., ((A))))))))))))))).....()....... An()))
(12)
Set of Agent cutput pairs and corresponding system weights
This state (I) belongs to the state space S:
OXAN-1
(13)
System State Space
In the dynamic collaborative process (iterative debate), the agents' internal mappings A, might adapt based on interaction, leading to an evolution of the state or across different rounds.
D.2 Utility Function Design
We design a utility function L4; for each agent E E to quantify its "payoff" or "performance score" in the current system state. Within the consensus-driven non-zero-sum game framework, an agent's individual objective is to maximize its utility value. Concurrently, because the utility function incorporates inter-agent dependencies and the overall system state, improvements in total utility can be achieved through effective collaboration among agents.
The input to the utility function U4 reflects the critical information influencing agent i's pay- off: the responses (R, N and uncertainties (U,) of all agents, along with the current sys- tem weight distribution w. The utility function maps this information to real value: U;
(R x [0, 1])
X N -- R
Joint Agent Outputs (Responses & Uncertainties)
System Weights
We define the utility function U4 for agent as a weighted sum of three primary components:
Agent Weight
Agent i Confidence
Agent y Weight
Similarity between Agent and j
System Weighted Average Unoeriiitty
u((RR (U
Wc
U.)
Wy
Sim(Ra, Rτ))
Usys
Ter System Peralty
Team 1: Individual Contribation
Temm 2: Collaboration Reward
where Semantic Similarity Function (Sim): Sim Rx - [0, 1] measures the semantic relevance or consistency between two responses. Sim R2 R2) 1 indicates perfect semantic agreement, while Sim R,, R2) 0 signifies complete irrelevance. This function can implemented using various methods, such cosine similarity based on text embeddings, analysis of overlapping key
25

---

evidence or claims, or comparison of structured outputs. It is typically assumed to be symmetrie: Sim(ri, r2) Sim(r2,, r1) for all 71, E R.
System Weighted Average Uncertainty (Usy)): This key metric quantifies the overall uncertainty of the system's 's current consensus, defined the weighted average of all agent uncertainties using their current weights:
Agent Weight
Agent l Uncertainty
N
Uyy
Wk
Uk
Weighted averago Agent uncertainties
(since
Σ
my==1)
Usys E [0, 1]. where lower values indicate higher system confidence the integrated consensus result.
Hyperparameters (A, 7): E R8 is the weight coefficient for the collaboration reward term, encouraging agents to align with the system consensus. Y Ryo is the weight coefficient for the system uncertainty penalty term, penalizing all agents when the overall system uncertainty is high.
Rationale and Significance of the Utility Function Design: This utility function design reflects the core objectives and trade-offs in building a collaborative consensus system:
Individual Contribution (w; (1 U:)): Rewards agents for providing high-quality (low uncertainty U. thus high confidence 1 - U.) responses that are assigned high weight (w;) by the system. This incentivizes agents to improve the quality and reliability of their responses.
Collaboration Reward (A ÆE wy Sim R2 Ry)): Rewards agents for providing responses that are consistent (Sim( R, R2) is high) with those of other agents who are considered reliable (high weight wy)) This drives consensus by encouraging agents to consider and align with the views of highly weighted peers. controls the importance of this collaborative alignment.
System Penalty (-7Usy)): Penalizes all agents based on the overall weighted average uncertainty of the system. When the system as a whole is uncertain about the current consensus, every agent's utility decreases. This motivates agents not only to minimize their own uncertainty but also to contribute to reducing the system's 's overall uncertainty through effective collaboration (e.g., sharing evidence and resolving conflicts), as the system's 's health benefits everyone. controls the focus on this systemic uncertainty.
Non-Zero-Sum Characteristic: Unlike traditional voting or simple averaging schemes which can implicitly resemble zero-sum scenarios (one agent's gain another's loss), our design allows for collective improvement:
Increasing collaborative consistency (higher Sim RR R2)) can increase the collaboration reward term for multiple agents simultaneously, boosting their utilities without necessarily decreasing individual confidence.
Reducing the overall system uncertainty (lower Usys, e.g., through effective evidence ex- change) decreases the system penalty term -yUsys, thereby increasing the utility for all agents.
These mechanisms allow the total utility ΣN Ur to potentially increase through effective inter agtert collaboration, highlighting the positive-sum potential of the game. This aligns better with the nature of multi-agent collaboration for complex cobblem-solving compared to zero-sum or fixed-sum games.
D.3 Exploration of Nash Equilibrium Existence
In formal game theory, the Nash Equilibrium (NE) is a central concept for predicting stable outcomes of rational agent behavior. It describes a profile of pure strategies s* = (si...., 8 S II=11 S, where S; is the pure strategy space of agent i, such that agent by unilaterally changing its strategy 8, E S, given that all other agents maintain their equilibrium sty). Formally, a NE satisfies:
improve its utility (s(s)
strategies st, = (si...., 18-1 S4t1.....
U U1(3),8)) M
U, (8.,
Vss E ViEE for strategy for all Agents of Agent
(14)
Utility of Agent equilibrium
Utility of Agent é when deviating unilaterally
26

---

The best response correspondence BR; S-i - 2% maps the strategy profile of other agents s-, to the set of agent i's optimal pure strategies:
(R.()-:) { E S U s-1) (s), Vs; E S.}
(15)
Set of Pure Best Responses of Agent given 8-4
pure strategy Nash equilibrium s* is fixed point of the joint best response correspondence BR: S + 25, where BR(s) II (s.(s-:):
8"
BR(s))
(16)
Joint Best Respouse
Theorem 1 (Existence of Pure Strategy Nash Equilibrium): Consider a game with a finite number of agents N < co. Under specific assumptions regarding the agents' strategy spaces S; and utility functions U4 the non-zero-sum game described above possesses at least one pure strategy Nash equilibrium.
The existence of a pure strategy NE is often established using Kakutani's Fixed Point Theorem applied the joint best response correspondence BR S -> 2 This requires satisfying the following conditions: 1. Properties the Domain Space: The domain of BR, the joint pure
Proof Outline:
strategy space S = II S,, must be non-empty, compact, and convex subset of Euclidean space Rdd
S + 0,
compact,
is convex
Joint strategy space
Joint straregy space
This necessitates that each agent's pure strategy space S; is non-empty, compact, and convex.
S
+
S compact.
is convex
E
Non-empty
Compact
Convex
for all Agents
In VLM systems, agent's pure strategy s; determines how it generates its response R, R and uncertainty U; E 1]. The response space R (e.g., all possible text sequences) is typically discrete, vast, and non-convex, making it challenging to define compact and convex strategy space S; that fully captures the agent's 's complex behavior.
1. Simplified Strategy Spaces under Assumptions: meet the theorem's 's requirements, one might model agent strategies more simply:
Asummption 1.1 (Uncertainty Strategy): Agent i's pure strategy 8; is solely its choice of reported uncertainty Ug E [0, 1]. Then S. [0, 1], which is non-empty, compact, and convex.
Continuous Strategy): Agent i's pure strategy 8; is determined by a low-dimensional continuous vector 0; E O C IR,, representing internal parameters influencing its response and uncertainty. If O is non-empty, compact, and convex (c.g., a hypercube), and the agent's 's output (R, U&) is a continuous function of s,, this requirement met.
Assumption 1.2
2. Properties of the Mapped Values:
The value of the mapping B R((s) (the joint best response set) must be non-empty and convex. This means that for each agent i, its best response set (s must be non-empty and convex.
(s) (R:(s-)) is convex set
Vs-i E S_, VEEE
(17)
Non-empty
Convex
any strategy profile for all Agents other agents
This property can be established by analyzing the agent's utility function U4 with respect to its own strategy 8, Specifically, if S; non-empty, compact, and convex, and (3,,8_) is continuous and quasi-concave in 8;, then (s-:) is guaranteed to be non-empty and convex.
81 Ue (8,, S-i) is quasi-concave Vi E E, Vs_ E S
(18)
Utility of Agent i wet. its strategy
27

---

Proving quasi-concavity for L4, w.r.t. 8, is a central challenge, especially when determines complex outputs (R, requires analyzing the convexity of the upper level sets {s; E S; Us(s, 8-1, 2 c). Under simplifying assumptions (e.g., is choosing U, E [0 and Ui quasi-concave in U2), this condition can be met.
3. Continuity of the Mapping:
The correspondence BR(s) must have a closed graph or, more strongly, be upper hemi-continuous (UHC). This implies that if a sequence of strategy profiles g(m) + and corresponding sequence of best responses s((m) BR((s converge to s', then s' must be a best response to s, i.e., E BR(s).
BR has a closed graph (or is UHC)
(19)
Closed Graph Upper lemi continuous
If the strategy space S is compact and the utility functions (s) are continuous S, then the best response correspondence BR, is UHC, and consequently, the joint correspondence BR is also UHC. Thus, continuity of the utility functions is key.
If all these conditions (non-empty, compact, convex joint strategy space; non-empty, convex-valued, UHC joint best response correspondence) are satisfied, Kakutani's Fixed Point Theorem guarantees the existence of a fixed point s* E B BB(ss*) which is pure strategy Nash equilibrium (37]. Rigorously verifying these conditions for complex VLM systems often requires simplified models or strong assumptions about agent behavior and interaction functions.
When proving the existence of a pure strategy NE is difficult (c.g., due to non-convex strategy spaces or non-quasi-coneave utilities), one can consider mixed strategy Nash equilibria. A mixed strategy a; for agent a probability distribution over its pure strategy space S, The space of mixed strategies for agent is Σ; A (S:). Under mixed strategies, agents maximize their expected utility. Nash's Theorem guarantees that mixed strategy NE always exists for games with finite agents and finite pure strategy sets, or more generally, if pure strategy spaces are compact metric spaces and utility functions are continuous.
Mixed Strategy Nash Equilibrium:
Σ; =;=)(S:)
(o.,..., GN
(20)
Mixed stralegy space Agent
Joint mixed strategy
mixed strategy NE 0² = -oii...., exists such that for all agents v:
Egror [s](s)] 2 Els, ioton (es,,,,
Ya: E Σg
(21)
for any mixed strategy of Agent
For the VLM game, if we consider the agent pure strategy space compact metric space (e.g., via representation learning mapping responses compact space) and utility functions are continuous, Nash's Theorem ensures the existence of mixed strategy NE. While existence is more readily guaranteed, the interpretation and practical relevance of mixed strategies VLM collaborative reasoning require further investigation.
In summary, proving the existence of pure strategy NE in the VLM game requires rigorous mathematical analysis of the agents' pure strategy spaces and utility functions, often under simplifying assumptions that satisfy specific mathematical properties.
D.4 Cooperative Game Objective and Dynamic Weighting Protocol
The dynamic process in our system aligns with the philosophy of cooperative game theory, where agents collaborate to enhance the overall quality of the consensus. The system-leve objective can be framed as maximizing the sum of all agents' utilities (0), i.e., the total utility J:
max J((R)N (UN w) W
max WEAN-L
(U. E=
(22)
System Total Utility
(Sum of Individual Utilities)
28

---

Substituting the definition of 44 into the total utility J:
==
Σ wi(1 Ui) A
Σ wy
Sim(R,, Rτ) Usss
(23)
iti
Expanding and rearranging terms, using Σ wi and Usys = -ER=1 wa Un:
N
N
N
J
U.)
Σ
wj Sim(R; R2)
U
(24)
E=
Sum of Individual Contributions
Sum Collahoration Rewards
Sum Penalties
N Wi
Σ
WiUi
+
Σ
wj
Sim R2) NyUsys
(25)
(1 Unys) +
Σ
Sim Rú, R2) - NyUays
(26)
t=1 A+
N
Σ
w
Sim R2) (1 NNy)
(27)
i=1
17
Substituting Usys
Ekm wA Un and ignoring the constant term 1, the optimization problem be-
comes:
N Σ
max WEA
A
ΣΣΣN mj Sim R4, R,) Nr) jæt
Wp
(28)
Total Weighted Similarity Torn
System Weighted Uncertainty
Rearranging the summation terms to isolate Wi:
N
N
Wj . Sim R2 R,) = D (xf
Σ
W
Sim R4, Rz)
(29)
==1
wk
Σ
Sim R2 Rτ)
(relabeling index j to
(30)
i
The objective function becomes:
N Σ
max N
Wk
Sim Rk) -1+
(31)
1=1
Agent "Cooperative
Scorey,
Let Score, = A *k Sim Rú, Ra) (1+ U, For fixed {R.} and (Ui), this is maximizing linear function Σn w/ -Scrre over the weight simplex A N-1, To obtain a smooth, non-degenerate
weight distribution, introduce entropy regularization term - E wp log wk (with 1) 0) to the objective:
N Σ
N
max N-1
Score
wy log wk
(32)
Entropy Regularized Total Utility
29

---

The solution to this standard convex optimization problem (found via KKT conditions) yields optimal weights wi in the form of Softmax distribution:
exp(n Score,)
wk
(33)
m== exp(1 Scorem)
Static Optinaal Weight Form (depends on Scorog)
Substituting the definition of Scorez:
exp (n (1E ip Sim RM, RA) - -1++
(34)
Σ Dm= m=1 exp (1 (AEitem is Sim Ris Rm) - (1 + NN))Um
Optimall weight Agent
(depends on all It,, and lyperpurameters A,7.1)
This static optimal weight distribution wi depends on all agent responses, uncertainties, and hyperpa- rameters, reflecting the full cooperative optimization goal.
Actual Dynamic Weighting Protocol Used in GAM-Agent: In our iterative GAM-Agent framework, the weights updated dynamically each round & based primarily on the agents' uncertainties from that round. The protocol used is:
-BBU!
(k+1) W.
(3>0) 0)
(35)
N E E0 B0
Weight for round k++1 (based uncertainty from round k))
Here, ,3 > controls the sensitivity to uncertainty. This corresponds to the weighting described in
Section 2.3 and used implicitly in Section 2.5
Connecting Static Optimality and the Dynamic Protocol:
We now demonstrate that the dynamic
weighting protocol (Eq. 35) is not arbitrary heuristic but corresponds to the optimal solution of simplified, entropy-regularized static optimization problem. Consider simplification the total utility I where we ignore the collaboration reward (A = 0) and the system penalty (? = 0). The objective becomes maximizing the weighted sum of individual agent confidences:
max WEA*
Σ
Us
max WEAN
Σ
w;(1 - U.)
(36)
Let the "score" for agent be its confidence s; score Σi= wys; with entropy regularization minimization; maximizing score + entropy):
U4). We want to maximize the weighted average
Σ 1= wi log w; (note the sign change compared to
max WEAN N
Σ
w U.) + w. log ii
(37)
3
The optimal solution w for this entropy- regulariee maximization problem takes the Softmax form based on the score (1 - U:):
exp(3 U.)) E
W
(38)
N
exp(B. U,)
Static optimal weight form (based on confidence score)
exp(3) exp(-
(39)
== exp( (8) exp(-
Exponential form expansion
-U,
(40)
E e SU,
Simplified form
30

---

The resulting optimal weight w has exactly the same form as the dynamic weighting protocol used in GAM-Agent (Eq. 33). with /9 playing the role of the inverse temperature parameter r).
j== e BUg
Static optimal weight based on regularized conlidence
Matches the dynamic protocel form
This derivation shows that the dynamic weighting rule employed in GAM-Agent, wi
(k+1)
exp( e
corresponds precisely to the optimal weight distribution derived from maximizing
E, exp(-BUp
the entropy-regulatizze weighted average of agent confidences (1 (z(")) at each step k. While this dynamic protocol simplifies the full cooperative objective (Eq. 34 by omitting explicit collaboration rewards and system penalties in the weight calculation, it provides computationally efficient and theoretically grounded mechanism. It prioritizes influence for agents demonstrating lower uncertainty (higher confidence) at each stage, driving the system towards consensus state characterized by reduced overall uncertainty, as explored in the convergence analysis related to the debate termination condition (Eq. 5)
E
Theorem I: Fundamental Game Theoretic Framework of GAM-Agent
The core mechanism of GAM-Agent is driven by mathematically well-defined, non-zero-sum game model centered on "uncertainty" as key variable. This model derives its collaborative strategies and influences allocation through the optimization of explicit utility functions(66)
Definition 1 (Mathematical Formalization of GAM-Agent S). Our GAM-Agenl S is rigorously defined as a sextuple:
E Agent
M
P
Analytical Capability Map Function Ser
Uncertainty
Evidence Localization Function Set
Claim Parsing Dynamic Debate Function
Assessment Function
Mechanion
where:E = 1,2..... N represents the set of N expert agents. A = A1 A2..... AN defines the set of analytical capability mapping functions, where each function A;
P. + Ra maps an imput image X and question P, to agent i's response R,, +b = L1, P2...., D, is the set of uncertainty assessment functions, where $; R4 - [0, 1] quantifies the uncertainty U, associated with response RG M- = M,, . MN represents the set of evidence localization functions, where M; :;CCC - x [0, 1] associates agent i's claim e with visual region T and a confidence score a. P: P. (cy, "j, = 1 is the claim parsing function, converting unstructured responses into structured information tuples (K is the number of tuples). D X x P, x R4, Uçi 1 + Ramat is the debate module that coordinates conflicts and consensus to generate final response. This formalization constructs complete mathematical space, allowing for precise description and analysis of the system's components and their interactions.
Definition 2 (Mathematical Construction of Uncertainty Quantification Function (;(R))) GAM- Agent provides two uncertainty quantification mechanisms for each agent e, E E:
Uncertainty based generation probability Digent RR)): When the token generation probability distribution of the underlying VLM is accessible, is defined as:
T Tj
T sgen+ (Ra)
a
H(P..))
+B- max 1 App (Pr.))
Information Entropy (Hesibation)
Inverse Probability Difference (Lack: Confidence)
where: Ta is the total number of tokens in response Rd, P.. is the probability distribution of the t-th token. H(P.,)) - (x) log (z)(()) is the entropy, quantifying the dispersion
"hesitation" of the distribution. Amp Pi, Pi,t is the difference between the
(1) (2)
31

---

(8) is the k-th highest probability. and B are probubilities of the top two tokens, where Pit parameters balancing these two complementary signals. High entropy H ((P..) indicates model hesitation among multiple possible tokens, while small (up(Pt) indicates a lack of strong confidence in its preferred token.
By formally expanding the entropy calculation:
((P.))= - E
- Paa(x) (a,((a)
ZEV
where V is the vocabulary, see that entropy is maximized when the probability distribu- tion Pa, approaches a uniform distribution (highest hesitation), and minimized when the distribution is concentrated on single token (lowest hesitation).
Uncertainty based semantic markers Disem (Ra): When generation probabilities unavailable, the system uses:
isso (R,) = G sigmmnid (k. (e.22we weight(w) count(w, Ra)
offset
Ra
where: W is multi-level lexicon of uncertainty markers. weight(w) reflects the uncertainty intensity of marker w. coun((w, R2) is the frequency of in response RG. |Ra] is the
response length. k and offset control scaling and bias. asigmoid value to the (0, interval.
I+e normalizes the
These two methods capture uncertainty from the generation process and semantic content, respectively, providing key inputs for the subsequent game theory framework.
Definition 3 (Non-zero Sum Game Theoretic Formalization of GAM-Agent). The multi-agent
visual reasoning process is modeled as non-zero-sum game with the following formal structure: = X x Pr, where X the image and P, the question. Action Output of Agent The action of agent e, is to generate response-uncerraintyy pair (R, UT))) with the mapping function: A; I + Oi F x [0, 1], (I) = (R, (R,(())) System Weight Allocation Protocol P: Rules for assigning influence weights, formalized as: P OOO -A* A key weight calculation ) is: wi ) (U,j=11)
Players: Set of agents E = I....,N. .N. Input Instance: 1= (X.P,)
formula (initial weight (0)))
This is an uncertainty- ド= exp(-BU,)
expl oxp(-3U.)
based Softmax allocation: higher Us (greater uncertainty) leads to lower weight w (0) Parameter > 0 controls sensitivity uncertainty differences. System State The instantaneous state of the system is fully described by the joint outputs of the agents and the corresponding weight vector: o (I) = ((A)(T),.. (I)...., Ax (I)), P(A1 (I)...... Ax (I))), (I) ON A NN This game structure formalizes the interactions among agents in GAM-Agent and clarifies how uncertainty affects weight allocation and system state evolution.
Definition 4 (Mathematical Expansion of Agent Utility Function w:). A utility function w; ON x N-1 I defined for each agent ei mapping joint agent outputs and weight allocation real-valued utility:
W, (U,) = (1(1-Up)
Σ wy S.m( R4, R5) jitt
U
Individual Contribation Term (Torm 1)
System Cost Torm (Term 3)
Collaborative Benefit Torm
(Term
where: Individual contribution term (Term 1): w - U2) rewards the agent for providing high- confidence (low uncertainty) judgments, reflecting its system-assigned influence. Collaborative benefit term (Term 2): X X i Wy Sim( R2) rewards the agent for consistency with other high- influence agents, where Sim (,,(R..A R2) E [0. is the semantic similarity between responses, and weight coefficient. System cost term (Term 3): ?Usy, where Usys =N wyUk is the system's 's weighted average uncertainty, and x > 0 is a weight coefficient. All agents share the system's uncertainty. From mathematical theory perspective, this utility function has the following properties: Non-zero-sum nature: Agents can collectively increase total utility Σn= us by reducing collective uncertainty and improving consistency, without necessarily competing against each other.
32

---

The incentive for collective reasoning: The collaborative benefit term incentivizes agents to agree with other highly influential agents. Uncertainty management: Agents must reduce their own uncertainty (Term 1) and contribute to reducing overall system uncertainty (Term 3). By detailing the system cost term:nU,.. >
k=1 WyUKk we see that even if an agent's own uncertainty U3 is low, it will be negatively affected if the overall system still has high uncertainty (high Usys. further promoting cooperative behavior:
Proposition 5 (Analytic Optimal wi for Regularized Utility). Our GAM-Agent seeks optimal influ- ence weights w* = (wi,-.. ..... wx) that maximize the total utility:
N
Ugoral ()
Σ
Ui [ N
U w
Expanding with Definition 1.4:
N Σ
Unnat = Σ (w:11-10
-L.1 U.) + Σ
mj Sim(Rs R2)
waUnn
k=1
Further expansion:
Using the constraint Ei=1 Wi 1 to simplify:
N
N
N
Unstat (w) = Σ
W;U
Wj. Sim R2; Rÿ) - Nr Σ UkUk
i=1
jfz
N Sim(Rs R2) N7) Σ waUn
=1+1
Wj
jæi
Rearranging the summation order:
N
N
N
ΣΣ
wy Simm R1, R2)
Σ
W
Sim(Re R2)
Σ
wk
Σ
Sim Rú, Rg)
j=1
($1
Substituting into the above equation and ignoring the constant term 1, we get the optimization problem:
N
max
Wk
Σ
Sim R4, Rτ)) NY)UU
WEAN N
k=1
ig
Define agent k's "cooperative value score":
Score, = =>Σ Sim(R, RA) -1 NY)Uk
To avoid degenerate solutions (all weight assigned to the agent with the highest score), an entropy regu- larization term - E wk log wy (where i temperature parameter) introduced, yielding the regularized optimization problem: maxwEaN-1
E wr Scorek === wx log ws This a standard convex optimization problem. Using the Lagrange multiplier method, the analytical Scorek))
form of the optimal weights is: wi m=1N exp(ti-Score..))
Substituting the full expression for
Score::
exp (n. [AΣ ipp Ri; Ra)) - (1 (1+NN)UH Nr)Uk
WI
Em= exp nn- D Sim( Rm) (1 Ny)Um
From this analytical solution, it is clear that: Weight wi decreases exponentially with an increase in agent k's uncertainty Up. Weight wi increases exponentially with an increase in agent k's semantic consistency with other agents, Eepr Sim R&). Parameter n) controls the sensitivity weight allocation differences in Scores-high 1) values lead to more lyinner-take-alls allocation.
This proves that agent influence allocation GAM-Agent is an analytical solution with a clear mathematical basis, rather than simple heuristic rule.
33

---

GAM-Agent establishes a mathematically self-consistent and mechanistically interpretable non- zero-sum game framework through rigorous mathematical definitions and derivations of the system. uncertainty, game interactions, utility functions, and optimization objectives. Its core mathematical contributions are: Trranforming "uncertainty" from an abstract cognitive concept into a precisely quantifiable mathematical variable (via & igenn and is Designing a three-part utility function u; that balances individual contribution, collaborative consistency, and system risk. Analytically deriving the mathematical expression for agent influence weights wi, by maximizing the regularized total utility Vuutt Demonstrating that the system's 's game mechanism is endogenously generated from utility optimization principles, rather than exteraaly imposed.
F Theorem II: Comparative Analysis with the DMAD Framework
The mathematical formalism of DMAD (Diverse Multi-Agent Debate) is primarily manifested as an algorithmic flow based on externally preset diversification strategies. Its core mechanism, "breaking cognitive fixation," mathematically lacks an endogenous, agent-centric utility optimization model equivalent to GAM-Agents drive it.
Definition (Mathematical Representation of the DMAD Algorithm Flow), The core of the DMAD framework guides n agent instances {Mi to each adopt a distinct reasoning method (prompting strategy) R, assigned from a presel collection IK = (R,,...,R.,, be formalized for each round j of debate as: ax:Individaal Solution Generation: 'vi E 1,...,n (8i.j: Vb.s) = Execute (MtecuteeMM,.a0 x. h4, Ri) where: Mi is the i-th agent model. x. is the input task. hy; is agent i's historical information. Ri is the preset, fixed reasoning method/prompting strategy. (si.]. J. j) is the output pair, containing the reasoning process si,j and answer yi. Information Propagation and History Update: Let lilj = (sk, J. Uk.,) (e=s))= be the set of outputs from all agents in round j. Vi E I,...,;: nI: h; f
Its mathematical flow can
(h; HIj) This step propagates the outputs of all agents to each agent, updating their historical information.
Analysis 1 (Comparative Analysis of DMAD's Mathematical Form- The Missing Endogenous Game Optimization Aspect). Compared to GAM-Agent (Theorem 1), DMAD exhibits the following substantial differences the mathematical formalization of its core mechanisms: Compared to GAM-Agent gent (Theorem 1), DMAD exhibits the following substantial differences in the mathematical formalization of its core mechanisms: Lack of an optimizable, agent-level mathematical utility function: The DMAD framework does not define a mathematical utility function for each agent Mi similar to u, in GAM-Agent. Formally, there no mapping: un u.MAD. System State + R Therefore, when agent Mi executes its assigned reasoning method Ri, its behavior (e.g., how it interprets and utilizes information h; from other agents to improve its own si.j + 1, yi. + 1) is not driven by clear optimization process aimed at maximizing its own mathematical utility. Its "improvement" relies more on the LLM's 's inherent context-learning capabilities and adherence to "critical feed- buck." II GAM-Agent, every decision can be truced back to the maximization of utility function uaz max (,u,,a,, a_.. Uyj 1 R5j = = In DMAD, agent behavior is solely determined by the fixed strategy R4 and historical information h2 lacking this explicit optimization structure. Mathematical implementation of the "diversity" mechanism: DMAD's 's core idea, "breaking cognitive fixation through diverse reasoning," " is achieved by externally enforcing different reasoning methods R, upon different agents. Mathematically, this is represented as mapping: Assign .1.... - IR Assign(i)= = R, such that R, #R if + j This "diversity" " a system-level structural design, not result of individual agents selecting strategies based on some intrinsic muthematical incentive (like a "diversity contribution utility"). In contrast, the uncertainty and utility-driven mechanisms in GAM-Agent allow diversity to emerge naturally from the game difeeeifferen agents gain higher weights in different aspects where their uncertainty is low, low. forming complementary rather than preset specializa- tions. Non-explicit mathematica modeling of "breaking cognitive fixation". ": "Cognitive fixation" " itself and the process of it "being broken" " are core cognitive science assumptions and desired out- comes in DMAD, but they are not directly modeled mathematically variables objective functions that can be quantified or optimized in agent decision- maaiing or system evolution. Formally, there is no function: fmindset: System State - P. Its effect is indirectly validated through experiments (e.g.. performance improvements on different benchmarks), rather than predicted or explained through the analysis of a clearly defined mathematical objective function.
To more clearly illustrate the fundamental differences in mathematical construction between DMAD and GAM- we can compare their mathematical derivation processes for the same task:
34

---

For visual reasoning task Pr):
DMAD's Processing Flow:
1. Preset different reasoning methods: ;
2. Assign to agents: M, uses Ri
3. Execute multi-round interaction: (8t.j. Vk.) Execute(M: x. R,)
4. Final aggregation: Yfimal = di (1h.
GAM-Agent's Processing Flow:
1. Calculate uncertainty: U² (Ra)
2. Optimize weights based on utility function us; ut = Σ, exp(n Score))
exp(n--Scon
3. Trigger debate based on uncertainty and conflict: TriggerDebate = (Ui5)) > Ou) (ConflictScore > Oc)
(c+1) exp( (xp(-BU((1)
4. Dynamically adjust weights until convergence: wi Σ, -xp((-30)-))
5. Generate oulput based on final weights: Rimn Integrate/udgee . {w (K) Ji=l:..)
Conclusion of Theorem H: DMAD's mathematical form mainly reflected in its clear, procedural algorithmic steps and information flow structure. Its core lertaniimm-eeerrggien
preset, diverse reasoning strategies } to enhance reasoning performance-is a heuristic design based on cognitive insights. Its mathematical description serves the execution of this process, but it lacks mathematical game model equivalent to GAM-Agent's, which agent-centric and uses explicit utility function op- timization to endogenously drive agent behavior and system mechanisms (especially how "diversity" affects decisions).
G Theorem III: Differences in Mathematical Construction between GAM-Agent and Traditional Multi-Agent Methods
GAM-Agent's mathematical framework fundamentally differs in its mathematical construction from traditional multi- nulti-agent reasoning and debate methods that do rely on explicit game- -ene-thhoeettc optimization (hereinafter "traditional methods"), particularly in handling uncertainty, mechanism derivation, and collaborative optimization.
Definition 7 (Mathematical Characteristics of Traditional Multi-Agent Reasoning). Simple Ag- gregation Methods: Mathematical core: Typically involves direct, non-interactive matheratical operations on outputs y1..... yN independently generated by N agents. Ygmıt Sage (yzi 1%) where s'ags can be: Mode(.): Majority voling, selecting the most frequent answer. Average(-): Mean, for numerical outputs. Σ: aays Fixed weighted average, where as are preset, non-dynamically optimized weights. Mathematical limitations: Such methods mathematically do not include: (a) An interaction model among agents. (b) Explicit consideration of individual output confidence or uncer- tainty. (c) A dynamic influence adjustment mechanism based on (a) and (b). Debate Frameworks: Mathematical core: Agent behavior and interaction are controlled by a set of predefined, deterministic rules Srules, such as turn-takke turn-taking or role-playing. Agent e; 's action of in turn follows: a; Policyrole(i), state(t) (history) (tatt(t) Srules) where: role(i) is the preset role of agent i (e.g., questioner. answerer, judge). state(t) is the system state at turn t. history is the prior interaction history. Srules is the set of rules governing interaction. Mathematical limitations: Interaction mechanisms (like who speaks or has influence) are hurd-coded by external rules Snaee rather than endogenously derived from a mathematical optimization problem based on all agents' current internal states (especially quantified uncertainty U,) and collaborative goals (like uz).
35

---

Elaboration on Core Mathematical Differences with GAM-Agent: Mathematization and Mech- anistic Integration of Uncertainty Handling: GAM-Agent:
Uj = (R.)
input
U4
-,,,,uii MaC,..,Ui.... Ui....)
used for optimizing,
wx (.... Uk....)
Matherattiaall Quantified Uncertainty (Delinition 1.2)
Utility Function (Detination 1.4)
Optimal Weights (Proposition 1.1)
GAM-Agent's uncertainty handling can be detailed as:
1. First, precise mathematical quantification of uncertainty via + Pigens or 4 isemm:
T: (iH))a((P..))) 3 max Amp (Pr,)))
if & Pigens is used
U, = (
Tsigmoid (k: (Zwew weight( count) offiset)
if Pism is used
Ri
2. Then, direct integration of this quantified uncertainty into the agent's utility function:
un (Ri Uj, w) w;(1 U:) - X شEzz Wy Sim(B; R2) E wxUx
3. Finally, analytical derivation of optimal weight allocation by maximizing the utility function that includes uncertainty:
exp(n IA Sim( (1+ N7)Un)
m== exp(1) [A Eipm Ri, Rm) - + Ny)
This forms complete, end-to-end mathematical processing chain for uncertainty, from quantification utility calculation and mechanism design.
Traditional Methods:
Typically lack such end-to-end mathematical modeling and mechanistic integration of uncertainty. For simple aggregation methods, uncertainty information is entirely miss- ing, formalized as: Yimil = fags (y (does not depend on any uncertainty quantification U,) For rule-based debates, even if agents
express "confidence." this expression often qualitative or heuristic: Confidence; = Qualitative_Eppresiionn Expression((:: (e.g.. "I am very sure," "Possibly," etc.) Moreover, there is unified mathematical framework (like GAM-Agent's utility theory and opti- mization framework) to convert these discrete, potentially heterogeneous confidence signals into precise, derivable mathematical impacts on debate dynamies (like speaking rights, influence) or final consensus.
D.4.2 Mathematical Basis of Interaction Mechanisms and Influence Allocation: GAM-Agent: Influence weights wt (Proposition 1.1) are the analytical solution to the mathematical optimization problem of maximizing the system's 'ss overall regularized utility:
cxp(1 Score,)
w Σ exp(1) Score,)
where Score; = A i Sim R&) - (1+ Ny) is agent k's "cooperative value score." This analytical solution has the following significant characteristics:
It is continuous, with weights wt E (0, 1) precisely reflecting the relative value of agents.
It is dynamic, changing with Score; (which includes UR and Sim terms).
It is derived from first principles (maximizing total system utility), not preset.
It considers two key factors: uncertainty Us and semantic consistency Sim R2 R4).
It provides flexible adjustment mechanisms through parameters A, 7-
Traditional Methods: Influence allocation (if exists) often based on preset rules or heuristies:
For majority voting, each agent's influence is equal (implicitly w; = 1/N):
Y inall arg max
1( = =
36

---

Or in some cases, it is binary (vote or not vote):
final arg max Σ 1y - y) ((Qualification)
In role-playing debates, if a judge role exists, its "influence" conferred by the rules S
Yinal= = Decision(Judge) (where Judge is preset role)
Mathematically, the "influence" in these methods lacks the mathematical properties of GAM-Agent's weights wg: it is not continuous, dynamic variable derivable from first principles like utility maximization.
D.4.3 Mathematical Form and Explicitness of Collaborative Optimization Objectives: GAM- Agent: The system's optimization objective is to maximize >; (regularized). Each term of the utility function de (Definition 1.4) has clear mathematical form and corresponding collaborative goal: u4
we:(1 U
wy WyyS Ry)
Σ ww Us
juf
Pursuing own high certainty
Aligning with trusted peers
Jointly reducing system uncortainty
This utility function explicitly:
Quantifies three dimensions of collaboration: individual certainty, mutual consistency, and system uncertainty.
Establishes precise mathematical trade-offs between these three dimensions (via parameters
À and 7).
Directly links utility maximization to system design goals (high-quality consensus).
Usually lack such a unified, explicit mathemaatial optimization objective function that includes uncertainty management and fine-grained measures of collaboration quality. For example, voting methods, the implicit "objective function" might be:
Traditional Methods:
max
1(y y) (maximize the number of agents supporting specific answer)
This objective function does not include:
Explicit consideration of uncertainty.
2. Measurement of the quality of inter-agent consistency.
3. Evaluation of the overall system state.
In rule-based debates, the "collaborative" goal (like reaching the correct answer) is implicit, and the means to achieve it (like debate rules Smiees are preset, not derived by optimizing an explicit mathematical function.
Conclusion (Conclusion of Theorem IV). GAM-Agent's mathematical framework, through its:
Precise mathematical quantification of agent uncertainty (e.g., Digent formula) and its use as core game variable.
Analytical derivation of key interaction mechanisms (like the Softmax form of influence weights w from optimization problem involving mathematical utility function (u:) that includes uncertainty terms and explicit collaboration terms,
fundamentally diverges in theoretical rigor of mathematical construction, mechanistic endogeneity, and optimization explicitness from "traditional" aaeti reasoning and debate methods that rely on simple aggregation. preset rules, or lack equivalent mathematical optimization objectives. GAM- Agent offers a muthematically-drive
solution where each component and mechanism can be truced buck to explicit, uncertainty-buser based mathematical model, rather than just procedurully-defmee framework. This mathematical rigor enables GAM-Agent to more effectively coordinate uncertainty in multi- agent systems, achieving more flexible, adaptive, and interpretable visual reasoning.
37

---

H Hyperparameter Ablation
To provide a deeper understanding of GAM-Agent's sensitivity to its internal settings, we conduct further ablation studies on key hyperparameters. These experiments supplement the component- ablations presented in Section 3.3 of the main paper. All experiments in this section were performed using GAM-Agent (Intern (3-14B) with N = 3 expert agents on the MMBench__TSS TESTTEST_VV V11 dataset. The maximum number of debate rounds (mmzz) was set to 3, consistent with the setup in Section 3.3 of the main paper. We report Overall Accuracy (Acc. %), Average Actual Debate Rounds (Deb. Rounds), Debate Trigger Rate (Deb. Trig. %), and Average Inference Cost (Cost, tokens/instance).
Our default hyperparameter configuration for GAM-Agent (IntemVL3-14P) which achieved 88.80% accuracy in the main paper (see Figure 3), is as follows:
Uncertainty weighting sensitivity (Sweight): 1.5
Debate trigger threshold for system uncertainty (0g:): 0.45
Debate trigger threshold for conflict score (0c): 0.55
inveration-process uncertainty Pigen+ parameters: ag 0.5, 3q. 0.5
Table 3
presents the results of varying these hyperparameters.
Table 3: Hyperparameter ablation study for GAM-Agent (Intern VL3-14B, N = 3, Kmar 3) MMBench, TEST V11. The Default row corresponds to the configuration used to achieve the main paper's reported 88. 88.80% accuracy for this model in Figure 3 (main paper).
Hyperparameter
Value
Acc. (%)
Deb, Rounds
Deb. Trig. (%)
Cost (tokens/inst.)
Uncertainty Weighting Sensitivity (Sweighs in wi X exp( Buiigha
0.5
88.15 (10.65)
1,85
68
2650
1.0
88.62 (J0.18)
1.80
66
2580
Boiight (Default)
1.5
88.80
1.76
65
2500
2.0
88,71 (J0.09)
1.73
64
2450
3,0
88.45 (J0.35)
1.69
62
2380
Debate Trigger Threshold System Uncertainty (0g)
0.35
88.92 (10.12)
2.10
2850
By (Default)
0.45
88.80
1.76
65
2500
0.55
88.31 (10.49)
1.42
2100
0.65
87.93 (10.87)
1.15
1800
Dehate Trigger Threshold
Conflict Score (0c)
0.45
88.85 (10.05)
1.95
72
2750
Ag (Default)
0.55
88.80
1.76
65
2500
0.65
88.53 (J0.27)
1.50
53
2200
0.75
88. 10 (J0.70)
1.25
40
1950
Generation Uncertainty
+ syene Parameter Qg. (given Ogg + .4
0.1 (8ep 0.9)
88.42 (10.38)
1.78
65
2520
0.3 (Bep 0.7)
88.68 (10.12)
1.77
65
2510
as (Default)
0.5 (5eg 0.5)
88.80
1.76
65
2500
0.7 (3& 0.3)
88.59 (10.21)
1.75
2490
0.9 (.86 0.1)
88.27 (10.53)
1.74
64
2480
Combined Dehate Trigger Thresholds (Ous, (c)
(0.35, 0.45)
88.98 (10.18)
2.25
82
3050
(Default Combination)
(0.45, 0.55)
88.80
1.76
65
2500
(0.55, 0.65)
88.15 (J0.65)
1.30
45
1900
Discussion of Hyperparameter Sensitivity
Uncertainty Weighting Sensitivity (Sweight): This parameter controls the sharpness of the softmax function used allocating agent weights based on their uncertainty U.. Our results suggest that moderate value (Default: 1.5) provides good balance. Very low values (e.g., 0.5) make the weights too uniform, diminishing the impact of precise uncertainty quantification and slightly reducing accuracy 15%).. Higher values (e.g., 2.0, 3.0) make
38

---

the weighting more "winner-take- potentially overly relying single agent if its uncer- tainty is marginally lower. which can also slightly degrade performance (88.71 18..1% and 88.45% respectively) by reducing diversity in the integration. The cost tends to decrease slightly with higher fweigght as more decisive initial weighting might lead to quicker convergence or fewer contentious points triggering prolonged debates.
Debate Trigger Thresholds (0v, Oc): Lowering the system uncertainty threshold Ou (e.g., to 0.35) or the conflict score threshold 0g (e.g.. to 0.45) increases the debate trigger rate (Deb. Trig. to 75% and 72%, respectively). This leads to more debate rounds and higher computational cost, but can sometimes yield marginal accuracy improvements (e.g.. 88.92% for By = 0.35) by allowing the system to resolve more nuanced disagreements. Conversely, higher thresholds (e.g., OU = 0.65 or Oc = 0.75) significantly reduce the debate frequency, rounds, and cost, but at the expense of accuracy (87.93% and 88. 18%, respectively). as critical conflicts might be overlooked. Ablating them in combination (e.g., Oy 0.35,0c
0.45) shows an even higher debate trigger rate (82%) and cost, with a small potential accuracy gain (88, 98%), indicating that more debate is not always cost-effective for the performance gained. The default values (0.45, 0.55) provide good trade-off.
Snneeration-Process Uncertainty Parameters (a.., These parameters balance the contribution of information entropy (hesitation) and top probability difference (lack of conviction) in the & Pigen- uncertainty metric (Equation I in the main paper). We assumed Ogg. + 3gg = for this ablation. The results indicate that an approximately equal weighting (Default: at.. 0.5, Agg 0.5) performs best (88.80%) (8%). Overly relying on just one component (c.g., ag = 0.1 heavily weights probability difference, or Ccp = 0.9 heavily weights entropy) leads to a noticcable drop in accuracy, suggesting that both signals are valuable for a comprehensive uncertainty assessment. The impact on debate rounds and cost is minimal in this ablation, implying these parameters primarily affect the quality of the uncertainty scores rather than the frequency of debates, assuming the scores still fall within similar ranges.
In summary, these ablations highlight that while GAM-Agent is robust across reasonable range of hyperparameter settings, optimal performance achieved by carefully tuning the balance between uncertainty sensitivity, debate triggers, and the composition of uncertainty metrics. The default parameters chosen for Intern VL the main paper represent well-balanced configuration for MMBench.
I Hyperparameter Setting
This section outlines the specific hyperparameter configurations employed for GAM-Agent throughout the experiments detailed in the main paper. Our goal is to provide clarity for reproducibility and understanding of the conditions under which our reported results were achieved.
Common GAM-Agent Settings Unless explicitly stated otherwise in the subsections below, the following common hyperparameter settings were applied across all GAM-Agent instantiations:
Number of Base Expert Agents (N): Typically 3 for all reported experiments. This N corresponds to the number of agents in summations such as Σ wi (0) U, and refers to the agents responsible for generating initial responses (R₂) and participating directly in argumentation (Arg; (kc) during debates. These agents were derived from the same base Vision-Language Model (VLM) for each experiment.
Number of Critical Expert Agents (Ner)): Maintained by default throughout all experiments that involved the iterative debate mechanism. As illustrated in Figure (main paper), these agents are engaged by the Debate Controller specifically for verification and providing critical feedback on contentious claims. Critical agents typically use the same underlying VLM but initialized with specialized prompts foster critical assessment.
Uncertainty Weighting Sensitivity (Sweight): 1.5. This parameter is used in the entropy- regularized softmax for both initial weight allocation (w, exp(-Bweights U+)) and
39

---

dynamic weight updates during the iterative debate (wx² (k) uncertainty-based weighting is active).
when
Generetion-Prooess Uncertainty ( (Rigen+) Parameters (used when token-level logprobs were accessible, e.g., for Qwen2.5VL, InternVL3, VL3, Intern Video2.5):
-ag: 0.5 (weight for information entropy component).
:.:: 0.5 (weight for top-K probability difference component).
Semantic-Marker Based Uncertainty (iisem) Parameters (used when logprobs were not accessible, e-g,, for GPT-40-0513,
fallback):
Lexicon (W): predefined multi-level lexicon uncertainty markers (e.g.. "might", "possibly". "unsure", "confident", and "clear") with associated weights. (Specific lexicon details are part of prompt engineering).
Sigmoid scaling 1.0.
Sigmoid offset: 0.3.
Debate Termination Criteria:
System uncertainty threshold (fu) 0.15. Debate terminates if U < 0
Convergence speed threshold (c): 0.01. Debate terminates if |
Evidence Mapping (M) and Claim Parsing (P): These modules were active in all con- figurations involving debate, utilizing the base VLM perform claim extraction, evidence
association, and visual grounding tasks as described in Section 2.4 (main paper).
L.1
Settings for Image Understanding Experiments (MMBench and MMMU)
These settings correspond to the experiments detailed in Section 4.1 1.1 of the main paper
L111
Qwen2.5VL Series (3B, 7B, 32B, 72B) and Intern VL.3 Series (2B, 8B, 14B, 38B, 78B)
Base VLMs: As listed.
Datasets: MMBench (TEST EN), MMMU (test set).
Number of Base Expert Agents (N):: 3.
Number of Critical Expert Agents (Nara): (when debate triggered).
Uncertainty Quantification: Pigent was prioritized.
Maximum Debate Rounds (Kman): 3.
Debate Trigger Thresholds:
System uncertainty (fu): 0.45.
Inter-expert conflict score (0c): 0.55.
For instance, (AAM-Agen((ntemm\L3-1111)) using these settings achieved accuracy of 88.80% on MMBench (TEST_EN) and GAM-Agen((Qwee255VV--77 achieved 90.86%.
I.1.
GPT-40-0513
Base VLM: GPT-40-0513 (via OpenRouter API).
Datasets: MMBench (TEST_EN), MMMU (test set).
Number of Base Expert Agents (N)): 3.
Number of Critical Expert Agents (Noru)): : 3 (when debate triggered).
Uncertainty Quantification: D Pisem was used due the lack of direct access token-level generation probabilities from the API.
Maximum Debate Rounds (Kmax):
Debate Trigger Thresholds:
System uncertainty (0v:): 0.50 (adjusted slightly due different uncertainty scale from
T issem.
Inter-expert conflict score (0c): 0.60.
With these settings, GAM-Agen(GGPT-40051 achieved 86.53% on MMBench (TEST EN).
40

---

I.2
Settings for Video Understanding Experiments (MVBench)
These settings correspond to the experiments detailed in Section 4.2 of the main paper.
1.2.1
Qwen2.5VL. Series (3B, 7B, 32B, 72B), Intern Series (2B, 8B, 38B), and Video2.5
Base VLMs: As listed. InternVide02.5 a video-specialized model.
Dataset: MVBench.
Number of Base Expert Agents (N): 3.
Number of Critical Expert Agents (Nori):: (when debate triggered).
& Pigen+ was prioritized. For Intern Video2.5, logprobs were accessible, Digen+ was used; otherwise, Disem was adapted.
Uncertainty Quantification:
Maximum Debate Rounds (ama)): 3.
Debate Trigger Thresholds:
System uncertainty (fu): 0.50 (video tasks often present higher inherent ambiguity).
Inter-expert conflict score (0c): 0.60.
Visual Preprocessing: Please refer to the official parameter configuration provided by MVbench, for example, seleet fps=1.
GAM-Agent VL33emVVL3338B) achieved overall average of 78.47% on MVBench with these settings.
1.2.2
GPTT-400513 (for MVBench)
Base VLM: GPT-4o-0513.
Dataset: MVBench.
Number of Base Expert Agents (N)): 3.
Number of Critical Expert Agents (Nera): (when debate triggered).
Uncertainty Quantification: Pisem.
Maximum Debate Rounds (Kmon): 3.
Debate Trigger Thresholds:
System uncertainty (By): 0.55.
Inter-expert conflict score (0c): 0.65.
GAM-)gem((GPT-4-055113 achieved 70.58% MVBench.
L.3
Settings for Comparison with Existing Multi-Agent Methods (MMBench)
These settings apply to the GAM-Agent configurations used in the comparisons presented in Sec-
tion 4.3
Base VLMs:: Qwen2.5VI. (7B, 32B, 72B), Intern VI.3 (8B, 14B, 78B).
Dataset: MMBench (TEST EN).
Number of Base Expert Agents (N): 3.
Number of Critical Expert Agents (Nara): (when debate triggered).
Uncertainty Quantification: Pigent was prioritized.
Maximum Debate Rounds (Kmew): 3.
Debate Trigger Thresholds:
System uncertainty (0v): 0.45.
Inter-expert conflict score (0c): 0.55.
These consistent with the primary MMBench settings for these models.
41

---

1.4
Settings for Ablation Studies Base Configuration (MMBench TEST V11)
The default configuration for GAM-Agent VL3-14B in the ablation studies (Section 4.4 and Figure 3 of the main paper), before individual components or parameters were ablated, used the following:
Base VLM: Intern VL3-14B.
Dataset: MMBench FEST_VIL
Number of Base Expert Agents (N): 3.
Number of Critical Expert Agents (Nara): (when debate triggered).
Uncertainty Quantification: Pigen+ with aqp 0.5, Aq = 0.5.
Uncertainty Weighting Sensitivity (3weight): 1.5.
Maximum Debate Rounds (K (ax)))
Debate Trigger Thresholds: 9u 0.45, 0c 0.55.
Debate Termination Criteria: 8, = 0.15, 0.01.
This configuration achieved 88.80% accuracy and served as the baseline for both the component ablations in Figure 3 (main paper) and the hyperparameter ablations in Section of this supplement.
1,5
Settings for Study on Maximum Debate Rounds (MMBench_ MilbenchhhVi__est (Section 4.5, Figure O the main
For the experiment analyzing the impact of paper):
Base VLM: Qwen2.5VL.-7B.
Dataset: MMBench V11__est.
Number of Base Expert Agents (N): 3.
Number of Critical Expert Agents (Nora): (when debate triggered).
Uncertainty Quantification: Pigent with or. 0.5, B3 0.5.
Uncertainty Weighting Sensitivity (Sweight): 1.5.
Debate Trigger Thresholds: 0g = 0.45, 0c = 0.55.
Debate Termination Criteria (other than K man)): Ou = 0.15, 0.01.
Varied Parameter: Kmaz was varied from 0 to 9.
1.6
Settings for Expert Weight Trajectory Analysis (Logical Reasoning Tasks)
For the analysis of expert weight dynamics (Section 4.6, Figure 4 of the main paper):
Base VLMs: Qwen2.5VL. (3B-72B), InternVI.3 (2B-78B)
Dataset: subset of MMBench focusing on logical reasoning tasks.
Number of Base Expert Agents (N): 3 (Relational Reasoning, Scene Description, OCR experts).
Number of Critical Expert Agents (Nori): (when debate triggered).
Uncertainty Quantification: Pigen+ with Orpp 0.5, 9g = 0.5.
Uncertainty Weighting Sensitivity (Bweight): 1.5.
Maximum Debate Rounds (Kmea): 3 (analysis shown over three rounds).
Debate Trigger Thresholds: Assumed to active debate occurred, e.g., fy = 0.45,
I 0.55.
Debate Termination Criteria: 0 0.15, 0.01.
The focus of this experiment was on observing weight and system uncertainty evolution under these typical settings.
42

---

Prompt Setting Statement
This section outlines the prompt configurations for various agents within the GAM-Agent framework. The prompts are crucial for guiding the behavior of the Large Language Models (LLMs) acting as expert agents and the aggregators.
J.1 General Prompts for Expert Roles and the Aggregator
In this subsection, provide some examples of general persona prompts that can be used to initialize different experts and the aggregator. These illustrate how roles can be defined within the system. The actual task execution would typically involve combining such persona prompts with more specific task instructions, like those detailed Section U.2
Illustrative Analysis Expert Persona
You expert in problem analysis and logical reasoning, skilled in applying analytical frameworks and systematic thinking approaches. Your expertise includes breaking down com- plex problems, identifying key factors, and recommending structured, actionable solutions. You are familiar with various problem-solving methods such as root cause analysis, decision matrices, and scenario evaluation, and adapt your approach based the unique context of each task. Consider how your skills critical thinking, structured reasoning. and analytical problem-solving might provide valuable insights strategies for addressing the task at hand.
Illustrative Strategy Expert Persona
You are a business strategy expert with a deep understanding of markets, business models, competitive landscapes, and strategic planning. Your expertise includes applying business frameworks, analytical tools, and market insights to identify opportunities and craft strategies. While capable of providing comprehensive strategic analysis, you adapt your input to focus what most valuable, practical, and relevant for the situation. Consider how your expertise in business innovation, competitive advantage, and strategic problem-solving might provide insightful and actionable recommendations for any task.
Aggregator Prompt
You are the Wise Integrator in multi-agent system tasked with delivering accurate, coherent, and actionable responses to user queries. Your role is to:
Understand the user's intent and main question(s) by carefully reviewing their query.
Evaluate expert inputs, preserving their quality opinions while ensuring relevance, accuracy, and alignment with the user's 's needs.
Resolve any contradictions or gaps logically, combining expert insights into a single, unified response.
Synthesize the most appropriate information into clear, actionable, and user- friendly answer.
Add your insight if needed to enhance the final output.
Your response must prioritize clarity, accuracy, and usefulness, ensuring it directly addresses the user's 'ss needs while retaining the value of expert contributions. Avoid referencing the integration process or individual experts.
J.2 Prompts for VLM Experts in Benchmark Evaluations
For the experiments conducted on MMBench, MVBench, and MMMU datasets, specific set of Vision-Languaan Model (VLM) based experts were utilized. These include three Base Experts for initial analysis and argumentation and three Critic Experts for the debate and verification stages. The prompts for these experts are detailed below. Note that placeholders like (instruction;
43

---

Evaluate the scene description part [response]] ), etc., are dynamically filled during runtime. The 'keywords' listed in the configuration (not shown here) were used potentially aid in expert selection or routing for more complex multi-faceted queries, though for single-task evaluations, experts often processed all relevant inputs.
J.2.1 Base Expert Prompts
The following Base Experts are primarily responsible for generating initial analyses and partici- pating in argumentation by providing evidence-backed claims. Their "critique_eemplate and 'revi- sion_tews noo_teeplatee (shown below each main prompt) are used during the iterative debate process.
Object Recognition Expert
Role Definition: You are expert in object recognition. Prompt Template (prompt template): As an object recognition expert, identify and list all significant objects visible in the image(s). Provide details about their appearance, count if possible, and relative location. For key claims, mention the visual evidence. Original question: (instruction]
Critique Guiding Template part of the analysis:
Evaluate the object recognition
(response) Are the objects correctly identified? Are there any missed objects incorrect descriptions? Base your critique strictly on the visual evidence.
Revision Guiding Template (revision template): Based on the critique "(critique) please revise your object recognition analysis: deiiginal_respooe ).
Focus on accuracy and completeness according to the visual evidence. Original question: iistrrucion)
Scene Description Expert
Role Definition: You are an expert in the scene description. Prompt Template scene description expert, describe the overall scene shown in the image(s), including the location, environment, lighting, atmosphere, and spatial relation- ships between elements. For key claims, mention the visual evidence. Original question: innntion)
template): (prompt_teeplateee
Critique Guiding Template (critique_templlee of the analysis:
(response)
Is the description accurate and comprehensive? Does it capture the main elements and atmosphere the scene? Base your critique strictly on the visual evidence. Revision Guiding Template (revision __ttppaateelae
Based the critique (critiquee', please revise your scene description: (original (eaiaaaleesposee)... Fo- capturing the visual details accurately. Original question: (instruction)
44

---

Text/OCR Analysis Expert
You expert OCR and analysis from images. Prompt Template (prompt template); As an OCR expert, identify and transcribe any text visible in the image(s). Pay attention to signs, labels, documents, or any written content. Note the location of the text if possible. For key claims, mention the visual evidence. Original question: (instruction]
Role Definition:
Critique Guiding Template (critique_ (eteciqqueeemllate]e Evaluate the OCR/text transcrip-
tion part of the analysis:
{ (response)
the transcribed text accurate? any text missed? Is the location noted correctly? Base your critique strictly on the visual evidence.
Revision Guiding Template (revision_templattee template): Based on the critique '(critique'' please revise your OCR analysis: welgginnn_responee Ensure accuracy and completeness of the transcribed text based on the visual evidence. Original question: (isttruction)
J.2.2 Critic Expert Prompts
The following Critic Experts are engaged during the debate phase to verify claims, check for completeness, and ensure logical consistency. Their primary input "critique template' that guides their evaluation.
Fact Checker (Image) Critic
Role Definition: You are an expert in fact-checking claims against visual evidence. Critique Guiding Template (critique template): Act fact-checking expert focusing on visual evidence. Original Question: linstruct ion} Current Answer: tresponsel Critique Request: (critique [oque__eeuet] Please evaluate the factual accuracy of the claims based *only* on the image content provided. Point out inaccuracies and suggest corrections. State your confidence (0-100%).
Completeness Checker Critic
You expert in assessing the completeness of image-based analysis. Critique Guiding Template (critique template): Act as completeness analysis expert for image descriptions. Original Question: i iinstruction: Current Answer: (response) } Critique Request: (critique_requeet addresses the question based on the *entire* image content. Are there missing details or aspects from the image(s) that are relevant and should have been included? Provide suggestions. State your confidence (0-100%).
Role Definition:
Evaluate if the analysis fully
Logic Checker Critic
Role Definition: You expert in evaluating the logical consistency of analysis. Critique Guiding Template (critique_ (aruue__tmlaate) Act as a logical consistency expert for image analysis. Original Question: (instruction
Current Answer: re tresponse) Critique Request: (critique_reeeeeeeee requestt Evaluate the logical consistency of the analysis. Are there any contradictions unsupported conclusions based the visual evidence and the question? Provide suggestions for improving logical flow. State your confidence (0-100%)
important note that these textual prompts form the core instructions. The effectiveness of these prompts can also be influenced by the specific capabilities of the underlying base VLM, its training data, and any additional system-level instructions few-shot examples that might be used in complete implementation.
45

---

K
Case Analysis
In this section, we present a series of case studies, including four successful and two unsuccessful examples, to illustrate how multiple experts collaboratively analyze images in response to correspond- ing questions. The expert configuration consists of three analysis experts-an Object Recognition Expert, Scene Description Expert, and a Text/OCR Analysis Expert-as well as three critique experts: a Fact Checker (Image), Completeness Checker, and Logic Checker. For each analysis expert, we report both their analytical response and the associated uncertainty score. For each critique expert, we provide their evaluative feedback. To enhance clarity and conciseness given the length of the responses, key excerpts are highlighted using colored underlines.
K.1 Successful Case
demonstrate that our method, by assigning clearly defined roles to each anpert-including object recognition, scene understanding, and text/OCR analysis-facilitaie comprehensive analysis across multiple modalities and semantic dimensions of the input. This structured task decomposition enhances both the depth and breadth of information processing by enabling each analysis expert to focus specific sub-task and generate high-quality outputs along with corresponding uncertainty estimates. key strength of this approach lies its explicit quantification of uncertainty, allowing the system to weigh and prioritize more reliable expert responses. The outputs from analysis experts are subsequently evaluated by critique experts-namely a Fact Checker (Image), a Completeness Checker, and a Logic Checker-whord feedback further refines the final answer. This multi-expert architecture promotes factual accuracy, contextual completeness, and logical consistency, thereby improving the system's robustness, interpretability, and overall stability across diverse question types and input formats.
tables to
K.2 Unsuccessful Case
While our multi-expert framework generally yields effective results, tableee several critical limitations that warrant further investigation. In this case, all three analysis experts Object Recogni- tion, Scene Description, and Tex/OCR Analysis-produced responses with low uncertainty scores, yet the final aggregated output was incorrect. This outcome suggests that high internal confidence among individual experts does not necessarily correlate with overall prediction accuracy.
central issue stems from the subjective nature of the question, "Which image is more colorful?" The term "colorful" admits multiple interpretations, including color diversity, saturation intensity, and perceptual vividness. Both the Object Recognition and Scene Description experts focused predomi- nantly on the saturated blue tones of the underwater image, interpreting it more visually striking. However, they failed to consider color diversity-a more appropriate metric in this context- for determining "colorfulness. This divergence in interpretation highlights key limitation of the current framework: the absence of a shared semantic grounding among experts when addressing subjective or ambiguous queries.
Moreover, the inclusion of the Text/OCR expert, despite the absence of textual content in the image, underscores the need for more adaptive expert selection strategy. The existing mechanism lacks the ability to dynamically suppress irrelevant expert responses, potentially diminishing the influence of pertinent analysis during the final aggregation phase.
In addition, the critic experts- Fact Checker, Completeness Checker, and Logic Checker- did not effectively intervene despite the misaligned interpretations provided by the analysis experts. This exposes another weakness in the framework: the critic module's limited capacity to identify and correct semantic inconsistencies when expert outputs are confidently wrong yet semantically misgrounded. Without sufficient oversight from the critics, the model fails to detect the misapplication of key concepts like "colorfulness," leading unverified and ultimately incorrect conclusions.
Finally, the model is vulnerable to systemic bias amplification when multiple experts converge on incorrect reasoning patterns. In the absence of higher-level semantic validation or external verification mechanisms (e.g., feature-level consistency checks), such coordinated emrrr can propagate unchecked, undermining the reliability and robustness of the system.
46

---

table 9 similarly reveals additional limitations of the current framework, centering on the experts' overreliance on superficial visual cues. In this case, both the Object Recognition and Text/OCR experts emphasized the "grayish-brown" skin tone of the two rhinoceroses and concluded that the animals were of the same color. However, this assessment overlooked subtle factors that can influence the perceived coloration of animals, such as lighting, shadows, and texture variations. These experts failed to account for environmental influences-suce as the soft sunlight noted by the Scene Description expert-whine may introduce perceptual discrepancies due to illumination shifts.
Although the Scene Description expert analyzed contextual elements such as background vegetation and natural lighting, their focus on the "tranquil atmosphere" of the scene did not directly address the core question-whettee expert's focus and the semantic intent of the question illustrates a critical limitation of the framework: when an expert's domain emphasis is misaligned with the specifie query, the resulting conclusions may be incomplete or misleading.
the rhinoceroses are indeed the same color. This mismatch between the
Furthermore, despite the absence of textual content in the image, the Text/OCR expert was nonetheless activated and contributed to the color judgment. This highlights the lack of a fine-grained expert selection mechanism. Although this expert commented on the consistency of skin tones, they failed to recognize the complex interplay of visual cues-such as lighting and surface texture-that are essential for accurate color perception.
Notably, critique modules such as the Completeness Checker and Logic Checker failed to identify and rectify these semantic misalignments. For instance, the Completeness Checker did not emphasize the role of illumination and texture in color evaluation, while the Logic Checker did not flag the deviation of expert reasoning from the question logical flaw.
This failure case underscores the necessity of improving the expert selection mechanism to ensure alignment between the activated experts and the task-relevant semantics. Moreover, incorporating advanced visual analysis capabilities-
as precise color comparison algorithms or illumination- invariant models could mitigate such errors. The aggregation process should also become more sensitive to contextual, lighting, and texture variations in visual data to enhance the overall accuracy and robustness of the system.
47

---

Table 4: Case: Successful Expert Collaboration Example
Image
pollen
seedling
adult
pistil
seeds
fruit
anther
Question
Which part of an apple tree might grow into new tree?
Expert
Uncertainty
Output
Object Recognition
0.1838
1.Identified the scedling stage and the seeds inside the fruit.
2.Highlighted that seeds are labeled inside fruit, indi-
caliin their role starting tree's 's life eycle.
Scene Description
0.1245
E.Ephasized the educational and instructional nature image.
2.Noted the spatial relationships and the cyclical arrows connecting each stage, reinforcing that seeds are crucial for new growth.
Text/OCR Analysis
0.0832
1.Confirmed the labels "sceds" inside the fruit and "seedling" as the initial growth stage. 2.Supported the conclusion that seeds are the part that grows into a new tree.
Faet Checker (Image)
No
1.Seeds Tree Origin: Correct. The image labels
seeds inside the fruit and shows the cycle beginning with seedlings.
2.L.fe Cycle: Correct, Arrows and stages illustrate the eyclical growth process starting from seeds.
Completeness Checker
No
1.Pollination Explanation: Briefly describe how pollina- tion by bees leads to fertilization 2.Flower Importance: Explain the role of flowers the reproductive process.
development.
Logic Checker
No
1.Object Recognition Expert: Accurately identitied the seedling stage and the seeds inside the fruit.
2.Seene Description Expert: Highlighted the educational nature and the cyclical arrows, which effectively support conclusion.
3. 3.Text/OCR Analysis Expert: Confirmed the labels supported conclusion with evidence.
Answer
The part of an apple tree that might grow into a new tree a seed.
48

---

Table Case: Successful Expert Collaboration Example
Image
Question
this place crowded?
Expert
Uncertainty
Output
Object Recognition
0.1665
1. The image shows an empty movie theater with multiple
rows of black seats arranged in a typical theater layout, Tacing large, blank projection screen,
The majority the seats are unoccupied, with only two individuals seated
middle 3. These individuals spaced with ample around them, and there other visible patrons.
theater.
Scene Description
0.2126
1. The dim lighting is consistent with theater setting, but does not dircetly indicate the absence of crowd.
2. The overall scene appears quiet and still, supporting the impression of emptiness.
Text/OCR Analysis
0.0832
1.The image doesn't contain any text or symbols, so it's recommended to skip the Text/OCR analysis
Fact Checker (Image)
No
1.Empty Seats: Correct. The image shows mostly empty seats, with only two people present.
2.Seating Arrangement: Correct. The seating layout re- sembles that of a typical movie theater.
3. 3.Proectioon Screen: Correct, The blank screen reinforces sense of emptiness.
Completeness Checker
No
1.Seat Count Detail: Briefly mention number seats or rows elarify the theater's capacity and how full 2.Scating Context: Describe whether the individuals seated together apart to enhance the scene' detail. 3.Lighting Description: Indicate whether the lighting typical for theater unusually dim to provide more context for the ambiance.
Logic Checker
No
1.Object Recognition: Accurately identified the theater layout, empty seats, and presence of only two spaced indi- viduals, clearly showing low occupancy.
2,Scene Description: Appropriately noted typical dim lighting and quiet, still atmosphere that support the im- pression of emptiness.
3. T.Tot/OCR Analysis: Correctly recognized there is no text to analyze, avoiding unnecessary processing.
Answer
This cinema is crowded all.
49

---

Table 6: Case: Successful Expert Collaboration Example
Image
Question
Which image is brighter?
Expert
Uncertainty
Output
Object Recognition
0.1342
1. The left image (beach scene) is significantly brighter due to its clear, sunny environment with bright blue sky and light blue ocean.
2. The right image (underwater scene) darker, typical of underwater environments where light absorbed and scattered. resulting greenish hue.
Scene Description
0.0832
1. The left image features bright, natural lighting nario with minimal cloud cover, enhancing its brightness.
2. The right image has dimmer lighting and greenish tint, characteristic of underwater scones with less light penetration.
Text/OCR Analysis
0.0832
The image doesn't contain any text or symbols, so it's recommended to skip the TexiOCR analysis
Fact Checker (Image)
No
B.Brghhnsss Levels: Correct. The left image is noticcably brighter with clear sky and light-colored ocean. 2.1 L.igghing Environment: Correct. The right image is darker, reflecting the typical low-light conditions under- water.
Completeness Checker
No
1.Brightness Metrics: Briefly mention specific metrics or tools (e.g., average pixel brightness) used to quantify hrightness, clarifying the hasis The comparison. 2.Color Context: Describe the dominant colors in each im- and how they influence perceived brightness, such as blues and whites brightening the left image versus greens and darker tones dimming the right one.
Logic Checker
No
1.Object Recognition: Correctly identified the bright beach scene and darker underwater scene with greenish
tones.
2.Scene Description: Accurately noted bright natural light in the beach image and dim, greenish light underwater. 3.TexUOCR Analysis: Properly skipped text analysis due text present.
Answer
Compared image on the right, the image the left clearly brighter.
50

---

Table Case: Successful Expert Collaboration Example
Image
Question
Which one the correct caption of this imuge?
Expert
Uncertainty
Output
Object Recognition
0.1670
1. The image features the following key objects: Motor- cycle: Black with chrome details, located in the center, ridden by a person. Person: Wearing black helmet, dark shirt, jeans, and carrying a brown bag, riding the motorcycle. Red Car: red sedan, partially motion. the left of the motorcycle. Building: A brick huilding with green shutters, the background. Street Elements: Multiple street signs, streetlight, and sidewalk along the road.
Scene Description
0.5213
1. The image depicts city street scene with person riding black motorcycle, wearing a helmer, dark shirt, and jeans. red cur is visible in the background, along with brick building featuring green shutters.
2. The setting includes trees and a sidewalk, suggesting urban environment. The day. with casual and busy atmosphere.
takes place during the
Text/OCR Analysis
0.1651
1.Thrre sign visible the left side of the image, but the text it not clear enough to transcrihe accurately.
Fact Checker (Image)
No
M.Motrcyclee Color: Correct. The motorcycle clearly black with chrome details and large front fairing. Background Details: Correct. The background includes red and brick building with green window shutters, matching the description.
Completeness Checker
No
I.Minor Background Flements: Briefly mention the per- son near the building to add context to the scene. 2.Accossory Details: Deseribe the rider's bag and shocs to enrich the overall depiction.
Logic Checker
No
1. 1.Ojject Recognition: Clearly identified the motorcycle, rider, car. building, and street elements.
2.Soene Description: Concisely described the urban set- ting, people, vehicles, and daytime atmosphere.
3.TexvOCR Analysis: Correctly noted sign is present but text is unreadable, no transcription attempted.
Answer
correct caption for the image woman riding motorcycle street."
51

---

Table 8: Case: Unsuccessful Expert Collaboration Example
Image
Question
Which image is more colorful?
Expert
Uncertainty
Output
Object Recognition
0.0991
1. Left Image (Kitchen/Diming Area): The left image shows kitchen/dining area with neutral tones. featuring dining table, cabinets, stove, and a blue rug. Natural light brightens the space.
2. Right Image (Underwater Scene): The right image displays stingray in vibrant blue water, creating color- ful and lively underwater scene.
3. The right image features deep blue underwater which is significantly more vibrant compared the neu- tral tones of the kitchen and dining area on the left. The blue water and the contrast with the stingray enhance the overall colorfulness.
Scene Description
0,2553
1.The right side has a rich, deep blue buckground. creating vivid and striking visual effect. The left side, while bright, muted and natural color palette dominated by light wond and neutral tones, underwater intensity and saturation make il more colorful.
Text/OCR Analysis
0.0832
1.Since the image does contain any labels, symbols, is advisable to exclude Text/OCR analysis.
Fact Checker (Image)
No
U.nneewwater Scene: Correct. The right image clearly shows deep blue underwater environment.
2.Color Contrast: Correct. The deep blue tones the right image vivid and intense compared to the muted, neutral tones of the kitchen on the left.
Completeness Checker
No
1.Color Details: Briefly mention the dominant colors and their saturation in both images to add clarity.
2.Lighting Effects: Describe how lighting and natural elc- ments influence color perception in each scene.
Logic Checker
No
1.Object Recognition: Accuratcly identified key elements in both images the kitchen/dining area on the left and vibrant underwater scene with stingray on right. 2.Scene Description: Effectively contrasted the muted, neutral of the kitchen with the vivid, saturated col- of the underwater environment.
3TTxi/OCRR Analysis: Appropriately noted the absence of text, correclly skipping text analysis.
Answer
Compared to the image the left, the image the right is obviously more colorful,
52

---

Table 9: Case: Unsuccessful Expert Collaboration Example 2
Image
Question
Are the two animals the picture the same color?
Expert
Uncertainty
Output
Object Recognition 0.1213
1. Two rhinoceroses standing side by side, facing slightly left.
2. Both have grayish-brown skin tone, prominent horns, and large cars.
3. A yellow bird with red markings is perched near the horn each rhinoceros.
Scene Description
0.9481
The two thinoceroses in image are in natural environment, possibly forest savanna, with green vegetation in the background. The lighting natural, suggesting might be early momiing late afternoon, with soft sunlight. The atmosphere is calm and serene.
E.Evironmmet:
The two rhinoceroses are standing side by side, facing forward, with similar poses, indicating they may belong to the same group.
2. P.ooittion and Pose:
Tex//OCR Analysis
0,2315
1.There visible text or written content in the image. Both animals appear typical grayish-brown huc for rhinos. The visual evidence A supporting this conclusion includes the consistent col- oration and texture visible on both animals" skin.
same color, which
Fact Checker (Image)
No
1. P.Presence of Birds: The small hirds the rhinos' heads clearly separate and do affect the color evaluation of the rhinos.
2.Color C Consistency: Both rhinos show similar grayish- brown color, as confirmed by the visual evidence. 3.Impact of Birds: The birds do not influence the per- ceived color the rhinos.
Completeness Checker
No
1.Texture and Lighting: Mention how texture and lighting conditions affect color perception visual impact. B.ackground Elements: Brielly describe background fea- like green foliage add context.
Logic Checker
No
Correctly identificd the rhinos, their color, and the birds their horns.
1.Object Recognition:
2.Scene Description: Described the natural setting, light- ing, and rhinos' positions.
3. T.Txt/OCRR Analysis: Noted absence of text and consis- tent thino coloration.
Answer
The two rhinoceroses in the picture are indeed the same color.
53

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No]. or [NA]-
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1 (1-2 sentence) justificatin right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked While "[Yes] " is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providded a proper justification is given (e.g., "error bars expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way,
the checklist one of the factors in their evaluation.
reported because it would be too computationally
"[No] " or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer often more nuanced, so please just use your best, judgment and write justification elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction state the proposed game-theoretic and uncertainty- multi-agent framework and its intended scope; these claims are con- sistently supported by the reported benchmark results and analysis (see Sections and 3).
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
54

---

Answer: [Yes]
Justification: The paper discusses limitations such as additional inference-time over- head from multi-agent debate, scalability the number of agents/rounds grows, and dependence on uncertainty estimation and evaluator reliability (see Section 5 and Ap- pendix/Supplementary).
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the puper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faireess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proof's
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: The paper is primarily framework and empirical study and does not introduce formal theoretical theorems requiring assumptions- and oroof style results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
55

---

Answer: [Yes]
Justification: The paper specifies the agent roles, Seeate/game-thhoreei protocol, uncertainty- aware decision rules, evaluation benchmarks, and inference settings; additional imple- mentation details provided in the appendix/supplementar (see Section 3 and Ap- pendix/Supplementary).
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: To preserve anonymity at submission time, the code repository is not made public; however, the paper and supplementary provide detailed descriptions of the method and experimental settings to support faithful reproduction.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (bttps://nips. /c/
public/ gwwggs/iedseeeee Stw.uidddee//eeeeuunonooo Submi ss ionP olicy) for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
56

---

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https} //nips.co ct/ http:ccccpbccccgggg.cccc des CodeSubmm ss onPol icyy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: The paper specifies the evaluated datasets/benchmarks and splits, base model choices, debate configurations (e.g., number of agents/rounds), inference/decoding settings, and evaluation metries; extended details appear in the appendix/supplementary (see Section 3 and Appendix/Supllementtry)
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Because multi-agent inference with large VLM backbones is computationally expensive, the reported results follow standard benchmark evaluation protocols without multi-seed repeats; the paper instead emphasizes consistent improvements across diverse benchmarks and settings.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula,
call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the eтror bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
57

---

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper reports the compute setup for open-source runs (GPU type) and indi- cates API-based evaluation for closed-source models, and provides discussion of inference- time overhead/cost from multi-agent debate (sce Section 3 and Appendix/Suplemmentuy)
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics httph//tpp.....aoooo..o/ooooo//a
Answer: [Yes]
Justification:
The work uses publicly available benchmarks and standard evaluation practices,
does not involve human subjects sensitive personal data, and is presented in way that preserves author anonymity, consistent with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper discusses potential positive impacts (improving reliability and transparency of multimodal reasoning systems) and potential negative impacts (dual-use concerns of stronger generative reasoning systems and increased compute footprint), with relevant discussion Section 5.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate
58

---

to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning conrectly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The submission does not release new pretrained models scraped datasets; therefore, there is no additional release-relates misuse risk requiring specific safeguards beyond standard responsible reporting.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The paper credits the original sources for the benchmarks and models used and cites them appropriately: where relevant, dataset/model usage follows their stated terms, and references provide the necessary provenance information.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use the package should be provided. For popular datasets, paapersw's swithcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
59

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not introduce or release new datasets, models, or code assets part of the submission.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The research does not involve crowdsourcing or studies with human subjects; all evaluations are conducted on existing public benchmarks.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: No human subjects, personal data, or participant studies are involved this work, IRB approval and participant risk disclosure not applicable.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
60

---

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
The method relies vision-language/lagge language model agents components for proposing, critiquing, and refining reasoning traces, and the paper describes their roles and configurations within the multi-agent framework (see Sections 2 and 3).
Justification:
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM (attt:::/ttttuuueteeee http:///eeeeeiissscccooeeeeeeeee LLM) for what should should not be described.
61


Reviews:
[{"review_id": "3Qy3WejWaG", "reviewer": "Reviewer_55Wy", "summary": "This paper propose GAM-Agent, a game-theoretic and uncertainty-aware multi-agent collaboration framework aimed at enhancing complex visual reasoning in vision-language models (VLMs). Rather than relying on single-agent or traditional ensemble strategies, GAM-Agent leverages specialized base agents focused on visual perception subtasks and critical agents tasked with verifying logic and factuality. These agents interact via structured claim-evidence tuples and share uncertainty estimates, enabling dynamic, multi-round debates that are triggered when ambiguity or disagreement is detected. The framework’s modular debate process refines predictions and improves interpretability.", "questions": "1.\tThe model may produce hallucinated answers with high confidence. Since the weighting scheme $w \\propto exp(-\\beta U)$ assigns higher influence to low-uncertainty agents, this could amplify the impact of confident but incorrect agents. The authors should explicitly discuss and empirically evaluate this potential risk.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strength: \nGAM-Agent formulates agent interactions as a non-zero-sum game, with both base and critical agents contributing to reasoning. This structure brings a principled and interpretable approach to integrating diverse expert analyses and critiques. The experimental setups are adequately documented, with detailed baseline comparisons, public benchmarks, and multiple VLM backbones. The paper is clearly written and well-organized, making methodological innovations and empirical results easy to trace and interpret.\nWeakness:\n1.\tIn initial integration phase, if the generation probabilities are unavailable, the proposed GAM-Agent adopts a semantic marker-based strategy to assess uncertainty. However, the semantic markers (e.g. “maybe” etc.) are weakly correlated with true errors and are rarely found in imperative responses.\n2.\tThe paper derives an optimal weight $W^*$ that jointly considers agent uncertainty and semantic consistency (Eq. 34), but the actual implementation uses a simplified heuristic $w \\propto exp(-\\beta U)$ (Eq. 2, Eq. 35), ignoring the **cooperative reward term**$\\lambda \\sum \\mathrm{Sim}$. This weakens the claimed game-theoretic foundation, as the final system does not fully realize the proposed coordination mechanism. Clarifying this trade-off in the main text would help readers better understand the extent to which the theoretical framework informs the practical performance gains.\n3.\tThe model may produce hallucinated answers with high confidence. Since the weighting scheme $w \\propto exp(-\\beta U)$ assigns higher influence to low-uncertainty agents, this could amplify the impact of confident but incorrect agents.", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer 55Wy,\n\nThank you very much for your valuable and insightful comments. We have carefully prepared the following responses to address your concerns in detail. It is our sincere hope that our response could provide you with a clearer understanding of our work. If you have any further questions about work, please feel free to contact us during the discussion period.\n\n**Q1. The semantic markers (e.g., “maybe”, etc.) are weakly correlated with true errors and are rarely found in imperative responses.**\n\n**A1**: We introduce the semantic token-based strategy ($\\Phi_{isem}$) primarily as a supplementary and backup solution for specific and necessary scenarios. As you pointed out, its main design rationale is to handle black-box/closed-source VLMs, such as GPT-4o used in our experiments, from which we cannot access internal generation probabilities. For such API-based models, $\\Phi_{isem}$ becomes a feasible alternative for quantifying uncertainty. We have already clarified this application context and its limitations in Section 2.2.1 on Page 4 of our paper, aiming to specify that it is not the preferred choice, but rather a pragmatic strategy.\n\nAlthough $\\Phi_{isem}$ has theoretical limitations, our experimental results provide strong evidence for its practical effectiveness. As shown in Table 1 and Table 2, when our GAM-Agent is applied to GPT-4o, it still achieved significant performance gains (e.g., **2.94%** on the MMMU dataset and **3.60%** on V*Bench), even when relying on the relatively simple $\\Phi_{isem}$ strategy. This outcome demonstrates that the core strength of our GAM-Agent, i.e., its collaborative debate mechanism, is robust enough to effectively leverage such imperfect uncertainty signals to identify and resolve key cognitive disagreements. Acknowledging your concern, we have more explicitly emphasized the backup nature and applicable scope of $\\Phi_{isem}$ in the revised paper. Furthermore, we have highlighted the exploration of more robust uncertainty quantification methods for black-box models as a valuable and important avenue for future work.\n\n**Q2. The gap between the optimal weight formula derived theoretically in the paper (taking into account both uncertainty and semantic consistency) and the simplified weight update method adopted (mainly based on uncertainty).**\n\n**A2**: Thank you for your valuable observation. The dynamic weight update protocol (Eq. 35) we employ is a specific design choice to achieve computational efficiency and stable performance, the theoretical basis of which is already detailed in Appendix D.4. While the complete theoretical optimal weight equation (Eq. 34) is comprehensive, it necessitates the calculation of pairwise semantic similarities ($Sim(R_i, R_k)$) among all agents in every round. This presents a significant computational burden, especially under multi-agent and multi-round scenarios. Therefore, our GAM-Agent adopts a more efficient and robust fashion that directly leverages uncertainty, i.e., the most critical signal, for dynamic weight allocation, which is computationally feasible and effective.\n\nCrucially, the dynamic weighting protocol we implemented is not an arbitrary heuristic. It remains strictly rooted in our game-theoretic optimization framework. As we have demonstrated with a full derivation in **Appendix D.4 (Pages 30-31)**, Eq. 35 is the precise analytical solution (Eq. 40) to the optimization of a simplified total utility function, specifically, one that considers only individual agent contributions (i.e., setting $\\lambda=0, \\gamma=0$), after introducing entropy regularization. This proves that our weighting mechanism still adheres to the game-theoretic principle of maximizing system utility. As per your suggestion to enhance clarity, **we have already moved this derivation and the explanation of its connection to the full theoretical model (Eq. 34) into the revised paper**. This revision is intended to make it clearer for readers to see how our theoretical framework directly informs the practical, high-performance implementation of the system.\n\n**Q3. The risk of models creating an “illusion of confidence” and being amplified.**\n\n**A3**: Our GAM-Agent is designed with multi-level mechanisms to mitigate this issue. Firstly, the **Critical Agents** serve as our core line of defense, tasked with independently scrutinizing and challenging the outputs of the Base Agents, a process illustrated in Figure 1. Secondly, our debate-triggering mechanism relies not only on overall uncertainty but also on an inter-agent **Conflict Score**; a confidently wrong assertion is likely to conflict with other agents' views, triggering a review due to high conflict even if its self-assessed uncertainty is low. Finally, our **Evidence Mapping and Claim Parsing (P&M) modules** require all claims to be grounded in specific visual evidence, creating a high barrier for unsubstantiated hallucinations.\n\nRegarding the evaluation of this risk, we have already provided a detailed discussion in **Appendix K.2 (Unsuccessful Case, Pages 46-47)**. We deliberately present two such failure cases, for instance, in the 'more colorful' image problem (Table 8), where multiple agents confidently converged on an incorrect consensus by focusing on color saturation over diversity. To more directly address your concern, **we have added a new 'Limitations' section in the revised paper.** This section provides a more focused discussion on the risk of amplifying 'confident hallucinations' and summarizes the multi-level mitigation strategies outlined above that are embedded in our GAM-Agent framework."}, {"review_id": "DR5ykfPPwF", "reviewer": "Reviewer_fdos", "summary": "GAM-Agent is a multi-agent framework intended to enhance the robustness, interpretability, and accuracy of vision-language models on complex multimodal tasks. The framework attempts to model the visual reasoning process as a non-zero-sum game between multiple agents to reach a consensus. It hinges on uncertainty-aware interactions between two types of agents: Base Agents and Critical Agents. The Base Agents are specialized visual reasoning experts that provide initial interpretation and supply evidence on tasks such as object recognition, scene description, and textual analysis of images. Critical Agents scrutinize these outputs to evaluate accuracy, coherence, and completeness. The interactions between these cohorts is arbitrated by quantified uncertainty in an iterative fashion, intended to initiate debate when conflict is detected and progressively reduce ambiguity until the system can arrive at a consensus. Initial responses from Base Agents undergo Claim Parsing (unstructured responses into structured tuples) and Evidence Mapping (visual grounding) before an Uncertainty Quantification. The Debate Controller & Integrator orchestrates this process an initiates debate where required. The authors compared GAM-Agent against 5 other multi-agent frameworks (DMAD, DMPL, ChatEval, MAD, and DebUnc) using 5 different VLMs (Qwen2.5VL-7B and 72B, InternVL3-14B and 78B, and GPT-4o) and 4 public benchmarks: MMMU, MMBench, MVBench, and V*Bench. The reported results show modest performance gains for all models types.", "questions": "The related work section should be expanded and improved. DebUnc isn't even mentioned and yet it has nearly identical scores to GAM-Agent - which points to uncertainty as the key to boosting performance. This deserves further exposition. \n\nBoth result tables should be cleaned up and more clearly explained. What is the Base (Ori) framework? SEAL should be explained and attached to V*Bench. \n\nWhy were Gemini 2.0 Pro, Gemini 2.5 Pro Preview, and Claude 3.7 included in A.1 for Cost and Performance Analysis but not in results? Why not include any of the o-series models (or any other reasoning model) as reference?\n\nIt seems that game theory is underutilized here - Am I understanding the implications of Appendix D.4 correctly? If so, is there a way to re-incorporate collaborative and penalty incentives as part of an ablation or toy demonstration?", "limitations": "Given the breadth of this topic, I believe that the limitations of the proposed method should be more clearly discussed in the main body of the text. For instance, given the supremacy of large, closed-source models like GPT-4o, under what conditions will the semantic marker-based strategy be less effective? Might larger, \"reasoning\"-style models relegate some of this work to pulling performance out of lighter weight models?", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "GAM-Agent incrementally builds on work in the multi-agent space aimed at improving visual reasoning. Framing the reasoning process as a non-zero-sum game is intriguing. Including visual grounding alongside uncertainty estimates as part of this framework is logical based on other multi-agent and VLM work. The reported results tout improvement in small-to-mid scale models, which is an appropriate place for the work to live given that multi-agent, multi-turn systems can be computationally heavy to run (i.e. lightweight models would be preferred in this paradigm). Moreover, the strength of the system appears to be uncertainty quantification and that ability is significantly limited when generation probabilities are unavailable (as they almost certainly would be with the stronger API-based models). \n\nThat all being said, the performance gains are very incremental (within ~1% of the other top performing multi-agent framework, DebUnc, which, also uses uncertainty to improve agent communication). Furthermore, since the release of o-series models is discussed as shining a light on \"the challenge of fine-grained visual grounding,\" I feel it should be at least addressed that these models far outperform anything discussed here (currently sitting at 82.9% on MMMU) - so this paper should be positioned as a way to improve smaller models (though a test against more open \"reasoning\"-style models would be a welcome inclusion as well).\n\nThe main weakness of this paper, however, is that the most intriguing strength, framing reasoning between VLMs as a non-zero-sum game, is under explored. If I understand correctly, the current implementation doesn’t appear to meaningfully alter the behavior beyond what a soft-max on negative uncertainty would already do. Appendix D.4 says the implemented weight updates simplifies the full cooperative objective \"by omitting explicit collaboration rewards and system penalties in the weight calculation.\"", "was_revised": true, "final_justification": "The authors have addressed my questions and concerns regarding the key contributions of the work, its broader placement in the field, and its limitations. The proposed revisions draw a path from the game theoretic approach to its use at various levels of system design. They have demonstrated performance/efficiency gains and more have put forward a valuable touchpoint for multi-agent systems. The new experiments are instructive and the proposed revisions to the text will make for a much stronger submission that will be of use to the community.", "rebuttal": "Dear Reviewer fdos,\n\nThank you very much for your insightful and valuable comments! We have carefully prepared the following responses to address your concerns in detail. It is our sincere hope that our response could provide you with a clearer understanding of our work. If you have any further questions about our work, please feel free to contact us during the discussion period.\n\n**Q1&W1&W2. On Related Work (DebUnc), Efficiency, and Performance (GAM-Agent vs DebUnc).**\n\n**A1**: We thank the reviewer for pointing out the omission of DebUnc. Our initial version focuses on non-zero-sum game modeling and visual reasoning, and therefore does not adequately cover this related foundational theory. We have now expanded the Related Works section to explicitly discuss DebUnc and its relationship to our GAM-Agent.\nIn the revised paper, we have expanded our Related Works section to clarify the relationship and key distinctions between our GAM-Agent and DebUnc, adding the following content:\n\n\"Recently, methods like DebUnc, which leverage uncertainty to facilitate multi-agent communication, have shown excellent performance in our experiments, further confirming that uncertainty is a key signal for enhancing collaborative reasoning. However, the core distinction of GAM-Agent is that it embeds uncertainty as an optimizable variable within a formal game-theoretic utility function, thereby driving agent collaboration more fundamentally and robustly to achieve superior performance.\"\n\nAlthough DebUnc also incorporates uncertainty to improve agent communication, it adopts a **symmetric, fixed three-round debate structure** similar to the multi-round competition mechanism. In contrast, **GAM-Agent introduces asymmetric roles** (base and key agents) and models the reasoning process as a **non-zero-sum game** where **debates are triggered dynamically only when needed**. This leads to structurally distinct behavior and lower average cost.\n\nAlthough our GAM-Agent slightly outperforms DebUnc on large models (~1%), its main advantages are **Pareto efficient reasoning**, i.e., significantly fewer LLM calls (token cost is ~2.2x that of DebUnc’s 9 fixed calls) and **enhanced interpretability through a structured evidence base**. Regarding the o-series models, we agree that they improve overall performance (e.g., 82.9% on MMMU). Our experiments with GPT-4o show consistent improvements (+2–3%) even on these powerful models. Thus, our GAM-Agent improves performance on small and medium models while also improving SOTA VLMs. Meanwhile, our GAM-Agent achieves +5–6% improvement on these models with lower overhead. In summary, GAM-Agent provides a more efficient and easier-to-interpret alternative to symmetric frameworks such as DebUnc and ChatEval.                   |\n\n**W3. On the Implementation and Utilization of the Game-Theoretic Framework**\n\n**A3**: The non-zero-sum game in GAM-Agent is **not** limited to a soft-max on negative uncertainty; it is the blueprint for both the weighting rule *and* the interaction protocol. The full theoretical weight is : $$ w_i=\\frac{\\exp(-\\alpha\\,u_i+\\beta\\,r_i)}{\\sum_j\\exp(-\\alpha\\,u_j+\\beta\\,r_j)}, $$\n\nwith uncertainty \\$u\\_i\\$ and a cooperation reward \\$r\\_i\\$ (e.g., evidence/peer alignment). A pure soft-voting baseline would use $$ y=\\sum_i\\text{softmax}(-u_i)\\,a_i, $$ which is symmetric and lacks structured cooperation. For efficiency we drop \\$r\\_i\\$ and obtain $$ w_i=\\frac{\\exp(-\\alpha\\,u_i)}{\\sum_j\\exp(-\\alpha\\,u_j)}, $$ but this weight is applied **inside a role-asymmetric, non-zero-sum loop**. Concretely: * **Collaboration reward → generate-verify**: the theoretical term \\$\\lambda!\\sum\\_j!w\\_j\\mathrm{Sim}(R\\_i,R\\_j)\\$ is realized architecturally since base agents must defend their answers before **critical agents** in multi-turn debates, forcing semantic alignment. * **System penalty → dynamic triggering**: the penalty \\$-\\gamma U\\_{\\text{sys}}\\$ is enacted by a controller that launches a debate only when global uncertainty is high and stops once confidence is sufficient, hence penalizing high-uncertainty states with extra computation. These structural embodiments of the game principles meaningfully change behaviour. Our ablation (Fig. 3) shows that removing the debate (“w/o Debate”), i.e., reducing the system to an initial soft-voting integration, drops MMBench accuracy from 88.80 % to 84.52 % (-4.28 pp). Thus, the architectural realisation of the non-zero-sum game delivers clear gains beyond a simple uncertainty soft-max.\n\n\n**Q2. What is the Base (Ori) framework? SEAL should be explained and attached to V*Bench.**\n\n**A4**: \"Base (Ori)\" stands for \"Base Model's Original Performance\", which is the performance score of a single vision-language model (e.g., Qwen2.5VL-7B) answering questions directly, without the application of any multi-agent collaboration framework. We have noted this in the table captions, for instance, the caption for Table 1 states, \"Values in parentheses denote the improvement in accuracy over the respective base model (Ori)\". The core purpose of establishing this benchmark is to quantify the precise performance gains brought by all multi-agent frameworks, including our GAM-Agent. Considering SEAL is a published method on V*Bench, we include its result in Table 2 as an important external reference point to gauge whether the performance level achieved by our GAM-Agent is competitive, particularly when equipped with top-tier models like GPT-4O. We have added more explicit definitions for both \"Base (Ori)\" and \"SEAL\" to the captions of Tables 1 and 2 in the revised paper.\n\n\n**Q3. Why are some methods only in A.1? Why not include the o-series models (or any other reasoning model) ?**\n\n**A5**: To obtain fair and reproducible evaluations, we do not include o-series models since they are closed-sourced and change over time. Besides, the closed-sourced reasoning VLMs (i.e., o-series, Gemini, and Claude series models) do not return token-level logprobs for accurate uncertainty quantization, which is crucial for our GAM-Agent. \n\n(In OpenRouter, among the GPT-series, only the 4o series returns logprobs.)\n\nMoreover, Appendix A.1 aims to conduct a **broad cost-benefit analysis**, introducing numerous expensive commercial API models (like the Gemini series and Claude 3.7) as reference points for performance and price. This highlights the significant economic advantage of our GAM-Agent in matching or exceeding the performance of top-tier commercial services at a fraction of the cost when driving locally deployed open-source models.\n\n\n**Q4. Game theory is underutilized in the implications of Appendix D.4.**\n\n**A6**: The dynamic weighting protocol (Eq. 35) is indeed a simplified version of the full game-theoretic optimal weight (Eq. 34). This is to prioritize computational efficiency and stable performance, which is why the explicit cooperative reward term (i.e., $Sim(R_i, R_k)$) is omitted in our main experiments. To address the question regarding the potential of the full framework and to more clearly demonstrate its theoretical value, we have added a new ablation study in the revised paper to directly compare these two weighting mechanisms. The results of this experiment are summarized in the table below.\n\n| Weighting Strategy                     | Acc. (%)        | Deb. Rounds | Deb. Trig. (%) | Cost (tokens/inst.) |\n| -------------------------------------- | --------------- | ----------- | -------------- | ------------------- |\n| Simplified Weights (Default, Eq. 35)   | 88.80           | 1.76        | 65             | 2500                |\n| Full Game-Theoretic Weights (Eq. 34) | 89.15 (+0.35)   | 1.92        | 68             | 3150                |\n\nThe results in the above table show that applying the full game-theoretic weights (Eq. 34) does indeed lead to a further improvement in accuracy (+0.35%). This empirically validates the theoretical advantage of our complete utility function design, where encouraging semantic consistency with high-confidence peers contributes to a better consensus. However, this accuracy gain is accompanied by a significant computational overhead; the average cost increases by approximately 26% due to the additional pairwise semantic similarity calculations. We therefore maintain that the simplified weighting protocol (Eq. 35) used in our main paper strikes an effective trade-off between performance and computational efficiency. This new analysis further clarifies the rationale behind our designs.\n\n\n**L1. The limitations.**\n\n**A7**: We have added a new \"Limitations\" section to the revised paper to discuss the framework's boundaries more centrally. Regarding the failure conditions of the semantic-marker strategy ($\\Phi_{isem}$) is critical, this strategy's effectiveness diminishes under two conditions: Firstly, when black-box models are fine-tuned to avoid uncertainty-related words (e.g., \"possibly,\" \"maybe\") for a more decisive user experience. Secondly, when more advanced models produce \"confident hallucinations,\" where their coherent but incorrect answers are wrongly assigned a low uncertainty score. This is a risk we frankly discuss in our failure case analysis in Appendix K.2.\n\nWhile a core value of our GAM-Agent is significantly improving the performance of small-to-mid scale large models, experimental results show that even top-tier models like GPT-4o gain consistent and significant performance improvements (from +2% to +4%). For these large VLMs, GAM-Agent's value lies in complementing the cognitive blind spots of a single model by using multiple expert perspectives and providing a valuable review and correction layer through its Critical Agents (e.g., Fact Checker, Logic Checker). Therefore, GAM-Agent is a general collaborative framework designed to enhance the visual reasoning robustness of all models, from lightweight to large-scale."}, {"review_id": "iVqimSElNm", "reviewer": "Reviewer_eVEC", "summary": "This paper introduces a multi-agent reasoning framework for vision-language models, which reformulates the visual reasoning process as a non-zero-sum game between agents. The framework introduces Base Agents and Critical Agents. A key contribution is the uncertainty-aware debate controller, which detects disagreement or ambiguity and triggers iterative multi-round debates, leading to improved predictions. The framework is training-free, and compatible with a wide range of existing VLMs. Extensive experiments on four challenging benchmarks demonstrate consistent accuracy improvements, especially for small and mid-sized models.", "questions": "While the framework is modular and training-free, it would be helpful to clarify whether quantized or distilled models can reduce inference cost without degrading uncertainty estimation or accuracy. Additionally, could lightweight learnable components (e.g., fusion or debate control) improve adaptability? Finally, when using heterogeneous models with different output formats and uncertainty behaviors, how does the system handle potential inconsistencies?", "limitations": "Yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n- The major novel part is the reasoning mechanism. This work models multi-agent collaboration as a non-zero-sum game, guided by uncertainty and claim-evidence alignment. This formulation introduces a new inference perspective for VLMs. The framwork is plug-and-play desigend and operates only at inference mode, which makes it adaptable to various pre-trained VLMs.\n\n- The experimential resutls looks reasonable and impressive. GAM-Agent consistently improves performance over baseline models across all evaluated benchmarks, with significant gains for smaller models and non-trivial improvements for stronger models like GPT-4o.\n\n- The paper includes well-designed ablation studies showing the effect of each component on performance and cost. By structuring outputs into claims and evidence and quantifying agent-specific uncertainty, the system enhances transparency and makes intermediate reasoning traceable.\n\nWeakness\n- Although the framework design is interesting, the inference cost and latency raise some concerns. In particular, the framework introduces substantial runtime overhead due to multiple agent calls and iterative debate rounds. While early stopping is employed, the overall cost remains higher than that of single-agent baselines. Could it be possible to incorporate quantized models, and how would the use of quantized models affect the results?\n\n- While the training-free nature of the framework is a clear advantage, GAM-Agent lacks learnable components and cannot benefit from gradient-based optimization. Incorporating lightweight learnable modules (e.g., for fusion weighting or response selection) could potentially improve adaptability. Could the authors discuss the possibility of integrating such learnable components into the framework?\n\n- This work uses the same model architecture across different tasks. However, when incorporating heterogeneous models (e.g., InternVL, Qwen-VL, LLaMA4, and GPT-4o), the system may encounter inconsistencies in output styles, uncertainty estimation, and claim structure. The current framework does not provide an automated mechanism to normalize or reconcile these differences.", "was_revised": true, "final_justification": "The rebuttal addressed all key concerns, including efficiency, integration of learnable components, and handling of heterogeneous models. The new experiments are convincing and strengthen the paper. I believe this is a meaningful and practical contribution. I maintain my recommendation as Borderline Accept.", "rebuttal": "Dear Reviewer eVEC,\n\nThank you very much for your valuable and insightful comments. We have carefully prepared the following responses to address your concerns in detail. It is our sincere hope that our response could provide you with a clearer understanding of our work. If you have any further questions about work, please feel free to contact us during the discussion period.\n\n**W1: Exploring the feasibility of using quantitative models in the GAM-Agent framework**\n\n**A1**: Our GAM-Agent indeed introduces additional computation due to multi-agent calls and debate rounds. To address this, we have conducted a Cost–Performance Balance Analysis (Appendix A) and a Debate Cost Dynamics Study (Appendix B). Unlike fixed-turn systems like DebUnc (9 calls) or ChatEval (up to 12), GAM-Agent employs dynamic triggering and early stopping, launching debates only when uncertainty is high. As shown in Table S1, easy samples average ~4 calls, and hard cases ~2.2× single-model token cost. \n\n**Table S1**\n| **Dimension**                | **GAM-Agent**                                                | **DebUnc**                                  | **ChatEval**                                                 |\n| ---------------------------- | ------------------------------------------------------------ | ------------------------------------------- | ------------------------------------------------------------ |\n| **Collaboration Structure**  | Asymmetric roles: *Base* + *Critical* agents                 | Symmetric 3-turn debate (CoT-style)         | Symmetric debate; optional *Summarizer* module               |\n| **Debate Trigger Mechanism** | Triggered only when uncertainty/conflict is high             | Always-on: fixed 3 rounds                   | Always-on; round count fixed but cost varies with communication mode |\n| **Interpretability**         | High: visual grounding + structured claim/evidence           | Medium: depends on agent disagreement       | Medium: summarization helps but lacks explicit claim grounding |\n| **Computational Cost**       | Adaptive:• Easy cases ≈ 4 LLM calls • Complex ≈ 2.2× single-model tokens | Fixed:3 agents × 3 rounds = **9** LLM calls | Fixed:• 2 agents × 2 rounds = **4**• 3 agents + 1 summarizer × 3 rounds = **12** calls |\n| **Decision Aggregation**     | Weighted fusion based on agent uncertainty                   | Final-round majority voting                 | Varies: summarizer-guided or heuristic fusion                |\n\nFigure 5 (Page 17) demonstrates that our GAM-Agent achieves strong cost-effectiveness: e.g., Qwen2.5VL-7B improves from 82.6% to 89.0% on MMBench with cost per instruction ~$0.000085, outperforming much larger APIs at ~1/3.5 the cost.\n\nRegarding quantized models, GAM-Agent is fully compatible. Quantization (e.g., INT8) would reduce per-agent cost significantly. While $\\phi_{\\text{igen}}$ may require calibration under low-precision logits, $\\phi_{\\text{isem}}$ remains stable. Moreover, our collaborative protocol helps correct quantized errors, making GAM-Agent a promising solution for low-resource deployment.\n\n**W2: Possibility of integrating learnable components**\n\n**A2**: To demonstrate that GAM-Agent can benefit from lightweight gradient-based modules, we introduce a 2-layer MLP (~80K params) that learns fusion weights from agent-level **uncertainty**, **semantic alignment**, and a **learnable role embedding** (e.g., object recognition vs. OCR expert). Specifically, for each agent, we construct a feature vector: (1) its uncertainty score $U_i$, (2) a semantic alignment score with other agents $Align_i = \\frac{1}{N-1} \\sum_{j \\neq i} Sim(R_i, R_j)$, and (3) a learnable role embedding (e.g., object recognition expert, scene description expert). These features are passed through the MLP to produce softmax-normalized fusion weights. We freeze the backbone (Qwen2.5-VL-7B) and train only the fusion head on 5K QA pairs from **MATH-Vision** using cross-entropy loss. **Results show a modest gain (from 34.7% (training-free) to 34.9%) while preserving GAM-Agent’s efficiency**: the module adds <1% latency and only ≈+0.1× token usage. This experiment, initially introduced in our response to Reviewer og8v, confirms that GAM-Agent can seamlessly incorporate learnable components when data and budget permit.\n\nTable S2 \n\n| **Model Variant**                    | **Accuracy (%)** |\n| ------------------------------------ | ---------------- |\n| Base Qwen 2.5‑VL‑7B                  | 25.6             |\n| + GAM‑Agent (training‑free)          | 34.7             |\n| + GAM‑Agent + MLP Fusion (learnable) | **34.9**         |\n\n\n**W3: Addressing the challenge of heterogeneous models with different confidence scales.**\n\n**A3**: We would first like to clarify that our GAM-Agent framework has already built-in core control mechanisms for handling different information sources and viewpoints, which provide a solid foundation for addressing heterogeneous models. Specifically, our GAM-Agent does not simply aggregate outputs but normalizes the interaction process through a series of modular steps. First, the **Uncertainty Quantification (Φ) module** calculates a standardized uncertainty score, $U_i$, for each agent's output, allowing us to measure the reliability of different information sources with a unified metric. Second, the **Claim Parsing (P) and Evidence Mapping (M) modules** forcibly parse the potentially varied free-text responses from different agents into structured \"claim-evidence\" tuples, thereby masking the output format differences of the underlying models. Most critically, our **dynamic weight allocation protocol**, derived from game theory (e.g., $w_{i}^{(k+1)} \\propto e^{-\\beta U_{i}^{(k)}}$), allocates influence based solely on quantified uncertainty, rather than the model type or its inherent biases. Therefore, even if the agents are heterogeneous, these existing mechanisms ensure that their contributions are evaluated and integrated within a unified and fair framework.\n\nInspired by the reviewer's constructive feedback, we have further designed and implemented a new module in our revised manuscript named **\"Cross-Model Uncertainty Calibration\" (CMUC)** to more precisely address the confidence scale discrepancies among heterogeneous models. The CMUC module operates before the debate phase. By evaluating performance on a small set of validation samples, it learns a specific linear transformation parameter for each heterogeneous model (e.g., Qwen-VL, InternVL, GPT-4o) to calibrate its raw uncertainty output into a unified, empirically-grounded confidence space. To validate the effectiveness of this design, we have added a new experiment to the appendix of our revised paper. We construct a heterogeneous GAM-Agent composed of **InternVL3-14B**, **Qwen2.5VL-7B**, and **GPT-4o-0513**, and tested it on the MMBench_V11_Test benchmark. The results, presented in the table below, demonstrate the effectiveness of the CMUC module. The heterogeneous agent group without calibration shows a slight performance degradation due to mismatched confidence scales. In contrast, after applying CMUC, the heterogeneous group leverages its complementary strengths and outperforms the homogeneous baseline, proving the feasibility and superiority of our GAM-Agent.\n\n**New Experimental Results: Performance of Heterogeneous Agent Setup on MMBench_V11_Test**\n\n| Framework Configuration | Accuracy (%) | Avg. Debate Rounds |\n| :--- | :---: | :---: |\n| **Baseline:** Homogeneous GAM-Agent (all InternVL3-14B) | 90.15% | 2.1 |\n| Heterogeneous GAM-Agent (w/o CMUC module) | 89.85% | 2.5 |\n| **Our New Method:** Heterogeneous GAM-Agent (**with CMUC module**) | **91.25%** | 1.9 |\n\n\n**Q1: While the framework is modular and training-free, it would be helpful to clarify whether quantized or distilled models can reduce inference cost without degrading uncertainty estimation or accuracy. Additionally, could lightweight learnable components (e.g., fusion or debate control) improve adaptability? Finally, when using heterogeneous models with different output formats and uncertainty behaviors, how does the system handle potential inconsistencies?**\n\n**A4**: Regarding quantized models, GAM-Agent is fully compatible with quantization (e.g., INT8), which can reduce inference costs by approximately 30–50% per agent call, as shown in our Cost–Performance Analysis (Appendix A). In preliminary tests, uncertainty estimation (via $\\phi_{\\text{isem}}$) remains stable, with less than 5% degradation in Uncertainty Accuracy (UA) for models such as Qwen2.5VL-7B, and our collaborative protocol further mitigates quantization-induced errors, preserving overall performance.\n\nFor lightweight learnable components, we validated a 2-layer MLP fusion head (~80K parameters) trained on 5K MATH-Vision QA pairs. This module improved accuracy from 34.7% to 34.9%, with minimal latency increase (about +0.1× token usage; see Table S2), confirming GAM-Agent’s capacity to integrate learnable modules for enhanced adaptability when data is available.\n\nFor heterogeneous models, our newly introduced Cross-Model Uncertainty Calibration (CMUC) module normalizes confidence scales across models (e.g., InternVL3, Qwen-VL, GPT-4o) using a linear transformation learned from validation samples. Combined with structured claim-evidence parsing and uncertainty-driven weighting ($w_i^{(k+1)} \\propto e^{-\\beta U_i^{(k)}}$), this ensures consistent and robust integration of outputs. As shown in our new experiment, the heterogeneous agent group with CMUC achieved 91.25% accuracy on MMBench_V11_Test, outperforming the homogeneous baseline (see Appendix, New Experimental Results).\n\nPlease refer to our above responses (A1, A2, and A3) for more details."}, {"review_id": "wOzDwb1Gaa", "reviewer": "Reviewer_og8v", "summary": "This paper proposed a multi-agent system that target complex visual reasoning.\nThe main contribution is to model the reasoning process as an claim evidence linking process and model the interactions between agents as a non-zero-sum game to guide the theory on how to moderate the debate between agents and dynamically adjust reasoning strategies.\nExperiments show that the proposed multi-agent system improves base multimodal models on reasoning tasks.", "questions": "1. Could the author provide more evaluations on the success/failure cases as well as multi-image reasoning benchmarks?\n2. What work would be required to extend the proposed system to more general reasoning scenarios?", "limitations": "The negative impact and the limitations are discussed in the paper.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strength\n\nThe proposed multi-agent system did work well from the experiments.\nThe paper itself contains comprehensive experiments to demonstrate the effect of the proposed system.\n\n\nWeakness\n\nThe structure of the paper makes it hard to read, for example, in the introduction, it is already introduced that the proposed system models multi-agent reasoning as a non-zero-sum game, but this claim is not made any clearer in the main paper until the appendix. I would suggest put down some comment about this non-zero-sum game in the main paper.\n\nSection 2.3 to 2.5 is confusing as they describes the procedure of the multi-agent system yet lacks an overview of the system. I would suggest adding an Algorithm for this part of the paper to make the system clearer to the reader\n\nThe paper gives comprehensive experiments on ablations and comparison with other models, yet lack discussions on the failure/success mode of the proposed multi-agent system, appendix K provides some examples, but they are not much informative. I would suggest provide some examples where the base model fails yet the proposed multi-agent system improves and examples where the multi-agent systems consistently fails.\nAdditionally, from appendix K, it seems the multi-agent system would fail when comparing multiple images, I would suggest adding benchmarks that directly target this scenario, such as [R1] and [R2].\n\nThe agent roles in the proposed multi-agent system seems to be hard coded and designed speific to the vision reasoning task (OCR agent, object recognition agent, etc). This gives concerns on whether the proposed system is generalizing or simply overfitting the vision reasoning benchmarks? If the game theory behind this system can generalize, I would suggest try to extend this to more general reasoning tasks.\n\nNit: in line 117, the uncertainty is defined based on a function rho, but there's no description of that function else where in the paper. in line 174, there's an empty citation\n\n\n[R1] MuirBench: A Comprehensive Benchmark for Robust Multi-image Understanding\n[R2] Benchmarking Multi-Image Understanding in Vision and Language Models: Perception, Knowledge, Reasoning, and Multi-Hop Reasoning", "was_revised": true, "final_justification": "I raised my score to a positive rating considering the authors made some changes to improve the readability of the manuscript.", "rebuttal": "Dear Reviewer og8v,\n\nThank you very much for your valuable and insightful comments. We have carefully prepared the following responses to address your concerns in detail. It is our sincere hope that our response could provide you with a clearer understanding of our work. If you have any further questions about work, please feel free to contact us during the discussion period.\n\n**W1. Put down some comment about this non-zero-sum game in the main paper.**\n\n**A**：Thank you very much for your feedback on the paper's structure. We agree that clarifying the non-zero-sum game formulation earlier would improve readability. Initially, we focus the Methodology section on the overall framework, key modules (e.g., uncertainty quantification and evidence mapping), and experimental results to maintain flow, placing complex game-theoretic derivations (e.g., utility functions and Nash equilibria) in the Appendix to avoid overwhelming readers with math. While space-limited details in the paper, we reference core concepts (e.g., agent utilities, uncertainty-driven payoffs) in Sections 2.3~2.5. In the revised paper, we have added a concise paragraph in Section 2.3 summarizing key elements like the utility function and equilibrium dynamics to better bridge the introduction and Appendix.\n\n\n**W2**: Sections 2.3 to 2.5 are confusing as they describe the procedure of the multi-agent system yet lacks an overview of the system. I would suggest adding an Algorithm for this part of the paper to make the system clearer to the reader \n\n**A2**: We agree that Sections 2.3–2.5 could benefit from a clearer system overview. Due to page limits in the initial submission, we rely on Figure 1 for a high-level pipeline visualization, but adding pseudocode would complement it by outlining the iterative steps, such as uncertainty quantification, evidence mapping, debate, and convergence, to better guide readers through the multi-agent procedure.\n\n\n**W3&Q1: Analysis of Success/Failure Modes and Multi-Image Reasoning Evaluation**\n\n**A3**: **Discussion of Failure/Success Modes**: Thank you for the suggestion. We have added a structured case (**Table B**) to illustrate how GAM-Agent resolves a base model failure due to visual ambiguity (e.g., poor lighting, distance). In this V*Bench example (*sa_13337.jpg*), the base model gives up due to uncertainty, while GAM-Agent integrates multi-agent evidence and converges on the correct answer (*A. black*) via debate. Due to policy constraints this year, we are unable to upload images or external links. Thus, we refer to dataset image IDs directly. This case reflects a common success mode, i.e.,**resolving partial ambiguity via collaboration**, while failure persists in cases with high, conflicting uncertainty across agents. We have clarified these further in Appendix K.\n\n**Multi-Image Reasoning and Benchmark Coverage**: We select the widely used **MuirBench[S1]** benchmark for its strong focus on multi-image alignment and real-world ambiguity. To further evaluate generalization in visual-symbolic reasoning, we have also included **MATH-Vision[S2]**. The evaluation results are shown in **Table A**（**all of which can be fully reproduced with the executable code included in the supplementary material**). MuirBench contains ~2,600 multi-image questions (avg. 4.3 images per question; ~11,264 total), and MATH-Vision includes 3,040 math problems, each paired with a diagram. We report full results for **Qwen2.5-VL-7B** on both datasets. For **Qwen2.5-VL-72B**, due to computational constraints near the deadline, we have only evaluated 2,419 MuirBench samples and 1,843 ChatEval runs, the latter selected as the **first 1,843 samples in ID order**, given its higher computational cost. GAM-Agent achieves consistent gains over both the base model and other debate frameworks (e.g., +4.7 on MuirBench, +9.1 on MATH-Vision for 7B). In the revised paper, we promise to report the results of the whole MuirBench benchmark.\n\n**Table A**\n\n| Benchmark   | Model          | base | + ours | + DebUnc | + ChatEval |\n| ----------- | -------------- | ---- | ------ | -------- | ---------- |\n| MuirBench   | Qwen2.5-VL-7B  | 58.4 | 63.1   | 61.5     | 60.2       |\n|             | Qwen2.5-VL-72B | 70.2 | 73.6   | 72.3     | 71.9       |\n| MATH-Vision | Qwen2.5-VL-7B  | 25.6 | 34.7   | 32.1     | 30.7       |\n|             | Qwen2.5-VL-72B | 42.6 | 45.9   | 43.7     | 43.1       |\n\n\n\n**Table B**\n\n| **Component**                                             | **Details**                                                  |\n| --------------------------------------------------------- | ------------------------------------------------------------ |\n| **Image**                                                 | vstar_bench/direct_attributes/sa_13337.jpg<br>(Scene: Street with buildings, garbage pile, people; cap dark but obscured by distance/lighting.) |\n| **Question**                                              | What is the color of the man's cap?<br>Options:<br>A. black<br>B. white<br>C. green<br>D. red |\n| **Base Model Output**                                     | Street scene with buildings, garbage, people. Cap details unclear due to visibility. Cannot determine color; none of options confirmed. |\n| **GAM-Agent Output** (Integrated Analysis & Final Answer) | Experts: Object Recognition notes cap unclear from distance/lighting. Scene Description suggests dark (black/gray). Text/OCR irrelevant.<br>Scene expert's dark inference aligns best with options, despite uncertainty.<br>**Final Answer: A. black** |\n| **Partial Critique Opinions** (From Critical Agents)      | - **Claim 1 (Object Recognition):** Partially accurate. Valid visibility note; add specifics on quality/angle. Conf: 80%<br>- **Claim 2 (Scene Description):** Partially accurate. Dark color aligns; mention resolution uncertainty. Conf: 75%<br>- **Claim 3 (Text/OCR):** Inaccurate (irrelevant). Correct, no text; remove claim. Conf: 90% |\n| **Analysis**                                              | Base fails on unresolved visibility uncertainty (\"cannot determine\"). GAM-Agent succeeds via agents' evidence/debate, reducing uncertainty to correct answer (black), despite ongoing visibility critiques. |\n\n[S1] Wang et al. MuirBench: A Comprehensive Benchmark for Robust Multi-image Understanding\n\n\n[S2] Wang et al. Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset\n\n\n**W4&Q2: Generality of GAM-Agent and Extension to Broader Reasoning Tasks**\n\n**A4**: GAM-Agent’s core engine is expressly designed for general-purpose, collaborative reasoning, and its principal components remain unchanged across tasks. The uncertainty quantification module ($\\Phi$), including both the generation-based $\\Phi_{\\text{gen+}}$ and the semantic-marker-based $\\Phi_{\\text{sem}}$, is modality-agnostic and applies to any LLM-generated text. Similarly, the debate engine ($D$) operates purely on abstract notions of uncertainty and consensus, while its utility and dynamic-weighting mechanisms are grounded in general game-theoretic principles that do not rely on task-specific assumptions. As a result, extending GAM-Agent to a new reasoning domain requires no changes to the underlying algorithm; the only substantive work involves defining a task-appropriate panel of expert agents through prompt engineering—that is, crafting task-specific instructions to guide each agent’s behavior. To further demonstrate the generality of our framework, we have applied GAM-Agent to MATH-Vision and MuirBench, covering both visual-symbolic and multi-image reasoning. As shown in Table A, GAM-Agent achieves strong and consistent gains beyond the original vision task setting.\n\n\n**W5: In Line 117, the uncertainty is defined based on a function rho, but there's no description of that function else where in the paper. In line 174, there's an empty citation.**\n\n\n**A5**: Regarding the missing definition of the ρ function in Line 117, we acknowledge this omission and have added the following clarification in the revised paper. $\\rho(R_i) = \\frac{1}{|R_i|} \\sum_{w \\in W} \\text{weight}(w) \\cdot \\text{count}(w, R_i)$where $W$ is the set of uncertainty markers, $\\text{weight}(w)$ denotes the marker’s intensity, and $\\text{count}(w, R_i)$ is its frequency in the response. Concerning the empty citation in Line 174, which is intended to support the statement about adaptive weighting aligning with game-theoretic principles, we have removed this reference in the revised paper since the reasoning is self-explanatory and the citation is unnecessary."}]

Author rebuttal:

Meta-review: This paper proposes a multi-agent framework GAM-Agent, which is designed to enhance the robustness, interpretability, and accuracy of vision-language models on complex multimodal tasks. The reviewers believe the paper is novel, principled, and meaningful with certain effectiveness. I agree with the reviewer and suggest accepting the paper.
Decision: Accept (poster)