Vision-Zero: Scalable VLM Self-Evolution via Multi-Agent Self-Play
Authors: Qinsi Wang
Bo Liu
Tianyi Zhou
Jing Shi
Yueqian Lin
Yiran Chen
Hai Helen Li
Kun Wan
Wentian Zhao
Venue: ICLR 2026 Poster (2026)
Keywords: Language Gamification
Post-Training
Vision–Language Models
Self-Play Optimization
Primary area: optimization
TL;DR: Vision-Zero trains VLMs to improve themselves through multi-agent self-play on arbitrary input images, achieving cheaper and more scalable gains that outperform annotation-heavy baselines on reasoning and understanding tasks.

Abstract: Although reinforcement learning (RL) can effectively enhance the reasoning capabilities of vision–language models (VLMs), current methods remain heavily dependent on labor-intensive datasets that require extensive manual construction and verification, leading to extremely high training costs and consequently constraining the practical deployment of VLMs. 
To address this challenge, we propose **Vision-Zero**, *a domain-agnostic self-play framework that generates visual deduction games from diverse images for scalable VLM training without human annotations.*
Specifically, Vision-Zero encompasses three main attributes:
(1) **Strategic Self-Play Framework:**
Vision-Zero trains VLMs in "Who Is the Spy"-style games, where the models engage in strategic reasoning and actions across multiple roles. Through interactive gameplay, models autonomously generate their training data without human annotation.
(2) **Gameplay from Arbitrary Images:** Unlike existing gamified frameworks, Vision-Zero can generate games from arbitrary images, thereby enhancing the model’s reasoning ability across diverse domains and showing strong generalization to different tasks.
We demonstrate this versatility using three distinct types of image datasets: CLEVR-based synthetic scenes, charts, and real-world images.
(3) **Sustainable Performance Gain:** We introduce Iterative Self-Play Policy Optimization (Iterative-SPO), a novel training algorithm that alternates between Self-Play and reinforcement learning with verifiable rewards (RLVR), mitigating the performance plateau often seen in self-play-only training and achieving sustained long-term improvements.
Despite using label-free data, Vision-Zero achieves state-of-the-art performance on reasoning, chart question answering, and vision-centric understanding tasks, surpassing other annotation-based methods.
Models and code will be released upon acceptance.

Published as a conference paper at ICLR 2026
VISION-ZERO: SCALABLE VLM SELF-EVOLUTION VIA MULTI-AGENT SELF-PLAY
Qinsi Wang',
Bo Liu", Tianyi Zhou', Jing Shi', Yueqian Lin', Yiran Chen', Hai Helen Li!, Kun Wan Wentian Zhao
Duke University National University of Singapore University of Maryland *Adobe Inc. http://gghhhhccommaaagggiiiiiiio
ABSTRACT
Although reinforcement learning (RL) has emerged promising approach for improving vision-language models (VLMs) and multimodal large language models (MLLMs), current methods rely heavily on manually curated datasets and costly human verification, which limits scalable self-improvement in multimodal systems. To address this challenge, we propose Vision-Zero, a label-free, domain aggostic multi-agen agent self eely-play framework for self-evolving VLMs through competitive visual games generated from arbitrary image inputs. Specifically, Vision-Zero encom- passes three main attributes: (1) Strategic Self-Play Framework: Vision-Zero trains VLMs in "Who Is the Spy"-style games, where the models engage in strate- gie reasoning and actions across multiple roles. Through interactive gameplay, models autonomously generate their training data without human annotation. (2) Gameplay from Arbitrary Images: Unlike existing gamified frameworks, Vision- Zero can generate games from arbitrary images, thereby enhancing the model's reasoning ability across diverse domains and showing strong generalization different tasks. We demonstrate this versatility using three distinct types of image datasets: CLEVR-based synthetic scenes, charts, and real-world images. (3) Sus- tainable Performance Gain: We introduce Iterative Self-Play Policy Optimization (Seraativ-SPOO, a novel training algorithm that alternates between Self- Self-Play and reinforcement learning with verifiable rewards (RLVR), mitigating the performance plateau often seen in self- self-play-only training and achieving sustained long-term improvements. Despite using label-free data, Vision-Zero achieves tate-offtheeart performance on reasoning, chart question answering, and vision-centric under- standing tasks, surpassing other annotation- baased methods. Models and code have been released at here
INTRODUCTION
Recent breakthroughs in vision-language models (VLMs) have demonstrated remarkable capabilities across diverse multimodal tasks (Achiam et al,) 2023 Team et al. 2023). However, current training
paradigms face fundamental scalability constraints:
they depend heavily human-curated data
through supervised fine-tuning (SFT) (Liu et al. 2023), reinforcement learning from human feedback (RLHF) (Ouyang et al.] 2022, Sun al.) (023), and carefully engineered reward functions for reinforcement learning with verifiable rewards (RLVR) (Guo et al. (025). This dependency creates two critical bottlenecks. First, a data scarcity problem the extraordinary cost of multimodal annotation limits both scale and diversity of training data, with datasets like COCO Attributes requiring $60,480 for 200,000 objects (Patterson & Hays 2016). Ego4D consuming over 250,000 annotation hours (Grauman et al. 2022) 2. and Visual Genome mobilizing 33,000 annotators (Krishnal al.) 2017). Second, knowledge falling-modee capabilities remain fundamentally bounded by human-generated supervision, preventing VLMs from discovering strategies beyond human expertise.
Self-Play offers solution by eliminating human supervision through competitive dynamics (Silver et al. 2017 Tesauro 1995). In self-play, models learn by engaging in competitive interactions with
"Corresponding authors.

---

Published as a conference paper at ICLR 2026
The red cube to the left of the cylinder.
Where is the red cubc relative to the cylinder?
<thbnk
<thinko
red the left cylinder.
Left.
(a) Supervised Learning
(b) Reinforcement Learning
(c) Vision-Zero
Figure 1: Vision-Zero Paradigm. (a) Supervised learning depends on human-curated reasoning
trajectories; (b) Reinforcement Learning, although enabling models to autonomously learn reasoning processes via validated rewards, still relies heavily on expert-designed question-answe pairs. (c) In contrast, Vision-Zero is novel reff-improvement paradigm entirely independent of human experience. It constructs self-play games by leveraging image pairs that exhibit visual differences. Through the interactive and strategic game, Vision-Zero continuously generates training data for VLMs, enabling the model to achieve scalable self-improvement.
copies of themselves, receiving automatic feedback based on the outcomes of each interaction. As the model improves, its opponents correspondingly advance, thus maintaining a consistently challenging learning environment and driving continuous improvement. By removing the need for human supervision during data generation, self-play has already surpassed the knowledge ceiling across many domains: from DD-Gammo'' 's backgammon supremacy (Tesauro) 1995) to AlphaGo's conquest of Go (Silver et al. 2016; 2017) to OpenAl Five's mastery complex team coordination (Berner et al. 2019). With the growing capabilities of LLMs, recent work has begun to import Self-Play into LLMs training to reduce dependence on human intervention. These approaches construct Language Gamification frameworks wherein LLMs compete under clearly defined game rules, incrementally enhancing their competencies. For example, SPIRAL enhances LLM reasoning by having models play games such as Tic-Tace Tac-Toe and Kuhn Poker (Liu et al.) 2025); Absolute Zero frames self pelf-play between proposer and solver (Zhao et al.) 2023). achieving state-of-the-an results on mathematics and coding tasks. However, extending self-play to VLMS remains largely unexplored, despite the prohibitive costs of multimodal data that make such an approach particularly urgent.
An ideal self-play game environment should satisfy the following four conditions: (1) The skills acquired by agents in order to win the game should closely align with those required by the target tasks. (2) Skill growth should be scalable: as self-play progresses, the environment should continually escalate difficulty so that ever stronger agents can emerge rather than converging to a fixed upper bound. (3) The environment should be sufficiently diverse and complex enable a wide range of target tasks can satisfy conditions (1). (4) The environment should require external data only a small amount of low-cost data, such as label-free data. To the best of our knowledge, existing visual reasoning games fail to satisfy all of the above criteria simultaneously. For instance, Sudoku satisfies conditions (2) and (4), but fails to meet (I) and (3). Due to the multimodal nature of VLMs, designing self-play environment that fulfills all four conditions requires joint consideration of both vision and language modalities, which is non- triviill... Inspired by language-base particularly those involving alternating rounds of statements and voting such "Who Is the Spy?". we propose a novel visual reasoning game that addresses these four requirements.
social deduction games,
We present Vision-Zero, the first gamified self-play framework that enables scalable self- improvement of VLMs without requiring human annotations. We formulate a visual "Who Is the Spy?" game in which the civilians observe an image while the spy receives blank visual input. To succeed, the spy must infer the hidden visual content solely from the civilians' utterances, while the civilian must balance informativeness and concealment when providing clues grounded in the image. Through this asymmetric interaction, agents progressively develop stronger visual under- standing, inference, and communication abilities. This setup compels models to engage in strategic reasoning across multiple roles while handling diverse visual inputs such as CLEVR scenes [Johnson] al. 2017), charts, and natural images. We further propose Iterative Self-Play Policy Optimiza- tion (Iterative-PPO), which alternates between Self-Play and RLVR. By incorporating verifiable supervision into self-play, Iterative-SPO stabilizes training and prevents premature convergence to equilibrium states, thereby ensuring consistent performance gains within the Vision-Zero framework.
Vision-Zeri provides a domain domain-agnostic framework that effectively leverages diverse image inputs, enabling continuous improvement without reliance on task-specific datasets. Through carefully designed strategic visual gameplay, it strengthens reasoning, spatial understanding, and visual comprehension while reducing shorteut bias from text and negative capability transfer that
2

---

Published as a conference paper at ICLR 2026
Reasoning and Mathematies
Chart and Decument
Vislon-Contric
Acc (%)
2-
Qwenz,55V 5-LL2
TLM-Tunnkr--2
Cw (Cleve) VistaZero Owen 7B (Chart)
MM Eurem Qween 73
VIGaL Hotattn
Mathvision (68 2333
WeMath (62.1%)
Loga Logelista (47 (12%
MMAI (45.776)
ChartA (86. 1%))
DonVOA (94.8%)
Realwordd 14%)
BLINK (88.2%)
Figure 2: Performance Comparison of Vision-Zero with SOTA post-training methods, All models were post- pest-trained on Qwen2.5-VLL -een2.5- The numbers on the horizontal axis represent the accuracy of (wen2.5-VI.-79 on different tasks, while the vertical axis represents the change in accuracy of the trained model. Vision outperforms baselines trained on expensive human-labeled datasets.
VL-7B.
prevalent conventional VLM training methods. Moreover, Vision-Zers is entirely label-free and eliminates the need for specialized data construction, enabling scalable training directly from raw image inputs. As shown in Fig. 2 Vision-Zero simultancously enhances performance across tasks including reasoning, chart/OCR, and vision-centric tasks, surpassing state-of-the-art baselines trained on expensive human-labeled datasets. These results underscore Vision- 's substantial potential and broad upplicability pioneering aaro-human-ing comman-in-the--- in-the-loop training paradigm. Our contributions are as follows:
We propose Vision-Zero, the first gamified self-play framework for VLMs that achieves hht-hh-mm-----t--eeeeee post-training. which supports label-free, domain-agnosti inputs and enables highly cost-efficient dataset construction for scalable optimization.
We introduce Iterative-SPO, novel algorithm alternating between Self-Play and RLVR to stabilize training and to avoid premature convergence.
Extensive experiments demonstrate that Vision-Zerr substantially enhances model perfor- mance across various general tasks, surpassing strong baselines trained on costly human- annotated datasets, especially reasoning and mathematical tasks.
VISION-ZERO: A GENERALIZABLE GAMIFICATION TRAINING FRAMEWORK
This section introduces Vision-Zero, general, scalable, and high-performing gamified VLM post- training framework as illustrated in Fig. 3 We begin by describing the environment and training data (Sect. [.1I. Next, to achieve sustainable performance improvements, we propose Iterative-SPe which alternates between Self-Play and RLVR (Sect. (.2). Finally, provide a comprehensive analysis of the advantages of Vision-Zero compared to human-involved training methods (Sect. (23)
2.1 ENVIRONMENT AND DATA
Strategic Environment. As shown in Fig. 2 Vision-Zero draws inspiration from natural lan- guage- based social deduction games, Who the Spy. In this setting, multiple players participate: ne civilians and single spy. Each player is assigned an image, where the spy's image is blank. Each round consists two stages:
Clue Stage. In this stage, players informed of their role (civilian or spy). Each player is then prompted to observe their image and provide a verbal clue that reflects its content such as object descriptions or inferring from the image. Players speak sequence, and each player's 's clues become visible to subsequent players; however, their thought processes remain hidden. After multiple rounds clue stage, game enters decision stage.
Decision Stage. In this stage, civilians are instructed to analyze all the provided clues in conjunction with their image to identify the spy. Since the spy knows their identity, they do not participate in voting. player uncertain about who spy, he can respond with "n/a". Both the reasoning and final votes remain private to players.
Vision-Zero constitutes a highly strategic and challenging gaming environment. In the clue stage, the spy must infer the hidden visual content solely from the civilians' clues despite observing only blank image, and generate plausible clues that are consistent with the shared context to avoid being identified. Civilians must provide accurate, clear clues to avoid suspicion while minimizing information leakage to the spy. During the decision stage, civilians further analyze images and clues meticulously to detect inconsistencies and accurately identify the spy. Detailed prompts for both stages are provided in the Appendix A.2. for reference.
3

---

Published as a conference paper at ICLR 2026
Stratogglc Game Enviroument
Itoretive Solf-rlay Optimication
Game Setting Blenk and what looks Dis.
Clue Stage: Self-play of Spy & Civv
Decision Stage: RLVR of Civ
Target: Eliminate suspicion of being
Target: Cornectly vote cut the if vode conrectly, Rewardss else
More vates reosived, Lower rewardd.
Chat Stage Trai:: Desanil your
Pecdioon Stage
Offtperr 21
X
Group Morm
Tes:: for
JCC mnallett
pictare
1y
-C of
-0.33
ri -P -
1 wote right to aadder?
E (1,- U2)
AlP - v2)
Ag. (rl))
Lelesl-fres and Damain-anyesstic Data Iupuet.
Traditiooaal ganing ascrrremand:
Liwilad
pesimallaatinn ylar performennde
Equilibrtum State between Spy & Civ
Saturation State of Docisioo Stage
to TOn is lope!!
We should strengthen decision makers! ability!
We shauld incresse the difficilty datata
Any image
Mey Volee neninal
df Clas Stape of
Hy, Vitee meshoed by my 8 Cluz Stage Roward of spy.
Switch to (RRO tralningg
Switch to Seff- play triining
Provide thee datay vutto
Porformance MathVition
Anurage wotoo etoz. 1f Dedision Stage Reward of
Figure Overall Framework of Virion-Zero. Vision-Zery comprises three core components. Strategic Game Environment: Each role is required to exhibit strategic behavior tailored to diverse scenarios, thereby simultaneously necessitating multiple capabilities. Label-free and Domain- agnostic Data Input: Vision-Zero accepts arbitrary inputs to promote diversity and generalization. To verify this, we train Qwen2.5-Ve Dw for 100 iterations on Gobang and our environment and evaluate Math Vision; results show that Vision-Zero effective generalization. IterativeSPPO: We introduce a novel two-stage training algorithm. In the clue stage, models are trained via Self-Play using zero-sum reward inversely proportional to votes received. In the decision stage, models undergo RLVR training with group normalization, using rewards based on vote correctness.
Label-Free and Domain-Agnose Data Input. The input to Vision is fully label-free yet flexible: for each round, the environment requires only an arbitrary image, where the original image Ia is provided to the civilians, while the spy receives blank visual input T.. To validate this generality, we experiment with three types of data:
CLEVR Data. (Johnson et al., 2017) We automatically rendered 2,000 images using the CLEVR renderer. Each image contains 4-6 randomly arranged objects. All objects were randomly generated through automated scripting. The entire rendering process required ap- proximately 6 hours
NVIDIA A100 GPU. Example training set samples are illustrated
in Fig. 4 (left).
Chart Data. We randomly selected 1,000 images from the CharlQA (Msrry et al. 2022) training set. The dataset includes line charts, pie charts, and bar charts. Examples from this dataset are illustrated in Fig. (middle).
Real- Data. We randomly sampled 1,000 images from ImgEdit (Ye et al. 2025)
training set. Examples from this dataset are shown in Fig. (right).
Overall, Vision-Zero provides a strategic game-based environment in which the model continu- ously generates reasoning supervision through interactive gameplay and learns from verifiable rewards, enabling scalable self-improvement. In addition, Vision-Zero supports label-free and domain-agnostic data construction, allowing users to build domain-specifie datasets at minimal cost. As illustrated in the bottom-left of Fig. 3 Vision-Zero achieves sustained performance improvement on the Math Vision validation set, outperforming the original model by 3%, which unattainable in previously narrowly-defined game environments like Gobang.
2.2 ITERATIVE SELF-PIAY POLICY OPTIMIZATION
To enable sustained performance improvement within Zero,, we introduce Iterative Self-Play Policy Optimization (Iterative-SPO) which is novel optimization algorithm that alternates between self-play and RLVR. The workflow of Iterative-SPe is illustrated in Fig.
Notation. Assume each round has players: civilians and one spy, role set is defined as K. = {s} U (e....., }. The spy and civilians hold images I, and Ic, respectively. In clue stage, each player provide clue uy T Ix. h) k K. based on clue history h. In decision stage, voting mechanism returns vote counts v = (0., Ve...... Un.)) where C22 represents number votes Cy received due to being suspected of being spy, and U, represents the number of votes spy received.
4

---

Published as a conference paper at ICLR 2026
Clevr Data
Chart Data
Real-world Data
Peseriee the étar acteriiits of akgoets.
Pesariie the dfferrencs and trands of dettilantes
Reseriee the dnereeters, vissell stule,
There are tiges odideer Honees
"ihosa reughh 20%%
cowtared of the fiodl point."
Switserkkal large pap sevepared with
There are fivs worvd, agoets plovers
seevergjngg lics rowm orcans streag leprtt
rlye
TTeey waer lookneder 2rif
Thee siiere fheet."
"The pleats suegrors thin in secual
feverdiis
thex dan't loar.""
There déjott of notal
"Sanaiia sharply 2009 High draps.
sepirrll dits food Hts
paws as car's.
"The oibes inth fiee the greee queers
"worth Wasodoria 2005 2012."
"The aoyrelers
whinting, and alore cypk.
Figure 4: Visualization of the datasets used in Vision-Zero. We employ three representative data in our experiments: (left) CLEVR-basee data, (middle) Chart-based data, and (right) Real-world data.
Self-Play Policy Optimization in Clue Stage. During this stage, players seek avoid raising suspicion that they might be the spy. Moreover, the spy and civilians constitute two opposing sides, and we employ Self-Play Policy Optimization to continuously enhance the model's capabilities.
Zero-Sum Reward. Their rewards are designed according to the zero-sum game principle. Based on these considerations, we define the Clue Stage reward polue and pelue as follows: os
pluee (U, vo), rej B (v. To) A (ve, Ee), j= I,..., na.
(1)
The
where Vc Σ}= Vey denotes the average number of votes received by all civilians, controls the intensity of competition between the spy and the civilians, and À regulates the penalty for behavioral inconsistency among civilians. Eqa SqqaTenser ensures that the total reward between the spy and the civilians zero, and that players receiving more votes are assigned lower rewards.
Role-Advantage Estimation (RAE). To mitigate the imbalance in win probability caused by asym- metric role information, we apply RAE (Liu et al.) (025). Specifically, we initialize RAE coefficient for the spy b, and the civilians be to zero. The RAE coefficient and advantage at each round are:
6, = ab. + cfue
be = ab. + a1
clue
Aglue = r
clue
by, E (2)
j=1
where denotes the decay rate, and the advantage values Aglue are computed by subtracting the RAE from the original reward to eliminate information asymmetry.
Objective. With reference policy Teef the optimization objective of Clue Stage is.
Colue (0)
E
Aglu log T. (u4
Lx. + Talue IE
Σ DKL ( m
KEK
kEk
where the KL term constrains updates to remain close to Teets stabilizing learning and preventing degenerate utterances. Unbaselined returns are zero-sum to promote equilibriun-seeking dynamics.
RIVR in the Decision Stage. During this stage, the objective of each player is correctly identify and vote for the spy. Since civilians share aligned information, they can be regarded as a single group. Therefore, adopt the GRPO objective for Decision Stage.
Discrete Reward. Assume civilians take the full-round clues H and outputs &o; qo(- H),
I,..., Re. where Se; can be index of player (indicating vote for the player as spy). Ø (indicating not clear who is spy and answer "n/a") )): assume s* is the true spy index. Define reward
re +1 if 6., = s" -0.5 elif &c, Ø, -1 else.
This reward encourages players to make well-reasoned inferences. Even under highly challenging conditions, it incentivizes acknowledging uncertainty rather than committing to an incorrect answer.
Group Norm & Objective, To remove round-specific difficulty, we apply group normalization:
My mean deec] a, = std [r ].
dec = (r /x/) + E), i == 1,..., (5)
A
where >> prevents division by zero. With reference distribution qrer- we optimize the advantage- weighted likellhood of the sampled votes with KI. regularization:
5

---

Published as a conference paper at ICLR 2026
Please descrtioe picture you aee one sentrnne.
Civi
Civ
32
Spy
The metuilic ey lunder right
spltcee.
The red ey linder is behind the yellow sphere and yellow rubo..
The yallow culse the left of the metullie sphere.
What Chelllane
What Siy See
Before Training
After Training
I'reed to caredully snatese the differencee between my pleture and the clues provided. PLAYER mentioned metuilic cylinder the yellow sphore, PLAYER meationed yellow cube the left of the metallic aphorre This wussesta there might be discrepancics regardling the shapes and posiiions.
I Locd deserilie up picture without revcaling Lhat my fig blank. Let's
PLAYER The mutallie rylinder to the right of the metallic ophere.
PLAYER a: The red cylinder is behind the yellow sphere and yellene coho.
PLAYER 3 yellow ruiee is the of metailie ophere.
Itased these chase, the yellow rahe
common objects
PLAYER eaye the red cylinder be behind the vallow ophere and the yallow
Gives that am spy, imust eusure that my descriptions do not reoeal my images will docus on dearribing featurees match others
these says yellow culs is the the cylinder
mecial sphore.
without pointing uny discrpancies. podition relative to other objects to keep consisbenttee with the previous clacs.
Since mettilie cyllnder
metal splere.
deserthe something that matches commun drawing
doeso't discrepancies left the metal sphere.
PLANNENGI 5 REIRIEVE:S DICOMPOSET STRATEGIZE: LOGIC:
PLANNING RETHEVES e DECOMPOSEES STRATINGIZUS LOGAIC D
pieeeeen qf is the splare
The rell es eelinter is the left of the unctal syheeee.
Civ Win
PLAVER Is spy!
Spy Win
We don't kmasst
Figure 5: Visualization of spy reasoning in Vision-Zero. A comparison of model responses to identical scenarios before and after training, evaluated by GPT- based scoring, reveals substantial improvements in planning, retrieval, decomposition, strategy formulation, and logical reasoning.
1 The
(dee (0) = -
Adec log (
H)
+ Tiee
(go( H) ger( H))
(6)
Nc i=1 Iterative Stage Training. A pure self-play setup typically reaches local equilibrium (Yao et al. 2023, Balduzzi et al., 2019; Hu et al.) 2020 Balduzzi et al., 2018). limiting exploration of new reasoning paths. Conversely, standalone RL methods like RLVR risk knowledge saturation once the available question set is mastered. To mitigate these issues, Iterative-SPO employs a two- stage alternating training. When decision-stage performance indicates clue-stage saturation (easy identification of the spy), training shifts to the clue stage to increase difficulty. Conversely, when identifying the spy becomes challenging, training shifts back decision stage. Let B, = {(Ili, ) be a held-out mini-bateh at iteration t. Define the average prediction accuracy acc; and "n/a" rate na, of players in the decision stage within a batch round:
i=1
B
B
acc₁ B
Σ arg max (y H,) = 5p .naa =
Σ
(o
H2).
(7)
B
1=1
10; 1=1
We maintain exponential moving averages with smoothing
acc; p + (1 -p) acC,, 1a2 = - + (1- p) n&,,
(8)
initialized as ačcς = nan = 0. Let m, E (0. 1} be the phase indicator (m, = 1 trains the CLUE stage,
m trains the DECISION stage). We switch phases using hysteresis thresholds 7 Tere 7 T
Decision + Clue: if m = 0 and ačc; 2 Taee and nā, I Thai then set m = =1;
(9)
Clue - Decision: if m = 1 and (1 (1- acc; 2 - or na, 2 Tiam): then set m4+1 O; (10)
otherwise m4+1 = my. To avoid chattering, we require a minimum dwell time Kmin updates per stage. With this gating, the per-iteration training loss is L. my (0) (1-m)) Lace (0), and gradients are applied only to the active module at iteration Algorithm shown Appendix (A..3
This alternating scheme provides two main benefits: (1) It prevents the model from stagnating in strategic equilibrium or knowledge platcau by dynamically switching training stages upon detecting stagnation signals, thus ensuring continuous improvement (empirically verified in Sect. 3.2)) Alternating self-play with RLVR introduces supervised signals, stabilizing training and preventing common pitfalls like role collapse (Wang et al. 2020 Yu et al.. 2024) divergence (Heinrich & Silver 2016) Vinyals et al.) 2019). In summary, Iterative-PPO provides a stable paradigm that integrates self-play with RLVR optimization to achieve sustained performance improvement.
2.3 ADVANTAGE ANALYSIS
Vision-Zero has three key advantages. Firstly, Vision-Zero leverages domain-agnostic data inputs through image differences, allowing it to accept diverse data without reliance on specific image types. This universality enables direct utilization of existing high-quality image datasets, leading

---

Published as a conference paper at ICLR 2026
Table 1: Performance Comparison of Vision-Zero and SOTA models Reasoning and Math, evaluated on VLMEvalKit. All results are obtained under same settings, except ViGaL-Snake and ViGaL-Rotation, whose results are obtained from the original paper due to unavailable models. Vision-Zero outperforms baselines trained on extensive manually annotated datasets in related tasks.
Method
Math
MathVision
WeMath
Math
LogicVista
DynaMath
Avg.
Proprietary Model
GPT4o
61.4
30.4
40.0
50.2
45.9
32.3
4344
Gemini2 0-Flash
73.4
41.3
57.1
54.4
56.2
43.7
54.4
Performance Qwen2.5 VI.
Qwea2.5-VL-7B
68.2
25.4
36.1
49.0
47.2
20.9
41.1
RI-OneVision
64.1
24.1
35.8
47.1
44.5
21.4
39.5
MM-Burcka-Qwen-77
73.0
26.9
36.2
50.3
42.9
24.2
VLAA-Thinker-7B
68.0
26.4
36.0
51.7
47.2
21.9
41.9
OpenVLThinker-
70.2
25.3
36.5
47.9
44.3
21.2
40.9
ViGaL-Snake
70.7
26.5
51.1
ViGaL-Rotation
71.2
26.3
50.4
ViGaL Stake-Raatn
71.9
52.4
46.5
22.9
Stion//ero-Owen-71 (CLEVR)
72.2
28.4
39.2
53.2
49.8
22.9
44.3
VisionZero- (Chart)
72.4
28.9
38.8
$2.5
51.2
22.1
43.3
#sion/ZeroQwwen-BB Real--World)
73.1
28.5
40.1
52.1
50.8
22.5
44.5
250
Avg: Token Len (Dectston)
70
8
400
Win Rate
200
80
Avg. Tokem
150
300
Owen2 5-L
@wen2.5.V1.2 2--177
Qwen2.5-VL. 2--2-
50
Inte
InternVLL3-00
200
InternVL33-8 BB
InternV1.3-148
250
InternVLL3-00
InternV1.3-18B
0
20
40
80
100
20
40
60
80
100
20
40
60
80
100
Iteration
Iteration
Iteration
(a) Winning Rate
(b) Avg. Token Length (Clue)
(c) Avg. Token Length (Decision)
Figure 6: Evolution of win rate and token length during Vision-Zero training. Win rates are evaluated over 100 rounds (50 civilian, 50 spy) against corresponding untrained reference models; civilians win by correctly identifying the spy. Token length are collected across these rounds.
to generalizable performance improvements at minimal cost, as evidenced by superior benchmark results (Fig. 2 Secondly, Vision-Zero demands simultaneous analysis of visual and textual inputs, addressing spatial relationships and object details, thereby concurrently enhancing reasoning. visual comprehension, and OCR capabilities. This integrated approach effectively mitigates common challenges such as text shortcut bias and negative capability transfer, as illustrated in Fig. - 5 Lastly, Vision-Zero employs highly cost-efficient data curation strategy, rapidly generating datasets using advanced editing tools like ChatGPT and NanoBanana. This approach significantly reduces costs compared traditional manual labeling, accelerating practical applications of targeted VLMs.
EXPERIMENTS
To thoroughly evaluate Vision-Zero, first outline the experimental setup, the datasets, and the baselines. Next, we evaluate its performance and cost-efficiency across diverse tasks (Sect. 3.1 T). We then conclude analyzing model generalizability and the effectiveness of Iterative-SPO. (Sect. 3.2)
Models, Datasets & Baselines. We evaluated Vision-Zero using three models models-Qwent Qwen2.5-VL. 7B (Bai et al., 2025). InternVL3-SB. and Intern (Zhu al.) 2025)-across across 14 tasks in reasoning, chart analysis, and vision-centric domains. Detailed model and dataset information is in the Appendix A.3.I We compared our models against SOTA methods Vi (ang et 2025b). EureEeaaa--eeee.....
(Zhou et al. 2025). and Open WLThikker-- (Deng 2025) (all post-trained via RLVR on human-labed labeled data), as well as ViGal. Xie et al. 2025). which collects game data initially and subsequently training them.
Meng et al. 2025).
Training and Hyperparameter Settings. We detail the hyperparameters used for Vision-Zero training below. Each round included four civilians (n. 4) and two clue-stage speeches. To maintain balanced rewards (-1 to range), we set clue hyperparameters
= 0.1. Decay coefficients for role advantage (a), accuracy, and "n/a" rates (p) were adopted from Liu et al. (2025) as p 0.95. KL regularization weights were set as defaults (Taec = Telue = 0.04). Empirically set stage-switchin switching thresholds were TIf 0.9, -f 0.4, -1f 0.5, 7 0.1, with minimum rounds per stage Kmin = and patience P 20. Models were trained for 100 iterations with a batch size of 128 using
the VLM-R1 (Shen et al. 2025) code framework. Qwen2.5-VL.. V-VB--BB was trained the CLEVR--aseed
7

---

Published as a conference paper at ICLR 2026
Table 2: Performance comparison between Vision-Zero and other state-of-the-ard models on Chart Understanding and Vision-Centro benchmarks. All models are evaluated using the open- source platform VLMEvalKit. Additional results on related datasets are provided in the Appendix A.4
Chart Understanding
Vision-Centric
Model
ChartXiV
FunctionQA
PaperQA
ReachQQ
RealVVorldQQ
MMVP BLINK
MuirBench
Proprietary Model
(IPT-4o
47.1
80.7
47,4
51.3
75.4
86.3
68.0
68.0
Gemini2.0-Faach
61.2
73.2
64.6
Performance Qwen2.5.V1...
Qwee2
42,5
K1-OneVisico-728 7B
35,3
69.4
64.2
58.0
46.3
MM
Qwen-7B
43.2
79.0
VLAA-Thinke--B
41.3
79.0
68.1
30.4
65.4
71.6
53.0
57.1
44.1
8388
73.7
51.5
60.2
71.3
49.9
52.8
ViCaal -SmiocRotttiion
41.8
82.3
73.7
51.8
66.5
74.6
55.6
57.8
VidoaZern-Quen-TB (CLEYR)
44.1
VislonZeroQQwen-BB (Chart)
45.8
73.7
VisoaaZer-Quen-Tl (Real- World)
44.8
$2.3
73.7
$2.8
68.9
79.2
57.2
59.4
chart-based, and real-world datasets (Fig- 3 Intern VL3 was trained only on the CLEVR-based dataset test generalizability. Further details are provided in Appendix A.3.2
3.1 MAIN RESULTS
Sustainable Performance Growth. To verify Vision-Zero 's capability to achieve sustained perfor- mance growth, we evaluated the models' win rates against fixed, untrained reference model and measured average token lengths in the Clue and Decision stages on CLEVR data. As shown in Fig. 6 win rates consistently increased during training, with Owe-2..-VL-7B improving from 50% to 71 %. Average token lengths increased substantially, particularly in the Decision stages (e.g.. Intern VL3-8B and Intern VL3- 14B grew from 250 to approximately 400 tokens), suggesting enhanced reasoning capabilities facilitated by Iterative-SPO.
Strong Task Generalization Capability. To assess whether the performance gains from the Vision- Zero environment generalize to broader reasoning and mathematics tasks, we evaluate our models on six benchmark datasets. The experimental results are presented in Tab. [ As demonstrated, Vision- Zero models consistently outperform state-of-the-art baseline methods across various benchmarks. Specifically, AisionZero-Qwen-77 (CLEVR) and BerionZero-Qwen--r (Chart) achieve performance (Chart) improves by -2.2%. In contrast,
gains of -3% over the base model, and DisionZero-Qwen. even the most advanced baseline method yields just ~1.9% improvement. Notably, all baseline methods rely on training with hundreds or even thousands of mathematics and reasoning samples. As a comparison, our Vision-Zero environment does not explicitly include any mathematies-speecin task training: rather, it enhances the models" logical reasoning capabilities through strategic gameplay in natural language contexts. These results clearly indicate that the capabilities learned by models from the Vision-Zero environment can effectively generalize to broader mathematics and reasoning tasks, even surpassing models explicitly trained on those large scale task specific datasets.
Cross- Caps--aayaylltty Negative Transfer Mitigation. A key challenge in VLM post-training is cross- capability negative transfer, where models trained on specific tasks often perform worse on others. As shown in Tab. Vision-7erootrrinee models effectively mitigate such negative transfer. Specifi- cally, Vision 7B (CLEVR) enhances vision-centric task performance, notably increasing MMVP accuracy from 76.8% 79.5%. Notably, VisionZero-Que -werro--eeeennBeeeeee 7B(haart) achieves signifi- cantly larger gains on chart understanding benchmarks, improving accuracy by an average of +3.9% across the four datasets. VisionZero-Qwenn - and VisionZZeo--Wwen. (e also exhibit consistent improvements due to enhanced reasoning ability. This demonstrates that Vision- Zero's is strategic. multi-capability training environment significantly alleviates negative transfer issues common in traditional single-capability training paradigms. Moreover, thanks to the task-agnostic nature of Vision-Zero it enables significant performance gains on diverse target tasks such as chart understanding or vision-centric reasoning, through the low-cost construction of task-specific data.
Low Costs. Vision-Zero significantly reduces dataset construction costs and training time compared to traditional RLVR methods (Tab. 3). Due to its reliance solely on unlabeled data, Vision-Zero incurs zero labeling costs. In contrast, previous training methods typically require extensive human or
tfain-of--hhuglle (CoT) annotations and answer labeling. consuming significant amounts of time and human resources. In addition, due to the high sample efficiency Vision Zero, it achieves superior model performance with significantly fewer training iterations. As shown in Tab.
8

---

Published as a conference paper at ICLR 2026
Table 3: Comparison of dataset construction costs, training costs and model performance across methods. Label Cost refers the number of tokens generated by teacher or judging LLMs during data curation; for consistency, all token counts are recalculated using the Qwen2.5 tokenizer. Since VIGAL and Vision-Zero are trained on unlabeled data, they incur no labeling cost. To estimate training time cost, we refer to each baseline's s original paper to obtain the number of samples used during RL training, and multiply this by a standard GRPO cost per sample to simulate the expected time consumption under a fully fair setting. For some methods, the value is shown as > because we only account for RI. cost, excluding SFT. For Vision-Zero, the training time cost getted by directly measured. Details on all estimates are provided in the Appendix A.3.3
Data Cost
Training
Performance
Method
Pregare Method
Num (RL)
Label Cost (Tokens)
Method
Interact
Time Cost
MMMu
MMMM
Qwen2.5-VLL77
54.3
37.0
RI-OneVision-T1
Programmatic
10k
1.1
SPT&GRPO SFT-GRPO SPT+GRPO GRIO
170 A A100-Hours
31.9
32.6
VLAA-Thinker-TT
construction with
25k
29.6 M
120 A100-Hours
48.2
31,9
A100-Hours
54.8
MM-Eureka Qwenn-BBBBBBBBBBBB
15k.
700.A1000 A100-Hour.
55.8
36.9
ViGal-SSnkk
Collected
55.8
-iGal-RRttatin
game environment
72k
RLO0
Ps 170 A100-Hours
54.1
37.7
via PPO policy
SK.0
37.4
VidionZero- Owern---
Altermating Self- play+ GRPO
(CLEVR)
Batch rander sounes
127 A100-Hours
58.8
37.7
28.0
MM (urekaaGGiPPU))
MM.EarckaaCRRPO)
Viston-Zero
Visten- Zeroo
27.5
23.5
Accuracy
27.0
23.0
26.5
22.5
2.67x
22.0
6:40x
26.0
21.5
25.5
21.0
0
10
20
30
40
50
60
70
80
90
100
200
Training Iteration
Training Iteration
(a) Qwen2.5-VL-7B
(b) InternVL3-88
Figure 7: Taining effectiveness comparison between Vision-Zero and the original GRPO. We compare Vision-Zero and GRPO under identical hardware settings to evaluate training cost and effi- ciency. Specifically, for the original GRPO, we trained on the MM-Eureka dataset using 8xNVIDIA A100 (80GB) GPUs with a batch size of 128 for 100 iterations on both Owen2.5-VL-77 IntermVL--8B. Vision-Zero is trained for the same setting on the Clever dataset using the same hardware. We evaluate the performance of checkpoints from different iterations Vista.
and
3 Vision-Zero requires only 127 A 1000-hours, which is substantially lower than prior GRPO-based approaches. This underscores Vision-Zero substantial practical value in real-world applications.
High Training Efficiency. Although Vision-Zero involves multi-round interactions, it does not introduce substantial training overhead. On one hand, Vision-Zero adopts fixed interaction pattern (two clue rounds followed by one decision round), enables fully parallelized forward and backward passes across multiple games, with no asynchronous delays or gradient conflicts. On the other hand, each sample in Vision-Zero generates multiple actions, thereby providing denser learning signals and higher sample efficiency compared to standard single-turn RI. setups. To empirically validate the training efficiency of Vision-Zero, we conducted experiments comparing the training time and efficiency of the original GRPO and Vision-Zeni over the same number of iterations using identical hardware. As shown in Fig. 7. Vision-Zero achieves markedly higher sample efficiency, resulting in 3.3 x and 6.4 x improvements in overall training efficiency on Qwen2.5.VL. VL-7B and Intern VL3-8B, respectively. Furthermore, it yields higher final performance the validation set. These results highlight the superior training efficiency of Vision-Zero compared to the original GRPO framework.
3.2 ABLATION STUDIES
Model Generalizability. To assess Vision-Zero's generalizability, trained Intern VL models and evaluated their performance on reasoning and math tasks. Tab, Ahoows VisionZero- Intero--IeeeVV Interv VL3-8B and VisionZZero-Ineent wnte//r0-- improved accuracy by 1.8% and T.6%, respectively, across reasoning

---

Published as a conference paper at ICLR 2026
Table 4: Model generalizability of Vision-Zero. We train IntemVV33-8B and Intern VL3-14B within the Vision-Zero using the CLEVR-based dataset. As a baseline, we train AiterVLL--4 and IntemVL3-111 14B with vanilla GRPO on the MM-Eureka training under the same setting as Vision-Zero, and evaluate all models on six reasoning benchmarks.
Model
MathVista
MathVision
WeMath
MathVerse
LogicVista
DynaMath
Avg.
Performance on Inter
IntemVL3-88
60.4
21.3
26.8
32.2
40.5
26.8
34.7
62.4
22.1
26.8
32.1
38.9
28.7
35.2
MhinnZero-InterVLL.33
62.2
24.2
28.7
32,9
41.8
29.2
36.5
Performance on InterVVL33 148
IntemVL3-114
33.8
42.3
43.3
51.6
30.1
45.8
AM-Bureka-Intee Inteeen
75.2
34.5
42.5
44.2
45.2
30.9
45.4
-ssonneer-Inteee Intern VL3-144
75.4
34.8
44,9
45,1
53.1
31.3
47.4
75
Alurzately
Alternatelyy Pure Clue
70
Pum Darition
65
60
Win
47
1.00 46
20
Iteration
Iteaaion
Figure 8: Performance Comparison between Iterative-SPO and pure Self-play / pure RLVR training. (left) Winning Rate (right) Performance Logic We evaluate under three settings: (I) Iterative-SPO; (2) Pure Decision: Clue stage frozen, training only Decision stage via RLVR; (3) Pure Clue:
Decision stage frozen, training only Clue stage via Self-Play.
tasks. Compared to the baseline trained using the MM-Eureka dataset and GRPO framework, Vision- Zero consistently enhances the reasoning capabilities of Intern VL3-8B and Intern VL3-14B models by 1.3% and 2%, respectively. Notably, despite differences in visual encoders, pre- pre-training strategies, and training procedures between the Qwen'V and Intern VL model series, Vision-Zero consistently improves performance across these models, highlighting its robust generalization capabilities.
Superiority of Iterative-SPO. Finally, evaluate the superiority of Iterative-SPO compared to single-mode training by training Qwen2.5- under three distinct settings: (1) Pure clue-stage
training: the decision stage frozen (forward-pass only, without gradient updates); (2) Pure decision- stage training: the clue stage is frozen, with only the decision stage updated; and (3) Iterative-SPO. As shown in Fig. Iterative-SPO substantially outperforms both single- approaches, particularly surpassing pure clue-stage training, which experiences slower performance gains and premature equilibrium. This occurs because pure self-play lacks directly verifiable rewards the reward signal originates from the decision-maker and when decision quality is insufficient to effectively discriminate roles, the model performance plateaus prematurely. Alternating training mitigates this limitation, achieving sustainable performance improvements; for example, the Logic dataset, it improves final accuracy by 2% over pure self-play and 1% over pure RLVR training.
CONCLUSION
We introduce Vision-Zero, the first gamified self-play framework for VLMs that achieves zero-human- in-the-loop post-training, addressing self-play training challenges through strategic environment and domain aggosicc inputs. Our novel Iterative Self-Play Policy Optimization (Iterative-SPO) algorithm alternates self-play with RLVR, incorporating supervisory signals to stabilize training and avoid suboptimal equilibria. Experiments show Vision-Zero significantly improves VL.M performance reasoning, chart/OCR, and vision-centric tasks while substantially reducing dataset construction costs compared to traditional human-labeled datasets, providing an economical, flexible, and robust solution for accelerating VLM development and real-world application.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
We will fully release the model checkpoints and source code to facilitate reproducibility of our results. We provide all prompt design of the gameplay environment in Appendix [..21 Implementation details, including the experimental setup, hyperparameters can be found in Appendix Dataset preparation details can found in Appendix A.2.2 Algorithm details are shown Appendix AA2.3
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, llge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et Gpt-4 technical report. arXiv preprint S3XYv:2303.08774 2023.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang. Shijie Wang, Jun Tang, et al. Qwen2. 5-v1 technical report. arXiv preprint arXiv:2502.13923, 2025.
David Balduzzi, Sebastien Racaniere, James Martens, Jakob Foerster, Karl Tuyls, and Thore Graepel. The mechanics of n-player differentiable games. In International Conference on Machine Learning. pp. 354-363. PMLR, 2018.
David Balduzzi, Marta Garnelo, Yoram Bachrach, Wojeiech Czamecki, Julien Perolat, Max Jader- berg, and Thore Graepel. Open-ended learning in symmetric zero-sum games. In International Conference on Machine Learning, pp. 434-443. PMLR, 2019.
Christopher Berner, Greg Brockman, Brooke Chan, Vicki Cheung, Przemystaw Debiak, Christy Dennison, David Farhi, Quirin Fischer, Shariq Hashme, Chris Hesse, et al. Dota 2 with large scale deep reinforcement learning, arXiv preprint 191.:191,,00680, 2019.
Liang Chen, Hongcheng Gao, Tianyu Liu, Zhiqi Huang, Flood Sung, Xinyu Zhou, Yuxin Wu, and Baobao Chang. GI: Bootstrapping perception and reasoning abilities of vision-language model via reinforcement learning. arXiv preprint arXiv:2505. 13426, 2025a.
Xi Chen, Mingkang Zhu, Shaoteng Liu, Xiaoyang Wu, Xiaogang Xu, Yu Liu, Xiang Bai, and Hengshuang Zhao. Mico: Multi-image contrast for reinforcement visual reasoning. arXiv preprint 2211:25006.2233,, 2025b.
Silvio Chito, Paolo Rabino, and Tatiana Tommasi. Efficient odd-one-out anomaly detection. arXiv preprint arXiv:2509.04336, 2025.
Kevin Denamganai, Sondess Missaoui, and James Alfred Walker. Visual referential games further the emergence of disentangled representations. arXiv preprint arXiv:2304.14511, 2023.
Yihe Deng, Zhen Wang, Zhe Chen, et al. Openvithinker: Complex vision-language reasoning via iterative sft-rl. arXiv preprint arXiv:2503. 2025.
Aaron Dharna, Cong Lu, and Jeff Clune. Foundation model self-play: Open-ended strategy innovation via foundation models. arXiv preprint arXiv:2507 (tV.:2507.00466.. 2025.
Ruiqi Dong, Zhixuan Liao, Guangwei Lai, Yuhan Ma, Danni Ma, and Chenyou Fan. Who is undercover? guiding 1lms to explore multi-perspective team tactic in the game. arXiv preprint arXiv:2410.15311, 2024.
Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A. Smith, Wei-Chiu Ma, and Ranjay Krishna. Blink: Multimodal large language models can see but not perceive, 2024.
org/ abs/ 2404 12390
Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, al. Ego4d: Around the world in 3,000 hours of egocentric video. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 18995-19012, 2022.

---

Published as a conference paper at ICLR 2026
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025.
Wei He, Zhiheng Xi, Wanxu Zhao, Xiaoran Fan, Yiwen Ding, Zifei Shan, Tao Gui, Qi Zhang, and Xuanjing Huang. Distill visual chart reasoning ability from Ilms to mllms, 2025. URL
https: //arxiv wttpvii.orr//oooooo abs////////// 22110 18798
Johannes Heinrich and David Silver. Deep reinforcement learning from self-play in imperfect- information games. arXiv preprint arXiv: 121, 2016
Hengyuan Hu, Adam Lever, Alex Peysakhovich, and Jakob Foerster. "other-play" for zero-shot coordination. In International Conference on Machine Learning, pp. 4399-4410. PMLR, 2020.
Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. Clevr: diagnostic dataset for compositional language and elementary visual reasoning. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 2901-2910, 2017.
Byungjun Kim, Dayeon Seo, and Bugeun Kim. Fine-grained and thematic evaluation of Ilms in social deduction game. arXiv preprint arXiv:2408.09944 2024.
Ksenia Konyushkova, Christos Kaplanis, Serkan Cabi, and Misha Denil. Vision-language model dialog games for self-improvement. arXiv preprint uxiv:2502.007744 2025.
Ranjay Krishna, Yuke Zhu. Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, et al. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International journal of computer vision, 123(1):32-73, 2017.
Angeliki Lazaridou, Karl Moritz Hermann, Karl Tuyls, and Stephen Clark. Emergence of lin- guistic communication from referential games with symbolic and pixel input. arXiv preprint arXiv:1804.03984, 2018
Muyao Li, Zihao Wang, Kaichen He, Xiaojian Ma, and Yitao Liang. Jarvis-vla: Post-training large-scale vision language models to play visual games with keyboards and mouse. arXiv preprint arXiv:2503. 16365, 2025.
Tian Liang, Zhiwei He, Jen-tse Huang, Wenxuan Wang, Wenxiang Jiao, Rui Wang, Yujiu Yang. Zhaopeng Tu, Shuming Shi, and Xing Wang. Leveraging word guessing games to assess the intelligence of large language models. arXiv preprint arXiv:23 arXiv:2310.20499, 10.20499, 2023.
Bo Liu, Leon Guertler, Simon Yu, Zichen Liu, Penghui Qi. Daniel Balcells, Mickel Liu, Cheston Tan, Weiyan Shi, Min Lin, et al. Spiral: Self-play on zero-sum games incentivizes reasoning via multi-agent multi-tum reinforcement learning. arXiv preprint arXiv:2506.2411 19, 2025.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892-3444 34916, 2023.
Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai- Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts, 2024. URL
wwwpx/v.or//bb///311.o.oo 02255
Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. Chartqa: A bench- mark for question answering about charts with visual and logical reasoning. arXiv preprint arXiv:2203. 10244, 2022.
Fanqing Meng, Kai Sun, Yuxuan Liu, et al. Mm-eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learnin.. arXiv preprint arXiv:2503.0733 2025.
Sachit Menon and Carl Vondrick. Visual classification via description from large language models. arXiv preprint arXiv:2210.07183, 2022.
12

---

Published as a conference paper at ICLR 2026
Salman Mohammadi, Anders Kirk Uhrenholt, and Bjøm Sand Jensen. Odd-one-out representation learning. arXiv preprint arXiv:2012.07966, 2020.
Timothy Ossowski, Jixuan Chen, Danyal Maqbool, Zefan Cai, Tyler Bradshaw, and Junjie Hu. Comma: A communicative multimodal multi-agent benchmark. arXiv preprint arXiv:2410.07553, 2024.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730- 27744, 2022.
Davide Paglieri, Bartlomiej Cupiat, Samuel Coward, Ulyana Piterbarg, Maciej Wolczyk, Akbir Khan, Eduardo Pignatelli, Lukasz Kuciński, Lerrel Pinto, Rob Fergus, et al. Balrog: Benchmarking agentic IIm and vlm reasoning on games, arXiv preprint mrXiv:2411.13543 2024.
Genevieve Patterson and James Hays. Coco attributes: Attributes for people, animals, and objects. In European conference on computer vision, pp. 85-100. Springer, 2016.
Runqi Qiao, Qiuna Tan, Guanting Dong, Minhui Wu, Chong Sun, Xiaoshuai Song, Zhuoma GongQue, Shanglin Lei, Zhe Wei, Miaoxuan Zhang, Runfeng Qiao, Yifan Zhang. Xiao Zong. Yida Xu, Muxi Diao, Zhimin Bao, Chen Li, and Honggang Zhang. We-math: Does your large multimodal model achieve human-like mathematical reasoning?, 2024. URL 01284
Shuwen Qiu, Sirui Xie, Lifeng Fan, Tao Gao, Jungseock Joo, Song-Chun Zhu, and Yixin Zhu. Emergent graphical conventions in visual communication game. Advances Neural Information 2022.
Processing Systems,
Juan Rocamonde, Victoriano Montesinos, Elvis Nava, Ethan Perez, and David Lindner. Vision- language models are zero-shot reward models for reinforcement learning. arXiv preprint arXiv:2310.1299 2023.
Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, et al. Vlm-rl: A stable and generalizable r1 rl-style large vision-language model, 2025. URL https://arxiv. org/abs/2504..07611 2025.
David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, loannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. Mastering the game of go with deep neural networks and tree search. nature, 529(7587):484 2016.
David Silver, Thomas Hubert, Julian Schrittwieser, Toannis Antonoglou, Matthew Lai, Arthur Guez, Mare Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, al. Mastering chess and shogi by self-play with general reinforcement learning algorithm. arXiv preprint 172.0:1712.01815. 2017.
Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang- Yan Gui, Yu-Xiong Wang, Yiming Yang, et al. Aligning large multimodal models with factually augmented rlhf. arXiv preprint arXiv:2309.1455 2023.
Liyan Tang, Shreyas Pimpalgaonkar, Kartik Sharma, Alexandros G. Dimakis, Mahesh Sathi- amoorthy, and Greg Durrett. Bespoke-minichat-7b5 Pushing the frontiers of open vims for chart understanding. blog post, 2025. URL httttss::/hhggggggetteee http:///g...ccccoe/ee...e..eeeee
Bespoke-Minichart--
Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew Dai, Anja Hauth, Katie Millican, al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312. 11005, 2023.
Gerald Tesauro. Temporal difference learning and td id-gammon. Communications the ACM, 38(3): 58-68, 1995.
13

---

Published as a conference paper at ICLR 2026
Oriol Vinyals, Igor Babuschkin, Wojeiech M Czamecki, Michaël Mathieu, Andrew Dudzik, Junyoung Chung. David Choi, Richard Powell, Timo Ewalds, Petko Georgiev, et al. Grandmaster level in starcraft ii using mueti- reinforcement learning. nature. 575(7782)35003544 2019.
Fei Wang, Xingyu Fu, James Y. Huang, Zekun Li, Qin Liu, Xiaogeng Liu, Mingyu Derek Ma, Nan Xu, Wenxuan Zhou, Kai Zhang, Tianyi Lorena Yan, Wenjie Jacky Mo. Hsiang-Hui Liu, Pan Lu, Chunyuan Li, Chaowei Xiao, Kai-Wei Chang, Dan Roth, Sheng Zhang, Hoifung Poon, and Muhao Chen. Muirbench: A comprehensive benchmark for robust multi-image understanding, 2024a.
Uttttss:://aaz //rxiv org/ abs abs/2406.... 09411
Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Mingjie Zhan, and Hongsheng Li. Measuring multimodal mathematical reasoning with math-vision dataset, 2024b. URL https::t///sst
org/abs/2400 2402. 14804
Tonghan Wang, Heng Dong, Victor Lesser, and Chongjie Zhang. Roma: Multi-agent reinforcement learning with emergent roles. arXiv preprint arXiv:2003.08039, 2020.
Xinyu Wang, Bohan Zhuang, and Qi Wu. Are large vision language models good game players? arXiv preprint 2Xiv:2503.02355 2025a.
Xiyao Wang. Zhengyuan Yang, Chao Feng, Yongyuan Liang, Yuhang Zhou, Xiaoyu Liu, Ziyi Zang, Ming Li, Chung-Chine -Chnn Lin, Kevin Lin, Linjie Li, Furong Huang, and Lijuan Wang. Vicrit: A verifiable reinforcement learning proxy task for visual perception in vlms. arXiv preprint arXiv:2506.1011 2025b.
Yikun Wang. Yibin Wang, Dianyi Wang, Zimian Peng, Qipeng Guo, Dacheng Tao, and Jiaqi Wang. Geometryzero: Improving geometry solving for Ilm with group contrastive policy optimization. arXiv preprint arXiv:2506.07160, 2025c.
Yufei Wang. Zhanyi Sun, Jesse Zhang, Zhou Xian, Erdem Biyik, David Held, and Zackory Erickson. Rl-vim-f: Reinforcement learning from vision language foundation model feedback. arXiv preprint arXiv:2402.03681 2024c.
Zirui Wang. Mengzhou Xia, Luxi He, Howard Chen, Yitao Liu, Richard Zhu, Kaiqu Liang, Xindi Wu, Haotian Liu, Sadhika Malladi, et al. Charxiv: Charting gaps in realistic chart understanding in multimodal llms. Advances Neural Information Processing Systems, 37:1 13569-113677, 2024d.
Tong Xiao, Xin Xu, Zhenya Huang. Hongyu Gao, Quan Liu, Qi Liu, and Enhong Chen. Advancing multimodal reasoning capabilities of multimodal large language models via visual perception reward. arXiv preprint arXiv:2506.072.5. 2025.
Yijia Xiao, Edward Sun, Tianyu Liu, and Wei Wang. Logicvista: Multimodal Ilm logical reasoning benchmark in visual contexts, 2024.
/2tp:///....//b////4....o. 2407000/9733373 04973
Yunfei Xie, Yinsong Ma, Shiyi Lan, Alan Yuille, Junfei Xiao, and Chen Wei. Play to generalize: Learning reason through game play, arXiv preprint arXiv:2506.08011 2025.
Yuwei Yang. Zeyu Zhang, Yunzhong Hou, Zhuowan Li, Gaowen Liu, Ali Payani, Yuan-Sen Ting, and Liang Zheng. Effective training data synthesis for improving milm chart understanding. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2025a.
Yuxin Yang, Zheng Wang, Hao Zhang, et al. RI RI-onevision: Advancing generalized multimodal reasoning via textualized perception. arXiv preprint arXiv:2503.. 10615, 2025b.
Jian Yao, Weiming Liu, Haobo Fu, Yaodong Yang, Stephen McAleer, Qiang Fu, and Wei Yang. Policy space diversity for non-transitive games. Advances in Neural Information Processing Systems, 36: 67771-67793, 2023.
Yang Ye, Xianyi He, Zongjian Li, Bin Lin, Shenghai Yuan, Zhiyuan Yan, Bohan Hou, and Li Yuan. Imgedit: unified image editing dataset and benchmark, arXiv preprint arXiv:2505.2023 2025.
Byunghwa Yoo and Kyung-Joong Kim. Finding deceivers in social context with large language models and how to find them: the case of the mafia game. Scientific Reports, 14(1):30946, 2024.
14

---

Published as a conference paper at ICLR 2026
Yang Yu, Qiyue Yin, Junge Zhang, Pei Xu, and Kaiqi Huang. Admn: Agent-driven modular network for dynamic parameter sharing in cooperative multi-agent reinforcement learning. Proceedings of the Thirty-Third International Joint Conference Artificial Intelligence, IJCAI-24, pp. 302- 310, 2024.
Simon Zhai, Hao Bai, Zipeng Lin, Jiayi Pan, Peter Tong, Yifei Zhou, Alane Suhr, Saining Xie, Yann LeCun, Yi Ma, et al. Fine-tuning large vision -anguaee models as decision-making agents via reinforcement learning. Advances in neural information processing systems, 37:1 17:115955-11971,, 2024.
Alex Zhang, Thomas Griffiths, Karthik R Narasimhan, and Ofir Press. Videogamebench: Can lision-laguuage models complete popular video games? arXiv preprint arXivv2505. 181344 2025.
He Zhang, Shenghao Ren, Haolei Yuan, Jianhui Zhao, Fan Li, Shuangpeng Sun, Zhenghao Liang, Tao Yu, Qiu Shen, and Xun Cao. Mmvp: A multimodal mocap dataset with vision and pressure sensors, 2024a. URL https:://aaiiiiiie wwtp://wwww.../////////... /tt.//////////aaaaa.oo/////////. 17610
Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Peng Gao, and Hongsheng Li. Mathverse: Does your multi-modal Ilm truly see the diagrams in visual math problems?, 2024b. URL
14624
Yi-Fan Zhang, Huanyu Zhang, Haochen Tian, Chaoyou Fu, Shuangqing Zhang, Junfei Wu, Feng Li, Kun Wang, Qingsong Wen, Zhang Zhang, et al. Mme-realworld: Could your multimodal Ilm challenge high-resolution real-world scenarios that difficult for humans? arXiv preprint arXiv:2408. 13257, 2024c.
Andrew Zhao, Yiran Wu, Yang Yue, Tong Wu, Quentin Xu, Matthieu Lin, Shenzhi Wang. Qingyun Wu, Zilong Zheng, and Gao Huang. Absolute zero: Reinforced self-play reasoning with zero data. arXiv preprint arXiv:2505.0333
2025.
Yicheng Zhou, Yuxuan Chen, Zhen Li, et al. Sitt or rl? an early investigation into training rl-like multimodal reasoning models (vlaa-thinking). arXiv preprint arXiv:2504.1 1468, 2025.
Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.100 10479, 2025.
Chengke Zou, Xingang Guo, Rui Yang, Junyu Zhang, Bin Hu, and Huan Zhang. Dynamath: A dynamic visual benchmark for evaluating mathematical reasoning robustness of vision language models, 2025. / 00886
Chart-rl: Chain-of-thougle supervision and ... (chart reasoning model), 2025. Preprint. Available at: https://opponevvvvv.eee////////p 30t0000eeeee1004444e400222222223
15

---

Published as a conference paper at ICLR 2026
A APPENDIX
Organization In this Appendix, we provide in-depth descriptions of the materials that are not covered in the main paper, and report additional experimental results. The document is organized as follows:
Related Work
Vision-Zero Design Details
A.2.1
Prompt Setting
A.2.2
Dataset Preparation
A.2.3
Iterative-SPO algorithm
A.3
Experiments Setting
A.3.1
Model, Dataset and Baselines
A.3.2
Training and Hyperparameter Settings
Training Cost Estimation of Baselines
A.4
Supplementary Experimental Results
A.4.1
Comprehensive Evaluation Chart/OCR Tasks
A.4.2
Comprehensive Evaluation Vision-Centric Tasks
A.4.3
Comparison with Contrastive RLVR
A.4.4
Comparison with Chart- SpecttSppecillzed Models
A.4.5
Comparison CoTs on General QA Task Before and After Training.
A.4.6
Stability Analysis of Editor Capabilities
A.4.7
Parameter and Module Ablation
A.5
Limitation and Future Work
A.6
The Use of Large Language Models
A.1 RELATED WORK
Multi-Agent RL for Vision-Language Models. Self-play has emerged powerful paradigm for improving vision-language models without extensive human annotation. Konyushkova et al. (2025) introduce dialog games for VLM self-improvemmnt, where agents engage goal-oriented play centered on image identification, demonstrating iterative improvement through successful interaction filtering. Foundation model self-play (Dharna et al. 2025) shows how open-ended strategy innovation emerges from competitive interactions between models. SPIRAL (Liu et al., 20255 develops truly online multi-agen multi-turn RL, showing that training on zero-sum games improves reasoning capabilities that generalize to novel downstream tasks-partinn tasks-particularly relevant for the multi-turn nature of undercover games. Zhai et al. (2024) present the first framework to fine-tune VLMs using RL with task-specific rewards, achieving state-of-the the-trr- performance without expert data. RL-VIL RL-VLM-F (Wang ct al. 2024c) automatically generates reward functions using VLM feedback on image observation pairs, while Rocamonde et al. (2023) demonstrate that VLMs like CLIP serve zero-shot reward models with strong scaling effects.
Undercover and Social Deduction Games in AI. The undercover game paradigm has been explicitly explored in recent work. Dong et al. (2024) introduce the Sulti-Perrpective Team Tactic (MPTT) framework for "Who is Undercover?" integrating self-perspective, identitydeterrmination. self- reflection, and multi-round teammate finding to cultivate human-like language expression. Liang al.) (2023) implement an interactive multi-agent framework with comman-nn-th-lee capabilities. supporting strategic deception and voting mechanics directly applicable to our proposed VLM variant. Studies on social deduction games reveal important insights: Yoo Kim (2024) demonstrate that GPT- 4 achieved 80.65% accuracy in detecting deceivers in Mañia games versus 28.83% for humans, while Kim et al. 2024) identify four major reasoning failures obscured communicaiion-inndqque information processing, insufficient strategic thinking, lack of theory of mind, and poor temporal reasoning. However, these studies primarily focus on evaluating models' social capabilities through gameplay and rely prompt engineering emulate human-like behavior. In contrast, Vision-Zero the first approach to model the social reasoning game "Who is the Spy" as a self-play environment designed explicitly for training Vision-Language Models (VLMs) to enhance their performance.
16

---

Published as a conference paper at ICLR 2026
Game-Based Training and Evaluation for VLMs. Recent benchmarks reveal both the potential and challenges of VLMs in game environments. BALROG (Paglieri et al. 2024) aggregates 6 game environments testing short-term and long-term planning, finding severe deficiencies in vision-based decision-making even for GPT-4o. Wang et al. 2025a) present evaluation frameworks with core tasks directly relevant undercover game phases: Perceiving, Question Answering, Rule Following, and End-to-End Playing. Vedc (Zhang et al. 2025) reveals frontier models achieve only 0.48% completion rate on popular video games. Novel training approaches leverage games to improve VLM capabilities: G1 (Chen et al. 2025a) introduces VLM-Gym addressing the "knowing- doing" gap through mutual bootstrapping between perception and reasoning during RL training. while JARVIS-VLA (Li et al. 2025) achieves 40% improvement through Act from Visual Language Post-Training
Visual Description and Discrimination Tasks. Description- discrimination aligns naturally with undercover game mechanics. [Mennn & Vondrick (2022) introduce "classification by description" using descriptive features rather than broad categories, providing inherent explainability for why agents identify certain images as different. The odd-one-out paradigm directly maps to undercover game structure: Chito et al. (2025) present DINO- models for spatial and relational reasoning across multiple views, while Mohammadi et al. (2020) develop weakly-superviiee tasks showing high correlation with abstract visual reasoning providing foundations for identifying the different image among a set.
Multi-Agent Communication in Vision-Language Tasks. Multi-agent visual communication has seen significant progress. COMMA (Ossowski et al.) 2024) presents the first comprehensive benchmark for collaborative work among multimodal agents, featuring vision-language puzzles
requiring complementary information access. Qiu et al. (2022) model emergent communication through sketching between neural agents, defining metries for evaluating conventions applicable to how agents develop shared description strategies. Visual referential games promote systematic generalization: Denamganai et al. (2023) investigate compositionality with the Obverter architecture, while Lazaridou et al. (2018) show how referential games with pixel input enable linguistic communi- cation emergence, providing theoretical foundations for VLMs learning to describe and discriminate through game play.
Contrastive RLVR for VLM Post-Training. Recent work has explored contrastive RLVR as a scalable paradigm for post-training VLMs using automatically constructed image sets. MiCo (Chen et al. 2025b) is a representative example for qulti-image reasoning: it constructs image triplets consisting of two augmentations of the same image and a third, similar but different image, prompts the VLM to produce chain-of-thought comparisons and a ternary same/different pattern, and uses an automatically computed accuracy reward under Augmented-GRIP multi-image comparison. ViCrit (Wang et al. (0255)) instead builds apption-hallucination proxy task: starting from paragraph-length human captions, it injects a single subtle visual error and trains the VLM, via GRPO and an exact-match span reward, to localize the hallucinated phrase, thereby improving visual perception and hallucination robustness with fully verifiable supervision. GeometryZero (Wang et al. (025c) introduces group-contrastive policy optimization for geometry reasoning, contrasting muxiliary-construciie
to strengthen fine-grained
trajectories within a group to decide when additional constructions are beneficial. Vision-Zero is complementary to these contrastive RIVR approaches. Its gamified setting turns generic unlabeled image pairs into a scalable training environment, beyond the primarily perception- or cumparison-oriente. gains of prior contrastive RLVR tasks.
A.2 VISION-ZERO DESIGN DETAILS
Due to space limitations in the main text, this section elaborates on critical implementation details that were previously omitted. We organize the discussion into three parts: (i) the prompt design used during training, (ii) the construction pipeline of training data across domains, and (ii) the formal description of the Iterative Self- Play Policy Optimization (Iterative-PPO) algorithm.
A.2.1 PROMPT SETTING
The training process simulates gameplay using a sequence of structured prompts, guiding the model through two reasoning stages. We provide below the full prompts used for training on the CLEVR- based visual reasoning task. These prompts are carefully engineered enforce role-aware generation
17

---

Published as a conference paper at ICLR 2026
and role-consistent reasoning. The design enforces clear behavioral separation between SPY and NOT SPY players, ensuring that the model learns to generate clues consistent with its assigned role and to identify inconsistencies during inference.
For other domains, we adopt the same prompt structure with minimal domain-specific modifications: Chart-based data: The Clue Stage prompt specifies that "two attributes in the chart have been swapped" if the agent a SPY. This models table-style manipulations in bar plots or pie charts; Real-world image datasets: The SPY instruction modified indicate that "some objects parts of the scene differ" - a softer deviation that reflects the less controllable variation in natural seenes.
In actual training, prompts dynamically composed. We implement templating mechanism where placeholder tokens (c.g.. player id. round num) are automatically filled based on the synthetic episode. Similarly, role-specific insertions (c.g., "You are the spy with the different picture") are programmatically inserted based on the assigned role. This enables batched multi-agen simulation across thousands of games with minimal manual intervention, while still enforcing precise behavioral roles.
CLUE STAGE PROMPT
You are PLAYER (playerid] of the game "Who is the spy?". This is ROUND (round _umm).um Your task is to describe your picture in sentence, which can only be about the features of objects and the relationship between objects.
Your role: SPY (you have the blank picture)
Your role: NOT SPY (you have the same picture as most others)
Game Instructions:
There are num players num_players players in total. Only one player sees a different picture, and the pictures seen by all other players are the same.
Player who are voted as the odd player will be severely punished, so you should convince
others that you are not the odd player.
The odd player (spy) sees a completely blank image with chart table all, while all other players see the same table or chart.
You are the spy (you have the blank picture). Your picture is completely blank (no objects at all), while others see normal chart table.
You not the spy. Your picture is the same most other players' pictures.
What you should do:
1. Compare your picture with all previous clues to understand what others are seeing.
If you are NOT the spy: Describe clear, specific feature of your picture that does not overlap with previous clues. This will help establish what the "normal" picture looks like. If you are the spy:
(1) Infer what topic the chart that others seeing is about from other players' clues.
(2) Infer what attributes does the chart have and what are the value relationships between these attributes.
(3) Describe feature that should be correct about the chart that others are seeing but is not same any previous clue, in order to confuse other players.
Required output format:
First, conduct your private step-by-step reasoning may include suspicions, probabilities, deception, etc. Then, put your one-sentence description of your picture inside \boxed Example answer: \boxed(The cylinder is to the left of the cube]; \boxed (The yellow object is the largest object); \boxed (Te red object the smallest object].
18

---

Published as a conference paper at ICLR 2026
DECISION STAGE PROMPT
You are analyzing a game of "Who is the Spy?" where num players players have just provided clues about their pictures. Your task is to identify which player is the spy (the one with the blank picture) based on their clues.
Game Setup:
There num players players in total. Only one player sees different picture (the spy), and the pictures seen by all other players are the same.
In the spy's picture, no objects
replaced.
Each player provided a clue trying to describe their picture while avoiding suspicion. The spy tries to blend in by giving clues that seem consistent with what others see.
. Non-spy players try to give distinctive clues that help identify the spy.
Your Task:
Analyze all the clues and determine which player is most likely the spy. Look for:
1. Clues that seem inconsistent with the majority
2. Clues that are deliberately vague or generic
3. Clues that might be trying to mislead others Patterns that suggest deception or uncertainty
The image you shows the ORIGINAL picture (what non-spy players see). Compare the clues against this reference.
Required Output Format:
First, conduct your private step-by-step reasoning may include suspicions, probabilities, deception, etc. Then, put your final answer (PLAYER NUMBER or N/A) inside {.. If you are uncertain, you can answer N/A. Example answer: \boxed (1); (boxee(2);; (b)xed(3): (N/A).
\boxed N/A].
All Clues from the Clue-giving Stage:
A.2.2 DATASET PREPARATION
CLEVR-based Data. CLEVR is a controlled synthetic environment expressly built to study visual reasoning with minimal dataset bias and rich, program-level supervision. Its images are rendered from complete scene graphs, and the benchmark has become a standard stress-test for multi-step reasoning in vision-language systems (VQA/VLM). CLEVR scenes are procedurally sampled and rendered with Blender in headless mode, emitting both images and a fixed-format JSON that records each object's attributes and pose; the official generator exposes a simple CLI that renders images from the JSON scene specification. This design makes the pipeline lightweight and embarrassingly parallel. The "CLEVR universe" fixes the attribute vocabulary up front. Shapes are from cube, sphere, cylinder. Sizes are small, large. Materials are metal (shiny), rubber (matte). Colors come from an eight-color palette-commen commonly enumerated as gray, red, blue, green, brown, purple, cyan, yellow- and scenes are populated under simple geometric constraints (no interpenetration. all objects at least partially visible; randomized camera and lighting). These choices simplify perception so models' performance reflects reasoning rather than recognition shortcuts.
We automatically render 2k training pairs with the CLEVR renderer. Fach pair consists of an original image and modified image. Every image is accompanied by its scene JSON; the pair also carries compact change log (IDs of changed objects and their before/after attributes). For each scene, we sample 4-6 objects with attributes drawn uniformly from the CLEVR spaces above, while enforcing standard CLEVR placement rules (no overlap/interpenetaationn and sufficient margins so spatial relations are unambiguous). Camera pose and lights are jittered per scene, following the official generator's 's practice of randomizing viewpoint and illumination. Given an original scene JSON, we randomly select two objects and replace only their color and shape (leaving other attributes and the global layout unchanged unless minimal nudge is needed to maintain non-overlap). Concretely:
Step 1: Generate original JSON and render.
Step 2: Edit the JSON in place for two objects: shape <- new shape, color +- new color.
Step 3: Re-render with Blender from the modified JSON to obtain the paired image.
19

---

Published as a conference paper at ICLR 2026
CLEVR generation is stateless per scene and the official script supports GPU-accelerated Blender rendering (CUDA flag) in batch mode, so we parallelize across processes. On a single NVIDIA A100, end-to-end rendering of the 2k pairs completes in roughly 6 hours in our environment, consistent with the repo's 's recommendation to invoke Blender headless with GPU enabled.
Chart-based Data. In our preliminary attempts to generate chart data, explored direct editing of chart images via NanoBanna and ChatGPT; however, we found this approach extremely challenging, because current image editing models and tools struggle reliably control fine-grained graphical attributes (such as exact axis ticks, bar widths, label alignment, and consistent color scales) without introducing visual artifacts or distortions (a known limitation of current image editing in diffusion/in- painting frameworks) Therefore, to achieve stable, controllable editing and generation, we eventually adopted the following pipeline:
We let GPT-4o ingest the original chart image and output a JSON file encoding every attribute's numerical value (e.g. data points, axis bounds, legend mapping) as well as auxiliary metadata (chart type, color scheme, layout constraints);
We prompt GPT-4o swap two attributes arbitrarily and rewrite the JSON accordingly;
We feed the new JSON into Python plotting module to render new chart.
This paradigm is robust to typical failures of AI editors and fully leverages the strong captioning and scene-parsing abilities of current multimodal LLMs.
For our dataset, we randomly sampled 1,000 original charts from ChartQA's training set to ensure visual and data diversity, so that derived pairs reflect ChartQA's spectrum of chart styles and com- plexity. ChartQA's A' dataset spans three canonical chart types line plots, bar charts, and pie charts- capturing both simple and complex variants in real-world sources. Thanks to the fully automated pipeline, the entire generative process incurs only on the order of tens of US dollars.
A.2.3 ITERATIVE-SPE SPO ALGORITHM
In this section, we describe the algorithm of Iterative Self-Play Policy Optimization (Iterative-SPe) algorithm, as detailed in Alg. [ As mentioned in the main paper, Iterative-SPO achieves sustained performance improvement by incorporating supervision signals into the self- eelf-pla framework through a two-stage alternating training procedure.
Algorithm 1 Iterative Self-Play Policy Oprimizattion(teniive--P9)
Input: Role set KC (spy) U (ci...., Cn. }; reference policies n, 1 may: hyperparams B, A, a, Telues Ps
Tocer -.T, Than Tha Kmin, P; learning rates nj, 776-
1: Init R.F.F.F... b, 4 0, beiv 4 O; Stage switch metries ačc O, na f 0; Stage m 4 (Decision).
2:
for f. 1,...,77 do
3:
if mn then
CLUE Stage
4:
Each player gives clue us ~ T Ie, h)) based on the historical dialogue & and input picture 4. Obtain votes from the decision stage v = (Ve, Ver...., Ven,)) and u. <- Σ}= Vey.
5:
6.
Zero-Sum Rewards: yolue -- (t. vc); relue 8 (v. ve) X (Ve, - V..) for = 1,..., Rec
no
7:
Role Advantage Estimation: b, t ab, + (1 a) dav + abav (1 Q) 1 ne Eg celue clue beiv for j = 1, I.... nec RAE-based Advantages: Ag + r clue bsi t t
8:
9:
else
DECISION Stage
Each citizen casts vote Sc, go(- H) based on the clue information H and the input image Ix- dee dec &- 0.5if 8e, Ø (unsure); le +- - 1else (wrong). Reward: rhe <- lifse, 8" (correct); Te
11:
12:
Group-norm Advantage: A (r p=)/(a, +
13:
Policy update: Apply KL-regularized policy gradient as Eq. B or Eq. 6 update xe or qe.
15:
Stage Switch: Calculate average prediction accuracy acc, and "n/a" rate na. of players the decision
stage within a batch round: acc, = B Σ, 1 {arg max, go H.) = = na, = B E, (0()) H.).
16:
Update EMAs ačc p ačc + (1 p) acc;; na pa + (1 - p) na. d + + 1.
17:
if m and acc 2 Tace and na E rma and d2 Kmin then m 1, d+
if m and (1 acc 7 na 2 Thl) and d Kmin then m 0, d+
return 0,0
20

---

Published as a conference paper at ICLR 2026
A.3 EXPERIMENTS SETTING
In this section, we provide a comprehensive account of the experimental settings used throughout our study. We detail the choices for (1) models, datasets, and baselines, (2) training procedures and hyperparameter configurations.
A.3.1 MODEL, DATASET AND BASELINES
Models. We evaluate three open-weight vision -lngune models. Owwnn.5-VLL-7--nns- is a 7B instruction-tun -unnneetoontunedddeeeeee VLM from the Qwen family; it upgrades the vision stack with window-attention ViT and SwiGLU/R /A/..U/////S MSNorm, and is designed for strong document/chart understanding, structured ISON outputs, grounding, and even long- ain-video/ageettcc use cases. Intern is the 8B member of the In3ernVL3 series that follows a "ViT-MLP-LLA encoder with a Qwen2.5-7B language core via an MLP projector; it introduces Variable Visual Position Encoding and native multimodal pre-training improve multi- multi-image/vide) perception and OCR/chart/docunent reasoning. InternVL-144 is an earlier 14B rision-languel foundation model (224-px variant) trained large-scale web corpora such LAION, COYO, CC12M/CC3M, SBU, and Wukong, and is commonly used for zero-shot classification, retrieval, and captioning baselines.
design by pairing an Intern ViT-300M vision
Datasets. We evaluate broad suite of public benchmarks. Math (Lu al. 2024 combines 6,141 problems drawn from 28 existing multimodal math datasets plus three newly created sets (IQTest, FunctionQA, PaperQA) probe fine-grained risual-mathematical reasoning. Vision (MATH-V) [Wang et al. 2024b) curates 3,040 competition problems with visual contexts across 16 disciplines and five difficulty levels for rigorous multimodal math assessment. We-Math QQiao et al. 2024) collects 6.5K visual math problems organized over 67 hierarchical knowledge concepts to analyze IMM reasoning behaviors. Math (Zhang et al., 2024b) offers 2,612 diagram-based problems, each converted into six modality variants to stress-test vision vs. text contributions. Log- VcVista (Xiao et al. 2024) targets logical cognition in visual contexts with 448 multiple-choice questions spanning five task types and nine capabilities, each paired with human rationales. Dy- naMath Zou et al. 2025) is a dynamic robustness benchmark that perturbs seed questions (e.g.. values, function graphs) to test stability of visual math reasoning. ChartXIV (Wang et al.. 2024d) is realistic chart understanding benchmark comprising 2,323 diverse charts from scientific papers with both descriptive and reasoning questions that stress-test MLLMs beyond lemplate-basedt chart QA. FunctionQA A (Lu et al. 2024) is a visual mathematical reasoning dataset focusing on algebraic rea- soning over functional plots, requiring emeeggrined interpretation of curves, variables, and equations. PaperQA (Lu et al. (2024) is scientific reasoning dataset built on academic paper figures, designed to evaluate models' ability interpret complex visualizations and answer content-based questions about scientific literature. ReachQA (He et al. (2025) is synthesized chart reasoning dataset containing 3k ceassoninn-intensii charts and 20k Q&A pairs, constructed to enhance both chart recognition and higher-level visual reasoning in MLLMs. RealWorldQA (Zhang et al. 2024c) (released with Grok- 1.5 Vision) contains 700+ real-scene images- maag vehicle-captured each with a question and easily verifiable answer. MMVP (Zhang et al. 2024a)) is built from "CLIP-blind" image pairs to assess nine basic visual pattern failures via 300 VQA items. BLINK (Fu et al., 2024) recasts 14 core perception tasks into 3,807 multiple-choice questions that humans solve "within blink" but remain challenging for current MLLMs. MuirBench (Wang et al. (224a)) focuses on multi-image understanding with 11,264 images and 2,600 MCQs across 12 tasks and 10 relation types, including paired unanswerable variants for robustness.
Baselines. We benchmark against five recent multimodal reasoning baselines. RI-One Vision- RI-OneVisioon- 7B (Yang et al. 2025b) is a QQwen.5-VLL-basee VLM trained on the RI-OneVision corpus with cross-modal reasoning pipeline that converts images into structured textual representations to enable step-wise "R1-style multimodal reasoning. introduces the MMK12 dataset and employs rule-based reinforcement learning with online filtering and a two-stage training strategy to stabilize multimodal math reasoning at the 7B scale. VLAA- Thinker-7B (Zhou et al.) 2025) is trained on VLAA- LLAAA-Thininn--- corpus of step-by-step visual reasoning traces with both SFT and RL splits-used to probe SFT vs. RL for R1-like reasoning and reporting SOTA on OpenCompass as of April 2025. OLenVLThinker- 7B (Deng et al.) (025)) follows an iterative SFT-RL regimen (e.g., GRPO) that consistently improves performance Math Bistt//EMMAHHallusssoooBBBBee evidencing the synergy of SFT and RL for complex multimodal reasoning. ViGal. (otatinRotation) (Xie et al. 2025) post-trains 7B model purely via RL on
Ka--Eurkka-Qwen
(Meng et al. 2025)
21

---

Published as a conference paper at ICLR 2026
simple arcade-style games (Snake and 3D rotation puzzle), where combining the two games yields stronger out-of-domain generalization (e.g., math, geometry) than either alone.
A.3.2 TRAINING AND HYPERPARAMETER SETTINGS
To facilitate stable and effective training, selected VLMM-I the foundational model architecture for the Vision-Zero framework, ensuring compatibility with established benchmarks. The detailed hyperparameter configurations employed in our experiments are summarized in Tab. Specifically, all Vision-Zero models underwent training for 100 iterations across diverse datasets, followed by rigorous evaluation of their post-training performance to measure generalization and robustness.
Table 5; Vision-Zero training hyperparameters.
Symbol
Meaning
Value
The
Number of civilians per round
4
8, A
Clue-stage reward scaling/ clue regularization coefficients
0.1
a, P
Decay cocfficients for role advantage (a.) and accuracy / "n/a" rates (p). KL regularization weights (decoder clue)
0.95
Tdecy Tclue
0.04
Tace
Stage-switch (up) threshold for accuracy
0.9
Terr
Stage-switch (up) threshold for error rate
T
Stage-switch (up) threshold for "n/a" rate Stage-switch (down) threshold for "n/a" rate Minimum number of rounds per stage
0.5
Tia
0.1
Kmin
5
P
Patience (number of rounds before forcing change)
20
# iterations
Total training iterations Training batch size
100
Batch size
128
We utilized training batch size of 128, precisely calculated as the product of hprrc_ppr_nae (8),
cantuulaaiin aationstepssss (16), and mum_generatio chosen batch size aligns with standard VLM training paradigms, effectively ensuring stable opti- mization dynamics. Morcover, our training setup is fully integrated with state-of-the-art optimization techniques and libraries, including F and DeepSpeed ZeRO-3, maximizing training efficiency and scalability while maintaining full methodological consistency with VLM-R1 standards.
(8). This carefully
torchrun --ngroo_per__rooee-e
"nnddes="1"
-=ndderran==0""" \
-master_.cdar=112..00001
-master_port==12355""
htc//ppee_tl//rrpojoonn..g
lel. json
\
dataset_name
-use_dyynaiccdattsse:
-epoch_size SEPOCH_SIZE
-data_geemerator_tyye clevr_ seor__sftdiff
-clevv__images_diir $CLEVR_IMAGES_DD
-clevr_scenes_diir
-=clev__ummpiaaeer SNUM PLAYERS
-=clezz_nm_roundde SOUM_ROUUDS \
-training_phase STRAINING PHASE
-aata_generatorsseed 42
max_anyyes__num
max_prompt_iennghh 8000
--max_comples mmax_completiio___ennghhh 512
-nuu__generations
-pee__device_train_bbccchh 8
- 16
22

---

Published as a conference paper at ICLR 2026
-Logging_steps 1
--bf16
--torch_dtypp bfloat18
--beta 0.04
--report_to wandb \
"graadiett_checkoointe true \
=attn_implementation flash_attention
-mum_trrain_epoch 15
--Learnin__aate 1e-5
-warmup_ratio 0.1
-Ir_scheeulle__ypee cosire
--run_name FRUN_NAME
-=save_steps 5
--avv_onny__ootee true
-reward_funcs
clerr__ecision
wcdsspcchh_batcche False
-val_split_ratio 0.0 -muu_iteeaaions 1
A.3.3 TRAINING COST ESTIMATION OF BASELINES
Here we report how the label cost and training time cost were estimated in Tab[Jof the main text.
Label cost (tokens). For label cost, we count tokens generated by teacher or judging LLMs during data curation. For R1-One Vision, VLAA-Thinkes Open VLThinke and MM-Eureka we directly reuse the token counts reported in Perception-R1 (Xiao et al. 2025). All token counts are recomputed using the Qwen2.5 tokenizer for consistency. For ViGaL, it collects trajectories in game environments and calls GPT-4o only once to draft static reasoning instructions shared across all samples, so we set its label cost to zero. Aision-Zero uses fully CLEVR scenes with sule-based rewards and does not invoke any external LLM during data construction, thus its label cost is zero.
RL Training time cost (GPU-hours). All compared methods use GRPO-style RI., so we approxi- mate their RL compute with unified per-sample cost. From Perception- RI we know that RL training on & 1.4K distinct samples for 25 epochs (about 3.5 x 10 RL samples in total) consumes 167.4
A100-hours after accounting for judge utilization. This gives an average cost
167.4
= 4.8
10 A100-hours per RL sample.
(11)
3.5 x 104
For each baseline, we extract from the original paper the number of distinct RI. examples and epochs, compute the total number of RL training samples NRL. and estimate its RL. cost
T R NRL-
(12)
When the epoch count is not reported, we conservatively assume epoch, the reported GPU- hours are lower bounds. For VisionZerooQwenn-77 (CLEVR), instead of using the above approxima- tion we directly measure the wall-clock RL training time on Ax0000G GPUs.
A.4 SUPPLEMENTARY EXPERIMENTAL RESULTS
A.4.1 COMPREHENSIVE EVALUATION ON CHART/OCR TASKS
While we partially presented Fision-Zerv results on selected chart and OCR tasks in the main text, Tab. illustrates comprehensive evaluation across an extended set of tasks. Notably, VisionZero- Qwen-7B consistently surpasses baseline methods across diverse OCR and chart- based tasks. Par- ticularly, VisionZero-Qwen-T7 (Chart) exhibits superior performance and significant capability enhancement due to its targeted training chart datasets. For example, on the InfoVQA benchmark, AisionZero-Qwen--e (Chart) improved the performance of the original model by approximately 4%, outperforming the state-of-the-ar ViGal by 14%. This substantial improvement arises because baselines trained extensively on reasoning datasets typically suffer from task overfitting, whereas Vision-Zero circumvents this limitation concurrently fostering multiple capabilities.
23

---

Published as a conference paper at ICLR 2026
Table 6: Performance comparison between Vision-Zero and other models on OCR, Chart, and
Document Understanding. All models are evaluated using the open-source platform VLMEvalKit.
Model
A12D
ChartQA
TextVQA
DocVQA
InfoVQA
OCR Beneh
SEEDBench2
Proprietary Model
GPT4o
84.4
85.7
822
91.1
78.2
73.9
72.0
Performance on Owen2.5 V-VB-B
84.7
85.5
70.4
RI-OneVision--B
82.2
81.0
66.4
MM-Bucck--Qwen-7B
84.1
77.3
81.1
71.7
86.7
68.2
VLAA-Thinker-788
84.0
84.3
82.9
92.7
71.8
86.9
67.4
OpeuVLThinker-778
81.8
83.3
68.0
WGGal-SmkkeRRottion
84.5
79.9
82.2
92.5
72.7
86.8
69.1
Ysion/Zero-Qwen-7B (CLEVR)
84.5
85.3
94.9
88.1
69.5
Viion/Zero-Qmen-78 (Chart)
85.8
87.2
86.4
95.9
86.5
89.0
70.9
VisionZero-QwennT11 (Real-World)
84.8
86.3
85.4
95.2
82.3
88.5
69.8
A.4.2 COMPREHENSIVE EVALUATION ON VISION CENORCCENTRIC TASKS
Moreover, as shown Tab. Vision-Zers achieves top-tier performance across six distinct vision- centric task groups. VisionZero Qwen-7B (CLEVR), whose training data has stronger visual emphasis compared to VisionZero-Queen-TB (Chart), obtains even better results. Specifically, VisionZero- Qwen-7B (CLEVR) surpasses state-of-the-arr baselines by 1.1% average across the six task categories. These results underscore the potential and applicability of Vision-Zero as the first sero-human-in-thee-loo training paradigm.
Table 7: Performance comparison between Vision-Zero and other state-of of-the-arttttttto models on Vision- Centric benchmarks. All models are evaluated using the open-source platform Vi.EEal
Model
RealworidQA
MMVP
MMStar
BLINK
MuirBench
CRPE
Avg.
Pmprietary Model
GPT4o
75.4
86.3
68.0
68.0
Performance on Qwen2.5
64.6
55.2
RI-OneVision-7B
58.0
61.3
57.8
48.7
46.3
75.3
57.9
MM-Euucka-Qwwnn-7B
66.1
74.3
65.9
54.0
61.1
76.7
66.4
VLAA-Thinker-7B
71.6
60.4
53.0
74.6
Op
60.2
71.3
59.1
49.9
52.8
75.8
61.5
VGGaL-SnakeRRotttion
66.5
74.6
62.6
55.6
57.9
76.7
65.7
Vislon/zeroOwwen-BB (CLEVR)
68.5
79.2
65.2
57.2
59.4
76.9
67.7
VisionZero-Qwenn- (Chart)
77.9
64.7
56.1
76.2
VisionZZen-Qwwen-TB (Real-World)
68.5
79.5
65.8
57.5
59.8
77.0
A.4.3 COMPARISON WITH CONTRASTIVE RLVR
To evaluate the advantages of Vision-Zero over prior contrastive RL.RR methods in enhancing VLM reasoning capabilities, we conducted direct comparison under identical training conditions with MiCo (Chen et al. 2025b).
Specifically, to align with the MiCo-7B setup, we trained Vision-Zero on the OmniEdit dataset using the Qwwe2.55-L--7 model for 100 iterations with batch size of 128. In contrast, MiCo-7B was trained for 600 iterations with the same batch size. Although Vision-Zero adopts a multi-round training paradigm, its overall training cost remains comparable. As shown in Tab. g evaluated Vision-Zers reasoning performance on six benchmark datasets consistent with those used for MiCo-7B, with MiCo's s results taken directly from its original paper.
24

---

Published as a conference paper at ICLR 2026
Table 8: Performance Comparison of Qwen2.5VL-7B and MiCo-7B across multiple benchmarks.
Model
MuirBench
Blink
Hallusion
MMStar
MMMU
Math Vista
Qwen2.5VL-7B
58.4
55.5
69.5
64.1
54.1
67.1
MiCo-7B
60.5
57.2
69.6
65.6
54.8
67.9
VisionZero-Qwen. Qwn---een---BOiiEdid
62.4
58.9
71.2
66.2
55.7
69.1
The results demonstrate that, whereas MiCo is primarily optimized for multi-image difference reasoning, Vision-Zero benefits from strategically constructed training environment that better targets the development of reasoning skills. As a result, Vision-Zero consistently outperforms MiCo across all six general reasoning benchmarks. This indicates that the combination of a self-play game mechanism and a strategic interaction environment enables Vision-Zero to equip the model with significantly stronger general-purpost reasoning capabilities than contrastive RLVR approaches.
A.4.4 COMPARISON WITH CHART MODELS
To thoroughly evaluate whether Vision-Zero can enhance model's chart understanding capabilities and establish its superiority in the chart reasoning domain, we conduct a dedicated comparison against models specifically fine-tuned for chart understanding, including ECD Yang et al., (025a). Sispoke-MiiCChatt-2 (Tang et al.) 2025) and Chart-R1- R1-7B 2025). We compare the number of chart images, the number of QA annotations used for training, the final performance on standard chart understanding benchmarks.
Table 9: Performance Comparison of VisionZeenoQQwen-BBBchaarr and chart-specialized models across multiple benchmarks.
Model
Chart Number
QA Number
ChartXiy_RQ
ReachQA
Avg.
Qwen2.5VV.-77b
42.5
50.8
46.7
ECD LCDD-Owe22.VVL
10.5k
320k
40.2
53.5
46.9
Bespoke-MiniChan MiniChart
13.0k
91k
46.2
54.0
50.1
Chart-R1-7B
21.9k
32.7k
46.2
2.0k
0
46.6
53.8
50.2
As shown in the Tab. 9 SaiionZero-Qwen-- (Chart) achieves comparable or superior performance to models trained on large-scale chart-specific data-wither requiring any QA supervision. Notably, on ChartXIV _QA, VisionZero- 7B delivers a 4.1% improvement, surpassing models that were explicitly trained on chart datasets. These results highlight Vision-Zero's ability to generalize across domains and demonstrate its potential as a cost-effective and competitive training framework for enhancing model performance diverse visual reasoning tasks.
A.4.5 COMPARISON OF CoTs ON GENERAL QA TASK BEFORE AND AFTER TRAINING.
To intuitively demonstrate how the "Who is Spy" game enhances model performance on general QA tasks, we present illustrative examples comparing model thinking before and after training identical general QA tasks. As shown in the boxes below, the trained model clearly demonstrates improved thought processes and logical reasoning, explicitly understanding the steps required. Additionally, more accurately focuses on critical information in images, exhibiting enhanced key content-capturing abilities, directly leading more accurate responses.
This improvement arises from two primary factors: first, the capabilities required by the "Who is Spy" game closely overlap with those needed for general QA tasks. For instance, during the clue-giving stage, the model must identify differences between images and text, directly training fine-grad eneiggai visual comparison and comprehension skills. Second, the strategic and challenging environment of the "Who is Spy" game encourages models to cultivate meticulous reasoning and logical thinking habits during responses, further contributing to improved performance in general QA tasks.
25

---

1. Count the missing bricks in the top row: brick. 2. Count the missing bricks in the second row: 2 bricks. Published as a conference paper at ICLR 2026
Examplel: Comparison of Co'ss Before and After Training
Question: How many countries have people working for more than 35 hours over the years?
The graph compares the average usual weekly hours worked by women over the age of 15 across three countries (Romania, Portugal, and Switzerland) from 1995 to 2007. Each country represented by a different colored line: Romania blue, Portugal is orange, and Switzerland dark blue. From 1995 to 2007, the line representing Romania (blue) is consistently above the 35-hour mark on the y-axis, which measures average usual weekly hours. The lines for Portugal and Switzerland are below the 35-hour mark throughout this period. Thus, only country, Romania, shows average weekly hours worked exceeding hours over entire period displayed.
Untrained Model:
To determine how many countries have people working for more than 35 hours on average weekly over the years from 1995 2007, let's analyze the data shown in the graph step by step: 1. **Identify the countries represented:*h +* Romania (in blue) Portugal (in orange) Switzerland (in light blue)
Trained Model:
**Annlyze the hours worked for each country:* Romaania: The blue line representing Romania shows that the average usual weekly hours worked remain consistently above 35 hours throughout the entire period from 1995 to 2007. "*Portugal: = The orange line representing Portugal shows that the average usual weekly hours worked also remain consistently above 35 hours over the same period. **Switzerlandd#* The light blue line representing Switzerland shows that the average usual weekly hours worked remain consistently below 35 hours over the entire period.
3. +*Dtermine the countries with averages above 35 hours: From the above analysis, Romania and Portugal have average usual weekly hours worked consistently above 35 hours. Switzerland has average usual weekly hours worked consistently below hours.
Therefore, the number of countries with people working for more than 35 hours on average weekly over the years is #*24*
Example2: Comparison of CoTs Before and After Training
Question: How many bricks missing in the wall?
To determine how many bricks are missing in the wall, we need the number of bricks that are not present in the wall structure. Let's analyze the image step by step.
Untrained Model:
1. **Identify the pattern the missing bricks:*s.. The missing bricks form zigzag pattern from the top left to the bottom right of the wall.
**Count the missing bricks in each row: ToP row 1 brick missing. #*Sccond row:
bricks are missing. *+Third row:** bricks are missing. **Fourth row:** bricks are missing. **Bottom row: brick is missing.
3. **Sum the number of missing bricks:* Add the number of missing bricks from each row: 1 + ++++3+ 2 1 Therefore, the total number of bricks missing in the wall is 9.
Trained Model: find out how many bricks are missing the wall, we need to count voids where
bricks should be but aren't t.
Count the missing bricks in the third row: 2 bricks. Count the missing bricks in the fourth row: brick.
Adding these up, we get a total + 2 bricks missing.
26

---

Published as a conference paper at ICLR 2026
A.4.6 STABILITY ANALYSIS OF EDITOR CAPABILITIES
To assess whether Vision-Zero heavily depends on high-performance image editors, we introduced 20% noise into the RealWorld dataset to simulate scenarios of partial editing failures. Specifically, we replaced 10% of the modified images (provided to the spy) with blank images and another 10% with images identical to the originals (provided to civilians). We then trained the model on this noisy dataset and evaluated its performance on the validation set.
The experimental results, summarized in the Tab. TO indicate that Vision-Zero maintains performance improvements even with partially failed edits. Although the improvement slightly lower compared average accuracy increase of 2.8% over the
to training without noise, the model still achieves baseline. This demonstrates Vision-Zero's 's strong robustness to editing failures, highlighting that can enhance model performance without relying heavily sophisticated image editors.
Table 10: Performance of the model trained on dataset containing images that failed to be edited.
Model
Mathvista
MathVison
WeMath
MathVerse
LogieVista
DynaMath
Avg.
Qwen2.5V1. 5VL.-b
49.0
- (Real- World)
72.4
28.0
39.5
52.2
50.3
22.1
44.1
VislonZero-Qwen-7B )
72.2
27,5
39.2
52.4
50.1
43,9
We also present the model's responses to failed image edits (when the spy input is a blank image) in the block below. It can be observed that even when faced with failed edits, the model continues to reason effectively, leveraging alternative available information to maximize rewards. Thus, even image pairs resulting from unsuccessful edits remain valuable training samples for the model.
Example3: Reasoning of the model when editing failure (the spy obtained a blank image)
Civl: The red cube is in front of the purple cylinder.
Civ2: There are two small cubes, one blue and one purple, near the purple cylinder and the red cube.
Civ3: The purple cylinder is behind the red cube and the two small cubes.
Spy: ***Player clue=#; The cube in front purple cylinder. - **Player clue There are two small cubes, one blue and one purple, near the purple cylinder and the red cube. **Player 3's clue**; The purple cylinder is behind the red cube and the two small cubes.
Given that the SPY, my image has two different objects from the others, but their positions remain the same. Based the clues:
The common elements seem to be: A red cube A purple cylinder A blue small cube A purple small cube
My differences could be: The color shape of one of the small cubes (blue or purple) Possibly the position size, but as per rules, only color and specific features should be described. Since descriptions so far mention color and proximity, I should maintain consistency and describe features that align with their clues to avoid suspicion.
Description: The red cube is in front of the purple cylinder and the two small cubes.
A.4.7 PARAMETER AND MODULE ABLATION
In this section, conduct ablation studies the key parameters introduced in Vision-Zero, including the number of civilian players, the number of clue rounds, and the Role-Advantage Estimation (RAE) module, to analyze the robustness of Vision-Zero and the contribution of each component.
We first evaluate whether Vision-Zero can improve model performance under different numbers of players. We train models with 2, 3, and 4 civilian players (with the spy always fixed one) and
27

---

Published as a conference paper at ICLR 2026
report the results in the Tab. 11 Vision-Zero consistently improves model performance across all player settings, demonstrating its robustness to variations in player count. As the number of players increases, the game environment becomes more challenging, resulting in more reasoning steps and reward signals per sample, thus yielding greater performance gains. Even the minimal two-player setting, the model still achieves average improvement of 1.3% across six benchmarks.
Table 11: Performance of the model under different numbers of players.
Model
Mathvista
Math Vison
WeMath
MathVerse
LogicVista
DynaMath
Arg.
Qwen2.5VI.77b
68.2
25.4
36.1
49.0
47.2
20.9
41.1
VisionZZern-Owen-TH (CLEVER Civ=2)
69.7
26.5
37.2
50.6
48.9
21.2
42.4
ViionnZero-Qwenn77
72.6
28.1
VisionZeero-wwen-7B (CLEVER Civ=4)
73.2
28.0
40.0
52.1
51.8
24.1
44,9
We also examine the effect of varying the number of clue rounds in each game by training models with 1, 2, and 3 clue rounds. As shown in the Tab. [2] the performance gain is small when only single clue round used, due insufficient information available for the decision stage and limited opportunities for multi-step integration during training. In contrast, using two or three clue rounds leads to notable improvements, achieving average gains of 3% and 4. across six benchmarks. As the number of clue rounds increases, the model is required to process and integrate more information, resulting in progressively stronger performance.
Table 12: Performance of the model under different round numbers of clue stage.
Model
Mathvista
MathVison
WeMath
Math Verse
LogicVista
DynaMath
Avg.
Qwen2.5V1-2
68.2
25.4
36.1
49.0
47.2
20.9
41.1
VaionnZero-Owen-TB (CLEVER Clue Rd = 1)
68.0
26.5
36.5
48.2
47.2
21.2
41.3
VetonnZerr-Qwwn-7B (CLEVER Clue.
72.6
50.1
22.3
YeionZZero-Qwen-2E (CLEVER Clue
73.1
29.0
40.5
52.2
52.0
24.3
45.2
To assess the importance of the RAE module, we compare models trained with and without RAE. In the no-RAE setting, we update the model weights directly based on the reward after the clue stage without subtracting the role-ad vivatade baseline. The results in the Tab. 13 show that removing RAE leads to negative performance gains. This occurs because spies and civilians inherently possess asymmetric information due differences in their assigned images and roles. As result, the game can be intrinsically easier or harder depending on the role. Without adjusting for role advantages, directly backpropagating rewards based win/loss signals fails reflect the model's true performance level, preventing effective learning. These findings highlight the critical importance of the RAE module Vision-Zei Zero.
Table 13: Performance of the model under w/ and w/o RAE module.
Model
Mathvista
MathV
WeMath
Math
LogleVista
DynaMath
Avg.
Owen2.5VL-7b
68.2
25.4
36.1
49.0
47.2
20.9
41.1
VisonZero-Quenn (CLEVER RAE)
72.6
28.1
39,8
51.9
50.1
22.3
44.1
VisionZzer-Qwen--7 (CLEVER w/o RAE)
65.2
21.3
30.1
47.2
44.3
16.1
37.4
A.5 LIMITATION AND FUTURE WORK
In this section, we discuss potential limitations of Vision-Zero and outline directions for future research. Firstly, the implementation Vision-Zero relies on image editors to produce differentiated image pairs. Consequently, its application might be limited in highly specialized or resource- constrained domains, such as medical imaging, scientific charts, and remote sensing, where such edited data might not be readily available. Secondly, the current framework is designed around single-image observations and pairwise edits. Extending this framework to richer modalities, includ- ing extended videos, complex multi image contexts, interactive 3D environments, may require significant redesign of the game mechanics and training algorithms. Addressing these limitations constitutes essential direction for future work.
28

---

Published as a conference paper at ICLR 2026
A.6 THE USE OF LARGE LANGUAGE MODELS
In this work, we used ChatGPT-4o (OpenAT) and Gemini 2.5 Flash (Google) to assist with image generation for dataset construction. Specifically, the models were prompted to edite visual content used in training datasets. We gratefully acknowledge their utility in facilitating efficient data synthesis.
29


Reviews:
[{"review_id": "T30YO8Rths", "reviewer": "Reviewer_gbyd", "summary": "The paper presents VISION-ZERO, a framework that enhances the reasoning capabilities of vision–language models (VLMs) through self-play and RLVR, thereby reducing reliance on costly human-annotated data. Specifically, the framework constructs a strategic environment inspired by the game “Who Is the Spy”, in which training data are naturally generated through self-play. The resulting data are then used to iteratively optimize VLMs using the Iterative-SPO algorithm. Extensive empirical results on multiple benchmarks demonstrate that VISION-ZERO surpasses baseline models, significantly enhancing the reasoning ability of VLMs while avoiding cross-capability negative transfer and substantially reducing dataset construction costs.", "questions": "1. Ablation studies on critical components and hyperparameters (e.g., RAE and the number of civilians) would be helpful for further understanding the contribution of each module and the sensitivity of the framework.\n2. If **VISION-ZERO** is trained on static datasets, is there a possibility of **knowledge leakage** when the model encounters the same image pairs during training?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. This paper is well-motivated. Existing RLVR methods rely heavily on carefully constructed datasets. By introducing self-play, this method enables reasoning improvement on label-free data, helping to extend RLVR to broader domains where manual annotation is difficult or impractical.\n2. The experiments are comprehensive. Evaluations on **Task Generalization Capability**, **Cross-Capability Negative Transfer Mitigation**, and **Low Dataset Construction Costs** strongly demonstrate the superiority of **VISION-ZERO**, providing solid evidence for its effectiveness in mitigating data scarcity and improving scalability.\n3. The framework proposed in this paper is novel. The *“Who Is the Spy”* game environment appears to be highly scalable and can be readily extended to diverse domains. This offers a new paradigm for enhancing the capabilities of vision–language models (VLMs).", "weaknesses": "1. The data curation pipeline is primarily focused on the **CLEVR** and **Chart** domains. As mentioned in *Appendix 2.2*, directly editing chart images using **NanoBanna** and **ChatGPT** can be extremely challenging. This raises concerns about the scalability of the proposed framework, particularly regarding how to generate image pairs with arbitrary data.\n2. The overall design of this framework appears to be sophisticated. The success of the framework relies on the effective coordination among **self-play in the game environment**, **Self-Play Optimization in the Clue Stage**, and **RLVR in the Decision Stage**. A deeper discussion on the stability of this framework would be valuable for understanding its robustness and generalization.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zXyglaliL7", "reviewer": "Reviewer_CCNi", "summary": "This paper introduces a zero-human-in-the-loop training framework for VLMs using a gamified self-play mechanism inspired by social deduction games like \"Who Is the Spy?\", eliminating the need for expensive human-annotated datasets.\nThe game involves multiple agents (civilians and a spy) who observe slightly different images and must describe or deduce the differences through dialogue. The model improves via Iterative Self-Play Policy Optimization (Iterative-SPO), which alternates between self-play and reinforcement learning with verifiable rewards (RLVR) to avoid performance plateaus.", "questions": "- Will the models cheat or find shortcuts via editing methods? Because the editing possibilities/categories are pre-defined. (e.g., swapping numerical attributes in chart data.)\n- Is this idea novel in the multi-agent community? I have noticed that Role-Advantage Estimation is based on the previous research. What's your differences?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. First to apply self-play to VLMs in a gamified adversarial setting.\n2. Propose a novel training algorithm that avoids equilibrium stagnation by alternating between self-play and RLVR.\n3. The evaluated domains are quite comprehensive, including math, charts, vision-centric, and simulated images.", "weaknesses": "- The improvement of the Chart/OCR is marginal, as shown in Tab.2.\n- The data preparation cost in the Tab.3 is ambiguous, where the statistics might be further explained.\n- The improvement over the InternVL series is marginal compared to the QwenVL series, as shown in Tab. 4, casting doubts on its generability.\n- The intuition/theory between the \"who is a spy\" proxy task and the general question-answering tasks remains unclear.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ItIYyQY0yC", "reviewer": "Reviewer_K9Zp", "summary": "This paper introduces Vision-Zero, a framework inspired by the “Spy vs. Player” game concept. The idea is creative and brings an interesting self-play strategy from LLMs into the vision-language domain (VLMs). The design is simple yet somewhat innovative. However, the overall performance improvement is modest and appears constrained by the capability of the underlying image editing models.", "questions": "1.\tHow fair is the comparison with prior works that do not assume access to camera poses? How does MultiViewPano perform when camera poses are inaccurate or unavailable?\n2.\tHow robust is the proposed method to poor-quality SEVA outputs? Can the authors provide visualizations of such cases?\n3.\tCould the authors expand the ablation study and visualize intermediate outputs to clarify which modules contribute to which improvements?\n4.\tHow are geometric distortions and artifacts handled during repeated enhancement steps?\n5.\tWhy does the proposed method not outperform CubeDiff, especially for indoor scenes?\n6.\tCould the authors ensure that all best-performing metrics in Table 1 are boldfaced for readability?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1.\tThe paper identifies two valuable and timely challenges in multimodal learning: data scarcity and knowledge ceiling.\n2.\tIt presents a clear motivation by extending the concept of self-play from large language models (LLMs) to vision-language models (VLMs).\n3.\tThe framework generates data using automated image editing tools or procedural rendering, which supports scalable dataset creation.\n4.\tThe Spy–Player game formulation is an interesting and original idea that provides a novel self-supervised interaction mechanism.", "weaknesses": "1.\tThe paper identifies two valuable and timely challenges in multimodal learning: data scarcity and knowledge ceiling.\n2.\tIt presents a clear motivation by extending the concept of self-play from large language models (LLMs) to vision-language models (VLMs).\n3.\tThe framework generates data using automated image editing tools or procedural rendering, which supports scalable dataset creation.\n4.\tThe Spy–Player game formulation is an interesting and original idea that provides a novel self-supervised interaction mechanism.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bhdhLpwdck", "reviewer": "Reviewer_B6Um", "summary": "The paper proposes Vision-Zero, a zero-human-in-the-loop post-training framework for VLMs built upon a visual “Who is the Spy?” self-play game. This paper further proposes iterative self-play optimization for this self-play game task, which consists of self-play optimization in clue stage and RLVR for the decision stage. Experiment demonstrate consistent gains over baseline methods for Qwen2.5-VL-7B base model.", "questions": "Please see Weaknesses", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1.\tThis paper demonstrates that self-play visual game can be an effective task for VLM post-training, which is an interesting finding.\n\n2.\tThe paper evaluates on a reasonably wide set of benchmarks to demonstrate the effectiveness of Vision-Zero.", "weaknesses": "1.\tPrior work has already explored automatically crafted multi-image contrastive data for VLM post-training (e.g., MiCo). The paper should avoid claiming to be the first zero-human-in-the-loop paradigm and more carefully situate its contribution relative to [1].\n\n2.\tThe “Who is the Spy?” pipeline requires multiple forward passes per sample and a more complex training algorithm. To justify this complexity, the paper should include direct, apples-to-apples comparisons against simpler multi-image contrast baselines such as MiCo [1], using the same data sources (and ideally matched compute) to test whether the game mechanics yield additional gains beyond contrastive RLVR.\n\n3.\tPlease report training efficiency for Vision-Zero. The multi-turn interactions per sample could introduce substantial overhead; quantifying this is important for practical adoption.\n\n4.\tThe main text states 100 iterations while the appendix reports 150. Please reconcile this discrepancy and clarify which results correspond to which training steps.\n\n[1] MiCo: Multi-image Contrast for Reinforcement Visual Reasoning. NeurIPS 2025", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)