Vision-Zero: Scalable VLM Self-Improvement via Strategic Gamified
  Self-Play
Authors: name: Qinsi Wang; name: Bo Liu; name: Tianyi Zhou; name: Jing Shi; name: Yueqian Lin; name: Yiran Chen; name: Hai Helen Li; name: Kun Wan; name: Wentian Zhao
Published: 2025-09-29 21:55:55

Abstract: Although reinforcement learning (RL) can effectively enhance the reasoning
capabilities of vision-language models (VLMs), current methods remain heavily
dependent on labor-intensive datasets that require extensive manual
construction and verification, leading to extremely high training costs and
consequently constraining the practical deployment of VLMs. To address this
challenge, we propose Vision-Zero, a domain-agnostic framework enabling VLM
self-improvement through competitive visual games generated from arbitrary
image pairs. Specifically, Vision-Zero encompasses three main attributes: (1)
Strategic Self-Play Framework: Vision-Zero trains VLMs in "Who Is the
Spy"-style games, where the models engage in strategic reasoning and actions
across multiple roles. Through interactive gameplay, models autonomously
generate their training data without human annotation. (2) Gameplay from
Arbitrary Images: Unlike existing gamified frameworks, Vision-Zero can generate
games from arbitrary images, thereby enhancing the model's reasoning ability
across diverse domains and showing strong generalization to different tasks. We
demonstrate this versatility using three distinct types of image datasets:
CLEVR-based synthetic scenes, charts, and real-world images. (3) Sustainable
Performance Gain: We introduce Iterative Self-Play Policy Optimization
(Iterative-SPO), a novel training algorithm that alternates between Self-Play
and reinforcement learning with verifiable rewards (RLVR), mitigating the
performance plateau often seen in self-play-only training and achieving
sustained long-term improvements. Despite using label-free data, Vision-Zero
achieves state-of-the-art performance on reasoning, chart question answering,
and vision-centric understanding tasks, surpassing other annotation-based
methods. Models and code has been released at
https://github.com/wangqinsi1/Vision-Zero.