Play to Generalize: Learning to Reason Through Game Play
Authors: Yunfei Xie
Yinsong Ma
Shiyi Lan
Alan Yuille
Junfei Xiao
Chen Wei
Venue: ICLR 2026 Poster (2026)
Keywords: reinforcement learning; large language models
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Developing reasoning capabilities in multimodal large language models (MLLMs) remains challenging. Motivated by literature suggesting that gameplay promotes transferable reasoning skills, we propose a novel post-training method, Visual Game Learning (ViGaL), where MLLMs develop generalizable reasoning skills through playing arcade-like games. Specifically, we show that training a 7B-parameter MLLM via reinforcement learning (RL) on simple games like Snake significantly enhances the downstream performance on multimodal math benchmarks like MathVista, and on multi-discipline questions like MMMU, without seeing any worked solutions, equations, or diagrams during RL.
Remarkably, our model outperforms specialist models post-trained on benchmark-oriented multimodal reasoning data, while preserving the model’s performance on general visual benchmarks, a challenge where specialist models often fall short.
Our findings suggest that multimodal reasoning can emerge from gameplay, pointing to a promising strategy of designing surrogate tasks for RL post-training. The code is available at https://yunfeixie233.github.io/ViGaL.

Published as a conference paper at ICLR 2026
PLAY TO GENERALIZE: LEARNING TO REASON THROUGH GAME PLAY
Yunfei Xie', Yinsong Ma², Shiyi Lan', Alan Yuille?, Junfei Chen Rice University, 2Johns Hopkins University, 'NVIDIA
Chen Wei
ABSTRACT
Developing reasoning capabilities in multimodal large language models (MLLMs) remains challenging. Motivated by literature suggesting that gameplay promotes transferable reasoning skills, propose novel post-training method, Visual Game Learning (ViGaL), where MILMs develop generalizable reasoning skills through playing arcade-like games. Specifically, show that training a 7B- parameter MILM via reinforcement learning (RL) on simple games like Snake significantly enhances the downstream performance multimodal math bench- marks like Math Vista, and on multi-discipline questions like MMMU, without seeing any worked solutions, equations, diagrams during RL. Remarkably, our model outperforms specialist models post-trained on cenchmark-orientee multi- modal reasoning data, while preserving the model's performance on general vi- sual benchmarks, challenge where specialist models often fall short. Our find- ings suggest that multimodal reasoning can emerge from gameplay, pointing to a promising strategy of designing surrogate tasks for RL post-training. The code is available at https: https://yunfess: http://wwuff/////// wtt.:/ie2...ttt..iieeee githtgtthhhh.g. Ctattuub.o//iGaa..
Post-Training
Evaluation
Average Accuracy Increase en Mathverse, Mathvision and Mathvista
Downstream
game
+2.9%
Emargent Ability...
RL on math
+2.4%
Math Data Required d for RL
on game
Snake Game
Math Reasoning
Discipline
Multi-
RL on math
12K
Figure/ Overview ViGaL. Left: We propose novel post-training method where MILMs finetuned via RL play arcade-style games such Snake (Kamradt) 2025). We demonstrate that gameplay post-training enables MLLMs to achieve out-of-domain generalization, enhancing their performance downstream multimodal reasoning tasks requiring math, spatial and multi- discipline reasoning, without using math or multi-displine data during RL. Right: Our ViGaL (RL on game) achieves higher average accuracy increase than MM-Eureka (Meng al. (025)) (RL on math) across three multimodal math benchmarks. This is notable because MM-Eureka trains on large-scale, curated math datasets, while ViGaL only uses game data. Details are Tab
INTRODUCTION
Games, beyond their entertainment value, provide rich and diverse structured environments for de- veloping and studying general reasoning and problem-solving abilities. Humans from early child- hood acquire foundational cognitive skills through diverse game-like activities such as arranging ob- jects, navigating spaces, and manipulating tools. These experiences foster essential building blocks of abstract thinking, including pattern recognition, spatial reasoning, and causal inference (Brändle) et al. 2021 Bertram 2020). In cognitive science, games are used as experimental platforms to re- veal the inductive biases of the human mind (Allen et al. 2024 Alhasoun & Alneghiemish 2021), such as planning depth in the game Bour-in-a-Row (Van Opheusden et al. (023), or the cognitive basis of tool use through the game Virtual Tools (Allen al. 2020).

---

Published as a conference paper at ICLR 2026
Al agents, too, have benefited from games resembling aspects of human play. These environments encourage exploration, robustness to sparse rewards, and learning from multimodal inputs. For ex- ample, emergent tool use has been observed in agents trained via hide-and-seck (Baker al. 2019), and Atari gameplay has been incorporated into training generalist agents (Reed et al. 2022). By learning in these environments, Al systems develop robust and transferable reasoning capabilities.
Recent work has shown that post-training with Reinforcement Learning (RL) can unlock reasoning behaviors from their base models (DeepScek-Al 2025 OpenAT 2024). These RL-trained models can "think before they speak". generating internal chain-of-thought traces before outputting a final answer. More importantly, growing evidence suggests that RL often generalizes more robustly to out-of-distribution samples than supervised fine-tuning (SFT). For example, models trained with RI. on mathematics transfer their reasoning skills to physics (Meng et al. (025), and navigation agents adapt to novel environments beyond their training domains (Chu et al., 2025a). Motivated by these findings, we ask: since games already serve as a natural medium through which humans acquire reasoning strategies, can post-training multimodal LLMs on gameplay similarly enhances their ability to reason across diverse tasks?
The results are striking (Fig. TT. We show that post-training 7B-parameter multimodal LLM, Qwen2.5-VL VL-BB (Bai et al. 2023), to play simple arcade-style games like Snake (Kamradt 2025)
yields two surprising outcomes: (1) the model generalizes to previously unseen games (Sec. 2.3
and (2) it exhibits strong reasoning abilities on multimodal math benchmarks like Vista LLu jet al. 2024), and multi-domain QA like MMMU (Yue al., (024a). Despite never observing worked solutions, equations, or diagrams during RL post-training, the model achieves competitive results not only against large-scale industrial systems like GPT-4o (Hurst al. 2024), but also against specialist models post-trained on math datasets (Tabs. 2 and 3}. Furthermore, it improves on reasoning benchmarks without degrading general visual understanding, common limitation of domain-specialist training (Tab. D). Overall, gameplay emerges as an effective surrogate task for incentivizing reasoning multimodal L.MMs.
Why does it work? Our ablation studies suggest that reasoning skills incentivized by gameplay can be helpful to other multimodal reasoning tasks. For example, Snake, a game set on a 2D grid where the player maneuvers the "snake" to avoid collisions and collect apples, significantly improves per- contrast, Rotation, a puzzle requiring
formance on math problems involving 2D coordinates. recognition of 3D object rotation angles, more strongly boosts performance on geometry questions involving angles and lengths (Fig. B). Furthermore, jointly training on both games yields consis- tently stronger results downstream benchmarks than training on either game alone, suggesting the compositionality of the acquired skills (Tab. 2
All these results point to new post-training strategy: rather than relying solely on domain-specifer datasets, we can design scalable and controllable surrogate tasks for post-training, such as games, that unlock reasoning behaviors transferable to downstream applications. Synthetic game environ- ments offer structured, rule-based rewards and fine-grained controllability, while also scaling far more easily than aanontadd data. This promising paradigm of post-training with surrogate tasks reminisces self superviee pre- pre-training in vision and language (He et al., 2222) Doerssh et al.) 2015 Radford et al. 2018), where carefully designed pretext tasks produce broad generalization.
REINFORCEMENT LEARNING ON VISUAL GAMES
We introduce ViGaL, a novel post-training paradigm designed to enhance generalization capabilities.
2.1 GAME ENVIRONMENT
As show in Fig. 2 under our ViGal. paradigm, the model is trained in a game environment where it receives states from game environment, outputs next actions, and obtains rewards as feedback from the environment. Formally, each task, given an instruction 1, can be formulated as a partially observable Markov decision process (POMDP): A, R, Q). where S is the set of possible environment states, O is the set of observations available to the model, and A represents actions model can do in this game environment. T S x A + S is the state transition function, while R is a binary reward from the environment representing the correctness of action. Due to partial observability, the agent perceives only observations Π(s).
2

---

Published as a conference paper at ICLR 2026
Snake Game
Use reasoning instructions...
MLLM Response
Obstacle Avoidance
<shink-oo
Snake
Shortest Path
soza: aoven, need evalucte the posstale novse and their consequencesi L'Querent Fositions head pisltion A/uhiok? <beet snseseers Reane theet _andeers ewcret ansvers ansker>
Snake
Callision Check
Movement Prodiction
Apples
Action Space
Observation
Reflection
Action
Rotation Game
In-Context Example
Game Task
Use reasoning instructions...
MLLM Response
Angle Estimation
<thinto the craneformation From Tmage Trage Las beed solated 90 degress clocksian. The ayaastey uses hav shifted dequoen..., andinating 30-Gugasu splatisn.///ilaal Cazaves/90//anaeee
Axis Alignment
pharacter
Tria ngle Reasoning
Polar Analysis
180
Action Space
Figure 2: Post-training MLLMs to reason through RL with games. We propose post-training MLLMs via RL by playing visual games. We demonstrate this with two games: the classic arcade game Snake (Kamradt) 2025), and Rotation, a self-designed task to investigate spatial reasoning. In each game, the model receives multimodal inputs and follows reasoning instructions, e.g. path planning in Snake, angle estimation in Rotation. It reflects to choose an action, outputs its chain- of-thoughts and decision, e.g., best/worst move or predicted angle, and receives reward. Through gameplay, the model obtains reasoning abilities that transfer to downstream multimodal reasoning tasks such math and multi-discipline question answering.
Snake and Rotation Games. We design two complementary games, Snake and Rotation, to study the proposed paradigm (Fig. (). each focusing different MLLM capabilities. The Snake game,
inspired by prior work showing that competition can enhance reasoning in MLLMs (Du et al.) 2023). emphasizes strategic decision-making. We set up a dual-snake game based on SnakeBench (Kam- radt 2025), where each model independently controls one snake. The objective is to reach apples, score points, and outcompete the opponent. At time t, the environment state st includes the co- ordinates of both snakes (arh, v².)) for i E (1,2), the apple location (rh. vε), and the previous 10 x 10 board. Each snake then selects its next action
actions A All elements are placed A E {up, down, left, right.. A snake dies if it collides with itself, the other snake, or the board boundary; the survivor wins, in the case of simultaneous death, the higher score decides. Unlike SnakeBench which uses only text to represent states, we provide both images of the game board and textual descriptions as observations of I2(')) for richer input. The Rotation game, inspired by rotation-ange prediction as a pre-text task in self-supervised learning (Gidaris et al., 2018], evaluates visual perception and spatial reasoning. The model is presented with two views of the same 3D object: an initial view Tinit and a rotated view Tot. obtained by rotating the object 90 or 180 around the z-axis (pointing toward the viewer). The task is to identify which rotation angle transforms lini into Im To guide reasoning, we include an in-context example with a known rota- tion. As in the Snake game, observations combine images and text. Together, these two games allow systematic exploration of reasoning and perception, two fundamental aspects of MLLM abilities.
2.2 RULE- BASED REINFORCEMENT LEARNING
We apply rule-based RL to directly post-train MLLMs for visual games, without relying on super- vised learning as a warm up. The algorithm is described as follows:
Reward design. We use simple rule-based reward function to avoid reward hacking (Gao al. 2022) and help the model learn how to play the games effectively. This reward function has two components: accuracy reward and format reward. The total reward is computed as the sum of an accuracy reward and a format reward r = r Faceurccy + Tfomaa- The accuracy reward Faceuracy is the answer is correct, and 0 otherwise. Details of reward for each game are in Appendix Sec. A.3
3

---

Published as a conference paper at ICLR 2026
Model
Wins (/10) Model
Acc. (%)
Game
ViGal.
Qwen2.5-VI.-77
ViGaL vs.
ViGaL
71.9
Space Invaders
280.0
85.0
Qwen2.5-VL-7B
Qwen2.5-VLL-7B
47.4
Ms. Pacman
1370.0
670.0
V1.-225-V1--725
Owen2.5.VL.-722
52.1
Seaquest
80.0
60.0
Llama-4-Man M-averic
Jlama---Maverick
66.2
Alien
540.0
450.0
Gemini-2.5-Pro
Gemini-2.5 Pro
51.0
Frogger
7.0
5.0
Claude-3.--Somnce
Claudo-3.
65.6
Breakout
0.0
9.0
GPT-4o
GPT-4o
61.5
Pong
-26.0
-26.0
04-mini
04-mini
70.8
Cumulative Reward 2251.0
1253.0
(a) Snake game.
(b) Rotation game.
(c) Atari game.
Table 1: Game Performance. (a) ViGaL gets high win rates (6-9 wins out of 10 matches) on
Snake playing against advanced proprietary models. (b) ViGaL shows best performance on Rotation. (c) ViGaL trained on Snake and Rotation shows zero-shot generalization to unseen Atari games, achieving nearly doubled cumulative reward compared to its base model (Qwen2.5-VL-7BB.
Advantage estimation and policy update. We employ REINFORCE Leave-One- Onave-One-Outt (RLOO) algorithm (Kool et al.; [2019] AAhmadian et al., 2024) in RL training phase. Following Group Policy Gradient (Chu et al. (025b), we omit KL divergence regularization. Without KL constraints limiting policy changes, the model explores the solution space more freely, potentially discovering better reasoning strategies. This enables more flexible adaptation during RI. training.
2.3 IMPLEMENTATION AND EVALUATION ON GAMES
Implementation details. We employ Qwen2.5 20enn255-VL-77 VL- IB-Itruutt (Bai et al. 2023) as our base model. We follow DeepSeek-RI (DeepSeck-Al AI 2025). using combination of fule-based format rewards and accuracy rewards, with RLOO (Koll et al. 2019 Ahmadian et al. (024) as the core RL algorithm. We implement our training within multimodal input RI. framework based on Open- RLHF (Hu et al. 2024). For hyperparameters, we adopt the default settings from MM-Eureka (Meng et al. 2025), including global batch size of 1 128, rollout batch size of 128, a rollout temperature of 1.0, and a learning rate of 1e-6, Training uses 6 A100-80G GPUs.
Game training data. We build game environments to collect training data for our experiments. For Snake, we leverage SnakeBench (Kamradt, (025) as our data engine. For Rotation, we utilize Hunyuan3D (Team 2025), which generates 3D meshes from images or text instructions. We render each mesh into 2D images from different orientations, creating image pairs with associated rotation angles as ground truth labels for RL training. Our comprehensive data generation pipeline enables producing training samples at any desired scale with fully customized settings. For experiments, synthesize 36K samples per game, sufficient for convergence. Details are in Appendix Sec. A.I
Competing with leading models on Snake and Rotation. To evaluate the game capabilities of ViGaL models, we initialize environments in diverse states unseen during training. For Snake (Tab. Tap, we randomly initialize games 10 times with two models competing directly, measuring win counts. For Rotation (Tab. lb), we measure rotation angle prediction accuracy on compre- hensive validation sets with 3D object meshes unseen during training. Our 7B-parameter model consistently outperforms proprietary models in both games. Results confirm that RL effectively unlocks small 7B models' ability to excel in visual games requiring environmental understanding, reasoning, planning, and interactive decision-making.
Out-of-distribution generalization to Atari games. We then test ViGaL on Atari-GPT (Way- towich et al. 2024), a benchmark for evaluating MLLMs as decision-making agents Atari video games such as in Fig. 3 The benchmark consists of seven difffernnt Atari games, with detailed set- tings in Appendix Sec. B. We follow most settings and prompts from Atari-GPT, with small modification to ensure format correctness for all models. Following Atari-GPT (Waytowich et al.) 2024), we report cumulative reward over IK steps as the evaluation metric, where higher rewards indicate better performance. As shown in Tab. Ic ViGaL shows significant cumulative reward improvement on Atari games despite being trained only on Snake and Rotation games. This is par- ticularly notable because Atari games differ substantially from training games in both visual appearance and gameplay strategies. These results suggest that our based RL training approach enables strong generalization to previously unseen game environments.
4

---

Published as a conference paper at ICLR 2026
Geometry Avg. GeoMath Geo3K
Model
Avg.
Avg.
Math Vista Math Verse
Math Vision
Proprictary Model
GPT-4o (Hurst et al. 2024
47.5
47.3
61.4
50.2
30.4
46.8
50.2
43.5
Gemini-2.0-Flash
Team 2023
55.4
56.4
73.4
54.6
41.3
54.4
55.3
53.5
Multimodal Reasoning Model Post- Truined Qwen2.5-VL-7B (Bai
2023
Base Model (Qwwen.5-VVL77))
46.3
47.7
68.0
49.0
26.0
44.8
44.0
45.6
RI R1-Onevision-7H Yong et al.
2025
40.9
46.8
64.1
46.4
29.9
35.0
45.4
24.5
RI-VE R1-VL-7B (Chen et al. 2025b
40.9
42.7
40.0
24.7
42.0
MM JM-Eureka-Qwent Meng et al. 2025
39.3
50.1
73.0
50.3
26.9
28.4
53.1
3.8
Deason-RFT-2zrr--8B (Tan et al. 2025
46.5
38.1
60.7
35.3
18.3
54.9
55.0
54.8
VLAA-Thinker-78
C hen et al. 2025a
51.3
48.7
68.0
51.7
26.4
53.9
51.1
56.6
OpenVLThinker--77 Deng et
2025
47.8
70.2
47.9
25.3
56.4
49.2
63.5
ViGaL Snake
51.6
49.4
70.7
51.1
26.5
55.0
49.9
60.0
ViGaL Rotation
52.8
49.3
71.2
50.4
26.3
57.9
51.7
64.1
ViGal. Snake Rotation
53,9
50.6
52.4
27.5
57.1
51.0
63.3
10.3
1003
1:04
102
1103
1005
1:03
20.4
Table 2: Results multimodal mathematical benchmarks. We compare other multimodal reasoning models. Results post-trained on the same subject as the evaluation are de-emphasized, while our ViGaL models only use games for post-training, Bold numbers are the best in each Avg. column. We include standard deviations of three independent runs for ViGaL Snake + Rotation.
CLEVR
Multi-Discipline
Model
Avg, Avg.
CLEVR-M S-CLEVR Avg. MMMU,,, AMMU-Promeell
Proprietary Model
GPT-4o Hurst al. 2024
55.9
51.2
68.1
34.3
60.5
69.1
$1.9
Gemini-2.00 F.0- (leam 2023
46.3
64.9
27.6
71.9
Multimodal Reasoning Model Post- Trained on Qwen2.5 V-L-7BBB (Bai al.
2023
Base Model: Qwen2.5-VL.7 VL-7B
50.3
54.9
74.6
35.2
45.7
54.3
37.0
RI 11-OOevision-7B (Yang al. 2025
53.7
65.1
75.5
54.7
42.3
51.9
32.6
RI-VL-7B Chen 2025b
53.9
68.0
87.4
48.6
39.7
50.0
29.4
MM-Fureka-QQwen-BB
(Meng et al.
2025
62.8
79.3
98.4
60.1
46.4
55.8
36.9
Reason-RFT-Zero-775 lan al. 2025
76.2
99.4
40.9
51.2
30.6
VLAAA-Thiner:-7B Chen al. 2025a
61.7
83.4
94.7
72.1
40.1
48.2
31.9
OpenVLLThinker-7B Deng et al.
2025
82.4
93.8
71.0
38.5
54.8
22.1
ViGaL Snake
64.4
82.6
92.6
72.6
46.2
55.8
36.6
ViGal Rotation
63.3
80.7
93.0
68.3
45.9
54.1
37.7
ViGaL Snake Rotation
81.7
91.9
71.4
47.7
58.0
37.4
Table 3: Results on multimodal spatial and multi-discipline reasoning benchmarks. CLEVR-M denotes CLEVR-Math (Lindström & Abraham, 22022), and S-CLEVR stands for Super-CLEVR (Li et al. 2023). Results post-trained on the same subject as the evaluation are de-emphasized, while ViGaL is exclusively post-trained using games. Bold numbers are the best in each Avg. column.
3 VISUAL REASONING GENERALIZATION
2024c). system- atically divide existing benchmarks into two broad categories: (i) reasoning-orion benchmarks requiring multi-step or mathematical reasoning, and (ii) general-purpos perception benchmarks assessing visual understanding and perception abilities.
Evaluation collection. Following prior studies (Tong et al. 2024a Liet et al.
For reasoning-oriented evaluation, test four key areas: Math (Math Vista (Lu et al. 2024
Math Verse (Zhang et al.] [2024), Math Vision (Wang et al. 2024b)), Geometry (GeoMath (Gao et al. 2023; Shi et al. (024), Geometry3K (Lu al. (0211). CLEVR+ (CLEVR-Math (iindström & Abraham 2022), Super-CLEVE (Li et al. (023)), and Stutti-Disciple (MMMU Yuu et al. 2024a), MMMU-Pro (Yue al. 2024b)). For general perception, evaluate across three cate-
5

---

Published as a conference paper at ICLR 2026
Accursoi Diforenoss (N)
Vole Saate Batadia
Vili Siate Banelin
Accuracy Differunces (S0)
+8.75
VIA Rutalen Seed
Vos Rodallen Seading
+6.25
+6.35
+6.16
+3.30
Appotee
14,22
Prospertsy tV
Expranation
4.55
Araa
+674
+0.00
Cover deatt
Expression
Lengri
Property
Vakume
Angle
40.18
-0.66
Angle
Lengit
Volume
Appiled
Conrdinate
0..44
3.75
(a) Snake vs. Baseline
(b) Rotation vs. Baseline
Figure 3: Per-category gains on MathVerss are not uniform. The eight math categories follow Math Verse (Zhag al., 2024). (a) Snake yields the largest gains on Coordinates and Expressions, consistent with its 2D grid structure. (b) Rotation boosts Angle and Length questions but reduces Exprexsion aceuracy, suggesting its training primarily incentivizes orientation recognition.
Snake
Expression
Body posttons 7L (L GJ
determine inequality that desctbbu region number ldentity
Possille Moves
Move -IGHT Moves
parding through paints
move RIGHT (1, distance apple bocomes [5-1] I1 R| 72+ 7-
Strtagy and Objective:
towards... Lart Not sussibee wall prosen.
Calourate
implls -I0 therrforee insquality that descrbbs the
poin (0, find vintercept -10 4/0
froc10
ly frac220(55)
Decieter Move sigit (1, to me rben central board positon and access more strotegic options, <ttinko....
region log 10 The final value (bonacily fi. 101)
Rotation
Angle
shown rotated that its back facin foroard, which would typcaaly ind 280 degroe rosation The orienatioon change suggest the panda has Rlpped amonn the vetiial acs, while alnn fwiing the opzoatte deectior. The symmetry and the way the legs apocor susports this a 180 degree retation rather than 50 de-degr rotation. </tini ca rowers 1804//mseer>
ahini front,
analps gleen DAD propertics find
Siice speetic type quadmiaters where the fateral night ammmaric. the sum of the angle around point (whsch the interection of the diogoratc) 350" By snalping the symmetry and the progertie the quadnbeeenl we can decuse that the sin of angln DAO and Dca 110 thersfoee, the cornect snoweris Whhooe]fi)
DCO. symmetry
ABC
infer
Figure 4: Reasoning trace of different games and math questions. Top: Algebraic functions and coordinate-level interpretations that emerge from playing the Snake game help solving Expression questions. Bottom: Spatial reasoning skills incentivized by playing the Rotation game appear when solving Angle-related problems.
gories: General (MuirBench (Wang et al. 2024a), CRPE (azzemzadeh et al. (0014)) CeittnCCentric (MMVP (Tong et al. 2024b), RealWorld (X.AI) 2024). MMStar (Chen et al. 20244)) MME (Fu
al. 2023), BLINK (Fu et al. 2024)), and OCR & Chart (AI2D (Kembhavi et al.) 2016). SEED.
Bench-2-Pics (Li et al. 20224)) Doc (Mathew et al. 202T). OCRBench (Liu et al. 2023
More detailed descriptions of each benchmark are provided in Appendix B.6
3.1 MAIN RESULTS
Zero-shot generalization from gameplay to multimodal reasoning. Our approach consistently shows remarkable generalization capabilities on mathematical and other reasoning tasks, despite having no direct exposure to in-domain training data during RL post-training. As shown in Tab. 2 our method notably outperforms models specifically RL-trained on mathematical tasks. For in- stance, ViGaL Snake + Rotation achieves 0.5% higher accuracy than Eureka-Qwen-78 (Meng (et al. 2025) 28.7% on Geometry, even though MM-Eureka- -ueeeer-e-----77 was explicitly trained on high- quality mathematical and geometry datasets.
This strong generalization extends beyond mathematics. Tab. shows that ViGal. Snake Rotation outperforms R1-One Vision-7B (Yang et al.) (025) by 5.4% average across MMMU series bench- marks, which test culti-disciplinary reasoning. This is particularly notable since RI R1-One RI-OOnVisionn-D was trained carefully curated comprehensive dataset spanning multiple subjects.
These empirical results suggest that gameplay-basee post-training develops fundamental reasoning capabilities that transfer more effectively than direct RL training diverse task- ses--specfiic datasets. Moreover, the gameplay environment appears to encourage general problem-solving strategies that consistently generalize well to out-of-domain tasks.

---

Published as a conference paper at ICLR 2026
Model
Avg.
Math
Math Verse
Vision
Base Model: Qwen2.5-VL-7B
47.7
68.0
49.0
26.0
MMN-EurekaQQwen-BB
50.1
73.0
50.3
26.9
ViGaL Snake + Rotation
50.6
71.9
52.4
27.5
ViGaL Snake + Rotation + Math Data
51.8
72.3
54.5
27.7
Table Gameplay complements math data. Adding math data MMK12 on top of ViGal. yields further gains in math performance. With access the same amount of math data, ViGaL outper- forms MM- MMM-Eurea (Meng et al. (2025) on average of the three math benchmarks.
Blending multiple games enhances generalization. As shown in Tab. 2 post-training on Snake achieves best performance on the CLEVR+ benchmark, while training on Rotation yields stronger results on geometry reasoning. Furthermore, training on both Snake and Rotation enables learning complementary skills, improving the overall benchmark average to 63.1% These findings indi- cate that combining game environments drives meaningful performance gains, demonstrating Vi- sual Gaming Learning's potential promising paradigm for enhancing generalizable reasoning without large-scale domain-specift data. Expanding the types of games during training consistently scales performance across different visual reasoning tasks.
Different games benefit distinct math subfields. To study which types of problems in the math benchmarks benefit from game play, we analyze accuracy differences across Math Verse (Zhang 2024) subcategories between ViGal models trained with Snake Rotation, shown in Fig. 3 We find that training on the Snake game significantly improves performance on the sub- categories like Expressions and Coordinates, while training on Rotation notably enhances perfor- mance on questions about angles and lengths. To understand why different games help with differ- ent types of math, we compare the reasoning processes required for playing games versus solving math problems. As shown in Fig. 4 solving Expressions questions involves algebraic functions and coordinate- level interpretations graphical representations, which closely align with the spatial rea- soning process in Snake. Similarly, solving angle-related questions is consistent with requirement of playing Rotation game to reason about rotational angles of 3D objects. These results suggest that playing different games develops fundamental skills like spatial modeling and algebraic calculation that transfer to visual math questions. To validate whether this pattern generalizes beyond Snake and Rotation, we extend this analysis to four additional games (Maze, Tetris, Sudoku, Sokoban) and identify systematic transfer patterns based on cognitive alignment between game mechanics and mathematical skills. A complete multi-game study using K-Means clustering analysis on Math- Verse subject-level categories is presented in Appendix Sec. [B.4] The experiment on quantitatively analyzing the correlation between math and game is in Sec. BB.9 in the Appendix. Furthermore, joint training on both games leads to improvements across all reasoning categories (see Appendix Sec. B.33) We also include qualitative analyses on improvements in math reasoning after RL in Appendix Sec. C
al.
Gameplay complements math data. We explore the complementary benefits of adding math data to the gameplay training pipeline, for which we implement a two-stage training process. Stage equals to ViGaL setup. training the model on Snake and Rotation games. In stage 2, we further finetune the stage 1 model MMK12 (Meng et al.) 2025), multimodal mathematical reasoning dataset containing approximately 12k examples. Stage 2 training uses the identical data and settings as MM--Errek-QQwen- cal data in stage 2 yields a continuous improvement of 1.2% on average across three mathematical benchmarks. This demonstrates the complementary relationship between our visual game learning approach and mathematical data post-training. Moreover, ViGaL with math data significantly out- performs
(Meng al. 2025). As shown in Tab. the integration of mathemati-
Baureka-Qwen-Pe Qw by 1.7% on mathematical benchmarks on average, using the same math data. These results suggest that visual game learning can serve as an effective surrogate task together with domain-specifi data to improve performance on target tasks.
Preserving general visual capabilities while reasoning enhancement. To examine whether gen- eralization on reasoning tasks leads to degradation in general visual capabilities. we evaluate ViGaL Snake + Rotation a broader set of MLLM benchmarks. As shown in Tab. g in Appendix, com- pared to Qwen2.5- VL-7B prior to RL tuning, our model maintains comparable general visual per- formance while achieving stronger math reasoning results. In contrast, other models that improve
7

---

Published as a conference paper at ICLR 2026
(a) Text prompt design.
(b) Reward design.
(c) Difficulty control.
peorpe
Marh
CLEVE.
Geo
Math CLEVR-
Gea.
Math CLEVR. Gee.
hose moded
42.3
54.3
masdel
4011
40.2
54.0
44.5
bas medill
417
549
66.5
We rencning leat.
595
48.0
104
590
41.2
9914
50.2
wit dificitty centrol
48,5
$1.4
51.5
09:1
K2.6
550
wind
62.1
09.11
876
1941
37.6
150
rasdom Lihel
4641
47.5
554
(d) Data scalability.
(c) Input modality.
(1) SFT vs. RL.
bainigg sumplex
Arg.
Math
CLEVR+
Gea.
ixpel soodalt
Math
CLEVB:
Gos.
posmiing
Muth CLEVE
Goa
hase recill
20.1
47.0
51.0
14.
Tase moode
477
58.9
66.
hos mall
27.
56.9
415
16K
18.9
31.
503
Ind
18.5
80.1
513
STT
473
38.0
71.5
35.
62.3
15/4
87.6
42.3
09.1
67.6
00.0
RI.
02.
100
93.
550
Table 5: Ablation study. We ablate different aspects of ViGaL with Snake and evaluate on down-
stream benchmarks. The similar evaluation with Rotation is in Sec. Sec. Each benchmark consists of several subtasks (Tab. [ and Tab. B}. and we report their averages. The base model is Qwen2.5-
VL-7B, whose results are in gray. The default settings in Tab. and Tab. 3 are highlighted in blue
math performance through RL post-training often exhibit substantial drops in general visual capa- bilities. These results demonstrate that our gameplay-bases approach enables math generalization without compromising other visual abilities.
3.2 ABLATION STUDY
We ablate key design choices in the Snake environment, evaluate each variant on downstream bench- marks, and report the results in Tab. [ The corresponding ablation for the Rotation environment is provided in Appendix Sec. B.2
Reasoning instructions in the text prompt help. We use reasoning instructions, such as "finding the nearest apple by calculating Manhattan distances", in the text prompts to guide the model thinking chains. The complete text prompts are in Appendix Sec. A.2 In Tab. [a we demonstrate that reasoning instructions brings a significant improvement of 2.8%, from 59.5% to 62.3%, for Snake in average accuracy over the three out-of-domain benchmarks.
Reward design of pre-text game matters for downstream tasks. We show that reward design of RL for games plays a crucial role for the downstream tasks. As shown in Tab. 5b we first ask the model to predict only the best next move, defined the action that moves toward the closest apple while avoiding death. In our improved reward design, we task the model with simultancously predicting both the best and worst next moves, where the worst move leads directly to losing the game. More importantly, it leads to improvements across all downstream tasks, bringing an average increase of 1.8%. These results suggest that proper reward design in pre-text game can improve not only gameplay capabilities but also generalization to downstream tasks.
Furthermore, inspired by several prior works that improve model performance without labeled re- wards (Zhao et al. (025) or with random labels (Shao et al. 2025), we also provide random reward ablation, where we still ask the model to predict both best and worst moves but use random moves as the labels. We report the results in the last row in Tab. 5b In our gameplay setting, RL with random labels reports 49.4% on averagne and does no provide significant gains over the base model, different from the conclusions in prior works (Shao et al., 2025). Potential explanations lie in the difference data domains and base models, where other works applied random labels to text-only mathematical data while our work applies random labels visual game data.
Controlling game difficulty for better reasoning. Gameplay for RL post-training offers unique opportunities to easily control task difficulty. We present an ablation study on difficulty control im- portance. We define difficulty based snake length, where longer snakes represent higher difficulty. For controlled difficulty, we collect training data using states where snake length falls within a mod- erate range of 1-5. Details are in Sec. A. As shown in Tab. 5cc, difficulty control achieves 61.4% overall accuracy compared to 60.6% without control. This suggests our game engine can easily generate appropriately difficult data, helping prevent model sub-optimization during RL training.
RL games shows data scalability. Thanks to using game engine, scale with high flexibility. To show data scalability on RL of visual games, we conduct experiments using 16k and 32k snake game samples, respectively. As in Tab. 5d scaling data from 16k to
generate data at any
8

---

Published as a conference paper at ICLR 2026
32k brings a performance improvement of 1.3% on average across all domains. This suggests the potential of the proposed ViGaL paradigm to improve downstream performance by easily scaling training data, which contrasts with the data scaling challenges of domain-specifte human annotated data, requiring extensive manual effort.
Both text and vision contribute to better visual reasoning. To isolate the contributions of text and vision modalities, we conduct an ablation study with text-only setting. In this setup, rep- resent game states-including snake positions, apple locations, and boundary constraints-usingg only textual descriptions during RL training. The model trained with text-only inputs on the Snake game demonstrates substantial improvements across all multimodal benchmarks, with average per- formance increasing from 49.1% % to 59.6%. Incorporating visual inputs yields an additional 1.8% performance gain. These results demonstrate that multimodal RL enhances visual reasoning capa- bilities, with complementary contributions from both text and vision modalities.
RL generalizes better than SFT from games to math. To evaluate the out-of-domain gener- alization of ViGaL, we compare it with supervised fine-tuning (SFT) using identical visual game data. Tab. 5T shows that SFT with Snake game data degrades the base model's performance on both mathematical reasoning and geometry tasks by a notable 9.7% and 12.7%, respectively. While SFT produces modest improvements on CLEVR+, these gains are substantially smaller than those achieved by RL. Overall, RL improves performance by 12.3%, whereas SFT decreases performance by 1.9% This stark contrast demonstrates that RL better preserves and extends the model's 's reason- ing capabilities to new domains.
RELATED WORK
Reinforcement Learning (RL) increasingly enhances rea- soning in Large Language Models (LLMs) beyond Supervised Fine-Tuning (SFT). Text-only mod- els like DeepSeek-R1 (DeepSeek-AT] 2025) demonstrate RL's efficacy, especially with rule-based rewards, for complex reasoning. This paradigm is now being extended to Multimodal LLMs (MLLMs). Recent MLLM research explores RL for improved visual reasoning. drawing from LLM successes. Various works (Peng et al.) 2025 Huang al.) 2025 Chen et al., (0256) in- vestigate multi-stage training, trace supervision, or rule-based RI for specific visual subdomains like geometry and counting. Others focus on different RI. algorithms like Process Reward Models (PRMs) (Luo et al.) [025 Xiang et al.) 2024). often moving beyond SFT-based Chain-of-Thouge Thought generation (Dong et al.) 2024 Thawakar et al. 2025). Many efforts favor simpler rule-based re- wards (Huang al. 2025 Zhou et al. 2025) over complex reward models prone to hacking (Eisen- stein et al. 2023). Unlike approaches training on costly, Nooainn-speciio reasoning datasets, our Vi- GaL paradigm extends rule-based RL to simple, synthetic visual games, demonstrating these serve as scalable, cost-effective pre-text tasks.
Reinforcement Learning in MLLMs.
Generalization in MLLMs. Achieving robust generalization to novel tasks, distributions, and domains is central to MLLM development. RL shows promise for better out-of-distribution (OOD) generalization compared to SFT (Chen et al. 2025b: Meng et al. (025). and developing multi-step reasoning like CoT (Wei et al. 2022) is itself generalization. This is often pursued through training on large, diverse cestrrctioo-folloviin
datasets (Li al. 2024b Liu et al. 2024 Chen et al. 2024b)
or explicitly training general reasoning capabilities (Yang al. 2025 Huang al. 2025). While these methods advance OOD generalization, they typically operate within the same broad domain of complex visual reasoning as training data. Our ViGaL paradigm investigates stronger out-of- domain generalization, showing fundamental skills learned from simple synthetic games transfer zero-shot to enhance performance on entirely different, complex domains like visual mathematics and multi-discipline questions, without domain-specifie data exposure.
Transfer Learning and Curriculum Learning. The idea that training on simpler tasks can im- prove performance on more complex ones is often framed as transfer learning. In this view, a model first learns from a source task domain, then reuses that knowledge to improve different target task (He et al., 2020). Before MLLMs, this principle was studied widely in computer vision through self-supervised pretext tasks, where models are trained without labels to solve auxiliary tasks such

---

Published as a conference paper at ICLR 2026
predicting image rotations (Gidaris et al. 2018), relative positions of patches (Doersch et al. 2013), or colorization, and the learned representations are then transferred to downstream tasks like detection segmentation. These methods show that solving carefully designed pretext tasks can produce robust visual features that support related vision problems. In reinforcement learning, cur- riculum learning (Bengio al. 2009) applies related idea by ordering tasks data from easy to hard so that agents gradually master more difficult behaviors. RL curricula and autocurricula have been used to grow complex behaviors from simple environments, for example in multi-agent games and robotics (Baker et al., 2019, Narvekar et al. 2020), typically within a single task family where later tasks share the same goal structure as earlier ones. Our ViGaL paradigm is different in two key respects. First, unlike self-supervised pretext approaches that transfer static feature representa- tions within the visual domain, we transfer learned reasoning policies, such as look-ahead planning. spatial verification, and constraint satisfaction, across tasks and domains. Second, unlike RI. cur- riculum work that mainly increases difficulty within the same environment class, ViGaL transfers policies learned in visual control games to abstract symbolic reasoning tasks in coordinate geometry and multi musti-dis question answering, without using domain -peaif- supervision in those target domains.
CONCLUSION
We introduced Visual Game Learning (ViGaL), a novel post-training paradigm where MLLMs learn transferable reasoning by playing simple arcade-style games. Our core finding is that RL games like Snake and Rotation, without any in-domain math data, significantly boosts MILM performance mathematical and multi-discipline benchmarks, surpassing specialized models and even large proprietary systems. Ablations confirm the importance of game design, reward structure, and that RL outperforms SFT, while distinct games unlock different skills. We posit that games instill fun- damental reasoning skills, suggesting a new avenue for using scalable, controllable synthetic games as powerful pre-text tasks to unlock generalizable reasoning. This work opens doors to exploring a broader range of game-based learning for generalizable AI.
ACKNOWLEDGMENT We thank Haoqin Tu and Yuxuan Cheng for their valuable feedback on this manuscript.
ETHICS STATEMENT
This work does not involve human subjects, private or sensitive data, or personally identifiable information. All training and evaluation data are either synthetically generated (Appendix [I.] or come from publicly available benchmarks (Appendix [B66). Our research adheres to the ICLR Code of Ethics, including principles of scientific integrity, fairness, and responsible stewardship. We believe the contributions of this work advance multimodal reasoning without raising foresccable ethical or societal risks.
Use of Large Language Models: An LLM used to assist with grammar refinement of text.
Further details are provided in Appendix D
REPRODUCIBILITY STATEMENT
We have taken steps ensure the reproducibility of our results. The training environments, rein- forcement learning setup, and hyperparameters are described in Section [ Details of synthetic data generation are provided in Appendix A. training prompts in Appendix A.2 and reward functions in Appendix A.3 Evaluation protocols for Atari-GPT visual reasoning benchmarks are specified in Appendix B. and Appendix B.6
REFERENCES
Pranjal Aggarwal and Sean Welleck. L1: Controlling how long reasoning model thinks with reinforcement learning. arXiv preprint arXiv:2003. 04697, 2025.
Arash Ahmadian, Chris Cremer, Matthias Gallé, Marzich Fadace, Julia Kreutzer, Olivier Pietquin, Ahmet Üstün, and Sara Hooker. Back to basics: Revisiting reinforce optimization for learn- ing from human feedback in lms. In ACI, 2024.
10

---

Published as a conference paper at ICLR 2026
Fahad Alhasoun and Sarah Alneghiemish. Probabilistic programming bots in intuitive physics game play. In AAAI, 2021.
Kelsey Allen, Franziska Brändle, Matthew Botvinick, Judith E. Fan, Samuel J. Gershman, Alison Gopnik, et al. Using games to understand the mind. Nature Human Behaviour, 2024.
Kelsey Allen, Kevin A Smith, and Joshua B Tenenbaum. Rapid tria--and-error learning with simulation supports flexible tool use and physical reasoning. PNAS, 2020.
Sanghwan Bae. Jiwoo Hong, Min Young Lee, Hanbyul Kim, Jeong Yeon Nam, and Donghyun Kwak. Online difficulty filtering for reasoning-oriented reinforcement learning. arXiv:2504.03335 2025.
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-VLL A versatile language model for understanding, local- ization, text reading, and beyond. arXiv:2308. 12966, 2023.
Bowen Baker, Ingmar Kanitscheider, Todor Markov, Yi Wu, Glenn Powell, Bob McGrew, and Igor Mordatch. Emergent tool use from multi-agent autocurricula. In ICLR, 2019.
Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Proceedings of the 26th annual international conference on machine learning, pp. 41-48, 2009.
Lara Bertram. Digital learning games for mathematics and computer science education: The need for preregistered rets, standardized methodology, and advanced technology. Frontiers in Psychology, 2020.
Franziska Brändle, Kelsey Allen, Josh Tenenbaum, and Eric Schulz. Using games to understand intelligence. In CogSci, 2021.
Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, al. SFT RL? an early investigation into training R1-Like reasoning large vision-language models. arXiv:2504.11460 2025a.
Liang Chen, Lei Li, Haozhe Zhao, Yifan Song, and Vinci. RI-V: Reinforcing super generalization ability in rision-language models with less than $3. /R1-V 2025b.
Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, et al. Are on the right way for evaluating large vision-language models? 2rXiv:2403.203300 2024a.
Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, et al. Intern Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In CVPR, 2024b.
Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, et al. SFT memorizes, RL generalizes: A comparative study of foundation model post-training. arXiv:2501.1716 2025a.
Xiangxiang Chu, Hailang Huang, Xiao Zhang, Fei Wei, and Yong Wang. GPG: A simple and strong reinforcement learning baseline for model reasoning. arXiv:2504.02544 2025b.
DeepSeck-AI. DeepSeek-RI: Incentivizing reasoning capability in Ilms via reinforcement learning. 11%4922501.12948, 2025.
Yihe Deng, Hritik Bansal, Fan Yin, Nanyun Peng, Wei Wang, and Kai-Wei Chang. Open W.Thhnner: An early exploration to complex vision-language reasoning via iterative self-imprrvemment arXiv:2503.17352. 2025.
Carl Doersch, Abhinav Gupta, and Alexei Efros. Unsupervised visual representation learning by context prediction. In ICCV, 2015.
Yuhao Dong, Zuyan Liu, Hai-Long Sun, Jingkang Yang. Winston Hu, Yongming Rao, and Ziwei Liu. Insight- V: Exploring long-chain visual reasoning with multimodal large language models. arXiv:2411.14493 2024.
11

---

Published as a conference paper at ICLR 2026
Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. Improving factuality and reasoning in language models through multi-agent debate. In ICML, 2023.
Jacob Eisenstein, Chirag Nagpal, Alekh Agarwal, Ahmad Beirami, Alex D' Amour, DJ Dvijotham, et al. Helping herding? reward model ensembles mitigate but do not eliminate reward hacking. arXiv:2312.09244, 2023.
Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang. Xu Lin, Zhenyu Qiu, Wei Lin, Jinrui Yang, Xiawu Zheng, et al. Mme: A comprehensive evaluation benchmark for multi- modal large language models. arXiv:2306. 13394, 2023.
Xingyu Fu, Yushi Hu, Bangzheng Li. Yu Feng. Haoyu Wang, Xudong Lin, Dan Roth, Noah A Smith, Wei-Chiu Ma. and Ranjay Krishna. Blink: Multimodal large language models can see but not perceive. In European Conference on Computer Vision, pp. 148-166. Springer, 2024.
Jiahui Gao, Renjie Pi, Jipeng Zhang, Jiacheng Ye, Wanjun Zhong, Yufei Wang, et al. G-LLaVA: Solving geometric problem with multi-modal large language model. arXiv:23 1370, 2023.
Leo Gao, John Schulman, and Jacob Hilton. Scaling laws for reward model overoptimization. vrXiv:2210.10700 2022.
Spyros Gidaris, Praveer Singh, and Nikos Komodakis. Unsupervised representation learning by predicting image rotations. wrXiv:1803.07222 2018.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In CVPR, 2020.
Jian Hu, Xibin Wu, Zilin Zhu, Xianyu, Weixun Wang, Dehao Zhang, and Yu Cao. OpenRLHE: An easy-to-use, scalable and high peeh-performance RLHF framework. arXiv:2405.11143, 2024.
Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, et al. Vision-R1: In- centivizing reasoning capability in multimodal large language models. arXi::2003.06749, 2025.
Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, al.
GPT-4o system card. arXiv:2410.21276, 2024.
Greg Kamradt. Snake Bench: Competitive snake game simulation with LLMs. https://gith ubb cccmm//kaanrrattt/aaeeeeeeet 2025.
Sahar Kazemzadeh, Vicente Ordonez, Mark Matten, and Tamara Berg. Referitgame: Referring to objects in photographs of natural scenes. In EMNLP, 2014.
Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Sco, Hannanch Hajishirzi, and Ali Farhadi. diagram is worth a dozen images. In ECCV, 2016.
Wouter Kool, Herke van Hoof, and Max Welling. Buy 4 REINFORCE samples, get baseline for free! In ICLR Workshop on Deep Reinforcement Learning Meets Structured Prediction, 2019.
Bohao Li, Yuying Ge. Yi Chen, Yixiao Ge, Ruimao Zhang, and Ying Shan. Seed- 1oodd-bench-2-ppe Benchmarking multimodal large language models with text-rich visual comprehension. arXiv preprint arXiv:2404. 16790, 2024a.
Feng Li. Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, et al. LLaVA-NeXT-Interre Tackling multi-image, video, and 3d large multimodal models. arXiv:2407.0788 07895, 2024b.
Jian Li, Weiheng Lu, Hao Fei, Meng Luo, Ming Dai, Min Xia, Yizhang Jin, Zhenye Gan, Ding Qi, Chaoyou Fu, et al. A survey on benchmarks of multimodal large language models. arXiv preprint (ttl:2408.0863222 2024c.
Zhuowan Li, Xingrui Wang, Elias Stengel-Eskin, Adam Kortylewski, Wufei Ma, Benjamin Van Durme, and Alan L. Yuille. Super-CLEVR: A virtual benchmark to diagnose domain ro- bustness visual reasoning. In CVPR, 2023.
Adam Dahlgren Lindström and Savitha Sam Abraham. Clevr-Math: A dataset for compositional language, visual and mathematical reasoning. In IJCLR, 2022.
12

---

Published as a conference paper at ICLR 2026
Haotian Liu, Chumyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang. Sheng Shen, and Yong Jae Lee. LLaVA-NeXT: Improved reasoning, OCR, and world knowledge. hattps://llaaaaaee
Chub. htw.oo//oooogg//11 -1----11-------ee---
2024.
Yuliang Liu, Zhang Li, Mingxin Huang, Biao Yang, Wenwen Yu, Chunyuan Li, Xucheng Yin, Cheng lin Liu, Lianwen Jin, and Xiang Bai. Ocrbench: On the hidden mystery of ocr in large multimodal models. arXiv:2305.00 07895,077955000 2023..
Pan Lu, Ran Gong, Shibiao Jiang, Liang Qiu, Siyuan Huang, Xiaodan Liang, and Song- Song-Chun Zhu. Inter-GPS: Interpretable geometry problem solving with formal language and symbolic reasoning. arXiv:2105.0410 0.05004165, 2021.
Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannanch Hajishirzi, et al. Math Vista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv:2310.02255, 2024.
Ruilin Luo, Zhuofan Zheng, Yifan Wang, Yiyao Yu, Xinzhe Ni, Zicheng Lin, et al. URSA: Understanding and verifying chain-of-thought reasoning in multimodal mathematics. arXiv:2501.04686, 2025.
Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. Doevqa: A dataset for vqa on document images. In WACV, 2021.
Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, et al. MM-Eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learning. WXXiv2503.07365 2025.
Sanmit Narvekar, Bei Peng. Matteo Leonetti, Jivko Sinapov, Matthew E Taylor, and Peter Stone. Curriculum learning for reinforcement learning domains: A framework and survey. Journal of Machine Learning Research, 21(181):1-50, 2020.
OpenAl. Introducing OpenAl ol. https::/ com/ 2024.
OpenAl. Gpt-5 system card. apenai..om/ineerrg--.-.ysemm... 2025.
YingZhe Peng, Gongrui Zhang, Xin Geng, and Xu Yang. LMM-R1. https:///ithub.co com /TideDra/Imm- x1. 2025.
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language under- standing by generative pre-training. OpenAI, 2018.
Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gomez Colmenarejo, Alexander Novikov, Gabriel Barth-Maron, Mai Gimenez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, et al. A generalist agent. arXiv:2205.06173 2022.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.
2017.
Rulin Shao, Shuyue Stella Li, Rui Xin, Scott Geng, Yiping Wang, Sewoong Oh, Simon Shaolei Du, Nathan Lambert, Sewon Min, Ranjay Krishna, Yulia Tsvetkov, Hannanch Hajishirzi, Pang Wei Koh, and Luke Zettlemoyer. Spurious rewards: Rethinking training signals in rlvr.
zlvr nwwionniiooooo g Trrininnnnn-i.. Signals- in RLVR Ttt.aa/aa..aaaaaa00803333333333.
2025.
Notion Blog.
Wenhao Shi, Zhiqiang Hu, Yi Bin, Junhua Liu, Yang Yang, See-Kiong Ng, Lidong Bing, and Roy Ka-Wei Lee. Math-LLaVA. Bootstrapping mathematical reasoning for multimodal large language models. arXiv:2406. / 17294, 2024.
Huajic Tan, Yuheng Ji, Xiaoshuai Hao, Minglan Lin, Pengwei Wang, Zhongyuan Wang, and Shang- hang Zhang. Reason-RFT: Reinforcement fine-tuning for visual reasoning. 2rX5v 2025.
Gemini Team. Gemini: A family of highly capable multimodal models. arXiv:2312. / 1805, 2023.
13

---

Published as a conference paper at ICLR 2026
Tencent Hunyuan3D Team. Hunyuan3D 2.0: Scaling diffusion models for high-resolution textured 3d assets generation. arXiv:2501.12000 2025.
Omkar Thawakar, Dinura Dissanayake, Ketan More, Ritesh Thawkar, Ahmed Heakl, Noor Ahsan, et al. LLaMAV-ol: Rethinking step-by-step visual reasoning in LLMs. arXiv:2501.66188 2025.
Jingqi Tong, Jixin Tang. Hangcheng Li, Yurong Mou, Ming Zhang, Jun Zhao, Yanbo Wen, Fan Song, Jiahao Zhan, Yuyang Lu, Chaoran Tao, Zhiyuan Guo, Jizhou Yu, Tianhao Cheng, Zhiheng Xi, Changhao Jiang, Zhangyue Yin, Yining Zheng, Weifeng Ge, Guanhua Chen, Tao Gui, Xipeng Qiu. Qi Zhang. and Xuanjing Huang. Game-rl: Synthesizing multimodal verifiable game data to boost vims' general reasoning, 2025. URL https::/aartt.. www.xxiivrrrroooooooo org/ abs abs///0b//55555/ 2205 13886
Peter Tong. Ellis Brown, Penghao Wu, Sanghyun Woo, Adithya Jairam Vedagiri IYER, Sai Charitha Akula, Shusheng Yang, Jihan Yang, Manoj Middepogu, Ziteng Wang, et al. Cambrian-l: A fully open, vision-centric exploration of multimodal Ilms. Advances in Neural Information Processing Systems, 37:87310 17781--77366, 2024a.
Shengbang Tong, Zhuang Liu, Yuexiang Zhu, Xingjian Chen, Ruoyu Zhang, Bo Li, et al. Eyes wide shut? exploring the visual shortcomings of multimodal llms. arXiv preprint arXiv:2401.06209 2024b.
Bas Van Opheusden, Ionatan Kuperwajs. Gianni Galbiati, Zahy Bnaya, Yunqi Li, and Wei Ji Ma. Expertise increases planning depth in human gameplay. Nature, 2023.
Fei Wang, Xingyu Fu, James Y Huang, Zekun Li, Qin Liu, Xiaogeng Liu, Mingyu Derek Ma, Nan Xu, Wenxuan Zhou, Kai Zhang, et al. Muirbench: A comprchensive benchmark for robust multi-image understanding. arXiv preprint arXiv:2406.09411 2024a.
Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. Measuring multimodal mathematical reasoning with Math-Vision dataset. In NeurlPS, 2024b.
Nicholas R. Waytowich, Devin White, M.D. Sunbeam, and Vinicius G. Goecks. Atari-GPT: Investi- gating the capabilities of multimodal large language models as low-level policies for atari games. arXiv:2408. 15950, 2024.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, al. Chain-of-thought prompting elicits reasoning in large language models. In NeurIPS, 2022.
X.AI. Grok-1.5 vision preview.
2024.
Kun Xiang, Zhili Liu, Zihao Jiang, Yunshuang Nie, Runhui Huang, Haoxiang Fan, et al. AtomThink: A slow thinking framework for multimodal mathematical reasoning. arXiv:2411.1199: 2024.
Tian Xie, Zitian Gao. Qingnan Ren, Haoming Luo, Yuqian Hong. Bryan Dai, Joey Zhou, Kai Qiu, Zhirong Wu, and Chong Luo. Logic-rl: Unleashing Ilm reasoning with rule-based reinforcement learning. arXiv preprint arXiv:2502. 14768, 2025.
Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang. et al. RI El-OOneVision Advancing generalized multimodal reasoning through cross- modal formalization. arXiv:2503.10615. 2025.
Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, al. Mmmu: A massive multi-discipline multi- modal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 9556-9567, 2024a.
Xiang Yue, Tianyu Zheng, Yuansheng Ni, Yubo Wang, Kai Zhang, Shengbang Tong, Yuxuan Sun, Botao Yu, Ge Zhang, Huan Sun, et al. Mmmu-pro: more robust multi-discipline multimodal understanding benchmark. arXiv preprint arXiv:2409.028/3 2024b.
14

---

Published as a conference paper at ICLR 2026
Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Shiji Song, and Gao Huang. Does reinforcement learning really incentivize reasoning capacity in LLMs beyond the base model? arXiv:2504.13837, 2025.
Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, el. MathVerse Does your multi-modal LLM truly see the diagrams in visual math problems? arXiv:2403. 14624, 2024.
Xuandong Zhao, Zhewei Kang, Aosong Feng. Sergey Levine, and Dawn Song. Learning to reason without external rewards. arXiv preprint urXiv:2505.19590 2025.
Hengguang Zhou, Xirui Li, Ruochen Wang, Minhao Cheng, Tianyi Zhou, and Cho-Jui Hsieh. R R1-Zero's 's "aha moment" in visual reasoning on a 2b non-sft model. 20X1::5503.5513,,220 2025.
15

---

Published as a conference paper at ICLR 2026
APPENDIX
Content
A. Data
A.I. Training Data Synthesis
A.2. Training Prompt in Visual Game Learning
18
Detail of Format Reward
20
Evaluation
Evaluation Detail of Atari Game
Ablation On Rotation Game
B.3. Synergistic Effects of Multi-Game Training
B.4 Auxiliary Task Selection: A Multi-Game Study
B.5. Reasoning Ability Boundary via Passük Evaluation.
25
B.6. Detail of Evaluation Benchmarks
B.
Evaluation on General Visual Capabilities
B.8. Inference Length Analysis
B.9. Reasoning Correlation Analysis Between Game and Math
28
10. Generalization Spatial Reasoning (VSI-Bench)
29
C. Case Study
30
D. Use Large Language Models
16

---

Published as a conference paper at ICLR 2026
A DATA
A. TRAINING DATA SYNTHESIS
Thanks to using the synthetic game data engine, data with precisely controlled difficulty levels. This completely eliminates the need for extensive data filtering strategies used in previous rule-based RL work training on domain-specific data like math (Meng et al.) 2025 Bae et al. 2025). where difficulty is hard to define and filtering can signif- icantly reduce dataset size.
flexibly generate large-scale training
For the Snake game, the environment consists of a 10 x 10 grid game board with two snakes of 1-grid initial length, where at each time step 4, each snake receives action to move, resulting in new game state $1+1. We define difficulty based on snake length-longee snakes create more complex game situations and more constrained movement options, closely aligning with how hu- mans perceive difficulty when playing Snake. To generate meaningful moves that accomplish the objective of collecting more apples while remaining alive, we implement policy network based on Proximal Policy Optimization (PPO) (Schulman et al. (017). The observation space is repre- sented as a 10 x 10 grid with distinct values indicating empty cells (0), apples (1), the agent's 's own body (2), and other agents' bodies (3), stacked across 4 time steps to incorporate temporal infor- mation, resulting in an input tensor E R 10x 10x4, The policy network architecture consists of two convolutional layers with 3 3 kernels (C1 16 and C2 32 output channels), followed by fully connected layers that output action logits for the four possible movements, transformed into probability distribution (als) using softmax. To prevent suicidal moves, incorporate action priors by masking logits for dangerous actions. The model employs the standard PPO objective with entropy regularization coefficient 3 0.01, value function coefficient
0.5, and clipping parameter s = 0.2. Agents receive rewards of r +1 for collecting apples and penalties of r = -1 for dying. enabling them learn complex behaviors such as obstacle avoidance, apple pursuit, and multi-step trajectory planning.
For the Rotation game, training data comprises synthetically generated visual puzzles focused on 3D spatial reasoning, utilizing 540 unique 3D object meshes (408 from Hunyuan3D 2.0 Team (025) and 132 from Hunyuan3D 2.5). Our custom pipeline produces pairs of images (Jmit Inx) representing objects before and after defined rotations. Difficulty in Rotation is determined by the rotation angle between two images, where smaller angle differences present greater percep- tual challenges. Each pair is generated through precise sequence: establishing diverse initial viewpoints through compound transformations (base orientation plus additional z-axis rotation from {0% 30" ..... 330"} prevent trivial pattern learning), then applying target rotations of either 90 or 180 exclusively around the z-axis. All objects are rendered at 512 x 512 pixel resolution using consistent perspective camera under standardized lighting conditions, resulting in approximately 32k unique pairs
Based on empirical results, we established optimal difficulty parameters for RL training across both games, which we ablate in Tab. 5c] This controlled progression of difficulty, made possible by our synthetic data generation approach, enables more effective learning trajectories compared to traditional data collection methods.
17

---

Published as a conference paper at ICLR 2026
A.2
TRAINING PROMPT VISUAL GAME LEARNING
Prompt for Snake Game
Your role is to guide snake within Snake game featuring multiple apples.
This game played board size by 10. board uses standard Cartesian coordinate system, where (0,0) represents the bottom-left position and (9,9) is the top-rightmost coordinate.
Apples at: {apple.position)
Direction of Your Last Action: (last_action)
Rules:
apple, you grow and gain point. 2) If your head moves to position where coordinates (x, y) are outside the board boundaries (meaning
1) If you move
x < 0, x > y. < O, or y > 9), or into a space occupied by another snake's body, or into a space occupied by your own body, you die. That's the worst move.
3) The goal is prioritize smake die, then efficiently collecting apples. First avoid the worst move, then for each apple, find the nearest apple by calculating Manhattan distances. But only choose best next move to get closer the nearest apple if you can confirm best next move will not run outside the range of the listed coordinates, run into the position of another snake, or yourself. Otherwise it will be the worst move.
Your snake with the ID {snake_id in [snakc_color] } has its head now positioned at {snake position]) and its body extends to (body- positionn You should avoid your next move into your own snake's 's position.
Enemy snakes in (enemy cenemy_color) positions: (enemy-positions
Decreasing your x coordinate is to the LEFT, increasing your x coordinate to the RIGHT. Decreasing your coordinate is DOWN, increasing your y coordinate is UP.
Read out another snake's position and apple position. Try to predict another snake's next move and avoid colliding with it.
Best answer is of next move that is the closest to the apple and not lead to your death. Worst answer is all of next moves 1. makes your head's coordinates y) outside the board boundaries, meaning < 0, > 9, y < or > 9. moves into a position occupied by another snake's 's body. 3. moves into position occupied by body of yourself.
Check all the next return None for worst answer, e.g..
list out all the worst in <worst.answer> tag. no
The hest answer and the worst answer are mutually exclusive and different.
You need first to give your reasoning process then choose of best next and worst from ['UP', 'DOWN'. 'LEFT', 'RIGHT".
<best </best_answer> and <worstaanswer> </worst_answe>> tags, respectively, i.c., "<think>> reasoning pro- cess here </think>><bssttassaaar one best move here </bestt.nswer> answer>> all worst moves here </worst_answer>"
The reasoning process and answer are enclosed within <think> </think>
18

---

Published as a conference paper at ICLR 2026
Prompt for Rotation Game
I'm showing you 4 images. Images 1-2 are example pair, and Images 3-4 are the test pair. In each pair, the first image shows the initial orientation, and the second shows the object after rotation.
### EXAMPLE OF ROTATION ###
Example: Image shows the initial view and Image shows the object after 180 degree rotation.
### YOUR TASK #*#
Now, considering the transformation from Image 3 (initial) to Image (rotated) Determine the angle of rotation from Image to Image the plane Analyze the rotation carefully using the example pair (Images 1-2) reference.
1. Coordinate System Transformation:
Draw an x-y coordinate system on both original and rotated images with origin at center Identify distinct feature point and note its coordinates in both images
Apply rotation matrix equations verify the transformation
Example: star icon at coordinates (3,1) in the original image appears at (-1,3) in the rotated image. Testing with the 90º clockwise rotation matrix [cos(90"), sin(90)); -sin(90"), cos(90")] confirms the transformation from (3,1) to (-1,3), verifying 90" clockwise rotation.
2. Angular Displacement Measurement:
Mark the image center as the origin in both images
Draw straight line from center to distinctive feature in both images Measure the angle between these two lines using counterclockwise as positive
Example: A line from center to red dot makes 30 angle with horizontal in the original image. In the rotated image, this line makes 210 angle with horizontal. The difference (1809) indicates clockwise 180' rotation.
3. Symmetry Axis Tracking:
Identify major symmetry axes in the original image
Locate the same symmetry axes in the rotated image
Calculate the angular displacement between original and rotated
Example: A rectangular logo has vertical and horizontal symmetry axes. After rotation, the vertical axis now points right and horizontal points down. This 90" shift of both axes confirms clockwise 90" rotation.
4. Triangle Configuration Analysis:
Select three non-collinear distinct points forming triangle in both images Compare the orientation of this triangle in both images using vector cross products
Determine rotation angle from the triangle's orientation change
Example: Three points form a right triangle with vertices clockwise arranged. After rotation, the same triangle has its vertices arranged in counterclockwise order while maintaining the same shape. This inversion indicates clockwise 180 rotation.
Polar Coordinate Comparison:
Convert key points to polar coordinates (1,6) relative to image center Compare values of the same features in original and rotated images
Calculate consistent angular difference across multiple points
Example: A feature at polar angle 45 in the original image appears at 135" in the rotated image. Another feature shifts from 10" to 100%, Both show a shift in polar angle, confirming a clockwise 90" rotation.
Choose the rotation angle from this list: ('counter clockwise 90', '180']
The reasoning process and answer are enclosed within <think> </think> and <answer'> </answer>> tags, respectively, i.e.. "<think> reasoning process here </think> <answer>> answer here </answer>"
19

---

Published as a conference paper at ICLR 2026
While the model takes images input to understand the current state of the game, design a structural text prompt framework to also provide game guidance. Our game prompts con- sist of two parts: (1) game settings and (2) reasoning instructions. (1) To help the model un- derstand the game environment, we describe the background, current game state, rules, goals, action space, etc. in text besides the input image. (2) In the reasoning instruction part, we provide specific thinking guidance since games can be approached with various thinking chains. To encourage broader thinking, we implement different types of reasoning instructions to guide decision-maie makinng process. Specifically, we used GPT-4o (Hurst et al., (024) to synthesize mathematical thinking instructions for Snake, such as "finding the nearest apple by cal calculation ing Manhattan distances", and spatial thinking instructions for Rotation, for example, "identif" fy major symmetry axes in the original image". With rea- soning instructions for games, the obtained reasoning abilities generalize to downstream evaluation on visual math questions (Tab. 5a). Bold text indicates reasoning instructions synthesized by GPT- 4o (Hurst et al. 2024).
A.3 DETAIL OF FORMAT REWARD
The format reward Pformat validates whether the response follows the task-specific format: if the response follows the required format
[0.1,
Pformat 0, otherwise
For Snake game, the desired format is:
<think>...< http:/hhii...ees.sssseeeee
As suggested by the format, encourage the model predict both positive toward the apple and negative move that leads to failure. This reward encourages contrastive makii,, which not only improves the model's gameplay abilities but also boosts down- stream reasoning performance on visual math benchmarks. We ablate the effect in Tab.5b] For the rotation task, the required format is simply <think>... wtt.s////hikknssssss.i..
that moves
answer>>.
20

---

Published as a conference paper at ICLR 2026
B
EVALUATION
B.1 EVALUATION DETAIL ATARI GAME
Alien
Frogger
Navigate through maze like environments shooting allen enemies and collecting Items.
Goal: Guide
a busy road and river to reach homes safely.
Example response:
Example response: Reasoning: The alien is right above, continue firing and take down
E3
Reasoning: There's
coming from the left. avoid
Moving up will help
260
Action:
Pong
Ms. Pacman
Goal: your hit the ball past your opponent's paddle to score points.
Goal: Novigate through maze, eating all dots while avolding ghosts or eating them when powered up.
Example response: Reasoning:
Example response:
is moving towards paddle, we must move the paddle down to it. Action: 3
Reasoning:
now directly above
the ghost. Moving down
should her it and gain points.
220
Action: 4
Seaquest
Space Invaders
Goal: Control
Goal: Shoot waves of descending alien inwaders while avoiding their attacks.
submarine to rescue divers while fighting creatures and managing oxygen.
# 17
Example response:
Example response:
Reasoning: The invaders
Reasoning: The
the the screen, Firing is the best aption.
the top of the screen. Firing is the best aption.
seruin
Action:
Action:
Breakout
Frame: 5000 30
Goal: paddle bounce bell to break bricks at the top screen.
Example response:
Reasoning: ball
moving right. need to move right intercept I'm nearing right side of the screen. Action:
Figure 5: Goal and example response from model of Atari games used for evaluation. We implement kinds of Atari games from Atari-GPT /Waytowich et al. 2024).
21

---

Published as a conference paper at ICLR 2026
To evaluate out-of-distribution generalization, test ViGaL Atari-GPT (Waytowich et al, 2.24). a benchmark for evaluating MLLMs as decision-making agents in Atari video games, as shown in Fig. 5 The benchmark consists of seven different Atari games: Alien, Frogger, Pong, Ms. Paeman, Seaquest, Space Invaders and Breakout. These games present diverse visual environments which is different from Snake game and Rotation game, and require different strategic approaches finish the goal, making them an ideal test bed for ViGaL evaluating out-of oul-of-distribution generalization capabilities.
For evaluation, we input game frames as pixel observations to our model, following the established protocol in Atari-GPPP Specifically, each game frame is resized from 210 x 160 x to 512 512 3, then provided to our model along with game-specific action information. We maintain a context buffer containing the two previous frames and responses together with the current frame to enable temporal reasoning. Following Atari-GPT, we implement frame skipping of 8 frames, which extends the standard 4- 4-frame skipping in ALE to reduce computational intensity while preserving gameplay continuity.
We evaluate our method through four independent rollouts 1,000 timesteps each and report the
average cumulative reward, with results presented in Tab. Tc
B.2 ABLATION ON ROTATION GAME
Table 6: Ablation study. Similar to the evaluation in Tab. we analyze how different aspects of our post-training strategy within the Rotation game affect downstream generalization benchmarks. The base model is Qwen2.5-VL-77
with results shown in gray. The default settings from Tab. and Tab. 3 are highlighted in blue We observe the same improvement trends for each strategy as reported in Tab.
(a) Prompt design. Avg.
(b) SFT vs. RL.
prompt
Math
CLEVR+
Geo.
post-training
Avg.
Math
CLEVR+
Geo.
base model
49.1
47.7
$4.9
44.8
model
47.7
$4.9
Reasoning Instruction
61.4
75.4
Reasoning Instruction
49.3
57.9
80.7
(c) Difficulty control.
difticulty control
Avg.
Math
CLEVR+
Geo.
base model
49.1
47.7
$4.9
44,8
w/o difficulty control
61.0
48.0
80.2
54.8
w/ difficulty control
62.6
49.3
80.7
57.9
As shown in Tab. 6. conduct similar ablation study Tab. but replace the Snake game environment with the Rotation game. Our results demonstrate the same consistent improvement trends on downstream generalization benchmarks for each strategy employed.
Specifically, we control the task difficulty by varying the rotation angles between two images, In the uncontrolled difficulty setting, the rotation angle between images can be clockwise 90", counter- clockwise 90%, or 180%. However, we found that explicitly requiring the model to distinguish be- tween clockwise and counter-clockwise rotations leads training difficulties. Therefore, we remove it and only retain option of clockwise 90º and 180 rotations.
Unlike the Snake game, we cannot conduct the ablations shown Tab, 5e because the Rotation game is inherently visiontdependent and requires visual input. Similarly, we cannot perform the ablations in Tab. (56] because the Rotation game provides only binary answer options, making impossible to meaningfully designate both "best" and "worst" answers simultaneously.
22

---

Published as a conference paper at ICLR 2026
+6.25
Accuracy Differences (%)
+5.72
+3.73
+3.12
+2.82
+1.86
+1.10
+0.78
+0.63
Coordinate
Length
Angle
Applied
Property
Analytic
Area
Volume
Expression
Figure 6: Accuracy differences between ViGaL-Snakk-RRttatin and base model without RL training across mathematical subfields Mathverse. The synergistic effects jointly training on two games observed suggest that complementary games can enhance overall mathematical reasoning capabilities.
B.3 SYNERGISTIC EFFECTS OF MULTI-GAME TRAINING
As discussed in Sec. 3.1 our analysis reveals that each game develops distinct reasoning abilities in the model. To investigate potential combined benefits, we conducted experiments where models were trained simultaneously on both the Snake and Rotation games. Fig. shows that joint training effectively combines the strengths of each individual game, improving performance across the math- ematical areas where each game shows particular effectiveness, resulting in greater overall gains on Mathverse. These results suggest that strategically combining games with complementary strengths offers simple yet effective approach to enhance model generalization abilities.
B.4 GENERALIZATION BEYOND SNAKE AND ROTATION: A MULTI-GAME STUDY
key question is whether the observed benefits are specific Snake and Rotation, whether they generalize across diverse game mechanics. To investigate this, we extend our study to four additional games: Sokoban, Maze, Tetris, and Sudoku [Tong et (025). Sokoban requires pushing boxes to target positions while navigating spatial constraints; Maze involves pathfinding through obstacles reach a goal; Tetris requires rotating and positioning falling shapes to complete rows; Sudoku demands filling grids while satisfying row, column, and box constraints. These games introduce cognitive challenges beyond Snake and Rotation, including strategic planning and logical deduction. This extended study allows to examine whether the transfer of reasoning skills from gameplay to mathematical tasks is general phenomenon or an artifact of our initial game selection.
Consistent improvements across diverse games. Tab. 7 shows that all games yield consistent gains across mathematical reasoning benchmarks, with improvements ranging from 1.5% to 1.7% average. This demonstrates that developing transferable reasoning skills from visual gameplay is robust across diverse game mechanics and not specific to Snake and Rotation.
Systematic transfer patterns through subject-level analysis. To understand how different game mechanics transfer to specific mathematical sub-skills, we analyze performance on Math Verse using its official subject split, which assigns each problem to one of three subjects: Plane Geometry (angles and lengths), Functions (coordinate graphs and related diagrams), and Solid Geometry (volume and surface area). We use this subject-level split instead of the finer subfields because many of those subfields depend overlapping skills. For example, the Coordinate. Property, and Expression subfields all require interpreting coordinate systems. The subject-level grouping therefore offers a more stable and disentangled proxy for distinct mathematical reasoning abilities, for instance by placing all questions that require interpreting coordinate systems under Functions.
Tab. g reports performance gains for all six games separately for each subject. Based these performance profiles, we grouped the six games into three clusters using K-Means clustering to identify systematic transfer patterns:
23

---

Published as a conference paper at ICLR 2026
Table 7: Comparison of different auxiliary games on mathematical reasoning benchmarks. All models use Qwen2.5-VL-77 as the base. All games yield consistent improvements, demonstrating the generalizability of game-based transfer learning.
Model
Math Avg.
Verse
Math Vista
Vision
Qwen2.5-VL-7B
47.7
49.0
68.0
26.0
+ Maze
49.3
50.9
70.3
26.7
+ Rotation
49.3
50.4
71.2
26.3
+ Snake
49.4
51.1
70.7
26.5
+ Sokoban
49.3
50.5
70.1
27.2
+ Sudoku
49.3
51.2
69.0
27.6
+ Tetris
49.2
50.8
69.4
27.4
Table 8: Performance gains Math subjects for each auxiliary game, showing accuracy im- provement relative to the Qwen2.5-VI. VL--B baseline. We use the official subject split from Math- Verse: Plane Geometry (angles and lengths), Functions (coordinate graphs and related diagrams), and Solid Geometry (volume and surface area). This subject-level grouping provides a more stable proxy for distinct mathematical reasoning abilities compared to finer subfields that depend on over- lapping skills.
Model
Plane Geometry
Functions
Solid Geometry
Baseline (Qwen2.5-VL-7B)
52.43
49.81
33.45
+Snake
+1.81
+3.77
+0.84
+Rotation
+2.32
+1.13
+0.50
+Maze
+1.96
+1.76
+1,84
+Tetris
+1.73
+2.01
+1.34
+Sokoban
+1.49
+1.89
+0.84
+Sudoku
+2.28
+2.27
+1.34
Cluster 1 (Snake, Sokoban): Strongest transfer to Functions (+2.83% avg). Snake and Sokoban require tracking multiple objects moving coordinate space: the snake's grow- ing body segments or boxes being pushed to target positions. Players must reason about how coordinate relationships between objects evolve as actions unfold, mirroring the core skill in Functions problems, understanding coordinate transformations (e.g., translations, scaling) and how values change under sequential operations.
Strongest transfer to Plane Geometry (+2.32%) This puzzle quires recognizing 3D object rotation angles, directly engaging angle reasoming and spatial relationships fundamental plane geometry.
Chuster 2 (Rotation):
Cluster 3 (Maze, Sudoku, Tetris): Balanced improvements across all three subjects. These games engage distinct reasoning modes: Maze requires navigating to find optimal paths; Sudoku demands symbolic constraint satisfaction without spatial movement; Tetris involves shape transformations and pattern completion. This diversity of non-overlapping cognitive demands supports reasoning across mathematical domains more broadly rather than specializing to single subject.
These findings reveal systematic relationships between game mechanics and mathematical reasoning categories. Beyond confirming the generalizability of our approach, the results provide practical guidance for auxiliary task selection: to improve a specific downstream skill, one can select existing games whose cognitive requirements align with the target skill. Our framework offers method for
24

---

Published as a conference paper at ICLR 2026
systematically choosing and combining off-the-shelf games based on observed cognitive transfer patterns.
B.5
REASONING ABILITY BOUNDARY VIA PASS@k EVALUATION
MathVista
0.85
Ours
MMM-Eureka-Qwen-7B-.ee
Qwen2.5-VL-78-Inst-
0.80
Coverage (pass@k)
0.75
16
Number of Sample K
Figure 7: @ass@k performance curves Math Vista comparing base models with their zero-RL counterparts trained on mathematical data and game data, respectively.
We explore the reasoning ability boundary of models trained with different RL approaches by eval- uating the passêč metric. This metric measures the probability that least one of k: independent model samples solves given problem, indicating the true scope boundary of model's reason- ing capability essentially what problems the model can potentially solve given enough sampling attempts.
We evaluate the pass@k performance three models: the Base Model without RL training, MM- Brecka-Qwen-7B-Insse
and our ViGaL. As shown in Fig. 7 our ViGaL consistently demonstrates increasing p scores on Mathverse as & increases. This finding suggests that our approach can effectively solve complex problems when allowed multiple reasoning attempts, uncovering capabil- ities not apparent in single-sample evaluations.
Moreover, compared the other RL-trained model, achieves a steeper improvement in pass@k as k: increases. This indicates that ViGaL possesses a broader reasoning boundary and stronger reasoning abilities, enabling it solve wider range of problems when given sufficient opportunities to explore different solution paths.
Beuurka--wee-7BBB Qwen-7B-Instruct, model
Finally, our results demonstrate that k increases, base models without RL training eventually outperform RL-trained models. This aligns with the findings in (Yue et al. 2025) that highlight fundamental limitation of reinforcement learning with verifiable rewards (RLVR); while RL training significantly improves performance at small & values (e.g.. pass@1), base models possess wider coverage of solvable problems. This suggests trade-off where RL optimization focuses on solving high-probabilits problems at the expense of broader solution coverage. Future work should explore RLVR algorithms that can improve @ performance across all values of effectively extending the reasoning boundary beyond that of the base model.
B.6 DETAIL OF EVALUATION BENCHMARKS
To obtain a clearer picture of the various facets MLLM performance, we follow prior stud- ies (Tong et 2024a Li et al. 2024c) and systematically and carefully divide existing benchmarks
25

---

Published as a conference paper at ICLR 2026
Math
Geometry
CLEVR+
Multi- Discipline
Breakzut
Mathvista
GeoMaahh
CLEVI-Math
MMMU & MMMUU-Pr
Reaconng
the
Question Whigh imslnnttooue at
Questoon: What angle panullddggua
Queetion: SLBt al blo0s, leno many bitdtt ane
Comprehensive Disciplines
poritl in aleady swrb-
Action 0
ADCO a degruss?
bnown
Bepinerring (NV
(3%%) gk 76
Davign
Ms. Pacman
Mathvsion
Geometry3K
Super-CLEVR
Rastram &
Roaconig
disance glosis Action
nelen keegs
siraag 3o
Quedion: ne
(AT (176
Cha
angie
dbjects and left?
And cmst tilee
(2X5)
Malkins
Hamambe
(00 12
(a) Atari games.
(b) Out-of -f--mann tasks.
Figure 8: Samples from our generalization reasoning benchmarks. We evaluate the proposed Vi- GaL with two types of generalization: (a) out-of-distribution generalization. where models trained on our visual games are tested on unseen Atari games (Waytowich et al. (024); and (b) out-of-
domain generalization, where models trained only on game tasks are evaluated diverse multi- modal reasoning tasks including mathematical reasoning, geometric mooblem-solving. 3D under- standing on CLEVR+ and multi multi-discipline reasoning on MMMU series.
into two broad groups: (i) crasoning-orientee benchmarks, which require multi-step or mathematical reasoning to solve the problems, and (ii) general-purpose purpose perception benchmarks, which primarily assess broad visual understanding and perception abilities.
For ceasoning-orientte benchmarks, we comprehensively evaluate the visual reasoning generaliza- tion capabilities of RL through gaming advanced visual reasoning skills, including math-focused tasks like Math and Geometry, and other comprehensive reasoning benchmarks beyond math, like CLEVR+ and Multi-Discipling Fig. 8b illustrates specific examples from each benchmark.
diverse collection of tasks that specifically demand
Math evaluates multimodal math reasoning with widely-used datasets: Math Vista (test- mini) (Lu et al. 2024). MathVerse (testmini) (Zhang et al.) 2024). and Math Vision (test) (Wang et al., 2024b). Math Vista offers diverse problems spanning VQA, logic, alge- bra, and geometry: Math Verse emphasizes algebraic and geometric image comprehension; Math Vision tests abstract visual reasoning.
Geometry evaluates structural interpretation skills across mathematical diagrams, medical images, charts, and architectural layouts. It uses datasets GeoMath (Geo170K (Gao al., (023), Math360K (Shi et al., (024)) and Geometry3K (Lu et al. (021). featuring both choice and non-choice questions. Following Reason-RFi (Tan et al. 2025). we test with 820 GeoMath and 800 Geometry3K samples.
CLEVR+ evaluates the integration of mathematical and spatial reasoning skills through challenging arithmetic problems in complex 3D block-based scenes, including sub-tasks on CLEVR-Math (Lindström & Abraham 2022) and Super-CLEV (Li et al., 2023). Fol-
lowing Reason-RFT (Tan et al. 2025). we use 1K test samples from each of CLEVR-Math and Super-CLEVR.
Multi-Discipline evaluates college-level expert knowledge across six disciplines: Art &
Design, Business, Science, Health & Medicine, Humanities & Social Science, and Tech & Engineering. We follow the evaluation setting of MMMU (Yue et al.] 2024a) val set (900 questions) and MMMU-Pro (Yue et al. 2024b) overall score (average of standard 10-option and vision-only settings).
For general-purpont perception benchmarks, we systematically evaluate comprehensive visual ca- pabilities. Following previous work, these benchmarks are categorized into three distinct types: General, Vision-Centric, and OCR & Chart.
General benchmarks assess fundamental visual understanding capabilities. We evaluate MuirBench (Wang et al., (024a) for multi- understanding and CRPE (Kazemzadeh et al. 2014) for relation understanding.
26

---

Published as a conference paper at ICLR 2026
General
Visdon Centric
OCK & Chart
Model
Aog.
Bench
Muir- CRPE Avg. MMVP WorldQA
Real-
MMSur BLINK.. MME, Arg.
A12D
Bench-2+
SEED- DecVQA OCR-
val
Bench
Proprietiry Medel
GPT-44 (Burst
74.//2.3
680
75.4
64.7
68,0
(82.6
84.6
72.0
General Muliiedal Langinge
Qwen2.5-V1-77 atal 203
72.4/68.
59.6
65.8
68.5
56.4
1698
[83.3 83.9
70.4
95.7
Muhtmacal Recscning Model Wost-Trained on Qwen2 V-L-777
RI Oneviiion-11 76 eoneet 2025
86.8
46.3
$7.3
56.5
61.3
58.0
57.8
48.7
1504
RI- LB-VL-7B (Chea Graie 20250
67.4
63.4
63.3
54.1
72.4
59.6
70.3
61.4
55.6
51.0
1657
79.2
81.7
66.4
39.4
31.0
daal 2025
71
68.9
68.9
61.1
76.7
65.1
74.3
66.1
65.9
54.0
1626
51.5
84.3
68.2
92.0
87.0
Reason: RFT Zero
2025
68.4
66.9
58.5
15.2
58.5
65.3
59.1
51.6
79.8
83.3
68.0
88.1
VLAA-Thinker-778
65.4
83.4
Vical.Srile
72.2 68.6
68.6
65.7
67.3
55.6
84.
60.1
Table 9: Main results on multimodal language benchmarks targeting more general and com- prehensive visual ability. We compare with models post-trained on Qwen2.5-VL-77 (Bai et al. 2023). Best category averages are highlighted in bold. Note that MMF, excluded from vision- centric category average accuracy due scale differences.
Vision-Centric benchmarks thoroughly evaluate perception, real-world understanding, and multi-modal capabilities. We assess MMVP (Tong et al. 2024b). RealWorldQA (X.AT
2024). MMStar (Chen et al., 2024a), MME (Fu et al. 2023), , and BLINK (Fu et al. 2024).
OCR & Chart understanding benchmarks focus on text-rich visual content. We specif- ically use AI2D (Kembhavi et al., 2016) for diagram understanding, JEED-Bench-2
Plus (Li et al., 2024a) for text-rich visual comprehension, Doc (Mathew et al. 2021)
for document understanding, and OCRBench (Liu al. 2023) for comprehensive OCR evaluation.
B.7 EVALUATION ON GENERAL VISUAL CAPABILITIES
As discussed in Sec. 3.1 we have already demonstrated that our game RL training can generalize to visual math reasoning and multi-discipline reasoning benchmarks. To evaluate whether reasoning improvements come at the cost of general visual understanding, we test ViGaL Snake + Rotation on diverse general visual benchmarks. The results, summarized in Tab, 9. show that our model preserves general visual performance at a level comparable to the Qwen2.5-VL-77 base model, while delivering stronger gains in mathematical reasoning. In contrast, prior RL-based approaches often sacrifice visual accuracy when optimized for reasoning. This confirms that our gameplay- driven strategy enhances reasoning ability without compromising broader visual competencies.
B.8 INFERENCE LENGTH ANALYSIS
Recent reinforcement learning studies (Xie al.) 2025 Aggarwal & Welleck 2025) have raised questions about whether performance improvements stem from genuinely enhanced reasoning ca- pabilities or merely from models generating longer responses. address this concern, we analyze the relationship between response length and performance for models trained with our game-based approach.
Model
Response Length
Math Avg.
Qwen2.5-VL (baseline) ViGaL. (ours, RL on games)
250
47.7
268
50.6
Table 10: Response length and performance on visual math benchmarks. Our game-based RL
approach achieves significant performance gains while maintaining comparable inference costs.
Table 10 demonstrates that our performance improvements are not simply due to increased ver- bosity. Our ViGaL model achieves substantial performance gains (50.6% vs. 47.7% while main- taining nearly identical inference costs- the response length increases by only 7% (268 vs. 250 tokens). This minimal increase in response length, coupled with the significant accuracy improve- ment, indicates that the model has learned transferable skills rather than merely generating longer outputs.
27

---

Published as a conference paper at ICLR 2026
These results suggest that game- based RL training enables effective knowledge transfer from game environments to mathematical problem-so/e solvin.. For example, spatial reasoning skills acquired from the Rotation game and coordinate recognition abilities developed through Snake gameplay transfer effectively to visual math tasks. The model thus learns genuine problem-solving strategies while maintaining inference costs.
B.9 REASONING CORRELATION ANALYSIS BETWEEN GAME AND MATHI
To understand the mathematical reasoning patterns in snake game playing, developed system- atic approach to extract and analyze reasoning steps from multiple gameplay traces. Our methodol- ogy uses GPT-5 OpenAl OpenAl (2025) an analytical tool in a two-stage process.
In Stage A, we collect multiple snake game "thinking traces", which are detailed reasoning se- quences generated during gameplay, and distill them into generalized of 8 core reasoning steps. These steps abstract away specific details like exact coordinates or particular board configurations to capture fundamental cognitive operations. The operations include parsing board state, enumerating moves, safety screening, path metric selection, distance computation, target identification, enemy anticipation, and move ranking. This summarization ensures our analysis focuses on transferable reasoning patterns rather than game-specific instances.
In Stage B, we quantify how mathematical each reasoning step is by evaluating its correlation with nine distinct mathematical aspects. We use simple 3-level scoring system where 0 means cor- relation, means low correlation, and means high correlation. GPT-5 analyzes how strongly each step relates to mathematical concepts such coordinate manipulation, distance metrics, analytical reasoning, and geometric properties.
The resulting correlation matrix in Tab. reveals clear patterns. Coordinate-basse reasoning domi- nates steps that involve spatial parsing and movement planning, particularly Steps through 3, Step 5, and Step 7. Meanwhile, analytical and length-based reasoning become prominent in optimization steps like target identification and move ranking, seen in Steps and 8. Steps 4 and 5, which involve path metries and distance computation, show high correlation with both coordinate systems and length calculations. This confirms the geometric nature of pathfinding in grid- based environments. Our systematic analysis demonstrates that even seemingly simple game-playing behaviors require sophisticated integration of multiple mathematical reasoning capabilities.
Prompt Template for Reasoning Step Extraction and Correlation Analysis
Stage A Step Extraction:
Given multiple snake game thinking traces, extract N general reasoning steps (6-9 steps) that capture the core operations. Abstract away instance-specific details and output:
Short, action-oriented step names with one-line deseriptions
General patterns covering: state parsing, move generation, safety screeming, target selection via distance, opponent awareness, scoring/ti--breas,, decision, reporting
Stage B Mathematical Aspect Correlation:
For each extracted step, assign correlation levels (0/1/2) these mathematical aspects:
Expression: Formatiing/stnccuringg outputs
Coordinate: Reading/writing positions, mapping (x.y)
Area: Board regions/bounds as areas
Volume: 3D spatial reasoning (if applicahle)
Applied: Goal-directed task execution
Property: Rules/invariants (bounds, occupancy, collision)
Angle: Angle-hased path reasoning
Analytic: Selection/optimizzaiinn, tie-break logic
Length: Distance metrics (Manhattan/L.1, grid paths)
Output structured table with integer only (0 correlation, low, high).
28

---

Published as a conference paper at ICLR 2026
Table 11: Correlation Matrix of each step reasoning trace of playing snake game with solving math
questions. (O=No Correlation, 1=Low Correlation, 2-High Correlation)
Expression
Coordinate
Volume
Applied
Property
Analytic
Length
Angle
Step
Operation
Parse state
Enumerate candidate moves
Safety screening (worsi-move lest)
4
Choose path metric
5
Compute distances to apples Identify nearest target Anticipate enemy motion (when
present)
8 Rank safe moves
Correlation Legend:
No Correlation
= Low Correlation
High Correlation
B.10 GENERALIZATION TO 3D SPATIAL REASONING (VSI-BENCH)
Obj. Count
Abs. Dist.
Obj. Size
Roont Size
Rel. Dist.
ReL Dir.
Route Plan
Appr. Order
Avg.
Proprictary Model
GPT-4o Hurst al 2024
34.0
43.8
37.0
41.3
31.5
28.5
Gemini- (2))
Gemini-1
Open-source Models
ILtemVLL2-SB Chen et al
2024b
34.6
23.1
28.7
48.2
39.8
36.7
30.7
29.9
39.6
InternVL2-400 CCen
2024b
36.0
34.9
26.9
46.5
31.8
42.1
32.2
34.0
39.6
LongVILA-8B (3)
21.6
29.1
16.7
29.6
30.7
32.5
25.5
VILA--1.5400
LongVA- (2)
Multimodal Reasoning Model Pust-Trained on Owen2.5-VL--2 (?)
Base Model (Qwen2.5-VL-7B)
36.7
41.9
21.4
50.4
36.8
38.5
40.9
29.9
34.1
Video
Visual Jigsaw (?)
38.5
ViGal.
38.7
42.9
22.6
46.7
45.8
38.3
41.7
35.1
36.1
Table 12: Results on VSI-Bench 3D spatial reasoning. We compare proprietary models, open- source multimodal models, and multimodal reasoning models post-trained Qwen2.5-VL--B.
Tab. 12 demonstrates that post-training on two games using synthesized 2D images successfully
generalizes to 3D spatial reasoning tasks. On VSI-Bench (?), ViGaL improves by 2.0% after game- based post-training, slightly surpassing Visual Jigsaw (?). This is notable because Visual Jigsaw is post-trained on 3D video data. whereas approach uses only 2D game data. These results further support the hypothesis that gameplay-bars based RL post-training develops transferable spatial reasoning skills that extend beyond 2D visual domains.
29

---

Published as a conference paper at ICLR 2026
C
CASE STUDY
Rotation
Question:
As shown in the figure, the straight line and the straight line b are intercepted by the straight line Line is perpendiculat to the foot of the perpendicular point A. 21= 70". line is parallel to line a, how many degrees clockwise can line be rotated around point A? Choices: 70" 50 3:"
Qwen2.5-VL-77B 5----------
Ours
lleettlyyy the given internater Unee perp the of
Mantify
ofethe perpenciazar i pallr A. In NP, Line partiietoo line
Unaerstaee The genmettts intttortapp mop bi peos cotor cortrrotee toteee
gsomnetic rebtionntipee CBlc Ser Land pesaiic
so woreecereeeee ng anglo squall
angle between the and line the
between
Ho angle c ginario 73t
da a
2. Deterriee the rstation angle angl reprovets angle betweenn the and vertical lie Since line ia peparditutoo
actweer the olongle 10 and the fieil angle since livel imperdiosarr
orellegl beteeece fine line
paraled kive
Thersfore, the rotation io Than buaan rotated tockwaree ywu auf purrs & The cxvedtt aesaee A
Therstore, the can rotaned 20r doccalte around the pdirt The
(a) A case study from Mathverse. Base model misinterpreted geometric configuration and rotation direction, while our model correctly identified the perpendicular relationship and calculated the proper angle.
Coordinate
Question:
Determine whether the graphed function is even, odd, neither.
Cwwe2255VL-P2 1i--5---V-----tst-t
Ours
Spermetry respect Jeis groph were serrrctric respect
yn the graph
Let's ssarmine the graph the function. The graph shows and
paraiocla opens
about y-
anis symmotry indicates
function even.
y-adis oumple, the point (A yJ dons howe corn monesponding point E-l, on the graph.
symmetri with
To confirm, we can check few points:
Symmerty with respect to the ongle: prash were symmettic with respect the crigi,, groph.
For xim 2, the furction value fiz). for x x the function
cac's point Ou, on glaph, point
value ft-2) Since graph symmetiic
groph
This contirms that f/aa - 100, which deflnitionn an Leven function.
painttt does have cormpordin an (-L,
X
Sance the graph does not schirt symmetry with respect to atter the y- bak origie the function aoither sonn for oud The final ansserr
Therefore, the function is even. The final answer is: even.
metther
(b) A case study from Mathverse. Base model misperceived critical visual information like symmetry and coordinates graphs, while our model demonstrated accurate visual perception for mathematical elements.
Figure 9: Comparison of base model and our model after rule-based RL training, showing improved visual-mathematiial reasoning on geometric and coordinate problems.
We provide quantitative comparison examples below to demonstrate reasoning improvements on mathematical problems after RL training. In Fig. Pal when solving a geometric angle problem, the base model fails to correctly interpret the critical relationship between perpendicular lines and corre- sponding angles. It makes contradictory assumptions about angle measures, leading to an incorrect calculation of the required rotation. In contrast, our ViGaL precisely tracks the geometric constraints and properly calculates the angle difference between initial and target positions. In Fig. 9b] when analyzing function properties from a graph, the base model incorrectly claims the function lacks symmetry despite clear visual evidence. It fails to recognize the fundamental y-axis symmetry of the parabola shown in the image. Our model immediately identifies this critical symmetrical pattern and correctly applies the appropriate mathematical definition of even function, demonstrating enhanced visual perception of mathematical structures.
D USE OF LARGE LANGUAGE MODELS
In accordance with ICLR 2026 policies, we disclose that a large language model (LLM) was used during paper preparation. Specifically, L.MMs were employed for grammar correction, wording
30

---

Published as a conference paper at ICLR 2026
refinement, and drafting of non-technical text passages. All research ideas, methods, experiments, and analyses were conceived, implemented, and validated by the authors, who take full responsibility for the correctness and integrity of the content.
31


Reviews:
[{"review_id": "q3KlZLDyrg", "reviewer": "Reviewer_ZnCJ", "summary": "The paper proposes Visual Game Learning (ViGaL)  that uses RL post-training on simple visual games (e.g., Snake and a Rotation task) to elicit reasoning capability in multimodal LLMs in other domains such as math. Instead of training on math or benchmark-style reasoning data, a MLLM is fine-tuned to play these games with rule-based rewards; the resulting model then transfers the acquired skills to out-of-domain tasks while retaining general visual abilities. The paper argues that gameplay can serve as a scalable surrogate task for RL post-training to unlock broadly useful multimodal reasoning.", "questions": "* The prompts include thinking instructions synthesized by GPT-4o. Are they necessary? How much do they affect the results?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper shows that RL post-training purely on simple visual games (Snake, Rotation) yields measurable gains on other seemingly unrelated domains such as math despite no direct supervision from those tasks. This is a surprising finding and is worth spreading.\n- The gameplay setup enables verifiable rule-based rewards that are friendly to reasoning training, avoiding the need for expensive reward models or human labels. The fact that it can generalize to other domains shows that it has high potential.\n- Another benefit of using game play for reasoning training is that there are a large amount of games out there with highly diverse contents. They cover all kinds of aspects of human skills, and can be a rich supplement to reasoning training data.\n- The fine-grained analysis in section 3.1 is interesting. For example it connects specific games to specific math subfields, e.g. the game Rotation aligns with angle/length questions.", "weaknesses": "* The games are relatively simple, which is understandable though because it is the first effort to explore this direction.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ceQbUN3i6p", "reviewer": "Reviewer_DBfm", "summary": "The paper proposes using gameplay as a surrogate reasoning task to improve multimodal reasoning abilities of MLLM.\nThe model is post-trained with reinforcement learning on two simple games — Snake (2D grid planning) and Rotation (3D object rotation prediction). It uses a lightweight reward (accuracy + format) and no KL regularization. After training, the model shows improved performance on downstream reasoning benchmarks (math, geometry, and multimodal QA), while maintaining its visual perception capabilities.", "questions": "Overall, this is a great paper. I would like to see the authors answers to my weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The idea of using structured games to indirectly train reasoning is novel and very interesting to me.\n- The performance of using games improves  5–8% on math and spatial tasks, showing gameplay can transfer reasoning skills.\n- The authors provided careful analysis on reward, prompts, and difficulty to show the effectiveness of each component.", "weaknesses": "- It is not very clear how to design a game for different types of reasoning abilities. That being said, although games are useful for post training, it requires design for each task.\n- The authors are currently only designing two kinds of games with spatial/math-like games used, it is unclear whether other reasoning abilities can also be solved by games.\n- The paper provides limited analysis of why the game can improve the math reasoning ability. Specifically, what kind of questions in the benchmark are these post-trained models most benefited from.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "club8Gis6t", "reviewer": "Reviewer_w1f6", "summary": "This paper proposes ViGaL (Visual Game Learning), a post-training method that applies reinforcement learning (RL) on simple visual games—primarily Snake—to improve the multimodal reasoning performance of a 7B MLLM on downstream benchmarks such as MathVista and MMMU. The authors show that models trained on these games achieve modest gains on reasoning tasks, even though no domain-specific reasoning data is used during training. The core claim is that gameplay can serve as a surrogate task to develop transferable reasoning skills.\n\nWhile the empirical results demonstrate some performance improvement, the work largely repackages well-established ideas—such as multi-task learning, curriculum learning, and pretext tasks—under a new but superficially motivated paradigm. Crucially, it fails to provide any principled insight into why or when such transfer occurs, nor does it offer a methodology for designing effective auxiliary tasks for a given downstream goal. As such, the contribution appears incremental rather than foundational.", "questions": "1) Has the phenomenon of using simple, structured tasks to improve performance on complex ones been studied before the LLM era? If so, shouldn't this paper engage more deeply with that literature to justify its claimed novelty?\n\n2) Please see the weakness.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1) Clean Experimental Setup and Broad Benchmarking: The paper evaluates the proposed method on a wide range of established multimodal reasoning benchmarks (e.g., MathVista, MMMU, CLEVR+), which lends credibility to the reported performance gains. The experimental design is generally sound, and the use of rule-based RL avoids the complexity of reward modeling.\n\n2) Demonstration of Cross-Domain Performance Gain: It is empirically shown that training on a simple game can lead to measurable improvements on unrelated reasoning tasks. This serves as a proof-of-concept that some form of transfer is possible, even if the mechanism remains opaque.", "weaknesses": "1) Lack of Novelty: Repackaging Known Ideas Without Substantial Advance\n\nThe idea that training on one task can improve performance on another—i.e., multi-task learning (MTL) or transfer learning—is decades old. The use of pretext tasks in self-supervised learning has been standard in vision and NLP. Even within RL, curriculum learning and autocurricula have long demonstrated that simple environments can give rise to complex behaviors. The paper does not convincingly argue why using Snake as a surrogate task is fundamentally different or more effective than prior approaches. It presents an anecdotal case rather than a general principle.\n\n2) No Theoretical or Mechanistic Insight\n\nWhat specific skills are learned in Snake that transfer to math reasoning?\n\nAre there measurable properties of an auxiliary task (e.g., action space complexity, reward sparsity) that predict transfer success?\n\nThe most pressing practical question—how should one design or discover an effective auxiliary task for a given downstream goal?—is completely unaddressed. The choice of Snake appears arbitrary. Why not Tetris? Why not a maze? The paper provides no criteria, heuristics, or framework for answering this.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "b39qUoDOhq", "reviewer": "Reviewer_uv38", "summary": "The paper proposes Visual Game Learning (ViGaL), a post-training paradigm where an MLLM (7B) is fine-tuned via reinforcement learning on simple arcade-style games (e.g., Snake and a Rotation task) to acquire transferable reasoning skills. Without using any math/benchmark-specific data during RL, ViGaL improves downstream multimodal reasoning (MathVista and MMMU) while preserving general visual abilities, outperforming specialist models trained on curated reasoning data.", "questions": "1. Can you articulate a principled game-selection taxonomy (e.g., planning, geometric transforms, partial observability, stochasticity) and explain why Snake and Rotation instantiate distinct buckets?\n\n2. What ex-ante criteria (beyond convenience) determined these two games, and which candidate games were considered but rejected? If any rejected candidates, were they rejected before or after experiments, and why? \n\n3. Do you observe negative transfer for any added game types?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Overall, this is a clearly written and well-structured paper with polished figures that make the pipeline easy to follow from setup through results. \n\n2. The core idea—leveraging controllable visual games with an RL loop as a scalable surrogate to induce transferable reasoning—is novel and timely, pushing beyond curated math datasets toward a more generalizable training signal. \n\n3. Empirically, the paper shows that gameplay-driven RL improves downstream multimodal reasoning without eroding general vision capabilities, and it consistently outperforms SFT when trained on the same data. The evidence is strengthened by broad evaluations and careful ablations (modalities, difficulty control, data scaling), suggesting robustness rather than benchmark luck. \n\n4. The approach is also practical: simple game environments, rule-based rewards, and a straightforward RL recipe lower the barrier to adoption and invite community exploration.", "weaknesses": "1. The paper proposes a promising direction, but the evidence base feels narrow: only two games (Snake and Rotation) are used, which makes it hard to argue for broad “play-to-generalize.” The paper explains that these two games let the authors probe “reasoning and perception” (Snake for spatial/planning, Rotation for angle perception) , yet this rationale is thin and the selection criteria appear ad hoc.\n\n2. The idea of the paper would read as substantively stronger with diversity and robustness checks across more games.\n\n3. The paper proposes that different games improve different math sub-skills, but the current analysis, while suggestive, is correlational and limited to two sources.\n\n4. The paper proposes training via two games (Snake, Rotation) and shows encouraging transfer, but it does not yet establish games as a better alternative to standard post-training (e.g., curated math/logic data or instruction tuning) under controlled, budget-matched conditions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)