Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with Jigsaw Puzzles
Authors: Zifu Wang
Junyi Zhu
Bo Tang
Zhiyu li
Feiyu Xiong
Jiaqian Yu
Matthew B. Blaschko
Venue: Accepted by TMLR ()

Abstract: The application of rule-based reinforcement learning (RL) to multimodal large language models (MLLMs) introduces unique challenges and potential deviations from findings in text-only domains, particularly for perception-heavy tasks. This paper provides a comprehensive study of rule-based visual RL, using jigsaw puzzles\footnote[2]{For a preliminary study with another pretext task, image rotation, please refer to~\Cref{app:rotation}.} as a structured experimental framework. Jigsaw puzzles offer inherent ground truth, adjustable difficulty, and demand complex decision-making, making them ideal for this study. Our research reveals several key findings: \textit{Firstly,} we find that MLLMs, initially performing near to random guessing on the simplest jigsaw puzzles, achieve near-perfect accuracy and generalize to complex, unseen configurations through fine-tuning. \textit{Secondly,} training on jigsaw puzzles can induce generalization to other visual tasks, with effectiveness tied to specific task configurations. \textit{Thirdly,} MLLMs can learn and generalize with or without explicit reasoning, though open-source models often favor direct answering. Consequently, even when trained for step-by-step reasoning, they can ignore the thinking process in deriving the final answer. \textit{Fourthly,} we observe that complex reasoning patterns appear to be pre-existing rather than emergent, with their frequency increasing alongside training and task difficulty. \textit{Finally,} our results demonstrate that RL exhibits more effective generalization than Supervised Fine-Tuning (SFT), and an initial SFT cold start phase can hinder subsequent RL optimization. Although these observations are based on jigsaw puzzles and may vary across other visual tasks, this research contributes a valuable piece of jigsaw to the larger puzzle of collective understanding rule-based visual RL and its potential in multimodal learning. The code is available at: \href{https://github.com/zifuwanggg/Jigsaw-R1}{https://github.com/zifuwanggg/Jigsaw-R1}.

Published in Transactions on Machine Learning Research (10/2025)
Ji A Study of Rule-based Visual Reinforcement Learning with Jigsaw Puzzles
Zifu Wang"
ESAT-PSI, KU Leuven
Junyi Zhu"
ESAT-PSI, KU Leuven
Bo Tang"
University of Science and Technology of China Memory Tensor
Zhiyu Li Memory Tensor
Feiyu Xiong Memory Tensor
Jiaqian Yu Samsung R&D Institude China
Matthew B. Blaschko
ESAT-PSI, KU Leuven
Reviewed on OpenReview: https: //
// openrevie.. net/ forun? id= XqQCsuyPue
Abstract
The application of rule-based reinforcement learning (RL) to multimodal large language models (MLLMs) introduces unique challenges and potential deviations from findings in text- only domains, particularly for perception-heavy tasks. This paper provides a comprehensive study based visual RL, using jigsaw puzzles as a structured experimental framework. Jigsaw puzzles offer inherent ground truth, adjustable difficulty, and demand complex decision- making, making them ideal for this study. Our research reveals several key findings: Firstly, find that MLLMs, initially performing near random guessing the simplest, jigsaw puzzles, achieve near-perfect accuracy and generalize complex, unseen configurations through fine-tuning. Secondly, training on jigsaw puzzles can induce generalization to other visual tasks, with evenesiveness tied to specific task configurations. Thirdly, ML.LMs can learn and generalize with or without explicit reasoning, though open-source models often favor direct answering. Consequently, even when trained for step-by-step reasoning, they can ignore the thinking process in deriving the final answer. Fourthly, we observe that complex reasoning patterns appear to be pre-existing rather than emergent, with their frequency increasing alongside training and task difficulty. Finally, results demonstrate that. RL exhibits more elfective generalization than Supervised Fine- Tuning (SFT), and an initial SFTT cold start phase can hinder subsequent RL optimization. Although these observations are based on jigsaw puzzles and may vary across other visual tasks, this research contributes valuable piece of jigsaw to the larger puzzle of collective understanding rule-based visual RL and its potential in multimodal learning. The code is available at: httpp:: ctaps:://gittuubcomn http:///uuuubboo///////agggg//// www.w//ggwwgggg/eeoaa/ Jigssa/-Il
* "quual contribution. Correspondence wang
For preliminary study with another pretext task, image rotation, please refer Appendix

---

Published in Transactions on Machine Learning Research (10/2025)
1 Introduction
Post-training has emerged a critical step for enhancing the performance of large language models (LLMs). A significant contribution in this area is DeepSeek-R1 (Guo et al. 2025), which employs a simple yet effective
mule-based reinforcement learning (RL) strategy. This approach can mitigate reward hacking (Gao et al.
2023
without relying on traditional scaffolding techniques (Lightman et al. 2024 Wang ct al.
2024a
Xic
et al.
2024
Xin et al. 2025a), and has shown robust generalization capabilities in LLMs across various
domains such as mathematics, coding, sense reasoning and logic puzzles (Chen et al. 2025b
Guo
ct al.
2025
Liu al. 2025a Xie al. 2025b
Despite these advancements, the application of rule-based RL to multimodal contexts is still in its early stages. Unlike purely textual environments such as DeepSeek-RL. multimodal large language models (MLLMs) face the complex challenge of integrating and reasoning over both textual and visual information. This introduces unique difficulties and potential deviations from findings in purely linguistic domains (Gandhi et al. 2025
Guo et al. 2025: Hassid et al. 2025 Lee et al.) 2025 Liu et al. 2025e
Marjanović et. al. 2025 ie el al..
2025bb.
For instance, a key insight from DeepSeek-R1 R1 is the model's natural achievement of test-time scaling (Snell et al. 2025) through pure RL, evidenced by increased completion lengths and the emergence of complex reasoning patterns (Gandhi et al. 2025), a phenomenon termed the aha moment. Nevertheless, perception-heary tasks, such as spalial reasoning, oflen permit concise answers derived directly from visual underslanding This contrasts sharply with beasoning-intenssie
problems like mathematics and coding that benefit from extended, step-by-step reasoning. In such perceptual domains, an explicit, lengthy thinking process characteristic of some rule-based RL successes in text might even prove detrimental (Jiang et al., 2025
This paper presents a comprehensive study of rule- RL within the multimodal domain. Rather than relying on verifiable answers from cxisting MLLM benchmarks, we revisit classic pretext task in computer
vision: solving jigsaw puzzles (Carlucci et al. 2019 Chen et al. 2023 Doersch et al. 2015 Du et al. 2020
Kim et al., 2018; Noroozi & Favaro, 2016). This task (a visual illustration is presented Figure offers a compelling testbed for studying rule-based visual RL for several reasons:
Firstly, jigsaw puzzles inherently provide a ground truth. This allows for the direct generation of rule-based rewards across various visual domains, eliminating the need for expensive human annotation.
Secondly, the complexity of these puzzles is readily adjustable by varying the number of pieces, facilitating a structured experimental framework.
Lastly, solving jigsaw puzzles involves an interplay of step-by-step reasoning and visual perception. The human approach iteratively placing pieces while considering local and global visual coherence provides a rich analog for the complex decision-malel making processes we aim explore in MLLMs.
Using jigsaw puzzles as our experimental framework, this rescarch undertakes an in-depth exploration of multifaceted aspects within rule-based visual RL. Our investigation yields findings that address the following key research questions:
Research Question #1: How do contemporary MLLMs perform on the classic pretext task of jigsaw puzzles?
Without lask-specific training, the performance of contemporary MLLMs on the simplest jigsaw puzzles (i.., 2x1) is comparable to random guessing. However, fine-tuning enables these models to effectively solve such puzzles with near-perfect accuracy. Importantly, these Iearned abilitics generalize to more complex configurations (e.g., 3x1) not encountered during training.
Research Question #2: Can MLLMs trained to solve jigsaw puzzles develop generalizable abilities applicable to other visual tasks?
Training models on jigsaw puzzles enables generalization to downstream tasks. The effectiveness of this generalization is dependent on specific task configurations, including puzzle size, question type and training dataset.
2

---

Published in Transactions on Machine Learning Research (10/2025)
Research Question #3: Given that extended reasoning may be detrimental for some perceptual tasks, is an explicit thinking process still beneficial when employing rule-based visual RL to solve jigsaw puzzles?
MLLMs can Iearn and generalize with or without an explicit reasoning process. However, open-source MLLMs typically show stronger performance in direct answering. As a result, even when trained to employ step-by-step reasoning, they tend to disregard the thinking process in deriving the final answer.
Research Question #4: Considering that many visual tasks can be solved with concise outputs, docs the aha moment still emerge in MLLMs trained on jigsaw puzzles?
The aha moment, characterized by the sudden emergence of complex reasoning patterns, is not observed. Instead, these patierns are pre-existing within MLLMs and are readily elicited by tasks with inherent reasoning structures, like jigsaw puzzles. Furthermore, the frequency of these reasoning patterns demonstrably incrcases throughout training and in response to grcater task difficulty.
Research Question #5: How does supervised fine-tuning (SFT) compare with RL in terms of generalization?
SFT generally demonstrates less effective generalization compared to RL. Besides, initiating training with a SFT cold start phase can make later RL optimization less effective.
2
Related Work
2.1 Jigsaw Puzzles
Since their inception, jigsaw puzzles have been closely linked to learning. Around 1760, British cartogra- pher John Spilsbury created the first dissocted map-an an carly jigsaw phy (Wikipedia, 2025). More recently, the lask of solving jigsaw puzzles has gained considerable attention within the computer vision community pretext task. The central idea is that neural networks, by training to reassemble images from shuffled patches, can develop rich feature represcntations transferable to varions downstream applications.
to teach googra-
For instance, Doersch et al. (2015 ; Noroozi & Favaro 2016 propose pre-training Convolutional Neural Networks (CNNs) on jigsaw puzzles before fino-tuning them for downstream tasks such as image classification and object detection. Building on this, Kim al. (2018) increase task complexity by integrating jigsaw puzzles with inpainting and colorization, demonstrating transferable representations for image classification and semantic segmentation. Moreover, Carlucci et al. (2019) employ jigsaw puzzles self-supervised regularization term, achieving strong domain generalization. [u et al. (2020) further advance this area by incorporating jigsaw puzzles into a progressive training pipeline for fine-grained classification. Most recently, Chen et al. (2023) investigate the efficacy of jigsaw puzzles in the context of vision transformers (ViTs), highlighting the importance for architect ure-peciic modifications. Nevertheless, to the best of our knowledge, the adoption of jigsaw puzzles as pretext tasks in the era of LLMs remains an unexplored area.
2.2
based Reinforcement Learning in Textual Domains
In order to mitigate reward hacking (Gao et al. 2023); DeepSeek-R1 (Guo et al. 2025) adopts a simple
yet effective rulc-basod RL approach. This method diverges from traditional scaffolding techniques such as process reward models (Lightman et al.] 2024 Wang et al., (0224a) and Monte Carlo Tree Search (MCTS) (Xie)
et al. 2024 Xin et al.) 2025a), and has proven effective in acquiring reasoning skills transferable across diverse
domains, including mathematics, coding, common-sense reasoning and logic puzzles (Chen et al. 2025b
Guo
et al.
2025
Liu et al. 2025a Xie et al. 2025b
Beyond the generalization capabilities, the reasoning chains produced by models trained with rule- based RL have been the subject of extensive study. For example, DecpSock-R1 demonstrates natural achievement of test-time scaling (Snell et al., 2025), evidenced by increased completion lengths and the sudden emergence of complex reasoning patterns, phenomenon termed the aha moment. However, these findings are debated.
3

---

Published in Transactions on Machine Learning Research (10/2025)
Hassid et al. 2025 Xie al. (2025b) contend that longer responses do not guarantee better reasoning and Liu et al. [2025e] challenge the notion of sudden emergence, positing that these complex behaviors might be inherent. in the base model rather than appearing abruptly. Separately, the reasoning processes within DeepSeek- R11 have also been noted for cxhibiting human- language processing characteristics (Marjanović et al. 2025).
Investigating these further, Gandhi et al. (2025) draw connections to human psychology, developing a framework that categorizes these reasoning patterns into four key cognitive behaviors. Their findings suggest that Qwen2.5 (Qwen et al. 2025 naturally exhibits these behaviors, whereas Llama3.2 (Grattafiori al.,
2024 initially does not.
2.3 Rule-based Reinforcement Learning in Multimodal Domains
Applying rulc-based RL to multimodal domains is an emerging field. Unlike models that operate solely on lext (e.g., De) MLLMs operate in more complex environment by processing both textual and visual information. This inherent complexity introduces unique difficulties and potential deviations from established findings in purely linguistic domains. Recently, several contemporary studics have emerged, reporting varied findings across diverse task settings.
A significant line of exploration, motivated by the success of DeepSeek-RL R1 in mathematical reasoning, involves adapting rule-based RL techniques to multimodal mathematical tasks (Chen et al., 2025a Deng ct al. 2025
Liu et al. 2025c Meng et al. 2025 Peng et al. 2025; Wang et al. 2025b Yang et al. 2025). These efforts
have consistently demonstrated strong out-of-domain generalization. Notably, studies by Chen et al. (2025a)
Meng et al. (2025) have documented the emergence of the aha moment tte spontaneous exhibition of sophisticated reasoning patterns like self-correction-ariiee
from end-to-end RL training.
However, the landscape appears different for visual perception tasks, including visual classification, visual grounding and spatial reasoning. While research in this area (Bai et al.) 2025b Chen et al. 2025d Lai et al.
2025 Li et al. 2025c Liao et al.. 2025 Liu et al. 2025f d Shen et al.) 2025 Yu et al. 2025a
Zhou al.) 2025
also show robust out-of-distribution generalization, the aha moment has not been observed when employing instruction-tuned models. Indeed, the necessity of explicit reasoning steps for these perception-intensive tasks has been questioned, as direct answers may often suffice (Jiang el al. 2022). In fact, work by Lai el al. (2025 0 Li et al. (2025c): Yu et al.] (2025a) suggest that training models for direct answering often leads to superior performance compared to models trained for explicit step-by-step reasoning.
2.4 based Reinforcement Learning without Labeled Datasets
While effective, rule-based RL often requires extensive human elforts to curate verifiable data. To miligate this, Xie al. (2025b) leverage logic puzzle called Knights and Knaves (K&K), which allows for controllable difficulty and algorithmically generated ground truth. They demonstrate that models trained on this synthetic dataset can develop complex reasoning skills transferrable to mathematical tasks. Building on this, Chen et al. (2025b); Liu et al.] (2025a); Stojanovski et al. (2025) extend it to a wider array of textual puzzles, such Sudoku and the game of 24. In a similar spirit, Feng et al. (2025); ; Tong et al. (2025) design several synthetic visual puzzles, demonstrating generalized reasoning abilities in MLLMs. For a greater level of autonomy, Zhao et al. (025al) introduce a framework where single model learns to propose tasks that maximize its own lcarning progress and improves reasoning by solving them, without relying on any external data.
Another stream of research bypasses the need for external annotations by leveraging signals from the models themselves. For instance, Xin et al. (2025b) use the format and length of model outputs as surrogate signals. Shafayat et al. (2025) Wei et al. (2025): Zuo ct al. (2025) employ majority voting from model outputs as the supervision signal. Agarwal et al. (2025): ; Gao el al. (2025); Prabhudesai et al. (2025); Zhang el al. (2025); ); Zhao et al. (2025b) utilize a model's own confidence as the intrinsic reward. Perhaps most surprisingly, Shao
et al. (2025) discover that even spurious rewards, such as random rewards or incorrect labels, can yicld gains nearly matching those from ground truth rewards. Their analysis suggests that RL may not teach models new reasoning capabilities but instead activates latent ones already present in the base model.
4

---

Published in Transactions on Machine Learning Research (10/2025)
Original Image
Jigsaw Image
Other Sizes
Shuffle
2x2
Full Jigsaw Puzzle
Pair Jigsaw Puzzle
patches currently positions and determine their relative position in the original image...
...For each patch, determine its correct position index in the original image...
Thinking:
Non-thinking:
Thinking:
Non-thinking:
<think?...
1 1324
<think?..
(B) 2 is on the upper left
<ithink>
<ithink
<answer>4 3</answer?
<answer>(C).... <lanswer>
Figure 1: An overview of the lask design. An original image divided into an mxn grid of patches, which are then randomly shuffled to create the jigsaw image. We consider two question types (full or pair) and two prompting strategics (thinking or non-thinking).
3
Task Design
This section outlines the formulation of Jigsaw puzzles in a format suitable for processing by MLLMs. Subsequently, we introduce rule-based reward system designed for RL training. All specific prompts are detailed in Appendix I and Figure provides conceptual overview.
3.1 Jigsaw Images
The creation of jigsaw puzzles begins with an input image. The image is first partitioned into an mxn grid of patches and the task's difficulty can be readily adjusted by varying the values m and n. Optionally, masked region can be added between patches to highlight the grid layout. If the image's height is not perfectly divisible by m, its width by n, the image trimmed from the bottom right edges ensure its dimensions are exact multiples of the patch count. Subsequently, these patches are randomly shuffled to create the jigsaw images. To uniquely identify each patch's location within this grid, position indices are assigned sequentially in row-major order, from 1 (top-left) to mn (bottom-right).
3.2 Question Types
Based on these shuffled images, we formulate distinct question types. These questions either directly assess the MLLM's ability to reconstruct the original image require it to reason about the relative positions between the shuffled patches in the initial image. An additional question type, assessing both spatial reasoning and visual grounding, is presented in Appendix
Full. In this task, MLLMs are required to identify the initial position index for each shuffled patch, thereby enabling the reconstruction of the original image. The answer list of mn numbers arranged in an mxn grid, where each number corresponds to shuffled patch and indicates its original position index. The complexity of this task is therefore mn!.
5

---

Published in Transactions on Machine Learning Research (10/2025)
Pair. For this task, two patches are randomly selected, and the MLLM's objective is to identify their relative positions in the original image. If the image is divided into a single row (m 1) a single column (n =), only two relative positions are possible (e.g. left/right or top/bottom, respectively). Otherwise, eight distinct relative directions are possible (c.g., top-left, directly above, to the right, bottom-right). This task is structured as a multiple-choice question, requiring the model output a single letter corresponding to the correct relative position. Consequently, the task complexity either or 8, depending on whether the image is divided into single row/column or not.
3.3 Thinking or Non- oon-thinking
For any given question, regardless of its type, we investigate two prompting approaches for MLLMs. One approach instructs MLLMs to include an explicit thinking process in their response, similar to the format used in DeepSeek-R1 (Guo et al.) 2025). The inclusion of explicit reasoning has been shown to improve generalization across diverse downstream tasks (Hu et al. 2025 Xie et al., 2025b) and is considered valuable for enhancing safcty and transparency (Chen al. 2025f Wang et al. 2025a). Conversely, as explicit step-by-step reasoning might be detrimental for tasks heavily reliant on visual perception (Jiang et al.) 2025) also explore alternative: prompting the ML.LM to provide the final answer directly, without detailing intermediate reasoning. In summary, examine the following two distinct instructions:
Thinking, MLLMs are instructed to first output their thinking process, which should be enclosed within <think> and </think> tags. Subsequently, they must provide the final answer, enclosed within <answer> and </answer>> tags.
Non-thinking. MLLMs are prompted directly output the final answer to the posed question.
3.4 Rule-based Rewards
The reward serves as the primary training signal in rulo-based RL. Our reward system consists of two componenis: an accuracy reward and format reward. The total reward is the sum of these two components.
Accuracy reward. This reward assesses the correctness of the response, For full questions, the reward is calculated as the proportion of correctly identified position indices to the total number of indices (mn), resulting in a fractional value between 0 and 1. For pair questions, the reward is binary: 1 for a correct choice and 0 otherwise.
Format reward. The final answer must be extractable in the prescribed format: list of mn integers arranged in an mxn grid for full questions single letter for pair questions. With thinking instructions, the answer is extracted from within the <answer>> and </ans //answer>> tags. For non-thinking instructions, it is extracted directly from the raw output.
Furthermore, for thinking, the model must adhere to the instruction of enclosing its reasoning process within <think> and </think> tags and the final answer within <answer> and < </answer> tags. Each tag must appear exactly once and in the correct sequence (the thinking process before the final answer).
The format reward is 0.5 if the output adheres to all these requirements, and 0 otherwise. In Appendix B present an ablation study of our reward design, comparing binary versus fractional accuracy rewards for full questions and analyzing the impact of format reward weights.
4 Experimental Setups
4.1 Datasets
COCO (Lin et al., 2014). This datasot serves as the foundation for training and evaluating jigsaw puzzles. We exclusively use the images and randomly generate the ground truth permutations. For training, we employ the train2014 split, and for testing, randomly select 1,000 images from the test.2014 split.

---

Published in Transactions on Machine Learning Research (10/2025)
CV-Bench (Tong et a 2024a). This benchmark repurposes standard vision datasets such COCO with a multimodal context, offering 2,638 test examples. It includes four distinct tasks: spatial relationship and object counting for 2D understanding, and depth order and relative distance for 3D understanding.
MMVP (Tong et al., 2024b). Similar to CV-Bench, MMVP adapts classic vision datasets like Ima- geNet. (Deng al.) 2009) to create 300 multimodal questions. This benchmark assesses MLLMs on nine fundamental visual patterns, such as orientation, perspective, and structural characteristics.
SAT (Ray et al., 2024). This synthetic dataset features indoor scenes, from which we exclusively use its static split. We categorize the original questions into the four task types defined in CV-Bench. For testing, we randomly sample 500 questions per task, yielding a total of 2,000 test questions. The remaining 96,924 questions constitule the Training set.
Super-CLEVE CLEVR (Li et al.) 2023). This is another synthetic dataset containing various vehicle models like cars and motorcycles. Following (Chen al. 2025d). we select 200 images from the test split and adapt the dataset counting problems.
4.2 Models
Proprictary Models, We evaluate GPT-4.1 OOpenA 2025a), GPT-4.1-mini OPenAA 2025a). and Claude
3.5 Haiku [Anhhropic] 2024).
Open-Source Models. We consider Qwen2-VL V--2--L--BBBBBB
22- Base
[Wagg al. 2024b) and several Instruction-tunet
models: Qwen2.5- 5LL----L-BBBB///-// (Bai et al. 2025a), Qwen2-VL-21
[aang et al. 2024b), and InternVL2.55
2B
CCen et al.
2024
4.3
Implementation Details
We use GRPO (Shao et al. 2024g as the reinforcement learning algorithm. The GRPO iteration μ 1, the KL efficient B = 0.04 and the clipping value 0.2. Given that thinking is substantially more computationally expensive, we perform 1,000 training steps for it, compared to 2,000 steps for non-thinking. More details on training costs are in Appendix D and an ablation on the number of training steps in Appendix E In each training step, 64 unique prompls are processed, with each prompt being sampled 8 times to calculate the advantages. The sampling temperature is set to 1, and top-k sampling is used with 50. The learning rate initiates at 1e-6 and linearly decays to 0.
As for SFT, we prepare the thinking data using the same prompt (as provided in Appendix D to instruct the model to include a reasoning process. We then apply rejection sampling to retain only those instances where the thinking process correctly leads to the final answer. This complete output, including both the reasoning chain and the final answer, subsequently used for fine-tuning. For non-thinking, we fine-tune the model directly on ground-truth answers. Both configurations are trained for 1,000 steps, with a batch size of 512. All other hyperparameters such as the lcarning rate are identical those used in RL.
5 Experiments
This section presents results designed to address the proposed rescarch qucstions. The main paper focuses on rustriccion-luned models; for a discussion regarding Qwen2-VV--2B-Basee please refer to Appendix F
Research Question #1: How Do MLLMs Perform on Jigsaw Puzzles?
To answer the question, we first. train models on 2x1 jigsaw puzzles using the training split of the COCO dataset. To introduce task diversity for non-square puzzles, the piece order is randomly shuffled in 50% of instances (c.g. 2x1 becomes 1x2 and vice verse). We then evaluate model performance on the same question type but with varying puzzle sizes, utilizing the test split of the COCO dataset. Evaluation results for pair questions shown in Table and the corresponding training dynamics illustrated in Figure Additional
7

---

Published in Transactions on Machine Learning Research (10/2025)
results, including full jigsaw puzzles and few-shot learning, provided in Appendix A and Appendix respectively.
Finding 1.1: MLLMs struggle with Jigsaw puzzles before fine-tuning. As demonstrated in Tables and [ jigsaw puzzles are notably difficult for MLLMs without task-specific training. Prior to fine-tuning, even powerful proprietary models perform levels comparable to random guessing, struggling even with the simplest jigsaw puzzles (i.e., 2x1).
Finding 1.2: MLLMs exhibit efficient learning and generalization for jigsaw puzzles after fine- tuning. Despite the initial difficulty, MLLMs show strong capacity to learn and solve these puzzles after fine-tuning. For example, the reward progression for (wen2.5-VL--Bt depicted in Figure 2 indicates rapid convergence to near-perfect accuracy. Notably, models trained exclusively on 2x1 jigsaw puzzles successfully generalize their learned abilities to larger puzzle sizes beyond the training distribution (e.g., 3x1).
Takeaways #1. Without sask-sp training, modern MLLMs perform no better than random guessing on the simplest jigsaw puzzles (i.c., 2x1). Nevertheless, after fine-tuning, they can solve these puzzles almost perfectly and can generalize the learned abilities to more complex configurations (e.g., 3x1) unseen during training.
Table 1:
Evaluation results on pair jigsaw puzzles with different sizes. For thinking and non non-thinking of the same model, the better result underlined.
Thinking
Method
2x1
3x1
4x1
2x2
AVG
Random
50.00
50.00
50.00
12.50
10.63
GPT-411
54.10
53.40
54.70
20.70
45.73
GPT-4.1-mini
61.90
54.50
54.80
20.30
47.88
Clande 3.5 Haiku
61.00
49.10
51.30
15.20
44.15
Swen2.5-VL-221
43.40
50.20
52.80
18.00
41.10
Qwen2.5-VIL-7B
49.40
48.70
50.60
15.80
41.12
+ Jigsaw-R1
97.80748.40
61.70"13:00
54.8074.20
15. 15.20
57.38
Qwen2.5-VT--37
48.50
47.50
48.80
12.20
39.25
+ Jigsaw-Hs
748.30
58.80
11:30
52.2013.00
13. 13.10
55, 15.22 15:19
Qwen2-Vil VL-2B
32.80
33.90
32.10
10.30
27.27
+ Jigsaw-Bl
13730
56.50122:60
48. 16:00
10.7070.40
46.40 19.13
InternVL.2.5.2
44.90
41.90
48.60
9.70
36.28
+ Jigsaw-R.
99.30
63.00 72100
53.00
13.70 13.70
57.25 57.25120.97
Non- Mon-thinking
Method
2x1
3x1
4x1
2x2
AVG
Random
50.00
50.00
50.00
12.50
40.63
GPT-4.1
53.80
49.70
50.90
16.50
42.73
GPT--411mmii
62.50
52.70
53.90
16.20
46.32
Claude 3.5 Haiku
31.30
42.40
43.00
13.20
32.47
Q Wwen2.5-VL-722
52.50
51.60
55.60
14.20
43.48
Qwen2.5-VL-77
50.40
49.60
54.20
13.20
41.85
+ Jigsaw-R1
148:00
65.90
16.30
16:30
53.901-0.33
14.90
58.40" 401166
Qwen2.5-VT--3B
52.20
48.30
48.60
13.70
40.70
+ Jigsaw-R.
98, 80146.66
66.00 17.770
53.20 2..20
16.80 3.10
58,
58.70f18.00
Qwen2-VI.-2B
50.90
53.60
16.90
9.60
10.25
+ Jigsaw-R1
98. 147607
65. 00 11:44u
53.50 76.00
12.30 12:70
57.35 57.35
117.10
InteruVL2.5-225
51.00
48.50
53.50
10.90
40.98
+ Jigsaw-R1
99.30 94.30
18.33
63, 20 74.20
14:70
53.40
0:110
12.90
57.20 16:22

---

Published in Transactions on Machine Learning Research (10/2025)
1.0
140
0.9
Reward
Length
0.7
M MWW
0.7
00
Qwen2.5-VL-7B
0.6
Qwen2. 3B
Owen22-V1-28
Accuracy Reward
Accuracy Reward
InternVL2 22.0-
0.6
Reward
0.4
20
800
1200
1600
2000
800
1000
Training Step
Training Stop
Training Stop
Figure The training dynamics of Jigsaw-R1. Left: Rewards of Qwen2.5-VL-3B (thinking). Middle: Rewards of Qwen2.5-VL-3B (non-thinking). Right: The completion length of various models. All curves are exponentially smoothed for visualization.
Research Question #2: How Do Jigsaw Puzzles Generalize to Downstream Tasks?
To address this question, evaluate the performance of models trained jigsaw puzzles across several downstream tasks, including CV-Bench, MMVP, SAT, and Super-CLEVR. Further details regarding these datasets can be found in Section
Jigsaw puzzles generalize. Our primary investigation (Table 2 reveals that models trained on jigsaw puzzles generally achieve improved performance on downstream tasks, indicating robust. generalization. Notably, despite being trained exclusively on the COCO dataset for jigsaw puzzles, these models successfully adapt to spatial reasoning tasks synthetic image datasets such SAT and Super- CLEVR. Nevertheless, the improvements seen in thinking models might be superficial. They learn to neglect the reasoning process (further discussed in Research Question #3), and their performance after fine-tuning remains inferior to that of non-thinking models that have not undergone fine-tuning.
Finding 2.1:
Finding 2.2:
Jigsaw puzzle configuration impacts generalization. To understand the factors influencing generalization, we analyze the performance of Qwen2.5-VIL-3B under various jigsaw puzzle configurations:
Puzzle size:
The choice of jigsaw puzzle size significantly affects downstream performance (Table 3h. For the non-thinking setting, training on larger, more challenging jigsaw puzzle size leads to better generalization. Furthermore, employing a curriculum learning approach that mixes different puzzle sizes (e.g. 3x1-+411 proves more effective than training exclusively with a single size.
Question type: Pair jigsaw puzzles result in superior gencralization on downstream tasks compared to full jigsaw puzzles (Table 4). We believe that this advantage stems from the analogy of pair jigsaw puzzles to downstream tasks (e.g. requiring models to answer multi-choice questions and explicitly asking them to reason about spatial relationships between visual elements).
Traóning dataset: As demonstrated in Table 5 aligning the training dataset with the target domain yields improved performance. For example, training directly on the SAT dataset enhances performance on SAT tasks. Given that jigsaw puzzles are label-free, it is even feasible train on the test set of SAT for further performance gains.
Takeaways #2. Training on jigsaw puzzles can induce generalization downstream tasks. The degree of generalization is affected by specific task configurations, including puzzle size, question type and training dataset.

---

Published in Transactions on Machine Learning Research (10/2025)
Table 2: Evaluation results on downstream tasks. For thinking and non-thinking of the same model, the better result underlined.
Thinking
Method
CV-Bench
MMVP
SAT
Super-CLEVR
AVG
GPT-4.1
83.69
88.66
73.70
52.00
74.52
GPT-4.1-mini
84.42
82.00
72.00
60.50
74.73
Claude 3.5 Haiku
73.38
71.33
59.30
48.00
63.00
Qweu2.5-VL-72B
82.98
76.33
71.00
72.00
75.57
Qwen2.5-VL-77
64.89
72.66
65.85
59
65.60
+ Jigsaw-R1
75.97f110s
77.00"4.34
69.15
66.00"7.00
72.03
Qwen2.5-VL-33
63.87
61.66
57.05
48.00
57.64
+ Jigsaw-R1
69.48 "0.81
65.00
61.95744990
47.00*
60.86
Qwen2-VL-2B VL-2B
51.55
63.33
45.75
55.00
53.91
+ Jigsaw- R1
59.36"7.81
61.33 11334-2.00 2:00
53. 15.440
66.00112.00
59.9616.0
LiteruVL2.5-255
56.02
54.66
47.60
15.50
43.44
+ Jigsaw-R1
60.73"
63.67 79.01
56.25 79.60
46.00 730.00000
56.66"t 6...22
Non-thinking
Method
CV-Bench
MMVP
SAT
Super-CLEVR
AVG
GPT-1.1
81.95
86.33
73.30
55.75
74.33
GPT-4.1-mini
81.46
80.66
69.90
65.50
74.38
Claude 3.5 Haíku
63.87
67.00
56.90
37.00
56.19
Owen2.5-VL-72
82.83
78.00
72.00
97.50
82.58
Qwen2.5-VT-77
79.87
78.00
69.55
92.50
79.98
+ Jigsaw-R1
80.44 00.57
77.67 0.33
69.80 (9:25
92.50
80. 10.12
Qwen2.5-VL VL-3B
70.35
66.00
65.50
76.50
69.59
+ Jigsaw-R1
73.57
3:22
70.00 14:00
65.65 15:15
83.50 7.00
73.18
Qwen2- VL-2B
64.89
66.33
61.65
72.00
66.21
+ Jigsaw- R1
67.40 22:1
66.00 0.33
64. 64 3:00
72.50 10:50
67. 67.65 67.65111:4 11:11:444
InternVL22.522
65.84
66.00
61.50
51.00
61.09
Jigsaw-R1
67.36 1.52
72.00
61.30
83.50132550
83.50
132.50
71.03 19.94
71.0319.99
Table Averaged downstream task performance of Qwen2.5-VTT-33
VT-3B when trained on different jigsaw puzzle sizes. 2x1-+3x1 (3x1-+x1)): Mixing 2x1 and 3x1 (3x1 and 4x1) jigsaw puzzles in a curriculum setting. For thinking and non-thinking of the same model, the better result is underlined.
Thinking
Baseline
2x1
3x1
4x1
2x2
2x1-+311
57.64
60.86
58.86
59.05
58.79
61.92
Non-thinking
Baseline
2x1 73.18
3x1
4x1
2x2
3x1-+41
69.59
74.95
73.03
72.68
75.29
Research Question #3: Thinking or Non-thinking?
In this section, we compare models that employ reasoning chain against those that provide an answer directly.
Finding 3.1: MLLMs can learn from jigsaw puzzles and generalize to downstream tasks with without explicit reasoning. As demonstrated by the jigsaw puzzle results (Tables 1 and 7), models can effectively learn with rule-based visual RL, whether or not an explicit reasoning chain generated. Importantly, these learned capabilities can be generalized to various downstream tasks (Table 2
10

---

Published in Transactions on Machine Learning Research (10/2025)
Table Evaluation results on downstream tasks when Qwen2.55VV-33 (non-thinking) is trained on different question types.
Question Type
CV-Bench
MMVP
SAT
CLEVR-
AVG
Full
71.76
69.67
65.20
72.65 83.50
$4.00
Pair
73.57
70.00
65.65 05.6510.45
10:45
73.18t0:53
Table 5: Evaluation results on rowmstream tasks when Qwen2.5-VI.33B VT.3BB (non-thinking) is trained on different.
datasets.
Training Dataset
CV-Bench
MMVP
SAT
Super-CLEVR
AVG 1.00
COCO train
73.57
70.00
65.65
83.50
73.18
SAT train
72.29 1:28
68. 08.00 2:00
67.00
82.00
72.32
-0.86
SAT train + SATtest test
72.46
68.33 08.33--167
67.40
81.00--24
72.29
Finding 3.2: Open-source MLLMs often benefit from direct answering, while proprietary models tend to perform better with explicit reasoning, Consistent with observations in (Li et al. 2025c Jiang
et al. 2025), our results on jigsaw puzzles (Tables and 7 and downstream tasks (Table 2 confirm that open-source models tend to achieve stronger results when prompted to output the answer directly. Indeed, even after line-tuning, models that adopt a reasoning process show weaker generalization on downstream tasks than direct-answering models that have not undergone fine-tuning. Conversely, we find that proprietary models generally demonstrate improved performance when explicit reasoning process precedes the final answer, which mirrors findings multimodal reasoning tasks (Hao et al.) 2025). It is important to note, however, that this does not necessarily mean proprietary models are inherently stronger. For example, Claude 3.5 Haiku performs comparably to Qwen2.5- 5- 5-BL on downstream tasks.
Finding 3.3: MILMs can neglect the reasoning process after fine-tuning. As illustrated in Figure (Right), the completion length of InternVT.2.5-22 significantly decreases during training. This occurs because the model increasingly circumvents step-by-step reasoning and often includes only the final answer in its thinking process (see examples in Appendix J]. Conversely, while Qwen models do present explicit reasoning steps, these steps may not consistently inform the derivation of the final answer (examples are provided in Appendix J. To quantify this, we utilized GPT-4.1 to assess the consistency between the reasoning process of Qwen2.5- 5- VL-3B and its final answer. As Figure 3 (Left)) demonstrates, although the model's final answer becomes more accurate with training, its reasoning chain becomes progressively more inconsistent.
Takeaways #3. MLLMs can learn and generalize, irrespective of whether an explicit reasoning process is included. Nevertheless, open-source MLLMs usually excel direct answering. Consequently, even when trained to include step-by-step reasoning, they may ignore the thinking process when deriving the final answer.
Research Question #4: Does the Aha Moment Emerge?
This section focuses exclusively on the reasoning chains produced by thinking models. By design, non-thinking models provide direct answer without generating an explicit reasoning process, so they are not included in this part of the analysis.
The aha moment is often associated with the emergence of complex reasoning palterns (Guo et al. 2025). While contemporary studies focusing on perception-heavs tasks like visual classification, visual grounding and spatial reasoning (Bai
Finding 4.1: Complex reasoning patterns are pro-existing in MLLMs.
ct al. 2025b Chen et al. 2025d Lai al. 2025 Li et al., (025c]] Liao et al.) 2025 Liu et al. 2025) Shen
et al. 2025 Yu 2025a Zhou et al.] 2025) typically do not observe these patterns in instruction-tunct
models, investigation into jigsaw puzzles reveals distinct phenomenon. Although the completion length
11

---

Published in Transactions on Machine Learning Research (10/2025)
4800
80
Keyword SeywordErequee Erequency
Prequency
60
80
4400
4200
56
30
bit
3800
60
3600
10
50
Consisteccy
3400
48
200
400
600
800
1.000
200
400
600
8.00
1000
2X1
3x1
2X2
Training Step
Training Step
Jigsaw Puzzle Size
Figure 3: Left: Accuracy of the final answer and consistency of the reasoning process during training. Middle: Evolution of keyword frequency throughout the training process. Right: Comparison of keyword frequency when trained on different jigsaw puzzle sizes. All results are demonstrated using Qwen2.5-VL-3B..
is not incrcasing, as illustrated in Figure [Rihht), we find that all these models, including InternVI.2.522 and also @wen2-VVL-2B-Baae (see Appendix M. exhibit complex reasoning patterns, such as verification and backtracking, even before training starts. Indeed, throughout the training process, we successfully identify all four cognitive behaviors as defined in (Gandhi et al. 2025).
Examples of the Four Cognitive Behaviors When Solving Jigsaw Puzzles
Verification: "Let me check the numbers ...'
Backtracking: "After re-evaluating the patches, 1 observe the following ..."
Subgoal Sctting: "Lot's try match the descriptions of the patches to
Backward Chaining: 'I can work backwards find the correct placement
Finding 4.2: Complex reasoning patterns evolve during fine-tuning. To monitor the evolution of these bchaviors, track the frequency of keywords indicative of backtracking and backward chaining (detailed in Appendix H As depicted in Figure 3 (Middle), the occurrence of these keywords demonstrates steady and significant increase throughout the training process. In addition to keyword matching, we also utilize GPT-4.1 to identify these behaviors in the reasoning chains. Further details are provided in Appendix H
Complex reasoning patterns emerge more frequently with harder jigsaw puzzles. To further investigate these reasoming patterns, we plot the frequency these keywords as the Qwen2.5-VI.-3B model is trained on jigsaw puzzles of varying sizes, shown in Figure 3 (Right). Our analysis reveals clear trend: the frequency of these keywords increases when the model is trained more challenging (i.e., larger) jigsaw puzzles.
Finding 4.3:
Takeaways #4: Rather than emerging abruptly, complex reasoning patterns intrinsic within MLLMs. Tasks that inherently require struct ured reasoning, such as jigsaw puzzles, readily activate these pre-existing patterns. Furthermore, they become demonstrably more prominent both throughout the training process and when MLLMs face more challenging jigsaw puzzles.
Research Question #5: SFT or RL?
This section evaluates the gencralization capabilities of SFT in comparison to RL. For these experiments, SFT data for thinking is curated via rejection sampling, while non-thinking utilizes ground-truth data. Please refer to Section for more details.
Finding 5.1: SFT exhibits wcaker generalization than RL. As demonstratod Table 0 applying SFT either the reasoning chain (thinking) or directly to ground-truth answers (non-thinking) can yield generalization. However, it is generally less effective than RI.
12

---

Published in Transactions on Machine Learning Research (10/2025)
Finding 5.2: A cold start phase with SFT preceding RL can be detrimental. Compared to single-stage RT process, a two-stage pipeline that incorporates cold start phase with SFT prior to RL can help models learn specific output formats (Guo et al. 2025). However, this is not essential in our experimental setting, as indicated by the rapid increase in rewards (Figure 2p. More importantly, we observe that this cold start phase can diminish the effectiveness of subsequent RI. optimization (Table 6
Takcaways #5. SFT typically shows weaker generalization compared to RL. Additionally, initiating the training process with a SFT cold start phase may limit the efficacy of subsequent RL optimization.
Table 6: Evaluation results of SFT and RL models on downstream tasks. For thinking and non-thinking of the same model, the better result is underlined.
Thinking
Method
CV-Bench
MMVP
SAT
Super-CLEVR CLEVR
AVG
Qwen2.5-VT-33
63.87
61.66
57.05
48.00
57.64
Jigsaw-RI
69.4819.61
65.0013.34
61.9514.90
47.00" 1:00
60.8673.22
+ SFT
66.41 12.54
64.0012.34
58.85 58.8511.80 11.00
42.00 42.00--6.00 --600
57.81 10.17
+ SFT Jigsaw-R1
15.99
63.6612.00 12.00
60.05 13.00
43.00 43.00--5.00 -0.00
58.91 11:27
Non- Mon-thinking
Method
CV-Bench
MMVP
SAT
Super-CLEVR
AVG
Qwen2.5-V2
VL-3B
70.35
66.00
65.50
76.50
69.59
+ Jigsaw-R1
73.57
70.00
85.65
73.50
73.1813.0
+ SFT
71.27
67.73
62.20
-3.30
76.75
69. $9.48 -0111
+ SFT + Jigsaw-RI
71.01 10.69
66.67
10.67
62.00 62.00--3.50 -3.50
80.00 13.50
69.9210.u 10.33
Limitations and Future Work
Skill alignment. The efficacy of jigsaw puzzles is contingent upon skill alignment. The primary skill cultivated is spatial reasoning, which is rcadily transferable to applications where the spatial relationship between objects is critical, such the downstream lasks in our experiments. In contrast, for tasks less reliant on this visual ability (e.g. mathematical reasoming), the training may yield diminishing returns. Therefore, we recommend integrating jigsaw puzzles with other training methods for broador gains rather than using them in isolation. Determining whether to always include jigsaw puzzles and similar pretext tasks (e.g. image rotation, as discussed Appendix C in training data requires careful, data-centric ablation studies (Ti et al., 2025d), which we leave for future work. Encouragingly, recent studies have already shown that incorporating jigsaw puzzles into training can enhance visual reasoning capabilities of MLLMs (Li et al. 2025a Wu et al. 2025
Zeng et al. 2025)-
Data contamination. It is important to consider our findings in the context of potential data contamination, a well-known challenge in the evaluation MLLMs. Because the models are pre-trained on vast and opaque web-scale datasets, their training data likely overlaps with our evaluation benchmarks. Consequently, the development of evaluation protocols that are robust against such contamination represents a critical and unresolved challenge for the research community.
Visual reasoning models. Recent advancements from OpenAl, particularly the o3 and o4-mini mod- cls [OpenA] 2025b), have shown significant promise in rcasoning with images for enhanced perception. While our work does not incorporate these visual reasoning models [Li et al.) 2025b Liu el al. 2025g Qi el. al. 2025 Su et al. 2025 Wang et al.) 2025c), we believe jigsaw puzzles are an ideal candidate for exploring based visual RL in this context due to their inherent reliance on imago-based reasoning. For an early exploration, we conduct small-scale experiments using the ChatGPT console, where these models are equipped with tool-use capabilities. Our preliminary experiments indicate that OpenAT o3 can effectively solve 2x2 jigsaw puzzles,
13

---

Published in Transactions on Machine Learning Research (10/2025)
substantially outperforming other models considered in this paper. However, it still faces challenges with more complex puzzles (e.g. 3x3), highlights areas for further investigation.
Multimodal generative models. Our study does not consider models capable of both understanding and
generating multimodal content (Chen et al. 2025c ej Hurst et al. 2024 Team 2024 Wu et al. 2024 Xie
et al. 2025a promising future research direction involves integrating jigsaw puzzles with these advanced models and let them generate their own inputs. This could reduce dependence on external datasets like
COCO and create an autonomous environment to learn from experience (Silver & Sutton 2025
Test-time training. Jigsaw puzzles inherently provide readily available annotations, making them suitable for direct training on the test set during test time. We have demonstrated that aligning the training dataset
with the target domain can yield enhanced performance. Therefore, exploring the use of jigsaw puzzles as a
technique for test-time training [Akyurek et al. 2024 Behrouz et al. 2024 Zhu et al. 2024 Zuo et al. 2025 presents an interesting avenuc for futurr work.
Other pretext tasks. While this work primarily focuses on jigsaw puzzles the pretext task, numerous
alternatives exist and worth exploration [Gidaris et al.) 2018 Gui et al. 2024). As preliminary step in this direction, we include study on image rotation in Appendix
Besides, future research could extend our
approach to pretext tasks in other modalities, including text Lan et al. 2020 video (Ahsan et al. 2019
Kim et al. 2019; Wang et al. 2020 andio (Carr et al. 2021 point clouds (Poursaeed et al. 2020 and
tabular data (Lee et al. 2024
Other RL algorithms. We exclusively employ GRPO in our current experiments, leaving other RL algorithms such PPO (Schulman al. 2017). DPO (Rafailov et al. 2023) and Reinforce+++ (Hu 2025 unexplored. Furthermore, investigating recent advancements and variations of GRPO, including DAPO Yu et al. 2025b), Dr. GRPO (Liu et al.) 2025c), GPG (Chu et al. 2025 and NoisyRollout (Liu et al. could offer valuable insights and potential performance gains.
2025c),
Acknowledgements
We acknowledge support from the Research Foundation Flanders (FWO) through project numbers G0A1319N and S001421N, and funding from the Flemish Government under the Onderzoeksprogrammnn Artificiële Intelligentie (AI) Vlaanderen programme.
We acknowledge LUMI-BE for awarding this project access to the LUMI supercomputer, owned by the EuroHPC Joint Undertaking, hosted by CSC (Finland) and the LUMI consortium through LUMI-BE Regular Access call.
14

---

Published in Transactions on Machine Learning Research (10/2025)
References
Shivam Agarwal, Zimin Zhang, Lifan Yuan, Jiawei Han, and Hao Peng. The unreasonable effectiveness of entropy minimization in Ilm reasoning. arXiv, 2025.
Unaiza Ahsan, Rishi Madhok, and Irfan Essa. Video jigsaw: Unsupervised learning of spatiotemporal context for video action recognition. WACV, 2019.
Ekin Akyurek, Mehul Damani, Adam Zweiger, Linlu Qiu, Han Guo, Jyothish Pari, Yoon Kim, and Jacob
Andreas. The surprising effectiveness of test-time training for few-shot learning. arXiv, 2024.
Anthropic. Introducing computer use, new claude 3.5 sonnet, and claude 3.5 haiku, 2024. https
//www. wtt.:aaattrrrppccccooooo/reeeesa cotp:////cco/////////e// hatee//---mmmeess-aae and -nd-tooeeee---uesee
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2.5-vl technical report. arXiv, 2025a.
Sule Bai, Mingxing Li, Yong Liu, Jing Tang, Haoji Zhang, Lei Sun, Xiangxiang Chu, and Yansong Tang. Univg-rl: Reasoning guided universal visual grounding with reinforcement learning. arXiv, 2025b.
Ali Behrouz, Peilin Zhong, and Vahab Mirrokni. Titans: Learning to memorize at test time. arXiv, 2024.
Fabio M Carlucci Antonio D'Innocente, Silvia Bucci, Barbara Caputo, and Tatiana Tommasi. Domain generalization by solving jigsaw puzzles. CVPR, 2019.
Andrew N Carr, Quentin Berthet, Mathieu Blondel, Olivier Teboul, and Neil Zeghidour. Selfssueeviisdd learning of audio reprosentations from permutations with differentiable ranking. SPL, 2021.
Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. Sft or rI? an carly investigation into training r1-like reasoning large vision-language models. arXiv, 2025a.
Jiangjie Chen, Qianyu He, Siyu Yuan, Aili Chen, Zhicheng Cai, Weinan Dai, Hongli Yu, Qiying Yu, Xuefeng Li, Jiaze Chen, Hao Zhou, and Mingxuan Wang. Enigmata: Scaling logical reasoning in large language models with synthetic verifiable puzzles. arxiv, 2025b.
Jiuhai Chen, Zhiyang Xu, Xichen Pan, Yushi Hu, Can Qin, Tom Goldstein, Lifu Huang, Tianyi Zhou, Saining Xie, Silvio Savarese, Le Xue, Caiming Xiong, and Ran Xu. Blip3-o: A family of fully open unified multinodal models-architecctur, training and dataset. arXiv, 2025c.
Liang Chen, Lei Li, Haozhe Zhao, Yifan Song, and Vinci. Ri-v: Reinforcing super generalization ability in vision-language models with less than $3, 2025d. https://github wwt.://eeeeeego/////eeeeeeeeeeee
Xiaokang Chen, Zhiyu Wu, Xingehao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus-pro: Unified multimodal understanding and generation with data and model scaling. arXiv, 2025c.
Yanda Chen, Joe Benton, Aush Radhakrishnan, Jonathan Uesato Carson Denison, John Schulman, Arushi Somani, Peter Hase, Misha Wagner Fabien Roger Vlad Mikulik, Sam Bowman, Jan Leike Jared Kaplan, et al. Reasoning models don't always say what they think. arXiv, 2025f.
Yingyi Chen, Xi Shen, Yahui Tin, Qinghua Tao, and Johan AK Suykens. Jigsaw-vit: Learning jigsaw puzzles in vision transformer. PRL, 2023.
Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv, 2024.
Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Yantao Li, Jianbing Zhang, and Zhiyong Wu. Seeclick: Harnessing gui grounding for advanced visual gui agents. ACL, 2024.
Xiangxiang Chu, Hailang Huang, Xiao Zhang, Fei Wei, and Yong Wang. Gpg: A simple and strong reinforcement learning baseline for model reasoning. arxiv, 2025.
15

---

Published in Transactions on Machine Learning Research (10/2025)
Huilin Deng, Ding Zou, Rui Ma, Hongchen Luo, Yang Cao, and Yu Kang. Boosting the generalization and reasoning of vision language models with curriculum reinforcement learning, arXiv, 2025.
Jía Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. CVPR, 2009.
Carl Doersch, Abhinav Gupta, and Alexei A Efros. Unsupervised visual representation learning by context. prediction. ICCV, 2015.
Ruoyi Du, Dongliang Chang, Ayan Kumar Bhumia, Jiyang Xie, Zhanyu Ma, Yi-Zhe Song, and Jun Guo. Fine-grained visual classification via progressive multi-gramularity training of jigsaw patches. ECCV, 2020.
Yichen Feng, Zhangchen Xu, Fengqing Jiang, Yuetai Li, Bhaskar Ramasubramanian, Luyao Niu, Bill Yuchen Lin, and Radha Poovendran. Visualsphinx: Large-scale synthotic vision logic puzzles for rl. arXiv, 2025.
Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D. Goodman. Cognitive behaviors that enable self-inpooiing reasoners, or, four habits of highly effective stars. arXiv, 2025.
Leo Gao, John Schulman, and Jacob Hilton. Scaling laws for reward model overoptimization. ICML, 2023.
Zilian Gao, Lynx Chen, Joey Zhou, and Bryan Dai. One-shot entropy minimization. arxie, 2025.
Spyros Gidaris, Praveer Singh, and Nikos Komodakis. Unsupervised representation learning by predicting image rotations. ICLR, 2018.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, A Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXie, 2024.
Jie Gui, Tuo Chen, Jing Zhang, Qiong Cao, Zhenan Sun, Hao Luo, and Dacheng Tao. A survey on self-supervised learning: Algorithms, applications, and future trends. TPAMI, 2024.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in lims via reinforcement learning. Xńv, 2025.
Yunzhuo Hao, Jiawei Gu, Huichen Will Wang, Linjie Li, Zhengyuan Yang, Lijuan Wang, and Yu Cheng. Can milms reason in multimodality? emma: An enhanced multimodal reasoning benchmark. ICML, 2025.
Michael Hassid, Gabriel Synnaeve, Yossi Adi, and Roy Schwartz. Don't overthink it. preferring shorter thinking chains for improved Ilm reasoning. arXiv, 2025.
Jian Hu. Reinforco+ A simple and efficient approach for aligning large language models. arxi 2025.
Zhiyuan Hu, Yibo Wang, Hanze Dong, Yuhui Xu, Amrita Saha, Caiming Xiong, Bryan Hooi, and Junnan Li. Beyond 'aha!': Toward systematic meta-abilities alignment in large reasoning models. arXiv, 2025.
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-40 system card. arXiv, 2024.
Dongzhi Jiang, Renrui Zhang, Ziyu Guo, Yanwei Li, Yu Qi, Xinyan Chen, Liuhui Wang, Jianhan Jin, Claire Guo, Shen Yan, et al. Mme-cot: Benchmarking thoough in large multimodal models for reasoning quality, robustness, and efficiency. ICML, 2025.
Dahun Kim, Donghyeon Cho, Donggeun Yoo, and In So Kweon. Learning image representations by completing damaged jigsaw puzzles. WACV, 2018.
Dahun Kim, Donghyeon Cho, and In So Kweon. Self-supervised video representation learning with space-time cubic puzzles. AAAI, 2019.
Yuxiang Lai, Jike Zhong, Ming Li, Shitian Zhao, and Xiaofeng Yang. Med-r1: Reinforcement learning for generalizable medical reasoning in vision-language models. arXiv, 2025.
16

---

Published in Transactions on Machine Learning Research (10/2025)
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. Albert: A lite bert for self-supervised learning of language representations. ICLR, 2020.
Kyungeun Lee, Ye Seul Sim, Hye-Seung Cho, Moonjung Eo, Suhee Yoon, Sanghyu Yoon, and Woohyung Lim. Binning as a pretext task: Improving self-supervised learning in tabular domains. ICML, 2024.
Seongyun Lee, Seungone Kim, Minju Seo, Yongrae Jo, Dongyoung Go, Hyeonbin Hwang, Jinho Park, Xiang Yue, Sean Welleck, Graham Neubig, Moontae Lee, and Minjoon Seo. The cot encyclopedia: Analyzing, predicting, and controlling how a reasoning model will think. arXiv, 2025.
Ang Li, Charles Wang, Kaiyu Yue, Zikui Cai, Ollie Liu, Deqing Fu, Peng Guo, Wang Bill Zhu, Vatsal Sharan, Robin Jia, Willie Neiswanger, Furong Huang, Tom Goldstein, and Micah Goldblum. Zebra-cot: A dataset. for interleaved vision language reasoning. arXiv, 2025a.
Chengzu Li, Wenshan Wu, Huanyu Zhang, Yan Xia, Shaoguang Mao, Li Dong, Ivan Vulić, and Furu Wei. Imagine while reasoning in space: Multimodal of arXiv, 2025b.
Ming Li, Jike Zhong, Shitian Zhao, Yuxiang Lai, and Kaipeng Zhang. Think or not think: A study of explicit thinking in rulc-based visual reinforcement fine-tuning. arXiv, 2025c.
Yu Li, Zhuoshi Pan, Honglin Lin, Mengyuan Sun, Conghui He, and Lijun Wu. Can one domain help others? a data-centric study on multi-domain reasoning via reinforcement learning. arXiv, 2025d.
Zhuowan Li, Xingrui Wang, Elias Stengel-Eskin, Adam Kortylewski, Wufei Ma, Benjamin Van Durme, and Alan L Yuille. Super-clevr: virtual benchmark to diagnose domain robustness in visual reasoning. CVPR, 2023.
Zhenyi Liao, Qingsong Xie, Yanhao Zhang, Zijian Kong, Haonan Lu, Zhenyu Yang, and Zhijie Deng. Improved visual-spatial rcasoning via r1-zcro-like training. arXiv, 2025.
Hunter Lightman, Vincet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Nya Sutskever, and Karl Cobbe. Let's verify step by step. ICLR, 2024.
Tsung-Y Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. ECCV, 2014.
Junteng Liu, Yuanxiang Fan, Zhuo Jiang, Han Ding, Yongyi Hu, Chi Zhang, Yiqi Shi, Shitong Weng, Aili Chen, Shiqi Chen, Yunan Huang, Mozhi Zhang, Pengyu Zhao, Junjie Yan, and Junxian He. Synlogic: Synthesizing verifiable reasoning data scale for learning logical reasoning and beyond. arXiv, 2025a.
Mingjie Liu, Shizhe Diao, Ximing Lu, Jian Hu, Xin Dong, Yejin Choi, Jan Kautz, and Yi Dong. Prorl: Prolonged reinforcement learning expands reasoning boundaries in large language models. arx 2025b.
Xiangyan Liu, Jinjic Ni, Zijian Wu, Chao Du, Longxu Dou, Haonan Wang, Tianyu Pang, and Michael Qizhe Shieh. Noisyrollout: Reinforcing visual reasoning with data augmentation. arXiv, 2025c.
Yuqi Liu, Tianyuan Qu, Zhisheng Zhong, Bohao Peng, Shu Liu, Bei Yu, and Jiaya Jia. Visionreasoner: Unified visual perception and reasoning via reinforcement learning. arXiv, 2025d.
Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Leo, and Min Lin. Understanding rl-zero-like training: A critical perspective. arxiv, 2025e.
Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. Visual-rft: Visual reinforcement finc-tuning. arXis, 2025f.
Ziyu Liu, Yuhang Zang, Yushan Zou, Zijian Liang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. Visual agentic reinforcement fine-tuning. Xrn, 2025g.
17

---

Published in Transactions on Machine Learning Research (10/2025)
Sara Vera Marjanovié, Arkil Patel, Vaibhav Adlakha, Milad Aghajohari, Parishad BehnamGhader, Mchar Bhatia, Aditi Khandelwal, Austin Kraft, Benno Krojer, Xing Han Tå, Nicholas Meade, Dongchan Shin, Amirhossein Kazemnejad, Gaurav Kamath, Marius Mosbach, Karolina Staúczak, and Siva Reddy. Deepseek- rl thoughtology: Let's think about Imm reasoning. arXiv, 2025.
Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanleng Lu, Daocheng Fu, Tiancheng Han, Botian Shi, Wenhai Wang, Junjun He, et al. Mm-eureka: Exploring the frontiers of multimodal reasoning with rulc-based reinforcement learning. arXiu, 2025.
Mchdi Noroozi and Paolo Favaro. Unsupervised learning of visual representations by solving jigsaw puzzles. ECCV, 2016.
OpenAl. Introducing gpt-4.1 in the api, 2025a. 4ptp
htp///idexx/gg/
OpenAL. Introducing openai o3 and o4-mini, 2025b.
https://openai.com com/inder/
introducing-o2 and-o4-mini
Yingzhe Peng, Gongrui Zhang, Miaosen Zhang, Zhiyuan You, Jie Liu, Qipeng Zhu, Kai Yang, Xingzhong Xu, Xin Geng, and Xu Yang. Lmm-rl: Empowering 3b Imms with strong reasoning abilitics through two-stage rule-based rl. Xiv, 2025.
Omid Poursaeed, Tianxing Jiang, Han Qiao, Nayun Xu, and Vladimir G Kim. Self-supervised learning of point clouds via orientation estimation. 3DV, 2020.
Mihir Prabhudesai, Lili Chen, Alex Tppoliti, Katerina Fragkiadaki, Hao Liu, and Deepak Pathak. Maximizing confidence alone improves reasoning. arv, 2025.
Ji Qi, Ming Ding, Weihan Wang, Yushi Bai, Qingsong Lv, Wenyi Hong, Bin Xu, Lei Hou, Juanzi Li, Yuxiao Dong, and Jie Tang. Cogcom: A visual language model with chain-of-maniitlaaiin reasoning. ICLR, 2025.
Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report. arxiv, 2025.
Rafael Rafailov, Archit Sharma, Erie Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. NeurIPS, 2023.
Arijit Ray, Jiafei Duan, Reuben Tan, Dina Bashkirova, Rose Hendrix, Kiana Ehsani, Aniruddha Kembhavi, Bryan A Plummer, Ranjay Krishna, Kuo-Hao Zeng, et al. Sat Spatial aptitude training for multinodal language models. arXiv, 2024.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arxiv, 2017.
Sheikh Shafayat, Fahim Tajwar, Ruslan Salakhutdinov, Jeff Schneider, and Andrea Zanette. Can large reasoning models self-train? arXiv, 2025.
Rulin Shao, Shuyue Stella Li, Rui Xin, Scott Geng, Yiping Wang, Sewoong Oh, Simon Shaolei Du, Nathan Lambert, Sewon Min, Ranjay Krishna, Yulia Tsvetkov, Hannaneh Hajishirzi, Pang Wei Koh, and Luke Zettlemoyer. Spurious rewards: Rethinking training signals in rlvr. arXiv, 2025.
Zhihong Shao, Poiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingehuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. Xrv, 2024.
18

---

Published in Transactions on Machine Learning Research (10/2025)
Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, et al. Vlm-rl: A stable and generalizable rl-style large vision-language model. arXiv, 2025.
David Silver and Richard S Sutton. Welcome to the era of experience, 2025. http:://storrggegoggleee. http:///owao.aoaoooooooooooooo/o
pdf
Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling Ilm test-time compute optimally can be more effective than scaling model paramcters. ICLR, 2025.
Zafir Stojanovski, Oliver Stanley, Joe Sharratt, Richard Jones, Abdulhakeem Adefioye, Jean Kaddour, and Andreas Köpf. Reasoning gym: Reasoning environments for reinforcement learning with verifiable rewards. arXiu, 2025.
Zhaochen Su, Linjie Li, Mingyang Song, Yunzhuo Hao, Zhengyuan Yang, Jun Zhang, Guanjie Chen, Jiawei Gu, Juntao Li, Xiaoye Qu, and Yu Cheng. Openthinking: Learning to think with images via visual tool reinforcement learning. arxi 2025.
Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models. arx, 2024.
Jingqi Tong, Jixin Tang, Hangcheng Li, Yurong Mou, Ming Zhang, Jun Zhao, Yanbo Wen, Fan Song, Jiahao Zhan, Yuyang Lu, et al. Code2logic: Game-code-driven data synthesis for enhancing vlms general reasoning. arv, 2025.
Peter Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Adithya Jairam Vedagiri IYER, Sai Charitha Akula, Shushong Yang, Jihan Yang, Manoj Middepogu, Ziteng Wang, et al. Cambrian-1: A fully open, vision-centric exploration of multimodal llms. NewrIPS, 2024a.
Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xic. Eyes wide shut? exploring the visual shortcomings of multimodal 1lms. CVPR, 2024b.
Cheng Wang, Yue Liu, Baolong Li, Duzhen Zhang, Zhongzhi Li, and Junfeng Fang. Safety in large reasoning models: A survey. arXiv, 2025a.
Haozhe Wang, Chao Qu, Zuming Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. Vl-rethinker: Incentivizing self-reflection of vision-langue languaee models with reinforcement learning. arXiv, 2025b.
Jiangliu Wang, Jianbo Jiao, and Yun-Hur Liu. Self-supervised video representation learning by pace prediction. RCCV, 2020.
Peiyi Wang, Lei Li, Zhihong Shao, R. X. Xu, Damai Dai, Yifei Li, Deli Chen, Y. Wu, and Zhifang Sui. Math-shepherd: Verify and reinforce iims step-by-step without human annotations. ACL, 2024a.
Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-land asioo-languuae model's perception of the world at any resolution. Xrxi 2024b.
Yaoting Wang, Shengqiong Wu, Yuecheng Zhang, Shuicheng Yan, Ziwei Liu, Jiebo Luo, and Hao Fei. Multimodal chain-of-thought reasoning: comprehensive survey. arx, 2025c.
Lai Wei, Yuting Li, Chen Wang, Yue Wang, Linghe Kong, Weiran Huang, and Lichao Sun. Unsupervised post-training for multi-modal Ilm reasoning via grpo. arxi 2025.
Wikipedia. Jigsaw puzzle, 2025.
Chengyue Wu, Xiaokang Chen, Zhiyu Wu, Yiyang Ma, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, Chong Ruan, et al. Janus: Decoupling visual encoding for unified multimodal understanding and generation. arv 2024.
Penghao Wu, Yushan Zhang, Haiwen Diao, Bo Li, Lewei Lu, and Ziwei Liu. Visual jigsaw post-training improves milms. ar Nau, 2025.
19

---

Published in Transactions on Machine Learning Research (10/2025)
Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. ICLR, 2025a.
Tian Xie, Zitian Gao, Qingnan Ren, Haoming Luo, Yuqian Hong, Bryan Dai, Joey Zhou, Kai Qiu, Zhirong Wu, and Chong Luo. Logic-rl: Unleashing Ilm reasoning with rule-based reinforcement learning. ar arXiv, 2025b.
Yuxi Xie, Anirudh Goyal, Wenyue Zheng, Min-Yen Kan, Timothy P. Lillicrap, Kenji Kawaguchi, and Michael Shieh. Monte carlo tree search boosts reasoning via iterative preference learning. NeurIPS Workshop, 2024.
Huajian Xin, Z.Z. Ren, Junxiao Song, Zhihong Shao, Wanjia Zhao, Haocheng Wang, Bo Liu, Liyue Zhang, Xuan Lu, Qiushi Du, Wenjun Gao, Haowei Zhang, Qihao Zhu, Dejian Yang, Zhibin Gou, Z.F. Wu, Fuli Luo, and Chong Ruan. Deepsock-prover-v/ and monte-carlo tree search. ICLR, 2025a.
Harnessing proof assistant feedback for reinforcement learning
Rihui Xin, Han Liu, Zecheng Wang, Yupeng Zhang, Dianbo Sui, Xiaolin Hu, and Bingning Wang. Surrogate signals from format and length: Reinforcement learning for solving mathematical problems without ground truth answers. arxiv, 2025b.
Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, et al. Rl-onevision: Advancing generalized multimodal reasoning through cross-modal formalization. arXiv, 2025.
En Yu, Kangheng Lin, Liang Zhao, Jisheng Yin, Yana Wei, Yuang Peng, Haoran Wei, Jianjian Sun, Chunrui Han, Zheng Ge, et al. Perception-rl: Pioneering perception policy with reinforeement learning. arXiv, 2025a.
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. Dapo: An open-source Ilm reinforcement learning system at scale. ar 2025b.
Yu Zeng, Wenxuan Huang, Shiting Huang, Xikun Bao, Yukun Qi, Yiming Zhao, Qiuchen Wang, Lin Chen, Zehui Chen, Huaian Chen, Wanli Ouyang, and Feng Zhao. Agentic jigsaw interaction learning for enhancing visual perception and reasoning in vision-languaag models. arXiv, 2025.
Qingyang Zhang, Haitao Wu, Changqing Zhang, Peilin Zhao, and Yatao Bian. Right question already half the answer: Fully unsupervised 11m reasoning incentivization. arXiu, 2025.
Andrew Zhao, Yiran Wu, Yang Yue, Tong Wu, Quentin Xu, Matthieu Lin, Shenzhi Wang, Qingyun Wu, Zilong Zheng, and Gao Huang. Absolute zero: Reinforced self-play reasoning with zero data. arXiv, 2025a.
Xuandong Zhao, Zhewei Kang, Aosong Feng, Sergey Levine, and Dawn Song. Learning to reason without. external rewards. arxie, 2025b.
Hengguang Zhou, Xirui Li, Ruochen Wang, Minhao Cheng, Tianyi Zhou, and Cho-Jui Hsich. R1-zero's 'aha moment' in visual reasoning on 2b non-sft model. Xrv, 2025.
Junyi Zhu, Shuochen Liu, Yu Yu, Bo Tang, Yibo Yan, Zhiyu Li, Feiyu Xiong, Tong Xu, and Matthew B Blaschko. Fastmem: fast memorization of prompt improves context awareness of large language models. EMNLP Findings, 2024.
Yuxin Zuo, Kaiyan Zhang, Shang Qu, Li Sheng, Xuekai Zhu, Biqing Qi, Youbang Sun, Ganqu Cui, Ning Ding, and Bowen Zhou. Ttrl: Test-time reinforcement learning. arXiv, 2025.
20

---

Published in Transactions on Machine Learning Research (10/2025)
Appendices
A Other Jigsaw Puzzles
22
A.1 Full Jigsaw Puzzles
22
A.2 Box Jigsaw Puzzles
22
B Other Reward Designs
24
B.1 Binary Accuracy Reward
24
B.2 Format Reward Weight
C Image Rotation
25
Task Design
25
C22 Task Performance
25
33 Goneralization
25
D Training Costs
27
E Training Steps
28
F Qwen2- VL-2B- Base
29
G Few-shot Learning
30
H Reasoning Patterns
31
Keywords
31
H.2 Content Analysis
31
Prompts
32
Full Jigsaw Puzzles
32
I2
Pair Jigsaw Puzzlos
33
3
Box Jigsaw Puwzles
35
14
Image Rotation
36
Examples
37
J.1
Qwen2.5-VL-33
37
J.2
Intern InternVL..5.2 L2.5-BB Examples
39
21

---

Published in Transactions on Machine Learning Research (10/2025)
A Other Jigsaw Puzzles
A.1 Full Jigsaw Puzzles
Table 7: Evaluation results on full jigsaw puzzles with different sizes. For thinking and non-thinking of the same model, the better result is underlined. Claude 3.5 Haiku fails to output answers in the required grid format.
Thinking
Method
2x1
AVG
Random
50.00
16.67
4.17
4.17
18.75
GPT-4.1
79.00
27.30
8.20
6.40
30.23
GPT-1.1-mini
60.80
26.60
8.40
6.20
25.50
Claude 3.5 Haiku
69.40
19.00
5.30
6.50
25.05
Qwen2.5-VL-72B
53.40
20.40
5.10
5.60
21.13
Qweu2.5-VL-7B
25.60
15.80
3.10
4.20
12.18
+ Jigsaw-R1
97 97.30
31.20t15:40
31.20 115.40
8,20
7.80
36.12 123.94
Qwen2.5-VV VL-33B
45.90
11.30
2.60
3.40
15.80
+ Jigsaw-he R1
97. 97.20151:30
31. 31.401200
7.30
6.50
35.60
Qwen2- VL-2B
46.90
5.30
0.60
4.00
14.20
Jigsaw-RL RR1
98.00191.10
30.60145.30
8.40
1.80
35.15121.22
InternVL22.5221
16.00
8.60
2.90
2.70
7.55
+ Jigsaw-Rt
19:30
30.10
30.10 121:50
t.50
0.00
2270
33.48 125.93 33.48125:93
Non- Non-thinking
Method
2x1
3x1
4x1
2x2
AVG
Random
50.00
16.67
4.17
4.17
18.75
GPT-4.1
68.50
17.30
7.00
7.30
25.03
GPT--41-mini
49.50
18.80
3.80
5.60
Claude 3.5 Haiku!
0.00
0.00
0.00
0.00
0.00
Qwen2.5-VLL-723
91.70
26.40
9.20
8.00
Qwen2.55VL-77
30.10
17.40
1.50
0.50
12.38
Jigsaw-R1
99.
99.20169.10
30.40
8.6017.30
9.00
36.80
12..42
Q 2.ee2.5-VV-33
51.60
16.70
4.40
3.70
19.10
+ Jigsaw-Rt
98.80147:20
29.70
9.1014.70
7.70
36.32 117/22
Qwen2-VT. VL.2B
11.40
2.00
0.20
0.00
3.40
+ Jigsaw- R1
99.00ts7.66
33010
7.30
3.50
35.471.9.07
InteruVVL2.5-2BB
20.90
11.30
3.40
2.80
9.60
+ Jigsaw-R R1
17.20178.30 30
31.40
120:10
7.1013:70
7.30
36.25 126.65
126.53
A.2
Box Jigsaw Puzzles
Unlike other jigsaw puzzle tasks (i.c., full and pair), this task evaluates MLLM's ability both reason about spatial relationship between image patches and to visually ground its understanding by identifying the correct bounding box.
The process bogins by dividing an input image into mxn grid of regions. Each region is assigned a unique position index. From each of these ma regions, a patch of equal size is randomly selected, resulting in mn equally-sized patches. Next, a target position index, (where 1 1 i < mn), randomly chosen. All mn patches are then shuffled. A key constraint during shuffling is that the patch originally from position must be relocated to a different position. The MLLM's task is provide the bounding box coordinates for the patch that originally belonged in position f. Further details the prompts used can be found in Appendix [
22

---

Published in Transactions on Machine Learning Research (10/2025)
The accuracy reward is measured by the Intersection over Union (IoU) between the predicted and ground truth bounding boxes. The format reward, consistent with other jigsaw puzzle tasks, requires that the final answer be extractable in the prescribed format.
As shown in Table training MLLMs on this task can promote generalization to ScreenSpot (Cheng et al, 2024 a downstream task requiring visual grounding within a Graphical User Interface (GUT) environment.
Table 8: Evaluation results on ScreenSpot. For thinking and non-thinking of the same model, the better result is underlined.
Thinking
Method
ScreenSpot
Qwen2.5- VL-3B
57.38
+ Jigsaw-R1
72.48 72.48116.10
Non-t thinking
Method
ScreenSpot
Qwen2.5 5- VL-3B
79.08
+ Jigsaw- R1
8 81.05 1.97
23

---

Published in Transactions on Machine Learning Research (10/2025)
B Other Reward Designs
B.1 Binary Accuracy Reward
For full questions with fino-grained reward signals, we default to using fractional rewards. As an alternative, we Lest binary rewards, similar to pair questions. To compare these two designs, we evaluate the generalization performance of Qwen2.5-VE. VL-BB (non-thinking) on downstream tasks. As shown in Table 9 the fractional reward yiclds slightly better results on larger jigsaw puzzles.
Table 9: Averaged downstream task performance of Qwen2.3 VL-3BB (non-thinking) when trained on full questions with different accuracy rewards.
Accuracy Reward
2x1
3x1
4x1
2x2
Binary
72.65
72.70
72.16
71.55
Fractional
72.65
72.7710:07
72.7710:61
71.94 19:99
B.2 Format Reward Weight
We assign weights of 1.0 and 0.5 to the accuracy and format rewards, respectively. We also perform an ablation study and explore alternative values such as and for the format reward. The downstream task performance of Qwen2.5-VI-33B (non-thinking) is measured for each configuration. As presented in Table I0 the choice of the format reward weight has only minor effect the final performance.
Table 10:
Averagod downstream task performance of Qwen2.5-VLL-3B (non-thinking) when trained with different format reward weights.
Weight
1.0
Performance
72.97
73.18
73.06
24

---

Published in Transactions on Machine Learning Research (10/2025)
C
Image Rotation
C.1
Task Design
As an alternative pretext task, we also conduct a preliminary study with image rotation. In this setup, an input image is randomly rotated clockwise by one of a predefined set of angles. Given & size parameter nt the possible rotation angles are evenly distributed within a 360 range. We frame this as an n-choice question where the model needs identify the appliod rotation angle. For instance, if n=4, the possible choices are 0% 90", 180", and 270". As with jigsaw puzzles, the difficulty can be adjusted by varying the value of n. The specific prompts are available in Appendix
Consistent with our jigsaw puzzle experiments, we investigate both thinking and non-thinking. Besides, the reward design and training hyperparameters are identical to those used for the jigsaw puzzles. For example, a correct choice receives an accuracy reward of 1 (0 otherwise), and a response from which a letter choice can extracted receives a format reward 0.5 (0 otherwise).
C.2 Task Performance
We first train Qwen2.5-VL-3B on the rotation task with 4 choices and then evaluate its performance across a range of sizes. The results are presented in Table [I Similar to our findings with jigsaw puzzles, the model initially struggles but masters the task effectively after fine-tuning. Moreover, the model demonstrates the ability to generalize to more complex configurations (i.e., larger rotation sizes) not encountered during training.
Table 11: Evaluation results on image rotation with different sizes. For thinking and non-thinking of the same model, the better result underlined.
Thinking
Method
AVG 11.50
Random
25.00
12.50
8.33
6.67
5.00
Qwen2.5-VL-31 VL-3B
29.80
15.10
9.10
6.50
5.30
13.16
+ Rotation-Ri R1
91.80 76.000
39. 39.10
10
27.90
28.90718.80
18..0
10.20 3.70
12.20
36.23
123.07
Non- Thinking
Method
8
12
15
20
AVG
Rondom
25.00
12.50
8.33
6.67
5.00
11.50
Qwen2.5-VI-22
30.60
16.80
8.00
6.40
5.50
13.45
Rotation-R1
764.40
191:40
t2.60
12460
12.50
16
18.20
"12.0"
722.0
11.30
"27.8
C.3 Generalization
We train the model using various rotation sizes and assess its generalization capabilities on downstream tasks, with the results shown in Table I2 Again, consistent with the jigsaw puzzle experiments, the model trained on image rotation generalizes to unseen (asks, and the degree generalization varies based on rolation sizes. Furthermore, Qwen2.5-Ve VL-3BB learns and generalizes both with and without explicit reasoning, though it tends to benefit from direct answering.
A comparison between the generalization performance of jigsaw puzzles (Table 3) and rotation tasks (Table 12 reveals that the former yields more effective generalization. However, we do not claim that jigsaw puzzles are an inherently superior pretext task. Our primary focus on jigsaw puzzles in this paper is to maintain a controlled experimental setup for studying various aspects of rule- based visual RL. Nevertheless, we believe that more effective generalization could be achieved by integrating multiple visual pretext tasks, a strategy analogous to contemporary practices in linguistic domains where various logic puzzles are jointly employed to enhance reasoning capabilities (Chen et al. 2025b
Liu et al. 2025a b).
25

---

Published in Transactions on Machine Learning Research (10/2025)
Table 12: Averaged downstream task performance of Qwen2.5-VI-3B when trained on different rotation sizes. For thinking and non-thinking of the same model, the better result underlined.
Thinking
Baseline
4
8
12
15
20
57.64
64.19
62.87
63.58
62.77
63.38
Non-thinking
Baseline
8 71.33
12
15
20 72.00
69.59
71.37
72.55
72.74
26

---

Published in Transactions on Machine Learning Research (10/2025)
D
Training Costs
On-policy RL algorithms like GRPO are computationally demanding, as they require generating multiple roll-outs during training. This cost is siguificantly exacerbated when the model must also produce an explicit reasoning process. We measure these training costs on a cluster of eight 64GB AMD MT250X GPUs. As detailed in Table 13 RL is substantially more expensive than SFT, especially when the model required to articulate its reasoning.
Table 13: A comparison the time (in seconds) required to complete a single training step between RL and SFT for both thinking and non-thinking.
Thinking
Method
RL
SFT 9.11
Time (s)
297.51
Non- ton-thinking
Method
RL 51.57
SFT 8.66
Time (s)
27

---

Published in Transactions on Machine Learning Research (10/2025)
E
Training Steps
We compare thinking and non-thinking across training steps. As Table 14 illustrates, extending the number of training iterations consistently improves downstream generalization for both approaches, albeit at a decreasing rate. This confirms the value of additional training steps when computationally feasible. Notably, thinking consistently outperforms non-thinking across different training steps, aligning with our observations.
Table 14: Averaged downstream task performance of Qwen2.5- 2---2-3--LL when trained with different nunber of steps. For thinking and non thinking of the same model, the better result is underlined.
Thinking
Training Steps 1,000
2,000
Performance
60.86
61.24
Non-thinking
Training Steps
1,000
2,000
Performance 72.47
73.18
28

---

Published in Transactions on Machine Learning Research (10/2025)
F Qwen2-VL-2B- Base
The training dynamics of Qwen2-VL-2BBBan 2B-Basc on 2x1 pair jigsaw puzzles are illustrated in Figure While rewards show a rapid initial increase, mirroring trends seen in instruction-luned models, the improvement in accuracy reward is largely superficial. It primarily reflects the model learning to adhere to the specified output format, which allows for the extraction of a final answer, rather than indicating a genuine enhancement in its capabilitics solve jigsaw puzzles. Subsequently, the accuracy reward stagnates, not surpassing 0.5, which is equivalent to the performance of random guessing. This leads to hypothesize that jigsaw puzzles present significant challenge for the base model, potentially requiring substantially extended training period to achieve meaningful performance gains.
Furthermore, the completion length also shows a swift imitial growth. This parallels the reward behavior: the model initially tends to provide direct answers but then learns to include the required explicit reasoning process within the specified format, leading to longer outputs. After this adjustment period, the completion length remains relatively stable.
Similar to observations with instruction-tuned models, the keywords already present in the model before training begins (at step 0). This aligns with findings in textual domain (Liu et al. 2025e) Besides, while these keywords appear more often as training progresses, their overall occurrence remains relatively infrequent.
0.5
140
Koyword Proquencyy
Reward
0.3
100
0.2
60
Accuracy Reward
Completion Length
20
800
Training Step
Training Step
Training Step
Figure 4: The training dynamics of Jigsaw-R1 using Qwen2-VT.-2B-
29

---

Published in Transactions on Machine Learning Research (10/2025)
G Few-shot Learning
This section details the few-shot learning performance of Qween..5VVI33 (non-thinking) on 2x1 pair jigsaw puzzles. For each lest run, we randomly sample a fixed set of few-shot examples from the training set. The process is repeated five times with different random seeds to compute the mean and standard deviation.
As shown Table 15 the model's performance slightly increases with two shots but then deelines as morc examples are added. The initial improvement likely serves provide additional contextual clues, whereas the subsequent drop suggests the model struggles with longer contexts. Ultimately, the performance ceiling of approximately 53.7% highlights that jigsaw puzzles remain significant challenge for the model.
Table 15: Evaluation results on 2x1 pair jigsaw puzzles of Qwen2.5-VL-3B (non-thinking) with different.
number of shots.
Shot
4
16
Performance
52.20
53.74 + 1.08
50.56 0.86
49.50 + 0.41
50.30 + 1.03
30

---

Published in Transactions on Machine Learning Research (10/2025)
H Reasoning Patterns
H.1 Keywords
A key characteristic in our sctting is that MLLMs often describe image content when answering the question. Consequently, many keywords may appear in these descriptions rather than reflecting the cognitive behaviors (e.g., the word "wait" in "two people waiting for trains"). To address this, after carefully examining model outputs, we select keywords specifically indicative of backtracking and backward chaining. While this targetod selection minimizes false positives, it inherently increases false negalives, resulting in a relatively low observed frequency. Specifically, they might appear only once in hundreds of samples.
Selected Keywords
Backtracking:
recheck, reverify, reevaluate, reexmamine
Backward Chaining: work backwards
H.2 Content Analysis
Except keyword matching, we also use GPT-4.1 to analyze whether a reasoning process exhibits the four cognitive bchaviors, adopting prompts from Gandhi al.] (2025). As shown in Figure 5 the frequencies of verification and backtracking increase throughout training, mirroring the observation from keyword matching. In contrast, the frequencies of subgoal setting and backward chaining remain largely unchanged.
40
35
30
Frequency
25
15
10
Verification
Backt.rackng
0
200
400
600
800
1000
200
400
600
800
1000
300
24
250
21
Frequency
200
150
100
Subgoal Setting
Backward Chaining
200
400
600
800
1000
200
400
600
800
1000
Training Step
Training Step
Figure 5: Evolution of cognitive behavior frequencies throughout the training process.
31

---

Published in Transactions on Machine Learning Research (10/2025)
Prompts
1.1 Full Jigsaw Puzzles
2x2 Full Jigsaw Puzzle (thinking)
The imput image is divided into 2x2 patches that have been randomly permuted from their original positions. Your task to solve this 2x2 jigsaw puzzle and reconstruct the original image.
Consider 2x2 grid, where each number represents a position index ranging from (top- left) to (bottom-right):
2
3 4
For each patch, determine its correct position index in the original image. a patch cur- rently at position should belong at position Y, place "Y" at position X.
First, output the thinking process within <think> </think> tags. Then, provide the final answer within <answer> </answer>> tags. The final answer should be the position indexes arranged in 2x2 grid.
2x2 Full Jigsaw Puzzle (non-thinking)
The imput image is divided into 2x2 patches that have been randomly permuted from their original positions. Your task to solve this 2x2 jigsaw puzzle and reconstruct the original image.
Consider 2x2 grid, where each number represents a position index ranging from (top- left) to (bottom-right):
2
3 4
For each patch, determine its correct position index in the original image. a patch cur- rently position should belong position Y, place 'Y" position X.
Directly output the final answer. The final answer should be the position indexes arranged in 2x2 grid.
32

---

Published in Transactions on Machine Learning Research (10/2025)
1.2 Pair Jigsaw Puzzles
2x2 Pair Jigsaw Puzzle (thinking)
The imput image is divided into 2x2 patches that have been randomly permuted from their original positions. Your task is to solve this 2x2 jigsaw puzzle and reconstruct the original image.
Consider a 2x2 grid, where each number represents a position index ranging from (top- left) to (bottom-right)::
2
3 4
For patches currently at positions 3 and 2, determine their relative position in the original image.
Select the correct answer from the following choices:
(A) 3 is on the upper right of 2
(B) 3 is on the lower left of 2
(C) 3 is on the upper left of
(D) is directly to the right of 2
(E) 3 is directly below 2
(F) 3 is directly above
(G) is directly to the left of 2
(H) 3 is on the lower right of 2
First, output the thinking process within <think> </think> tags. Then, provide the final answer within <answer> </answer> tags. The final answer should be a single letter.
33

---

Published in Transactions on Machine Learning Research (10/2025)
2x2 Pair Jigsaw Puzzle (non-tine monnthinkiigg)
The imput image is divided into 2x2 patches that have been randomly permuted from their original positions. Your task to solve this 2x2 jigsaw puzzle and reconstruct the original image.
Consider 2x2 grid, where cach number represents a position index ranging from (top- left) to (bottom-right):
2
3 4
For patches currently at positions 3 and 2, determine their relative position the original image.
Select the correct answer from the following 8 choices:
(A) 3 is on the upper right of 2
(B) 3 is on the lower left of 2
(C) 3 is on the upper left
(D) directly the right of 2
(E) 3 is directly below 2
(F) 3 is directly above 2
(G) 3 is directly to the left of 2
(H) 3 is on the lower right of 2
Directly output the final answer. The final answer should be a single letter.
34

---

Published in Transactions on Machine Learning Research (10/2025)
1.3
Box Jigsaw Puzzles
2x2 Box Jigsaw Puzzle (thinking)
The input image is divided into 2x2 regions. Consider 2x2 grid, where each number represents region index ranging from (top-left) to 4 (bottom-right)::
2
Some patches of equal size have been randomly swapped from their original positions, sulting in unnatural appearance. Your task is to find the original location of the patch that currently belongs in region 4.
First, output the thinking process within <think> </think> tags. Then, provide the final answer within <answer> </answer> tags. The final answer should be bounding box coordinates, formatted as integers and separated by comma.
2x2 Box Jigsaw Puzzle (non-thinking)
The input image is divided into 2x2 regions. Consider 2x2 grid, where each number represents region index ranging from (top-left) to 4 (bottom-right)::
1
Some patches of equal size have been randomly swapped from their original positions, sulting in unnatural appearance. Your task is to find the original location of the patch that currently belongs in region 4.
Directly output the final answer. The final answer should be bounding box coordinates, formatted as integers and separated by comma.
35

---

Published in Transactions on Machine Learning Research (10/2025)
1.4 Image Rotation
Image Rotation with 4 choices (thinking)
The imput image has been rotated clockwise by angle that is multiple of 90. Your task is to determine the angle of clockwise rotation.
Select the correct answer from the following choices:
(A)
(B) 180
(C) 90
(D) 270
First, output the thinking process within <think> </think> tags. Then, provide the final answer within <answer> </answer> tags. The final answer should be single letter.
Image Rotation with 4 choices (non-thinking)
The input image has been rotated clockwise by an angle that multiple of 90. Your task is determine the angle of clockwise rotation.
Select the correct answer from the following 4 choices:
(A) 0
(B) 180
(C) 90
(D) 270
Directly output the final answer. The final answer should be single letter.
36

---

Published in Transactions on Machine Learning Research (10/2025)
J Examples
J.1 Qwen2.5-VL-3B
In this section, we present two illustrative examples of Qwen2.5-VL-3B where the model's final answer, while correct, is inconsistent with its preceding reasoning process. In the first example, the model's reasoning chain indicates that the patch containing rice should be placed on the left, yet it chooses option A, which is indeed the correct outcome. Similarly, in the second instance, the model's rationale suggests that patch should be positioned directly above patch 1; nevertheless, again selects option A, also the correct answer.
Qwen2.5-VI--3B VL--3B Example
Prompt:
Select the correct answer from the following 2 choices:
(A) is directly the left of
(B) 2 is directly the right of
Output:
<think>
The image on the left shows a bowl rice with a spoon and fork placed next to it. The image on the sauce. If we consider the typical layout of meal tray, the rice would typically be served first, followed by the main dish. Therefore, the image on the left should be positioned to the left of the image the right. </think>
right shows a plate of mixed vegetables
<answer> AaAneer><//assvae a/anser>>
Ground truth:
37

---

Published in Transactions on Machine Learning Research (10/2025)
Swenn..5-VT.-3) Example
POT OF THOTS
Shor
15
Prompt: Select the correct answer from the following choices:
(A) 2 is directly below 1
(B) 2 is directly above 1
Output:
<think>
The image shows a scene of game being played on sandy surface with spectators in the background. The patch labeled "2' appears to be the top part of the image, showing the upper body of player and the frisbee in motion. The patch labeled '1" seems to be the lower part of the image, showing the lower body of the same player and the ground.
Given the layout the image, it is clear that the top part (patch 2) is above the bottom part (patch 1). Therefore, the correct answer that patch directly above patch </think>
<answer> A</answer
Ground truth: A
38

---

Published in Transactions on Machine Learning Research (10/2025)
J.2 InternVL2.5-2B Examples
This section includes two illustrative examples from InternVL.2.5-2B.. In both cases, the model's thinking process merely contains the final answer, instead of providing a detailed step-by-step reasoning.
InternVL2.5-22 Example 1
Prompt: ... Select the correct answer from the following 2 choices:
(A) 2 is directly to the left of
(B) 2 is directly to the right of
Output:
<think>
is directly to the left of 1. </think>
<answer>> A</answer>> ansswr>>>>>
Ground truth:
39

---

Published in Transactions on Machine Learning Research (10/2025)
InternVT.2.5-22 Example 2
POT OF THOTS
15
Prompt: Select the correct answer from the following choices:
(A) 2 is directly below 1
(B) 2 is directly above 1
Output:
<think>
directly below 1. </think>
<answer>A<, A >nsswrr>>
Ground truth:
40


Reviews:
[{"review_id": "J1tGFWwu0P", "reviewer": "Reviewer_yXd6", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The paper is easy to follow and clearly summarizes its contributions.\n- The authors provide experimental results backing each claim, clearly define the task and give details on the datasets and hyperparameters.\n\nWeaknesses:\n- The reason for some design choices and comparisons are not mentioned.\n- The authors do not address the chances of data contamination in the MLLMs.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper presents an analysis of rule-based visual reinforcement learning with Jigsaw Puzzles. They find the following:\n\n- The puzzle solving ability of MLLMs without any training is near random guessing. However, it improves with fine-tuning on jigsaw puzzles.\n- Training MLLMs on jigsaw puzzles improves generalization capabilities.\n- MLLMs with rule-based visual RL can learn and generalize without explicit reasoning steps.\n- Reasoning is pre-existent in MLLMs and there is no aha moment. The reasoning however is more frequent for complex tasks.\n- SFT does not generalize as well as RL. Even SFT cold start diminishes the performance of RL.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- For accuracy rewards, the authors use partial rewards (0-1) for full questions, while the pair questions have binary rewards (0 for wrong answer and 1 for right answer). What is the reasoning behind this? Isn't the full question much tougher? Like if you have a 4x1 Puzzle and the MLLM gets 2 of them correct, the reward is 0.5. But in doing so it also gets 2 pair questions right as the two correct blocks have the right relative position and even some from the wrong blocks may have correct relative positions? Then shouldn't the reward be calculated on this basis even for full question? And shouldn't the reward be bigger for the full question?\n- Why are the number of training steps different for thinking (1000) and non-thinking (2000) models?\n- For RQ1, the authors should also compare with few-shot prompting.\n- In RQ3, what does thinking and non-thinking models mean exactly? Are they the base models over which rule-based visual RL fine-tuning is applied? If so can the reason for thinking models struggling be that they are already fine-tuned with RL to think for certain kind of tasks and further RL on top of that drops their performance? If so I would not think this as a contribution as it is sort of expected from the setup? The authors should clarify this.\n- In RQ4, the authors show an example of cognitive behaviors and say that such behaviors increase with problem complexity. However, the authors also mentioned before that non-thinking models and open source models prefer to give direct answer. Then where does this reasoning chain come in?\n- Is their any chance of data contamination, i.e. the models already seeing the datasets on which the authors test the model capabilities in this paper?", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "wvdFPZWHwD", "reviewer": "Reviewer_ZEk1", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength: \n\nCreative Task Design: Using jigsaw puzzles as a pretext task is innovative and practical - it provides automatic ground truth without manual annotation, has adjustable difficulty levels, and naturally combines perception with reasoning.\nComprehensive Experimental Coverage:\n\nTests multiple models (both proprietary and open-source)\nVaries puzzle configurations systematically (2×1, 3×1, 4×1, 2×2)\nExplores different question types and reasoning modes\n\n\nWeaknesses\n\nLimited Mechanistic Understanding: While showing that jigsaw training helps downstream tasks, the paper doesn't explain the underlying mechanisms. Why do pair questions generalize better than full puzzles? What specific visual features or skills are being learned?\nShallow Analysis of Reasoning Patterns: The keyword-based analysis of reasoning patterns (backtracking, verification) is limited. The authors acknowledge low keyword frequencies but don't adequately address this limitation or provide more sophisticated analysis methods.\nMissing Technical Details: No discussion of computational costs, training times, or efficiency comparisons between different approaches. The SFT coldstart not being effective is only reported as evaluation results, but no details goes into how the data was prepared. Is it long cot or shortr cot", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper investigates rule-based visual reinforcement learning (RL) using jigsaw puzzles as a structured experimental framework for multimodal large language models (MLLMs). The work addresses five key research questions about how MLLMs perform on and learn from jigsaw puzzles, revealing several important findings:\n\nPerformance and Generalization: MLLMs initially perform at random-guessing levels on simple jigsaw puzzles but achieve near-perfect accuracy after fine-tuning and can generalize to more complex unseen configurations.\nCross-task Transfer: Training on jigsaw puzzles induces generalization to other visual reasoning tasks, with effectiveness dependent on specific configurations (puzzle size, question type, training dataset).\nReasoning Modes: MLLMs can learn effectively with or without explicit reasoning chains. \nTraining Methods: RL demonstrates superior generalization compared to supervised fine-tuning (SFT), and SFT cold-start phases can hinder subsequent RL optimization.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Strengthen Reasoning Pattern Analysis: Go beyond keyword frequency analysis: Develop more sophisticated metrics for measuring reasoning quality. Analyze the semantic content and correctness of reasoning chains. \n\n\nProvide Mechanistic Analysis of Generalization: Add experiments or analysis to understand why jigsaw puzzles improve certain downstream tasks but not others. \n\nAdd more details on how the SFT coldstart was experimented \n\nIf you can also add aseline Comparisons: Include comparisons with other visual pretext tasks (e.g., rotation prediction, colorization) i think it would me effective", "broader_impact_concerns": "This work does not raise significant ethical concerns. The research focuses on improving visual reasoning capabilities of MLLMs through a benign pretext task. The paper appropriately acknowledges limitations and does not make exaggerated claims about capabilities.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "7rC389pWYc", "reviewer": "Reviewer_fBqt", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strengths\n\n1. The proposed experiment demonstrates sound design. The authors provide a clear rationale for selecting jigsaw puzzles, detailing task design, reward setup, and other key aspects.\n2. The results presented in this paper are intriguing, with the finding that training on jigsaw puzzles can generalize to other tasks being particularly noteworthy.\n3. Another interesting outcome is that thinking models tended to ignore the thought process when predicting output.\n4. The paper is well-written and easy to follow.\n\n# Weaknesses\n\n1.The authors' choice to focus on a single pretext task may limit the generalizability of their findings, leaving open the question of whether their results can be extended to other scenarios.\n2. While the presented results are intriguing, they could benefit from further discussion regarding their implications for future work. Specifically, it would be helpful to clarify the motivations behind exploring these specific research questions.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper conducts an exploratory investigation into the application of rule-based reinforcement learning (RL) to multimodal large language models (MLLMs). To facilitate this inquiry, the authors design a structured experimental framework centered around jigsaw puzzles. This framework enables them to address five research questions:\n\n1. How do MLLMs perform in this context?\n2. Can MLLMs trained on jigsaw puzzles generalize to other tasks?\n3. Does explicit thinking process contribute to performance in this scenario?\n4. Do \"aha moments\" arise during training?\n4.How does supervised fine-tuning (SFT) compare with RL in these tasks?\n\nThe study contributes to a deeper understanding of the capabilities and limitations of MLLMs.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. The authors' proposal to use two rewards - accuracy and format - raises questions about the rationale behind their chosen format reward value (0.5 or 0). It would be helpful to understand how this value was selected and whether alternative values could impact the outcomes.\n2. The finding that training on jigsaw puzzles generalizes to other tasks is an important one. A more comprehensive discussion of this point would be valuable, including consideration of potential limitations and implications. For instance, does this suggest that MLLMs should always incorporate jigsaw puzzles in their training process? What are the potential drawbacks of doing so?\n3. As mentioned previously, I believe it would be beneficial to see a broader discussion about how the results of this paper can inform future research work. This could include exploring potential applications and limitations of the proposed approach.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)