Visual Jigsaw Post-Training Improves MLLMs
Authors: Penghao Wu
Yushan Zhang
Haiwen Diao
Bo Li
Lewei Lu
Ziwei Liu
Venue: ICLR 2026 Poster (2026)
Keywords: Multimodal Large Language Models
Self-supervised Learning
Post-training
Reinforcement Learning
Visual Jigsaw
Primary area: foundation or frontier models, including LLMs

Abstract: Reinforcement learning based post-training has recently emerged as a powerful paradigm for enhancing the alignment and reasoning capabilities of multimodal large language models (MLLMs). While *vision-centric* post-training is crucial for enhancing MLLMs’ intrinsic understanding of visual signals, current post-training paradigms are predominantly *text-centric*, where dense visual inputs are only leveraged to extract sparse cues for text-based reasoning. There exist a few approaches in this direction, however, they often still rely on text as an intermediate mediator or introduce additional visual generative designs. In this work, we introduce **Visual Jigsaw**, a generic *self-supervised* post-training framework designed to strengthen visual understanding in MLLMs. Visual Jigsaw is formulated as a general ordering task: visual inputs are partitioned, shuffled, and the model must reconstruct the visual information by producing the correct permutation in natural language. This naturally aligns with reinforcement learning from verifiable rewards (RLVR), requires no additional visual generative components, and derives its supervisory signal automatically without any annotations. We instantiate Visual Jigsaw across three visual modalities, including images, videos, and 3D data. Extensive experiments demonstrate substantial improvements in fine-grained perception, temporal reasoning, and 3D spatial understanding. Our findings highlight the potential of self-supervised vision-centric tasks in post-training MLLMs and aim to inspire further research on vision-centric pretext designs.

Published as a conference paper at ICLR 2026
VISUAL JIGSAW POST-TRAINING IMPROVES MLLMs
Penghao Wu'
Yushan Zhang²
Haiwen Diao'
Bo Li
Lewei Lu
Ziwei Liu
SS-Lab, Nanyang Technological University
"Linköping University
"SenseTime Research
Project Page: htgp://ss.eeeeaoo.......ooooooo.
ABSTRACT
Reinforcement learning based post-training has recently emerged as powerful paradigm for enhancing the alignment and reasoning capabilities of multimodal large language models (MI.Mss). While vision-centric post-training is crucial for enhancing MILMs' intrinsic understanding of visual signals, current post- training paradigms predominantly text-centric, where dense visual inputs are only leveraged to extract sparse cues for text-based reasoning. There exist few approaches in this direction, however, they often still rely text diate mediator or introduce additional visual generative designs. In this work, we introduce Visual Jigsaw, generic self-supervised post-training framework de- signed to strengthen visual understanding in MILMs. Visual Jigsaw is formulated general ordering task: visual inputs partitioned, shuffled, and the model must reconstruct the visual information by producing the correct permutation in natural language. This naturally aligns with reinforcement learning from verifiable rewards (RLVR), requires no additional visual generative components, and derives supervisory signal automatically without any annotations. We instantiate Vi- sual Jigsaw across three visual modalities, including images, videos, and 3D data. Extensive experiments demonstrate substantial improvements in fine- fine-grained per- ception, temporal reasoning, and 3D spatial understanding. Our findings highlight the potential of self-supervised vision-centric tasks in post-training MLLMs and pretext designs.
interme-
aim to inspire further research
Visual Tigsow
Owen25
SuparCrope
Vasal Agow
Viedfpatia
1008Benchh
Ialleahh
NNNVO
Borch
Vinerrant
Ondelspatia
Val
Image Jigsaw
Video ligsaw
3D Jigaw
Figure 1: We propose Visual Jigsaw, a self-supervised post-training task that enhances visual per- ception and understanding in MLLMs. Training visual jigsaw tasks substantially strengthens fine-grained perception, monocular spatial perception, and compositional visual understanding in images; temporal understanding in videos; and geometry-aware understanding in 3D, demonstrat- ing its generality and effectiveness across modalities. For clearer visualization, the value ranges differ across benchmarks in each radar chart.
INTRODUCTION
Multimodal large language models (MLLMs) have recently demonstrated remarkable progress, achieving strong performance on a wide range of vision-language tasks. Following the success of Reinforcement Learning from Verifiable Reward (RLVR) (Lambert et al. 2024 Guo et al. 2025) in the large language models domain, which has unlocked substantial breakthroughs in complex rea- soning abilities, the research community has largely shifted its focus toward replicating this success

---

Published as a conference paper at ICLR 2026
in the multimodal domain. This has led to a predominant focus advancing text-based Chain-of Thought (CoT) multimodal reasoning to enhance multimodal mathematical and scientific reasoning
(Huang et al. 2025 Meng et al. 2025 Yuan et al. 2025a Wang al. (025f).
Within this paradigm, dense visual information often serves merely as contextual evidence, from which the model extracts sparse information support text-based reasoning. Consequently, a deep, fine-grained understanding of the visual signal itself has been considerably undervalued. Some recent studies (Wang al., 2025b a) have shown that explicitly incorporating visual reconstruc- tion objectíves during the training of MLLMS can improve visual understanding. However, such approaches necessitate the integration of additional visual generation components and learning ob- jectives onto the existing understanding- based MLLM architectures. Furthermore, it remains an open question whether forcing models to achieve pixel-level reconstruction the optimal strategy for enhancing MILMMs visual understanding. This raises a pivotal question: Can we enhance an MLLM's visual understanding without altering its architecture or output format?
Delving into the history of self-supervised visual representation learning reveals rich set of pretext tasks, such as reconstruction- approaches (He et al. 2022a) and discriminative approaches (He) et al. 2020). In parallel. jigsaw-style tasks have emerged as lightweight yet effective paradigms: reordering shuffled image patches (Noroozi & Favaro 2016), recovering video frame order (Ahsan ct al. 2019). While these jigsaw-style approaches provide structural ordering signals, they have generally shown weaker performance compared to more dominant approaches and thus have not become mainstream in vision representation learning. Nevertheless, they demonstrate that the struc- tural ordering jigsaw task, which can be viewed as a simpler version of the wwtnsstrucctooggeeratttt task, can still offer effective self-supervised signals without requiring pixel-level fidelity.
In this work, we introduce Visual Jigsaw, a self-supervised task designed for the RL post-training phase of MLLMs to enhance their visual perception and understanding. The task is formulated as a lightweight ordering problem: visual inputs are partitioned, permuted, and presented to the MLLM, which must then generate the correct permutation order using natural language. Importantly, this for- mulation requires no additional visual generative designs and is seamlessly compatible with existing MLLMs that produce text-only outputs. Moreover, this task naturally fits in the RLVR framework with deterministic ground-truth and requires no other annotations. We position Visual Jigsaw in the post-training phase, as solving it requires the model to already possess a foundational level of visual understanding. Furthermore, post-training with RL has been shown to offer stronger gen- eralization than Supervised Fine-Tuning (SFT) (Huan et al., [025] Chu et al.) [025 [Cen et al.] 2025a), enabling the model to better transfer the vision-centric skills acquired from the jigsaw task to downstream applications.
We implement Visual Jigsaw across three visual modalities: images, videos, and 3D data. Through a post-training phase with Group Relative Policy Optimization (GRPO) (Shao et al. (024) on visual jigsaw tasks, we substantially improve the ability of MLLMs perceive and comprehend these vi- sual modalities (shown in Fig M. In the image domain, we partition the input into patches, shuffle them, and require the model to recover the correct spatial arrangement. We find that this task en- hances fine- mme-grained perception, monocular spatial understanding, and compositional visual under- standing. For video, we segment the input along the temporal axis, shuffle the clips, and challenge the model to reconstruct the original sequence, leading to marked improvements in temporal under- standing. In the 3D domain, we sample points with distinct depth values from an RGB-D image, shuffle and annotate them in the RGB view, and require the model to recover their order from nearest to farthest, thereby augmenting its 3D perceptual capabilities.
Our main contributions are: 1) We introduce Visual Jigsaw, a lightweight and verifiable self- supervised post-training task that enhances vision-centric perception and understanding capabilities in MLLMs. It requires no additional generative modules and integrates scamlessly with existing text-only models. 2) We instantiate Visual Jigsaw across three visual modalities moddalssees-mmages videos, and 3D data-and demonstrate consistent improvements in fine-grained perception, temporal un- derstanding, and 3D spatial reasoning, thereby establishing its generality and effectiveness. 3) We highlight the potential of self-supervised tasks focused explicitly on the visual signal a promising, complementary direction for enhancing the iisionccentric abilities of MLLMs.
2

---

Published as a conference paper at ICLR 2026
2
RELATED WORKS
2.1 SELF-SUPERVISED LEARNING
Self-supervised learning (SSL), wherein pretext tasks derive supervision directly from input data, has become cornerstone of visual representation learning. Early approaches included context- based tasks such as predicting relative patch positions (Doersch et al., (013) and patch order- ings (Noroozi & Favaro 2016). While these works revealed the potential of such proxy tasks, they were limited in scalability. More recently, SSL has been dominated by two major families:
(1) reconstruction-based methods (Zhou et al.] [02T He et al.) 2022b Bao al., 2021: Assran
itt al. 2023) and (2) discriminative methods (He et al. 2020 Chen et al. 2020 Caron et al., 2021)
These approaches have demonstrated impressive scalability and transferability, establishing strong
foundations for large-scale vision pre-training (Oquab et al., 2023; Siméoni et al. 2025).
Parallel these approaches, jigsaw pretext tasks explicitly formulate visual learning as an ordering problem, requiring the model to recover the spatial or temporal structure of visual inputs. Noroozi
Favaro (2016) pioneered the 3 x image jigsaw puzzle, which was later extended for iterative refinements [Wei et al. 2019), domain generalization (Carlucci et al. 2019), and fine-grained rea- soning (Du et al. 2020). Extensions to video include spatiotemporal jigsaws (Ahsan et al.) 2019 Huo et al. 2021] Wang et al. 2022) that jointly exploit appearance and motion cues. Recent jigsaw- style variants (Caron et al.) 2024] Wang et al.] 2023) Zhai et al.] 2022) have shown competitive results with contrastive learning and masked image modeling when properly designed and scaled.
The characteristics of jigsaw-style tasks also make them a good fit for understanding based MLLMs, which are optimized for visual understanding with textual outputs rather than dense reconstruction. A visual jigsaw task thus provides a lightweight, verifiable objective that requires no additional generative modules. Building these advantages, work introduces Visual Jigsaw self- supervised post-traine fost-training stage to enhance vision-centric perception in MLLMs across image, video, and 3D modalities.
2.2 MLLM VISUAL UNDERSTANDING
MLLMS (Hurst al. 2024 Comanici et al. 2025 Bai al. 2025 Zhu et al.) 2025a) have rapidly advanced, achieving strong performance across diverse multimodal tasks. These improvements have largely stemmed from more powerful LLM backbones, better image resolution strategies, improved vision encoders, higher-quality training datasets, and post-training techniques. However, relatively little attention has been devoted to enhancing the intrinsic visual understanding MILMs. Most existing efforts rely scaling data to indirectly improve ceeecpptio-rellttee tasks.
F-Former(Swetha et al. 2024) adds visual reconstruction objective in MLLM training but its tar- get is to better extract and combine vision features from two vision encoders and only supervise the connector with the reconstruction objective. Recent works (Wang et al.] 2025b a) demonstrate that explicitly adding visual reconstruction objectives enhances visual understanding, but such ap- proaches require introducing extra generative modules and objectives, have only been demonstrated in settings where the MLLM is trained jointly with reconstruction from the beginning. and have not been validated on stronger models like Qwen2.5-VL (Bai et al. 2025). Meanwhile, unified multi- modal models (UMMs) (ie et al. 2025b; Chen et al. 2025c Deng et al. 2025 Chen al. 2025b) explore combining vision understanding and generation in one model, but it has only been shown that understanding benefits visual generation Xie et al. (2025a) while optimizing generative objec- tives sometimes harm understanding abilities Pan et al. (2025); [Cenn et al.] (2025b). In contrast, we propose a lightweight, post-training self-supervised task that strengthens visual perception and understanding in MLLMs without altering the architecture.
2.3 MLLM RL POST-TRAINING
RL post-training has played pivotal role in advancing LLMs. Early paradigms such RLHF OOuyang et al. 2022) and DPO (Rafailov et al. 2023) focused on improving alignment with human preferences, while recent developments like RLVR (Lambert al. 2024: Shao et al.) (024) have been shown to substantially enhance reasoning capabilities. Inspired by this success, the MLLM community has begun to apply similar paradigms. Most works concentrate strengthening multi-
3

---

Published as a conference paper at ICLR 2026
Image Jigsaw
3D Jigsaw
Accuracy:
Accurscy: 1.00
Policy
Policy
Accurscy: 0.67
%
Rank closest fanthest
0.00
Accurscy: 0.00
Video Jigsaw
1.00
Policy
0.00
Figure 2: Illustration of the Visual Jigsaw tasks. In the Image Jigsaw (top left), image is partitioned into non-overlapping patches, shuffled into a sequence, and the model is tasked with predicting the correct raster order. In the Video Jigsaw (bottom), video is segmented into temporal clips, shuffled, and the model predicts their original chronological order. the 3D Jigsaw (top right), points with distinct depth values sampled from RGB-D image, shuffled and annotated the RGB view, and the model is required to recover the correct depth order from nearest to farthest. Across all tasks, the policy model outputs an ordering that compared against the ground truth, and a partial accuracy reward is assigned when only some elements are correctly ordered.
modal reasoning for mathematical and scientific tasks (Meng et al. 2025 Huang et al. 2025 Yuan et al. 2025a Wang al. (0225)). These RL-based approaches have also been extended to video (Feng et al. 2025 Chen et al., 2025d) and 3D domains ( (Yuan et al.) (025b). Other methods focus on specific vision tasks such grounding (Liu al. (025b) and segmentation (Liu al. 2025a). More recent efforts (Zheng et al. 2025 Su et al.] 2025) ) also explore teaching MI..Ms to use vision tools, enabling models think with images and better retrieve and perceive the visual input via operations like crop and zoom-in.
However, the majority of these approaches still target text-based reasoning, task-specific objectives. or visual tool-calling, rather than directly improving intrinsic visual perception. Vicrit (Wang et al.) 2025ep and LLaVA-Critic-- ing eπrors in captions or judging textual responses, but their training signals are ultimately tied to text-image alignment instead of intrinsic visual signal understanding. The most closely related work, Jigsaw-R1 Wang et al. (2025g). also attempts to introduce jigsaw task for MILLM post- training. However, its approach struggles even on simple 2 2 image jigsaws, thus focusing mainly predicting the relative position of pair of patches. In contrast,
(Wang al.) 2025d) enhance perception and reasoning by detect-
method leverages the standard, more complex visual jigsaw tasks to systematically enhance MLLM perception, and we demonstrate its effectiveness not only images but also across video and 3D modalities.
METHOD
3.1
VISUAL JIGSAW
Our proposed Visual Jigsaw framework (illustrated Fig 2b is formulated as a general visual or- dering problem. Given some data from certain visual modality (image, video, 3D), derive a set of K jigsaw elements by applying a modality-specifee partitioning rule, such as splitting an image into patches, segmenting video into clips, sampling points in 3D scene. These elements

---

Published as a conference paper at ICLR 2026
then shuffled, and the model is tasked with predicting their original structural arrangement. For- mally, the model predicts a permutation of size K as a list of indices, which is then compared against the ground-truth permutation. We optimize this task using the GRPO algorithm. The following sub- sections detail our reward design and describe the specific instantiations of Visual Jigsaw across each of the three visual modalities.
3.1.1 REWARD DESIGN
The ground-truth of the visual jigsaw tasks is a list of indices that is directly verifiable. Instead of assigning only a binary accuracy reward, we design graded reward function. An output that exactly matches the ground-truth permutation receives an accuracy reward of 1. For a valid but partially correct permutation, the reward is the fraction of correctly placed indices, scaled by discount factor E (0, 1). This discount penalizes incomplete solutions, preventing the model from overestimating partial matches while still providing learning signals. To avoid reward hacking (e.g. predicting the same index for all positions), any output that is not valid permutation of size K is assigned a reward of O. Formally, the accuracy reward function is given by
if
Reward(o, g)
lo, = -g.. ValidPermutation()) A Ao DK K 0,
?
otherwise
where o denotes the model's predicted permutation, g the ground-truth permutation, the number of jigsaw pieces, 7 the discount factor for partial correctness, and ValidPermutation(() indicator of whether o a valid permutation size K.
Besides the accuracy reward, <think></think and the final answer within <answer><> answer>, A format reward of 0.2 will be assigned to outputs following the correct format, while outputs with an incorrect format will receive 0 values for both format and accuracy rewards.
also require the model to put its thinking process within
3.1.2 IMAGE JIGSAW
Given an input image I Hxxx' we first partition it into a grid m x n non-overlapping patches, each of size H x W. This produces K m x n patches arranged in raster order (row- major, top-left to bottom-right)
H W R x3
P= P= P2. ................................ .P......... Ps
We then apply random permutation æ : (1.2..... K} + {1, K} that maps an original position index i to its shuffled position (1). The shuffled sequence of patches can therefore be written as
Px = P. (1): Pr-1(2) .... Pr)1(K))
where the j-th element corresponds the patch originally at position i (j). Given Par the model's objective is to recover the original arrangement by predicting the correct permutation the input patch indices [I),, x(2). (K)).
For training, we use 118K images from the COCO dataset Lin et al. (2014). We set m = n 3, yielding 9 patches per image, and filter out images with side lengths smaller than 84 pixels to avoid overly small patches. The prompt template for this task is provided in Appendix A.6
3.1.3 VIDEO JIGSAW
Given video V RTxHxWx3 with T frames, segment it uniformly along the temporal axis into K non-overlapping clips, each containing T consecutive frames: K
T Uy E R #xxxx
V [V1, U2 ... vkl.

---

Published as a conference paper at ICLR 2026
We then apply random permutation (1.2...., - f1,2,.... ..,,,..... K), where t()) denotes the shuffled position of the i-th clip in the original chronological order. The shuffled sequence is written
V (1); (2):
(x))
The model's objective is to restore the original chronological order by predicting the correct permu- tation (1), (2), . {K])..
For training, we use 100K videos from the LLaVA- Video dataset ZZhang et al. 2024c Each video divided into clips (K" 6). To prevent the model from exploiting simple frame- matching cues at clip boundaries, we trim 5% of the frames from both the beginning and end of each clip. The maximum number of frames for each clip set to 12, and the maximum resolution for each frame is set to 128 x 28 28 pixels. We remove videos with lengths smaller than 24 seconds. The prompt template for this task can be found in Appendix A.6
3.1.4 3D JIGSAW
A canonical 3D jigsaw task would mirror its 2D image and video counterparts, involving the par- titioning of 3D space into volumetric primitives (e.g. voxels, mesh fragments, or point cloud seg- ments) and tasking the model with recovering the original spatial arrangement. Such formulations would fully leverage geometric information native 3D representations. However, current general- purpose MLLMs typically process 3D-related tasks via 2D images or videos rather than directly operating on raw 3D data structures.
We therefore design practical variant of the 3D jigsaw based on RGB-D images. Given an RGB-D image, we randomly select K points with distinct depth values, forming sequence sorted by depth from nearest farthest:
P= P=Ip1.2 ,Kl,, der < dpz ... < dpx:
where dp. is the depth of point p:-
We then apply a random permutation π f1.22...... K} f1.2..... .., obtain a shuffled sequence of the points
P. P== +(1: Ps (2): Pr (K)
Each point is annotated with its index in Pa on the RGB image. The model is tasked with recovering the correct depth order by predicting the permutation (I),,, (2), ..... [[K] that restores P.
For this task, we use the RGB-D data from ScanNet (Dai et al. 2017) and generate 300K training samples in total. We construct training samples by randomly selecting 6-point combinations from depth maps, restricting the points to lie within a range of 0.1 to 10 m. To ensure diversity, any two points in a combination must be separated by least 40 pixels in the image and differ in depth by more than 0.2 m. The prompt template for this task can be found in Appendix A.6 We also experimented with alternative designs of 3D jigsaw tasks, which are provided in Appendix
4 EXPERIMENTS
4.1
IMPLEMENTATION DETAILS
We adopt Qwen2.5-VL --sttt-rtttttt the base MLLM for all experiments. We the GRPO algorithm and remove both the KL regularization and the entropy loss. The discount factor & for partially correct predictions is set to 0.2. The training is performed with global batch size of 256 for image jigsaw and 128 for video & 3D jigsaw, and the learning rate 1 x 10-6 For each prompt, sample 16 responses with decoding temperature of 1.0. Both image and video jigsaw tasks trained for 1000 steps, and the 3D jigsaw is trained for 800 steps.
4.2 MAIN RESULTS
This section presents quantitative results. Qualitative examples are provided in Appendix A.4

---

Published as a conference paper at ICLR 2026
4.2.1 IMAGE JIGSAW
We evaluate the model trained with image jigsaw across three categories of vision-centric bench- marks including 1) Fine-grained perception & understanding: MMVP (Tong et al. 2024), fine-
grained perception subset of MMStar (Chen et al., 2024). MMBench (Liu et al. 2024a HR- HR-Bench
(Wang et al., 2025c), V* (Wu & Xie, 2024), MME Real World (lite) (Zhang et al. (025b), LISA-
Grounding LLai et al. 2024), OVD-Eval ( (Yao et al.
2024); 2) Monocular spatial understanding:
VSR (L.iu et al. 2023). OminiSpatial (Jia et al. 2025
DA-2K (Yang et al. 2024); 3) Compositional
visual understanding: Winoground (Thrush et al.
2022). SugerCrepe++ (Dumpala al. 2024).
We include three baselines, which are all post-trained from Qwen2.5-VL-77 Think VL (Wang jet al. 2025f) mainly focuses on improving multimodal reasoning. VL-Cogito (Yuan et al. 2025a)
trained on a broader set of tasks, including general image understanding and counting, in addition to mathematical and scientific reasoning. LLaVA-Critic-RI (Wang et al.) 2025d) is trained with the critic task and shows improvement in image perception and understanding. As these vision-centric benchmarks mainly focus on direct visual perception and understanding, we directly evaluate the model to give the short answer without the thinking/reasoning process for fair comparison. This protocol is further motivated by our finding that enabling chain-of-thought reasoning can actually degrade the performance of some reasoning models on some specific benchmarks (e.g. 35.78 31.44 on OVD-Eval for Vhin
Table I:
Evaluation results image benchmarks. Image Jigsaw achieves consistent improve-
ments across fine-grained perception, spatial understanding, and compositional understanding tasks.
Fine-grained Perception Understanding
Spatial Und (Mono) Compositional Und
MMStar (flee-gr (eeeegranned)
WeMr-d
USA-Grounding
HR-Bench-SK
OmniSpatial
Winoground
SuparCreper+
MMBRenh
OVD-Eval
MMVP
DA-2K
VSR
Model
test
fine
en_dev
test
test
lite
test
test
test
test
val
g-acc
test
ThinkLite-VL VL
55.33 59.95 84.19 68. 12.127 76.96 46. 17 73.70 35. 78 78.09 42.60 (3..25
61.49
VL-Cogito
55.33 56.64 82.98 69.62 79.58 47.63 72.26 35.78 79.82
44.29 56.43
38.25
63.59
LLaVA-Critic-RI
53.33 57.80
83.16
67.50 78.01 48.18 68.52 35.28
78.50
42.73
53.82
34.75
61.93
Qwen2.5-VL-7B
54.66 59.75 83.33
67.38 76.96 43.41 71. 35.079995577777777..7
42.66 (7.00
61.59
Image Jigsaw (SFT)
56.00 60.94
83.67
69.75 80. 10 43.88 66.59 34.35180.68
43.55
61,46
(38.
62.03
Image Jigsaw 60.66 65.81 84.45
71.13 80.63 45.96 74.54 36.49
80.36
44.49
60.35
39.00
63.02
(Gain)
+6.06
+3.75 +2.55 +2.65 +1.42
+5.90 +2.00
+1.43
Tab shows that our method consistently improves the vision-centric capabilities on the three types of benchmarks. These results confirm that incorporating image jigsaw post-training significantly enhances MLLMs' perceptual grounding and fine- mne-grained vision understanding beyond reasoning- centrie post-training strategies. We attribute these improvements saw requires the model to attend to local patch details, infer global spatial layouts, and reason about inter-patch relations, which directly benefits ane-ggrieed, spatial, and compositional understanding.
the fact that solving image jig-
4.2.2
VIDEO JIGSAW
For video jigsaw, we evaluate on a comprehensive suite of video benchmarks: AoTBench Xue et al.
2025).
Vinoground (Zhang et al. 2024a), TOMATO (Shangguan et al. 2024), FAVOR- (Tu
ct al.
2025). TUNA-Bench (Kong ct al. 2025), Video-MME (Fu et al. 2025). TempCompass (Liu
ctt al.
2024b), TVBench (Cores et al. 2024). MotionBench (Hong et al., 2025), ,, LVBench (Wang
eet al.
2024b). VSI-Bench (Yang et al. 2025). Video-TT (Zhang et al. 2025c). CVBench (Zhu al.
20256).
We include the Video-R1 (Feng al.) 2025) baseline for comparison, which is trained with cold-
start SFT followed by RL for video understanding and reasoning. We enable the thinking process when evaluating Video-R1 1, find its performance is generally better than direct answering. For
7

---

Published as a conference paper at ICLR 2026
all models, we set the maximum number of pixels to 256 x 28 x 28 and evaluate under three different frame settings (16, 32, 64).
Table 2: Evaluation results on video benchmarks.
Video Jigsaw consistently improves over the
baseline across all benchmarks and frame settings.
FAVOR-Bend
Vinoground
TUNA-Bench
VidcoMME
TempCompas
MotionBench
AoTBench
TOMATO
TVBench
L.Bench
VSI-Bench
Video-TT
CVBench
Model
Frames
vqa group test
test
test
wo subs
me
test
val
test
test
meg
test
Video-RI
16
45.06 9,40 27.29 49.47 53.00
56.62
70.19
51.80 55.82 34.53 34.34 42.95
Video-R1
32
47.53
10.20 27.29 49.90
54.26
59.88
71.77
53.54 56.12 38.61 35.11 42.63 48.10
Video-RI
64
48.68
10.60 27.36
50.51
54.33
60.85
72.59 53.43 56.09 38.80
36.61
42.74448.6
Qwen2.5-VL-7B
45.52
12.60 25.87
53.14
57.44
71.77
49.94 55.56 33.51 32.79 38.39 47.70
Qwen2.5-VL--7B
32
49.48
18.20 26.34
49.34
54.88
60.70
72.59
51.96 56.47 39.19 35.34 41.57 49.60
Qwen2.5-VI-7B
64
52.41
21.80 26.35
50.86
55.79
63.44
53.74 56.29 40.35
37.25 51.50
Video Jigsaw
16
51.67
15.20 27.56
49.69
55.10
58.07
73.10
51.33
56.87 36.41
35.39
40.19 49.80
(Gain)
+6.15
+2.60 +1.69 +1.15 +1.96
40.63
+1.33
+1.39 +1.31 +2.90
+2.60
+1.00 +2.10
Video Jigsaw
32
55.00 40 28.03
50.56 56.49
62.37
73.60
53.31 39.79 38.47
39.70
43.27 51.60
(Gain)
+5.52
43.20 +1.69
41.22 +1.11
+1.67
+1.01
+1.35 +1.22 40.51
43.13
+1.00 42.00
Video Jigsaw
64
57.64
25.20 28.30 52.27 56.63
64.74
73.60
54.18 57.91 41.83 40.40 44.11 54.50
(Gain)
+5.23 +3,40 +1.95 +1.41 +0.84
+1.30
+0.76 +0.44 +1.22 +1.48 +2.66 +1.86 +3.00
From the results shown in Tab 2 we observe that Video Jigsaw brings consistent improvements across all video understanding benchmarks and frame settings. While our method enhances gen- eral video perception and comprehension, the gains are particularly pronounced on tasks requiring temporal-centric understanding and reasoning about temporal directionality (e.g. AoTBencc).. Fur- thermore, the strong gains on CVBench demonstrate improved cross-video understanding and rea- soning. These results confirm that solving video jigsaw tasks encourages the model to better capture temporal continuity, understand relationships across videos, reason about directional consistency, and generalize to holistic and generalizable video understanding scenarios.
4.2.3 3D JIGSAW
For the 3D modality, we evaluate the model on diverse set benchmarks that span various aspects of 3D understanding: SAT-Real (Ray et al. 2024). 3DSRBench (Ma et al. 2024), ViewSpatial (Li al. 2025), All-Angles (Yeh et al. 2025), OminiSpatial (Tia et al. | 2025). VSI-Bench (Yang al.
2025
SPARBench (tiny) (Zhang al. 2025a),, and DA-2K (Yang et al. 2024).
Table 3: Evaluation results on 3D benchmarks. 3D Jigsaw consistently enhances performance on both directly related depth comparison tasks (DA-2K) and broader 3D perception tasks spanning single-view, multi-view, and egocentric video inputs.
3DSRBench
Spatiaa
OmniSpatial
SPARBench
All-Angles
VSI-Bench
SAT-Real
DA-2K
Model
test
test
test
test
test
test
tiny
test
Qwen2.5-VL-7B
48.66
57.42
36.52
47.56
42.66
37.74
35.75
54.45
3D Jigsaw
64.00
58.13
38.62
49.06
45.99
40.64
38,31
71.56
(Gain)
+15.34
+0.71
+2.10
+1.50
+3.33
+2.90
+2.56
+17.11
As shown in Tab prisingly, the largest gain is on DA-2K. a depth estimation benchmark that directly related to our depth-ordering pre-training task. More importantly, observe consistent improvements wide
3D Jigsaw achieves significant improvements across all benchmarks. Unsur-
8

---

Published as a conference paper at ICLR 2026
range of other tasks, including those with single-view (e.g. 3DSRBench, OminiSpatial), multi-view (e.g. ViewSpatial, All-Angles), and egocentric video inputs (e.g. VSI-Bench). These results demon- strate that our approach not only teaches the specific skill of depth ordering but also effectively strengthens the model's general ability to perceive and reason about 3D) spatial structure.
4.3 ABLATION STUDIES AND DISCUSSIONS
SFT vs. RL. We investigate the difference between using SFT and RL to train the visual jigsaw task, focusing the image jigsaw setting. As shown in the Image Jigsaw (SFT) entry of Tab IT SFT leads to moderate improvements on some benchmarks, but the gains are notably smaller than those achieved with RL. Moreover, on certain benchmarks (e.g. L.IAA-Grounding and OVD-Eval), SFT causes a significant performance degradation, suggesting that the model overfits to the jigsaw task and fails to transfer the learned skills. This observation is consistent with recent findings that SFT tends to memorization, while RL is better at promoting generalization (Huan et al. 2025 Chu al. 2025). Our results confirm that RL enables the model to more effectively generalize the Viston-centric capabilities acquired from visual jigsaw training to related downstream tasks.
How does the difficulty of the visual jigsaw tasks affect the performance?
We conducted ablation experiments to investigate how the difficulty of the jigsaw task affects model perfor- mance. We varied the complexity of both the image and video jigsaw tasks: for the image task, we reduced the grid size from 3 to 2x2; for the video task, we reduced the number of clips from six four. We then mea- sured the average performance across all corresponding benchmarks (using 16 frames for the video evaluation). The results in Fig 3 show that while easier jigsaw tasks still yield perfor- mance improvements over the baseline, the gains are substantially smaller than those from the stan- dard, more difficult tasks. This indicates that higher degree of difficulty provides stronger su- pervisory signal for enhancing fine-grained perception and temporal reasoning. Critically, we also found that challenging setups (e.g. 3 x 3 for images), the design of the partial accuracy reward becomes crucial. Without this design, the model fails to learn the task, as sparse binary feedback is insufficient to bootstrap learning in the early stages of training.
Image Jigsaw
Video Jigsaw
64
Average Porformacce
62
61.0
46.2
45.0
58.9
44
Sa
58
Beseline
242 Jigaww
3+3 Jigaow
Besolina 4-clip Jigsaw 6-clip Jigaaw
Figure 3:
Performance with different jigsaw difficulties on image and video tasks.
Visual Jigsaw on other base models. To validate the general effectiveness of our Visual Jigsaw framework, we apply it on stronger base model MiMo-VLL-7B-SPP 2508 (Xiaomi (025) and show the results Table where Visual Jigsaw also yields consistent gains over the bascline.
Model
Image Avg
Video Avg
3D Avg
MiMo- VL-7B-SFT
63.77 65.14
51.84 54.47
50.67
Visual Jigsaw
52,91
Table 4: Results of Visual Jigsaw MiMo-VL.
Visual Jigsaw on Reasoning MLLMs.
Harder Jigsaw Configurations. We also conduct experiments on harder jigsaw configurations with 4 x 4 for image jigsaw and 8-clip for video jigsaw. For 4 image jigsaw, We find that the model failed to learn this task effectively, with jigsaw accuracy remaining low. This happens even when we do configurrtn adaptation by first training on three key factors: (1) Information Scarcity: At the typical resolution of our COCO training data (8840x480), 4 x 4 tiles are often too small and fragmented to contain sufficient semantic infor- mation for re-assembly. (2) Semantic Ambiguity: Small tiles from uniform areas (e.g., sky, walls) become perceptually indistinguishable, making the ordering task ill-posed (3) Combinatorial Ex- plosion. The permutation space expands from 9! to 16!, significant increase in complexity for the model. This suggests that scaling to harder configurations is not merely a matter of increasing grid size, but will likely require combination of higher-nesolution training data, tile-diversity
jigsaw task. We attribute this to

---

Published as a conference paper at ICLR 2026
straints, and curriculum learning strategies. As for the 8-clip video jigsaw, the average performance 46.24 with 16-frame setting which does not bring notable improvement over 6-clip. This is mainly because many LLaVA-Vidde samples around 30 splitting into 8 clips makes segments short and sometimes ambiguous. We leave the study 8-clip on longer videos for future works.
Table 5: Performance Visual Jigsaw on reasoning-oriente oriented MLLM (ThinkLite VL), showing improved visual perception while preserving reasoning ability.
Vision-Centric
Math Vista
Math Vision testmini
Verhe testmini
MMMU val
EMMA mini
Model
Avg
testmini
ThinkLite-VL 59.69
75.20 75.10
30.92
50.76
55.11 54.22
26.75
Image Jigsaw
61.60
35.20
50.50
29.00
We further explore whether Visual Jigsaw can also benefit reasoning MLLMs that have already undergone reasoning-intensive RL post-training. To this end, we select ThinkLLite-W VL. as the base model and apply the image jigsaw training. We enable the KI. constraint better preserve the reasoning capability during training. We evaluate the resulting model on both vision-centric bench- marks and multimodal reasoning benchmarks, including Vista (Lu et al., 2023), Math Vision [Wang et al. 2024a), Math Verse (Zhang et al. (2024b), MMMU (Yue et al. 2024) and EMMA (Ho et al. 2025). As shown in Tab 5 the reasoning MLLM trained with Visual Jigsaw achieves clear improvements in visual perception and understanding, while maintaining its strong reasoning ability.
How does model solve jigsaw problems? Here we provide example think trace for image jigsaw training Fig[4] We can that the model identifies semantic objects (trees, sky, a person) in different image tiles, infers spatial context (horizon line, pathway edges), and uses logical deduction place the tiles into a coherent global scene. More examples are provided in Appendix A.1
Tile
Tile
Tile
Tile
Tile
Tie
7
Tile 8
Tile 9
<thinio
mentally reassemblingg image pasitioned the upper right corner, showing dense cluster of mowwccveree trees with dark, shadowed
spatial relationships features
its right. suggest ng edge of forested arcs a hills Tile left portion of the image, appears to be part snowy, overcast sky with tree branches extending from snowy landstape
top-
the side, indicating a continuation the hottom-lett and lower miidiin-Ieff areas, shows a snowy stope with darknr, sharled sect on that transltions Into a lighter, anow-covered g ground, conslstent with a path trail leading Into wooded area, Tile 7, occupying the central end fight-middle parte the image, depicts walking through anowy anvironment.
clear horizon
1, situated at
weering winter
standing on snowy peth wit1 ski poles, likely engaged in pethway, with tile left tile 6 the right, Iramiin Uhe central figure and the trail. This amanngenent suggests a continuous smowy ouidloor scene navigating snowy terrain.
Tilee and
to form sides this
with focus </thinic
6, 5, 8, 4,9, 3</answer>
Figure 4: Example thinking process for solving image jigsaw.
5 CONCLUSION
In this work, we introduced Visual Jigsaw, verifiable self supe-supe post-training framework that enhances vision-centric understanding in MILMs. By formulating visual understanding as an or- dering problem and optimizing with RLVR, visual jigsaw avoids the need for dense visual re- construction and integrates seamlessly into text-only MLLMs. Our experiments demonstrate the generality of this approach, yielding consistent improvements across images, videos, and 3D data in fine- aen-igr perception, temporal reasoning, and 3D spatial understanding. Ultimately, our work highlights the potential of perception-focee focused self- and weakly supervisd tasks as powerful and complementary path toward developing more capable and robust multimodal models.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
This work uses only publicly available datasets that follow established licenses and guidelines. Our method focuses on improving vision-centrie perception and understanding in MILMS without intro- ducing additional risks beyond existing models. As with other MLLMs, potential misuse or biases may arise if training data are not carefully curated. emphasize responsible usage, transparency, and alignment with human intentions to maximize benefits while mitigating risks.
REPRODUCIBILITY STATEMENT
All datasets used the experiments are publicly available, and we provide detailed descriptions of the used datasets and preprocessing information in Sec The evaluation benchmarks and details are provided for each modality in Sec 4 The training setup and hyperparameters are described in 4 Our implementation is based on the open-source verl (Sheng al. 2024) library, with the main modifications including the construction of visual jigsaw data and the reward calculation. The corresponding code is provided in the supplementary materials. To further facilitate reproducibility, we will release the code, data, and models to reproduce all main experiments, along with instructions for running ablation studies and evaluations.
ACKNOWLEDGEMENT
This research is supported by cash and in-kind funding from NTU S-Lab and industry partner(s). This study is also supported by the Ministry of Education, Singapore, under its MOE AcRF Tier MOE-T2EP20223-0002
REFERENCES
Unaiza Ahsan, Rishi Madhok, and Irfan Essa. Video jigsaw: Unsupervised learning of spatiotem- poral context for video action recognition. WACV, 2019.
Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self-supervised learning from images with joint-embedding predictive architecture. In CVPR, 2023.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502. /3923, 2025.
Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. Beit: Bert pre-training image transformers. 2021.
arXiv preprint
Fabio M Carlucci, Antonio D'Innocente, Silvia Bucci, Barbara Caputo, and Tatiana Tommasi. Do- main generalization by solving jigsaw puzzles. In CVPR, 2019.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 9650-9660, 2021.
Mathilde Caron, Neil Houlsby, and Cordelia Schmid. Location-awarl self-supervised transformers for semantic segmentation. In WACV, 2024.
Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. Sft or rl? an early investigation into training r1 ri-kike reasoning large vasion-language models, 2025a. URL https: /ttp///axvvv...////v///e//la.aa/ org abs////4...2...////// 11468
Jiuhai Chen, Zhiyang Xu, Xichen Pan, Yushi Hu, Can Qin, Tom Goldstein. Lifu Huang, Tianyi Zhou, Saining Xie, Silvio Savarese, et al. Blip3-o: A family of fully open unified multimodal models-architecture, training and dataset. arXiv preprint arXiv:2505.0055. 095880 2025b.
11

---

Published as a conference paper at ICLR 2026
Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, et al. Are we on the right way for evaluating large vision-lange lision-aaguaage models? In NeurlPS, 2024.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In ICML, 2020.
Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus-pro: Unified multimodal understanding and generation with data and model scaling. arXiv preprint arXiv:2501. 17811, 2025c.
Yi Chen, Yuying Ge, Rui Wang, Yixiao Ge, Lu Qiu, Ying Shan, and Xihui Liu. Exploring the effect of reinforcement learning video understanding: Insights from seed-bench-rl. arXiv preprint arXiv:2503.24376, 2025d.
Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma. SFT memorizes, RL generalizes: comparative study of founda- tion model post-training. In ICML, 2025.
Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capa- bilities. arXiv preprini urXiv:2507.06261 2025.
Daniel Cores, Michael Dorkenwald, Manuel Mucientes, Cees G. M. Snoek, and Yuki M. Asano. Tvbench: Redesigning video-language evaluation, 2024.
Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias
Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In CVPR, 2017.
Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, et al. Emerging properties in unified multimodal pretraining. arXiv preprint Xrvv:22505. /4683, 2025.
Carl Doersch, Abhinav Gupta, and Alexei A Efros. Unsupervised visual representation learning by context prediction. In ICCV, 2015.
Ruoyi Du, Dongliang Chang, Ayan Kumar Bhunia, Jiyang Xie, Zhanyu Ma, Yi-Zhe Song, and Jun Guo. Vine-grained visual classification via progressive multi-granularity training of jigsaw patches. In ECCV, 2020.
Sri Harsha Dumpala, Aman Jaiswal, Chandramouli Shama Sastry, Evangelos Milios, Sageev Oore, and Hassan Sajjad. Sugarcrepe++ dataset: Vision- lasion-aanguae model sensitivity to semantic and lexical alterations. NeurIPS, 2024.
Kaituo Feng, Kaixiong Gong. Bohao Li, Zonghao Guo, Yibing Wang, Tianshuo Peng. Junfei Wu, Xiaoying Zhang, Benyou Wang, and Xiangyu Yue. Video-rl: Reinforcing video reasoning in milms, arXiv preprint arXiv:2503.2177, 2025.
Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever comprehensive evalua- tion benchmark of multi-modal Iims in video analysis. In CVPR, 2025.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Pciyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501.122: 2025.
Yunzhuo Hao, Jiawei Gu, Huichen Will Wang, Linjie Li, Zhengyuan Yang. Lijuan Wang, and Yu Cheng. Can milms reason in multimodality? emma: An enhanced multimodal reasoning benchmark. In ICML,, 2025.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In CVPR, 2020.
12

---

Published as a conference paper at ICLR 2026
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In CVPR, 2022a.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. CVPR, 2022b.
Wenyi Hong, Yean Cheng, Zhuoyi Yang, Weihan Wang, Lefan Wang, Xiaotao Gu, Shiyu Huang, Yuxiao Dong, and Jie Tang. Motionbench: Benchmarking and improving fine-grained video motion understanding for vision language models. In CVPR, 2025.
Maggie Huan, Yuetai Li, Tuney Zheng, Xiaoyu Xu, Seungone Kim, Minxin Du, Radha Pooven- dran, Graham Neubig, and Xiang Yue. Does math reasoning improve general IIm capabilities? understanding transferability of Ilm reasoning. arXiv preprint urXiv:2507.00432. 2025.
Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-rl: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.067445 2025.
Yuqi Huo, Mingyu Ding, Haoyu Lu, Zhiwu Lu, Tao Xiang, Ji-Rong Wen, Ziyuan Huang, Jianwen Jiang. Shiwei Zhang, Mingqian Tang, al. Self-supervised video representation learning with constrained spatiotemporal jigsaw. 2021.
Aaron Hurst, Adam Lerer, Adam Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21277 2024.
Mengdi Jia, Zekun Qi, Shaochen Zhang, Wenyao Zhang, Xinqiang Yu, Jiawei He, He Wang, and Li Yi. Omnispatial: Towards comprehensive spatial reasoning benchmark for vision language models. arXiv preprint arXivv25006003/35, 2025.
Fanheng Kong, Jingyuan Zhang, Hongzhi Zhang, Shi Feng, Daling Wang, Linhao Yu, Xingguang Ji, Yu Tian, Fuzheng Zhang, et al. Tuna: Comprehensive fine-grained temporal understanding evaluation on dense dynamic videos. arXiv preprint arXiv:2505.20124 2025.
Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia. Lisa: Rea- soning segmentation via large language model. In CVPR, 2024.
Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brah- man, Lester James V Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, al. Tulu 3: Pushing frontiers in open language model post-training. arXiv preprint arXiv:2411.1511 2024.
Dingming Li, Hongxing Li, Zixuan Wang. Yuchen Yan, Hang Zhang, Siqi Chen, Guiyang Hou, Shengpei Jiang, Wenqi Zhang, Yongliang Shen, et al. Viewspatial-bench Evaluating multi- perspective spatial localization in vision-language models. arXiv preprini arXiv:2555.21500. 2025.
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV,
Fangyu Liu, Guy Edward Toh Emerson, and Nigel Collier. Visual spatial reasoning. TACL, 2023.
Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: your multi-modal model an all-around player? In ECCV, 2024a.
Yuanxin Liu, Shicheng Li, Yi Liu, Yuxiang Wang, Shuhuai Ren, Lei Li, Sishuo Chen, Xu Sun, and Lu Hou. Tempcompass: Do video 1lms really understand videos? arXiv preprint arXiv:2403.00472 2024b.
Yuqi Liu, Bohao Peng, Zhisheng Zhong, Zihao Yue, Fanbin Lu. Bei Yu, and Jiaya Jia. Seg- zero: Reasoning-chee guided segmentation via cognitive reinforcement. arXiv preprint 0.Xiv:2503.06520. 2025a.
13

---

Published as a conference paper at ICLR 2026
Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. Visual-rft: Visual reinforcement fine-tuning. arXiv preprint arXiv:2503.01785, 2025b.
Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai- Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255, 2023.
Wufei Ma, Haoyu Chen, Guofeng Zhang, Yu-Cheng Chou, Celso M de Melo, and Alan Yuille. 3dsrbench: A comprehensive 3d spatial reasoning benchmark. arXiv preprint arXiv:2412.07825, 2024.
Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Tiancheng Han, Botian Shi, Wenhai Wang, Junjun He, al. Mm-eureka: Exploring the frontiers of multi- modal reasoning with rule-based reinforcement learning, arXiv preprint arXiv:2503.00 07365, 2025.
Mehdi Noroozi and Paolo Favaro. Unsupervised learning visual representations by solving jigsaw puzzles. In ECCV, 2016
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprini arXiv:2304.07190 2023.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. NewrIPS, 2022.
Xichen Pan, Satya Narayan Shukla, Aashu Singh, Zhuokai Zhao, Shlok Kumar Mishra, Jialiang Wang, Zhiyang Xu, Jiuhai Chen, Kunpeng Li, Felix Juefei-Xu, et al. Transfer between modalities with metaqueries. arXiv preprint 1XXv:2504.002566 2025.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly reward model. NeurIPS, 2023.
Arijit Ray, Jiafei Duan, Ellis Brown, Reuben Tan, Dina Bashkirova, Rose Hendrix, Kiana Ehsani, Aniruddha Kembhavi, Bryan Plummer, Ranjay Krishna, al. Sat Dynamic spatial aptitude training for multimodal language models. arXiv preprint arXiv:2412.07755, 2024.
Ziyao Shangguan, Chuhan Li, Yuxuan Ding, Yanan Zheng, Yilun Zhao, Tesca Fitzgerald, and Ar- man Cohan. Tomato: Assessing visual temporal reasoning capabilities in multimodal foundation models. arXiv preprint arXiv:2410.23266 2024.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi. Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024.
Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409. / 19256, 2024.
Oriane Siméoni, Huy V Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Mare Szafraniec, Seungeun Yi, Michaël Ramamonjisoa, et al. Dinov3. arXiv preprint ar0iv:2508. 10104, 2025.
Alex Su, Haozhe Wang, Weiming Ren, Fangzhen Lin, and Wenhu Chen. Pixel reasoner: In- centivizing pixel-space reasoning with auriosity-drive reinforcement learning. arXiv preprint arXiv:2505. 159666 2025.
Siman Swetha, Jinyu Yang, Tal Neiman, Mamshad Nayeem Rizve, Son Tran, Benjamin Yao, Tr- ishul Chilimbi, and Mubarak Shah. X-former: Unifying contrastive and reconstruction learning for milms. In ECCV, 2024.
14

---

Published as a conference paper at ICLR 2026
Tristan Thrush, Ryan Jiang, Max Bartolo, Amanpreet Singh, Adina Williams, Douwe Kiela, and Candace Ross. Winoground: Probing vision and language models for visio-linguistic composi- tionality. In CVPR, 2022.
Shengbang Tong. Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. Eyes wide shut? exploring the visual shortcomings of multimodal llms. In CVPR, 2024.
Chongjun Tu, Lin Zhang, Pengtao Chen, Peng Ye, Xianfang Zeng, Wei Cheng, Gang Yu, and Tao Chen. Favor-bench: A comprehensive benchmark for fine-grained video motion understanding. arXiv preprint arXiv:2503. 250.::2503.199333 2025.
Dianyi Wang. Wei Song, Yikun Wang. Siyuan Wang, Kaicheng Yu, Zhongyu Wei, and Jiaqi Wang. Autoregressive semantic visual reconstruction helps vlms understand better. arXiv preprint (xXiv:2506.090400 2025a.
Guodong Wang, Yunhong Wang, Jie Qin, Dongming Zhang, Xiuguo Bao, and Di Huang. Video anomaly detection by solving decoupled spatio-temporal jigsaw puzzles. In ECCV, 2022.
Haochen Wang, Junsong Fan, Yuxi Wang, Kaiyou Song, Tong Wang, and ZHAO-XIAME ZHANG. Droppos: Pre-training vision transformers by reconstructing dropped positions. NeurIPS, 2023.
Haochen Wang. Anlin Zheng, Yucheng Zhao, Tiancai Wang. Ge Zheng, Xiangyu Zhang, and Zhaox- iang Zhang. Reconstructive visual instruction tuning. In ICLR, 2025b.
Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. Measuring multimodal mathematical reasoning with math-vision dataset. In NeurlPS, 2024a.
Weihan Wang, Zehai He, Wenyi Hong, Yean Cheng, Xiaohan Zhang, Ji Qi, Xiaotao Gu, Shiyu Huang, Bin Xu, Yuxiao Dong, et al. Lvbench: An extreme long video understanding benchmark. 2024b.
arXiv preprint arXiv:2406.0805
Wenbin Wang, Liang Ding, Minyan Zeng. Xiabin Zhou, Li Shen, Yong Luo, Wei Yu, and Dacheng Tao. Divide, conquer and combine: A training-free framework for high-resolution image percep- tion in multimodal large language models. In AAAI, 2025c.
Xiyao Wang. Chunyuan Li, Jianwei Yang, Kai Zhang, Bo Liu, Tianyi Xiong, and Furong Huang. Llava-critic-ll: Your critic model is secretly strong policy model. arXiv preprint arXiv:2509.00676 2025d.
Xiyao Wang, Zhengyuan Yang, Chao Feng, Yongyuan Liang, Yuhang Zhou, Xiaoyu Liu, Ziyi Zang, Ming Li. Chung- Ching Lin, Kevin Lin, al. Vicrit: A verifiable reinforcement learning proxy task for visual perception in vlms. arXiv preprint arXiv:2506. 10128, 2025e.
Xiyao Wang. Zhengyuan Yang, Chao Feng. Hongjin Lu, Linjie Li, Chung- Ching-Chigggii Lin, Kevin Lin, Furong Huang, and Lijuan Wang. Sota with less: Mcts-guided sample selection for data-efficient visual reasoning self- eeff-improvement. arXiv preprint arXiv:2504.009.0 077344, 2025f.
Zifu Wang, Junyi Zhu, Bo Tang, Zhiyu Li, Feiyu Xiong, Jiaqian Yu, and Matthew B Blaschko. Jigsaw-rl: study of rule-based visual reinforcement learning with jigsaw puzzles. arXiv preprint ar XtXxx:25055.2500 2025g.
Chen Wei, Lingxi Xie, Xutong Ren, Yingda Xia, Chi Su, Jiaying Liu, Qi Tian, and Alan L Yuille. Iterative reorganization with weak spatial constraints: Solving arbitrary jigsaw puzzles for unsu- pervised representation learning. In CVPR. 2019.
Penghao Wu and Saining Xie. V*: Guided visual search as core mechanism in multimodal llms. In CVPR, 2024.
LLM-Core-Team Xiaomi. Mimo-vl technical report, 2025. URL URLnttps://axxivvoo///aa 2506. 03569
Xie, Trevor Darrell, Luke Zettlemoyer, and XuDong Wang. Reconstruction alignment improves unified multimodal models. arXiv preprint arXiv:2509.07295, 2025a.
15

---

Published as a conference paper at ICLR 2026
Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. In ICLR. 2025b.
Zihui Xue, Mi Luo, and Kristen Grauman. Seeing the arrow of time in large multimodal models.
arXiv preprint arXiv:2506.03333 2025.
Jihan Yang, Shusheng Yang. Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How multimodal large language models see, remember, and recall spaces. In CVPR, 2025.
Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything v2. NeurlPS, 2024.
Yiyang Yao, Peng Liu, Taacheng Zhao, Qianqian Zhang, Jiajia Liao, Chunxin Fang, Kyusong Lee, and Qing Wang. How evaluate the generalization of detection? benchmark for comprehensive open-vocabulary detection. In AAAI, 2024.
Chun-Hsiao Yeh, Chenyu Wang, Shengbang Tong, Ta-Ying Cheng, Ruoyu Wang, Tianzhe Chu, Yuexiang Zhai, Yubei Chen, Shenghua Gao, and Yi Ma. Seeing from another perspective: Eval- uating multi-view understanding milms. arXiv preprint arXiv:2504. 2025.
Ruifeng Yuan, Chenghao Xiao, Sicong Leng, Jianyu Wang, Long Li, Weiwen Xu, Hou Pong Chan, Deli Zhao, Tingyang Xu, Zhongyu Wei, et al. Vl-cogito: Progressive curriculum reinforcement learning for advanced multimodal reasoning. arXiv preprint arXiv:2507.22607, 2025a.
Zhihao Yuan, Shuyi Jiang, Chun-Mei Feng, Yaolun Zhang, Shuguang Cui, Zhen Li, and Na Zhao. Vidco-grounded
Scene-rl: Video- grounded large language models for 3d scene reasoning without 3d annotations.
arXiv preprint arXiv:2506. 17545, 2025b.
Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, al. Mmmu: massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In CVPR, 2024.
Shuangfei Zhai, Navdeep Jaitly, Jason Ramapuram, Dan Busbridge, Tatiana Likhomanenko, Joseph Yitan Cheng, Walter Talbott, Chen Huang. Hanlin Goh, and Joshua Susskind. Position prediction as an effective pretraining strategy. In ICML, 2022.
Jiahui Zhang. Yurui Chen, Yanpeng Zhou, Yueming Xu, Ze Huang, Jilin Mei, Junhui Chen, Yu- Jie Yuan, Xinyue Cai, Guowei Huang, et al. From flatland space: Teaching vision-language models to perceive and reason in 3d. arXiv preprint arXiv:2503.22 22976, 2025a.
Jianrui Zhang, Mu Cai, and Yong Jae Lee. Vinoground: Scrutinizing Imms over dense temporal reasoning with short videos. arXiv preprint arXiv:2410.02763 2024a.
Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Yu Qiao, et al. Mathverse: Does your multi-modal 11m truly see the diagrams visual math problems? ECCV, 2024b.
Yi-Fan Zhang, Huanyu Zhang, Haochen Tian, Chaoyou Fu, Shuangqing Zhang, Junfei Wu, Feng Li, Kun Wang, Qingsong Wen, Zhang Zhang, al. Mme-realworld: Could your multimodal I1m challenge high-resolution real-world scenarios that are difficult for humans? In ICLR, 2025b.
Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li. Video instruc- tion tuning with synthetic data, 2024c. URL https arwiv.iiiiv...... wtw.aiiww....o.ao..ooooooooooore abs 2410. 02713
Yuanhan Zhang, Yunice Chew, Yuhao Dong, Aria Leo, Bo Hu, and Ziwei Liu. Towards video thinking test: A holistic benchmark for advanced video reasoning and understanding. In ICCV, 2025c.
Ziwei Zheng, Michael Yang, Jack Hong, Chenxiao Zhao, Guohai Xu. Le Yang, Chao Shen, and Xing Yu. Deepeyes: Incentivizing" thinking with images" via reinforcement learning. arXiv preprint arXiv:2505.14433 2025.
16

---

Published as a conference paper at ICLR 2026
Jinghao Zhou, Chen Wei, Huiyu Wang, Wei Shen, Cihang Xie, Alan Yuille, and Tao Kong. ibot: Image bert pre-training with online tokenizer. arXiv preprint arXiv:2111.0832, 2021.
Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Yuchen Duan, Hao Tian, Weijie Su, Jie Shao, al. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprini arXiv:2504. 10479, 2025a.
Nannan Zhu, Yonghao Dong. Teng Wang, Xueqian Li, Shengjun Deng, Yijia Wang, Zheng Hong, Tiantian Geng, Guo Niu, Hanyan Huang, et al. Cvbench: Evaluating video synergies for complex multimodal understanding and reasoning. arXiv preprint arXiv:2508. 19542, 2025b.
17

---

Published as a conference paper at ICLR 2026
A APPENDIX
USE OF LARGE LANGUAGE MODELS
In this work, have used large language models assist in polishing the writing of the paper. Specifically, the LLM was employed to check grammar correctness and to provide alternative phras- ings or stylistic suggestions for certain sentences. All suggestions were carefully reviewed by the authors, and only adopted after manual verification and modification when appropriate.
A.I QUALITATIVE EXAMPLES OF THINKING PROCESS IN VISUAL JIGSAW
Tile
Tile
Tile
Tile Te
<thinko
montally rasssembiing ing image each a pertial view of atrect orurbar anwrenment, likly ndieating ng uppar-eftt
apatict clationns and conteat of butlding
postioned top
image,
with
buildings and poesiblly reo dentix, commercial struct inee. Tie accupivs the too-r (fit port the imago, si showing cont nuation of the ulban anvi anment with tracs and parts of build ng focade, consistent with street or placa area. Tite 7 appeare to be situated on the left side showi ng people from behind, intereeting near what acoms like market street vendor setua, with boscs and gooda vialbie,
buggesting could marketalase
focus Tile located centrolly
signago, consistent with bury activit erw romment. strect acone with vohicle and pavament, providing sense depch and perspective to the lower right hand oide of the image, complet the street teval view. By integrot ng these dbservttioss deducc the sparisl or canizaation of the image.. </thiii>
right, continuation the
being sold, including produce positioned
loweriight
comtnustion
Karmonrr 8, 6, 2.//rrrwer>
Clip1
C
Clip
Clip
Clips
Cipp 0
<thinks
determine correct we analyzz logical orogression drewing Hello Kitt:: ge* bugine with
"CCipp holking ig pen, than silnta dh awing thu inkial stwep the haud. luz2. CCiijp 4 builde on Clip 2 by adcing ng bow to the head, inz. "Cup g continuee by drowing the eyes, nose, and whickers on the head. vo4, *ril ge* moves to the body, drawing the ma and storting the lower part of the body. In5, "CCp 5 further dlevelops the body, addingg more details to the limbs. Ind, "Clio gea oo rppltttt tha body with add stonal dataila like stripes.
</thints
3,5, Tcleramen
<thinit
<thinio
Lut's anteluce thes iicetle--ip-steeeeeeee "xPuint thh; it's right the foregrounc, compet dircctly
Lat's smmlon this waup--r--slezzirrrtt "PPuint 61h:
1n2. "*Peint gen; than Point *PPunn dt*; It's the well to ridht In. *Puini I1yy the mirror on the left. tn8. "Poont tess the mirrr on the right.
table.In2. "*Point 3"n It's further 1* It's
2*1;
further back but
further back Point 3.1rd. PPoint 2-t
wall the oound....... *Puint 5** It'e also on out e bit higher up.ine. * 4e It's on the wall the left Print 2.
</thanko
<7thinio-
canswer> 1, 3.//rruer>>
canswers 8, de/earswer>
Figure 5: Examples of the thinking process solving visual jigsaw tasks.
In this section, we provide an example thinking process of our model solving the visual jigsaw task in Fig 5 For the image jigsaw task, the model needs to identify the content for each image patch,
18

---

Published as a conference paper at ICLR 2026
imagine the overall scene, and reason the relationships between the image patches to solve the jigsaw puzzle. For the vidco jigsaw, the model identifies the content for each clip and reasons about the conrect chronological order. As for the 3D jigsaw, the model grounds the point marker the actual object, infers the distance between points and the camera, and compares the distances among the points.
A.2 ADDITIONAL EXPERIMENTS ON 3D JIGSAW
Besides the depth ordering task for the 3D modality, we have also explored two variants of 3D jigsaw designs, which we detail below.
(1) View-Motion Matching. Given scene captured from multiple camera poses, we randomly select one view as the anchor view, and sample several candidate views that differ from the anchor while also being diverse from another. For each candidate, we construct natural language description of the ego-motion from the anchor to that candidate (e.g. "move forward 2.0 meters and rotate left by 15 The model is provided with the anchor image, the shuffled candidate images, and the corresponding ego-motion descriptions, and is tasked with correctly matching each candidate view its ego-motion description.
(2) BEV-Pose Matching, We render a bird's s-rde--we- (BEV) image of the scene and randomly select of candidate views with different camera poses. These camera poses are annotated on the BEV image with numerical identifiers. The model is then given the annotated BEV image and a shuffled set of candidate view images, and must correctly match each camera pose its corresponding candidate view.
Table 6: Evaluation of 3D Jigsaw Variants. Comparison of depth ordering, view-motion matching.
and BEV-pose matching tasks on 3D benchmarks.
3DSRBench
ViewSpatial
All-Angles
OmniSpatial
VSI-Bench
SPARBench
SAT-Real
DA-2K
Model
test
test
test
test
test
test
tiny
test
Owen2.5-VL-77
48.66
57.42
36.52
47.56
42.66
37.74
35.75
54.45
Depth Ordering
64.00
58.13
38.62
49.06
45.99
40.64
38.31
71.56
New-MMotion Matching
64.67
55,89
37.94
48.22
44.55
38.97
36.53
60.15
BEV-Pose Matching
62.00
57.17
36.69
48.22
44.22
38.78
34.31
58.99
Analysis and Results. Both tasks are intuitively reasonable, as they explicitly encourage the model to connect 2D visual observations with underlying 3D spatial configurations. However, our prelimi- nary experiments (shown in Tab show that these variants do lead to significant improvements on downstream benchmarks, and overall underperform the depth ordering formulation. We hypoth- esize that this may be due to the relatively weak 3D perception and reasoning capability of current base MLLMs, which limits their ability transfer the learned skills from these complex jigsaw formulations to downstream tasks. Exploring ways to strengthen this foundation and better exploit such 3D-aware self-supervised tasks remains interesting direction for future work.
A.3 VISUAL JIGSAW EXAMPLES
The visual jigsaw task examples for the three modalities are provided in Fig. Fig. and Fig.
A.4 QUALITATIVE EXAMPLES
Some qualitative examples of the model trained with image, video, and 3D jigsaw tasks are shown in Fig. Fig. 10 and Fig.
19

---

Published as a conference paper at ICLR 2026
Image Jigsaw Mentally image output patch raster scan order.
1 5.555 3. 71
Answer 001.11100000
1
3
5
6
8
Answer 22222222 2 8
19 00.33444000 18
Answer
3
8
Figure 6: Examples of the image jigsaw task. Each row shows a shuffled set of patches from an image, where the model is required reconstruct the correct raster scan order. The ground-truth answers are displayed on the right.
A.5 LIMITATIONS AND FUTURE WORKS
While our study demonstrates the effectiveness of Visual Jigsaw across images, videos, and 3D modalities, several limitations remain. First, for both image and video jigsaw, we adopt a stan- dard and relatively simple formulation. Future work could explore more complex hybrid jigsaw configurations, such as jointly partitioning video along spatial and temporal dimensions, or using heterogeneous piece sizes to introduce richer structural constraints. Second, due to computational constraints, we have not scaled the training data and model size extensively; investigating the scal- ability of this self-supe task remains promising direction. Third, some of our 3D jigsaw variants did not yield the expected improvements. We believe that applying these tasks to base models with stronger 3D reasoning capabilities and richer 3D priors could unlock further poten- tial. Finally, beyond jigsaw, worth exploring a broader range of self- and weakly-suppervised vision-centric tasks enhance the perceptual and reasoning abilities of multimodal large language models.
20

---

Published as a conference paper at ICLR 2026
Video Jigsaw
Mentally reconstruct video output the clip Indices the chronological order.
2
3
5
('''5' Answwr
Figure 7: Example of the video jigsaw task. Each row shows clip from the original video image, and the 6 clips are shuffled. The model is required to reconstruct the correct chronological order. The ground-truth answers displayed on the right.
21

---

Published as a conference paper at ICLR 2026
Jigsaw Order the points closest farthest relative
ss '' ' [2] JaMSUy
Answer 1 3 41
Figure 8: Examples of the 3D jigsaw task. The model is required to order the points each image from closest to farthest relative to the camera. The ground-truth answers are displayed on the right.
Fine-grained Perception
Spatiol Understanding
Compositional Understanding
What the color of the woman's handbog?
Decide whether the caption correctly descrbes image. Coption: suitcase is under the cat.
Does the caption describe the image correctly? Caption: white, beige baby under beige/white comforter.
Owenn_5-VL-7B: Black.
Image ligsaw: White.
Qwen2.5-Vi. V-----88. No.
S-Ve22.--VL-8.- No.
Image ligsow Ves.
Image ligsaw
Figure 9: Qualitative examples image tasks.
22

---

Published as a conference paper at ICLR 2026
Which caption best describes video?
Q2wen2.5-VL-7B. B.
Temporal Understanding
girl was front of toy refridgerator before she went nehindd it girl was behind toy refridgerator before she went front of it
Figsaw:
Why does brown dog show belly the end the video for owner pet resting C. f.llnttted soiffed to
Qwen22.5 VL 78: A.
Event Understanding
Video
Which direction does the cyon sphere move in the video?
Qwenn25-VVL-P7 C.
Understanding
left. Up the right. Up left. right
Video Jigpaw A.
Figure 10: Qualitative examples on video tasks.
Distance Comparison
View 1 and View are two different views that represent the same scene, In view, carton the image closer the spot where the comera pasitioned?
Perspective Taking
How many cranes have their heads facing their own right?
A. closer the spot where comero View was positioned B, closer to the spot spot
three
comero was positioned
Qwen2..5 Swwen2.5-VL-7B:
1 and View positioned is
ligsaw:
Qwen2.5-VLL.
3D ligsaw
Relative Direction
Camera Movement
View and View ore two different views that represent the
How did the comero likely rotate when shooting the video? A. rotated left 5, ratated right
View the man holding the bicycle tire is facing the ieft side of the view, bicycle facing A. facing from the spot where comero View was positioned facing to the left side of View 2 C. facing to the spat where camera View 2 was positioned
holding
Qwen2.5-VV-27 5-VL-78.8
Qwen2.5 3-VL-7B:
3D ligsaw: A
Figure 11: Qualitative examples on 3D tasks.
23

---

Published as a conference paper at ICLR 2026
A.6 TASK PROMPTS
Prompt for Image Jigsaw
You are given nine shuffled image tiles that were created by slicing one image into 3*3 grid.
Here are the tiles, each tagged with an index reflecting the current (shuffled) order in which they are shown:
Tile 1:
<image>>
Tile 2: <image>
Tile 3:
<image>
Tile <image>
Tile 5:
<image>>
Tile <image>
Tile 7: <image>
Tile <image>>
Tile 9:
<image>>
Task:
Mentally reassemble the original image, arranging the tiles into the correct 3#3 layout and provide the tile indices in raster-scan order (left-to-right, pop-t-bbttomm) separated by commas.
Answer format example:
J. 3. 9, 2, 4, 8, 6
You FIRST think about the reasoning process as an internal monologue and then provide the final answer. The reasoning process MUST BE enclosed within <think><<think>> tags. The final answer MUST BE put within <unswer><<answee tags.
Prompt for Video Jigsaw
You are given four **shuffled** video clips that were created by slicing one original video into 6 equal-length temporal segments.
Here the clips, each tagged with index reflecting the current (shuffled) order in which they are shown:
Clip 1: <video>
Clip 2:
<video>
Clip 3:
<video>>
Clip 4:
<video>
Clip 5: <ideo>>>
Clip 6: <video>
Task:
1. Mentally reassemble the original video by arranging the clips in their correct chronologi-
cal order (earliest segment first, latest segment last).
2. Output the clip indices in that order, separated by commas.
Answer format example:
2, 1, 6, 5
You FIRST think about the reasoning process internal monologue and then provide the final answer. The reasoning process MUST BE enclosed within <think></thinss tags. The final answer MUST BE put within <answer><<answwr>>> tags.
24

---

Published as a conference paper at ICLR 2026
Prompt for 3D Jigsaw
<image>> You are given an indoor RGB image. Six points are marked on the image with red circular labels (1, 2, ...).
Your task is to order the points from closest to farthest relative to the camera, judging the distance based the center of the red circular marker.
Answer with the ordered sequence of point numbers. You FIRST think about the reasoning process as an internal monologue and then provide the final answer. The reasoning process MUST BE enclosed within <think><<thinks tags. The final answer MUST BE put within <answer>> <unswr>><<answer>> tags.
25


Reviews:
[{"review_id": "0GgEX5XAv5", "reviewer": "Reviewer_Dhxc", "summary": "This paper introduces Visual Jigsaw, a self-supervised post-training framework designed to enhance vision-centric understanding in MLLMs. The method formulates a general visual ordering task, partitioning and shuffling images, videos, or 3D inputs and requires the model to predict the correct permutation via natural-language output without modifying architecture or adding visual generators. Experiments on Qwen2.5-VL show improvements across fine-grained perception, temporal reasoning, and 3D spatial understanding benchmarks.", "questions": "- How sensitive are the improvements to the number of jigsaw pieces (K)? Does performance saturate or degrade with grid size (4×4, 5×5) grids or > 6 video clips?\n\n- The reward combines partial correctness with a discount factor γ = 0.2. How sensitive are results to this value? Did the authors explore other reward formulations ?\n\n- What is the total cost of post-training ?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The jigsaw ordering task provides a simple yet effective way to reinforce visual perception in MLLMs, avoiding generative components.\n\n- Applying the post-training principle to images, videos, and 3D data convincingly demonstrates versatility.\n\n- Evaluation on various benchmarks covering visual perception, temporal reasoning, and 3D geometry provide good empirical support.\n\n- SFT vs RL comparison, jigsaw-difficulty analysis, and transfer to reasoning models (ThinkLite-VL) strengthen the claims.\n\n- The paper is well written and figures are intuitive.", "weaknesses": "- The paper has limited conceptual novelty. The idea extends classical self-supervised “jigsaw” pretexts (Noroozi & Favaro 2016) into RL post-training. The novelty mainly lies in adapting it to MLLM post-training rather than the task itself.\n\n- Missing related works of reconstruction-based methods in MLLMs and discussion in paper: Recent works like X-Former[1] explicitly combines contrastive and masked-reconstruction objectives with frozen encoder and decoders to improve visual understanding during pre-training with less data. In contrast, Visual Jigsaw uses structural ordering without dense pixel reconstruction. The paper should discuss this design trade-off in detail, why post-training with ordering might achieve similar benefits, and whether the two paradigms (reconstruction vs ordering) could be complementary.\n\n- The paper omits discussion on training overhead for Jigsaw compared to SFT.\n\n- Missing comparison to SoTA for image and video benchmarks.\n\n- The paper could further analyze reward sensitivity and scalibility (beyond 3×3 grid/6-clips).\n\n- All results rely on Qwen2.5-VL; would be good to show on other model architectures like LLaVA/Blip to confirm generality\n\n- Missing qualitative analysis. Visual reasoning traces (< think > outputs) could better substantiate the claim of improved visual understanding.\n\n\n\n[1] X-Former: Unifying Contrastive and Reconstruction Learning for MLLMs. Sirnam Swetha, Jinyu Yang, Tal Neiman, Mamshad Nayeem Rizve, Son Tran, Benjamin Yao, Trishul Chilimbi, Mubarak Shah. ECCV 2024", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uLQHSzVvIv", "reviewer": "Reviewer_znHd", "summary": "This paper proposes Visual Jigsaw , a self-supervised post-training framework designed to enhance the intrinsic visual understanding of MLLMs. The core idea is to formulate a visual ordering task, the model is presented with shuffled parts of a visual input and must generate a text-based permutation to reconstruct the original order. This approach cleverly leverages RLVR, specifically using GRPO, to optimize the model. The authors instantiate this method across images, videos, and 3D data . Extensive experiments on a wide range of benchmarks show that this post-training stage significantly improves fine-grained perception, temporal reasoning, and 3D spatial understanding, demonstrating the method's generality and effectiveness.", "questions": "1. Have you tried expanding on the harder puzzle tasks (like 4x4)?\n2. Given that some improvements are modest, the results would be more credible if variance  of evaluation are reported.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper presents an elegant and effective idea. It repurposes a classic self-supervised task (jigsaw puzzles) into a post-training stage for modern MLLMs. The framework is simple, requires no architectural modifications or extra generative modules, and is broadly applicable.", "weaknesses": "The improvement effect of some benchmarks is not significant.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "k8PR57k1CZ", "reviewer": "Reviewer_uvM4", "summary": "This paper introduces a novel post-training task called Visual Jigsaw, which encourages MLLMs to learn the inherent order within visual inputs. An RL-based supervision strategy is employed to train the model. Experiments on image, video, and 3D benchmarks demonstrate that the proposed method significantly improves performance.", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1.\tThe paper presents an effective post-training framework for enhancing MLLMs.\n\n2.\tThe idea of leveraging a self-supervised objective to strengthen the visual understanding capability of MLLMs is novel and promising.\n\n3.\tThe proposed approach achieves strong performance across diverse benchmarks, including image, video, and 3D tasks.\n\n4.\tThe manuscript is well structured, clearly written, and easy to follow.", "weaknesses": "1.\tThe Visual Jigsaw task employs a relatively simple supervised objective. Although the experimental results across multiple benchmarks are promising, the underlying mechanism of how this form of supervision contributes to model improvement remains unclear. It would strengthen the paper to include an analysis of what is learned during post-training and how this supervision enhances visual comprehension.\n\n2.\tThe experiments are conducted on only a single MLLM. Including additional baseline models would help demonstrate the generalizability and robustness of the proposed Visual Jigsaw framework.\n\n3.\tThe Visual Jigsaw post-training is performed using specific individual datasets rather than the original MLLM training data. It would be helpful to clarify the rationale behind this choice and provide guidelines for selecting appropriate post-training data.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BTp6ZCEg3S", "reviewer": "Reviewer_kYxc", "summary": "This paper considers Visual Jigsaw as a general self-supervised task for post-training. Specifically, it has three variants, including image jigsaw, 3D jigsaw (ranking from the closest to the farthest), and video jigsaw. Empirically, initialized from Qwen2.5-VL-7B-Instruct, the proposed method achieves improvements over baselines separately. Moreover, when incorporating with a text-centric reasoning MLLM (ThinkLite-VL), the proposed image jigsaw post-training task still brings improvements on vision-centric benchmarks and maintains the performance on reasoning-oriented benchmarks.", "questions": "N/A", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. This paper is well-written and easy to follow.\n2. The motivation of this paper is clear and reasonable.\n3. Empirical studies are partially sufficient.", "weaknesses": "1. The proposed three types of jigsaw tasks are evaluated *separately*. It is strongly encouraged to combine them and find out whether they can work together.\n2. Qualitative examples of the reasoning pathways of both (1) when solving the proposed jigsaw problems and (2) answering multiple-choice questions are missing. Moreover, the underlying reason why this pre-text task is beneficial for VQA seems to be missing. An analysis of the reasoning pathways might be helpful.\n\nSome suggestions on the claims:\n1. The claim \"While these jigsaw-style approaches provide structural ordering signals, they have generally shown weaker performance compared to more dominant approaches\" is actually wrong. There are actually some great works [1, 2, 3] that demonstrate jigsaw-like self-supervised pre-training tasks are at least comparable with contrastive learning and masked image modeling. Therefore, adding discussions on these works and clarifying the claim is important.\n2. The claim\"Current post-training paradigms are predominantly text-centric, where dense visual inputs are only leveraged to extract sparse cues for text-based reasoning\" is also not that appropriate. Recent o3-like approaches are definitely not text-centric reasoning methods. Discussions should include both early approaches like CogCoM [4] and Dyfo [5], recent approaches like DeepEyes [6], VGR [7], Pixel-Reasoner [8], and TreeVGR [9]. The advantages of visual jigsaw compared with these o3-like approaches should be discussed.\n\n**References**\n\n[1] Position prediction as an effective pretraining strategy. ICML, 2022\n\n[2] DropPos: Pre-Training Vision Transformers by Reconstructing Dropped Positions. NeurIPS, 2023.\n\n[3] Location-aware self-supervised transformers. WACV, 2024.\n\n[4] CogCoM: A Visual Language Model with Chain-of-Manipulations Reasoning. ICLR 2025.\n\n[5] Dyfo: A training-free dynamic focus visual search for enhancing lmms in fine-grained visual understanding. CVPR 2025.\n\n[6] DeepEyes: Incentivizing\" Thinking with Images\" via Reinforcement Learning. arXiv 2025.\n\n[7] Vgr: Visual grounded reasoning. arXiv 2025.\n\n[8] Pixel reasoner: Incentivizing pixel-space reasoning with curiosity-driven reinforcement learning. arXiv 2025.\n\n[9] Traceable evidence enhanced visual grounded reasoning: Evaluation and methodology. arXiv 2025.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)