Title: MiCo: Multi-image Contrast for Reinforcement Visual Reasoning
Venue: NeurIPS 2025
Accepted: yes
Keywords: Visual Reasoning

Chain-of-Thought

LLM

VLM

MLLM
Abstract: This work explores enabling Chain-of-Thought (CoT) reasoning to link visual cues across multiple images. A straightforward solution is to adapt rule-based reinforcement learning for Vision-Language Models (VLMs). However, such methods typically rely on manually curated question-answer pairs, which can be particularly challenging when dealing with fine-grained visual details and complex logic across images. Inspired by self-supervised visual representation learning, we observe that images contain inherent constraints that can serve as supervision. Based on this insight, we construct image triplets comprising two augmented views of the same image and a third, similar but distinct image. During training, the model is prompted to generate a reasoning process to compare these images (i.e., determine same or different). Then we optimize the model with rule-based reinforcement learning. Due to the high visual similarity and the presence of augmentations, the model must attend to subtle visual cues and perform logical reasoning to succeed. Experimental results demonstrate that, although trained solely on visual comparison tasks, the learned reasoning ability generalizes effectively to a wide range of questions. Without relying on any human-annotated question-answer pairs, our method achieves significant improvements on multi-image reasoning benchmarks and shows strong performance on general vision tasks.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces MiCo, a self-supervised framework to enhance multi-image reasoning in Vision-Language Models (VLMs). Current VLMs struggle with linking fine-grained visual cues across images, especially for tasks requiring spatial alignment, object correspondence, or tracking semantic changes. To address this, MiCo leverages contrastive triplet training and rule-based reinforcement learning. Experiments show MiCo outperforms other SOTA methods on multi-image benchmarks and generalizes to single-image tasks.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\nThe paper combines self-supervised learning with reinforcement learning, eliminating the dependency on annotated QA pairs. This approach leverages inherent structural constraints for reward calculation.\n\nMiCo achieves SOTA results on VLM2-Bench, surpassing GPT-4o and other advanced VLMs. It can also generalize to diverse tasks, improving multi-image reasoning and even single-image tasks.\n\nThe paper conducts comprehensive ablation studies to validate the contribution of each component.\n\n**Weaknesses**\n\nThere exists a certain degree of inadequacy in the construction of image triplets. For example, in the lower-left triplet of Figure 2, the first and second images are defined as identical, while the third is defined as different. Yet, compared to the first image, the second image has the person cropped out. While the distinction lies in \"the presence/absence of a license plate,\" I argue that the presence/absence of the person constitutes a more salient difference.\n\nI suspect there may be issues with the authors' SFT setup. Apart from the reported poor performance on the person track, the SFT approach actually underperforms the base model in most cases on both the general and object tracks, exhibiting a significant performance drop. I am curious about how the authors constructed the CoT for SFT. Additionally, I would like to understand why MiCo shows notably lower performance on the person track compared to other reasoning-based models.\n\nThe proposed model does not demonstrate a significant improvement over other SOTA methods (e.g., VLAA-Thinker-7B) on the VLM2-Bench. Moreover, the gains on general datasets are also quite limited.\n\nQUESTIONS:\nWhy do the results reported in Table 3 differ from those originally reported in the papers for MM-Eureka-7B and NoisyRollout-7B?\n\nI would like to know how many samples the authors collected from image edit and video datasets for training, respectively.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a post-training framework named MiCo (Multi-image Contrast), to enhance the multi-image reasoning capabilities of Vision-Language Models (VLMs) through reinforcement learning (RL). The major contributions of this work are its task definition for training and the automated pipeline to construct labeled training data.\n\nMiCo trains a VLM to find two augmented views of the same image in a triplet of three similar images (views). The model is optimized to generate chain-of-thought (CoT) reasoning to determine similarities/differences among the triplet. The training pipeline, including prompt format and reward design, is basically borrowed from the DeepSeek-R1 and its GRPO algorithm. Besides, the paper introduces a rollout augmentation strategy that uses weakly augmented views to rollout and strong augmented views to update the model.\n\nMiCo can constructs labeled data for RL training, without the need for human annotations. It constructs image triplets consisting of two augmented views of the same image and a third similar but distinct image, sourced from video frames (e.g., VidGen-1M) and image editing datasets (e.g., OmniEdit).\n\nExperiments show that MiCo, without relying on manual annotated training data, generalizes effectively to diverse multi-image reasoning benchmarks (e.g., VLM2-Bench, MuirBench) and achieves state-of-the-art results.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n- The training paradigm of MiCo is simple yet effective. The training task of comparing image triplets is conceptually straightforward and simple. Yet, it is effective in enhancing the multi-image reasoning capabilities of VLMs.\n- MiCo can automatically constructs training QA pairs with image intrinsic constraints, significantly reducing dependency on labor-intensive annotations. Also, it essentially implements a self-supervised learning paradigm.\n- It proposes a rollout augmentation strategy which uses weakly augmented views to rollout and use strong augmented views to update the model. This strategy is shown to be effective in improving the model's reasoning capabilities.\n- The trained model demonstrates strong multi-image reasoning capabilities and also generalizes well to general visual language tasks.\n- The ablation experiments are comprehensive and insightful.\n\n\n**Weaknesses:**  \n- The framework only applies to the Qwen2.5-VL-7B model, which rises concerns about its generalizability to other VLMs, including larger models as well as other model series (e.g., LLaMA, Mistral).\n- The paper lacks discussion on the data scaling for RL training. It is unclear the minimum amount of data required to train the model, and whether the model performance continues to improve with more data and more training steps.\n- The framework relies heavily on high-quality contrastive pairs (from videos or editing datasets). Its performance may degrade with less structured or noisier data sources. This also poses a challenge for the framework if we want to scale the training data.\n- The framework may degrade model's performance on person-centric tasks. The paper does not provide sufficient analysis on this aspect.\n\nQUESTIONS:\n1. The paper does not discuss the data scaling for RL training. What is the minimum amount of data required to train the model, and does the model performance continue to improve with more data and more training steps?\n2. In Section 3.3, the paper mentions \"To increase the diversity and balance the difficulties of questions, besides the image triplet, we also\n142 construct image pairs and design the corresponding prompts for comparing two images.\" What is the detail of this process? How does it affect the model performance?\n3. Do the training data include samples with triple views of the same image? In other words, can the labels be \"TTT\" or \"FFF\", or must have twice \"T\" and once \"F\"? Also, can the training data include samples with only two views? How does these modifications affect the model performance?\n4. In table 1, why is Qwen2.5-VL-7B-CoT marked as non-reasoning model?\n5. Does the Prompt Diversity in Table 2 (e) mean using variants of reasoning prompt (system prompt) or variants of user question template, or both (that are defined in Section 3.3)?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
The authors propose a method to enable vision-language models (VLMs) to perform multi-image reasoning. They begin by demonstrating that existing VLMs struggle with robust visual comparison. To address this, they construct training triplets consisting of two augmentations of the same image and a third image that is visually similar but semantically different. Using these triplets, they apply an augmented Group-Relative Policy Optimization (GRPO) approach to refine reasoning trajectories. Experimental results show that the method achieves competitive performance across multiple benchmarks.

STRENGTHS AND WEAKNESSES:
### Strengths

- The idea to perform augmented GRPO on image triplets is novel, addresses a well-defined problem (multi-image understanding and reasoning), and does not require manual annotations.

- The authors demonstrate significant improvements over the baseline, showcasing the strength of the method.

- The authors also conduct extensive ablation studies.

### Weaknesses

- The use of synthetic data and augmentations during training could lead to shortcut learning.

- For reproducibility, it would be important to include more details about how the editing pipelines, such as OmniEdit, were used to construct the data (e.g., what types of prompts were used).

QUESTIONS:
It would be interesting to see more failure cases and corresponding qualitative results. Could the authors also elaborate on the scale of the training data and whether they were manually validated?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 4
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03

comments: SUMMARY:
MiCo proposes a multi-image contrastive learning framework to train visual language models for cross-image reasoning by constructing image triplets (two enhanced versions of the same image + a similar but different image). The model is prompted to generate reasoning procedures to compare these images and is optimized through rule-based reinforcement learning without the need for manually annotated question-answer pairs.

STRENGTHS AND WEAKNESSES:
Strengths:
1. It uses the intrinsic constraints of images as supervisory signals to construct image triplets that are visually similar but have subtle differences, thus avoiding the high cost of manually constructing QA pairs. And Augmented GRPO, which uses weakly enhanced sampling trajectories and optimizes the strategy on strong enhancement.



Weaknesses:
1. The method lacks necessary insights, similar to the multi-image version of NoisyRollout
2. It is necessary to evaluate on more benchmarks to illustrate the superiority of the method and model, such as MuirBench, MMIU
3. Typo： LVAA-Thinking
4. In a sense, I think the results in Table 1 cannot explain the superiority of Mico. For example, Eureka is only trained in a single-image scenario, but it is only 3 points behind Mico, which is trained on multiple images.

QUESTIONS:
refer the weaknesses. I will consider increase my rate within the rebuttal.

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 2
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-07-03