Unveiling the Compositional Ability Gap in Vision-Language Reasoning Model
Authors: Tianle Li
Jihai Zhang
Yongming Rao
Yu Cheng
Venue: NeurIPS 2025 poster ()
Keywords: Multimodal Large Language Models
Multimodal Reasoning
RL for MLLM
Primary area: deep_learning
TL;DR: We introduce ComPABench to evaluate VLM compositional reasoning, showing that existing post-training methods struggle, while enhancing vision-text alignment and using progress rewards improves RL-based compositional ability.

Abstract: While large language models (LLMs) demonstrate strong reasoning capabilities utilizing reinforcement learning (RL) with verifiable reward, whether large vision-language models (VLMs) can directly inherit such capabilities through similar post-training strategies remains underexplored. In this work, we conduct a systematic compositional probing study to evaluate whether current VLMs trained with RL or other post-training strategies can compose capabilities across modalities or tasks under out-of-distribution conditions. We design a suite of diagnostic tasks that train models on unimodal tasks or isolated reasoning skills, and evaluate them on multimodal, compositional variants requiring skill integration. Through comparisons between supervised fine-tuning (SFT) and RL-trained models, we identify three key findings: (1) RL-trained models consistently outperform SFT on compositional generalization, demonstrating better integration of learned skills; (2) although VLMs achieve strong performance on individual tasks, they struggle to generalize compositionally under cross-modal and cross-task scenarios, revealing a significant gap in current training strategies; (3) enforcing models to explicitly describe visual content before reasoning (e.g., caption-before-thinking), along with rewarding progressive vision-to-text grounding, yields notable gains. It highlights two essential ingredients for improving compositionality in VLMs: visual-to-text alignment and accurate visual grounding. Our findings shed light on the current limitations of RL-based reasoning VLM training and provide actionable insights toward building models that reason compositionally across modalities and tasks.

Unveiling the Compositional Ability Gap in Vision-Language Reasoning Model
Tianle Li, Jihai Zhang, Yongming Rao, Yu Cheng
The Chinese University of Hong Kong, Tencent Hunyuan Research
tianleli@link cuhk. edu .kk
Abstract
While large language models (LLMs) demonstrate strong reasoning capabilities utilizing reinforcement learning (RL) with verifiable reward, whether large vision- language models (VLMs) can directly inherit such capabilities through similar post- training strategies remains underexplored. In this work, we conduct systematic compositional probing study to evaluate whether current VLMs trained with RL or other post-training strategies can compose capabilities across modalities or conditions. We design suite of diagnostic tasks
tasks under that train models on unimodal tasks or isolated reasoning skills, and evaluate them on multimodal, compositional variants requiring skill integration. Through comparisons between supervised fine-tuning (SFT) and RL-trained models, we identify three key findings: (1) RL-trained models consistently outperform SFT on compositional generalization, demonstrating better integration of learned skills; (2) although VLMs achieve strong performance on individual tasks, they struggle generalize compositionally under cross cross-modal and cross-tast scenarios, revealing significant gap in current training strategies; (3) enforcing models to explicitly describe visual content before reasoning (e.g., caption-before tefore-thinking). along with rewarding progressive vision-to-ted grounding. yields notable gains. It highlights two essential ingredients for improving compositionality in VLMs: visual-to -o-tett alignment and accurate visual grounding. Our findings shed light on the current limitations of RL-based reasoning VLM training and provide actionable insights toward building models that reason compositionally across modalities and tasks.
Introduction
Recent breakthroughs in large language models (LLMss) have shown that strong reasoning capabilities emerge through RI., exemplified by GPT-o1 [laech et al. 2024) and DeepSeek-RI [Guo et al.) 2025). These models demonstrate impressive performance on complex multi-step reasoning tasks in the language-only domain, revealing the potential of RL-style post-training enhance logical and compositional reasoning [Team et al. 2025 Hou et al. 2025 Shen et al. 2025b). Inspired by these advances, researchers have begun exploring whether similar training paradigms be extended to
VLMs, which integrate visual perception with language reasoning (Zhan et al. 2025 Huang et al.
2025
Hao et al. 2025 Yang et al. 2025 Wang al. 2025).
Previous attempts to apply RL with verifiable rewards to VLMs have shown promising gains on
individual vision-language tasks such visual math solving and object localization (Shen et al. 2225aa Meng et al. 2025 Pan and Liu 2025). However, it remains unclear whether these improvements extend beyond isolated benchmarks more complex, realistic scenarios that require the integration of multiple reasoning capabilities. While the compositional abilities of LLMs have been increasingly
studied in the context of skill composition [Zhao 2024 Xu al. 2024b), the extent to
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Choss modaliy Compostion Abilzz
PT MM
Pon-TTd (PP) Taak
Mub Medal (MB)) Task
Anturacy (N)
Pure-Test Grose-task
T
72
(Toad Slope Asoi)
Rlasaringg
Gäven the Zollenkog
totel canseersllis aneonts
slopes: s0c tanglo
whappe??
stonen in the smagle?
all
9222
Tosk
RL
Big Gap
There are 3 shapes on
Spotal Heesoning (Nearea Grid Posttion
Task B
6ot weist purp
cloxset to the (pupp in traperote?
clewest Poople taapszoud:
carunia klarswocs
Casseer 11, 2ic/mmee
Accurscy (14)
Musii Modiall Crose-taskk
Comprsitonal ANiby
76.6
Thare are staper. Hon trapersts bane Torn srea of the rearect shape and target
shppss
on a dod gxid total area
SFT
Big
(Toul Nexrestt Stupes
Composatinnal Task
target chape? 100c
2,3
Rig Gap
Figure I: Demonstration of the tasks and partial results for probing of cross-modality and cross-task compositional ability.
which VLMs exhibit similar capabilities remains largely underexplored. In particular, it is still an open question whether VLMs can coherently combine skills acquired independently either across modalities (e.g.. transferring textual reasoning to visual inputs) across reasoning domains (e.g.. integrating spatial and arithmetic reasoning) to solve tasks that demand such composition.
To better understand the performance of compositional generalization in VLMs, we investigate two core dimensions: cross-modas and cross-task reasoning. We focus on the following research (R)): RQ1 Can reasoning abilities acquired through pure-text training be composed with visual recognition solve multimodal reasoning tasks? RQ2 Can independently learned visual reasoning skills be integrated to tackle composite tasks that require both capabilities? RQ3 Can such compositional ability generalize to out-of-distribution (OOD) variants with altered task objectives? To support this investigation, we design set of diagnostic tasks and carefully curated training and evaluation splits, each aimed at isolating specific challenges such as cross- cross-modal reasoning, visual skill composition, and generalization to new task settings as partially demonstrated in Firgure [
We conduct comprehensive experiments across multiple post-training strategies to evaluate the compositional capabilities of VLMs. Our study yields three key observations: (1) RL-trained models consistently outperform SFT in compositional settings, particularly for cross-task generalization; (2) despite strong performance on individual tasks, VLMs exhibit significant limitations in compositional reasoning under multimodal input; and (3) explicitly structuring the reasoning process through visual- to-text prompting and reinforcing intermediate progress reward [Luo et al. 2024) lead to substantial gains in compositional performance.
In nutshell, our contributions to this work can be summarized as follows:
We introduce ComPABench, diagnostic benchmark that systematically evaluates compositional generalization in VLMs across modalities, reasoning tasks, and distribution shifts.
We conduct comprehensive empirical analysis of post-training strategies, and reveal their limitations in both cross- modal and cross-task compositional generalization.
We identify a simple yet effective solution, RL-Ground, that helps reduce the compositional gap in existing post-training strategies by aligning visual inputs text before reasoning and rewarding accurate grounding of visual content during intermediate reasoning steps.
Together, we hope these contributions can lay a foundation for advancing VLMs toward more robust multimodal reasoning with stronger compositional generalization.
Related Work
2.1 Post-training for VLM Reasoning
Following the success of reasoning-oriente I.LMs such as GPT-ol [Jaech et al., 2024), significant efforts have been devoted to developing advanced reasoning capabilities in VLMs after supervised

---

fine-tuning over various fundamental visual tasks Wang et al. 2024a Chen et al. 2024 Team et al.
2025 Wu al. 2024, Abouelenin et 2025]. Early approaches range from manually y designed
structured reasoning Xath et al. [2024a] to tree-based search strategies Xu et al. [0024a] Yao
et al. 2024]. The breakthrough of Deepseek-RI [Guo al. 2025] outcome- based reward with GRPO [Shao et al., 2024] for LLMs has inspired attempts to adapt this paradigm to VLMs.
However, directly transplanting Deepseek-RI Is training methodology to VLMs has proven ineffective.
Huang al. [025 identify two primary failure modes: (1) the learning algorithm struggles to obtain meaningful positive rewards for complex samples, and (2) models tend to bypass visual inputs and rely solely on textual cues for reasoning. Similarly, Zhan et al. 2025 demonstrate that direct application of outcome reward RL fails to enhance VLMs' reasoning performance. While Du et al. [025] partially validate the transferability of text-based reasoning capabilities multimodal tasks, their analysis remains confined to mathematics-oriented domains. Current VLMs, particularly open-source implementations, still exhibit significant gaps in multimodal reasoning compared to human-level performance Hao et al. 2025
2.2 Generalization Probing for Training Strategies
The post-training phase of large models, particularly the choice between supervised fine-tuning and reinforcement learning, has significant implications for generalization. Several recent works
have sought to explore these effects for different strategies respectively (Wang et al.] [024bb Zhao
al. 2025) Chu al. [2025] conducted a systematic comparison of SFT and RL, finding that SFT often leads to memorization of training patterns, whereas RL enables stronger generalization
by encouraging models to discover and apply more transferable principles. Kirk al. 2023) also confirmed that RLHF-trained models exhibit improved robustness to distribution shifts at the cost of reduced response diversity. In contrast, Yue al. 2025) revisits the widely held belief that RL with verifiable rewards (RLVR) enables LLMs to acquire fundamentally new reasoning abilities, showing instead that RLVR primarily reweights the model's existing reasoning distribution rather than expanding it, which limits exploratory capacity despite improved efficiency. Our work differentiates from these works by systematically analyzing the limitations of current post-training strategies when applied to VLMs, focusing specifically on their compositional generalization capabilities across modalities, tasks, and autto-ddistribution settings.
Preliminaries
To ground the experimental design and training protocols used in this work, first formalize the three training paradigms employed throughout our evaluation: Supervised Fine- Fine-Tuning (SFT), Reinforcement Learning with Verifiable Reward (RL), and RL training initialized from an SFT- trained checkpoint (SFT-init RL). We also detail the learning objectives used in each setting, with a particular emphasis the Generalized Reinforcement Policy Optimization (GRPO) strategy adopted in DeepSeek-RI training [Guo et al. 2025
3.1 Supervised Fine-Tuning
Supervised fine-tuning aligns pretrained VLM with the target task distribution using paired data (x, y). where is the input prompt and y = (y1,--.; Ur) is the corresponding output sequence. The model is trained to minimize the negative log- likeliiol (NLL) of the target output under the causal language modeling objective:
(SF Σ log (o Vet)
(I)
This objective encourages syntactic correctness and semantic alignment with human-provided exam- ples. our setup, & may contain text-only format, or both image and text formats. y includes both a reasoning trace (enclosed in <think> block) and a final response (enclosed in a <answer> block).
3

---

3.2 Reinforcement Learning with GRPO
adopt Group Relative Policy Optimization (GRPO) as the optimization strategy our RL training framework. GRPO generalizes token-level policy optimization with structured per-sample seward-to-avvantaee computation and includes KL regularization term to the reference policy Tel (typically the SFT model).
Let G denote the number of generated candidate answers for given question q, and |o denote the number tokens in the i-th output ( The GRPO loss is defined as:
Te To(o001 q, 10.00 A(i,t) B- KKL(TO |l T a 9, 0
(GRPO(0) = G [
(2)
Here, A(i,t) is the estimated advantage at time step &. for the i-th sample, and / controls the strength of the KL regularization term. The inner term represents a scaled policy gradient with a reward signal modulated at each token step.
In practice, ti, is typically derived from scalar reward r(g, " measuring task success, which may combine:
Answer correctness: Whether the generated prediction matches the ground-truth answer.
Format adherence: Whether the output satisfies the specified constraints of format.
This formulation ensures that the model improves generation quality while maintaining consistency with the reference distribution.
3.3 SFT-Initialized RL Training
To stabilize and accelerate the RL training process, we explore a hybrid strategy where reinforcement learning is initialized from model pretrained with SFT, similar R [[Goo al. [2025]. In this setting, the reference policy æo used in the KL term is set to the SFT-trained model, and the initial parameters Og of the policy Ty are inherited from the same checkpoint. This strategy offers two key benefits: (1) it enables faster convergence by leveraging prior alignment to task distributions, and (2) mitigates early-stage instability common pure RI. setups.
Together, these training paradigms define the backbone of our experimental pipeline, enabling us to probe the strengths and failure modes of VLMs compositional, multimodal, and generalization- intensive reasoning settings.
4 Experiments
To assess compositional generalization in VLMs under different post-training strategies, conduct experiments from three perspectives: cross-modal composition, cross-task reasoning, and out-of- distribution generalization. We first present ComPA Bench, diagnostic benchmark, followed by training setups and evaluation results addressing the three research questions.
4.1 Benchmark for Probing of Compositional Ability
Task Type
Training Subset
Test Subset
Crdal Composition
PT-GR, PT-SR
PT-GR, PT-SR, MM-GR, MM-SR PT-GR, PT-SR, PT-Comp
Cross- Tross-Task Composition (Purc-text)
PT-GR, PT-SR
Cross-Tias Task Composition (Multimodal)
MM-GR, MM-SR
MM-GR, MM-SR, MM-Comp
OOD Composition
MM-GR, MM-SR MM-GR-OOD, UM-SR-OOD, MM-Comp-OOD
Table I: Statistics of our proposed ComPABench for different task settings. The abbreviations of different types of dataset can be found in the right bottom of blocks in Figure

---

RQ1
Cres-mabiity Compusnional Ability
Pure-test Task
Muhs-oedl
Gormaric
beight Lrapezoid beses the total area all shapes?
following shtpess: cectangle
2kb=2 Mhat is total axea of shapes shown the Inage,, please round the resu into the nearest integez? <ansmer>3B<<anaeee
Reasoning
heightt: 7.
(StapeArea)
Please zound the zesult to the neaxest integez. cansier33k//ansee
PTGR
MM GR
Theze are shapes on a 4x4 grid: a puzple traperoid position (0,2), a cyan tropecoid at position 2), a grey rectangle at position (3,8). Which shape is
Theze axe shapes on Which shape closest the purple traperoid in torms of Karhattan dit Lance?? The final answer is the grid position of the closest shape like (row_ index, colurnn index)
grid.
Spatial
Rensming
closest to the purple trapecoid in tezes of Manhaltan distance? The final the closest <answerd (1.2)//a(..)
(Grst Position)
is grid position of indax, calumn, (ndax).
RQ2
PT-SR
canseer> (1.))er)(..)///aaaeee 2)////meer>>>
MM-SR
Compondional
Theze aze shages a 6x5 gEid: zed trapezold at positiio (2, 4) with base 8 and haight 4, blue position (4, with side 6, Find shapa nearest to red trapezoid at the nearest shape and the target shape? Please round the result into the nearest integerr canser>185c/anaae
These are shapes on 6x6 grid. Find the shape that nearest to the red trapezoo id teres Kanhattan distanca, what is the total azes of the nescest shape to the tazget shape and the
Alilty
positiion 2) with
Composttionl Task
Manhattan distance.
atsa
target shape? Please round the zesult into the
integer.
PT-Cup
consvero (arrseee155///aaae
MM-Camp
OOD Independent Tisks
000 Compustional Take
RQ3 0OD Geomeriic Reasoning
the largest azea anong shapes shown in the image, please round the result into the neaxest anteger?
There shapes the shape that farthest from the yellow square in terms of Nanhartan distance, what is the azea of the shage with the larger area between Flease round result into the neazest integer.
a 5x6 grid.
MM-0R-000
Theze are shapes on 6x6 grid. Mhich shape is farthest from the red square in torns of Manhattan distance? The rinal answee is the geid position of the farthest shape like (zon_index, volum__idex)..
0CD Spatil Rasssing
these
(answerD (4,9) 8) c/answer>
Figure 2: Demonstration of proposed ComPABench for RQ1, RQ2, and RQ3.
To systematically evaluate the compositional ability of VLMs trained under different post-training strategies, design a set of finely controlled tasks aligned with our three research questions, where we call this benchmark as ComPABench. Each task setting is implemented with paired pure- text and multimodal variants to allow cross- cross-mode moddaltyyllty and cross task comparisons. Figure 2 iillustrates the benchmark design across individual and compositional tasks, together with the corresponding OOD variants for evaluating transfer robustness.
Cross-Modal Compositional ability (RQ1). To evaluate whether reasoning abilities acquired from pure-text training can transfer visual inputs at inference, we construct parallel task formats with matched semantics but differing input modalities. In the geometric reasoning task, the model computes the total area of multiple shapes described either in pure-text or shown in an image with labeled dimensions. In the spatial reasoning task, it identifies the grid index of the shape closest to given target, based on either textual position descriptions or an image depicting a grid with embedded shapes. By comparing performance across modalities, we assess the model's 's ability to compose textual reasoning with visual perception.
Cross-Task Compositional ability (RQ2). We probe whether models integrate independently acquired skills by composing geometric and spatial reasoning in a single task. Here, we evaluate whether models trained on each skill individually computing the total area of a target shape and the shape closest to the target. We test VLM under both pure-text and multimodal settings to compare the compositional ability of different input types.
solve questions requiring both, such as
Compositional OOD Generalization (RQ3). To evaluate whether compositional reasoning extends to variants of seen tasks, we develop OOD tasks that modify the objective of individual task, so as the compositional ones. For example, instead of asking for the total area, the model must identify the largest area (for geometric reasoning), or select the farthest shape rather than the nearest (for spatial reasoning). In the compositional OOD setting, models are asked to perform combined tasks using these novel objectives (e.g., compute the area of the larger of between the target shape and the farthest shape of it), probing compositional ability in more challenging setting.
5

---

This benchmark provides unified and controlled evaluation for diagnosing cross cross-modal and cross- task generalization, as well as robustness distributional shifts. We present the detailed composition for each of the tasks in Table D More specifically, we generate 4K samples for each individual type of data training and 500 samples for evaluation. For instance, for Cross-Moder Composition task, we mix 4K PT-GR and 4K PT-SR data to train VLM, and test it on 500 PT-GR, 500 PT-SR, etc. The proposed ComPABenue directly supports our investigation into the compositional abilities of current reasoning VLMs under different post-training strategies. More details about the construction of ComPABench is provided the supplementary materials.
4.2 Experiment Settings
To evaluate the effect of different post-training strategies on compositional generalization, we conduct all experiments using backbone models: (wen2.5-VVL-3--Insttre and -ron2255-VL-7--Inttrree
For training configurations, we apply consistent hyperparameters: per-device batch size of 1, learning rate of 1c-6, and a total of training epoch. For RL experiments, we generate 8 completions per prompt in training. And we set the scale before KL divergence constraints 0, as observe dramatic performance degradation with KI. divergence in the objective of optimization. All experiments are conducted NVIDIA H100 GPUs. Our implementation GRPO based on the open-source R1-V
4.3
Evaluation Result
In this subsection, we present experimental results answering the three research questions introduced previously.
4.3.1
RQI: Cross-Modality Compositional Generalization
Shape Area Task
Grid Position Task
100
36 Image Format
36 Pure Toxt
23.8
100
28 Pure Toxt Image Tormar
95.2
50.2
93.5
Pure-Text
Pure- Text
76A
Accuracy 1%)
70.6
Accuracy
40
26.2
14
182
20
Original
SFT-trained
RL-trained
Original
SFT-trained
ALitrained
Figure 3:
Performance comparison when post-trained with pure-text and evaluated with either pure-text or mongge-frmat(mmlli-nn -oolll) questions.
Pure-text to multimodal generalization gap. We first evaluate whether reasoning skills acquired from pure-text training transfer effectively to visual input at inference time as shown in Figure 3 While the original Qwen2.5.V VL models (without post-training) already show high accuracy the proposed pure-text tasks, SFT boosts performance in the pure-text modality generally, reaching wear-perfect accuracy for the grid position task (99.2% for 3B and 99.8% for 7B), and maintains similar performance or improves moderately for shape area task. However, the models trained solely on pure-text data fail dramatically when tested the corresponding multimodal tasks, dropping sharply to 13% (3B) and 16.2% (7B) on shape areas, and to just 4.8% (3B) and 4.2% (7B) on grid positions. This large accuracy gap (exceeding 94 points in the worst case) indicates that purely textual training alone does not inherently enable visual reasoning for SFT post-training, despite semantic alignment between tasks.
Moderate improvement with RL training. RL generally achieves competitive performance com- pared to SFT in the pure-text modality after pure-text training, with only notable exception
http:::://gtttub..om/eeeeeeeepe Agent/RI--

---

Performance Trend with Pure-Text Initialization Training
3B Model
7B Model
SFT Shape Area
SFT Shopt Anb
40.3
20. 38. 52.
RL Shape Area
522
RL Shape Area
90
SPT Orid Pas
NL Grid You
794 12.
RL Grid Pos
SPT Grdd Pos
76.2
14.0
Acsunny PM
60
61.
40
20
44
Original
NRS-Trained
PH-int-MM--Triieed
Original
MK-Trained
Pi-nii-WW-Triieed
Figure 4: Trend of multi-modal performance without and with pure-text training initialization.
where RL underperforms SFT (82% vs. 99.2% on 3B grid position tasks). Despite this, RL enhances multimodal accuracy compared to pure-text-only SFT models. For instance, RL improves multimodal shape area accuracy from 20.8% to 28.0% (7B). yet still far below the pure-text setting. On the grid-position multimodal task, RL yields only modest improvements (6.2% for 3B and 5.2% for 7B), barely surpassing the performance of the original base model. These results indicate that while RI. enables partial compositional generalization from reasoning skills acquired on text to multimodal visual tasks, its effectiveness remains limited when trained exclusively on textual data.
Impact of initializing multimodal training with pure-text priors. To better understand how pure-text training influences subsequent multimodal reasoning, further examine models initialized with pure-text reasoning priors before multimodal training as shown in Figure 4.. In this scenario, initializing multimodal RL with models already trained on pure-text data significantly enhances visual task performance. Specifically, for the 3B wrid-position task, accuracy increases substantially from 49.6% (direct multimodal RI. to 64.4% (text-initialized multimodal RL... In contrast, the beneficial effect of pure-text initialization is minimal or even slightly detrimental for SFT (91.8% versus 89.6% on Grig Position for 7B model). This discrepancy between RI. and SFT likely arises because, during multimodal SFT training, the reasoning path is explicitly provided in the <think> block preceding the answer, whereas RL must discover the correct reasoning path solely from final-answer supervision. Therefore, initializing RL with text-trained models, where semantics closely match, may help the model more efficiently converge toward the appropriate reasoning strategy.
These findings collectively confirm that pure-text reasoning capabilities, even when trained to near perfection, do not automatically generalize to multimodal inputs. RL-based training provides moderate improvements over pure-text SFT but is insufficient by itself. Importantly, initializing multimodal RI. from a pure-text reasoning model can enhance performance, suggesting an effective strategy for scenarios where multimodal data is limited or costly.
4.3.2 RQ2: Compositional Reasoning from Independently Acquired Skills
Pure-text compositional performance. the pure-text setting (left column Fig. 5). the original Qwen2.5-VI. models (without additional post-training) already exhibit moderate compositional capabilitics, achieving 49.4% accuracy (3B) and 46.4% accuracy (7B). However, SFT on individual geometric and spatial reasoning tasks separately severely impairs compositional accuracy, dropping performance drastically to just 0.6% (3B) and 2.2% (7B), despite nearly perfect accuracy on each sub-skill individually. This catastrophic forgetting indicates that standard SFT actively disrupts the model's inherent compositional capability. In contrast, RL with a final-answer reward substantially improves compositional reasoning, raising accuracy significantly to 93% (3B) and 81.2% (7B). Thus, RL effectively preserves and enhances compositional generalization in text-only setting for VLMs.
Multimodal compositional performance. In the multimodal setting (right column of Fig. 5). the original models (without any multimodal post-training) struggle significantly, achieving low accuracy (5.8% for 3B and 13% for 7B). Similar to the pure-text case, multimodal SFT also fails, reaching only
7

---

Compositional Reasoning (Text-Only Setting)
Compositional Reasoning from Independently Acquired Skills (Muitimodal Setting)
Stape
Stape Anco
Compoiltinas
Compecttions
TEA
16.8
40
00
23:3
Owant XL-25 Instuct
Selcwaned
Nuiaandd
Sefioaasd
ALtrunad
SPTcnd
Shaps Areo
Spatial
Wid
99
6B4
Compan Biona
18.6
612
Over2.5VL -B-.rruu
SPTrolneed
1Liruined
SPTinA
Cruuddddruuh
Figure 5: Comparison of Results for compositional reasoning from independently acquired skills.
2.2% (3B) and 7.2% (7B), indicating that standard SFT alone does not enable effective cross-task multimodal compositional reasoning. While multimodal RI. training improves upon SFT, achieving 17.4% (3B) and 31.2% (7B), remains far below pure-text RI levels, highlighting inherent challenges in multimodal compositional reasoning for cross-task scenario.
Limitations of SFT-init RL. Initializing multimodal RL training from SFT checkpoint (SFT-init RL) does not enhance compositional performance; instead, accuracy remains extremely low (2.6% for 3B, dropping to 1.0% for 7B). This indicates that RL struggles correct flawed compositional strategies established by prior SFT training. We attribute this issue primarily to our hybrid training strategy, which alternates evenly between SFT and RL updates (half-half half step attribution). Although SFT-init successfully boosts RL performance on individual tasks as expected due to explicit reasoning path supervision during SFT, it simultaneously imposes strong biases that limit RL's flexibility in adjusting compositional strategies. Consequently, while SFT-init can facilitate faster learning of individual skills, it may inadvertently hinder compositional generalization across tasks compared to RL trained without SFT initialization.
Impact of progress-reward grounding (RL-Ground). Motivated by these failures, explore strategy that explicitly targets visual-to-text alignment and reasoning decomposition. Our proposed potential solution, RL-Ground, combines two key components: (1) <caption> block that forces the model to first describe visual content natural language before entering the reasoning stage, and (2) fine- grained progress reward that provides supervision the level of intermediate vision- grounded reasoning (c.g., correct shape area computation or distance estimation), rather than only at the final answer.
Mhat total shapes
Progress Reward: 5/3 +2/3
Accuracy
coaptionnA blue 3xd blue tziangla, puiple trapezoid laboled 2, had and and yellow square labeled an2 on shire backgroundic captions
<thinlo Triangle Area- (aab(abbb/2233344)..2
D) CARSWOED 26 </ AnSIBEE
Trapezcie Area (ai+a2) (2t4))/22222))))/)))))))
2.4.16
Square Areass92-29-11 Total Arca=6=16ed:26 </thinio
Format Reward: scaption>..... h/wp:///wwwcotiioottii.......... Peeee wtw.://hii......./ttaaaaeeeeee..
Figure 6: Illustration RL-Ground framework.
As shown in Fig. 6 the <caption> module promotes an early transformation of visual inputs into language, triggering the reasoning together with text instead of merely visual inputs. Simultaneously. the progress reward allows the model to build up compositional reasoning over verifiable subgoals, reducing reliance sparse final accuracy reward signals. This structure leads to significant gains: RL-

---

Ground achieves 28.6% (3B) and 52.8% (7B), surpassing the other evaluated post-training strategies. We provide more ablations and training progress on RL-Ground in the supplementary materials.
These findings clearly illustrate that cross-task compositional generalization is challenging under multimodal settings. Simple exposure independently trained sub-skills via standard SFT RL alone proves insufficient, and even harmful (in the case of SFT). Instead, caption-before-lnn formats coupled with dense progress reward significantly improve multimodal compositional reasoning, offering a promising direction for future training paradigms.
4.3.3
RQ3: Generalization to OOD Compositional Task
RQ3 3B Models
RQ3 7B Models
100
96
Method
Original
83.2
03.0
SFT
Accuracy (%)
74.2
RL
RL-Ground
60-
41
44
38
45
40.4
27.8
26.4
20
14.4
10.0
1.2
6.6
1.4
4.4
Shape Area 000
Grid Position OOD
Compositional 00D
Shape OOD
Grid Position COD
Compesitional OOD
Figure 7: Comparison of Results for generalization to OOD independent and compositional tasks.
SFT vs. RL under OOD generalization. We evaluate model robustness on OOD tasks that modify the reasoning objective while maintaining similar multimodal inputs. Results show that SFT exhibits task-dependent generalization: it completely fails on the largest-area task (1.8% for 3B and 1.4% for 7B), but achieves moderate accuracy on the grid position of the farthest shape task (74.2% for 3B and 27.8% for 7B), suggesting limited transferability when visual structures closely align with training data. However, it fails on the OOD compositional task (1.2% for 3B and 2.8% for 7B), reaffirming its lack of compositional flexibility. In contrast, RL generalizes strongly to independent OOD tasks, achieving 95.8% (3B) and 95% (7B) on the largest-area task, and 44% (3B) and 83.8% (7B) on the farthest shape task, comparable to or better than in-domain individual task results from Fig. 5 For the OOD compositional task, RL reveals scale-dependent trend: while it performs poorly on 3B (6.6%), the 7B model generalizes better, achieving 40.4% and surpassing its in-distribution compositional performance. These results indicate that while SFT's generalization brittle and task-specific, RL better supports abstract reasoning transfer, particularly larger model scales.
RL-Ground achieves robust generalization. RL-Ground consistently achieves high accuracy across all OOD tasks. For individual OOD task, it outperforms all other methods, reaching 83.2% (3B) and 88.6% (7B) on the farthest shape task(Grid Position OOD). RL-Ground performs merely slightly behind the original RI. method in Shape Area OOD task. Notably, RI.-Grunnd also demonstrates best performance on the OOD compositional task, achieving 38% on 3B and 52.8% on 7B model, both matching exceeding its in-domain compositional performance. These results confirm that combining caption-before-eee ability, but also significantly improves robustness to unseen task objectives.
with progress reward not only enhances in-distribution compositional
In nutshell, while SFT exhibits limited and inconsistent generalization under OOD shifts, RL gener- alizes well to new reasoning objectives, particularly at larger model size. RL-Ground demonstrates the strongest and most stable generalization across all settings, showing clear advantages for both individual and compositional OOD tasks.
4.4 Generalization and Grounding Analysis of RL-Ground
While RL-Ground was developed and evaluated on synthetic tasks from ComPABench to facilitate controlled analysis, the techniques it introduces are broadly applicable to real-world VQA settings. In particular, the use of explicit visual grounding via the <caption> step and step-wise progress

---

reward are not bound to specific data domains. These mechanisms address fundamental challenges in VLMs such as compositional reasoning, grounding robustness, and sparse reward learning.
To validate that RL-Groundd is not overly tailored to synthetic setups, we conduct two complementary studies. First, introduce two auxiliary visual grounding tasks, Shape Area Grounding and Grid Position Grounding. The two tasks isolate grounding ability from high-level reasoning. In Shape Area Grounding, the model is prompted with question such as "What the area of the red triangle?" requiring it to detect the correct shape, recognize its geometry, and compute the corresponding area. Grid Position Grounding challenges the model to locate an object based on color and shape and return its precise row and column index on grid. Both tasks demand perception and test whether the model extracts the relevant visual facts without performing full reasoning chains.
Table reports results across grounding and reasoning tasks for all evaluated strategies. RL-Groonnd achieves high scores in both grounding subtasks with 96.2% on Shape Area Grounding and 88.6 38.6% on Grid Position Grounding, demonstrating its superior visual grounding alignment via the <caption> step and step-wise reward optimization. These improvements directly translate into the highest compositional reasoning accuracy (52.8%), validating the hypothesis that strong grounding enables generalizable multi-hop reasoning.
Interestingly, while standard RL already provides significant gains over SFT with substantial boost in SA Grounding (96.6% vs. 1.2%) and GP Grounding (74.8% vs. 65.4%), the RL-Ground variant consistently outperforms both, indicating the added benefit of incorporating explicit grounding and structured rewards. This performance trend mirrors the results on the compositional task, where RI.- Ground again leads by a wide margin, affirming that improved visual grounding tightly correlated with stronger generalization and reasoning capabilities.
Table 2: Evaluation of VLMs across grounding and compositional reasoning tasks. "SA" = Shape Area, "GP" = Grid Position. RL-Ground achieves the strongest visual grounding and generalization.
Model SFT
SA
SA Grounding
GP
GP Grounding
Compositional
20.8
89.4
24.4
13.8
73.6
1.2
91.8
65.4
7.2
RL
74.6
96.6
83.2
74.8
31.2
SFT-init-RL
76.6
1.2
89.8
43.6
1.0
RI-Ground
73.8
96.2
88.4
88.6
52.8
Beyond ComPABench, we assess whether RL-Ground generalizes to real-world language patterns. We perform zero-shot evaluation on the Geometry3K dataset [Lu et al. 202T]. which is a natural language-based VQA benchmark involving spatial and geographic reasoning. RL-Ground, trained only on synthetic data, achieves 21. 21.2% accuracy, outperforming the base Qwen2.5-VL-3B-Inset (14.8%), SFT (9.2%), and RL-only (17.8%) models. This demonstrates that caption-before.e beaoorereasooinng and progress reward confer robustness that transfers across tasks and domains.
Conclusion
We introduce ComPABench, benchmark for evaluating compositional ability in VLMs across cross-modal, cross-task, and OOD settings. Inspired by recent RLVR progress in language models, assess whether similar training strategies improve compositional reasoning in VL.Ms. Through comparisons of SFT, RI,, and SFT-initialized RI, we find that RI. better integrates independently learned skills, especially in cross-task and OOD scenarios. Yet, compositional reasoning with visual inputs remains challenging, Our proposed RL-Ground strategy, combining caption-befored reasoning and progress rewards, yields strong in-distribution and -iito--disttibution gains in terms of compositional ability across tasks, underscoring the value of structured prompting and grounded supervision for improving the compositional generalization of multimodal reasoning. To further understand its effect, we evaluate RL-Ground on auxiliary visual grounding tasks and find that substantially improves grounding fidelity compared to baselines, which in turn correlates with higher compositional performance. Moreover, RI-Ground generalizes effectively to real-world visual question answering, despite being trained only on synthetic data. These findings highlight the transferability of visual-groue -isuudegruuned RL and point to new directions for robust multimodal reasoning.
10

---

References
Abdelrahman Abouelenin, Atabak Ashfaq, Adam Atkinson, Hany Awadalla, Nguyen Bach, Jianmin Bao, Alon Benhaim, Martin Cai, Vishrav Chaudhary, Congcong Chen, et al. Phi-4-mini technical report: Compact yet powerful multimodal language models via mixture-of-loras. arXiv preprint arXiv:2503.01743, 2025.
Zhe Chen, Weiyun Wang. Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, Lixin Gu, Xuehui Wang, Qingyun Li, Yiming Ren, Zixuan Chen, Jiapeng Luo, Jiahao Wang, Tan Jiang, Bo Wang, Conghui He, Botian Shi, Xingcheng Zhang, Han Lv, Yi Wang. Wenqi Shao, Pei Chu, Zhongying Tu, Tong He, Zhiyong Wu, Hui Deng, Jiaye Ge, Kaiming Chen, Min Dou, Lewei Lu, Xizhou Zhu, Tong Lu, Dahu Lin, Yunfeng Qiao, Jifeng Dai, and Wenhai Wang. Expanding performance boundaries of open- source multimodal models with model, data, and test-time scaling. ArXiv, abs/2412.05271, 2024.
URL https
//api.semanticscheee
org hetg/orppSID::2200 274581884
Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc Le. Sergey Levine, and Yi Ma. Sft memorizes, generalizes: A comparative study of foundation model post-training. arXiv preprint arXiv:2501.1716. 2025.
Yifan Du, Zikang Liu, Yifan Li, Wayne Xin Zhao, Yuqi Huo, Bingning Wang, Weipeng Chen, Zheng Liu, Zhongyuan Wang, and Ji-Rong Wen. Virgo: A preliminary exploration on reproducing o ol-like milm. arXiv preprint arXiv:2501.01904 2025.
Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in lims via reinforcement learning. arXiv preprint arXiv:2501.12944 2025.
Yunzhuo Hao, Jiawei Gu, Huichen Will Wang, Linjie Li, Zhengyuan Yang, Lijuan Wang, and Yu Cheng. Can milms reason in multimodality? emma: An enhanced multimodal reasoning benchmark. arXiv preprint 2oXXh:2501.05444 2025.
Zhenyu Hou, Xin Lv, Rui Lu, Jiajie Zhang, Yujiang Li, Zijun Yao, Juanzi Li, Jie Tang, and Yuxiao Dong. Advancing language model reasoning through reinforcement learning and inference scaling. arXiv preprint arXiv:250l.1163: 2025.
Qihan Huang, Long Chan, Jinlong Liu, Wanggui He, Hao Jiang, Mingli Song, Jingyuan Chen, Chang Yao, and Jie Song. Boosting milm reasoning with text-debiased hint-grpo. arXiv preprint arXiv:2503.233900 2025.
Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai ol system card. arXiv preprint arXiv:2412.16720, 2024.
Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina, Eric Hambro, Edward Grefenstette, and Roberta Raileanu. Understanding the effects of rlhf on Ilm generalisation and diversity. ArXiv, abs/2310.06452, 2023. URL httpp:://ppp..eeettticcccoooo org/
semanticschoolrrr
CorpusID:2638899223
Pan Lu, Ran Gong, Shibiao Jiang, Liang Qiu, Siyuan Huang, Xiaodan Liang, and Song-Chun Zhu. Inter-gps: Interpretable geometry problem solving with formal language and symbolic reasoning. In The Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-IJCNLP 2021), 2021.
Liangchen Luo, Yinxiao Liu, Rosanne Liu, Samrat Phatale, Meiqi Guo, Harsh Lara, Yunxuan Li, Lei Shu, Yun Zhu, Lei Meng, et al. Improve mathematical reasoning in language models by automated process supervision. arXiv preprint arXiv:2406. 240::2992..0 2024.
Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Botian Shi, Wenhai Wang, Junjun He, Kaipeng Zhang, et al. Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforeement learning, arXiv preprint 25X3::2503.0766 2025.
11

---

Zhenyu Pan and Han Liu. Metaspatial: Reinforcing 3d spatial reasoning in vlms for the metaverse. arXiv preprint MXiv:2503.184700 2025.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang. YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint PxXiv22422003300 2024.
Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang. Kangjia Zhao, Qianqian Zhang. Ruochen Xu, and Tiancheng Zhao. VIm-rl: A stable and generalizable rl style large vision-language model. arXiv preprint arXiv:2504.07615. 2025a.
Maohao Shen, Guangtao Zeng, Zhenting Qi, Zhang- Zhang-Wei Hong, Zhenfang Chen, Wei Lu, Gregory Wornell, Subhro Das, David Cox, and Chuang Gan. Satori: Reinforcement learning with oh action-thoughh enhances IIm reasoning via autoregressive search. arXiv preprint wrxv:2550.00258. 2025b.
Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang. Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi kl. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501. 2025.
Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang. Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing rision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409.12 12/91, 2024a.
Weiyun Wang, Zhangwei Gao, Lianjie Chen, Zhe Chen, Jinguo Zhu, Xiangyu Zhao, Yangzhou Liu, Yue Cao, Shenglong Ye, Xizhou Zhu, al. Visualprm: An effective process reward model for multimodal reasoning. arXiv preprint arXiv:2503. 10221, 2025.
Xinyi Wang, Antonis Antoniades, Yanai Elazar, Alfonso Amayuelas, Alon Albalak, Kexun Zhang, and William Yang Wang. Generalization vs memorization: Tracing language models' capabilities back to pretraining data. arXiv preprint arXiv:2407.14985 2024b.
Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang, et al. Deepseek-v12: Mixture-of-expers vision-language models for advanced multimodal understanding. arXiv preprint arXiv:2412. 2024.
Guowei Xu, Peng Jin, Li Hao, Yibing Song, Lichao Sun, and Li Yuan. Llava-ol: Let vision language models reason stepeby-step. arXiv preprint vrXiv:2411.10444 2024a.
Zhuoyan Xu, Zhenmei Shi, and Yingyu Liang. Do large language models have compositional ability? an investigation into limitations and scalability, ArXiv, abs/2407.15520, 2024b. URL https://api.
org/ http//CoosID::20887707
Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, et al. R1-onevision: Advancing generalized multimodal reasoning through modsl formalization. arXiv preprint arXiv:2503. 10615, 2025.
Huanjin Yao. Jiaxing Huang, Wenhao Wu, Jingyi Zhang, Yibo Wang, Shunyu Liu, Yingjie Wang, Yuxin Song. Haocheng Feng, Li Shen, et al. Mulberry: Empowering milm with oike reasoning and reflection via collective monte carlo tree search. arXiv preprint arXiv:2412.18319, 2024.
Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Shiji Song, and Gao Huang. Does reinforcement learning really incentivize reasoning capacity in Ilms beyond the base model? arXiv preprint arXiv:2504.13837 2025.
Yufei Zhan, Yousong Zhu, Shurong Zheng, Hongyin Zhao, Fan Yang, Ming Tang, and Jinqiao Wang. Vision-rl: Evolving human-free alignment in large vision-language models via vision-guide guided reinforcement learning. arXiv preprint arXiv:2503. 18013, 2025.
Jun Zhao, Jingqi Tong, Yurong Mou, Ming Zhang, Qi Zhang, and Xuanjing Huang. Exploring the compositional deficiency of large language models in mathematical reasoning. arXiv preprint arXiv:2405.06680 2024.
Rosie Zhao, Alexandru Meterez, Sham Kakade, Cengiz Pehlevan, Samy Jelassi, and Eran Malach. Echo chamber: RI post-training amplifies behaviors learned in pretraining. arXiv preprint
arXiv:2504.07912, 2025.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Explicitly stated in the last two paragraphs of Sec
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how be expected to generalize other settings.
much the results
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
See Limitations section supplementary materials.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
13

---

Justification:
no theoretical results
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: as seen in experiment settings at Section 4.2
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
14

---

Answer:
[Yes]
Justification:
Code and data are provided in supplementary materials.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https Cooeeeoodeesssseeeeee 7/nips.com http://bbb///iiiiee/eeeeeeaaaaaa cassiicyncccyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification: As seen Sec
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Inference has run for times get the average score for inference, but error bar is reported.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
15

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
As seen Sec
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
The research conforms with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
As seen in Broader impacts section in supplementary materials.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
16

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The proposed dataset has no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The appropriate license is included for each asset.
Guidelines:
existing assets. The authors should cite the original paper that produced the code package or dataset.
The answer NA means that the paper does
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
17

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: includes in the supplementary materials.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
18

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
Justification: As seen in See !
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy (https: httpp//rrppssccoooofererre/s///e for what should or should not be described.
19


Reviews:
[{"review_id": "6nhJ04AaA0", "reviewer": "Reviewer_tgW9", "summary": "The paper introduces ComPABench, a carefully-controlled diagnostic suite for probing compositional generalization in vision-language models (VLMs). They evaluate three learning paradigms—SFT, pure-RL with GRPO, and SFT-initialised RL—along cross-modal (text → image), cross-task (integrating spatial + geometric reasoning) and out-of-distribution (OOD) variants. Results on Qwen-VL 3B/7B show: (i) RL consistently beats SFT on compositional tasks; (ii) simply mastering each sub-skill (or pure-text reasoning) does not imply multimodal composition; and (iii) their RL-Ground scheme—“caption-before-think” plus dense progress rewards—substantially narrows the gap, especially on OOD composition.", "questions": "- Dataset leakage – Are any ComPABench images or prompts derived from existing public sets? How is overlap with Qwen-VL pre-training filtered?\n- Sample size sensitivity – Each test split has 500 items; did the authors compute confidence intervals or run multiple seeds to ensure stability?\n- Reward shaping details – Progress rewards are briefly sketched (Fig. 6). Could the authors release the exact reward code and ablative weights?\n- Scale trend – Only 3B/7B backbones were tried. Does RL’s advantage persist on ≥32 B models where SFT tends to overfit less?", "limitations": "- Synthetic, narrow domain – Shapes-on-grid tasks capture discrete arithmetic & spatial reasoning but miss open-world semantics, natural language variety, and real-image noise.\n- Small evaluation suite – 500-sample splits and two backbones limit statistical power; no human baselines or comparison to larger proprietary models.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "- Clear scope – Compositionality is a core open challenge for MLLMs; the benchmark isolates it cleanly.\n- Well-designed benchmark – Tasks are paired PT/MM, include controlled OOD splits, and report per-skill vs composed scores, exposing failure modes overlooked by single-task evals.\n- Insightful empirical findings – The paper shows RL’s compositional edge and the dangers of SFT “memorisation,” echoing recent LLM studies but in the multimodal space.\n- Actionable training recipe – RL-Ground is simple (no new architecture), yet improves both in-domain and OOD performance, offering practitioners a concrete knob to turn.", "was_revised": true, "final_justification": "Thanks the authors for response, I would say the response address most of my questions.\n\nHowever, after careful considerations and observing other reviewers ratings, I tend to maintain the current ratings due to the significance of this work.", "rebuttal": "We thank the reviewer for the thoughtful comments and detailed feedback. And we answer the questions raised by the reviewer as follows:\n\n**Q1: Dataset Leakage – Are ComPABench prompts/images derived from public datasets? How is overlap with Qwen-VL filtered?**\n\nAll ComPABench images and prompts are synthetically generated in-house using deterministic rendering pipelines (see Supplementary Sec. A). Our datasets are constructed from parametric shape definitions and grid layouts with predefined color palettes, ensuring no overlap with real-world datasets. Therefore, leakage is strictly avoided.\n\n**Q2: Sample Size Sensitivity – Did you compute confidence intervals or run multiple seeds?**\n\nWe acknowledge the concern regarding statistical robustness. As noted in the NeurIPS checklist, we ran inference 3 times for all evaluation settings and report average results. We will state it clearly in the revision, which confirms that the key trends remain stable under variance.\n\n**Q3: Reward Shaping – Can the authors release the reward code and ablative weights?**\n\nYes, we have released the complete reward shaping code in the supplementary zip, and will publicly host it upon acceptance. As detailed in Supplementary Sec. B (Table 2), we conducted ablations isolating the impact of:\n\n- **Caption format only:** Improves grounding but insufficient alone for compositional reasoning.\n- **Progress reward only:** Boosts compositional accuracy from 31.2% to 39.6%.\n- **Full RL-Ground:** Combines both, achieving 52.8% compositional accuracy.\n\nThese results validate the design of RL-Ground as both principled and effective.\n\n**Q4: Scale Trend – Does RL’s advantage persist at larger scales (e.g., 32B)?**\n\nDue to compute constraints, our study focuses on 3B and 7B models. While we have not yet evaluated larger models (e.g., 32B), prior work [1] demonstrates that SFT tends to overfit even at 11B, whereas RL provides better generalization under distribution shifts. We leave large-scale evaluation as an important direction for future work.\n\n[1] Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V. Le, Sergey Levine, and Yi Ma. *SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-Training.* arXiv preprint arXiv:2501.17161, 2025.\n\n**Limitations – Synthetic domain and narrow reasoning types**\n\nWe explicitly discuss this in Supplementary Sec. 1. ComPABench is designed as a controlled diagnostic suite (like CLEVR), not as a naturalistic benchmark. While it lacks open-world visual complexity, it systematically isolates reasoning failure modes that broader datasets often conflate. We view it as complementary to large-scale evaluations.\n\n**Typo – Line 49 “Firgure”**\n\nThank you for noticing. We will correct this in the final version.\n\nIn summary, we appreciate the reviewer’s recognition of our benchmark design, training insights, and actionable findings. We hope our clarifications, ablations, and reproducibility efforts address the reviewer’s concerns and justify a higher score."}, {"review_id": "r1nNbefQHx", "reviewer": "Reviewer_3k4H", "summary": "This paper investigates the compositionality gap which is a significant limitation of current VLMs about combining cross-modal and cross-task reasoning. \nIt proposes a newly constructed diagnostic benchmark, ComPABench and finds that while reinforcement learning outperforms supervised fine-tuning in enhancing compositional generalization, models still struggle with complex tasks that require the integration of multiple skills.\nTo address this, the paper proposes an effective strategy RL-Ground, which significantly improves both in-distribution and out-of-distribution compositional reasoning through a \"describe-then-think\" approach and a progressive reward mechanism.", "questions": "Please add bad cases study and check the training issues.", "limitations": "Yes", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strength:\n- This paper designs and introduces a new ComPABench which is specifically aimed at diagnosing the compositional capabilities of VLMs in cross-modal, cross-task, and out-of-distribution (OOD) settings.\n- The paper conducts a comprehensive empirical comparison between two mainstream post-training strategies—Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) on the benchmark.\n- It explores the weaknesses of VLMs in compositionality and proposes a simple solution named RL-Ground. RL-Ground combines a \"describe-then-think\" prompting structure and achieves significant performance improvements across multiple tasks.\n\nWeakness:\n\n- The paper does not include a qualitative error analysis of model outputs, which would provide deeper insight into the specific failure modes.\n\n- The reported 2.2% performance for SFT on the multimodal compositional task is unconvincingly low, which may suggest a training failure or inappropriate hyperparameters.\n\n- The benchmark tasks are somewhat artificial and detached from practical applications, and the study does not test the models on more realistic multi-task reasoning scenarios.\n\n- Figure 4 inappropriately uses line curves to present different conditions; a bar chart would be a more suitable representation for this data.\n\n- The visualization in Figure 1 is confusing. A clearer approach would be to consistently compare the Pure-Text (PT) and Multi-Modal (MM) results for each of the three task scenarios.", "was_revised": true, "final_justification": "Following the rebuttal and discussion with the authors, the reviewer remained unconvinced and decided to keep the original rating.", "rebuttal": "We thank the reviewer for their constructive feedback and thoughtful suggestions. Below, we provide responses to the key concerns.\n\n\n**Weakness 1–2: Lack of qualitative error analysis and the low SFT performance on the compositional task**\n\nWe thank the reviewer for these important observations. To better understand the behaviors of different post-training strategies, we introduce and analyze two auxiliary tasks:\n\n- **Shape Area Grounding** evaluates whether the model can correctly localize a specific shape and extract its relevant geometric attributes (e.g., base, height) to compute its area. For example, given a prompt like `\"What is the area of the purple trapezoid?\"`, the task assesses whether the model correctly identifies the purple trapezoid in the image and performs the area computation accordingly.\n\n- **Grid Position Grounding** measures the model’s ability to spatially localize a target shape on a 2D grid. For instance, given the question `\"What is the position of the orange right_triangle?\"`, the model is expected to identify the correct grid coordinates of the shape in the image.\n\nThese grounding evaluations help us identify where failures occur and how they correlate with compositional generalization.\n\n**Table: Evaluation of VLMs across tasks and their associated visual grounding metrics**\n\n| **Model**               | **Shape Area** | **Shape Area Grounding** | **Grid Position** | **Grid Position Grounding** | **Compositional** |\n|-------------------------|----------------|---------------------------|-------------------|------------------------------|--------------------|\n| Qwen2.5-VL-7B-Instruct  | 20.8           | 89.4                      | 4.8               | 24.4                         | 13.8               |\n| SFT                     | 73.6           | 1.2                       | 91.8              | 65.4                         | 7.2                |\n| RL                      | 74.6           | 96.6                      | 83.2              | 74.8                         | 31.2               |\n| SFT-init-RL             | 76.6           | 1.2                       | 89.8              | 43.6                         | 1.0                |\n| RL-Ground               | 73.8           | 96.2                      | 88.4              | 88.6                         | 52.8               |\n\n**SFT overfits to training patterns and fails to adapt to new task prompts**\n\nAs shown in the table above, SFT achieves high accuracy on individual reasoning tasks such as Shape Area (73.6%) and Grid Position (91.8%). However, it shows extremely lower grounding scores (1.2% for Shape Area), and its compositional accuracy drops to just 7.2%. This suggests that SFT learns to associate surface-level prompt features with memorized reasoning templates seen during training, rather than learning to flexibly combine reasoning skills. \n\n**Example failure case**\n\n> Question: There are 3 shapes on a 6x6 grid. Find the shape that is nearest to the red trapezoid in terms of Manhattan distance, what is the total area of the nearest shape to the target shape and the target shape?  \n> SFT Output:  \n> \\<think\\>Target shape: red trapezoid at (2, 4)  \n> Distance from target shape to blue square at (4, 1): 5  \n> Distance from target shape to green square at (1, 2): 3  \n> Closest shape is the green square at (1, 2) with distance 3  \n> Total area of closest shape and target shape: 3 + 8 = 11\\</think\\>  \n> \\<answer\\>11\\</answer\\>  \n> Ground Truth: 105\n\nIn this case, SFT correctly identifies the nearest shape (e.g., the green square) but hallucinates the areas of both the target and the nearest shape, outputting an incorrect total without grounding either value in the actual image. This illustrates that the model overfits to previously seen single-task prompt templates, failing to adapt when the task requires integrating multiple skills or resolving an unfamiliar instruction composition.\n\n**RL improves grounding and partial composition**\n\nPure RL shows consistently high grounding accuracy (96.6% for Shape Area, 74.8% for Grid Position), enabling better generalization on the compositional task (31.2%). This indicates that RL encourages more faithful visual attention and reasoning behavior. However, the performance gap shows that RL alone is insufficient for strong compositionality without further inductive guidance.\n\n**SFT-init-RL underperforms due to inherited SFT biases**\n\nAlthough SFT-init-RL performs well on atomic tasks, its grounding scores remain low (e.g., 1.2% for Shape Area Grounding), and its compositional accuracy collapses to 1.0%. This suggests that starting RL training from an SFT checkpoint may preserve the shallow reasoning habits learned during SFT, making it harder for RL to correct them later.\n\n**RL-Ground achieves robust and generalizable behavior**\n\nBy enforcing a `<caption>` step and applying step-wise rewards to guide intermediate progress, RL-Ground attains high grounding accuracy (96.2% and 88.6%) and the best overall compositional performance (52.8%). These results offer deeper insight: RL-Ground enhances compositional ability by strengthening the model’s atomic visual grounding capability.\n\n**Weakness 3: Artificial benchmark design and lack of real-world tasks**\n\nWe appreciate the reviewer’s concern. The synthetic nature of **ComPABench** is intentional and serves a clear diagnostic purpose: it allows us to precisely control task structures, systematically vary compositional difficulty, and isolate failure modes such as visual grounding, spatial localization, and multi-hop reasoning. While not a replacement for open-domain evaluation, ComPABench complements existing real-world benchmarks by providing a controlled lens to probe compositional reasoning—an ability often underrepresented in current VLM benchmarks.\n\nTo assess the generalization of our findings beyond synthetic data, we conducted a zero-shot evaluation on GeoQA, a more natural visual QA benchmark consisting of 754 test samples. Despite being trained only on synthetic data, RL-Ground significantly improves performance:\n\n| Model                      | GeoQA Accuracy (%) |\n|---------------------------|--------------------|\n| Qwen2.5-VL-3B-Instruct     | 14.8               |\n| SFT                       | 9.2                |\n| RL                        | 17.8               |\n| **RL-Ground (ours)**      | **21.2**           |\n\nThis evaluation provides three key insights:\n\n1. **SFT underperforms even the base model**, dropping from 14.8% to 9.2%. This supports our central claim that SFT tends to overfit to synthetic prompt distributions and weakens out-of-distribution generalization.\n\n2. **RL recovers and surpasses base model performance**, demonstrating that reinforcement learning improves robustness through optimizing the path of thinking more efficiently.\n\n3. **RL-Ground (ours) achieves the best generalization**, outperforming both RL and SFT. Its design—explicit `<caption>` followed by progress-guided reasoning—encourages the model to extract and structure visual information more effectively, leading to better real-world QA performance even without exposure to in-domain examples.\n\nWe will include this GeoQA result and comparative analysis in the revised version to clarify that: While ComPABench is synthetic by design for diagnostic control, the caption-before-thinking strategy and progress-based reward design introduced in RL-Ground can transfer effectively to more naturalistic, multi-task reasoning scenarios, improving model generalization even without training on in-domain examples.\n\nMoreover, we are actively working to extend ComPABench with **real-world image sources** as future plan.\n\n**Weakness 4-5: Visualization issue in Figure 4 and Figure 1**\n\nThank you for the helpful suggestions. We agree that a bar chart is more appropriate for comparing discrete conditions and will revise Figure 4 accordingly to improve readability. For Figure 1, we will adopt a clearer layout with improved labeling and a legend to better illustrate the task structure and highlight the compositional gap.\n\n**Summary of Planned Revisions**\n\nWe thank the reviewer again for highlighting both the strengths and areas for improvement. We will:\n\n- Include qualitative error cases in the final version\n- Clarify the significance of low SFT performance with empirical evidence\n- Revise Figures 1 and 4 for better readability\n- Emphasize that ComPABench is a diagnostic tool, complementary to open-world benchmarks, and reveal the conclusions drawn in our paper are transferable to real-world datasets.\n\nWe hope these improvements and clarifications address your concerns and merit a higher score."}, {"review_id": "mmHFQmes4M", "reviewer": "Reviewer_i1z4", "summary": "This paper introduces ComPABench, a diagnostic benchmark for systematically probing the compositional generalization ability of Vision-Language Models (VLMs) under different post-training strategies, including Supervised Fine-Tuning (SFT), Reinforcement Learning (RL), and a hybrid SFT-initialized RL approach. The benchmark evaluates cross-modality, cross-task, and out-of-distribution (OOD) compositional generalization. The authors propose a novel RL training strategy, RL-Ground, that incorporates a caption-before-thinking mechanism and progress reward signals to improve performance on compositional tasks.", "questions": "Please address the weaknesses above.", "limitations": "yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- The paper presents several important observations, e.g., RL tends to preserve compositional skills better than SFT; SFT can overfitting previously acquired reasoning skills.\n- The compositional generalization of VLMs is a meaningful research area. This paper targets an important gap in the current research landscape.\n\nWeaknesses:\n- The experiments are only conducted on Qwen models of 3B and 7B parameters. This raises concerns about the generality of the findings across model scales and architectures. Larger models (e.g., LLaMA 11B, Gemma 12B/27B) may exhibit different compositional behaviors due to their capacity, and omitting these from evaluation limits the robustness and applicability of the conclusions. While computational cost is understandable, this limitation should be more explicitly acknowledged and, ideally, partially addressed.\n- The organization of the paper could be improved. Key components such as the dataset (introduced in the \"Benchmark for Probing of Compositional Ability\" subsection) and the proposed method (described in the \"Impact of progress-reward grounding (RL-Ground)\" subsection) are both presented under the Experimental section. This conflation of experimental results with dataset/method description makes it harder to clearly distinguish between contributions and evaluations. A more conventional structure separating methodology and experimentation would enhance clarity and readability.\n- While the proposed RL-Ground strategy is practical and achieves notable improvements, its core components—captioning before reasoning and progress-based intermediate rewards—are relatively standard and lack significant innovation. Moreover, RL-Ground fails to address RQ1 (Cross-Modality Compositional Generalization), which is arguably one of the most critical and challenging aspects of VLM compositional reasoning. In addition, the paper does not provide sufficient detail about the experimental setup of RL-Ground.", "was_revised": true, "final_justification": "Thanks the authors' discussions. I still have some concerns about writing and empirically generality, I'll raise my score to 3.", "rebuttal": "We thank the reviewer for the constructive feedback. Below we address each concern in turn.\n\n**Weakness 1: Limited model scale and architecture diversity**\n\nWe acknowledge this limitation and will explicitly state it in the Limitation section. Due to compute constraints, our experiments focus on Qwen models (3B and 7B). Evaluating compositional generalization across other model families such as LLaMA, Gemma, and even closed-source models like GPT-4o is an important direction, and we plan to include these in future work using the same benchmark setup.\n\n**Weakness 2: Organization and motivation of RL-Ground**\n\nWe appreciate the suggestion regarding paper organization. Our current structure reflects the probing nature of this work. Our primary goal is to reveal and analyze compositional gaps in MLLMs, a dimension that has received limited attention in prior work. We do not position RL-Ground as a novel algorithmic contribution, but rather as a practical guidepost that highlights how compositional ability may emerge from vision-to-text conversion and visually grounded intermediate reasoning.\n\nWe agree that clearer separation of contributions and evaluation will improve readability. In the revised version, we will move benchmark details into a standalone “Benchmark” section and place result analysis, including RL-Ground, into a separate “Results and Findings” section.\n\n**Weakness 3: RL-Ground novelty, RQ1 impact, and experimental details**\n\nWe thank the reviewer for this feedback. We emphasize that our goal is not to propose a new algorithm, but rather to highlight two practical directions that improve compositional generalization in RL training:  \n(i) explicit conversion from vision to text to trigger subsequent reasoning with text, and  \n(ii) incorporating progress-based intermediate rewards to ground visual perception tasks.  \n\nRegarding its effectiveness on RQ1 (Cross-Modality Compositional Generalization), we note that RQ1 uses models trained on pure-text corpora, while RL-Ground is a post-training scheme applied to multimodal models. However, to probe the potential transfer of its design principles, we also experimented with applying the `caption-before-think` prompting pattern at inference time during RQ1 evaluation. As shown below, we observe consistent improvements across both tasks:\n\n| Model                       | Shape Area (%) | Grid Spatial (%) |\n|----------------------------|----------------|------------------|\n| RL-3B                      | 16.2           | 4.6              |\n| RL-3B + caption-before-think | 16.6         | 6.2              |\n| RL-7B                      | 28.0           | 3.0              |\n| RL-7B + caption-before-think | 31.4         | 5.2              |\n\nThese results suggest that the structured prompting design in RL-Ground may also benefit models evaluated on RQ1, even when trained in purely textual settings.\n\nAs for implementation details, RL-Ground uses the same hyperparameter setup as the RL baseline, with two key changes:\n1. A modified format reward that only gives credit when outputs follow the full `<caption>...</caption><think>...</think><answer>...</answer>` structure.\n2. An additional progress reward for intermediate reasoning steps, increasing the maximum possible reward from 2 to 3.\n\nWe will clarify these implementation specifics in the revised version."}, {"review_id": "LzgXJ2zdis", "reviewer": "Reviewer_JamV", "summary": "This paper investigates the compositional reasoning capabilities of Vision-Language Models, exploring why skills learned independently often fail to integrate in complex, multimodal tasks. To systematically probe this \"compositional gap,\" the authors introduce ComPABench, a diagnostic benchmark designed to test cross-modal and cross-task generalization. The paper proposes a novel and effective solution, RL-Ground, which enhances compositionality by enforcing a \"caption-before-thinking\" step and rewarding intermediate reasoning progress, leading to significant gains on both in-distribution and out-of-distribution tasks.", "questions": "1. The RL-Ground method seems tailored to your synthetic benchmark. How can its \"progress reward\" be scaled to complex real-world tasks where sub-goals are not easily verifiable? Furthermore, can you provide an error analysis on the brittleness of the \"caption-before-thinking\" step, particularly when the initial caption is flawed?\n2. The paper's conclusions are based solely on the Qwen architecture. What is your argument for why the identified \"compositional gap\" and the relative performance of SFT vs. RL are fundamental issues that would generalize to other VLM families (e.g., Gemma, Phi-4)?\n3. Given that ComPABench is synthetically generated, its linguistic diversity may be limited. Have you tested your RL-Ground model on any existing, more naturalistic VQA datasets to confirm that its reasoning improvements generalize beyond the benchmark's specific language patterns?", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n1. The paper introduces ComPABench, a significant contribution. The benchmark is designed to isolate different facets of compositional reasoning, such as cross-modal, cross-task, and out-of-distribution (OOD) generalization, which are often conflated in other evaluations.\n2. The study conducts a comparison of post-training strategies (SFT, RL, SFT-init RL) across multiple model scales. The finding that SFT can be actively harmful to compositional reasoning, leading to \"catastrophic forgetting\" despite high performance on individual skills, is an important result for the community.\n3. The paper does not merely identify a problem but also proposes a simple and effective solution, RL-Ground. This method, which combines \"caption-before-thinking\" with intermediate progress rewards, is intuitive and shows performance gains, offering a promising direction for future research on improving VLM reasoning.\n\n**Weakness:**\n1. The experiments are conducted on the Qwen2.5-VL family of models. While using two different model sizes is good for showing scaling effects, validating the findings across different VLM architectures (e.g., Gemma, Phi-4) would significantly strengthen the claim that the \"compositional ability gap\" is a fundamental issue rather than an artifact of a specific model family.\n2. The RL-Ground method relies on providing a \"fine-grained progress reward\" for intermediate reasoning steps. While this is feasible for the paper's synthetic tasks where ground-truth for sub-steps is clear, defining and verifying these sub-goals for complex, open-ended real-world tasks would be extremely challenging and costly, potentially limiting its practical applicability.\n3. The proposed RL-Ground solution relies on the model first generating a caption, but the paper does not explore the failure modes of this approach. If a model generates an incomplete or subtly incorrect caption, it may then reason perfectly from flawed premises, leading to a wrong answer. This \"point of no return\" at the captioning stage could make the model brittle in real-world scenarios where visual details are ambiguous or hard to describe.\n4. Because the ComPABench dataset is synthetically generated, the language of the prompts is likely very structured and repetitive. This lack of linguistic diversity means models may be learning to exploit simple textual patterns rather than developing a robust, generalizable reasoning ability. The conclusions drawn might not fully transfer to real-world datasets where questions are more varied.", "was_revised": true, "final_justification": "Regarding Q1: The new diagnostic experiments are convincing. The data demonstrating that RL-Ground enhances visual grounding effectively mitigates the concern about the brittleness of the <caption> step.\n\nRegarding Q2: I appreciate the authors' candid acknowledgement of this limitation and their commitment to address it in the revised paper. This is a reasonable approach.\n\nRegarding Q3: The new zero-shot evaluation on the GeoQA dataset provides evidence that the method's improvements generalize to realistic, natural-language tasks and are not confined to the synthetic benchmark.", "rebuttal": "We thank the reviewer for the constructive feedbacks, and answer the questions raised as follows:\n\n**Q1: Is RL-Ground overly tailored to synthetic tasks? How scalable is progress reward, and how robust is caption-before-thinking to errors?**\n\nWe thank the reviewer for raising these important concerns. While our benchmark is synthetic for controllability and diagnosis, the underlying challenges—visual grounding, multi-hop reasoning, and compositional capability—reflect real-world VQA situations. RL-Ground is not tightly coupled to our benchmark; instead, it proposes two broadly applicable directions:\n1. Enforcing explicit visual grounding (`<caption>`) before reasoning to trigger reasoning with text, and  \n2. Rewarding intermediate progress to encourage visual-grounded reasoning.\n\n**Regarding the generalizability of _progress reward_:**  \nWe acknowledge that defining sub-goals in open-domain tasks can be challenging. However, many real-world VQA tasks contain implicit intermediate signals (e.g., object detection, caption generation, OCR) that could be repurposed into reward shaping. We envision future work leveraging pretrained vision models (e.g., SAM, GroundingDINO, BLIP-2, YOLO) to estimate such sub-goals in-the-wild as the progress reward.\n\n**On the robustness of the `<caption>` step:**  \nWe appreciate the concern regarding potential brittleness of the `<caption>` step. To better understand how RL-Ground contributes to compositional generalization, we introduce two supporting grounding tasks that measure the model’s ability to extract critical visual information:\n\n- _Shape Area Grounding_: asks for the area of a specified shape\n- _Grid Position Grounding_: requires locating a shape on the grid using (row, column) indices\n\nThese tasks serve as diagnostic tools for analyzing how well the model grounds visual entities before reasoning over them.\n\nTable: Evaluation of VLMs across tasks and their associated visual grounding metrics\n\n| **Model**               | **Shape Area** | **Shape Area Grounding** | **Grid Position** | **Grid Position Grounding** | **Compositional** |\n|-------------------------|----------------|---------------------------|-------------------|------------------------------|--------------------|\n| Qwen2.5-VL-7B-Instruct  | 20.8           | 89.4                      | 4.8               | 24.4                         | 13.8               |\n| SFT                     | 73.6           | 1.2                       | 91.8              | 65.4                         | 7.2                |\n| RL                      | 74.6           | 96.6                      | 83.2              | 74.8                         | 31.2               |\n| SFT-init-RL             | 76.6           | 1.2                       | 89.8              | 43.6                         | 1.0                |\n| RL-Ground               | 73.8           | 96.2                      | 88.4              | 88.6                         | 52.8               |\n\nAs shown in the above table, RL-Ground achieves high grounding scores  (_Shape Area Grounding_: **96.2%**, _Grid Position Grounding_: **88.6%**)  compared to all other post-training strategies, along with the best performance on the compositional task (**52.8%**).  This confirms that the `<caption>` step, combined with step-wise reward, enables the model to better structure and retain relevant visual cues, thereby enhancing multi-step compositional reasoning. In contrast to the concern that caption errors could cascade, our results indicate that RL-Ground actually reduces such failure cases by explicitly guiding the grounding process before reasoning begins.\n\nTo further mitigate possible risks from incorrect or misleading captions, we identify one promising future direction: introducing a stricter supervision mechanism that treats the caption as a self-contained summary and only assigns reward if the downstream answer can be derived from the caption alone, without using the image. Such an extension would encourage more semantically complete and robust visual summarization. Nonetheless, we emphasize that RL-Ground is not proposed as a final solution, but rather as a light-weight and effective direction that highlights how enhancing atomic visual alignment and grounding can unlock compositional capabilities in VLMs.\n\n**Q2: Does the compositional gap and RL/SFT difference generalize beyond Qwen?**\n\nWe believe the observed compositional gap and the relative robustness of RL over SFT are not specific to Qwen, but indicative of broader challenges in current VLM post-training.  \nWe plan to validate our findings across architectures such as Gemma, InternVL, and Phi-4, as future work.  \nTo clarify our current scope, we will more explicitly acknowledge the architectural limitation in the paper’s **Limitations** section.\n\n**Q3: Is RL-Ground’s improvement specific to synthetic language patterns? Can it generalize to real-world VQA?**\n\nTo assess the generalization beyond our benchmark, we conducted a new zero-shot evaluation on the **GeoQA** dataset (754 test samples), a realistic, natural-language VQA benchmark, using the checkpoint trained with our synthetic dataset and the original  Qwen2.5-VL-3B-Instruct base.\nWe evaluated:\n\n- **Qwen2.5-VL-3B-Instruct**: 14.8% accuracy \n- **SFT**: 9.2% accuracy \n- **RL**: 17.8% accuracy  \n- **RL-Ground**: 21.2% accuracy\n\nThese results confirm that RL-Ground improves performance on naturalistic questions despite being trained only on synthetic data. This highlights that the proposed strategies—captioning and progress reward—enhance reasoning robustness and are not limited to benchmark-specific language patterns.\n\nWe will include this experiment and analysis in the revised version to reinforce the practical relevance of RL-Ground beyond ComPABench."}]

Author rebuttal:

Meta-review: (a) **Summary:**
This paper explores the compositionality gap in VLMs while reasoning, specifically testing whether VLMs inherit compositional abilities during post-training. To benchmark, they introduce a diagnostic suite called ComPABench. Models are first trained on either unimodal tasks or isolated reasoning skills then tested on generalizing to multimodal, compositional variations. They compare SFT and RL settings and find that (i) RL outperforms SFT, showing better integration of learned skills (ii) VLMs perform well on individual tasks yet struggle in cross-modal and cross-task scenarios and (iii) models show better performance in the “caption-then-think” scenario to enforce reasoning.


(b) **Strengths:**
- The paper introduces a new benchmark, ComPABench, and has a very clear scope of testing multimodal compositional understanding (**Reviewer tgW9, Reviewer 3k4H**). There are controlled OOD splits as well as cross-model and cross-task settings.
- The finding that RL preserves compositionality better than SFT is an important and unique finding (**Reviewer i1z4**). Additionally, this setup shows that SFT leads to more catastrophic forgetting compared to RL while maintaining high performance on individual skills (**Reviewer JamV**).
- This work also proposes a solution to the compositionality gap, RL-Ground (**Reviewer JamV**). The idea is intuitive and also leads to improved performance.

(c) **Weaknesses:**
- The authors only tested on Qwen models (**Reviewer i1z4, Reviewer JamV**), which limits the generalizability of the results. Including additional model families and sizes would strengthen the findings.
- ComPABench is synthetic, which means the language is likely more simplistic or repetitive than natural language (**Reviewer JamV**). Similar to only testing on Qwen models, this may limit the generalizability of the findings. It’s unclear if they’ll hold for compositional understanding of natural language.
- The proposed approach of RL-Ground is not novel as the caption first, then reason is used in other paradigms as well (**Reviewer i1z4**). There are also some unexplored failure modes here, such as what happens for incorrectly or poorly generated captions (**Reviewer JamV**).

(d) **Recommendation:** The scores were 3,3,4,4, essentially right at the accept/reject borderline.  The strengths are that this work presents a clean experimental setup alongside a new evaluation suite called ComPABench. ComPABench uses well designed settings (cross-modal, cross-task generalization) and tests RL and SFT settings. Additionally, they propose a new method called RL-Ground to lead to better compositional reasoning. There are also valid concerns, such as only Qwen models being evaluated and the benchmark suite using synthetic language. Overall, weighing the pros and cons, I suggest this paper should be accepted.

(e) **Discussion/Changes during Rebuttal:**
- **Point:** One reviewer questioned the sample size (500 samples) and whether the results are based on multiple seeds/have confidence intervals. The authors ran 3 times and report the average, observing the trends are stable. **Impact:** This experimental setup makes sense and did not impact my decision.
- **Point:** Another reviewer wondered whether RL-Ground is robust to the captioning step (i.e. does a poorly captioned image break the approach). The authors ran an experiment to grounding tasks, evaluating the area and the location of a shape. **Impact:** RL-Ground achieves high performance compared to baseline methods, validating its robustness.
Decision: Accept (poster)