CoPRS: Learning Positional Prior from Chain-of-Thought for Reasoning Segmentation
Authors: Zhenyu Lu
Liupeng Li
Jinpeng Wang
Yan Feng
Bin Chen
Ke Chen
Yaowei Wang
Venue: ICLR 2026 Poster (2026)
Keywords: Reasoning Segmentation
Reinforcement Learning
Positional Prior
Multi-Modal Chain-of-Thought
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: This paper introduces CoPRS, a reasoning segmentation model that first uses "chain-of-thought" to generate an interpretable positional prior (i.e., an explicit heatmap), hinting at where in the image it's focusing before decoding the final mask.

Abstract: Existing works on reasoning segmentation either connect hidden features from a language model directly to a mask decoder or represent positions in text, which limits interpretability and semantic detail.
To solve this, we present CoPRS, a Multi-modal Chain-of-Thought (MCoT)-based positional perception model that bridges language reasoning to segmentation through a differentiable and interpretable positional prior instantiated as a heatmap.
By making the reasoning process clear via MCoT and expressing it as a dense, differentiable heatmap, this interface enhances interpretability and diagnostic analysis and yields more concentrated evidence on the target.
A learnable concentration token aggregates features of the image and reasoning text to generate this positional prior, which is decoded to precise masks through a lightweight decoder, providing a direct connection between reasoning and segmentation.
Across the RefCOCO series and ReasonSeg, CoPRS matches or surpasses the best reported metrics on each standard split under comparable protocols, with performance at or above the prior state of the art across both validation and test partitions.
Extensive experiments demonstrate a strong positive correlation among the CoT trajectory, the generated heatmap, and the decoded mask, supporting an interpretable alignment between the reasoning output and downstream mask generation.
Collectively, these findings support the utility of this paradigm in bridging reasoning and segmentation and show advantages in concentration driven by reasoning and in more precise mask prediction.

Published as a conference paper at ICLR 2026
CoPRS: LEARNING POSITIONAL PRIOR FROM CHAIN- OF-THOTGHT FOR REASONING SEGMENTATION
Zhenyu Lu 12.5, Liupeng , Jinpeng Wang" Yan Feng", Bin Chen?, Ke Chen', Yaowei Wang
Shenzhen Institutes of Advanced Technology. Chinese Academy of Sciences
Peng Cheng Laboratory
Harbin Institute of Technology, Shenzhen
*Meituan, Beijing
University of Chinese Academy of Sciences
zhenyalute u22 @m. fudan. edu. en; wangjjp26@gail.com com; wangyaowei@hit edu. en
ABSTRACT
Existing works on reasoning segmentation either connect hidden features from language model directly to a mask decoder or represent positions in text, which limits interpretability and semantic detail. To solve this, present CoPRS, Multi-modal Chain-of-Thoughh (MCoT)- positional perception model that bridges language reasoning segmentation through differentiable and inter- pretable positional prior instantiated as a heatmap. By making the reasoning process clear via MCoT and expressing it as a dense, differentiable heatmap, this interface enhances interpretability and diagnostic analysis and yields more concentrated evidence on the target. A learnable concentration token aggre- gates features of the image and reasoning text to generate this positional prior, which is decoded precise masks through lightweight decoder, providing a direct connection between reasoning and segmentation. Across the RefCOCO se- ries and ReasonSeg, CoPRS matches or surpasses the best reported metrics on each standard split under comparable protocols, with performance at above the prior state the art across both validation and test partitions. Extensive experiments demonstrate strong positive correlation among the CoT trajec- tory, the generated heatmap, and the decoded mask, supporting an interpretable alignment between the reasoning output and downstream mask generation. Col- lectively, these findings support the utility of this paradigm in bridging reason- ing and segmentation and show advantages in concentration driven by reason- ing and in more precise mask prediction. Code has been released at https: 7/github. http wttom//hhnyyu--eeeeooooooo Hil.yy/LLD--eelloooee////e..
INTRODUCTION
Visual perception is increasingly expected to not only assign labels to pixels but also follow natural- language instructions with compositional constraints, such as "Segment the UAV that is trailing the quadcopter and partially occluded by trees." This demand advances the long are of visual under- standing, starting from semantic segmentation (category labels) (Guo al. 2018), instance seg- mentation (object masks) (Hafiz & Bhat (2020), and further to open-vocabulary segmentation (open- set text categories) (Ren et al. 2024a), and most recently, toward reasoning segmentation (free-form instructions) Lai et al. (2024). Meeting this goal requires coupling language reasoning with spatial grounding by converting textual instructions into perceptual decisions.
Existing attempts to bridge language reasoning with segmentation fall into two distinct camps. La- tent reasoning methods (Pi et al., 2024] Lai et al., (024) predict the masks by directly decoding hidden features from the language models, which keep intermediate decisions non-transparent and uncontrollable. Text-based reasoning methods (Lan et al. 20u5 et al. 22025), on the other hand, readout positions in text and generate discrete coordinates. While explicit, such an interface is in- flexible to capture and reflect fine- -nniiid visual semantics, and also fragile to practical issues like
"Corresponding authors.

---

Published as a conference paper at ICLR 2026
part morkey's body picture helps it meintain balance and stability when moving another?
Tainable
Trozen
V diffrnataaail
Multi Modal LLM
Multi Modal LLM
Multi Modal LLM
<thintThee tall otten used for balance, especially
The monkey's tall is the key anatomical festure diding balance and slability during arboriel movement.
corawerll thon 2d (16,15762558555
suery:
"polnt_ad" 1380,199 [B00sser
Decoder
Reformat
QK'
Queryy Head
Decoder
Decoder
(9) Lalent Reasoning
(b) Text-Based Ressoning
(c) MCCT-BBaae Positional Perception
Figure 1: Illustration of paradigms for reasoning segmentation. is exemplified by LISA Laii
et al. 2024), and (b) by Seg-Zero (Liu et al. 2025). Our CoPRS (c) bridges MCoT reasoning to segmentation through a differentiable and interpretable positional prior.
formatting
out-of-image coordinates. In essence, limitations in the polarized paradigms highlight the need for a better trade-off between interpretability and representational fidelity.
To close this gap, introduce CoPRS, CoT-based Positional perception model for Reasoning Segmentation. CoPRS is one-stage and end-to-end: given an image-instruction input, it first reasons before producing a perception heatmap concentrating the target region, which provides positional prior to enhance the segmentation mask decoding. As compared in Figure T the positional prior serves as differentiable and interpretable connection between MCoT (Wang et al. 2025b) and segmentation, which is direct and effective to enhance visual perception of a Multi-modal Large Language Model (MLLM) and align instruction semantics with mask decoding.
Specifically, we first introduce learnable concentration token to aggregate text and generate a concentration query. Next, we convert this query the positional prior to con- centrate the target for mask prediction. This dense, differentiable heatmap is more interpretable than purely hidden features, and provides finer detail than discrete textual coordinates. Coneur- rently, establish unified training framework by adopting the Group Relative Policy Optimiza- tion (GRPO) (Shao et al., 2024) jointly with segmentation supervision. This framework enhances reasoning capability through GRPO, jointly supervising the MLLM and segmentation model via differentiable positional prior and offering an effective solution to the limitations of prior paradigms.
con-
CoPRS matches exceeds the best reported cloU/gIoU each split under comparable protocols across RefCOCO, RefCOCO+ (Kazemzadeh et al. 2014). RefCOCOg (Mao et al.) 2016). and Rea- sonSeg (Lai et al. 2024). We further find strong positive correlation among the quality of the CoT trajectory, the generated heatmap, and the decoded mask, indicating strong concentration driven by reasoning and precise mask generation. Beyond reasoning segmentation, the unified framework and its positional prior naturally extend to region concentration tasks such as trajectory prediction.
To summarize, we make the following contributions this paper.
CoPRS Formulation. We present an end-to-end MCoT-driven positional perception model for reasoning segmentation, where a language-conditione positional prior serves as an interpretable intermediate aligning instruction understanding with mask prediction.
Unified Framework. We establish unified training framework by combining a GRPO strategy with supervised objective, enhancing reasoning and segmentation single loop and overcom- ing the limitations of prior paradigms.
Positional Prior Interface. A learnable concentration query produces heatmap as a dense po- sitional prior, and lightweight decoder refines into a precise mask. Our design provides both interpretable concentration and strong boundary quality.
Strong Results. CoPRS performs strongly on each split across the RefCOCO series and Reason- Seg, and further analysis elarifies how reasoning output aligns with segmentation performance.
2

---

Published as a conference paper at ICLR 2026
2 RELATED WORK
Referring and Reasoning Segmentation. Referring segmentation requires a model to produce a mask for the entity described in a short instruction. Prior methods such as VLT (Ding et al. 2021), CRIS Wang et al. 2022). LAVT (Yang et al. 2022), ReLA (Liu et al. 2023a), X- X-Decoder (Zou Brounded-SAN (Ren al., 2024a). typically rely on specific text encoders rather than large language models (LLMs) to parse the text and predict the mask. Reasoning segmentation extends this setting longer, compositional instructions with stricter grounding requirements, motivating the two method families outlined next.
et al.) 2023a), SEEM (Zuu et al. 2023b), CD- VITO (Fu et al. 2024).
Latent Reasoning Methods. Advances in multimodal large language models (MLLMs) (Liu al., 2023b Bai et al. 2023) have substantially improved the reasoning capability of vision-language perception. LISA (Lai et al.) 2024) bridges the gap between MLLMS and reasoning segmenta- tion by introducing a special token. Subsequent works, including PerceptionGPT (Pi et al. 2024),
PixelLM (Ren et al. 2024b), SegLLM (Ren et al. 2024b), LaSagnA (Wei et al. (024). OMG-
LLaVA (Zhang et al.) 2024a), GroundHog (hhagg et al., 2024b). ObjectRelator (Fu et al. 2025),
RAS (Cao et al., 2025), leverage LLM latent features and decode them into segmentation masks.
However, they neither reveal intermediate reasoning before the final prediction nor expose it through a transparent interface. In contrast, our approach makes the reasoning process clear via MCoT and visualizes the intermediate as a heatmap, improving interpretability and diagnostic analysis.
Text-based Reasoning Methods. Since SAM (Kirillov et al. 2023) achieves strong segmentation quality when prompted with boxes or points, it is feasible prompt SAM using textual coordinates after a simple format conversion. Recent works, such as SAM4MLLM (Chen et al.] 2024), Seg- Zero (Liu et al. 2025) and Seg-R1 (You & Wu 2025), use MILMS generate textual coordinates of boxes and points via thain-of-thought, and then feed them to SAM for mask prediction. In a similar vein, Text4Seg (Lan et al. 2025) generates textual patch indices and applies CRF (Krähenbühl & Koltun 2011) or SAM for mask refinement. Such sparse, discrete outputs provide limited semantic detail and are sensitive to formatting errors and out-of-image coordinates. To address these issues, our model introduces dense, differentiable positional prior that captures richer semantic detail.
3 METHOD
We first present the model design and data flow in Section 3.1 We then formalize the learning objectives, unifying policy optimization via GRPO on the language path with segmentation super- vision on the vision path Section 3.2 Finally, we detail the training and inference procedures in including data preparation, tokenization, group rollouts, and deterministic inference.
Section 3
3.1 MODEL ARCHITECTURE
Overall Architecture. As shown in Figure 2 CoPRS is built upon a multimodal LLM (MLLM), vision backbone, query head and mask decoder. Given image and text inputs (Zimg: Fxa), a policy model T-)-) generates a token sequence that includes the chain-of- -ffthoughtt (CoT) and a concentration token, and we read the MLLM's hidden states to obtain the concentration token em- bedding. Then the query head Freed maps this embedding to a concentration query. The vision encoder Fene(.) extracts image features image keys. Subsequently, the query attends to the image keys with multi-head attention, yielding a heatmap that serves as a positional prior. Finally, the mask decoder (-) decodes this prior to the predicted mask M.
MLLM Backbone. We use Qwen2.5-VL (Bai et al., 2025) as our MLLM backbone. Follow- ing DeepSeck-RI R1 (Guo et al.] 2025), we adopt multimodal chain-of- (MCoT) to leverage the reasoning capabilities of MLLM on compositional instructions. Specifically, we use an in- struction prompt to elicit both the CoT and a concentration token: given (Zimg. Rmx), the model </think> block and then (ii) output the concentration
is asked to (i) reason in <think>. token <REF -OOS, We obtain the concentration token's embedding Econe via Foome which finds its occurrence and reads the hidden states of LLM. Under this setup, the policy To generates the token sequence yor via next token prediction. Formally, the process given in
1/1 Te(: Yoiz-15 Fimg, P(xt): L I,...,
(1)
Éçone Feonc (H1rr)
3

---

Published as a conference paper at ICLR 2026
Text
Prompt
Samples
CeRoo
Advantages
The family water activity. ensuring their safety while niiig on the water?
0.45
Concentrtion Token
1.18 -0.64
Multi-Modal LLM Tig
0.9
0.4
QK"
Col
Scores
K
Query Head Toest
Ma a
Casa
Vision
Mosk Decoda
Backbone
Fene
Triinide
Positional Prior
Troznn
Figure 2: Overall architecture. Given image and text inputs, the policy generates CoT and concen- tration tokens, which query image keys to generate positional prior, that is then decoded to masks. The policy and segmentation modules are jointly optimized.
where Vir includes both the CoT and the concentration token.
From Keys and Query to Positional Prior. The vision backbone encodes Zime into image fea- tures, which we map vision keys K via multilayer perceptron (MLP) applied to the backbone output. In practice, we choose ViT-H-an an image encoder from SAM (Kirillov et al. 2023) as the
vision backbone and an MLP query head projects Evons into the concentration query Q. Subse- quently, we compute scaled dot product multi-head attention scores (Vaswani et al. 2017) between
Q and K, and we use two stacked 2D convolutional layers denoted Fiue (+) aggregate features across heads. Formally, the computation is defined in the following equations.
K Fone ((img)))))
Fixcdd (Econc)
H perior Fruse (I(QW)) (WWK) /de
del
Thooo
where QERd,, K KERHxWxd, W ERd,xdn W.K. Rddx da the head dimension, Thead is the number of heads, and Fruse R + HWW Details are provided in Algorithm
Lightweight Decoder. Our mask decoder comprises two submodules. First, three stacked 2D con- volutional blocks resample the fused positional prior, producing feature map the decoder res- olution. Second, we choose Two-Way Transformer following the SAM decoder design (Kirillov al. 2023), which performs bidirectional cross attention between the image features and the po- sitional prior. This lightweight design has 4.7M parameters and enables the prior to guide dense segmentation. Formally, formulate the process as
M = Facc (K, H peiior)
(3)
3.2 LEARNING OBJECTIVES
Unified Objective. We train the whole system end-to-end with single objective that couples re- inforcement learning on the language path with segmentation supervision on the vision path. For
each (Xing F.x)), the policy no rolls out group of responses {y)(), G with the group size G, and we compute a GRPO loss LoRRO from the advantages. In parallel, the positional prior Hprior and
the predicted mask M are supervised against the ground truth mask Mg to yield the segmentation loss Lsec. The overall objective is
= Lanro (111)))11)) + Assa (ppiie, M, Mg)
(4)
4

---

Published as a conference paper at ICLR 2026
We compute both terms for each batch and take single backward pass through all trainable modules.
GRPO Objective. Following Shao et al. (202), we optimize те with the GRPO objective. The update ratio Ft, is the likelihood ratio between the current policy we and the old policy Tew at token og,s which is clipped with e introduced in PPO (Schulman et al. 2017) for stability. The
advantage A is computed relative rewards within each group only; details are given in Section A.1 Formally, the policy loss is
L = E., min ra. Ar,, clip(ra,, 1- 11+)+++++) A..)] (++))..))+ t= 1=1: T, i i= 1::G.
(5)
where the update ratio
To (o( 01, Ding: F(a)
Titt To 0z,t O Zing, Tixt
(6)
and the token Oig (0) GRPO further regularizes with a KI. divergence term between the trained policy and the reference policy:
Leroo = L. - 13 Dxx. No Ter)
where / is the coefficient of the KL penalty (See Section A.I)
For each sampled response in the group, design reward function that combines mask quality and CoT format compliance. Specifically, the mask reward score aggregates soft IoU, soft dice, and hard IoU, while the CoT format reward score computed via multiple regular expressions for the string matching. We then normalize both rewards to the range [0, 1] using fixed coefficients. Further implementation details are provided in Section 4.1
Supervised Segmentation Objective. The segmentation loss comprises three complementary terms. (i) A binary cross-entropy (BCE) loss applied to Hprior encourages positional evidence and accurate concentration. (ii) A dice loss (Milletari et al. 2016) on the predicted mask M directly supervises mask quality. (iii) focal loss (Lin et al., (017) on the mask logits emphasizes hard pixels and fine-graines structures. All losses are computed only over the original image region and averaged per image over the batch, with the dice loss coefficient Ad and focal loss coefficient Xy being reported in Section 4.1 Formally, the segmentation loss is
LSFG = LACE Hpriory Mg) + >aLrICR (M, Mg) + LFOCA Mg.).
(8)
3.3 TRAINING AND INFERENCE
Data Preparation. Before entering the Fene. we resize each image so that its longer side is 1024 pixels while preserving aspect ratio, then pad it to 1024 x 1024. We apply the same transforms to the masks to maintain coordinate alignment during loss computation. For the policy To. we cap the input at 705,600 pixels (900 vision tokens). If an image exceeds this cap, downsample while preserving aspect ratio for the policy input.
Training Procedure. As shown in Figure 2 during training we tokenize (Zmmg Fra), replicate each pair for G times, and feed these copies to the vg to generate G responses. For each response in the group, the reward function assigns a scalar score, and the scores are converted into advantages for computing Larro, which updates only the MLLM parameters. In the same batch, Timg is resized and padded, then encoded by the vision backbone, and decoded to M for computing Lsec. which updates all trainable modules. We optimize both losses jointly in each iteration.
Inference Procedure. At inference, (Zing. Fira) ) is used without replication. vo with determin- istic next token prediction to produce a single response that includes the concentration token. We then apply the same forward path as in training produce mask logits. Finally, remove padding, resize to the original image size, and threshold the logits at zero to obtain the binary mask.
EXPERIMENTS
Research Questions. In this section, we aim answer the following research questions:
5

---

Published as a conference paper at ICLR 2026
Table 1: Comparison of methods on RefCOCo, ReiCOCO+, and ReiCOCOg datasets.
RefCOCO
RefCOCO+
RofCOCOg
Model Type
Method
VLT
67.5
70.5
56.3
61.0
50.1
55.0
57.7
CRIS
70.5
73.2
66.1
62.3
68.1
53.7
59.9
60.4
Methods without LLMs
LAVT
72.7
75.8
68.8
62.1
68.4
55.1
61.2
62.1
RcLA
76.5
70.2
71.0
65.0
66.0
X- X-Decoder
64.6
SEEM
LISA-7B
74.9
79.1
72.3
65.1
70.8
58.1
67.9
70.6
LISA-13B
76.0
78.8
72.9
65.0
70.2
69.3
70.5
Pereeption(iP1-78
75.1
78.6
71.7
68.5
73.9
61.3
70.3
71.7
PenceptionPP.133
79.1
PixelT M.
73.0
76.5
68.2
66.3
71.7
58.3
69.3
70.5
Reasoning
Latent
LaSagnA-7B
78.7
73.8
66.4
70.6
70.6
71.9
Mr
80.2
81.5
75.4
70.3
73.0
62.5
72.6
73.6
OMG-LLaVA
78.0
80.3
74.1
69.1
73.1
63.0
72.9
72.9
GrundHogg-7B
78.5
79.9
75.7
70.5
75.0
74.1
74.6
GLaMM-7H
79.5
83.2
76.9
72.6
78.7
64.6
74.2
74.9
SAM4MLLM-7B
79.6
82.8
76.1
73.5
77.8
74.5
75.6
Seg. R1-3B
69.9
76.0
64.9
59.1
66.8
67.9
67.3
Text-based
Sog-R1-7B
74.3
78.7
67.6
62.6
70.9
$7.9
71.0
71.4
Reasening
Sog-Zero-3B
79.3
73.7
71.5
Seg-Zero-7B
80.3
76.2
72.6
Texi4Seg
72.1
Tex--Seg-13BB
80.2
82.7
77.3
73.7
78.6
67.6
74.0
75.1
Positional
CoPRS-3B
83,9
75.6
71.8
78.9
66.5
74.8
73.7
Prior
CoPRS-7B
81.6
85.3
79.5
75,9
80.3
69.7
76.2
76.2
RQ1: Does CoPRS achieve higher accuracy in reasoning segmentation and state-of -fat-------h-rtttt results on standard benchmarks compared to prior methods?
RQ2: How the CoT, the positional prior H Hpriors and the predicted mask M mutually corre- lated, i.e., does higher CoT quality align with stronger positional priors and better segmen- tation accuracy?
RQ3: Do the GRPO settings, supervised segmentation losses, and MLLM/vision backbone choices each contribute to performance, and does our unified objective with the default backbones outperform these alternatives?
4.1 EXPERIMENTAL SETUP
Datasets and Metrics. We evaluate CoPRS by conducting experiments on four datasets. We train CoPRS-3B and CoPRS-7B separately on the training sets RefCOCO, RefCOCO+ Ref- COCOg. To prevent data leakage, remove from the training data all COCO images that appear in the validation or test splits of Re(COCO(+/g). We evaluate on the official validation and test splits of RefCOCO(+/g). We further assess zero-shot reasoning segmentation by evaluating Reason- Seg (validation and test) without training on its images. Consistent with common practice in prior work (e.g., Lai et (2024)), we adopt intersection over union (IoU) metrics. Specifically, we report cloU (the cumulative intersection over the cumulative union) on RefCOCO(+/g). and both cloU and gloU (mean of per-image loU) ReasonSeg.
Baselines. We compare our method with 20 prior works grouped into three categories. Meth- ods without LLMs. including VLT (Ding et al., (202)), ReLA (Liu et al. 2023a). X-Decoder (Zou et al. (023a). SEEM (Zou et al. 2023b),
202I), CRIS (Wang et al. 2022). LAVT (aang
Crounded-SAN (Ren 2024a). do not rely on LLM to encode instruction texts for generating masks. Latent reasoning methods, including LISA (Lai et al. 2024), PerceptionGPT (Pi et al. 2024).
PixeILM (Ren et al.) 2024b), LaSagnA (Wei et al. 2024). SegLLM (Wang et al.) 2025a). OMG-
LLaVA (Zhang al. 2024a). GroundHog (Zhang et al. 2024b), GLaMM (Rasheed et al. 2024
RAS (Cao et al. 2025), take hidden features from a large language model and decode them into
segmentation masks. Text-based reasoning methods, including SAM4MLLM (Chen et al. 2024 Seg-Zero (Liu et al. 2025). Seg-RI (You Wu 2025), Text4Seg (Lan et al. 2005),, use an MLLM emit discrete location tokens- box/poin coordinates patch indices, and then convert them to

---

Published as a conference paper at ICLR 2026
masks. For approaches available in multiple parameter scales, report results for all the variants. RAS provides only a version with 13B parameters.
Implementation Details. We train 8 NVIDIA A100 (80 GB) GPUs. Our implementation builds on the VERL codebase. Concretely, we weight the two components of reward function as 0.7 for mask and 0.3 for CoT format. Within the mask score, the coefficients for soft IoU, soft Dice, and hard IoU are set to 0.5, 0.2, and 0.3, respectively, and the format score is computed under specific regular expression rules for five conditions (see Section B.I]. For GRPO, we use sampling num- bers of 2, 4, and 8. Loss coefficients AsEa- X and A are set to 0.3, 3.0 and 10, respectively, for most batches. The base learning rate for the MLLM backbone is set to 2e-6; we apply multipliers of 25x for the concentration query head, and 10x/5x for two submodules of mask decoder. We use the A (Loshchilov Hutter 2019) optimizer with weight decay 0.01. We adopt OneCy- eleLR (Smith & Topin 2019) as the learning rate scheduler, applying cosine decay to each parameter group down to one tenth of its peak learning rate. Full configurations are provided in Section B.3
4.2 OVERALL PERFORMANCE (RQ1)
We compare CoPRS with prior state-of-the-art reasoning segmentation methods two standard benchmarks: the RefCOCO series and ReasonSeg.
Results on Re(COCO(+/g). We follow standard evaluation protocols (Lai et al. 2024) and eval- uate on the RefCOCO series. At matched model sizes, CoPRS-3B and CoPRS-7B 7B achieve the best performance across all RefCOCO, RefCOCO+, and RefCOCOg splits (Table T. Specifically, CoPRS-7B outperforms the latest reasoning methods on all the splits, trailing RAS-13B on only of 8 splits. This advantage stems from learning objectives, strengthening the CoT reasoning capability of CoPRS, which is crucial in reasoning segmentation.
Moreover, compared to Seg-RI and Seg-Zero trained via GRPO, CoPRS achieves significant improvements both model scales, with the 3B model surpassing their 7B counterparts. This fully demonstrates the effectiveness of our de- signed learnable concentration query in con- necting reasoning and segmentation.
Table 2: Zero-shot comparison of methods on ReasonSeg dataset.
Model Type
Method
val
test
gloU
cloU
glol.
cloU
ReLA
22.4
19.9
21.3
22.0
Methods without LLMss
X-Decoder
22.6
SEEM
25.5
21.2
24.3
18.7
Results on ReasonSeg. We evaluate on Rea- sonSeg in a zero-shot setting to validate the generalization ability of CoPRS on complex reasoning segmentation scenarios. From Ta- ble [ our CoPRS also demonstrates superior results on the complex reasoning segmentation task. Meanwhile, we find that methods trained with reinforcement lcarning, such as Seg-R1, Seg-Zero and our CoPRS, consistently outper- form other methods, demonstrating the gener- alization benefits of reinforcement learning for segmentation models.
Grounded-SAM
26.0
14.5
21.3
16.4
IISA-7B
53.6
$2.3
48.7
48.8
LISA-13B
57.7
60.3
53.8
50.8
Latent Reasoning
LaSagnA-7B
47.2
SegL
57.2
GroundHog-7B
56.2
SAM4MLLM-7D
46.7
48.1
Seg-R1-3B
60.8
56.2
55.3
46.6
Text-based
Reasoning
Seg Zero-3 3B
Seg-R1-7B
58.6
58.2
53.1
56.1
48.6
53.7
Seg-Zero-78
62.6
62.0
52.0
Positional Prior
CaPRS 3B
67.3
60.6
57.8
52.7
CoPRS-7B
65.2
64.5
59.8
55.1
4.3 CORRELATION ANALYSIS AND VISUALIZATION (RQ2)
Correlation Analysis Methodology. We first analyze the correlation between the positional prior Hpeier and the predicted mask M during both training and inference. We then analyze how the quality of CoT correlates with both Hprior and M, thereby linking the linguistic reasoning to the visual outputs. We plot the corresponding training losses and evaluation metrics as scatter points to make the relationship clear. Additionally, we use ordinary least square regression to plot the regression line y â + år and the mean confidence bands ((x) + s.e. (f(z)). where s.e. //1 (5r (x-a)) E.= with 77 = 10 for visual clarity and being residual standard error.
Correlation between Heatmap and Mask. During training. panels (d) in Figure 3 blue points, each representing one training batch. The x-axis is 1 LEce prioo:; Mg). which increases

---

Published as a conference paper at ICLR 2026
00
Mam Mal
T- M,.""
1- -- M,J (c) Ears AAKE Bcf0000
7- 1- Mal
Cocs Coka BaCOoo
(b) Laca Epru RdCDCO
(d) Eun Aasa BdCOCO Saio
1A Mad
1:01
141 M0
My 34.1
(0) Flyne ve. AZ Ioll on SHOOCU
(1) Myeies ve. A2: tall on BBR000O+
(e) Mlgie vo. MP: toll StvtOOOCOO
(h) Mpuier Ne. M IoU on Beowowniing
Figure 3: Correlation analysis between the positional prior H prior and the predicted mask M during training and inference on RefCOCO(+/g) and ReasonSeg. Each blue point represents one training batch, while each red point represents one inference instance. Ordinary least squares (OLS) regression lines and mean confidence bands are overlaid.
as the prior better matches Mg- The y-axis is 1 Luicu (M, Mg).. which is higher when M converges to Mg. The points exhibit low dispersion, reflecting stable loss with batch size of 128. Across all datasets, the scatter patterns and correlation coefficients R >> 0.7 indicate strong positive association between H prior and M.
During inference, panels (h) in Figure show red points, each representing one inference in- stance. The x-axis is IoU between prior and Mg.. i.c., the mask quality if the prior were used directly with no decoding. The y-axis is the IoU between M and Mg. standard segmentation metric. As in training, the scatter pattern and correlations R > 0.7 reveal strong positive rela- tionship across test splits.
observed that the regression lines, confidence bands and most points lie above y x. This trend indicates that the positional prior already concentrates well, while the decoder further refines it to a precise mask.
Correlation between CoT and Segmentation Quality. While Figure 3 already confirms the alignment between the heatmap and the final masks, it does not yet quantify how well the CoT reasoning itself aligns with these visual outputs. To make this link more explicit, we additionally use Gemini-2.5-Flashh (Comanici al. 2025) as an independent automatic evaluator. Inspired by Yin et al. (202)). ) we compute a consistency score in 1, (weighted average over four dimensions: logical correctness 0.3, task relevance 0.2, vi- sual consistency 0.3, localization accuracy 0.2) between the image-instructiin pair and the gen- erated CoT the RefCOCO+ testA split. The scatter plots in Figure 4 show clear positive cor- relation between CoT consistency scores and both Heatmap IoU and Mask IoU, This quantitative evidence directly supports that better CoT reasoning quality leads to better segmentation perfor- mance in CoPRS.
Hostasap Ioll
Mask Kol-
R 0.48
1001
ČGT Consistancy Score
CoT Consistency
Figure 4: Correlation between CoT quality and segmentation quality (Heatmap/Mask IoU) on RefCOCO+. OLS results are overlaid.
Visualization Results. We present zero-shot visualizations on ReasonSeg, shown in Figure 5 After MCoT reasoning, the positional prior indicates all instances relevant to the instruction (yel- low), with the target instance most strongly concentrated (deep red). Figure in Appendix presents
8

---

Published as a conference paper at ICLR 2026
txt
Vimg
H
prioe
M
17811203011 T835wilbed
The the picture
Smoking is common activity for some while having picture What nshed of ciganethe?
holf other the brean boud. eneion make nurhbble temporanly
dispesing dedicated oohnney. though
The kandmanii in Purii cxpicitd in the imoge in the Are de Thompph,, The icenc necckassict crckitechurs, fesst lange cantiol anch adorned aculptural relieff and atchaec, dominates the soese The ennuctandl slewened poettin
11311045000 Sodiiiieie
20868976846 4c76596925 O
On a summer beroecue,
excess naterr pisture used colonder (strciner))
common to serve grilled corn the cob. What corn before serving?
corn, per
water drain caay while holding the camn plack, essrrig by sccensiblc aurving.
from
bettane mavoeqaie like nahural ottire interactione resting howd, slight distingsse stotic.
1011247804 0telich2005.
The real person
Figure 5: Sample visualizations. With sample ID exposed, all samples are from the ReasonSeg test split. From left to right: image-text pair, positional prior, predicted mask, and chain of thought.
additional visualizations. Additional failure cases in Figure 7 in the Appendix show that CoPRS mainly struggles with very small objects that disappear at our current input resolution, and dense groups of similar instances where text alone cannot reliably disambiguate the target.
4.4 ABLATION STUDY (RQ3)
To gain deeper understanding of the contributing factors, we perform ablation studies on Ref- COCO+ with different MLLM backbones and varied vision backbones, and further ablations of CoPRS-7B on RefCOCO+, RefCOCOg, and ReasonSeg. We systematically examine MILM back- bone choice, vision backhone choice, GRPO group size, training mode, reward coefficients, and segmentation loss combinations.
MILM Backbone. For ablating the MILM
Table 3: Effect of MLLM Backbone Choice.
backbone, we additionally train CoPRS with
Gray denotes the default backbone.
LLaVA-1.5-7B/13BB on RefCOCO+. Table 3
reports cloU metries of CoPRS versions with both LLaVA-1.5 and Qwen2.5-VL series. As
Method
Backbone
val
testA
lestB
CoPRS-3B
Qwen2 5- 5-VL
71.8
78.9
66.5
expected, performance increases with backbone capacity, but the gains across different MI.LM backbones are relatively modest. This indicates that CoPRS is not sensitive to the specific MLLM
CoPRS-7B
Qwen2
75.9
CoPRS-7B
LLaVA-155
73.1
66.4
CoPRS-13B
LLaVA-155
75.5
80.3
70.7
architecture and that our improvements largely transfer across different backbone choices. Together with the comparisons to prior work under the same VA-155 backbone (Table [ this suggests that our gains are complementary to backbone strength rather than being tied particular MLLM.
Vision Backbone. As shown in Table 4 we ab- late SAM backbones (VYT-B/L/H) on RefCOCO+ with a fixed Vwen2.5-VVL-7B MILM and report the total parameters of the full pipeline. Larger vision backbones bring slightly better segmenta- tion performance, but the improvement modest and the overall trend remains stable across sizes. WITH
Table 4: Effect of Vision Backbone Choice.
Gray row denotes the default backbone.
Backhone
#ParamstB)
val
testA
testB
VIT-
8.60
74.8
78.9
68.5
8,93
75.9
80.3
69.7
Additionally, vision backbones constitute only a small portion of the total parameters, so scaling them up only marginally increases overall computational cost.

---

Published as a conference paper at ICLR 2026
100
BattOC Hemontic
TusA bwih
4EB
dgl (Tog)
200
168
Size
Training Modo
Misk Coof
(Dice. Fosal) Coof
(a) Effect of GRPO group (b) Effect of training mode (c) Effect of mask reward (d) Effect of segmentation size
choice
coefficient.
loss coefficients.
Figure 6: Ablation studies on GRPO group size, training mode, mask reward coefficient, and seg- mentation loss coefficients. (a) is evaluated on all splits of RefCOCO+, while (b)-(d) are evaluated on the test split of each dataset. Bold x-axis labels mark the default settings.
GRPO Group Size. We study the effects of GRPO group size during training. The group size G denotes the number of responses sampled per question during rollout. As shown in Figure 6aa increasing G' improves performance across splits of RefCOCO+. To quantify efficiency, we also report the total number of GRPO samples required to reach convergence (loss fluctuation < 10%
over 300 steps) for G {2, 4,8, 16). Particularly, the number of samples for convergence does not grow linearly with G. because larger groups offer more diverse candidates per step, improving exploration and the contrast between positive and negative samples. Empirically, we find that G strikes good trade-off between efficiency and performance.
Training Modes. We compare reinforcement learning, segmentation supervision. and a combined objective for CoPRS-7B. As shown in Figure 6bb the combined objective achieves the best perfor- mance. This suggests that reinforcement learning strengthens reasoning, while supervised signals sharpen mask generation. Together they are more effective for complex reasoning segmentation.
Reward Coefficients. We evaluate the impact of reward mixing ratio between mask reward score and format score. Figure comparee their combinations, where the format score is one minus the mask score. As the coefficient on the mask reward increases from 0 to 0.7, cloU improves across all three datasets, but pushing it further to 1.0 slightly degrades performance. 'his pattern suggests that the segmentation term is the main driver of segmentation quality, while keeping a small contribution from the format score helps regularize the policy and improves generalization, especially out of distribution data (ReasonSeg). We set the 0. 0.7/0.3 weighting by default, with the segmentation reward dominant and the format score acting regularizer, and Figure 6c supports this choice.
Segmentation Loss Combinations. We compare segmentation loss configurations with varying coefficients (see Figure bdh assess the contribution of each component, with BCE weight fixed at I. To avoid the prohibitive cost of LLM experiments, we only probe a few representative weight settings, which already show trends consistent with our expectations. Adding a focal loss term, which emphasizes hard pixels and fine-grained structures, improves segmentation performance. The relative weight between focal and dice loss also affects the balance between global and local mask quality.
CONCLUSIONS
In this work, we propose CoPRS, connecting language reasoning with segmentation via an inter- pretable and differentiable interface. CoPRS implements this idea with a learnable concentration query to produce a positional prior instantiated as heatmap, from which precise masks are de- coded, within unified framework combining reinforcement learning and segmentation supervision. This interface avoids feeding hidden features to the decoder or representing positions in text, instead providing direct, interpretable alignment between reasoning and mask generation. Empirically, CoPRS attains strong performance across datasets. Further analysis shows that CoT trajectory and heatmap quality strongly correlate with final mask accuracy, and sample visualizations show the same pattern. Overall, CoPRS delivers strong concentration from reasoning and predicts precise masks in unified formulation, providing starting point for perception aligned with instructions.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
Reproducibility Statement. We point readers to the fundamental setup in Experimental Setup (Sec- tion and the appendix Implementation Details (Section B which concisely summarizes the pipeline implementation (Section B.I.. the design details (Section B.2 and the training configura-
tion (Section B.3). These sections contain the information needed to reproduce our results.
LLM Usage Statement. Consistent with policies on LLM usage, we used an LLM only for language polishing (see Section B.4 for details). All ideas, experiments, and analyses were produced and verified by the authors, who take full responsibility.
ACKNOWLEDGMENTS
We sincerely thank the anonymous reviewers and chairs for their efforts and suggestions, greatly helping us improve the manuscript. This work is supported in part by the National Natural Science Foundation of China under grants 62536003 and 624B2088, and in part by the project of Peng Cheng Laboratory (PCL2025A14).
REFERENCES
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, local- ization, text reading, and beyond, 2023.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2.5-vl technical report. arXiv preprint 13323,, 2025.
Shengcao Cao, Zijun Wei, Jason Kuen, Kangning Liu, Lingzhi Zhang, Jiuxiang Gu, HyunJoon Jung, Liang- Yan Gui, and Yu-Xiong Wang. Refer anything with vasion-language prompts. arXiv preprint arXiv:2506.05342, 2025.
Yi-Chia Chen, Wei-Hua Li, Cheng Sun, Yu-Chiang Frank Wang, and Chu-Song Chen. Sam4mllm: Enhance multi-modal large language model for referring expression segmentation. European Conference Computer Vision, pp. 323-340. Springer, 2024.
Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capa- bilities. arXiv preprint arXiv:2507.06261 2025.
Henghui Ding, Chang Liu, Suchen Wang, and Xudong Jiang. Vision-language transformer and query generation for referring segmentation. In Proceedings the IEEE/CV international con- ference on computer vision, pp. 16321-16330, 2021.
Hanze Dong, Wei Xiong, Bo Pang, Haoxiang Wang, Han Zhao, Yingbo Zhou, Nan Jiang, Doyen Sahoo, Caiming Xiong, and Tong Zhang. Rlhf workflow: From reward modeling to online rlhf. arXiv preprint 44X5v:2405.0788 00440000076,,,. 2024.
Yuqian Fu, Yu Wang, Yixuan Pan, Lian Huai, Xingyu Qiu, Zeyu Shangguan, Tong Liu, Yanwei Fu, Lue Van Gool, and Xingqun Jiang. Cross-domain few-shot object detection via enhanced open-set object detector. In ECCV, 2024.
Yuqian Fu, Runze Wang, Bin Ren, Guolei Sun, Biao Gong, Yanwei Fu, Danda Pani Paudel, Xuan- jing Huang, and Lue Van Gool. Objectrelator: Enabling cross-view view object relation understanding across ego-centric and exo-centric perspectives. ICCV, 2025.
Daya Guo, Dejian Yang, Haowei Zhang. Junxiao Song. Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, al. Deepscek-rl incentivizes reasoning in Ilms through reinforcement learning. Nature, 645(8081):633 -388 2025. ISSN 1476-4687. doi: 10.1038/ $415866-02--09422-7

---

Published as a conference paper at ICLR 2026
Yanming Guo, Yu Liu, Theodoros Georgiou, and Michael S Lew. review of semantic segmentation using deep neural networks. International journal of multimedia information retrieval, 7 93, 2018.
Abdul Mueed Hafiz and Ghulam Mohiuddin Bhat. A survey on instance segmentation: state of the art. International journal of multimedia information retrieval, 93)):111-189, 2020.
Jian Hu, Zixu Cheng, Chenyang Si, Wei Li, and Shaogang Gong. Cos: Chain-of-shot prompting for long video understanding. arXiv preprint arXiv:2502006428 2025.
Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao. Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-rl: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.06744 2025.
Sahar Kazemzadeh, Vicente Ordonez, Mark Matten, and Tamara Berg. Referitgame: Referring to objects in photographs of natural scenes. In Proceedings of the 2014 conference on empirical methods natural language processing (EMNLP), pp. 787-798, 2014.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yer Lo, et al. Segment anything. In Proceed- ings of the IEEE/CVF international conference on computer vision, pp. 4015 4015-4026, 2023.
Philipp Krähenbühl and Vladlen Koltun. Efficient inference in fully connected erfs with gaussian edge potentials. Advances neural information processing systems, 24, 2011.
Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia. Lisa: Rea- soning segmentation via large language model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9579-9589, 2024.
Mengcheng Lan, Chaofeng Chen, Yue Zhou, Jiaxing Xu, Yiping Ke, Xinjiang Wang, Litong Feng, and Wayne Zhang. Text4seg: Reimagining image segmentation as text generation. In The Thir- teenth International Conference Learning Representations, 2025.
Tsung-Y Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, pp- 2980-2988, 2017.
Chang Liu, Henghui Ding, and Xudong Jiang. Gres: Generalized referring expression segmentation. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp-
23592-23601, 2023a.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892-34916, 2023b.
Yuqi Liu, Bohao Peng, Zhisheng Zhong, Zihao Yue, Fanbin Lu. Bei Yu, and Jiaya Jia. Seg- zero: Reasoning-chain guided segmentation via cognitive reinforcement. arXiv preprint arXiv:2503.065520 2025.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Confer- ence on Learning Representations, 2019.
Junhua Mao, Jonathan Huang, Alexander Toshev, Oana Camburu, Alan L Yuille, and Kevin Murphy. Generation and comprehension of unambiguous object descriptions. Proceedings of the IEEE conference on computer vision and patlern recognition, pp. 1 11-20, 2016.
Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmadi. V-net: Fully convolutional neural net- works for volumetric medical image segmentation. In 2016 fourth international conference on 3D vision (3DV), pp. 571- Icee, 2016.
Long Ouyang, Jeffrey Wu, Xu Jiang. Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to fol- low instructions with human feedback. Advances in neural information processing systems, 35: 27730-27744, 2022.
12

---

Published as a conference paper at ICLR 2026
Renjie Pi, Lewei Yao, Jiahui Gao, Jipeng Zhang, and Tong Zhang. Perceptiongpt: Effectively fusing visual perception into Ilm. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 27124-271333 2024.
Hanoona Rasheed, Muhammad Maaz, Sahal Shaji, Abdelrahman Shaker, Salman Khan, Hisham Cholakkal, Rao M Anwer, Eric Xing, Ming-Hsuan Yang, and Fahad S Khan. Glamm: Pixel grounding large multimodal model. In Proceedings of the IEEE/CVE Conference on Computer Vision and Puttern Recognition, pp. 13009-13018, 2024.
Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, al. Grounded sam: Assembling open-world models for diverse visual 2024a.
tasks. arXiv preprint arXiv:2401.144
Zhongwei Ren, Zhicheng Huang, Yunchao Wei, Yao Zhao, Dongmei Fu, Jiashi Feng, and Xiaojie Jin. Pixellm: Pixel reasoning with large multimodal model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 26374-26383, 2024b.
John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel. High-
dimensional continuous control using generalized advantage estimation. arXiv preprint 0rxivv1506.02438,, 2015.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint 1rXiv:1707003477, 2017.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models. arXiv preprint arXiv:2402.03300 2024.
Leslie N Smith and Nicholay Topin. Super-convergence: Very fast training of neural networks using large learning rates. In Artificial intelligence and machine learning for multi-domain operations applications, volume 11006, pp. 369-386, SPIE, 2019.
Cheng Tan, Jingxuan Wei, Zhangyang Gao, Linzhuang Sun, Siyuan Li, Ruifeng Guo, Bihui Yu, and Stan Z.L. Li. Boosting the power of small multimodal reasoning models to match larger models with self-consistency training. In European Conference on Computer Vision, pp. 305-322. Springer, 2024.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, tukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural informa- tion processing systems, 30, 2017.
XuDong Wang, Shaolun Zhang, Shufan Li, Kehan Li, Konstantinos Kallidromitis, Yusuke Kato, Kazuki Kozuka, and Trevor Darrell. SegLLM: Multi-round reasoning segmentation with large language models. In The Thirteenth International Conference on Learning Representations, 2025a.
Yaoting Wang, Shengqiong Wu, Yuecheng Zhang, Shuicheng Yan, Ziwei Liu, Jiebo Luo, and Hao Fei. Multimodal chain-of-thouge reasoning: A comprehensive survey. arXiv preprint arXiv:2503.12600 2025b.
Zhaoqing Wang, Yu Lu, Qiang Li, Xunqiang Tao, Yandong Guo, Mingming Gong, and Tongliang Liu. Cris: Cliv-driven referring image segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1 11686-11695, 2022.
Cong Wei, Haoxian Tan, Yujie Zhong, Yujiu Yang, and Lin Ma. Lasagna: Language-based segmen- tation assistant for complex queries. arXiv preprint arXiv:2404.08500 08506, 2024.
Zhao Yang, Jiaqi Wang. Yansong Tang, Kai Chen, Hengshuang Zhao, and Philip HS Torr Lavt: Language-aware vision transformer for referring image segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 18155 18155-18165, 2022.
Zhangyue Yin, Qiushi Sun, Zhiyuan Zeng, Qinyuan Cheng, Xipeng Qiu, and Xuan-Jing Huang. Dynamic and generalizable process reward modeling. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 4203- 0233, 2025.
13

---

Published as a conference paper at ICLR 2026
Zuyao You and Zuxuan Wu. Seg-rl: Segmentation can be surprisingly simple with reinforcement learning. arXiv preprint arXiv:2506.22624. 2025.
En Yu, Kangheng Lin, Liang Zhao, Jisheng Yin, Yana Wei, Yuang Peng, Haoran Wei, Jianjian Sun, Chunrui Han, Zheng Ge, al. Perception-rl: Pioneering perception policy with reinforcement learning. arXiv preprint arXiv:2504.07954 2025a.
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, al. Dapo: open-source Ilm reinforcement learning system at scale. arXiv preprint arXiv:2503: 14476, 2025b.
Tao Zhang, Xiangtai Li, Hao Fei, Haobo Yuan, Shengqiong Wu, Shunping Ji, Chen Change Loy, and Shuicheng Yan. Omg-llava: Bridging image-level, object-level, pixel-level reasoning and understanding. Advances in neural information processing systems, 37:71737-71766 2024a.
Yichi Zhang. Ziqiao Ma, Xiaofeng Gao, Suhaila Shakiah, Qiaozi Gao, and Joyce Chai. Groundhog: Grounding large language models to holistic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 14227-14238, 2024b.
Zhuosheng Zhang, Aston Zhang, Mu Li, hai zhao, George Karypis, and Alex Smola. Multimodal chain-of- thougtt reasoning in language models. Transactions Machine Learning Research, 2024c. ISSN 2835-8856.
Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, et al. Group sequence policy optimization. arXiv preprini arXiv:2507.1807,, 2025.
Xueyan Zou, Zi-Yi Dou, Jianwei Yang. Zhe Gan, Linjie Li, Chunyuan Li, Xiyang Dai, Harkirat Behl, Jianfeng Wang, Yuan, et Generalized decoding for pixel, image, and language. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 15116- 15127, 2023a.
Xueyan Zou, Jianwei Yang, Hao Zhang, Feng Li, Linjie Li, Jianfeng Wang, Lijuan Wang, Jian- feng Gao, and Yong Jae Lee. Segment everything everywhere all at once. Advances in neural information processing systems, 36: 16799999-19782, 2023b.
14

---

Published as a conference paper at ICLR 2026
A APPENDIX: GRPO THEORY AND ADDITIONAL RELATED WORK
A.1 GROUP RELATIVE POLICY OPTIMIZATION
The reasoning ability of MLLMs is a key factor that influences the reasoning segmentation perfor- mance. Since Reinforcement Learning (RL) is an effective way to improve the reasoning ability of LLMs and MLLMs, we employ it to enhance the reasoning segmentation capability of our method.
Proximal Policy Optimization (PPO) (Schulman et al. 2017) is widely used in the RL fine-tuning stage of LLMs. PPO is an actor-critic RL algorithm, which optimizes LLMs by maximizing the following surrogate objective:
Luro = E ~ P Tew (0l)) (े E/ min Ta [o._
[aofell A,, clip [eloollooall [ 1 - -, s.11+s) A.] (9)
where To and Tews are the current and old policy models, and q, are questions and outputs sampled from the question dataset and the old policy Te. respectively. E is a clipping-relate hyper-parameter introduced in PPO for stabilizing training. The advantage, At, is based the ward [r2x} and a learned value function Vy,, computed by applying Generalized Advantage Estima- tion (GAE) (Schulman al., 2015). Furthermore, per-token KL penalty from reference model is added to the reward at each token to mitigate over-optimization of the reward model (Ouyang et al.,
2022 denoted as:
To(o:19. oce)
Tt rqp o5+) 3 log
(10)
Ter (g. ocer)
where rip is the reward model, Net is the reference model, which usually the initial policy model, and (8 is the coefficient of the KL penalty.
PPO relies on separate value function that is typically another model of comparable size to the policy model, imposing heavy memory and computational costs. Additionally, the value function is treated as baseline in the calculation of the advantage for variance reduction. Moreover, in the LLM context, usually only the last token is assigned a reward score by the reward model, which may complicate the training of a value function that is accurate at each token. Group Relative Policy Optimization (GRPO) (Shao et al. 2024) is proposed to address these drawbacks by obviating the need for additional value function upproximation
PPO, and using the average reward of multiple sampled outputs, produced in response to the same question, as the baseline. Specifically, for each question q. GRPO samples a group of outputs {01, 02, og from the old policy Towi and then optimizes the policy model by maximizing the following objective:
CGRPO E
mo (Olo)
(I)) Te (o op,e)) A,,,,elpp Tolor,lu. or.e)) -, +e) Acc - -pnL (mellaal)
min
ne (or.
where E and 3 are hyper-parameters. and A is the advantage calculated based on relative rewards of the outputs inside each group only. For each question g. group of outputs {0r, (01,02," , are sampled from the old policy model To..s- The score of the outputs obtained through a reward model, yielding G rewards fri,72.... .rG} correspondingly. The advantages Å. for all tokens in output are defined the normalized reward, i.e., A., Å. r. n--mca(rr) In addition, GRPO sidrr) directly adds the KL divergence between the trained policy and the reference policy to the loss, avoiding complicating the calculation of A,,-- The KI. divergence is estimated by the following unbiased estimator:
Tneroondl..0.com log Ter(on Or.ce)
DeL. [
(12)
To (o q1 0i,ct)
To (1, Oi,ct)
A.2 ADDITIONAL RELATED WORK
GRPO Guided Reinforcement Learning. The GRPO (Shao et al. 2024) strategy addresses re- ward hacking in RLHF (Dong et al., 2024) by penalizing deviation from reference policy. How- ever, its reliance on a static reference limits adaptability. This spurred key optimizations: Dynamic Advantage-based Policy Optimization (DAPO) YYu et al. (025b) introduces moving trust region by dynamically updating the reference policy via an exponential moving average, enabling more sta- ble, long-term improvement. Another significant limitation of the original GRPO is its token-level
15

---

Published as a conference paper at ICLR 2026
optimization, which can be computationally intensive and may lead training instability. Address- ing this, Sequence Policy Optimization (GSPO) (Zheng et al., 2025) was proposed to shift the optimization granularity from the token level to the sequence level. By defining sequence-level importance ratio and advantage, GSPO significantly reduces computational overhead and improves training stability, especially for large-scale models.
Multimodal Chain-of- Thoughh. Multimodal thain-of-thought (MCoT) (Wang et al. 2025b) rea-
soning has recently attracted substantial attention, particularly in its integration with MLLMs. Early implementations, such as Multimodal-CoT (Zhang et al.) 2024c), have established a basic MCoT pattern by generating intermediate rationales before predictions. MC-CoT (Tan et al.] 2024) fur- ther refines this paradigm by employing word-level majority during training to enhance the quality of generated rationales. The dependence on high-quality MCoT training data hinders the further improvement of the inference ability of traditional methods. Most recently, the great success of Deepseek-RI (Guo et al., (025) has provided a way (i.e.. GRPO) to enhance L.M inference capa- bilities through model autonomous exploration without the need for expensive CoT annotation data. Inspired by this, subsequent works utilize the GRPO strategy to efficiently enhance the reasoning ability of MLLMs. For example, Vision-R1 (Huang al., 2025) first utilizes existing MLLM and DeepSeek-RR as well as data filtering, through modal bridging to generate multimodal cold start CoT data, and then applies GRPO to further enhance the model's inference capability. Perception- R1 (Yu et al.] (025a) explores the effects of RL on different perception tasks and optimizes the reward modeling to support perception policy learning. In addition, Chain-of-Shot (Hu et al. 2025) further extends GRPO strategy to optimize frame sampling via binary video summaries. In this work, we study hhatmap-basse positional prior that couples MCoT with precise positional percep- tion in unified training framework for GRPO strategy and segmentation supervision, addressing the gap between high-level reasoning and pixel-level segmentation.
B APPENDIX: IMPLEMENTATION DETAILS
B. PIPELINE IMPLEMENTATION
We build on the VERL codebase, which was originally designed for PPO and extended with GRPO functionality.
Sharding Strategy. We shard the VLL.M/policy component using Fully Sharded Data Parallel (FSDP), partitioning parameters across devices during training. The lightweight segmentation mod- ules (query head, Q-V attention, and mask decoder) are left unsharded to avoid FSDP overhead and keep their compute/memory costs low. We apply tensor parallelism across attention heads during autoregressive decoding.
ESDP Workers. We precompute image features offline to reduce compute, the frozen vision backbone is excluded from the training loop. Our framework uses three FSDP workers. (i) The actor contains all trainable modules (the MILM and the segmentation components) and is responsible for parameter updates. (ii) The rollout worker runs the MILM only, taking image and text inputs to generate responses via next token prediction. (iii) The frozen reference worker runs an MLLM as the reference policy to compute the KL term in CoRrO (eq. 7 and includes the segmentation modules decode masks used for computation of mask reward scores and group advantages.
Training Pipeline Implementation. For each annotation, the rollout worker generates G responses for the image-text pair with the current policy by next token prediction, caching the tokens and their log probabilities. The frozen reference worker then runs forward without gradients on the same inputs to compute reference log probabilities for those sampled responses and to decode a mask used in the mask based reward. From each response and its mask signal we compute scalar reward and convert rewards to group advantages. Next, the actor worker runs forward to obtain the policy log probabilities for the sampled responses and the predicted mask. We form the GRPO objective from the actor log probabilities, the stored old log probabilities from rollout, the reference log probabilities, and the advantages, and we form the segmentation objective from the predicted mask and the ground truth mask. The two objectives are summed and optimized jointly in a single backward pass, updating all trainable modules.
16

---

Published as a conference paper at ICLR 2026
Algorithm 1 Generation of positional prior H prior
Require: Image Wing; concentration token embedding Gconei image encoder Fenei query head Fhead
fusion network i projection matrices { W Thass
Jiml
Ensure: Positional prior H prior E H
1: K f Fene (Zing)
> E R
2: Q - -heee (cone)
> Q R4
3: for = 1 to Thead do
4: K, + KW
K E 11 xx Wxd,
5:
qi < QW
E RR
6:
for (u, v) H. x f1,..., W) do
7:
S;(u,v) t ddh
q. (u,
4 S. v) E
8: end for end for
9:
H Price +- Fluee (S:] [5i)==1)
F small conv fusion head, IR X Hxw + HxW
11:
return H prior
B.2 DESIGN DETAILS
Reward Function Design. We scalar mask score in [0, 1]: given predicted mask and ground truth mask, we compute three overlap metrics (soft IoU, soft Dice, and hard IoU) and take their weighted sum with fixed coefficients 0.5, 0.2, and 0.3, respectively, providing stable localiza- tion signal for how well the prediction covers the instance. For valid outputs, the score is 1.0 by default and reduced 0.9 the <think> content is longer than 2048 characters, or if any non-whitespace text appears before <think> or after the special token. Thus the five canonical cases are: invalid (0.0); valid and clean (1.0); valid but long <think> (0.9); valid but extra text before <th nk> (0.9); valid but extra text after the special token (0.9). For each sample, we take weighted sum of these two components as the final reward that is assigned to the last valid response token so that GRPO updates the entire trajectory. The relative weights are specified in Section 4.4
Positional Prior Heatmap Generation. make the computation of the positional prior Hprior fully reproducible, we detail the heatmap generation procedure in Algorithm [ starting from the image keys K, the concentration query Q, and the per-head scaled dot-product scores S, v). The convolutional fusion head Fruse then aggregates (S: 35) into the final positional prior H prio HW
B.3 TRAINING CONFIGURATION
Data and preprocessing. We train on the RefCOCO series. The maximum prompt length is 1300 tokens and the maximum response length is 2000 tokens. For the policy input, images are capped 705,600 pixels and downsampled needed; minimum of 3, 136 pixels enforeed. SAM ViT-H features initialize the vision branch.
Hardware and precision. Experiments run on single node with 8 GPUs. Computation uses 6 for model parameters and fp32 for reductions and buffers.
Parallelism. The policy (VLLM) is trained with Fully Sharded Data Parallel. The rollout service uses tensor parallelism of size 4. The reference worker is also sharded; optimizer state is offloaded.
Batching. Global batch size 16 (before repeating G times for GRPO). For the actor, micro-batch per device is 2 for updates and 8 for experience collection. Rollout batch size is 16 and the group size is = 8 responses per input.
Optimization. We use AdamW with weight decay 0.01 and (81,82) (0.9, 0.999). The base learning rate 1.6 x 10-6 with multipliers 25x (query head). 10x (position/prompt encoder), and 5x (mask decoder). Gradient clipping final division factor of about 6.7 and no warmup. Total planned training steps are 3 31,250. Gradient checkpointing enabled.
max norm of 1.0. The schedule is one eycle with
17

---

Published as a conference paper at ICLR 2026
GRPO settings. We GRPO with sampling number 8. clip ratio 0.2, group-relative advantages, and a fixed KL penalty coefficient 0.2 (low-variance form). The entropy coefficient is 0.0.
Segmentation objectives. Unless noted, ASEo 0.3, Ao 1.5, and Ay 0.0 at the start; at step 1,500 we set Ao = 3.0 and y = 10.0. Losses are computed only on the valid (unpadded) region.
Rollout and decoding. Rollouts 1.0, top-k disabled). Execution uses bfloat16, up to 64 concurrent sequences, and a cap of 17,408 batched tokens. Chunked prefill is enabled. One image is used per sample.
VLLM backend with sampling enabled (temperature 1.0, top-p
B.4 LLM USAGE STATEMENT
In preparing this paper, we used large language model (LLM) for polishing at the sentence level. We do not directly include the text generated by LLM in our paper. Instead, we use it solely reference and for guidance. The model was given the following prompt to guide the text refinement process:
"Slightly polish it sentence by sentence, and give the reasons. Not latex code. Disable online search and do not find citations yourself. You must avoid changing any statistics and avoid distorting my statements."
This prompt was specifically designed to ensure that the LLM's revisions were limited to language refinement and that statistics experimental results were altered. The LLM also instructed not to perform any online searches or generate citations. All final content, including experimental data and results, remains the responsibility of the authors.
C APPENDIX: ADDITIONAL SAMPLE VISUALIZATIONS
Successful Cases. In Figure we present instances from the same category and those relevant to the instruction, all showing elevated responses in the heatmap (yellow regions). More importantly, the heatmap concentrates
the instance specified by the instruction, producing sharp peak over the target (deep red regions). This concentration guides the decoder, yielding masks with accurate boundaries. These results indicate that the MLLM reasons over the image and text input and iden- tifies the correct referent, while the positional prior concentrates the instances for further precise mask prediction.
Failure Cases. In Figure 7 the first two rows depict scenes with many nearby instances, while the last three rows contain very small targets. Two failure modes emerge. (i) Resolution bottleneck: the positional prior is computed at 256x256 and the SAM embeddings at 64x64; when the longer image side exceeds 2k pixels, tiny objects can vanish after resizing and the decoder cannot reliably recover them. (ii) Same class crowd ambiguity: in dense groups of similar objects (e.g., crowds of people), the positional prior often spreads across many candidates with weak contrast, suggesting that text only instruction is insufficient disambiguate near duplicates and that the model has not fully learned the subtle semantic cues needed. These observations suggest that higher resolution inputs multi-scale features, together with stronger instance level language grounding. are likely to improve performance on such cases.
18

---

Published as a conference paper at ICLR 2026
C txt
C Timg
H prior
M gt
3476471336 fo848f1 5 The animal being dragged.
5813271004 eb5e96cf78
In musical performance, there often person leads and directs the musicians singers. Who the picture could be fulfilling this role?
10482337565 990f5dd710. 9910233766599005dd7700.
Something that eats leaves.
12965644696 64bc52ee39.o .
The body parts the bird grip branches,
6524426212 6700231c7d
Crows ore known for their intelligence and curiesity. What part the crow would most likely used to observe its surroundings and gather information?
picture
Figure 7: Failure cases. With sample ID exposed, all samples are from the ReasonSeg test split. From left to right: image-text pair, positional prior and ground truth mask.
19

---

Published as a conference paper at ICLR 2026
xtx
Zimg
H price
M
y 1
50538944557
image
When warling public tronsportat
people watting
large connry covering
people often sook shether to escupe from direct sunl gigtt What the picture can offer shade for people wathing
proucise openvoir
deege, bisscing
San iight slllwing low.
stop?
The place where the leacher write the
6950081780 cf6o4290222
warnebeand mounted well behand him attnnarr slied, blank surface dasigned for anting draaiig with nur kara shalk, perthoned ongle vistble both the teocher and the sackence.
The place where the Teacher
write and
buring a fermil motch, players
004231 1968, 30cb49dded_ o
Morgan
Morgan
Morgan
quertion asior acout the used the slringg furiid game, us shown pe churn. -ow matches distoription of temii rudlect.
often hit the ball with specific tool What tool in the
picture be used to hit the
brand "Pence" wabble confirming
ball
racket....
5100750816 bd297o8792
The question wich part piettanc resnnsunta key dbjechive festball or right whore piay try to acone pointe on torget for scaring the key objach part gool net.
What
represents key jectiveee football or rugby gome, where plyers ettempt to score points?
(gnel) le the
108/5382200,
most suiteble conta for surving holding sliced eap
In various containers fruit Which container would be most suttable for serving and halding sliced apples?
image
lorge black cenl Itss speciosi especily depth unoune accommsdate muth pie opple slices without svencreading offering stobil by accessiblity
The ploces whene
23522064552 17441
The places here people
kitchen idannd The
4581513454 Rca
Some plants have delicate structures that ane visually and ofhen aftract pollinators, In the picture.
The staject is the picture that can used hold the cskes of digenette the brown boul. This size and opos tuo dlsiis make sut le for temponaniyy dispsiigg of sigare sahes, thoajh dedicateel us may.
esomple of such a plant?
4831092391 7859854277.9
hights
The reflection of the
élage retlective profile...
suitsble
apples
4858616208 414004067c.0
The more edvanced eircraf
copecity degth
apple
occusalllly
80152 19832,
image hand
objects learvig on car
ongles slihhtly hundle
creating appearanze loaning
14544353885 12334581133
In donse rainforest, there types plants growing providing habitttt for many animalx What a key feature of the roleforest that helps support the growth these plants?
The key feature of the roinforest depicted the image that supperte pir growth the parsoriial Preshwethr streae wricing throage the fancil flozr
This water dource providei conslitent
supply of stturture esscett of for plant survell
The parsin proporing satch the the the woman nod shiit. She seriiely helling table ternis sacdle (with the brand "Zol"" vaibl) and peattioneed the huble with trakk fecsi, suppesting sls red
16479902818, 0605006044
the person who currently preparing cotch the bal.
Figure 8: Additional successful cases. With sample ID exposed, samples are from the Reason- Seg test split. From left to right: image-text pair, positional prior, predicted mask, and response.
20


Reviews:
[{"review_id": "lPr4POFoie", "reviewer": "Reviewer_vsKC", "summary": "The paper proposes CoPRS, a CoMT approach that extracts a query from the MLLM. The query is used to generate a differentiable heatmap with cross-attention and acts as a positional prior to the segmentation decoder. The key contributions are:\n- A dense, differentiable heatmap is an interpretable link of the MLLM and mask generation. \n- A training framework that combines GRPO and supervised objective.\n- CoPRS achieves good results on RefCOCO series and ReasonSeg.", "questions": "- How sensitive are the reported gains to the MLLM backbone? Ablations with a smaller model or from a  different family (e.g. LLaVa) will show how dependent the model is on the specific architecture.\n- In the GRPO reward design, the choice “0.7 for mask and 0.3 for CoT” might be reasonable, but it is left unexplained.\n- How correct are the CoT reasoning trajectories, and how do they contribute to interpretability?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper addresses a relevant multimodal problem, bridging language reasoning with spatial localization.\n- The positional prior offers intermediate interpretability, given the correlation between the quality of the prior and that of the final segmentation mask\n- Clear empirical gains on evaluation benchmarks.\n- Cleverly combining reinforcement learning with supervised learning", "weaknesses": "- The new architecture adds complexity that must be quantified to justify a reduction in speed by the performance gains.\n- Although CoT reasoning is emphasized, the actual CoT outputs and their role are not examined. There is no evidence that linguistic reasoning aligns with the generated heatmap.\n- Interpretability was studied only through correlation between the heatmap and the final mask, both in the visual domain. The interpretability role of reasoning is unclear.\n- GRPO, although more efficient than PPO, is still expensive on a large backbone. The ablation only compares with/without GRPO, without exploring how reasoning reward affects segmentation quality.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zU1RdGHqsu", "reviewer": "Reviewer_eH3B", "summary": "Free-form Reasoning Segmentation requires coupling language reasoning with visual segmentation. Existing work either uses latent reasoning which offers limited interpretability or text-based reasoning which has limited flexibility. This paper introduces CoPRS, which produces concentration tokens through multimodal chain-of-thought, which are then used to generate heatmaps over the images through attention, and finally producing the segment masks. All components of CoPRS are trained end-to-end using two objectives: a GRPO objective over the output tokens and a supervised segmentation objective over the heatmap and the final predictions. Experiments show that CoPRS achieves competitive performance on RefCOCO and ReasonSeg over existing methods.", "questions": "I don’t quite see how H_prior can always be visualized as in figure 2. Does H_prior always have the fixed dimension size equal to the image? What does each element in K represent? \n\nHow does this differ from latent reasoning method where the latent reasoning is done via an attention mechanism where the attention scores can be extracted (ex. the coder in PixelLM)? Wouldn’t the attention score also be interpreted as heatmaps? \n\nI don’t think it is mentioned in the paper, what is CoPRS an acronym for?\n\nMinor:\n\nTypo: line 297, 406\n\nThe citation for SegLLM is incorrect.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "End-to-end training that allows the model to learn to perform CoT reasoning, generate the query embeddings, and generate the mask simultaneously. \n\nThe approach shows strong results across two benchmarks, comparing against an extensive set of existing methods.\n\nThe paper is well-written and easy to follow.", "weaknesses": "The paper aims to balance interpretability and representational fidelity by generating heatmaps over the images as positional priors. However, this heatmap generation process itself is not interpretable. From my understanding, the training objective encourages the heatmap to match the ground truth masking, which could be seen as a one of the latent reasoning methods discussed in the introduction, and suffers the same downsides of interpretability. \n\nCoPRS relies on a vision backbone to produce the key embeddings that are used to generate the positional prior. This introduces additional computation compared to other methods. Furthermore, the quality of the vision backbone model seems very important, but the paper does not examine the effect of it on the overall performance. \n\nTable 5 and 6 shows ablation on the model’s training objectives. However, there is no in-depth analysis of these results. How are the weights decided?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mbPR1zd1CX", "reviewer": "Reviewer_HRqs", "summary": "This paper introduces CoPRS, a novel Multi-modal Chain-of-Thought (MCoT)-based framework that connects language reasoning to visual segmentation through a differentiable positional prior represented as a heatmap. The model integrates a learnable concentration token within a multimodal LLM to aggregate textual reasoning and visual context, producing a heatmap that serves as an interpretable intermediate between reasoning and segmentation. The approach is trained end-to-end by combining Group Relative Policy Optimization (GRPO) for reasoning and supervised segmentation losses for mask prediction.\n\nEmpirically, CoPRS achieves state-of-the-art results on the RefCOCO, RefCOCO+, RefCOCOg, and ReasonSeg benchmarks, demonstrating both superior performance and better interpretability compared to prior latent- or text-based reasoning methods. The paper also presents correlation analyses showing that the quality of the learned heatmap aligns strongly with final segmentation accuracy, highlighting the causal link between reasoning and perception.", "questions": "- How sensitive is CoPRS to the design of the concentration token prompt (e.g., `<REF_POS>`)?\n\n- Does the GRPO-based reward generalize to new reasoning templates unseen during training? (Does the RL signal capture general reasoning ability rather than just overfit to specific CoT structures?)\n\n- Can the positional prior mechanism transfer to other grounding tasks (e.g., referring tracking or VQA grounding)? (Is the proposed differentiable positional prior is a general paradigm?)", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Overall, writing is clear with intuitive figures. \n\n- The paper introduces an interpretable and differentiable positional prior as an intermediate representation linking language reasoning (via MCoT) to visual segmentation.\n\n- The joint GRPO + supervised objective elegantly integrates reasoning quality and mask precision within a single training loop.\n\n- The heatmap prior provides transparent evidence of where the model is “attending,” supporting qualitative interpretability and quantitative correlation analysis (R > 0.7 between heatmap and mask quality).\n\n- Consistently outperforms both latent reasoning (e.g., LISA, SegLLM) and text-based reasoning methods (e.g., Seg-Zero, Text4Seg) across all RefCOCO variants and ReasonSeg.\n\n- Experimental details are available to reproduce the result. \n\n- The ablation studies on different setups (training mode, reward function, etc) demonstrate further the advantage of the proposed method.", "weaknesses": "- The core modules (GRPO, SAM decoder, etc) are existing frameworks; the contribution mainly lies in combining them rather than introducing fundamentally new architectures. Further clarifications on the technical novelty would strengthen the paper.\n\n- Results are tied to Qwen2.5-VL and SAM encoders; unclear how robust the method is across different MLLM backbones or smaller models.\n\n- Although some failure cases are available in the appendix, the paper does not discuss the limitations of the proposed method explicitly in the main text. Adding discussions on limitations will enhance the clarity of the paper. \n\n- GRPO-based multimodal optimization on large MLLMs is computationally heavy; practical feasibility for wider adoption is not discussed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "l1Aq04uFid", "reviewer": "Reviewer_XDcj", "summary": "This paper introduces CoPRS, a multimodal chain-of-thought (MCoT)-based method for reasoning segmentation that bridges language reasoning to pixel-level segmentation via a differentiable positional prior (heatmap). The key contributions include:  \n\n- A novel interface using a learnable \"concentration token\" to generate positional priors from MCoT reasoning, providing interpretable intermediate heatmaps.  \n- A unified training framework combining Group Relative Policy Optimization (GRPO) for language reasoning and segmentation supervision for mask refinement.  \n- State-of-the-art performance on RefCOCO series and ReasonSeg benchmarks, with ablation studies demonstrating the correlation between heatmap quality and segmentation accuracy.", "questions": "**Q1**: How does CoPRS handle ambiguous positional priors in crowded scenes (Figure 5)? The failure cases suggest sensitivity to instance density – could multi-scale features or contrastive learning between instances mitigate this?  \n**Q2**: For GRPO sampling numbers (Table 4), what computational overhead does G=8 introduce compared to G=2? A latency/accuracy tradeoff analysis would help practitioners.  \n**Q3**: The correlation analysis (Figure 3) shows association but not causation between heatmaps and masks. Could you disentangle whether performance gains stem primarily from GRPO-enhanced reasoning or the decoder architecture?  \n**Suggestion 1**: Add comparisons with RAS-13B and PerceptionGPT-R1 using comparable model sizes.  \n**Suggestion 2**: Include an ablation on the vision backbone (e.g., ViT-H vs. ViT-L) to clarify performance dependencies.  \n**Suggestion 3**: Provide pseudocode or extended equations for the heatmap generation process (Section 3.1) to improve reproducibility.  \n\n**Rebuttal Potential**: Addressing the baseline comparison gap (Suggestion 1) and providing theoretical justification for GRPO’s effectiveness in segmentation could significantly strengthen the contribution narrative. Clarifying the heatmap implementation details (Suggestion 3) would enhance methodological rigor.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "*Originality*: The integration of MCoT reasoning with dense positional priors offers a fresh perspective compared to existing latent-feature or text-coordinate paradigms. The use of GRPO for joint language-reasoning segmentation optimization is a creative combination of reinforcement learning and segmentation tasks.  \n*Clarity*: The architecture diagram (Figure 2) effectively illustrates the pipeline. The distinction between training modes (GRPO vs. supervised) is well articulated.  \n*Significance*: Addresses a critical gap in interpretability for reasoning segmentation systems. The reported 3B/7B model improvements over text-coordinate baselines (e.g., +4.7 cIoU over Seg-R1 on RefCOCOg) demonstrate practical value.", "weaknesses": "*Limited Baseline Comparison*: While compared to latent-reasoning and text-based methods, there is no direct comparison with recent hybrid approaches like PerceptionGPT-R1 or RAS-13B under identical training protocols. This leaves uncertainty about absolute performance claims.  \n*Ablation Depth*: The GRPO hyperparameter study (Table 4) only tests sampling numbers {2,4,8} without justifying this range. More analysis is needed on how group size affects exploration-exploitation tradeoffs in segmentation contexts.  \n*Methodological Ambiguity*: The heatmap generation process (Eq. 3-4) lacks critical implementation details, e.g., how the MLP maps vision backbone outputs to keys, or why two convolutional layers were chosen for F<sup>fuse</sup>. This hinders reproducibility.  \n*Theoretical Limitations*: While empirical correlations are shown, there is no formal analysis of why GRPO’s group-relative advantages are particularly suited for segmentation tasks compared to standard PPO.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)