Perception-R1: Pioneering Perception Policy with Reinforcement Learning
Authors: En Yu
Kangheng Lin
Liang Zhao
jisheng yin
Yana Wei
Yuang Peng
Haoran Wei
Jianjian Sun
Chunrui Han
Zheng Ge
Xiangyu Zhang
Daxin Jiang
Jingyu Wang
Wenbing Tao
Venue: NeurIPS 2025 poster ()
Keywords: MLLM; Perception; Rule-based RL
Primary area: deep_learning
TL;DR: Exploration of  rule-based reinforcement learning (RL) in MLLM post-training for perception policy learning.

Abstract: Inspired by the success of DeepSeek-R1, we explore the potential of rule-based reinforcement learning (RL) in MLLM post-training for perception policy learning. While promising, our initial experiments reveal that incorporating a thinking process through RL does not consistently lead to performance gains across all visual perception tasks. This leads us to delve into the essential role of RL in the context of visual perception. In this work, we return to the fundamentals and explore the effects of RL on different perception tasks. We observe that the perceptual perplexity is a major factor in determining the effectiveness of RL. We also observe that reward design plays a crucial role in further approaching the upper limit of model perception. To leverage these findings, we propose Perception-R1, a scalable RL framework using GRPO during MLLM post-training. With a standard Qwen2-VL-2B-Instruct, Perception-R1 achieves +4.2% on RefCOCO+, +17.9% on PixMo-Count, +4.2% on PageOCR, and notably, 31.9% AP on COCO2017 val for the first time, establishing a strong baseline for perception policy learning.

Perception-Ri: Pioneering Perception Policy with Reinforcement Learning
En Yu 11, Kangheng Lin Liang Zhao
Jisheng Yin Yana Wei', Yuang Peng", Haoran Wei Jianjian Sun
Chunrui Han", Zheng Ge Xiangyu Zhang', Daxin Jiang', Jingyu Wang', Wenbing Tao"
Huazhong University of Science and Technology
Beijing University of Posts and Telecommunications StepPuun "Johns Hopkins University Tingshua University {yuen, wenbingtao) @hust. edu. cn
Abstract
Inspired by the success of DeepSeek-R1, we explore the potential of rule-based reinforcement learning (RL) in MILM post-training for perception policy learn- ing. While promising, our initial experiments reveal that incorporating a thinking process through RL does not consistently lead to performance gains across all visual perception tasks. This leads us to delve into the essential role of RL the context of visual perception. In this work, we return to the fundamentals and explore the effects of RL on different perception tasks. We observe that the percep- tual perplexity is major factor in determining the effectiveness of RL. We also observe that reward design plays a crucial role further approaching the upper limit of model perception. To leverage these findings, we propose Perception-RI, scalable RL framework using GRPO during MILMM post-training. With a standard Wwen2-VL-22B-Insee
Perception-R achieves +4.2% RefCOCO+, +1.99 on PixMo-Count, +4.2% on PageOCR, and notably, 31.9% AP on COC02017 val for the first time, establishing a strong baseline for perception policy learning. Project code is available at wtttttt//// /:t::/giiit github........ wwwtuu...oomm//// wwt. http///innnnnnnggPPi.
Introduction
"We do see the world it is, but
conditioned it."
Stephen R. Covey
The landscape of large language model (LLM) has undergone a paradigm shift from non-reasoning foundation model, e.g., GPT-4/4o 44, 19,. DeepSeek- V3 [33] to strongly reasoning model, e.g.,
OpenAl o1/o3 [45], DeepSeek-RI [2). and Kimi-1.5 [77 DeepSeek-RI. I, particular, introduced simple yet effective rule-based reinforcement learning (RL) approach (55), enabling emergent reasoning patterns without relying on traditional scaffolding techniques such as Monte Carlo Tree Search (MCTS) 67] Process Reward Models (PRM) (3I]. This has catalyzed new revolution in LLM post-training techniques, prompting researchers to develop more powerful reasoning language
models 42 24
Despite these advancements, current explorations predominantly focus on the purely linguistic domain, and the unimodal nature of these reasoning models limits their ability to engage with the world in a truly perceptive way. To bridge this gap, this work takes a pioneering step in exploring
CCoresponning author, Core contribution
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

the potential of perception policy learning within multimodal LLMs 66 3 from lens of RL. While transferring RL techniques with reasoning processes, i.e.. chain thouuht [66]. from the language domain shows promise on certain visual tasks, our empirical studies reveal that this approach is not universally effective. This inevitably prompts us to reexamine the role that RI. play visual perception tasks, and how the utilization of RL. can lead to better and scalable perception policy.
Current understanding of reinforcement learning as post-training technique is primarily rooted in linguistic tasks (24) and language-centric multimodal tasks [T]. This work. however, posits that perception is critical prerequisite for visual reasoning. We argue that only by fully unlocking the perceptual patterns of Multimodal LLMs (MLLMs) can these models achieve complex visual reasoning. Visual perception tasks, fundamentally distinct from natural language tasks, necessitate a revised understanding of RL in this context due to two unique properties:
Visual perception is grounded in the objective physical world. It possesses definite physical truth values (e.g., points, lines, bounding boxes) but lacks the semantic depth of language.
Visual perception tasks, such as visual grounding and counting, are often "single-step" direct predictions. This limits the structured reasoning search space typically explored by RL.
These characteristics indicate that applying RL to visual perception will yield different properties, motivating exploration of perception- RL cognition. This work investigates the RL post- training of MLLMs in visual perception, complementing and extending current understanding. Through extensive experimental analysis, we have identified several bitter yet valuable lessons.
Explicit thinking process (CoT) during RI. not necessary for current perception policy. ($ ($5.2) We observe that the model without thinking process performs better than the one with thinking process.
Reward design plays pivotal role in perception policy learning. ($ 5.3) An appropriate reward function will lead a healthier learning curve and explore stronger perceptual patterns of MLLM.
Perceptual perplexity determines RI. superiority over SFT. ($ 5.2 We observe that RL can bring more significant improvement compared to SFT on more complex visual tasks, e.g., object detection.
Driven by these findings, we present a simple, effective, and scalable RL framework, i.e., Perception- RI, for efficient perception policy learning. Inspired by mainstream language reasoning models [12]
57.. Perception applies rule-based RL algorithm GRPO (55) during MLLM post-training stage. With vanilla Qwen2-VL-22-Immnn VL-2B- [6], Perception-R achieves significant improvement on multiple visual perception benchmarks, e.g., +4. 2% on RefCOCO+ 40), - 17.9% on PixMo-
Count [3], and +-4.2% F1-score on PageOCR (4 More importantly, Perception-R serves as the first time to enable pure MI.LM to reach 31. .%% mAp on the object detection benchmark
COC02017 32 val, showcasing the great potential of general foundation models to surpass expert models in mainstream visual tasks. We hope our method, results, and analysis will inspire future research on perception policy learning with RL.
2 Related Works
Multimodal Foundation and Reasoning Models. Recently, vision-language models 37
3 have demonstrated remarkable capabilities in visual comprehension [64] 68 and generation 14 48
70
through large-scale pretraining (2 61 and visual instruction tuning 37 These models integrate
visual modalities into unified semantic space via visual encoders
49 and adapters [1 37. while
leveraging auto-regressive large language models [59] as decoders for output generation. Despite the advancements in multimodal foundation models, their visual reasoning capabilities remain in an
carly developmental stage. Recent approaches S 39 41 have explored reinforcement learning (RL) post-training to enhance visual reasoning. However, they primarily focus on language-centric tasks such as ambiguous reference resolution (39] and geometrie problem-solving [4I, while overlooking critical aspects of perception-driven reasoning. In this work, we take pioneering step in utilizing RL for perception policy learning, aiming to bridge this gap and advance multimodal reasoning.
Visual Perception in Multimodal Models. Visual perception, a core concept in computer vision 21 52 20 69 29], involves interpreting and understanding sensory (visual) information from the real world. In the context of MI..Ms, visual perception is crucial for integrating, comprehending and reasoning about visual data from images or videos. Current MILMS typically bolster their visual perception by employing advanced visual architectures [63] 64), optimized visual- language modeling

---

and sophisticated post-training techniques 174). This work explores the potential of reinforcement learning (RL) further enhance these visual perception capabilities.
strategies 77 68
RI.-based Post-training in LLMs and MLLMs. Reinforcement learning (RL.) has emerged as a pivotal paradigm for refining L.MMs through alignment with human preferences and task-specific objectives. Prominent approaches like Reinforcement Learning from Human Feedback (RLHF) 446 and Direct Preference Optimization (DPO) [50] have demonstrated remarkable success in enhancing safety, coherence, and instruction-following capabilities of LLMs 43 44 and MLLMS [74
Recently, culebbsed RL techniques. represented by GRPO [5 have demonstrated the potential for large-scale RL upplications. LLMs have officially entered the era of strongly reasoning models. Subsequently, MLLMs [8] (39 41 have also quickly followed this technology. However, so far. there has been no exciting, true "Aha Moment" in the multimodal domain. This study aims to investigate the potential contributions of RL to multimodal models, focusing on visual perception.
3 Preliminaries
Perception Policy Definition. The goal of perception policy in visual-language context is enabling
the model first (i) extract and understand visual information from the environment 37 68], then (it) perform logical reasoning based on this understanding [73] 70) to (iii) accomplish specific tasks and further interact with the environment [5] 22). In this work. we aim to empower the model to deal with series of pure visual, e.g., counting, detection, and visual- language, e.g., grounding, optical character recognition (OCR), tasks through perception policy learning.
Group Relative Policy Optimization (GRPO (55]) is a rule-based reinforcement learning algorithm tailored for post-training LLMs. Its core idea is use group relative rewards optimize the policy, eliminating the need for a separate critic model [54]. Specifically, GRPO samples multiple outputs (0, ~ Og Figure T) from the old policy for the same input, calculates the average reward of these outputs as the baseline, and uses the relative rewards to guide policy updates. The optimization objective of GRPO can be formulated as following:
JGRPO(0) (0) E Eg P(). (O1g)l
1
loel
min
Tg f.t
Ar,, clip
1, Tg i,t
1 - e, +e Ar,
xL [To||
|edd
G
1=1
TBuu
Tee (oi, oi,
log Tiet (0i,t 41 0i,<t)
DKL.
[Tol
m
1,
(1)
To (0s, (7 0z,=()
To (o,,elg, 0s,<1)
where e and 3 are hyper-parameters, and A., is the advantage, computed using a group of rewards
{ri, (2,, "o,.",ra" corresponding to the outputs within each group. Refer to [22 55) for more details.
4 Perception-R1
In nutshell, our Perception-RI applies the rule-based RL algorithm GRPO [55] the post-training stage of MLLM and optimizes the reward modeling to support perception policy learning. Figure illustrates the idea, more approach and implementation details introduced next.
4.1 Rule-based Reward Modeling
The reward function serves as the principal training signal in reinforcement learning (RL), directing the optimization process. Existing LLM methods [12] 57 24 basically apply a highly resilient, rule-based reward system consisting of only two reward types: Format Reward and Answer Reward.
Format Reward. In existing LLM and MLLM, the output format is comprised of two essential components: the final output format and the intermediate reasoning process format. The reward for the final output is defined in accordance with specific task requirements and is typically encapsu- lated within <answer></ </nsswr>>> tags, whereas the reward for the intermediate reasoning process generally mandates that the reasoning steps be enclosed within <think>< /iite</ttinkae tags. Formally,
3

---

reward modeling
rollout
bipertite matching
mlim
image
03
reward
0g
There are 11 flamingos.
How many llamingos in the image?
ground truth
grpo
prompt
Illustration of Perception-RI framework. Following DeepSeck-RI [12]. we prompt MLLM model to generate several rollout responses, conduct reward modeling, and then apply GRPO [55 during post-training stage.
Figure 1:
1,
format is correct
Stormat
format is incorrect
In Perception-R1, we follow this setting. A subtle difference emerges that visual perception task frequently require the output of object coordinates, e.g., bounding box, lines, points. Consequently, the output format must be strictly constrained to the [x1,y1,x4 ,22,2]] structure.
Answer Reward. The Answer Reward pertains to the correctness of model-generated responses, serving as central consideration in reward design. Typically, outputs from language models are abstract and semantically rich, requiring validation through external mechanisms such as code-based ADE [12 or mathematical answer verification (55]. In contrast, visual perception tasks benefit from clearly defined physical ground truths, which simplify the development of robust reward function.
Perception-RI diverges from LLM approaches by anchoring the reward mechanism in visual discrimi- nation. This departure is pivotal, as it replaces the often implicit and subjective feedback mechanisms typical of language models with an explicit, quantifiable metric. Formally, discriminative reward r; can be represented as:
T: = (0;, 2),
where (--) indicates the discriminative function, for example, IoU for bounding box and euclidean distance for point. By leveraging visual discrimination, we provide the model with clear and objective feedback signal, ensuring the model's policy update with precise measured margin.
4.2 Multi-Subject Reward Matching
In natural environments, physical objects rarely appear isolation and instead frequently co-oceur in groups. This inherent complexity gives rise to a challenge we define reward matching, which entails aligning the model's output with the corresponding ground truth before reward computation. Specifically, when prompting the model to predict the attributes of multiple subjects within an image, e.g., points and bounding box, necessary to determine the appropriate ground truth reference for each subject to ensure accurate reward assignment.
Formally, let y = N denote the set of predicted attributes for N subjects, and let 1M represent the corresponding ground truth attributes. We model the reward matching problem as bipartite graph matching task, where one set of nodes corresponds to predictions and the other to ground truths. The edge weight between prediction and a ground truth lj is determined by the reward function & (yy, 2,) defined in Eq. 3 which measures their similarity or compatibility. The objective is to find the optimal assignment that maximizes the total reward:

---

N
δ = max
+ (o))))
(4)
where Sy is the set of all valid assignments between predictions and ground truths. To solve this optimization problem efficiently, we employ the Hungarian algorithm [27]. a well-established method for bipartite graph matching that guarantees the optimal pairing by maximizing the overall reward (or equivalently, minimizing the cost). This ensures that each predicted attribute is accurately matched with its corresponding ground truth, thereby optimizing the reward computation process.
After the optimal reward assignment determined, we calculate the answer reward by aggregating the individual rewards for each subject. Mathematically, the overall reward score is defined as:
N
1
answer
+ (1ti
(5)
Swatl Stormat
+ Sanswer
where is the optimal assignment obtained via the Hungarian algorithm.Ih In Perception-RE we primarily use reward matching for visual counting and object detection tasks, as these involve multiple objects.
4.3 Perception-R1 R1 Configuration
Model Setting. Our model implementation follows Qwen2-VL VL [[]. We mainly use the Qwen2- VL- Instruct-2B the baseline model. We also utilize -----2.---L----Innsttete [3] for training object detection tasks, due to its specialized optimization for localizing bounding boxes. The input image
resolution for Qwen2-VL. is dynamic cooperated with 2D-RoPE (56).
Task and Data Setting. Given that Perception-RR RI is primarily oriented towards pure visual and visual- language tasks, select several mainstream and representative downstream tasks for perception policy learning, specifically including visual grounding, e,g. refCOcO (7T) / / / 40 OCR, iee. PageOCR [34,, visual counting, i.e., Pixmo-Count [13], and object detection, i.e., COCO2017 [32 For each task, a subset (5k ~ 10k) of samples are respectively extracted as base data for individual post-training. More details are in the appendix A.I
Training Setting. We focus on the RL-based post-training stage of MILM. All the selected base models have already undergone pre-training and SFT stage. During RL stage, the initial learning rate set as le with rollouts by default and a batch size of The following are some important during post-training. Prompts detailed settings are in the appendix A.
Gradient Acemulation
Rollout G
KL Coefficient
Max Response Len
Temperature
2
8
0.04
2048
1.0
Reward Setting. We tailor distinct discriminative rewards for various visual perception tasks. For grounding task, the reward based on the Intersection over Union (loU) between the predicted output and the ground truth. In the counting task, we adopt paradigm similar to Qwen2.5-VI. which first detects points and then counts them. Here, the reward is derived from the Euclidean distance computed during reward matching. For OCR, the edit distance serves the primary reward metric. Lastly, in object detection, we combine multiple rewards: an object number reward based on the Fl score, location reward using IoU, and binary classification reward with missing penalty.
Sampling Setting. Following Kimi-1.5 [57] we adopt curriculum sampling strategy that begins with easier data and gradually transitions to more challenging examples. Specifically, for the object detection task, we first conduct offline training on COCO dataset to compute reward values. Based on the selected rewards, i.e., number reward, we partition the dataset accordingly. As training advances, we progressively replace the data with more difficult samples (i.e., those associated with lower rewards) while concurrently increasing the rollout to broaden the model's exploration space.
5

---

Table 1: Visual grounding benchmark evaluation. To comprehensively assess the model's ground- ing capability, we select referring experssion comprehension (REC) benchmark, i.e., RefCOCO 771, RefCOCO+ [11,, and RefCOCOg 40 for evaluation. The expert model is denoted in gray. RefCOCO
method
size ge g50 valgrs lestAgrr 75, vallas testA goo testB and valuz testA testB
MDETR
25
87.5
90.4
82.6
OFA
62
88.4
90.6
83.3
LLaVA-1.5
35
49.1
54.9
43.3
10.7
13.6
6.9
0.4
0.3
0.3
20.1
22.9
16.8
LLaVA-NcXT
36
7B
82.5
88.4
74.0
45.7
54.8
35.6
1.9
2.6
0.7
43,4
48.6
36.8
LLaVA-OV
28
7B
73.0
82.3
63.5
24.2
29.6
15.9
0.5
0.5
0.5
32.6
37.5
26.6
Qwen2- VL 61
2B
86.8
89.6
82.0
77.2
80.6
70.1
33.0
35.7
26.9
65.7
68.6
59.7
Perception-RI
2B
89.1
91.4
84.5
79.5
83.6
72.4
35.0
38.5
28.8
67.9
71.2
61.9
RefCOCO+
method
salesa gestA valerr test Agaro test val test aestA testB
valugg testAAuy testB
MDETR
81.1
85.5
72.9
OFA 62
81.3
87.1
74.2
LLaVA-1.5
35
42.4
49.7
36.4
9.8
12.4
6.4
0.5
0.5
0.2
17.6
20.8
14.3
LLaVA-NeXT
36
7B
74.5
84.0
64.7
41.5
51.8
30.0
1.9
2.7
1.0
39.3
46.2
31.9
LLaVA-OV
28
7B
65.8
79.0
57.2
23.6
28.8
15.3
0.6
0.6
0.4
30.0
36.1
24.3
Qwen2- VL [61
28
77.1
82.5
70.1
68.7
73.8
60.0
29.4
32.3
23.0
58.4
62.9
51.0
Perception-R1
2B
81.7
86.8
74.3
73.6
79.3
64.2
32.6
36.9
26.7
62.6
67.7
55.1
RefCOCOg
method
valeso testaso
valara testars
valeos testags
valaug
testos
MDETR
25
83.3
83.3
OFA 62
82.2
82.3
LLaVA-1.5
35
43.2
45.1
8.5
9.3
0.3
0.3
17.3
18.2
LaVA-NeeTT
36 7R
77.5
77.1
40.7
39.9
1.8
1.7
40.0
39.6
LLaVA-OV
28
70.8
23.3
23.6
0.6
31.6
31.7
Qwen2 VL 61
2B
83.3
83.1
72.7
73.0
28.9
27.9
61.6
61.3
Perception-R R1
85.7
85.4
75.7
76.0
32.1
33.1
64.5
64.8
Table 2: PageOCR evaluation, compared with various strong expert and general models. "en" means English and "zh" means Chinese.
Edit Distance
F1-score 1
Precision
Recall
BLEU
METEOR
size
zh
zh
zh
en
zh
en
zh
zh
Nougat
250M
25.5
74.5
72.0
80.9
66.5
76.1
DocOwl1.5
25
7B
25.8
86.2
83.5
96.2
78.8
85.8
GOT 65
3.5
3.8
97.2
98.0
97.1
98.2
97.3
97.8
94.7
87.8
95.8
93.9
Qwen2 VL
61
2B
8.0
10.0
94.4
93.0
96.9
96.1
93.0
90.5
90.9
78.0
94.1
87.2
LLaVA-NeXT
36
7B
43.0
64.7
57.3
88.1
47.8
58.2
Perception-RI
2.8
8.4
98.2
96.9
98.6
97.2
97.8
96.7
96.6
74.7
98.1
93.8
Experiments
The experimental section evaluates Perception- RI's performance on visual perception tasks (8 ($5.1) followed by analytical experiments exploring reinforcement learning (RL)'s role in perception policy learning (S 5.2) Finally, discusses the interplay between visual perception and RL, along with key insights for perception policy learning ($ 5.3).
5.1 Performance Landscape in Perception Tasks
We evaluate Perception-R1 on mainstream perception tasks: visual grounding, counting, OCR, and object detection. Experiments use the datasets described in 4.3 and benchmarks for image understanding. Results are in Tables
Visual Grounding is task that involves localizing visual objects based on linguistic descriptions. Specifically, given language prompt, the model is required to output the spatial coordinates of the subject (typically single entity) described in the prompt. As shown in Table [ evaluate

---

Table 3: Mainstream visual tasks evaluation including (a) visual object counting and (b) challenging
general object detection. Notably, the results of expert model in (b) are copied from MMDetection 7
f means Perception-R for object detection is build based on -.Ven2.5-VI-3IB-Jmm
3
Viusal Counting
Object Detection
method
size
Pixmova1
Pixmorest
method
size epoch
AP
APso
APrs
LLLVA-1.5
335
7B
33.3
31.0
YOLOV3
51
273
27.9
49.2
28.3
LLaVA-1.6
[58]
7B
32.7
31.9
Faster-RCNN
52
12
35.6
55.7
37.9
LLaVA-OV
28
7B
55.8
53.7
DETR 0
41M
500
42.0
62.4
44.2
Qwen2-VL VL
[61
60.2
50.5
Qwen2.5-VV VL
3B
1
16.1
23.7
16.7
Perception-He R1
78.1
75.6
Perception-
3B
1
31.9
46.7
33,4
(a) Visual counting evaluation on Pixmo-Cone Count val set and test set.
13
(b) Object detection evaluation on COC02017
32
validation set.
Table 4: General image understanding and reasoning evaluation, compared with various baselines. We select 8 mainstream multimodal benchamrks, i.e., MMBench [38], MMVet [72), MMStar (9) Sci- enceQA [53] SeedBench [18. MME [6).. LLaVA-Bench (37), and ai2D 226 for the comprehensive
understanding. We use the model after RL training in the counting tasks for the eval.
MMBench MM MMStar ScienceQA SeedBlench
MME
LLaVA.Bench Bench
LLAVA
AI2D
1lm
Avg
Avg
Avg
Avg
Avg
Cognition
Perception
Avg
Avg
LLaVA1.5 (35
Vicuna 1,57BB
62.8
32.8
32.6
65,4
60.1
302,1
1338.3
52.6
51.9
LLaVA-NeXT 36
Vicumal.5.7BB
66.0
37.9
37.7
68.2
69.1
195. 7
1419.5
52.7
67.4
Qwen2-V VL [61
Qwen2-2B
71.9
45.6
46.3
74.0
72.7
418.5
1471.1
46.5
71.6
Perception-Be R1
Qwen2-2B
71.8
48.9
45.7
73.4
73.0
430.0
1473.9
58.2
71.8
Perception-BE R1 on three mainstream benchmarks, refCOCO/+/ and report Acc@0.5, Acc@0.75. and Ace@0 0.95 to comprehensively assess its visual grounding capability. We surprisingly find that several SoTA MLLMs exhibit poor performance on the more challenging Acc @ 0.95 metric, with scores even below 1%. contrast, Perception-RI achieves a stable performance of over 30% on this metric. This observation suggests that the community should prioritize reporting more discriminative results in future evaluations. The experimental results demonstrate that Perception-R. exhibits strong competitiveness compared to both specialized and general-purposs models.
Optical Character Recognition (OCR) represents critical task in visual perception due to its substantial practical value. Current methodologies predominantly adopt either expert models or fine-tuned generalist models for OCR. Perception-RR pioneers the utilization of RL. to further unlock the OCR capabilities of MLLM. As shown in Table 2 our proposed Perception- R1 achieves SoTA performance on the highly challenging OCR benchmark, i.e., PageOCR (34), demonstrating significant superiority over existing expert models, e.g., GOT (98.27 2% vs. 97.2% F1-score) and robust generalist models, e.g., LLLVA-NeXT (98.2 2% vs. 64.7% Fl-score). Notably, Perception-RR R1 does not use the Chinese OCR data for training so it is zero-shot performance for Chinese metric. This breakthrough substantiates the formidable potential of RL applications in OCR tasks, establishing new frontiers for enhancing textual understanding and recognition in complex visual environments.
Visual Counting, as fundamental vision task, necessitates models to accurately quantify category- specific instances within images, requiring robust visual logic to identify and enumerate targets through structured recognition patterns. In Perception-RI, we adopt a detect-then-coot count paradigm that reformulates the counting problem into a point detection process. As shown in Table [3 Perception-RI achieves remarkable counting performance, surpassing the current strong baselines by substantial margin (17.9% improvement compared to Qwen2-VI. in Pixmo val set). This advancement substantiates that RL effectively stimulates models to explore intrinsic visual logic mechanisms (Although counting yields deterministic results, the sequence of counting can exhibit distinct patterns), thereby enhancing their capacity to resolve complex vision tasks.
General Object Detection, widely regarded as the crown jewel of computer vision tasks, has long been considered one of the most challenging problems in visual perception. As pioneering endeavor to integrate RL into object detection, Perception-BE R1 achieves a groundbreaking milestone, serving as the first pure MILM to surpass the 30+ AP threshold, i.e., 31.9 AP in Table 3hb on the COCO 2017 val set, matching or even exceeding the performance of specialized expert models. This achievement underscores rule-based RI's immense potential addressing complex vision tasks requiring sophisticated visual-logic integration.
7

---

Table 5: Allation Study of Perception-RI. We perform ablation studies investigate key properties of Perception-RI across a range of visual perception tasks. Specifically, we report the Acc@0.5 for RefCOCO / +/ / val set, the Fi-score for PageOCR, the average scores for Tixmo-Coumt and the AP metric for COC02017 val set. w/o means without. Notably, there is no reward matching applied visual grounding and OCR tasks, these tasks do not involve the multi-subject reward.
Visual Grounding
OCR
Visual Counting
Detection
case
RefCOCO
RefCOCO+
RefCOCOg
PageOCR
Pixmo,ai
Pixmouess
COCO2017
Perception-R
89.1
81.7
85.7
98.2
78.1
75.6
31.9
w/o reward matching
77.1
75.4
23.5
w/o RL
86.8
77.1
83.3
98.2
60.2
50.5
16.1
w thinking
75.1
67.9
71.3
93.8
74.9
72.8
25.7
w/o thinking
89.1
81.7
85.7
98.2
78.1
75.6
28.1 1
RL only
89.1
81.7
85.7
98.2
78.1
75.6
31.9
SFT only
88.2
80.7
84.6
97.2
58.0
59.9
25.9
SFT+RL.
88.4
80.7
85.1
98.3
77.1
75.4
30.8
Table 6: Reward design analysis of Perception-RI. cls reward indicates binary classification reward
and missing reward is penalty to penalize missed detections. To facilitate rapid experimentation, we randomly sampled 10k data from COC02017 train set for this experiment.
COC02017
reward function
AP APso
APrs
format reward
format reward + location reward (loU)
18.8
25.3
20.1
format reward
location reward (IoL) + cls reward
20.2
27.3
21.4
format reward location reward (IoU) + cls reward + recall reward (F1)
27.6
42.0
28.7
format reward location reward (IoU) + cls reward + recall reward (F1) missing reward 28.1
42.0
29.6
General Visual Comprehension extends heyond pure perceptual tasks, and we evaluate Perception- R1 multiple multimodal benchmarks. As shown in Table we observe intriguing phenomenon that models trained with RL for vision-specific tasks, e.g., counting task, exhibit concurrent perfor- mance gains in generic comprehension benchmarks. We attribute this cross-task enhancement the perception policy learning, which drives the model to discover superior image interpretation patterns.
5.2 Ablation Study of Perception-RI
In this section, we aim to conduct a comprehensive ablation study to systematically investigate the contributions of critical components within Perception-Re Experimental results are shown in Table From the experimental results, we can derive three principal empirical findings:
Reward matching enhances the explorability of multi-subject visual perception. As evidenced by the comparative results between row and in Table [5] replacing the bipartitle matching with sequential matching leads to substantial performance degradation in both visual counting and object detection task. This suggests that sequential matching constrains the RL exploration space. On the contrast, the bipartite matching mechanism provides more possibility in reward assignment, enabling the model to explore optimal visual perception patterns.
Explicit thinking processes prove non-essential for contemporary visual perception. Comparative analysis of row 3 and 4 reveals consistent performance degradation across all four evaluated perception explicit thinking process during both training and inference phases.
tasks when incorporating Similar phenomenon also emerges image classification tasks [30. We posit that this phenomenon arises because current visual perception lasks are more oriented toward visual logic rather than semantic logic. This shift implics that explicit language-centric reasoning processes are unnecessary, models tend to focus more learning implicit visual patterns.
Perceptual perplexity dictates RL's advantage over SFT. A comparison of post-training methods (SFT, RL, and SFT+RL) across four perception tasks (Table [5, rows 6-8) reveals that RL offers superior performance enhancement in tasks with high perceptual perplexity, such as counting and multi-objecttcategory detection. Conversely, for lowpperplyxity tasks like grounding and OCR, RL performs comparably to or even worse than SFT. This suggests that high perceptual perplexity is a significant factor influencing RL's effectiveness, indicating that RL techniques are most beneficial for tasks with greater perceptual complexity and a larger exploration space for the perception policy. Further analysis of perceptual perplexity is provided in the appendix A.2
8

---

1.72
86.0
1.70
Aog. refeoculiiy
74
Aug. pixno valfosst
168-
85.3
Bownd
166-
Sore
Roward
75
162-
L64
2x
85.0
2x
72
160-
1.58
16x
84.5
8x
71
1.6-
N2x
32x
160
240
sdo
84.0
32x
80
160
240
320
70
Triinig Staps
Rellouts
Training Steps
Rollouts
(a) Grounding reward
(b) Grounding performance (c) Counting reward
(d) Counting performance
Figure 2: Scalability analysis of Perception-RI. We select two primary tasks: grounding and counting. We visualize the training reward curves under varying numbers of rollouts and evaluate the final performance of each task. All experiments are conducted with 5k sampled data. And the default rollout number setting (1x) is 8.
5.3 More In-depth Analysis
In this section, we explore several key properties of Perception-Re R1 further enhance our understand- ing of Perception Policy Learning with RL.
Analysis of reward design for perception policy learning. We introduced the details of reward function of Perception-RI in $4.3 In this part, we examine the influence of these reward functions on perception policy learning. Specifically, using object detection as a case study, we incrementally integrate the designed answer reward into the format reward, as illustrated in Table 6 The results indicate that the progressive introduction of refined reward functions leads to consistent improvements in detection performance, ultimately exceeding the performance of expert models. This underscores the critical role of reward design in perception policy learning. Furthermore, it identifies a promising avenue for future research: the development of more refined and task-specific reward functions to enhance perception policy learning.
Analysis of scaling up rollout for perception policy learning. The scalability RL is a key concern of existing LLM post-training. In this part, we analyze the scalability of Perception-R1, focusing specifically on scaling up the number of rollouts. As shown in Figure 2 we conduct rollout-scaling experiments in two tasks: visual grounding and visual counting. The results indicate that increasing rollout count enhances reward optimization and final performance. This demonstrates Perception- R11's 'ss strong scaling properties and underscores the critical role of rollout quantity in scaling perception policies. By generating sufficient rollouts, the model broadens its exploration space, increasing the diversity of candidate solutions for reward evaluation. This expansion accelerates convergence to optimal visual perception patterns.
Limitation and Conclusion
"What can RL bring to MLLM?" is public question since the propose of DeepSeek-R1. Several latest works attempt to apply RL from the perspective of language-cent centric visual reasoning [3 15 41. However, in this paper, we take different pathway and argue that perception is crucial prerequisite for visual reasoning. Only by fully unlocking the perception patterns of MILMs can the models possess the ability to reason about complex visual tasks. Nevertheless, we regrettably find that many current perception tasks are overly simplistic, which limits the exploration space for RL. This, in turn restricts the possibility of MLLMs achieving a perceptual "Aha moment" through thinking process. Finding more appropriate perception tasks, aka. meta task, may be the key to addressing this issue.
In summary, this work takes pioneering step in exploring the potential of rule- based RL in MLLM post-training for perception policy learning. Through extensive experimental analysis, we establish several valuable cognition about perception policy learning with RL. Driven by these findings, we build Perception- RI, a simple, effective, and scalable RL framework for efficient perception policy learning. Perception-RI I sets new SoTAs across multiple visual perception tasks, particularly in object detection tasks. By introducing a novel paradigm, it achieves and even surpasses the performance of expert models, showing the great potential of perception policy learning with RL.

---

Acknowledgements
This research supported in part by the National Key Research and Development Program of China (Grant No. 20232D0121300), and National Natural Science Foundation of China (Grant No. 62495092).
This research is also supported by the National Natural Science Foundation of China under Grant 62576144 and Grant 62176096.
References
[1] Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint wwXiv:2309.66660 2023.
[2] Jinze Bai, Shuai Bai, Shusheng Yang. Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308. 12966, 2023.
[3] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang. Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprini arXiv:2502.1332. 13923, 2025.
[4] Lukas Blecher, Guillem Cucurull, Thomas Scialom, and Robert Stojnic. Nougat: Neural optical understanding for academic documents. arXiv preprini arXiv:2308. 13418, 2023.
[5] Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choro- manski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, al. Rt-2: Vision-language- action models transfer web knowledge to robotic control. arXiv preprint arXiv:2307.15816 2023.
[6] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European conference on computer vision, pages 213-229. Springer, 2020.
[7] Kai Chen, Jiaqi Wang, Jiangmiao Pang, Yuhang Cao, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jiarui Xu, Zheng Zhang, Dazhi Cheng, Chenchen Zhu, Tianheng Cheng, Qijie Zhao, Buyu Li, Xin Lu, Rui Zhu, Yue Wu, Jifeng Dai, Jingdong Wang, Jianping Shi, Wanli Ouyang, Chen Change Loy, and Dahua Lin. MMDetection: Open mmlab detection toolbox and benchmark. arXiv preprint arXiv:190.007155. 2019.
[8] Liang Chen, Lei Li, Haozhe Zhao, Yifan Song, and Vinci. R1-v: Reinforcing super generaliza- tion ability in vision- Langugee models with less than $3. R1-V 2025.
cow www.Deeeeeeeeeeeeeeeeeeeeeeeeeee Agent/
[9] Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, al. Are we the right way for evaluating large vision-language models? arXiv preprint arXiv:2403. 20330, 2024.
[10] Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma. Sft memorizes, rl generalizes: comparative study of foundation model post-training. arXiv preprint 2rXiv:2501.17106 2025.
[11] Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong. Junqi Zhao, Weisheng Wang, Boyang Li, Pascale N Fung, and Steven Hoi. Instructblip: Towards general-purpose vision-language models with instruction tuning. Advances in Neural Information Processing Systems, 36, 2024.
[12] DeepSeek-Ar AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli
10

---

Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Honghui Ding. Huajian Xin, Huazuo Gao, Hui Qu, Hui Li. Jianzhong Guo, Jiashi Li. Jiawei Wang. Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, .iin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang. Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng Li, Miaojun Wang. Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang. R. Chen, R. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang. Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Shengfeng Ye, Tao Yun, Tian Pei, Tianyu Sun, T. Wang. Wangding Zeng, Wanjia Zhao, Wen Liu, Wenfeng Liang, Wenjun Gao, Wengin Yu, Wentao Zhang. W. L.. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xiaosha Chen, Xiaowen Sun, Xiaoxiang Wang. Xinnan Song, Xinyi Zhou, Xianzu Wang. Xinxia Shan, Y. K. Li, Y. Q. Wang. Y. X. Wei, Yang Zhang. Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo. Yuan Ou, Yuduan Wang. Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You. Yuxuan Liu, Yuyang Zhou. Y. X. Zhu, Yanhong Xu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma. Ying Tang. Yukun Zha, Yuting Yan, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song. Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. Deepseek-rl: Incentivizing reasoning capability in lims via reinforcement learning. 2025.
[13] Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Mohammadreza Salehi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, et al. Molmo and pixmo: Open weights and open data for state-of-the-ard multimodal models. arXiv preprins arXiv:2409.1 7714/ 2024.
[14] Rumpei Dong, Chunrui Han, Yuang Peng, Zekun Qi, Zheng Ge, Jinrong Yang, Liang Zhao, Jianjian Sun, Hongyu Zhou, Haoran Wei, Xiangwen Kong, Xiangyu Zhang, Kaisheng Ma, and Li Yi. DreamLL.M: Synergistic multimodal comprehension and creation. In The Twelfth International Conference on Learning Representations, 2024.
[15] Kaituo Feng, Kaixiong Gong, Bohao Li, Zonghao Guo, Yibing Wang. Tianshuo Peng, Benyou Wang, and Xiangyu Yue. Video-r1: Reinforcing video reasoning in milms. arXiv preprins 25X 2025.
[16] Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin. Mengdan Zhang, Xu Lin, Zhenyu Qiu, Wei Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, and Rongrong Ji. Mme: A comprehensive evaluation benchmark for multimodal large language models. arXiv preprint Ttt::2300.13344 2023.
[17] Zitian Gao, Boye Niu, Xuzheng He, Haotian Xu, Hongzhang Liu, Aiwei Liu, Xuming Hu, and Lijie Wen. Interpretable contrastive monte carlo tree search reasoning, 2024.
[18] Yuying Ge, Sijie Zhao, Ziyun Zeng, Yixiao Ge, Chen Li, Xintao Wang, and Ying Shan. Making Ilama and draw with seed tokenizer. arXiv preprint arXiv:2310.012/8, 2023.
[19] GPT-4o. Hello got- 2024.
[20] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961-2969, 2017.
[21] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. pages 770-778, 2016.
[22] Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxiao Dong, Ming Ding, et al. Cogagent: A visual language model for gui agents.
11

---

In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pages 14281-14290, 2024.
[23] Anwen Hu, Haiyang Xu, Jiabo Ye, Ming Yan, Liang Zhang, Bo Zhang, Chen Li, Ji Zhang, Qin Jin, Fei Huang, et al. mplug-docowl 1.5: Unified structure learning for ocr-free document understanding. arXiv preprint arXiv:2403. /2895, 2024.
[24] Jingeheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, and Heung- Yeung Shum Xiangyu Zhang. Open-reasoner--7ro:: An open source approach to scaling reinforcement learning on the base model. https://gs thub. coom/ppenn Sheasoner-Zeroo
http://///aaaooooooo//aeeeeeeeer Open -eaesone-- Zero 2025.
[25] Aishwarya Kamath, Mannat Singh, Yann I LeCun, Gabriel Synnaeve, Ishan Misra, and Nicolas Carion. Mdetr-modulated detection for end-to-end multi-modal understanding, Proceedings of the IEEE/CVF international conference computer vision, pages 1780-177 1790, 2021.
[26] Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannanch Hajishirzi, and Ali Farhadi. A diagram is worth dozen images. In Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings. Part IV 14, pages 235 251. Springer, 2016.
[27] Harold W Kuhn. The hungarian method for the assignment problem. Naval research logistics quarterly, 2(1-2):83-97, 1955.
[28] Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.0m 03326, 2024.
[29] Jinyang Li, En Yu, Sijia Chen, and Wenbing Tao. Ovtr: End-to-end open-vocayylary multiple object tracking with transformer. arXiv preprint vrxiv:2503.00111 2025.
[30] Ming Li, Shitian Zhao, Jike Zhong, Yuxiang Lai, and Kaipeng Zhang. Cls-rl: Image classifica- tion with rule-based reinforcement learning. arXiv preprint arXiv:2503.16158 2025.
[31] Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step, 2023.
[32] Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Purt V 13, pages 740-755. Springer, 2014.
[33] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chenggi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19453 2024.
[34] Chenglong Liu. Haoran Wei, Jinyue Chen, Lingyu Kong, Zheng Ge, Zining Zhu, Liang Zhao, Jianjian Sun, Chunrui Han, and Xiangyu Zhang. Focus anywhere for fine ane-grained multi-page document understanding. arXiv preprint
140.: 2024.
[35] Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 86296-26306, 2024.
[36] Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024.
[37] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances neural information processing systems, 36, 2024.
[38] Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, al. Mmbench: your multi-modal model an all-around player? arXiv preprint arXiv:2307.0628/ 2023.
12

---

[39] Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan. Dahua Lin, and Jiaqi Wang. Visual-rft: Visual reinforcement tine-tuning. arXiv preprint arXiv:2503.0178. 2025.
[40] Junhua Mao, Jonathan Huang, Alexander Toshev, Oana Camburu, Alan L Yuille, and Kevin Murphy. Generation and comprehension of unambiguous object descriptions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 11 11-20, 2016.
[41] Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Botian Shi, Wenhai Wang, Junjun He, Kaipeng Zhang, et al. Mm-eureka: Exploring visual aha moment with rule- based large-scale scale reinforcement learning. arXiv preprint arXiv:2503.07365. 2025.
[42] Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling. arXiv preprint arXiv:2501. 2025.
[43] OpenAl. Chatgpt. https://openai: http:////ppppaaa.ccooobb//oaaapp 2022.
[44] OpenAl. Gpt-4 technical report. arXiv preprint ar1ii:333::0033.777744 2023.
[45] OpenAl. Learning to reason with llms, September 2024.
[46] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730-27744, 2022.
[47] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730-27744, 2022.
[48] Yuang Peng, Yuxin Cui, Haomiao Tang. Zekun Qi. Runpei Dong, Jing Bai, Chunrui Han, Zheng Ge, Xiangyu Zhang, and Shu-Tao Xia. Dreambench++: A human-aligned benchmark for personalized image generation. arXiv preprint arXiv:2406. 16855, 2024.
[49] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning. pages 1748-8766 PMLR, 2021.
[50] Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024.
[51] Joseph Redmon and Ali Farhadi. Yolov3: An incremental improvement. arXiv preprint arXiv:1804.022 027677 2018.
[52] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. IEEE transactions on pattern analysis and machine intelligence, 3966):117--114 2016.
[53] Tanik Saikh, Tirthankar Ghosal, Amish Mittal, Asif Ekbal, and Pushpak Bhattacharyya. Sci- enceqa: A novel resource for question answering on scholarly articles. International Journal on Digital Libraries, 23(3):289-300 2022.
[54] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv: 2017.
[55] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepscekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint 3rXiv:2402.03300 2024.
[56] Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding, Neurocomputing, 1680127063, 2024.
13

---

[57] Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changju Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi kl. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv: 0an::2511.129999 /2599, 2025.
[58] Shengbang Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Manoj Middepogu, Sai Charitha Akula, Jihan Yang, Shusheng Yang. Adithya Iyer, Xichen Pan, et al. Cambrian-Ik 1: A fully open, vision-centric exploration of multimodal llms. arXiv preprint arXiv:2406. 16860, 2024.
[59] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307 09288, 2023.
[60] Fei Wang, Wenxuan Zhou, James Y Huang, Nan Xu, Sheng Zhang, Hoifung Poon, and Muhao Chen. mdpo: Conditional preference optimization for multimodal large language models. arXiv preprint arXiv:2406. 1 1139, 2024.
[61] Peng Wang. Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, al. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409. 12191, 2024.
[62] Peng Wang, An Yang, Rui Men, Junyang Lin, Shuai Bai, Zhikang Li, Jianxin Ma, Chang Zhou, Jingren Zhou, and Hongxia Yang. Ofa: Unifying architectures, tasks, and modalities through learning framework. In International conference on machine
a simple requence-to-sequeee learning, pages 23318-23340. PMLR, 2022.
[63] Haoran Wei, Lingyu Kong, Jinyue Chen, Liang Zhao, Zheng Ge, Jinrong Yang. Jianjian Sun, Chunrui Han, and Xiangyu Zhang. Vary: Scaling up the vision vocabulary for large vision- language model. In European Conference on Computer Vision, pages 408-424. Springer, 2024.
[64] Haoran Wei, Lingyu Kong, Jinyue Chen, Liang Zhao, Zheng Ge, En Yu, Jianjian Sun, Chunrui Han, and Xiangyu Zhang. Small language model meets with reinforced vision vocabulary. arXiv preprint arXiv:2401.12555 2024.
[65] Haoran Wei, Chenglong Liu, Jinyue Chen, Jia Wang, Lingyu Kong, Yanming Xu, Zheng Ge, Liang Zhao, Jianjian Sun, Yuang Peng, et al. General ocr theory: Towards ocr-2.0 via a unified end-to-end model. arXiv preprint 24Xiv:2409.0/704 2024.
[66] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. of-touh prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24848 2022.
[67] Huajian Xin, Z. Z. Ren, Junxiao Song. Zhihong Shao, Wanjia Zhao, Haocheng Wang. Bo Liu, Liyue Zhang, Xuan Lu, Qiushi Du, Wenjun Gao, Qihao Zhu, Dejian Yang. Zhibin Gou, Z. F. Wu, Fuli Luo, and Chong Ruan. prepseek-prover-vie for reinforcement learning and monte-carlo tree search, 2024.
Harnessing proof assistant feedback
[68] En Yu. Kangheng Lin, Liang Zhao, Yana Wei, Zining Zhu, Haoran Wei, Jianjian Sun, Zheng Ge, Xiangyu Zhang. Jingyu Wang, et al. Unhackable temporal rewarding for scalable video milms. arXiv preprint 4rXiv:2502.12081, 2025.
[69] En Yu, Tiancai Wang, Zhuoling Li, Yuang Zhang, Xiangyu Zhang, and Wenbing Tao. Motrv3: Relcase-fetch supervision for end-to-end multi-object tracking. arXiv preprint 2305. 2023.
[70] En Yu, Liang Zhao, Yana Wei, Jinrong Yang, Dongming Wu, Lingyu Kong, Haoran Wei, Tiancai Wang, Zheng Ge, Xiangyu Zhang, et al. Merlin: Empowering multimodal lmss with foreiight minds. arXiv preprint arXiv:2312.00588,, 2023.
[71] Licheng Yu, Patrick Poirson, Shan Yang, Alexander C Berg, and Tamara L Berg. Modeling context in referring expressions. In Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11 11-14, 2016, Proceedings, Part II 14, pages 69-85. Springer, 2016.
14

---

[72] Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang. and Lijuan Wang. Mm-vet: Evaluating large multimodal models for integrated capabilities. arXiv preprini wrXiv:2308.024999 2023.
[73] Liang Zhao, En Yu, Zheng Ge, Jinrong Yang, Haoran Wei, Hongyu Zhou, Jianjian Sun, Yuang Peng, Rumpei Dong, Chunrui Han, et al. Chatspot: Bootstrapping multimodal Ilms via precise referring instruction tuning. arXiv preprint arXiv:2337.09444. 2023.
[74] Zining Zhu, Liang Zhao, Kangheng Lin, Jinze Yang. En Yu, Chenglong Liu, Haoran Wei, Jianjian Sun, Zheng Ge, and Xiangyu Zhang. Perpo: Perceptual preference optimization via discriminative rewarding. arXiv preprint arXiv:2502.0437,, 2025.
15

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction accurately reflect the paper's contributions and scope. We assert that this paper pioneers the exploration of rulc-based Reinforcement Learning (RL) Multimodal Large Language Model (MLLM) post-training for perception policy learning. Furthermore, we establish several key insights regarding perception policy learning with RL. Driven by these findings, we developed Perception-RI which achieves new state-of ffate-of-the-are (SoTA) results across multiple visual perception tasks.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The discussion of limitation is presented Section o
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex system might not be used reliably provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess..
16

---

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: We provide the full set of assumptions and complete (and correct) proof in Section 3 and Section including preliminaries and equations and detailed explanation.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proof' provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: All reproduce information including source code are provided in this paper.
Guidelines:
The answer NA means that the paper does include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
17

---

(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open- dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We provide the source code and detailed readme in supplemental material and also provide detailed data source in the appendices.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cc)
ioliciisssiioooiiyy for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https //nips http://i//ggiidss///eetttbiieeee
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We present the experimental setting in section 4.3
Guidelines:
The answer NA means that the paper does not include experiments.
18

---

The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification:
We do not report error bars in this work.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We report it on Section 4.3 and all experiments are conducted on NVIDIA A100 Tensor Core GPU.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
19

---

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: This work aligns with NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This work does not have potential negative societal impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (c.g., deploymentt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: There is no high risk for misuse of our models and datascts.
20

---

Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All code and data are following licenses properly. And all related works are cited.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website). the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, ppppppswiithheeecc thcode cwmccoo/aaaooooo aattaatseets has curated licenses for some datascts. Their licensing guide can help determine the license of dataset.
For existing datasets that are e-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is available online, the authors encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We introduce new code and models in this paper. Both the code and mode cards are well-documented, with detailed information regarding usage, training procedures, licenses, and limitations provided alongside the assets.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details of the dataset/code/moodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
21

---

The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing experiments and research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurlPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing experiments and research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration is not required.
Answer: [Yes]
Justification: This work build based on LLMS and we provide detailed introduction about them.
22

---

Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy /tteee//Deerripoccc///ofeeeeeees for what should or should not be described.
23

---

A Technical Appendices and Supplementary Material
In this appendix, we provide additional details about Perception-Rl, which are omitted due the 9- 9-page limit of the main paper. Specifically, Section A. elaborates on the detailed dataset and training settings. Section A.2 presents additional explanation about perceptual perplexity. Section A.3 presents more experimental results.
A.1 Additional Details about Experimental Setting
More detailed dataset information of Perception-RL. In Section 4.3 we introduced what data was used for RL post-training of Perception-R on which tasks. In this part, we will provide more detailed information about the datasets, as shown in Table
Table 7:
Training dataset statistics. Notably, do not mix the data from different perception tasks for joint training because the rewards for different tasks vary.
tasks
datasets
Original
Used
Ratio
visual grounding
RefCOCO/ RefCOCO+ RefCOCOg
320k
1.56%
OCR
PageOCR
50k
5k
10%
visual counting
PixMo-Count
1.9M
0.5%
object detection
COCO2017
110k
110k
100%
overall
2.38M
130k
More detailed training setting information of Perception-RU Section 4.3
elaborates on several
key parameters of Perception-R1. In this part, we further demonstrate the diverse prompts employed for distinct perception tasks, as shown Table g
Table Prompts of Perception-RI. The system prompt of Perception-RR follows Qwen2- VL 61 and Qwen2.5-VL
[3.
tasks
system prompt
user prompt
visual grounding
Qwen2-VL
Output the bounding box the [question] in the image.
OCR
Qwen2-VL
OCR this image.
visual counting
Qwen2-VL VL Qwen2.5-VL.
Output all the bounding boxes the (label)
object detection
Please output bhox coordinates and names of (90 categories of COCO).
A.2 Additional Explanation about Perceptual Perplexity
In this work, we point out that perceptual perplexity is major factor in determining the effectiveness of RL. And RL techniques are most beneficial for tasks with greater perceptual complexity and a larger exploration space for the perception policy. In this part, we aim to demonstrate the impact of perceptual perplexity across different tasks on the performance of RL through quantitative analysis. Specifically, utilize the theoretical possibility of matching outcomes in reward matching to quantify the perceptual perplexity of the task, that is, how many theoretical matching results exist between the model's 's predictions and the ground truth. We also count the probabilities of actual matches across different datasets, that is, the average number of permutations of the ground-truth output per image.
Table 9: Perceptual perplexity analysis of different perception tasks. is the number of ground truth. The statistical value of perplexity is calculated based on sampled data.
tasks
dataset
theoretical
statistical
visual grounding
refCOCO
OCR
PageOCR
visual counting
Pixmo-Coum A
229,202
object detection
COC02017
A
3,47c+83
As shown in Table visual grounding and OCR tasks, there only one fixed target for the ground truth, leading to just possible matching outcome. However, for visual counting and

---

Input:
Output:
Question:
Outptt bounding boxes http:///.....//...///././///....
Perception-RI:
(259 5912999 115 (40,,,44440099,,54427733..4
Num boxes
Figure 3: Demo of Percpetion- R1 on visual counting task.
Input:
Output:
Question:
Outatt the bounding box of the woman holding child in the image.
Perception-RE:
The woman holding chi1d(155,459))2288888
Figure 4: Demo case of Percpetion-R on visual grounding task.
object detection tasks, multiple target objects exist in an image. The various permutations of these targets create numerous matching possibilities, offering ample exploration space for RL to optimize perception capabilities.
A.3 Additional Experimental Results
More qualitative results. In this section, we provide more qualitative analysis of Perception-RR R1 on multiple visual perception tasks. The selected cases shown in Figure innlluding visual counting, visual grounding, object detection, and OCR task, respectively.
25

---

Input:
Output:
Question:
<image>Plesan outpul bbox courdinates and names of (90 caleguries of COCO).
Perception-R1:
('bbox 28": [435, 1,666, 415). "label": "person"",
("bhox [218, 323, 253],
"lahel": "person"
("bbox 2d" [147. 165, 225. 280). "label": "person").
("bbox 2d' [183, 239, 242). "label":
"porson"),
("bhox [276, 338, 192], "lahel":
"person"),
("bbox 2d": [127. 54, 471, 406), "label": "laplop"),
("bbox 2d!:: [108, 174, 156, 243], "label"; "cell phomp"
('boox 2d": [26, 107, 103, 214), "label!": "potted plant").
(bboxx 2d": [64; 280, 137], "label"; "chair")
Figure 5: Demo case of Percpetion-Re on general object detection task. The color of bounding boxes correspond to the category.
26

---

Input:
Riding Dirty
Question:
OCR this image.
Perception-RI:
GEAR L Bin'nRiding Dirty' muddy mix road- and mountain-bike racing, eyclocross doubles the thrill both sports. Here's the gear get you started. Inyybby Berne pooudylannCyycoorreee is as cool it looks: Racers pedal drop-hundlebar bikes with trail real-ready tires through a course that often includes pavement, dirt, mud, and grass, as well as obstacles force you to dismount and sling your bike over your shoulder. 'All you really need tough and have good attitude and desire to suffer little bit," Cannondale professional eyclocross team. The right gear helps, too.|n) BBest for Entry Level',n' (eveeoonn (CCanonddaee
Stu Thorn,, founder and director of the
CAADX 105)m) (nWhtt you sacrifice with extra weight you make up for with highly capable aluminum frame that'll keep you going between races. carbon-blade blade fork helps dependable 2x11 speed Shimano 105 drivetrain powers through any grade, and TRP cable disc brakes perform well even in mud. Meanwhile, the 35 tires and stable geometry make the CAADX superfun commuter if you're looking for bike can multitask (n) meannondalc.com com $1,970 Best mestt for Experienced Cyelists' un @Specialized CruX Elite Xiln you already log long miles on premium road bike, this should be carbon frame is as light it is. AnA low bottom bracket and relatively slack head angle, which make it stable, helped us through rocks and roots and corner quickly. The tires can run tubeless to better resist flats, and extra clearance
of the pounding the courses dish out. The
rig. The
they spin freely when eaked with mud. The CruX Elite playful and fast and something won't It outgrow you collect medals. yn uspecialized com $3.000 Glove n nhehe cross scason typically runs from September through February, so you'l need hearty gloves like these, waterproof base, warmth on wet race days. graftsports. ns 878 na Tough eyclocross requires you be foot times. So light,
MACCESSORIESS anl.ESnnnaCaaaa nCCaf Shield
with fleece lining
Crewin in Unliie other bike aren't likely to wear out, but Darn Tough will replace them if they do. daratough.com $ 118) Parrk Tool Brush Setta)) The mud, dirt, and grime that builds up during off-road rides can damage key components. This kit does $80 ninRapha
socks are key, These
and Leg Warmer'un in TTheeee merino layers, which have bit of Lyera for stretch, peel off easily when the weather warms up. And they dry quickly, whether you sweat profusely or get caught in a sudden squall. rapha.ce From 870' onnTopakk SmartGauge De'm (nFo peak performance, adjust your tire pressure suit the terrain. (On soft trails, lower pressure makes grip better.) The SmarrGauge makes snap with fast, readable result, topeak..co 840
than keep your bike looking fresh; it keeps it healthy,
parktool
Figure 6: Demo case of Perepetion-RI on OCR task.
27


Reviews:
[{"review_id": "wf8fG8Sq8D", "reviewer": "Reviewer_S8da", "summary": "Perception-R1 introduces a RLVR framework that fine-tunes multimodal LLMs for perception tasks including visual grounding, OCR, counting and object detection. The authors argue that when paired with carefully designed rewards (including their reward-matching scheme) and no chain-of-thought reasoning, RL can unlock stronger visual skills for multimodal LLMs. A key element is a reward-matching step that aligns each predicted object with the most appropriate ground-truth instance before scoring, ensuring the model is rewarded for true perceptual accuracy rather than output order. Using only 5k–10k samples per task, this approach shows superior performance compared to baselines.", "questions": "- Why—and under what inductive bias—can RL outperform SFT when no chain-of-thought reasoning is used? In other words, if explicit reasoning is absent, what advantage does a policy-gradient objective confer that standard maximum-likelihood training cannot. Does this also imply that training an encoder with RL instead of SL would also obtain superior simple image-classification tasks?\n- In Tables 1–4, are the MLLM scores zero-shot or supervised fine-tuned results? If any numbers are zero-shot, please also report supervised-only results so readers can fairly assess the benefit of RL tuning. Also please including the Qwen-2.5-VL backbone results for fair comparison. Perception-R1 uses Qwen2.5-VL selectively. \n- Table 5 shows the “no RL” baseline outperforming “SFT only.” What training differences separate these rows, and if \"no RL\" baselines indicates a zero-shot baseline (a baseline with no training) why would additional supervised training lower accuracy for certain tasks?\n- At line 256-257, the authors state that \"for low-perplexity tasks like grounding and OCR, RL performs comparably to or even worse than SFT\", but I don't see any values where RL outperforms SFT (row 6 vs row 7). What is the reason behind this?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n- The paper is one of the first to study RL post-training for core visual perception in MLLMs, filling a gap left by language-centric RL work.\n- The authors propose a simple yet effective GRPO based recipe which shows consistent gains across four perception benchmarks.\n- The study provide ablation studies which isolates three important lessons (no CoT, value of reward matching, role of perceptual perplexity) that can guide future RL practitioner later in this field.\n- The paper is overall easy to follow.\n\nWeaknesses\n- Beyond empirical wins on high-perplexity tasks, the paper offers little theoretical reason RL should outperform SFT once the thinking component is removed from the post-training stage. \n- The definition and justification of the perceptual-perplexity metric are opaque.\n- Several comparison and reporting issues flagged in the questions section need clarification for full transparency.", "was_revised": true, "final_justification": "The reviewers has resolved most of my concerns and hence, raise my score to Borderline Accept.", "rebuttal": "Thank you very much for recognizing our work and for providing such valuable comments and suggestions. We have responded to each of your comments and questions as follows:  \n* **Q1. Theoretical explanation about why RL outperforms SFT without CoT process.**\n\n\tThanks for your question. This is a profound and valuable topic. Indeed, some existing works [1][2][3] have already explored the relative merits of RL and SFT. In the context of visual perception, we think the following key reasons are crucial:\t \n\t 1. **RL conducts better negative supervision compared to SFT.** Visual representation learning and downstream visual task like *object detection* are highly dependent on the quantity and quality of *negative samples* [4][5]. In the context of post-training, the negative sampling spaces and strategies for RL and SFT differ significantly. With RL, any action that yields a low reward is treated as a \"negative sample.\" This allows RL model to effectively explore and learn from a wide range of trajectory samples on-line. While SFT methods only sample one trajectory (answer text) off-line and the negative supervision is very limited (only from a few tokens that are not strictly aligned with the answer).\n\t 2. **Policy gradient allows models to explore the perception pattern freely not just clone the expert behavior.** SFT is fundamentally a form of behavior cloning (BC), where the model learns to maximize the likelihood of an expert's demonstration (the ground-truth text). This approach can suffer from covariate shift: when the model makes a mistake, it enters a state distribution not seen during training, leading to compounding errors. In contrast, RL, particularly through policy gradient methods, optimizes the policy $\\pi$ to maximize the expected cumulative reward. This objective encourages the model to explore and discover novel strategies or implicit ”perception patterns\" that lead to high rewards, even if those patterns deviate from the expert's specific demonstration. It learns to solve the task robustly, rather than simply mimicking one particular solution path.\n\t 3. **RL allows for more flexible and task-aligned reward shaping.** The objective for SFT is fixed: minimizing the cross-entropy loss against a reference text. This is an indirect proxy for the actual task performance. For many visual tasks, performance is better measured by non-differentiable or complex metrics. RL's reward function $R$ can be designed to directly incorporate these metrics. For instance, in object detection, the reward can be the Intersection over Union (IoU) score, directly optimizing for localization accuracy. For image editing, the reward could be based on a combination of a CLIP score for semantic alignment and a score from an aesthetic quality model. This flexibility allows RL to align the model's learning process much more closely with the true end-goal of the visual task.\n\n\n\tAnd this conclusion still holds true for the simple image classification task [6].\n\n\n\t[1] Chen, Jierun, et al. \"The Synergy Dilemma of Long-CoT SFT and RL: Investigating Post-Training Techniques for Reasoning VLMs.”  \n\t[2] Zhu, Ke, et al. \"Continual sft matches multimodal rlhf with negative supervision.\"  \n\t[3] Wang, Bo, et al. \"Implicit Reward as the Bridge: A Unified View of SFT and DPO Connections.\"   \n\t[4] He, Kaiming, et al. \"Momentum contrast for unsupervised visual representation learning.\"  \n\t[5] Lin, Tsung-Yi, et al. \"Focal loss for dense object detection.\"    \n\t[6] Li, Ming, et al. \"Think or not think: A study of explicit thinking in rule-based visual reinforcement fine-tuning.\"\n\n* **Q2. The definition and justification of the perceptual-perplexity metric.**\n\n\tWe are sorry that we did not provide detailed formal definitions in the main text due to the page limit. We offer the following clarifications of perceptual perplexity below and will add them to the next revision.  \n\t**Definition:** Given a perceptual task dataset, the perceptual perplexity *P* of this task can be represented as:\n\t$$P = \\prod_{i=1}^{N} m_i!$$\n\twhere $m_i$ is the number of ground-truth objects belonging to category $i$, $i$ ∈ {1, 2, ..., *N*} *N* is the total number of possible object categories for the task. The sum of objects across all categories must equal the total number of objects: $\\sum_{i=1}^{N} m_i = M$.  \n\t**Justification:** For visual grounding and OCR task, $M = N = 1$. For visual counting task, $M > 1, N = 1$. For object detection, $M > 1, N > 1$. Consequently, the perceptual perplexity associated with these tasks increases from low to high.\n\n* **Q3. Experiment setting in Table 1-4 and add Qwen2.5-VL baseline result.**\n\n\tThe results for Perception-R1 presented in Tables 1-4 were obtained in a zero-shot setting. Concurrently, we report the SFT-only results in Table 5. Following your advice, we have added the results of Qwen2.5-VL as a baseline, as shown in the table below. These results will be included in the next version. \n\t\n\t| case | base model | Visual Grounding | | | OCR | Visual Counting | | Detection |\n  |------|------|------------------|------------------|------------------|---------|------------------|------------------|-----------|\n  | | |  RefCOCO | RefCOCO+ | RefCOCOg | PageOCR | Pixmo_val | Pixmo_test | COCO2017 |\n  | Qwen2-VL-2B | - | 86.8 | 77.1 | 83.3 | 94.4 | 60.2 | 50.5 | 0.1 |\n  | Qwen2.5-VL-3B | - | 86.7 | 79.3 | 84.3 | 94.9 | 60.8 | 57.4 | 16.1 |\n  | Perception -R1 | Qwen2-VL-2B | 89.1 | 81.7 | 85.7 | **98.2** | 78.1 | 75.6 | 28.0|\n  | Perception-R1 | Qwen2.5-VL-3B | **89.5** | **83.5** | **86.5** | 97.1 | **83.8**| **81.1**| **31.9** |\t\n\t\n\n* **Q4. Issue of the comparison between SFT-only and baseline setting in Table. 5.**\n\n\tWe sincerely apologize that there is an oversight in the OCR column of Table 5. In fact, the correct baseline metric for row 3 in the OCR column is 94.4, which is consistent with the Qwen2-VL baseline reported in Table 3. With this correction, it is clear that the performance in the SFT-only setting is generally higher than the baseline performance. We will rectify this in the revised manuscript.\n\n* **Q5. Confusion about the statement at line 256-257.**\n\n\tThank you for pointing out the ambiguity of this sentence. To clarify, our intention is to convey that for tasks with low perceptual perplexity, such as visual grounding and OCR, the RL-only setting does not yield a significant performance improvement over the SFT-only method. The gain is only about one point, which is why we describe the results as \"comparable”. In the next version, we will revise this statement to be more precise and avoid any ambiguity."}, {"review_id": "DNupOx8afu", "reviewer": "Reviewer_AEua", "summary": "This paper proposes Perception-R1, a rule-based reinforcement learning (RL) framework aimed at improving visual perception capabilities in multimodal large language models (MLLMs). Inspired by the success of DeepSeek-R1 in pure language domains, the authors apply the GRPO algorithm during the post-training phase of MLLMs to optimize perception-oriented tasks like visual grounding, counting, OCR, and object detection.\n\nKey findings include: 1) Explicit \"thinking\" processes (e.g., Chain-of-Thought) hurt perception tasks. 2) Reward design is crucial, especially in multi-object tasks. 3) RL benefits tasks with high perceptual perplexity (e.g., object detection) but less so in simpler tasks (e.g., OCR). 4) Perception-R1 achieves strong or state-of-the-art results across multiple benchmarks, especially hitting 31.9 mAP on COCO2017 — the first for a pure MLLM.", "questions": "*  Most benchmarks are academic; demonstrating robustness in noisy or real-world environments would enhance impact. Therefore, limited evaluation on real-world applications is a big concern for me. \n* The paper doesn’t state whether code, models, or data subsets will be released, which is key for community adoption. \n* **Why SFT + RL Performs Worse than RL?** SFT + RL is often the common practice of many previous research efforts. \n\nI appreciate the overall pipeline and the proposed method for incorporating reinforcement learning into training grounded multimodal LLMs. However, I have some concerns about the potentially overreaching title and naming choices (e.g., Perception-R1), which may give the impression of a more comprehensive coverage of perception tasks than what is actually explored in the paper. Also, I think it is necessary to perform experiments on more than one base models to demonstrate the generalizability of the proposed method.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### **Strengths**\n\n* Offers a new angle by targeting perception-first reinforcement learning, rather than language-first reasoning, for MLLMs.\n* Implements GRPO without complex scaffolding (e.g., MCTS), making the approach scalable and accessible.\n* State-of-the-art results: Sets new benchmarks on visual grounding (RefCOCO/+), counting (PixMo), OCR (PageOCR), and general object detection (COCO2017). \n* Detailed Ablations & analysis: Provides thorough studies on reward design, the (in)effectiveness of thinking processes, scaling rollouts, and task complexity.\n\n### **Weaknesses / Questions**\n\n* The name Perception-R1 suggests a general-purpose perception policy framework, but the main performance gains and experimental focus appear to be centered on object detection. Could the authors clarify whether the method generalizes across a broader range of perception tasks (e.g., segmentation, depth estimation, tracking), or is it primarily tailored for detection and counting?\n* While results show GRPO helps, there's little theoretical grounding as to *why* rule-based RL is well-suited for visual tasks beyond empirical trends.\n* Most experiments are built on Qwen2-VL; broader model diversity (e.g., LLaVA, MiniGPT) might improve generality of claims. Overreliance on one base model. \n* While the reward complexity claimed to be simple, reward functions are still tailored per task and depend on carefully tuned heuristics, which may not scale easily.", "was_revised": true, "final_justification": "I appreciate the authors addressing my questions in the rebuttal. I like the core idea of extending the findings in DeepSeek-R1 from the purely language domain to visual perception. The paper applies GRPO in the post-training stage of MLLMs to optimize perception-oriented tasks such as visual grounding, counting, OCR, and object detection. Therefore, I will maintain my positive score and recommend acceptance.", "rebuttal": "We greatly appreciate the Reviewer for the recognition of our work and the valuable comments and suggestions you provided. We have responded to each of your comments and questions as follows.\n\n* **Q1. The \"Perception-R1\" name and its scope for broader perception tasks.**\n\n\tThanks for your suggestion. We would like to emphasize that ***the core contribution of Perception-R1 is to propose a new, perception-first RL post-training paradigm.*** So we name our framework “Perception-R1”. To demonstrate its effectiveness, we selected four foundational visual perception tasks: visual grounding, OCR, visual counting, and object detection. Theoretically, our framework can be adapted to any visual perception task by defining an appropriate reward function. In fact, some latest works have already attempted to use rule-based RL techniques for segmentation [1] and tracking [2], which can be viewed as extensions of our work. In the future, we will also explore applying Perception-R1 to 3D visual perception tasks (such as 3D object detection and tracking, depth estimation, etc.) to further expand its application scope.\n\t\n\t[1] You, Zuyao, and Zuxuan Wu. \"Seg-R1: Segmentation Can Be Surprisingly Simple with Reinforcement Learning.\"  \n\t[2] Wang, Biao, and Wenwen Li. \"R1-track: Direct application of mllms to visual object tracking via reinforcement learning.\"\n\n* **Q2. Theoretical explanation of why rule-based RL is well-suited for visual tasks.**\n\n\tThis question is similar to the **Q1** of Reviewer # **S8da**. To further elaborate on our reasoning, we highlight the following points: \n\t 1. **RL conducts better negative supervision compared to SFT.** Visual representation learning and downstream visual task like *object detection* are highly dependent on the quantity and quality of *negative samples* [1][2]. In the context of post-training, the negative sampling spaces and strategies for RL and SFT differ significantly. With RL, any action that yields a low reward is treated as a \"negative sample.\" This allows RL model to effectively explore and learn from a wide range of trajectory samples on-line. While SFT methods only sample one trajectory (answer text) off-line and the negative supervision is very limited (only from a few tokens that are not strictly aligned with the answer).\n\t 2. **Policy gradient allows models to explore the perception pattern freely not just clone the expert behavior.** SFT is fundamentally a form of behavior cloning (BC), where the model learns to maximize the likelihood of an expert's demonstration (the ground-truth text). This approach can suffer from covariate shift: when the model makes a mistake, it enters a state distribution not seen during training, leading to compounding errors. In contrast, RL, particularly through policy gradient methods, optimizes the policy $\\pi$ to maximize the expected cumulative reward. This objective encourages the model to explore and discover novel strategies or implicit ”perception patterns\" that lead to high rewards, even if those patterns deviate from the expert's specific demonstration. It learns to solve the task robustly, rather than simply mimicking one particular solution path.\n\t 3. **RL allows for more flexible and task-aligned reward shaping.** The objective for SFT is fixed: minimizing the cross-entropy loss against a reference text. This is an indirect proxy for the actual task performance. For many visual tasks, performance is better measured by non-differentiable or complex metrics. RL's reward function $R$ can be designed to directly incorporate these metrics. For instance, in object detection, the reward can be the Intersection over Union (IoU) score, directly optimizing for localization accuracy. For image editing, the reward could be based on a combination of a CLIP score for semantic alignment and a score from an aesthetic quality model. This flexibility allows RL to align the model's learning process much more closely with the true end-goal of the visual task.\n\n\t[1] He, Kaiming, et al. \"Momentum contrast for unsupervised visual representation learning.\"  \n\t[2] Lin, Tsung-Yi, et al. \"Focal loss for dense object detection.\"   \n\n* **Q3. More baseline results.**\n\n\tFollowing your advice, we add more baselines including Qwen2-VL, Qwen2.5-VL, and LLaVA to apply our Perception-R1 framework. The results are shown as following: \n\t\n\t| case | base model | Visual Grounding | | | OCR | Visual Counting | | Detection |\n  |------|------|------------------|------------------|------------------|---------|------------------|------------------|-----------|\n  | | |  RefCOCO | RefCOCO+ | RefCOCOg | PageOCR | Pixmo_val | Pixmo_test | COCO2017 |\n  | LLaVA-1.5-7B | - | 57.2 | 49.8 | 50.2 | - | - | - | - |\n  | Perception -R1 | LLaVA-1.5-7B | 61.8| 54.0| 55.6| - | - | - | -|\n  | Qwen2-VL-2B | - | 86.8 | 77.1 | 83.3 | 94.4 | 60.2 | 50.5 | 0.1 |\n  | Perception -R1 | Qwen2-VL-2B | 89.1 | 81.7 | 85.7 | **98.2** | 78.1 | 75.6 | 28.0|\n  | Qwen2.5-VL-3B | - | 86.7 | 79.3 | 84.3 | 94.9 | 60.8 | 57.4 | 16.1 |\n  | Perception-R1 | Qwen2.5-VL-3B | **89.5** | **83.5** | **86.5** | 97.1 | **83.8**| **81.1**| **31.9**|\t\n\n\n* **Q4. Reward functions are still tailored per task and depend on carefully tuned heuristics, which may not scale easily.**\n\n\tThank you for your concerns regarding the reward function design. The reward function is central to rule-based RL. Given the discriminative nature of visual perception tasks, most reward functions are discriminative metrics such as Intersection over Union (IoU) or Euclidean distance. These different task-specific reward functions are not mutually exclusive, rather, they can be mixed and scaled up. Recent work [1] has explored this approach, demonstrating the effectiveness and scalability of a unified RL framework for multiple downstream tasks.\n\n\tFurthermore, from an RL perspective, this multi-component reward structure functions can be regarded as task-aligned reward shaping. As discussed in the response of **Q2**, for many visual tasks, performance is better measured by non-differentiable or complex metrics. RL's reward function $R$ can be designed to directly incorporate these metrics. By providing a dense and multifaceted objective rather than a single sparse signal, it inherently mitigates the risk of policy exploitation (i.e., reward hacking), thereby compelling the model to learn more robust and generalizable policies.\n\t\n\t[1] Ma, Yan, et al. \"One RL to See Them All: Visual Triple Unified Reinforcement Learning.\"\n\n\n* **Q5. Limited evaluation on real-world applications.**\n\n\tThanks for your advice. To address your concerns, we have conducted additional evaluations in real-world and robotic scenarios. Specifically, we select two challenging real-world understanding and reasoning benchmarks, i.e., RealWorldQA, and OmniSpatial [1] to evaluate Perception-R1.  The results are presented in the table below.\n\t\n\t| models | RealWorldQA | OmniSpatial |\n  |--------|-------------|-------------|\n  | Baseline (Qwen2.5-VL-3B) | 60.26 | 41.81 |\n  | Perception-R1 | **61.83** | **42.07** |\n\n\tWe can observe that Perception-R1 still demonstrates its effectiveness in noisy real-world scenarios.\n\n\t\n\t[1] Jia, Mengdi, et al. \"OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for Vision Language Models.\"\n\n* **Q6. Open code and model weights.**\n\n\tThank you for this valuable suggestion. In fact, we are already prepared to make our work open-source including data, code, and model weight. In order to adhere to the NeurIPS double-blind review policy, we have omitted this information in the current submission. We will be sure to include a direct link to the open-source repository in the final camera-ready version.\n\n* **Q7. Why SFT + RL performs worse than RL-only?** \n\n\tThank you for your question. We attribute the reason to two primary factors:\n\t\n\t1. Our base model, i.e., Qwen2-VL and Qwen2.5-VL, have already undergone a comprehensive SFT stage. Applying an additional SFT phase on this foundation risks disrupting the model's originally learned data distribution. \n\t2. As we analyze in the response for **Q2**, SFT is fundamentally a form of behavior cloning (BC) [1]. This suggests the model can become constrained by the rigid mimetic patterns acquired during the SFT phase, thereby impeding effective exploration in the subsequent RL stage.\n\n\tAs a result, performance can actually degrade, making the direct application of RL a more effective approach.\n\t\n\t[1] Chen, Jierun, et al. \"The Synergy Dilemma of Long-CoT SFT and RL: Investigating Post-Training Techniques for Reasoning VLMs.”"}, {"review_id": "L5Eakc49cL", "reviewer": "Reviewer_z4S9", "summary": "This paper introduces Perception-R1, a framework for enhancing the visual perception capabilities of Multimodal Large Language Models (MLLMs) through rule-based Reinforcement Learning (RL) during post-training. The paper argues that improving fundamental visual perception is a critical and currently under-explored prerequisite for achieving advanced visual reasoning. From the name, the paper positions itself as a visual-domain analogue to language-only reasoning models like DeepSeek-R1, by applying GRPO to Qwen2-VL series MLLM. The reward function uses task-specific, rule-based rewards such as IoU for visual grounding/object detection, and edit-distance for OCR tasks. For tasks with multiple objects, they used the Hungarian algorithm to solve the bipartite matching problem between prediction and ground truths. \n\nThe experiments showed 3 key findings: (1) an explicit CoT is found to be unnecessary and often detrimental to performance on the visual perception tasks studied (even though this was known to be helpful in language domain); (2) the effectiveness of RL over SFT is strongly correlated with the task's \"perceptual perplexity,\" a measure of the output space's combinatorial complexity, with RL showing greater benefits on high-perplexity tasks like object detection; and (3) meticulous reward engineering, including multi-subject matching and a curriculum sampling strategy, is crucial for guiding the model toward learning robust perception policies.\n\nIn terms of empirical results, Perception-R1 achieved SOTA results on several perception benchmarks, including a +4.2% improvement on RefCOCO+, a +17.9% improvement on PixMo-Count, and a +4.2% F1-score on PageOCR. The most notable result is achieving 31.9% Average Precision (AP) on the COCO 2017 validation set, which the authors claim is the first time a pure MLLM architecture has reached this level of performance on this challenging object detection benchmark.", "questions": "1. On Formalism and Definitions: Could you provide a more formal RL-centric definition of a \"perception policy\"? Specifically, how do you define the state, action, and observation spaces for the MLLM agent in tasks like object detection?\n2. On \"Perceptual Perplexity\": As mentioned above, how would you define perceptual perplexity with a formal equation? \n3. On the Role of Chain-of-Thought: Your work finds that an explicit thinking process is detrimental for perception, which directly contrasts with the findings from DeepSeek-R1 that inspired your work. How do you explain this fundamental divergence? Does this suggest that visual perception and linguistic reasoning are governed by different cognitive mechanisms within these models that require distinct alignment strategies?\n4. On the Choice of RL Algorithm: The paper uses GRPO, a policy-gradient algorithm. Could you justify this choice over preference-based optimization methods like DPO, or more specifically, PerPO, which targets the same problem of perceptual alignment in MLLMs? What are the hypothesized advantages of GRPO in this specific \"perception policy\" learning context?\n5. On Attributing Performance Gains in COCO: The 31.9% AP on COCO is a very strong result. To help the community better understand the source of this gain, could you provide an ablation where Perception-R1 is applied to the non-specialized Qwen2-VL-2B model for the detection task? This would help disentangle the contribution of your RL framework from the strong localization priors of the Qwen2.5-VL base model.\n6. On Statistical Significance: The paper reports impressive point-estimate improvements across all benchmarks. However, the checklist indicates that no error bars were reported. Could you please comment on the variance of your results across different training runs (e.g., with different random seeds) to establish the statistical significance of the claimed improvements?", "limitations": "The authors provide a dedicated limitations section (Section 6), acknowledging that current perception tasks may be too simplistic to fully leverage RL's potential and that finding more complex \"meta tasks\" is a key future direction. However, the paper does not discuss the potential negative societal impacts of creating more powerful perception models. While the work is foundational, enhanced capabilities in object detection, OCR, and visual grounding could be applied to surveillance or autonomous systems with societal consequences. A brief discussion of these dual-use possibilities would strengthen this section.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**:\n\n**Originality and Significance**: On the surface it seems like a straightforward application of GRPO to RLVR CV tasks. Most attempts at applying RL to MLLMs followed a language-centric paradigm. This paper instead proposes a “perception-first” approach, with the core argument that robust perception is a prerequisite for high-level visual reasoning, is a valuable hypothesis. This reframes the problem from “how do we make MLLMs reason better?” to “how do we first make MLLMs see better, so that they can then reason?”. \n\n**Quality**: The paper’s claims are supported by empirical validations across a diverse set of perception tasks. The result of 31.9% AP on COCO 2017 validation set is strong for MLLMs, although still behind expert models like Faster R-CNN (from 10 years ago), it is almost double that of the SFT baseline (16.1% AP). They also report consistent performance gains across visual grounding (RefCOCO+), visual counting (PixMo-Count), and dense OCR (PageOCR), which adds to the paper’s claim about enhancing foundational perception.\n\n**Clarity and Reproducibility**: The paper is well-written and clearly organized. The experimental results includes sensible ablations, showing for example a clear performance degradation when removing reward matching (from 31.9 to 23.5 AP on COCO), supporting the need for bipartite matching mechanism. The counter-intuitive finding that explicit CoT harms performance is a well-supported. Finally, there is a detailed analysis of reward components.\n\n**Weaknesses**:\n\n1. **Lack of Conceptual Rigor**: Several key concepts central to the paper's thesis are not defined with sufficient precision, which limits their generalizability and testability.\n  * The term \"**Perception Policy**\" is used throughout but its definition in Section 3 is a high-level, descriptive paragraph rather than a formal RL definition. The state space, action space (e.g., coordinate tokens, control tokens), and observation space for the MLLM agent are not formally specified. Without this formalism, the term remains ambiguous, making it difficult to precisely situate the work within the broader RL literature.\n  * The concept of \"**Perceptual Perplexity**\" is introduced as a key determinant of RL's effectiveness. Having some sort of formal equation for perceptual perplexity and how Table 9 is computed would be more informative.\n2. **Insufficient SOTA baselines**: The paper's positioning is narrow, primarily comparing its RL-based approach against SFT. It critically overlooks the most relevant and concurrent work on MLLM alignment, particularly methods based on preference optimization. Why is a policy-gradient approach like GRPO superior to, or different from, a preference-based approach for this specific problem?\n  * One example that can be good to include for comparison is PerPO ([74] in the submission), which uses \"discriminative rewarding\" and \"listwise preference optimization\" to enhance the same visual discrimination capabilities in MLLMs and has results in RefCOCO/+/g as well (Table 1 in the submission). \n3. **Limited Scope of Generalization Claims**: The paper trains task-specific models but makes claims about improving general capabilities. This claim is supported by limited evidence. The observation that RL training on the counting task improves performance on general comprehension benchmarks is intriguing but anecdotal. It is unclear if a model trained on object detection would also improve on ScienceQA, or if an OCR-trained model would improve on MM-Vet. Without a more systematic study, the claim of cross-task enhancement remains speculative.\n4. **Potential Confounding Factors in Performance Gains**: The paper's strongest result (COCO detection) is achieved using a different, more powerful base model (Qwen2.5-VL-3B) than the other experiments. The Qwen2.5-VL model is explicitly designed and optimized for \"precise object localization\". While the ablation shows a large gain over the SFT version of this same model, it is difficult to disentangle the contribution of the Perception-R1 framework from the strong inductive biases of the base model's architecture. The claim would be more robust if a similar relative gain were demonstrated on a less-specialized base model, such as the Qwen2-VL-2B used for other tasks.", "was_revised": true, "final_justification": "I am increasing my rating for this paper. The authors' thorough rebuttal and the additional experiments they conducted have successfully addressed most of the significant concerns raised in my initial review.\n\nHere is a summary of my final assessment:\n\n* **Resolved: The critical issue of performance attribution.** My primary reservation was whether the impressive COCO result was due to the framework or the powerful base model. The new ablation applying Perception-R1 to the weaker Qwen2-VL-2B model convincingly demonstrated the framework's effectiveness, resolving this concern. This was the most important factor in my decision to raise the score.\n* **Resolved: Lack of conceptual and empirical rigor.** The authors provided formal definitions for \"perception policy\" and \"perceptual perplexity,\" which solidified the paper's conceptual foundations. The new experiments testing for statistical significance and generalization across tasks have also strengthened the paper's empirical claims.\n* **Partially Resolved: Comparison to SOTA methods.** The authors provided a reasonable theoretical justification for their choice of GRPO over preference-based methods like DPO/PerPO. However, a direct empirical comparison is still missing. I view this as a minor remaining weakness, but not a critical flaw, as the primary comparison against SFT is sound and the paper's contribution is clear without it.\n\nIn summary, the authors' diligent response to feedback has significantly improved the paper. The paper now presents a well-supported and valuable contribution, with its core claims backed by much stronger evidence.", "rebuttal": "We sincerely appreciate your recognition of our work, as well as your thoughtful and professional review. The suggestions you provided are highly valuable for the further improvement of our work. We have responded to each of your comments and questions as follows.\n\n* **Q1. Lack of conceptual rigor including Perception Policy and Perceptual Perplexity.**\n\n\tWe are sorry that we did not provide detailed formal definitions in the main text due to the page limit. We offer the following clarifications below and will add them to the next revision.\n\n\tFirst, we aim to provide an RL-centric definition of **Perception Policy**. In the context of Perception-R1, the learned policy $\\pi_{\\theta}$ is the MLLM itself (e.g., Qwen2-VL). The learning process uses the GRPO algorithm to update the model’s parameters $\\theta$ based on rule-based rewards calculated from generated outputs. \n\n\tDuring this process, the observation space $O$ can be defined as $O=(I,P)$, where $I$ is the image input, $P$ is the text-based prompt that specifies the task. For object detection, for example, the prompt is a request to output the bounding box coordinates and the class names of objects. The state at step $t$ can be formally defined as: $s_t = (O, a_1, a_2, ..., a_{t-1})$, where $(a_1, a_2, ..., a_{t-1})$ is the action sequence of tokens based on next-token prediction mechanism. For the specific task of object detection, the action sequence must conform to a strict structure encodes both object locations and their categories that looks like:`{\"bbox_2d\": [127, 54, 471, 406], \"label\": \"laptop\"}`. For the formal equation of **Perceptual Perplexity**, given a perceptual task dataset, the perceptual perplexity *P* of this task can be represented as:\n    $$P = \\prod_{i=1}^{N} m_i!$$\n\twhere $m_i$ is the number of ground-truth objects belonging to category $i$, $i$ ∈ {1, 2, ..., *N*} *N* is the total number of possible object categories for the task. The sum of objects across all categories must equal the total number of objects: $\\sum_{i=1}^{N} m_i = M$. For visual grounding and OCR task, $M = N = 1$. For visual counting task, $M > 1, N = 1$. For object detection, $M > 1, N > 1$. \n* **Q2. Comparison between policy-gradient approach like GRPO and preference-based approach like DPO.**\n\n\tThank you for your valuable suggestion. While the relative merits of policy-gradient RL like GRPO versus preference optimization like DPO have already been extensively studied in prior work [1][2][3], our work focuses primarily on their application within visual perception contexts. In this domain, we identify two principal advantages of GRPO over DPO:\n\t1. **Dynamic on-Policy exploration**. Through a dynamic and interactive training process, GRPO samples data from the current policy model and continuously updates the policy online. This enables it to explore the policy space, learn from its own mistakes, adapt to its evolving capabilities, and constantly create new, more challenging \"curricula\" for itself. This implies that GRPO affords the model a broader exploration space, enabling it to discover superior visual perception polices. In contrast, DPO, as an offline method, relies on a fixed preference dataset, which leads to a \"distribution shift\" problem and limits its further performance improvement.  \n\t2. **Generation of high-quality hard negatives.** GRPO's online sampling naturally generates high-quality negative examples that are relevant and challenging. Since the model explores at the edge of its capabilities, its failures are meaningful mistakes rather than random errors. These \"hard negatives\" are incorrect but plausible solutions that provide maximum learning value. DPO is limited by its pre-collected dataset, where the quality of negative examples may not target the model's specific weaknesses as it evolves. GRPO therefore creates a more effective learning process by continuously finding its own most challenging examples.\n\n\tAs for the work on PerPO [4], although it is specifically optimized for visual perception tasks through \"discriminative rewarding\" and \"listwise preference optimization”, its approach still falls within the paradigm of DPO. Consequently, it cannot fundamentally resolve the two aforementioned deficiencies compared to GRPO.\n\n\t[1] Wang, Bo, et al. \"Implicit Reward as the Bridge: A Unified View of SFT and DPO Connections.\"  \n\t[2] Tang, Yunhao, et al. \"Understanding the performance gap between online and offline alignment algorithms.\"  \n\t[3] Lanchantin, Jack, et al. \"Bridging Offline and Online Reinforcement Learning for LLMs.\"  \n\t[4] Zhu, Zining, et al. \"Perpo: Perceptual preference optimization via discriminative rewarding.\"\n\n* **Q3. More systematic study to support the claim of generalization enhancement.**\n\n\tBased on your suggestion, we ran additional experiments to test our generalization claim. We used models post-trained on different tasks. Note that we excluded OCR because its focus on text extraction differs fundamentally from the other tasks, which require general visual capabilities like spatial perception and object identification. The results are presented below.\n\t\t\n\t| | post-training task | MMBench | MMVet | MMStar | ScienceQA | SeedBench | MME | | LLaVA-Bench | AI2D |\n  |---|---|---|---|---|---|---|---|---|---|---|\n  | | | Avg | Avg | Avg | Avg | Avg | Cognition | Perception | Avg | Avg |\n  | Qwen2-VL-2B | -- | 71.9 | 45.6 | 46.3 | 74.0 | 72.7 | 418.5 | 1471.1 | 46.5 | 71.6 |\n  | **Perception-R1** | visual counting | 71.8 | 48.9 | 45.7 | 73.4 | 73.0 | 430.0 | 1473.9 | 58.2 | 71.8 |\n  | | visual grounding | 71.4 | 49.2 | 46.6 | 73.6 | 72.9 | 425.4 | 1457.2 | 52.0 | 71.3 |\n  | Qwen2.5-VL-3B | -- | 71.4 | 57.7 | 53.3 | 75.8 | 70.9 | 560.7 | 1470.8 | 62.7 | 79.0 |\n  | | object detection | 78.5 | 56.4 | 55.7 | 78.1 | 73.7 | 611.4 | 1546.7 | 65.2 | 80.5 |\n\n\tThe results show that the Perception-R1, when trained on diverse visual tasks, consistently enhances general visual understanding and reasoning. This demonstrates the effectiveness and generality of our method.\n\t\n* **Q4. More experimental results based on Qwen2-VL and Qwen2.5-VL.**\n\n\tIn response to your suggestion, we have conducted additional experiments comparing our Perception-R1 framework with the Qwen2-VL and Qwen2.5-VL baselines. The experimental results are presented as follows:\n\t\t\n\t| case | base model | Visual Grounding | | | OCR | Visual Counting | | Detection |\n  |------|------|------------------|------------------|------------------|---------|------------------|------------------|-----------|\n  | | |  RefCOCO | RefCOCO+ | RefCOCOg | PageOCR | Pixmo_val | Pixmo_test | COCO2017 |\n  | Perception -R1 | Qwen2-VL-2B | 89.1 | 81.7 | 85.7 | **98.2** | 78.1 | 75.6 | 28.0|\n  | | Qwen2.5-VL-3B | **89.5** | **83.5** | **86.5** |97.1 | **83.8**| **81.1**| **31.9**|\t\n\n\n* **Q5. Does the difference in reasoning between Perception-R1 and DeepSeek-R1 imply that their visual and linguistic abilities rely on separate mechanisms, each requiring its own alignment strategy?**\n\n\tYes, as we argue in the paper, this difference arises because visual perception tasks are primarily oriented toward visual logic rather than semantic or linguistic logic. For tasks like object counting or localization, a model learns implicit patterns directly from image pixel. Imposing an explicit, language-based reasoning chain introduces an unnecessary and potentially detrimental step that degrades visual perception capabilities. Similar findings are also documented in [1].\n\n\tFurthermore, we wish to clarify that although the reasoning paradigms for visual perception and linguistic reasoning differ, they can still be unified. By designing and combining distinct reward functions, a model can be trained to accommodate both modes of thinking, an avenue that has already been explored in [2].\n\t\n\t[1] Wei, Yana, et al. \"Open Vision Reasoner: Transferring Linguistic Cognitive Behavior for Visual Reasoning.\"  \n\t[2] Ma, Yan, et al. \"One RL to See Them All: Visual Triple Unified Reinforcement Learning.\"\n\n* **Q6. Provide statistical significance such as with different random seeds.**\n\n\tWe appreciate you highlighting the importance of statistical significance. In response, we conducted stability experiments for Perception-R1 on the RefCOCO/+/g. Each experiment utilized a different random seed to assess the robustness of our findings, and the results are shown below.\n\t\t\n\t| case | seed value | Visual Grounding | | |\n  |------|------|------------------|------------------|------------------|\n  | | |  RefCOCO | RefCOCO+ | RefCOCOg | \n  |1 | 123 | 88.7 | 81.0 | 85.1 |\n  | 2| 124 | 89.1 | 81.5 | 85.3 |\n  | 3| 125 | 88.8 | 81.0 | 85.4 |\n  | Average | -- | 88.9 | 81.2 | 85.3 |\n\n\tWe can observe that under different random seed settings, the model's performance fluctuation on benchmarks does not exceed 1 point, indicating that Perception-R1 possesses sufficient stability to support reproducibility.\n* **Q7. Negative societal impacts of creating more powerful perception models.**\n\n\tWe appreciate you raising the critical issue of societal impact. We agree this was an important omission and, following your suggestion, have added a discussion about this as follows:\n\t“It is essential to acknowledge the broader societal impacts and dual-use nature of our work. Like many AI technologies, the perception models that offer benefits in visual grounding, OCR, and object detection can also be exploited. For instance, highly accurate person detection could fuel mass surveillance and threaten civil liberties, while powerful OCR could scan sensitive documents without consent. As these models become foundational to AI systems, ethical considerations become paramount. We urge the community to lead the conversation on responsible development, establish clear guidelines, and create robust safeguards to mitigate these risks.”"}, {"review_id": "Iu1W9lLqyB", "reviewer": "Reviewer_8gLg", "summary": "Perception-R1 is a collections of Qwen VL 2 and 3B models finetuned using GRPO on a variety of vision tasks. The resulting models are performing well across a wide range of benchmarks, The authors provide interesting ablation on RL, SFT and SFT + RL and on the reward components.", "questions": "* You report the model achieving higher performance without thinking. How come the model cannot automatically learn to skip thinking if it is detrimental?", "limitations": "The author must acknowledge more explicitly that their method is limited to train specialist models since \"the rewards for different tasks vary\".", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "# Strengths\n\n* If the multi subject reward function is novel, it is an interesting contribution.\n* The degradation of the performance for Perception with thinking is interesting, but requires further investigation, i.e., why can't the policy learn to not think if it's detrimental to the performance?\n* The authors test their method across a variety of tasks: grounding, counting, OCR, etc.\n\n# Weaknesses\n\n## Specialist model\n> Notably, we do not mix the data from different perception tasks for joint training because the rewards for different tasks vary.\n> We mainly use the Qwen2-VL-Instruct-2B as the baseline model.We also utilize Qwen2.5-VL-3B-Instruct [3] for training object detection tasks, due to its specialized optimization for localizing bounding boxes. \n* It is not obvious from the text, but I understand that different models are trained for different tasks, instead of a single generalist model. This makes the results less impressive since the performance of Perception R1 is closer to the specialized baselines..\n\n## Statistical significance and learning curves\n\n* How was the RL checkpoint chosen for evaluation? The best performing checkpoint on the test set?\n* Are the results consistent across seeds and training steps?\n\n## Clarity:\n\n* MLLM is used in the first sentence of the abstract and only defined on the 2nd page. Same for GRPO which is used at the end of the abstract.\n* Please make it obvious that Perception-R1 is a suite of models and not a single generalist model.\n\n## Missing prior work\n\n* Some relevant important prior work is missing. For example,  \n\nPinto, André Susano, et al. \"Tuning computer vision models with task rewards.\" International Conference on Machine Learning. PMLR, 2023.\n\nMathe, Stefan, Aleksis Pirinen, and Cristian Sminchisescu. \"Reinforcement learning for visual object detection.\" Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.", "was_revised": true, "final_justification": "I am not in a position to recommend the paper in this current form. In my opinion, the main limitation of this work is that they trained a suite of specialize model, while the main challenge is to train a single model of a suite of tasks.", "rebuttal": "We sincerely appreciate the Reviewer for providing valuable feedback and suggestions. We have responded to each of your comments and questions as follows:\n\n* **Q1. Clarification and explanation regarding \"specialist model”.**\n\n\tThank you for your question regarding whether Perception-R1 is a single generalist model or a collection of specialist models. In fact, ***the core contribution of Perception-R1 is to propose a new, perception-first RL post-training paradigm.*** Our focus is more on the ***effectiveness*** and ***generality*** of this method rather than build a single unified model. For the sake of training efficiency, we trained different tasks separately using only a very small amount of data in our experiments. However, it is entirely feasible to mix the rewards from different tasks to perform joint training, and some recent work [1] has already done so. To further demonstrate this feasibility, we also conducted a simple mixed-reward experiment, as shown in the table below:\n\t\n\t| case | data | Visual Grounding | | | Visual Counting | |\n  |------|------|------------------|------------------|------------------| ------------------| ------------------|\n  | | |  RefCOCO | RefCOCO+ | RefCOCOg | Pixmo_val | Pixmo_test |\n  |single reward | 5k grounding | 89.1 | 81.7 | 85.7 | - | - |\n  |single reward| 5k counting | - | - | - | 78.1 | 75.6 |\n  |mix reward| 5k grounding + 5k counting | 88.6 | 81.1 | 85.2 | 78.0 | 75.2 |\n\n\tWe can observe that mixing rewards from different tasks for joint training does not cause significant negative impact on task performance. Therefore, it is entirely feasible to train a single unified model if training efficiency is not a primary concern.\n\n\t\n\t[1] Ma, Yan, et al. \"One RL to See Them All: Visual Triple Unified Reinforcement Learning.\"\n\n* **Q2. How was the RL checkpoint chosen for evaluation?**\n\n\tAll experiments were evaluated using the final checkpoint of the trained model.\n\n* **Q3. Are the results consistent across seeds and training steps?**\n\n\tRegarding the training seed and the number of training steps, we compiled the model's performance under different training seeds and numbers of training steps, as shown in the table below:\n\t\n\t| exp | seed value | Visual Grounding | | |\n  |------|------|------------------|------------------|------------------|\n  | | |  RefCOCO | RefCOCO+ | RefCOCOg | \n  |1 | 123 | 88.7 | 81.0 | 85.1 |\n  | 2| 124 | 89.1 | 81.5 | 85.3 |\n  | 3| 125 | 88.8 | 81.0 | 85.4 |\n  | Average | -- | 88.9 | 81.2 | 85.3 |\n\n\t| exp | training step | Visual Grounding | | |\n  |------|------|------------------|------------------|------------------|\n  | | |  RefCOCO | RefCOCO+ | RefCOCOg | \n  | 4| 100| 88.5| 80.9| 84.5|\n  | 5| 200| 88.7| 81.3| 85.1|\n  | 6| 300| 88.8| 81.5| 85.7|\n  | last|357 | 89.1| 81.7| 85.7|\n\n\twe can observe that model training is not sensitive to seed settings (performance fluctuation does not exceed one point). Additionally, RL training steps show a positive correlation with final performance, and as training steps increase, performance gradually converges to an upper limit.\n\t\n* **Q4. Clarify of some abbreviations and missing prior work.**\n\n\tThank you very much for your meticulous review. Based on your feedback, we have carefully examined all occurrences of technical term abbreviations to ensure that full explanations are provided upon first appearance. We have also added citations for the missing prior work you identified. These updates will be presented in the next version.\n\n\n* **Q5. Why not make model automatically learn to skip thinking?**\n\n\tThank you for this interesting question. We would like to answer your query from two aspects:\n\n\t1. First, as our experiments in the paper demonstrate, an explicit linguistic thinking process is not beneficial for visual perception tasks. We observed that including such a process consistently degraded the model's final performance. This finding prompted us to disable the explicit linguistic thinking process and instead allow the model to learn an implicit \"visual logic.\" For a more detailed explanation of \"visual logic\" and \"linguistic logic,\" please refer to our response to **Q5** of Reviewer # **z4S9**.\n\n\t2. Second, it is indeed feasible to have the model autonomously learn to skip the thinking process. We conducted experiments on this as well (shown in the table below). Specifically, we enabled an explicit reasoning process training on the RefCOCO by incorporating `<think>` and `<answer>` special tokens. We then tracked the number of tokens generated within the `<think>` tags as training progressed. We can observe that during the RL process, the model tends to gradually shorten the length of its \"thinking\" tokens. However, this approach is highly inefficient, and the final performance is not as good as when we disable the thinking process directly.  \n  | exp | training step | thinking_length |\n    |------|------|------------------|\n    | 1| 10 | 72|\n    | 2| 50 | 39|\n    | 3| 100| 27|\n    | 4| 150| 25|\n    | 5| 200| 20|\n\n\tTherefore, we opted to bypass the explicit reasoning steps, prompting the model to generate the final answer directly."}]

Author rebuttal:

Meta-review: This paper proposes to improve multimodal LLMs in object detection, counting, and visual grounding through rule-based RL (with GRPO). The authors make several insightful findings, such as explicit chain-of-thought can hurt perception, and that the effectiveness of RL is correlated with the task's "perceptual perplexity."

The reviewers raised valid concerns regarding the conceptual rigor of terms like "perception policy," the lack of comparison to preference-based methods, and using a highly specialized base model. The authors responded by offering formal definitions for their key concepts, provided strong theoretical justifications for their choice of RL algorithm, and ran additional ablation studies. The new experiment applying their framework to Qwen2-VL-2B demonstrated that the performance gains are attributable to their method and not just the specialized base model.

The reviewers were largely satisfied with the authors' responses, with 3/4 recommending acceptance and raising their scores after the rebuttal period. The paper is well-executed, the results are significant, and it opens up a valuable new direction for grounding multimodal LLMs. The paper makes valuable contribution to the conference.
Decision: Accept (poster)