Perception in Reflection
Authors: Yana Wei
Liang Zhao
Kangheng Lin
En Yu
Yuang Peng
Runpei Dong
Jianjian Sun
Haoran Wei
Zheng Ge
Xiangyu Zhang
Vishal M. Patel
Venue: ICML 2025 poster (2025)
Keywords: Perception
Reflection
LVLMs
Primary area: deep_learning->large_language_models

Abstract: We present a perception in reflection paradigm designed to transcend the limitations of current large vision-language models (LVLMs), which are expected yet often fail to achieve perfect perception initially. Specifically, we propose Reflective Perception (RePer), a dual-model reflection mechanism that systematically alternates between policy and critic models, enables iterative refinement of visual perception. This framework is powered by Reflective Perceptual Learning (RPL), which reinforces intrinsic reflective capabilities through a methodically constructed visual reflection dataset and reflective unlikelihood training Comprehensive experimental evaluation demonstrates RePer's quantifiable improvements in image understanding, captioning precision, and hallucination reduction. Notably, RePer achieves strong alignment between model attention patterns and human visual focus, while RPL optimizes fine-grained and free-form preference alignment. These advancements establish perception in reflection as a robust paradigm for future multimodal agents, particularly in tasks requiring complex reasoning and multi-step manipulation. Project Page: [https://weiyana.github.io/Perception-in-Reflection](https://weiyana.github.io/Perception-in-Reflection)

Perception in Reflection
Yana Wei
Liang Zhao
Kangheng Lin En Yu* Yuang Peng" Rumpei Dong Jianjian Sun"
Haoran Wei?
Zheng Ge Xiangyu Zhang? Vishal M. Patel
Abstract
Firat look:
What beauthil scenet The boaseettoll and pal rices stand quictly against the piik sursil, creating such pascoful vibe.
We present a perception in reflection paradigm designed to transcend the limitations of current large vision-language models (LVLMs), which are expected yet often fail to achieve perfect per- ception initially. Specifically, we propose Re- flective Perception (RePer), a dual-model reflec- tion mechanism that systematically alternates be- tween policy and critic models, enables iterative refinement of visual perception. This framework is powered by Reflective Perceptual Learning (RPL), which reinforces intrinsic reflective ca- pabilities through a methodically constructed vi- sual reflection dataset and reflective unlikelihood training. Comprehensive experimental evalua- tion demonstrates RePer's quantifiable improve- ments in image understanding, captioning preci- sion, and hallucination reduction. Notably, RePer achieves strong alignment between model atten- tion patterns and human visual focus, while RPL optimizes fine-grained and free-form preference alignment. These advancements establish percep- tion in reflection as robust paradigm for future multimodal agents, particularly in tasks requiring complex reasoning and mul manipulation. Project Page: https: www.yna..aaaaaaaa//.aa/.. github. in Reeeon-eiiieeeeeiiinnnn
Reflection thought
carctully more dctails.
Second CUPPERS logo the bosketball hood
Question:
can you irage?
Human perceptron process
coen wee Expeootheil horp snowen people
LVLM
Halluchation
(b) Existing LVLM methods
First turn:
hoop palnn
Feedback:
Your anower correct but missing some details. Can you Iryil again?
d Critic model
Policy model
Sacond turn:
Ckay! can sea baskettaal hoop palm
CLIPPERS
(c) Perceptron reflection (Ours)
Figure I. Existing LVLMs expected to deliver accurate percep- tions initially, but humans often reflect and refine answers gradu- ally. We introduce perception in reflection, employing policy and critic model interactions to fully harness perceptual capabilities.
well-trained models can achieve sufficiently accurate initial perception. Such perceptual accuracy enables the model to process visual inputs and generate appropriate responses in a single pass (Liu et al., 2024c;b; Wang et al., 2024). However, the frequent occurrence of hallucinations and mis- perceptions hinders their wider applicability in real-world scenarios. As shown in Figure 1, even for simple scenes, models may generate hallucinatory descriptions (e.g., as seen in (b)) or fail to capture essential details (e.g., as ob- served in the initial response in (c)). This raises an important consideration: Are current perception paradigms inherently limited, or might there be a more reasonable paradigm?
1. Introduction
In advancing large vision-language models (LVLMs)) (GPT- 4o, 2024; Liu et al., 2024c; Bai al., 2023), considerable attention has often been focused on enhancing the mod- els' visual perception capabilities for image understanding. This emphasis stems from fundamental assumption that
'Equal contribution 'Johns Hopkins University "StepFun BBPT "HUST 'Tsinghua University UIUC. Correspondence to: Yana Wei <ywei66@jh.cduu Vishal M. Patel <vpa- je..@jju..duu.
Some methods (Chen et al., 2023; Liu et al., 2024d; Yu et al., 2023) attempt to mitigate this through a sort of vi- sual chain-of-thought (CoT) (Wei et al., 2022) reasoning. They establish paradigm that first executes fine-grained perceptual tasks (such as grounding object locations (Chen et al., 2023; Shao et al., 2024), structures (Liu et al., 2024d) or identities (Yu et al., 2023)) before engaging in broader
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Perception in Reflection
perception. However, these approaches face key limita- tion: the reliance specialized tasks and data formats that are difficult to generalize across all vision-language tasks, e.g., box CoT can not be used in math geometry problems, making it challenging to achieve consistent visual percep- tion across diverse scenarios. Furthermore, CoT does not change the original single-pass manner. When perceptual errors occur, it is unable adjust and rectify them.
capabilities from both discriminative and generative perspec- tives. Comprehensive ablation studies on data construction, training strategies, reflection rounds, and critic designs ver- ify RePer's 's generalizability, establishing it as a fundamental paradigm for advancing multimodal perception.
In order to thoroughly unveil the underlying mechanisms be- hind perception in reflection, we further conducted a series of analytical experiments. Our comprehensive experimental analysis reveals two key findings:
Shifting the view to the real world, we observe that humans, as shown in Figure/ 1, typically do not perceive in a single step, rather, they establish cognition through gradual observation. This iterative process enables humans to continually enrich, refine, and enhance their perceptual outcomes. Drawing inspiration from this, we think that a reasonable perception paradigm for LVLMs should be iterative rather than a single-pass. In other words, the ability to reflect and improve over multiple rounds is not just desirable feature; it's a fundamental requirement for LVLMs to achieve robust and generalizable perception.
RePer can effectively migrate image attention towards human--lligne regions through iterative refinement. This implies that the perceptual patter utilized by RePer aligns more closely with that of humans.
RPL can be regarded as free-form preference opti- mization framework that unifies various preference learn- ing paradigms, e.g. DPO (Rafailov et al., 2024), and LiPO (Liu et al., 2024e), while enabling fine-grained su- pervision through explicit feedback signals.
In this paper, we propose a novel perceptual mechanism, termed Reflective Perception (RePer). Its purpose is to enable LVLMs to, like humans, use a perception- feedbacc loop to gradually establish precise visual cognition. To achieve this, we make RePer a dual-model architecture, i.e., policy model and critic model, to enable LVLMs to conduct pereption and reflection separately in terms of multi- turn dialogues between policy and critic model. In this way, LVLMs distill lessons from past experiences, gradually direct attention toward critical visual primitives, and thereby produce more accurate and refined responses.
These two key findings underscore the crucial value of per- ception in reflection in enhancing multimodal understand- ing and reasoning capabilities. We believe it will become an essential capability for multimodal agents in the future, particularly in complex visual reasoning (Xie et al., 2024; Malkiński & Mańdziuk, 2022) and multi-step manipula- tion (Sampat al., 2022; Kim et al., 2024) tasks.
2. Perception in Reflection
Although LVLMs inherently possess reflective perception capabilitics, this ability is instable and has not been ef- fectively activated (Kumar et al., 2024). To this end, we further propose a Reflective Perceptual Learning (RPL) approach. Through strategic temperature sampling and a hybrid evaluation system combining model and rule-based rewarding, we construct an online, multi-turn visual reflec- tion datuset. This dataset exhibits progressive improvements in both perception accuracy and response quality across di- alogue turns. Building upon this, we propose reflective unlikelihood training, an imitation learning approach (Ross et al., 2011; Swamy et al., 2023) that calibrates the model's preferences across responses of varying quality, thereby mitigating behavioral collapse (Kumar et al., 2024) where models tend to generate suboptimal responses in early turns.
In this section, we first define our problem and formal- ize the objective from a reinforcement learning perspective (Section 2.1). We then elaborate on how models learn to perceive through reflection, encompassing both data con- struction and training strategies (Section 2.2). Finally, we present the inference algorithm for reflective perception during deployment (Section 2.3).
2.1. Problem Definition and Formulation
Perception in LVLMs. Perception, as concept in the field of computer vision (He et al., 2016; Ren et al., 2016; He et al., 2017),, refers to the process of interpreting and understanding sensory, ie., vision, information from the environment. In the context of LVLM, we typically define perception as the process by which the model recognizes and understands the image or video. The perception capability of the model will directly determine the accuracy of its understanding and reasoning towards real world.
Extensive experiments demonstrate that RePer achieves su- perior performance across various benchmarks including image understanding, hallucination detection and detailed image caption, e.g., 54% CAPTURE on DetailCaps (Dong al., 2024a) and 51% % accuracy on HallusionBench (Guan et al., 2024). Using GPT-4o (GPT-4o, 2024) and DALLE- 3 (Betker et al., 2023), we validate its enhanced perception
Perception in Reflection. Our goal is to mimic human perception, establishing a
feedback loop through LVLM's iterative attempts to enhance image comprehen- sion and response accuracy. In pursuit of this, we model our
2

---

Perception in Reflection
challenge through the lens of reinforcement learning (RL), inspired by SCoRe (Kumar al., 2024) and RISE (Qu et al., 2024). To be specific, given dataset D= = { 24, yi J of images I,, questions I₁, and oracle responses y;, we aim to train an LVLM policy To(- [ <; 01.t, Sta)). This model, given an image I and question æ, along with f pre- vious attempts U1: and feedback prompts f1.r, is designed to perceive the image as accurate as possible and deliver the most correct possible answer y. Formally, given a veri- fier r(y.y" to assess the correctness of model response y compared to oracle answer y", we aim to derive policy that utilizes the aforementioned information to produce the outputs with the highest correctness reward over T rounds:
Siep-I: Caplion Generation
Aloy Iapul Gooo
Model: L1VV2-155-308
espert answer the queston bowrd on the given image. For the question, generate several condidate answers with diffeeent temperaturn.
System Prompt:
Given the question: [Can you provide a comprebensive caption for this image?)
Candidate This image shows a bed two by white headboards- Gudpng 1
Candidate 2: I can observe twin bed with... ...
Step-2: VL.M-Baso Rewani Sconing
Input
Model: GPT4-o / /Gmini
Kating Criteria: 3. Authenticity (4 points): the answerr should
Correctness points):
Candidate Anewers: the image, can
observe twin
Scores: [AAuthentictty: Correctness: 2, Final score: 7) (Authenticity: 4.... Final
...]
Reason: [The generated incorrectly deserihes the headboards white.., The reason that.... ..]
max 30
(la v*).
Step-3 Rule-Rosed Reward Scoring
Model: Factual parser
(1)
Input
Two Reference Captions (from GPT-qp and Gemini-Pro)
Section 2.1 resembles a multi-round Markov Decision Pro- cess (MDP) (Qu et al., 2024) or can be viewed as an RL or supervised finetune (SFT) objective. It is noteworthy that every historical attempt is synchronously optimized to maximize the ultimate reward.
Candidate Answers: 1. In the imago, we con see...
2. can observe a twim
(Stediee
Visual Elements (oljject, Attribute, Relations):
Por each reference: (obj: [bed, ..., attr: [white..... re): the center.....
Por each candidates: (obj: [yillow, --., attr: [red,--], mel: [on the bed,...B
Scores Candidatese [o.55, 066.... ...
Stop-4: Multi-Ture Reflective Dialogue
2.2. Reflective Perceptual Learning
Oulpit Filer Tnp
Candidate Answers: In the imnge, we can sut... 2. can obseeve a Iwlik ...
based Reward and Ressons [(%, The genursted answer incoraccly..... (8, The reason is that........
Despite existing LVLMs often possessing intrinsic self- reflection capabilities (Liu al., 2024a), these abilities have been shown to be remarkably fragile (Kumar et al., 2024). In other words, they struggle to adaptively refine their re- sponses based on given feedback (as shown in Figure 12). To address this limitation, we propose Reflective Perception Learning (RPL), a methodology that trains models to contin- uously enhance their previous responses through imitation learning (Ross et al., 2011; Swamy et al., 2023). We first elaborate on the data collection and training objective.
Rule-based Reward [o.55, o......)
Pilter
Score Gap Threshold
Candidate Answers Rank TETL Answer >> Turna Answer -o Turas Answer Per defrree Torplase
Multi turn Reflective Dialogue:
Question: Given an image, can you provide a comprehensive caption foe this image? Turni Answer: This image shonee a twin had with two side-by-side whit....
Turni Feedback: soore of given this caption. The descriptton inacourately mentions different pilloweases each side; both visible pillows have pillowcases....
Figure 2. Data construction pipeline of visual reflection dataset.
Data Construction. Naturally, we structure a multi-turn dialogue encompassing the sequence of posing questions, providing responses, receiving crroncous feedback, and sub- sequently re-responding and re-evaluating. This iterative process is designed to cultivate and demonstrate reflective perception capabilities within the trained models.
maintaining semantic relevance.
Step-2 VLM- Reward Scoring. For the generated multiple candidate responses, we employ a robust Visual- Language Model (VLM) to conduct a comprehensive and multifaceted evaluation, yielding fine-grained scores.
Practically, we expect the model to, (1) generate diverse responses based on all past answers and feedback, thereby enabling the exploration during reflection towards a percep- tion with fewer errors; (2) gradually produce more accurate answers in multi-turn dialogues, ensuring the convergence of the reflective chain. To meet these requirements, we construct visual reflection dataset for model imitation. Figure 2 gives an overview, with detailed steps as follows:
Step-3 Rule-Based Reward Scoring. Then we design a pipeline to extract key elements, e.g., objects, attributes, and relations, from both images and responses, and establish matching rules to compute alignment scores.
Step-4 Reflective Dialogue construction. After obtain- ing the candidate answers and their corresponding reward scores, we select samples meeting two criteria: (a) mini- mum score gap between the highest and lowest responses, and (b) at least one response scoring above the specified points. Then the filtered responses are structured into N rounds of reflective dialogue, progressing from low- est to highest scores. To this end, we curate a dataset
Step-I: Initial Candidate generation. We employ tem- perature sampling to generate diverse candidate answers per image-question pair. This approach ensures sufficient variation in response style, detail level, and accuracy while
3

---

Perception in Reflection
== H (trr))))))))) where it is sampled from model outputs, f represents specifie feedback, and denotes the corresponding reward score.
fo
Critic Model Fa
Ga.ral D
Policy Model igg
Algorithm 1 Reflective Perception (RePer)
1:
Initialize Policy, Critic model: No. To
2: Generate initial perception response Yo using Th given image and language instruction
Projection
He
Turn
Visual Encoder
Inferance flow
Text response
Text feedback
Image
Generate initial evaluation ro. fo using re given (I, x, yo)
Figure 3. Inference pipeline of reflective perception.
4: Set 0-
5: while t < max trials do
unlikelihood loss scale.
6:
Generate perception response Yr using ne given
(I, x. yo. Tu. .o.... 11-1, ft-1 Generate evaluation Ft: Jt using ro given (I,x, yo. ro. fo. ..., 1t-1, 1o-
Essentially, we employ rewards to balance likelihood and unlikelihood. In the initial rounds where the reward is lower (smaller loss weight), there is a predisposition towards un- likelihood, promoting the penalization of the response. Con- versely, in subsequent rounds where the reward is higher (larger loss weight), there is tendency towards likelihood, encouraging rewarding of the response. We will elaborate this on Section 3.2.
7:
8: Increment
9: end while
10: return
Two points merit attention. First, it is crucial to reward answer of each round using a hybrid scoring mechanism. This approach aims to align the model with both rule-based and model-based reward systems (Mu et al., 2024), thereby maximizing its ability to generalize to complex real-world scenarios. Second, we aim to devise responses based on the self-generated outputs of the model, thereby facilitating an online optimization process. This is intended to minimize the risk of the model overfitting to non-reflective capabili- ties (Kumar et al., 2024; Qu al., 2024; Tang al., 2024). Reflective Unlikelihood Training. Based on the con- structed data, we apply imitation learning (Ross et al., 2011; Swamy et al., 2023) to simulate reflective perception. This learning process necessitates the disregard of textual pat- terns, focusing instead on the cultivation of capabilities.
2.3. Reflective Perception
As shown in Algorithm and Figure define reflective perception during the inference process collaborative interaction between the well-trained policy and critic agents.
Initially, the policy model observes and provides its percep- tual results, which are then evaluated by the critic model. The critic model assesses the policy's perception, providing both a score gauge the quality of the perception and a ra- tionale for the score given. Following this, the policy model reflects on its perceptual errors, informed by the critic's eval- uations on self tllf-generated responses from previous rounds, and produce a new perceptual response. The critic model then delivers updated feedback based on all previous per- ceptions and critiques. This iterative process continues until predetermined limit reached.
More critically, we seek to prevent the model from overfit- ting to multi-turn responses and avoid the behavioral col- lapse (Kumar et al., 2024) where the model consistently generates suboptimal initial replies. In previous efforts, both RISE (Qu et al., 2024) and SCoRe (Kumar et al., 2024) pri- marily utilized SFT for imitation learning. However, RISE employed the exponent of centered rewards to mitigate this issue, while SCoRe utilized reward shaping to counteract. In this paper, we propose a method that simultaneously bal- ances likelihood and unlikelihood (Welleck et al., 2019), formalized as follows:
Through iterative reflective perception mechanism, the model can achieve human-like visual perception patterns, as empirically validated by the experimental results demon- strated in Figure 4.
3. Discussion
3.1. RePer Progressively Aligns Human Attention.
The reflective capabilities of language models have been widely studied (Liu et al., 2024a; Kumar et al., 2024; Qu et al., 2024). It is suggested that these models inherently possess a reflective ability that, although unstable, can im- prove the certainty of outputs and activate deeper levels of knowledge (Liu et al., 2024a). Yet, the effectiveness of this capacity LVLMs remains an unexplored question.
T
max E. B of log To (T(le)) + a<11 ot) log(1 no mo (Vilo))).
(2)
where o denotes a single sampling instance from our con- structed dataset D. and of = F(r)) represents the normal- ization of reward rÿ. a is a constant term that adjusts the

---

Perception in Reflection
Image Attention Maps
Input Image
(a) LLAVA-1.5
(b) RePer (Turn 1)
(e) RePer (Tura 2)
HaVA--15: The image festures a benntiful nighttime view of large, filuminatted collle with a dork tower. The essile is surrounded by crowel of people, background. people appear livily atmosphere around the aerle. The scere cxpiures the cesemoo of peguiarr tourist destirution, with the cvelle serving as a stunning focal point.
Output Answers
somte stunding closer foroground and others further
aduttitgg the majestle structure, creating
RelPr (Turn The image features large, majertic castle... The castle has purple and white color scheme, and appears be popular tourist attraction.... a few people can sect....
RePar (Turn The image festures lurg, majestic castle... The people the image likeely tourists, admiring the beauty this iconle structure, and the scene set a location that likely a popalar tourist destination. The image captures the esseuce of fairy tale. and the eastle is the ecnter of atttenttion making perfect picture for postcard or travel brochure.
Figure 4. Comparison of image attention maps between LLL 1.5 and RePer, highlighting RePer's broader activation of image tokens and its ability to generate more detailed and accurate answers. While LLaVA-1.5 over-focuses on "people", RePer correctly attends the main subject, "castle," progressively activating more relevant tokens for improved perception.
To investigate, we begin by visualizing the detailed image at- tention of the model for each round of the model's responses. As shown in Figure 4, the rounds progress, the model accurately shifts its attention emphasis towards the correct image tokens, rather than over-focusing on few insignif- icant ones. Hence correspondingly, the hallucinations and erroneous perceptions initially displayed by the model grad- ually decrease. Essentially, this represents a progressive alignment towards ground-truth human attention.
As aforementioned, to develop reflective perception capa- bilities, we create multi-turn data that progresses from poor to good responses, with rewards increasing linearly from rounds to T. As result, in the initial rounds, the model mainly penalizes poor samples (more unlikelihood), while in later rounds, it gradually shifts to rewarding good samples (more likelihood). This helps the model avoid overfitting to poor initial samples and, importantly, allows it to progres- sively learn to distinguish between good and bad samples.
Figure 6a provides further quantitative support, showing log-linear increase in average image token activations over five rounds of reflection. Specifically, we calculated the av- erage image token activations across 1,000 cases to measure how the model's attention to images varies during reflective perception. This is important because responses with fewer hallucinations are associated with higher average activations of image tokens (Huang al., 2024). Our findings suggest that visual reflection gradually unlocks the model's inher- ent visual capabilities, focusing attention salient image context and progressively mitigating hallucination.
From another perspective, we can view RPL as form of reward modeling. Unlike popular LLM-based reward mod- eling methods such as DPO (Rafailov et al., 2024) and LiPO (Liu et al., 2024e), RPL does not propagate gradients to the remaining negative samples. Yet, back-propagation over multi-round dialogues is actually not isolated. With each response contextualizing all previous responses, as denoted by o [,, ;1:t-1; f1tt1), each sample im- plicitly establishes partial increasing preference order.
Moreover, it is worth noting that RPL holds significant ad- vantage over previous reward modeling approaches: flexibil- ity in handling diverse preference samples-pairwiss or list- wise, scalar or fine-grained - feedbuck-basee maintaining stable training. Additionally, the use of detailed feedback aids error highlighting, facilitating object-level even token-level preference that direct optimization more precisely. Our analyses in Section 4.6 further confirms this.
3.2. RPL is a Free-Form Preference Optimization.
howards-whild
Revisiting the data construction in RPL, we essentially trans- form listwise preference data with precise feedback and scores into multi-turn dialogues grading from poor to good quality. This prompts the inquiry: is RPL fundamentally a preference optimization process?
4. Experiments
Revisiting Equation (2), for a given sample and its T dialogue iterations, the objective is articulated as follows:
4.1. Implemental Details
L' log (Ti + a(1 0.) log(1 No (5 (o1)) +
Datasets. To construct the training dataset as illustrated in Section 2.2, we begin by randomly sampling 10,000 images from the LLaVA-665K (Liu et al., 2024c) dataset. For each image, we prompt the model to generate 8 different
less likelthood
more unlikehibavel
or log no (urlor) + a(1 or) log(1 No (ūT oT))
more Mellihoo
less unlikelihoud
5

---

Perception in Reflection
Table Model Performance Comparison of RePer with Baselines and State-of-the-Art Models. RePer outperforms across six
benchmarks, with the best results highlighted in bold.
Model
MMHHallBeaa
HellusionBench fAce 1
Detallcaps-457
VABench
GAPE Conrect
Score ? Hal mee
aAce
gAcc
CAPTURE 1 Precision 1 Recall
Relevancy
Accunccy
Authen.
Total
MiniGPT-4 7B
35,78
10.12
8.79
45,1
MPLUG-OWL
9,45
InstmerBLIPP IP 7B
45.26
10.11
9.45
51.81
65.
45.01
59.8
LLaVA-SFTs 78
1.88
0.68
33.66
8.96
5.93
51.13
64.38
44,28
44.5
6.68
4,85
27.02
12.47
70,09
MaVA-RI 7R
31.33
7.69
52.21
63.61
45.93
44.9
4.85
27.91
12.64
70.68
VOLCANO 7B
2.06
0.62
26.50
10.69
6.37
30.88
66.23
13.35
$1.0
7.12
5.35
31.63
14.52
78.78
LLaVA- 13B
1.9
0.65
46.37
22.25
18.24
51.08
64.48
44,34
55.8
6.85
5.20
30.00
13.44
74,88
LLVA-RLHH 13R
14.75
56
45.35
62.6
4,66
30.6
75.36
VOLCANO 13B
2.15
0.64
40.69
19.36
13:40
51.21
66.47
43.65
66.0
7,55
5.59
31.34
14.32
78.17
LLaVA-155 78
2,02
0.61
35.66
17.92
11.21
51.03
67.27
42.19
60.2
6,50
5.28
30.19
13,58
75.16
+RePor
14.29
52.89
66.81
60.7
33.16
14.94
80.88
MaVA-15533 13B
0.58
43.85
20.81
14.95
51.23
43.77
66.05
6.65
31:27
14.12
77.37
+Releer
2.61
0.52
51.00
22.83
20.00
54.73
61.74
49.1
67.6
7.67
6,86
34.11
15.33
62.54
captions sampled with temperatures ranging from 0.0 to 1.4 in increments of 0.2. To filter high-quality samples, we retain instances from VLM-based scoring where the highest score exceeds 9 and the score disparity (difference between the highest and lowest scores) is greater than 4. Similarly, for rule-based scoring, we retain cases with a highest score above 0.55 and a score disparity exceeding 0.2. Using the generated captions, rewards, and templates from Figure 2, we create the visual reflection dataset, containing 11,065 samples from 8,101 images. These samples are distributed as follows: 3.649 for one conversation turn, 2,621 1 for two turns, and 3,795 for three turns.
LLaVA-RLHF (Sun et al., 2023a). 1L5 (Liu et al., 2024b) but also with Volcano (Lee al., 2023), a multi- modal model trained with self-feedback guided refinement.
As shown in Table 1, RePer consistently outperforms base- line models across benchmarks and model scales. Its notable improvement on DetailCaps (+3.64% in 7B and +6.83% in 13B) highlights its ability to generate more accurate and de- tailed captions through multi-turn refinement and RPL. The increased recall rate (+8.30% in 7B and +12.17% in 13B) for visual elements demonstrates RePer's enhanced pereep- tion of details. This results in consistent improvements on general and ballucination--elated benchmarks, reducing hallucinations without sacrificing image understanding.
Models Training and Inference. Our experiments are based on the VLaVA-115 (Liu et al., 2024b) architecture. We directly supervised finetune the instruct model on our generated datasets. All models are trained for one epoch on 8 NVIDIA A100 GPUs with batch size of 8 and learning rate of 1e-6. Only the parameters of the L.M module are fine-tuned, while the rest remain frozen. In reflective unlikelihood training (Equation (2)), rewards are normalized to [0.1] by dividing with their maximum values (F), serving as likelihood weight (a). The constant term set as 10.0. During the inference stage mentioned in Section 2.3, we defaultly use LLaVA-Critic (Xiong al.,, 2024) as the critic model.
4.3. GPT--o-Assisted Perception Evaluation (GAPE)
We introduce GPT-4o-Assisted Perception Evaluation (GAPE) to simulate human-like perception assessment. De- signed to complement traditional closed-set image caption- ing benchmarks (Chen et al., 2015; Agrawal al., 2019), GAPE evaluates model-generated captions by leveraging human- aaiaan prompts with GPT-4o (Peng et al., 2024) without the need for numan-annotater groundtruth answers. Specifically, given an image and prompt, GPT-4o evaluates the generated captions across five dimensions: Authenticity, Correctess, Detail, Coherence, and Completeness. The evaluation prompts align with the "Rating Criteria" outlined in Figure 8. To better highlight differences in caption qual- ity, these dimensions are scored on larger scale from 0 to 100, offering himan-like and nuanced assessment of caption performance.
4.2. Main Results
To evaluate the visual perception capabilities of RePer, we conducted assessments across five widely-used bench- marks, covering a range of tasks: image understanding (LLaVABench (Liu et al., 2024c)), hallucination detection (HallusionBench (Guan et al., 2024), MMHal-Bendl (Sun et al., 2023b), GAVIE (Liu et al., 2023a)), and detailed image captioning (DetailCaps (Dong et al., 2024a)). As shown in Table 1, we compared RePer not only with classic oate-off-hheart multimodal baselines including MiniGPT- 4 (Zhu et al., 2023), mPLUG-Owl (Ye et al., 2023), In- structBLIP (Dai et al., 2024), LLaVA (Liu et al., 2024c),
Results As shown in Table and Table our RePer con- sistently outperforms other methods, demonstrating its ef- fectiveness in enhancing model's perceptual capabilities. Notably, we observe the most significant improvement in Authenticity, which evaluates the model's tendency to hal- lucinate non existent objects. This substantial gain can be attributed to our unlikelihood training objective, which ef-
6

---

Perception in Reflection
application, using the CLIP-Image-Scone metric from Vi- sualFactChecker (Ge et al., 2024). This metric evaluates caption accuracy and detail by comparing the similarity between an original image and its text-to-image generated version (DALLE-3 (Betker et al., 2023)), using the caption as a prompt. By comparing the raw and reconstructed im- ages, the metric detects mallucination-relate discrepancies, providing unique perspective on caption quality. To en- hance this evaluation, substitute the CLIP model with DINO (Caron et al., 2021) and DINOv2 (Darcet et al., 2023) for a more thorough assessment.
Table Comparison of human preference metries and GAPE scores for 13B scale models.
Model
Mean Rank
Top-1
1.
2.46
15.13%
77.37
Volcano
2.01
35.29%
76.17
1.53
63.87%
82.54
Table Image captioning comparison on 13B models using the CLIP-Image-Score metric and its variants with DINO/DINOv2 as Image encoders.
Model
CLIP
DINO
DINOv2
1.5
67.43
40.56
41.02
As shown in Table 3, our RePer consistently outperforms the baselines, underscoring the superior quality of its cap- tions. Figure 5 presents visual examples of the reconstruc- tion process. In the second example, LLaVA 1.5 falsely mentions, "There are several birds scattered throughout the scene," exhibiting hallucination. In contrast, the caption from our RePer produces a reconstructed image closely re- sembling the original, demonstrating its superior accuracy and ability to avoid hallucinations.
+RePer
67.85
42.19
42.12
fectively penalizes misaligned visual descriptions.
Evaluation Reliability To assess the reliability of our au- tomatic evaluation, we conducted a human study comparing caption preferences across three 13B-scale models (L.aVV LLV- 1.5, Volcano, and RePer) on 119 samples from the GAPE benchmark. For each image, geeer--generated captions were anonymized, shuffled, and ranked by six expert annotators, following the same five rating criteria used in GAPE. We report two metrics: 1) Mean Rank the average posi- tion of each model across all rankings; and 2) Top-1 Rate the percentage of times a model's caption was ranked highest by humans. As shown in Table 2. the human prefer- ence rankings align closely with the GPT-4o-based GAPE scores, validating the reliability our automatic evaluation. Notably, RePer achieves the top rank in 63.87% of cases, demonstrating strong agreement with human judgment.
4.5. Ablation Studies
Reflection Turns We analyze the impact of reflection turns on model performance using LLaVA-Critic and GPT- 4o as the critie. As shown Figure 6b, increasing reflec- tion turns improves performance on the DetailCaps-4870 benchmark, reducing hallucinations and enhancing detail perception. This aligns with our attention analysis (Fig- ure 4), suggesting that iterative reflection helps the model better focus on relevant image regions.
Scoring Disparity for Data Construction We also exam- ine the effect of scoring thresholds in data selection (Sec- tion 2.2) on DetailCaps and HallusionBench. As shown in Figure 6c, optimal performance is achieved with samples having highest scores above 9 and score disparities of at least 4, indicating that high scoring disparity helps select challenging yet high-quality training samples.
4.4. Evaluation via Text-to-Imape Reconstruction
Original Image
RePer (Ours)
LLaVA-15
We further study the influence of un- likelihood loss weight a (from Equation (2)) on reducing behavior collapse in initial responses using DetailCaps and HallusionBench. As shown in Figure 6d, a weight of 10.0 achieves optimal performance by effectively balancing the penalization of undesirable responses while preserving valu- able content.
Unlikelihood Loss
Figure 5. We use DALLE-3 (Betker et al., 2023) as text-to-image model to reconstruct images using generated captions. Compared to the original image, reconstructed images from L.. (Liu et al.. 2024b) captions lack key objects or include extraneous ones, indicating incomplete descriptions or hallucinations.
4.6. Further Analysis
Critic matters, RPL matters more. To assess RPL and dif- ferent critics' impact on RePer, we compare its performance with and without RPL, using critics LLVVA-Criti (Xiong et al., 2024) and GPT-4o (GPT-4o, 2024), across multiple
We assess image captioning performance, a key perceptual
7

---

Perception in Reflection
L.5
CAPTURE Scorett)
5.25
15.4
5.00
15.2
Perfommna
35.0-
4.50
64.6
3775
46.4
(PT44
0.0
0.0
3.0
160
Rellection Turns
Rellection Turms
Scoring Disparity
LOGS Weight
(a) Attentive Img- Token Analysis
(b) Reflection Turns
(c) Scoring Disparity
(d) Loss Weight of Unlikelihood
Figure (a) Increase in activated average image attention across reflection turns. (b-d) Ablation studies.
counteract multimodal hallucinations.
Table 4. Comparison of RePer and RePer without RPL under vary-
ing critics and reflection turns Detailcaps-4870.
Critic
Torn
RePer
RePer w.. RPL
5. Related Work
1
54,29
51.22
GPT-4o (GPT-4o, 2024)
2
55.41
$2.28
The remarkable scaling laws (Kaplan et al., 2020) of LLMS (Touvron et al., 2023a; Xu et al., 2024) in terms of pa- rameters and data have driven the advancement of LVLMs. BLIP-2 (Li et al., 2023a) pioneered the use of Q-Former to bridge visual encoders with large language models, ex- plicitly supervising the vision-language alignment while autoregressively generating vision-related text. Works like LLaVA (Liu et al., 2024c:b), MiniGPT-4 (Zhu et al., 2023), and Qwen-VI. (Bai et al., 2023; Wang et al., 2024) have demonstrated the sufficiency of text autoregression for vi- sual understanding and have progressively simplified the vision languaee connector using techniques such as cross- attention (Ye et al., 2023), linear layers (Liu et al., 2024c; Zhao et al., 2023), MLPs (Liu et al., 2024b; Zhang et al., 2024; Dong al., 2024b), and convolutions (Yu et al., 2023; Wang et al., 2024), all while maintaining consistent perfor- mance.
55.55
53.9
54,29
51.22
LLaVA-Critic (Xiong ot al., 2024)
2
54,68
52.25
54.73
53.85
Table 5. RPL. vs. Preference Optimization Methods,
Method
DetailCaps
HallusionB
GAVIE
LLaVAB
LLaVA-1.5-13
51.22
24.43
5.65
66.95
+DPO (Rafnilov et al., 2024)
50.53
25.61
5.28
66.2
+LiPO (Liu et al., 2024e)
52.31
25.04
6.27
69,5
+RPL
54.73
31.28
6.86
67.6
reflection turns DetailCaps. In Table 4, GPT-4o yields perior results due to its strong generative and discriminative abilities, while LLa VLL-A-Crite also shows consistent improve- ments, indicating RePer's adaptability to different critics. Even without RPL, RePer benefits from reflection; how- ever, RPL further amplifies this effect, leading a stronger initial-turn performance and demonstrating the effectiveness of the imitation learning approach. For detailed view of RePer's iterative reflection process, refer to Figure 7.
Despite relentless scaling of visual encoders (Tong et al., 2024a; Wei et al., 2024a), language decoders (Wang et al., 2024), and visual-textual corpora (Li al., 2024; Wei et al., 2024b), LVLMs have yet achieve qualitative leap in per- ceptual acuity hallucination mitigation. Some approaches attribute hallucinations to visual (Tong al., 2024b) lin- guistic biases (Li et al., 2023b), seeking to counter them through online (Liu et al., 2023b) or offline (Leng et al., 2024) corrections. Others (Yu et al., 2024; Zhu et al., 2024; 2025) take a more direct route, modulating the model's visual attention preferences by aligning with human judg- ment via Reinforcement Learning from Human Feedback (RLHF) (Ouyang et al., 2022). Yet, disappointingly, these efforts have failed to tackle the root issue: models still re- flexively respond to perceptual challenges, regardless of their complexity.
RPL is essentially fine-grained preference optimization. As detailed in Section 3.2, RPL's imitation learning in reflec- tive dialogues can be seen as listwise preference optimiza- tion with detailed feedback and explicit rewards. We com- pare it to similar methods: DPO, which optimizes Bradley- Terry (Bradley& & Terry, 1952) using preference pairs with the largest score differences, and LiPO, which optimizes learning-to-ran (Liu et al., 2009) using all preference data ranked by reward. Specifically, for each multi-turn cor- rection trajectory ranked by reward, DPO selects only the highest- and lowest-scoring responses to form single pref- erence pair, while LiPO constructs all possible response pairs, treating the higher-scoring response in each as pre- ferred. Table 5 shows RPL's clear advantages, especially in caption and hallucination metrics. We speculate this success stems from: 1) fine-grain -reiggd critic feedback that fa- cilitates effective corrections, lacking in DPO/LiPO; and 2) unlikelihood training without KI. constraints, which helps
L.MSs often use step-by-step reasoning (Wei et al., 2022) to avoid giving premature answers. However, this linear process can falter with complex problems, leading to fac- tual inaccuracies and hallucinations (Miao et al., 2023).
8

---

Perception in Reflection
Numman:
trage
tingg Sumane car shtiing as vtndew el, with pesser hand nocking tovorde The appeore oxtice and liistiing prooeit hond, pordtly trying adcred cry pll valaud Theoeane is asanses therz dinins tublic and the bacipncall, howe spplic qudicl acnac, inumddes Bovons
jemarrancc aliands
Crfties
than. repeese prodidera desaled deceriptinn of eactiontag the can the yendee oi pwouprt roaching twordd the ou thetenee teterrtaee betwer skiih
shaap, chate bonk, lndkgrend, la Saauna is lodasy ond of vildio,, peoss's lead rundin lomaad and a of "Cluuic Waar dap valk: se the wasee all The wow at indog inseersiie
dast teacnantio
dees revetooned
Comeetly ileviifes ieeintevttion
axdapple
Laddsreniileeer the ofgof dfigs.
Rather:
tmage fhotereee car dhttigg yn wisdoor ynd boking wet pasrr' hasd morhingge rwarts a. The svee lose indorr eesiroe or and "Cuure abding
lnkeathin, souse lads salisa pluce is bass orea rederr whia The penne's a lend rudics out lowaed the oá, and the as apaams cutour and inkooted in the pence a laae, feswen
Mikia roaible
have
boy Toek cvendyy bis wher the expiiing the view and the proce itrinng
the cwriing varn onlese arresphen
Cettic
land moching eut rilcadd the net and "aaece Milie cope bag The cioopptio oinn insouaiio beown can and phopeno eugheiig fledyy ayy playt
iroge
deaapphess attos acumaiii douufaed
The rapoae also aiuliggte sciee cisn and inkenetis bdlvaon the on and the paice, orglaalde the wareth and oniaos of the sxeace, Overal the comerett sesdoo whho wwiing whw.hggghhggggggg
neted muse
deorttatne kithoregh vod pirnic dear girme coeee relling vary
dleneets fot, window pessod honneaaeenettilleee Cummerly deve hes the kecrw tio and dhregghee
"Te lecvlytie deceeaeed
of
ropalit nesgedionins soxadeel
Mather
image soorea ditteg en wildon 4lll with prnse' thoa mol lin de Tht lioe way inkor endioreea and bag Way' dips vilile the will son and strangheee to
lis the were plas ls hoze or tsdou wittrn, The tensel a bund oadiss leoand the cat, and the aroam cutina and antrrtie in the penss hard masrre soms liiry
eniille sen to freselly
eveynay Bi, ded epwinng the vnn the penoe trving dileeaeee fhe on, cpstings wer memorllles sree clane the in Leuatid repronaate the bes) and tiae
invraction
whene the
mallagpen
the they fo Iose.
The LaGt poparae provila catulled and viri dlompttto of the seas, outtaingg the canne, of setac and las anteedise latwoss the out and the paous acnaadd ohe
-tietedy roways pentent candie wads suinaating vaze didd Aeaweei
whar" the baod betowe and Foooeer deortpiin
placti mnvegg
ruperas deaded and cxptance oones the trags bochadie inkoratias betweu the an and parus the backgroend dhamara like the vindew theckge bag.
"The leurlytic offe thal wooespthe way and platt attoeat for coul dightly recpy withor bookg ingastor deaa..
of paining dear cfaeseeao
Figure 7. Multi-turn Refinement via Critic Feedback. Initially, RePer generates hallucinated objects but corrects them in the second turn based on critic feedback. By the third turn, it enhances the description with more details, improving a response that already scored 90. The increasing feedback scores highlight RePer's iterative refinement and progressive improvement.
To counter this, some approaches use external feedback to guide reasoning (Shinn et al., 2024; Yao et al., 2022), while others harness the model's reflective abilities for self- correction (Liu al., 2024a; Miao et al., 2023; Qu et al., 2024; Kumar et al., 2024). These methods employ itera- tive "answer-reflect-reansww performance on complex challenges.
or external (Liao et al., 2024) rewards. Despite promising re- sults, these approaches lack systematic training frameworks and do not sufficiently explore the underlying principles of their mechanisms. We address these limitations by propos- ing RePer and RPL, with comprehensive theoretical and empirical analysis.
loop, significantly improving
Some LVLMs require preliminary image parsing tasks like grounding (Chen et al., 2023; Shao et al., 2024), parsing (Liu al., 2024d; Wei et al., 2024c; Chen al., 2024), or iden- tification (Yu al., 2023; 2025) before responding. While approach moderately improves
6. Conclusion
Perception in reflection addresses a key limitation in cur- rent LVLMs: the unrealistic expectation of perfect initial responses. Instead, it provides robust fallback mechanism, empowering the model to adjust and converge on the correct answer even when initial predictions fall short. Powered by reflective perceptual learning, we create a system that can generalize more effectively across varied and complex visual scenarios, ensuring that the model is not only accurate but also resilient and adaptive in weal-world applications.
this teaain-o--thoughtttt performance, other methods (Cao et al., 2024; Wu & Xie, 2024) focus on locating relevant image regions and crop- ping them to assist with fine-grained perception. However, these methods often struggle with complex scenarios and may increase hallucination. Recent work explores iterative refinement using internal (Liu et al., 2024a; Lee al., 2023)
9

---

Perception in Reflection
Impact Statement
Chen, L., J., Dong, X., Zhang, P., He, C., Wang. J., Zhao, F., and Lin, D. Sharegpt4v: Improving large molti-modal models with better captions. In European Conference on Computer Vision, pp. 370-387. Springer, 2025.
This paper proposes a model that leverages reflective per- ception and structured preferences to improve image under- standing, captioning precision, and reduce hallucinations in Al- Al-generated text-a known concern in multimodal systems. As the dataset and training are based model-generated outputs and curated feedback, we believe the work poses minimal immediate societal risk. Nonetheless, responsible deployment should account for faimess,, bias, and potential misuse.
Chen, X., Fang, H., Lin, T.-Y., Vedantam, R., Gupta, S., Dollár, P., and Zitnick, C. L. Microsoft coco captions: Data collection and evaluation server. arXiv preprint arXiv:1504. 500.00025, 2015.
Dai, W., Li, J., Li, D., Tiong, A. M. H., Zhao, J., Wang, W., Li, B., Fung, N., and Hoi, S. Instructblip: Towards general-purpost vision-language models with instruction tuning. Advances in Neural Information Processing Sys- tems, 36, 2024.
References
Agrawal, H., Desai, K., Wang, Y., Chen, X., Jain, R., John- son, M., Batra, D., Parikh, D., Lee, S., and Anderson, Nocaps: Novel object captioning at scale. In Proceedings of the IEEE/CVF international conference computer 2019.
Darcet, T., Oquab, M., Mairal, J., and Bojanowski, P. Vision transformers need registers, 2023.
vision, pp.
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. Bert: Pre-training of deep bidirectional transformers for lan- guage understanding. arXiv preprint arXiv: 1810.04880 2018.
Bai, J., Bai, S., Yang, S., Wang. S., Tan, S., Wang, P., Lin, J., Zhou, C., and Zhou, Qwen-vl: A frontier large vision- language model with versatile abilities. arXiv preprint arXiv: 12966, 2023.
Dong, H., Li, Wu, B., Wang. J., Zhang, Y., and Guo, H. Benchmarking and improving detail image caption. arXiv preprint 2tti::2200.199999 2024a.
Betker, J., Goh, G., Jing, L., Brooks, T., Wang, J., Li, L., Ouyang, L., Zhuang. J., Lee, J., Guo, Y., et al. Improving image generation with better captions. Computer Sci- ence. https://cdn. openai. com/paperstdallle-.. pdf, 2(3):
Dong, R., Han, C., Peng, Y., Qi, Z., Ge, Z., Yang, J., Zhao, L., Sun, J., Zhou, H., Wei, H., Kong, X., Zhang, X., Ma, K., and Yi, L. DreamLLM: Synergistic multimodal comprehension and creation. In The Twelfth International Conference on Learning Representations, 2024b.
8, 2023.
Bradley, R. A. and Terry M. E. Rank analysis of incomplete block designs: the method of paired comparisons. Biometrika, 39:324, 1952. URL https: //api. www.eetticccooooioooo. wwwporrr///ooorr// 125209808.
Ge, Y., Zeng, X., Huffman, J. S., Lin, T.-Y,, Liu, M.- Y.. and Cui, Y. Visual fact checker: Enabling high- fidelity detailed caption generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14033-14042, 2024.
Cao, Y., Zhang, P., Dong, X., Lin, D., and Wang, J. Du- alfocus: Integrating macro and micro perspectives in multi-modal large language models. arXiv preprint arXiv:2402. 147777 2024.
GPT-4o. Hello gpt-4o, 2024. URL https: //opeai. com/ ie/iinnexx///
Guan, T., Liu, F., Wu, X., Xian, R., Li, Z., Liu, X., Wang, X., Chen, L., Huang, F., Yacoob, Y., et al. Hallusionbench: an advanced diagnostic suite for entangled language hal- lucination and visual illusion in large vision-language models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 14375- 14385, 2024.
Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., and Joulin, A. Emerging properties in self-supervised vision transformers. In Proceedings of the International Conference on Computer Vision (ICCV), 2021.
Chen, J., Kong, L., Wei, H., Liu, C., Ge. Z., Zhao, L.., Sun, J., Han, C., and Zhang, X. Onechart: Purify the chart structural extraction via one auxiliary token. In Proceedings of the 32nd ACM International Conference Multimedia, pp. 147-155, 2024.
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learn- ing for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Chen, K., Zhang, Z., Zeng, W., Zhang, R., Zhu, F., and Zhao, R. Shikra: Unleashing multimodal 11m's referential dialogue magic. arXiv preprint arXiv:2306. 15155, 2023.
He, K., Gkioxari, G., Dollár, P., and Girshick, R. Mask r- cnn. In Proceedings of the IEEE international conference on computer vision, pp. 19699, 2017.
10

---

Perception in Reflection
Huang, Q., Dong, X., Zhang, P.. Wang, B., He, C., Wang. J., Lin, D., Zhang, W., and Yu, N. Opera: Alleviat- ing hallucination in multi-modal large language models In
Liu, F., Lin, K., Li, L.,, Wang, J., Yacoob, Y., and Wang, L. Aligning large multi-modal model with robust instruction tuning. arXiv preprint arXiv: 14565, 2023a.
via over-trust penalty and vetrospection-allocaiii Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 13418-13427, 2024.
Liu, F., Lin, K., Li, L., Wang, J., Yacoob, Y., and Wang, Mitigating hallucination in large multi multi-modal -modal models via robust instruction tuning. In The Twelfth International Conference on Learning Representations, 2023b.
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D. Scaling laws for neural language models. arXiv preprint arXiv:2001.0888 2020.
Liu, G., Mao, H., Cao, B., Xue, Z., Johnson, K., Tang. J., and Wang, R. On the intrinsic self-correction capability of Ilms: Uncertainty and latent concept. arXiv preprint arXiv:2446.0024 02378, 2024a.
Kim, M. J., Pertsch, K., Karamcheti, S., Xiao, T., Balakr- ishna, A., Nair, S., Rafailov, R., Foster, E., Lam, G., San- keti, P., et Openvla: An open-source vision-language- action model. arXiv preprint wrXiv:2406.09244 2024.
Liu, H., Li, C.,, Li, Y., and Lee, Y. Improved baselines with visual instruction tuning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 86296-26306, 2024b.
Kumar, A., Zhuang, V., Agarwal, R., Su, Y., Co-Reyes, J. D., Singh, A., Baumli, K., Iqbal, S., Bishop, C., Roelofs, R., et al. Training language models to colrecorrect via reinforcement learning. arXiv preprint arXiv:2409.12917, 2024.
Liu, H., Li, C., Wu, Q. and Lee, Y.J. Visual instruction tun- ing. Advances in neural information processing systems, 36, 2024c.
Liu, M., Chen, D., Li, Y., Fang, G., and Shen, Y. Chartthinker: A contextual chain-of-thought approach to optimized chart summarization. arXiv preprint arXiv:2403.1 //36,, 2024d.
Lee, S., Park, S. H., Jo, Y., and Seo, M. Volcano: mitigating multimodal hallucination through self- fef-beee guided revision. arXiv preprint (tt.::2311.0736666 2023.
Leng, S., Zhang, H., Chen, G., Li, X., Lu, S., Miao, C., and Bing, L. Mitigating object hallucinations large vision- language models through visual contrastive decoding. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 13872-13882, 2024.
Liu, T., Qin, Z., Wu, J., Shen, J., Khalman, M., Joshi, R., Zhao, Y., Saleh, M., Baumgartner, S., Liu, J., al. Lipo: Listwise preference optimization through learning- to-rank. arXiv preprint arXiv:2402.01877 2024e.
Liu, T.-Y. et al. Learning to rank for information retrieval. Foundations and Trends@ in Information Retrieval, 3()): 225-331, 2009.
Ji,J., Li, D., Savarese, S., and Hoi, S. 2 Bootstrapping language-iman image pre-training with frozen image encoders and large language models. In International conference on machine learning, pp. 19730-19742. PMLR, 2023a.
Malkiński, M. and Mandziuk, J. Deep learning methods for abstract visual reasoning: A survey on raven's 's progressive matrices. ACM Computing Surveys, 2022.
Li, Q. Chen, Z., Wang, W., Wang, W., Ye, S., Jin, Z., Chen, G., He, Y., Gao, Z., Cui, E., et al. Omnicorpus: An unified multimodal corpus of 10 billion-level images interleaved with text. arXiv preprint arXiv:2406.0844
Miao, N., Teh, Y. W., and Rainforth, T. Selfeheck: Using Ilms to zero-shot check their own step-by-ster reasoning. arXiv preprint arXiv:2308 00436, 2023.
2024.
Li, Y., Du, Y., Zhou, K., Wang, J., Zhao, W. X., and Wen, J.R. Evaluating object hallucination in large vision- language models. arXiv preprint 23Xiv:2305.100 10355, 2023b.
Miller, G. A. Wordnet: a lexical database for english. Com- munications of the ACM, 38(1 38(11):39 1):39-41, 1995.
Mu, T., Helyar, A., Heidecke. J., Achiam. J., Vallone, A., Kivlichan, I., Lin, M., Beutel, A., Schulman, J., and Weng, L. Rule based rewards for language model safety. 2024.
Li, Z., Chai, Y., Zhuo, T. Y., Qu, L., Haffari, G., Li, F,, Ji, D., and Tran, Q. H. Factual: A benchmark for faithful and consistent textual scene graph parsing. arXiv preprint 2rXiv:2305.174499 2023c.
arXiv preprint arXiv:2411.011
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730-2777 27744, 2022.
Liao, Y.-H., Mahmood, R., Fidler, S., and Acuna, D. Can feedback enhance semantic grounding in large vision- language models? arXiv preprint arXiv:2404.06510, 2024.

---

Perception in Reflection
Peng, Y., Cui, Y., Tang, H., Qi. Z.,, Dong, R., Bai, J., Han, C., Ge, Z.,, Zhang, X., and Xia, S.-T. Dreambench++: A human-aligned benchmark for personalized image gener- ation. arXiv preprint S..iv:2406.668555 2024.
Tang, Y., Guo, D. Z., Zheng, Z.. Calandriello, D., Cao, Y., Tarassov, E., Munos, R., Pires, B, A., Valko, M., Cheng, Y., et al. Understanding the performance gap between online and offline alignment algorithms. arXiv preprint 2024.
arXiv:2405.cm
Qu Y. Zhang, T., Garg, N., and Kumar, A. Recursive introspection: Teaching language model agents how to self-improve. arXiv preprint urXiv:2407.18219 2024.
Tong, S., Brown. E... Wu, P., Woo, S., Middepogu, M., Akula, S. C., Yang, J., Yang, S., Iyer, A., Pan, X., et al. Cambrian-1: A fully open, vision-centric exploration of multimodal llms. arXiv preprint 24Xi:2440611660) 2024a.
Rafailov, R., Sharma, A., Mitchell, E., Manning, C. Er- mon, S., and Finn, C. Direct preference optimization: Your language model is secretly a reward model. Ad- vances in Neural Information Processing Systems, 36, 2024.
Tong, S., Liu, Z., Zhai, Y.. Ma, Y., LeCun, Y.. and Xie, S. Eyes wide shut? exploring the visual shortcomings of multimodal 1lms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9568-9578, 2024b.
Ren, S., He, K., Girshick, R., and Sun, J. Faster r-cnn: Towards real-time object detection with region proposal networks. IEEE transactions on pattern analysis and machine intelligence, 39(6): 137-1149, 2016.
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F,, et al. Llama: Open and efficient foundation lan- guage models. arXiv preprint arXiv:2302
Ross, S., Gordon, G., and Bagnell, D. A reduction of imita- tion learning and structured prediction to no-regret online learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pp. 627-635. JMLR Workshop and Conference Proceedings, 2011.
2023a.
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y.. Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine- tuned chat models. arXiv preprint arXiv:2307.09228, 2023b.
Sampat, S. K., Patel, M., Das, S., Yang, Y., and Baral, C. Reasoning about actions over visual and linguistic modalities: A survey. arXiv preprint 220.:22000005655 2022.
Wang, P., Bai, S., Tan, S., Wang. S., Fan, Z., Bai, J., Chen, K., Liu, X., Wang, J., Ge, W., et al. Qwen2-vl: vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint 2rX9v:
Shao, H., Qian, S., Xiao, H., Song, G., Zong, Z., Wang, L., Liu, Y., and Li, H. Visual cot: Unleashing chain- of-thought reasoning in multi-modal language models. arXiv preprint ar arXiv:2433.16990 2024.
2024.
Wei, H., Kong, L., Chen, J., Zhao, L.,, Ge, Z., Yang, J., Sun, J., Han, C., and Zhang, X. Vary: Scaling up the vision vocabulary for large vision-language model. In European Conference on Computer Vision, pp. 408-424. Springer, 2024a.
Shinn, N., Cassano, F., Gopinath, A., Narasimhan, K., and Yao, S. Reflexion: Language agents with verbal rein- forcement learning. Advances in Neural Information Processing Systems, 36, 2024.
Wei, H., Liu, C., Chen, J., Wang. J., Kong, L., Xu, Y., Ge, Z., Zhao, L., Sun, J., Peng, Y., et al. General ocr theory: Towards ocr-2.0 via a unified end-to-end model. 2024b.
Sun, Z., Shen, S., Cao. S., Liu, H., Li, C., Shen, Y., Gan, C., Gui, L.-Y., Wang, Y--.,, Yang, Y., Keutzer, K., and Dar- rell, T. Aligning large multimodal models with factually augmented rlhf. 2023a.
Wei, H., Yin, Y., Li, Y., Wang. J., Zhao, L., Sun, J., Ge, Z., and Zhang, X. Slow perception: Let's perceive geometric figures step-by-step. arXiv preprint arXiv:2412.206311, 2024c.
Sun, Z., Shen, S., Cao, S. Liu, H., Li, C., Shen, Y., Gan, C., Gui, L.-Y., Wang, Y.-X., Yang, Y., Keutzer, K., and Darrell, T. Aligning large multimodal models with factually augmented rlhf, 2023b. URL https: //arxiv. org/ abs/2309 14525.
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V. Zhou, D., et al. Chain-of-though prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24844-24837, 2022.
Swamy, G., Wu, D., Choudhury, S., Bagnell, D., and Wu, S. Inverse reinforcement learning without reinforcement learning. In International Conference on Machine Learn- ing, pp. 33229-333118 PMLR, 2023.
Welleck, S., Kulikov, I., Roller, S., Dinan, E., Cho, K., and Weston, J. Neural text generation with unlikelihood training. arXiv preprint arxiv: 04319, 2019.
12

---

Perception in Reflection
Wu, P. and Xie, S. V?: Guided visual search core mechanism in multimodal llms. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern 2024.
Zhu, D., Chen, J., Shen, X., Li, X., and Elhoseiny, M. Minigpt-4: Enhancing vision-language understanding with advanced large language models, 2023.
Recognition, pp.
Zhu, K., Zhao, L., Ge, Z., and Zhang, X. Self-supervised visual preference alignment. In Proceedings of the 32nd ACM International Conference Multimedia, pp. 291- 300, 2024.
Xie, J., Chen, Z., Zhang, R., Wan, X., and Li, G. Large multimodal agents: A survey. arXiv preprint irXiv:2402.55166 2024.
Xiong, T., Wang. X., Guo. D., Ye, Q. Fan, H., Gu, Q.. Huang, H., and Li, C. Llava-critic: Learning to evaluate multimodal models. arXiv preprint arXiv:2410.02712, 2024.
7hu, Z., Zhao, I.., Lin, K., Yang, J., Yu, E.,, Liu, C., Wei, H., Sun, Ge. Z., and Zhang, X. Perpo: Perceptual pref- erence optimization via discriminative rewarding. arXiv preprint arXiv:2502.0 2025.
Xu, H., Zhao, R., Wang, J., and Chen, H. Restful-llama: Connecting queries restful apis. In Proceedings of the 2024 Conference Empirical Methods in Natural Language Processing: Industry Track, pp. 1433, 2024.
Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., and Cao, Y. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:22/0.03629, 2022.
Ye, Q. Xu, H., Xu, G., Ye, J., Yan, M., Zhou, Y., Wang. J., Hu, A., Shi, P., Shi, Y., et al. mplug-owl: Modulariza- tion empowers large language models with multimodality. arXiv preprint arXiv:2304. 14178, 2023.
Yu, E., Zhao, L., Wei, Y., Yang, J., Wu, D., Kong, L., Wei, H., Wang, T., Ge, Z., Zhang, X., et al. Merlin: Empowering multimodal Iims with foresight minds. arXiv preprint ar Xiv::23/2.00586 2023.
Yu, E., Lin, K., Zhao, L., Wei, Y., Zhu, Z., Wei, H., Sun, J., Ge, Z.,, Zhang, X., Wang, J., al. Unhackable tempo- ral rewarding for scalable video milms. arXiv preprint arXiv:2502. 12081, 2025.
Yu, T., Yao, Y., Zhang, H., He, T., Han, Y., Cui, G., Hu, J., Liu, Z., Zheng, H.-T., Sun, M., et al. Rlhf-v: To- wards trustworthy milms via behavior alignment from fine-grat fine-graieed correctional human feedback. In Proceed- ings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pp. 13807-13816, 2024.
Zhang, Y., Li, B., Liu, h. Lee, Y. j.. Gui, L., Fu, D., Feng, J., Liu, Z. and Li, C. Llava-next: A strong zero-shot video understanding model, April 2024. URL -ett--0-vaaa-ett-----e -eddee/ee/. blog/200 200gg/224--0 00044-4433--220
Zhao, L., Yu, E., Ge, Z., Yang. J., Wei, H., Zhou, H., Sun, J., Peng, Y., Dong, R., Han, C., et al. Chatspot: Bootstrap- ping multimodal ilms via precise referring instruction tuning. arXiv preprint arXiv:2307.09474 2023.
13

---

Perception in Reflection
Appendix
In this appendix, we provide additional details to complement the main paper. Specifically, Appendix elaborates on the Visual Reflection Dataset, while Appendix B further validates RePer's alignment with human visual focus. Appendix presents details of the proposed GAPE benchmark along with its complete results, Finally, Appendix D showcases additional examples illustrating the strong capabilities of RePer.
A. Construction Details of Visual Reflection Dataset
This section provides additional details on the data construction process introduced Section 2.2 and Section 4.1.
A.1. Step-1: Initial Candidate Generation
To generate diverse responses, we employ temperature sampling, producing eight candidate captions per image across different temperature values, ranging from 0.0 to 1.4 in increments of 0.2. Higher temperatures generally lead to lower response quality, often introducing hallucinated objects or less precise descriptions.
A.2. Step-2: VLM-Based Reward Scoring
We define evaluation criteria for high-quality image captions, which guide the reward scoring process through carefully designed prompts (as shown in Figure 9). The reward score ranges from 0 to and assesses five key aspects:
Authenticity: Whether the caption contains hallucinated objects.
Correctness: Whether all described attributes and relationships are factually correct.
Detailness: Whether the description is sufficiently detailed, covering all relevant attributes of objects.
Coherence: Whether the caption logically consistent, without contradictions.
Completeness: Whether the caption comprehensively covers all relevant aspects of the image, including both foreground and background elements.
A.3. Step-3:
Kule-Based Reward Scoring.
Inspired by (Dong et al., 2024a), we design rule-based rewards to quantify the alignment between image elements and textual descriptions. This evaluates visual preference through structured pipeline:
Reference Caption Generation We prompt strong VLMs (GPT-4o and Gemini-Pro) using "Please describe this image in detail." to generate reference captions for each image.
Element Extraction We extract objects, attributes, and relations from both reference captions and candidate answers using Factual Parser (Li et al., 2023c), while applying stop-word filtering to remove irrelevant terms. To filter irrelevant elements, a stop word list is curated for abstract nouns (e.g., "foreground", "background") that do not correspond to image content. 1aaMAA--1B-chee (Touvron al., 2023b) and Factual Parser are used to extract candidate nouns from ShareGPT4V- (Chen et al., 2025). Words recalled by Factual Parser but missing in LLaMA2-133 1-AA--11--char reviewed, and high-frreqency terms are validated by human experts. This process results in the final stop word list.
Elements Matching We implement three-stage matching strategy to evaluate visual elements:
Exact Matching: Directly aligns identical objects, attributes, and relations.
Synonym Matching: Uses WordNet (Miller, 1995) identify synonym sets and assigns a 1.0 match score for synonymous elements.
Applies BERT (Devlin et al., 2018) to compute cosine similarity between embeddings of unmatched elements, selecting the highest similarity score per element.
Soft Matching:
14

---

Perception in Reflection
Rating Criteria
As senior computer vision and NLP researcher, and an impartial data quality assessor, please evaluate the quality of the generated caption relation to the aspect has an upper bound
given image on
as follows:
Authenticity (4 points): answer correctly identify all the objects present in the image?
describe object aspect
the image.
Correctness points): Does the
3. Detail (2 points): How well does the answer describe each object and its attributes?
Coherence (1 points); Is the answer logically consistent and understandable, maintaining steady flow throughout?
Completeness (1 points): Does answer cover aspects image?
Give lower points if the answer falls of these aspects, score of 10 indicates that the generated answer accurate, detailed, coherent, complete and does not hallucinate any details not present in the image. The final score for the answer is the sumn of scores in each aspect. Please also offer your reason for this score, and output the dict as the answer.
Input Image
Strong LLMs
Generated Answers
The scene takes place In someone's garden, featuring small statue of crunchy red monster with purple standing outside. The monster's hat
GPT-4o
Gemini
set of ornament. It uniquely appears play musical role, holding not one but two guitars. One guitar can be seen hanging in front of the statue an lap
eyes it garden
Scores and Reasons
hand, while
"Authenticity": Correctness":
side. The overall arrangement suggests the monster might be part of whimsical and artistic garden setup with possibly some entertainment attached.
picked up
"Detail":
"Coherence":1
"Completeness": 1,
"Final Score":
"Reason": 'The monster halds only one guitar, not two. grips drum with hand."
Figure Pipeline for Generating VLM-Based Reward Scores. Given image a generated answer, a predefined rating criterion for high-quality captions is formulated prompt. A strong I.I.M then evaluates the response, producing reward score along with rationale for the assigned score.
Final Matching Score recall, and F1-score. The final caption quality score is weighted sum of the three F1 scores, with default weights of Object.AtrriuteeRelaaio 5:2:2.
The final Rule-based reward aggregates scores from all three stages to compute precision,
A.4. Step-4: Reflective Dialogue Construction
To construct reflective dialogues, we first apply data filtering based on the criteria outlined in Section 2.2 (Step-4) and Section 4.1 (Datasets). We then organize responses into a structured reflection-driven dialogue format, using the template illustrated in Figure 8.
For VLae- rewards, given an image, its candidate answers, and scores with reasons from GPT-4o, we sort candidate answers from lowest to highest score. The multi-turn dialogue constructed follows:
In the first turn, the model is given the initial question.
In subsequent turns, each reflective prompt includes the score, explanation, and question asking the model to improve its response.
We construct dialogues of 1-3 turns, ensuring that the final ground-truth -tuth answer is always the highest-scoring candidate.
For multi-turn cases, the first-turn response is always the lowest-scoring candidate, enabling progressive refinement process.
This answer-critic iterative refinement encourages the model to learn from mistakes, gradually correcting its responses over multiple turns. For the rule-based rewards, there is no reasoning provided, and the scoring criteria differ from those of VLM-based rewards, which leads to different interpretation the relative score differences. Therefore, each round's prompt selected from predefined prompt pool that expresses the meaning of "could you generate a better answer."
15

---

Perception in Reflection
Quesion: Please generate detailed caption that aotamtelly deseribes the image-
vase collectible sitting high other, Additionally, a collectible Popsé- Co bottle with yellow back stand and watchel is displey at ground level vntaThe image shuses collection of actique or valuable lepi--ollectibles arranged, which migit bekong to an anid collectur of Pepei Co. memorabiiis, empbasizing the historical value distinet features of the antiuues.
Rerpors:: image, can see an Popel
objects.
PEPSI-COLA COLLECTIBLES
bottle Rational: This caption only the The description hallocinates objects that don't cúit in the such illustrated collectible Pepsi bettes
Criti
arangement multipk
book "Papel-Cok Collectiles, showing image with collectbles
cover. test
Could you generate
Reguoert The image
book V"Popai-Oslo Callectiblst" Michael Hunt, which for
Papai-Cokk of
various sources, book items inleehhe froat cover the book features illustraion of Ptpei- Cola displap cabinet, which reflects the nustalgic atmoxpbere the collection The book likely filldd with faseinating illustations and infocmation about vintage Pepsi- Uola items, making it escliting read for collectors and mithusiaets of Polai-Cola Cola menomailiin.
providing information
history
Critie Soow and Rationale: This paption qualifirs for seore 9. generated answer accurately deseribos the title general subject of the book. Ilowever, the authorship attribution is partially incoreett, as the image shuws Tel Vehling and Miihhre Huutt. The description of the book's content and cover illustration appropriste but looks detailed specifies. Please suggest on improved one.
Respose:: image features
Vebling Michael white hudgrronnd with cover prominently visible. The book appears to be guide or catalog, for collectors Pepsi-Cola memorubilia, showrasing varsous items and their values.
("Pepsi Cullectibles""
displayed on
Figure 9. Illustration of data construction based on the designed template. The responses are structured progressively from the lowest to the highest score. Each reflective prompt consists of three components: reward score, rationale for the score, and follow-up question requesting an improved response. This iterative refinement process guides the model towards generating more accurate and detailed captions.
Table 6. Comparison of RePer's Performance with Baselines and of Ahe-Art Models on the GAPE Benchmark.
Model
GAPE
Authenticity
Correctness
Detail f
Coherence
Completeness
Total
LLaVA-SFI
27.62
12.47
12.27
9.61
8.11
70.09
LLaVA-RLHF
27.93
12.44
9.55
70.68
LLaVA-SFT+
30.00
13.44
13.09
9.76
8.58
74.88
LLaVA-RLHF 13B
30.06
13.59
13.39
9.71
8.61
75.36
VOLCANO 13B
31.34
14.32
13.76
9.85
8.9
78.17
LLaVA-1.5
30.19
13.15
9.78
8.46
75.16
+RePer
33.16
14.95
13.95
9.87
8.96
80.88
LLaVA-1.5 13B
31.27
14.12
13.48
8.69
77.37
+RePer
34.11
15.33
14.26
9.70
9.15
82.54
B. Alignment with Human Focus
To further support our claim in Section 3.1 that RePer's image attention better aligns with human visual focus, we conducted targeted human evaluation. We recruited six annotators (PhD/Master's students in computer vision and NLP) and randomly sampled 100 images from the test set. For each image, collected image attention maps generated by RePer and LLaVA-1.5 during inference. These maps were anonymized and randomly shuffled before being shown annotators, who were asked: "Which attention map better reflects your own visual focus if you were to answer this question?" The annotation interface is shown in Figure 10. We report the win rate, defined as the percentage of cases where RePer's image attention map was preferred over LLaVA-1.5's. RePer was favored in 70.27% of cases, indicating stronger alignment with human visual perception.
C. Details of GAPE
As introduced in Section 4.3, GAPE employs evaluation prompts aligned with the "Rating Criteria" outlined in Figure 8. The evaluation score ranges from 0 to 100, with the following weight distribution: Authenticity (40 points), Correctness (20 points), Detail (20 points), Coherence (10 points), and Completeness (10 points).
GAPE serves as complement to traditional image captioning benchmarks. Existing benchmarks, such as COCO Cap-
16

---

Perception in Reflection
Human Align
Image: test_ 02047
Merson NCC
Question: describe Image detall.
map your focus?
Figure Human annotation interface for assessing attention map alignment with human focus. For each image, annotators selected the attention map that best matched their own viewing behavior.
tion (Chen et al., 2015) and NoCaps (Agrawal et al., 2019), rely on language metrics like BLEU, ROUGE, and CIDEr, which are constrained by closed-set vocabularies and require human-annotated ground truth captions, making them less scalable for evaluating vision- Laiiuag models (VLMs) that generate diverse descriptions. contrast, GAPE provides a more flexible and robust evaluation pipeline, operating without uaman--nnotated ground truth and leveraging LLM-based assessment to better align with human judgment, while accommodating the open-ended nature of caption generation.
Table 6 presents detailed comparison of model performance on GAPE across all evaluation aspects.
Figure 11 shows the interface of human annotation mentioned in Section 4.3.
D. Case Study
Figure 12 illustrates failure case of the baseline model LLa (Liu et al., 2024b) in multi-turn conversation with critic model (LLaVA-Crité (Xiong et al., 2024)). In this setup, LLaVA-1.5 first generates an initial response and then attempts to refine its answer based on feedback from the critic. Despite receiving detailed and accurate feedback, the model fails to correct its mistakes in the second turn, continuing to include hallucinated objects (highlighted in red) from the initial response. Furthermore, in the third turn, it introduces additional hallucinated objects, demonstrating its inability to properly leverage feedback for refinement. This failure also results in stagmant or even lower scores assigned by the critic. This underscores the necessity of equipping models with stronger feedback- driven self-correction mechanisms to improve response accuracy and mitigate hallucinations.
Figure 7 illustrates successful refinement of RePer. While the initial response contains hallucinated objects, RePer corrects these errors in the second turn based on feedback from the critic model. By the third turn, it further enhances the description by adding more details to an already accurate response, which had received a score of 90 from the critic. Throughout the multi-turn conversation, the feedback score steadily increases, reflecting the model's 's progressive improvement. This demonstrates RePer's strong ability to iteratively refine its answers based on prior responses and feedback, effectively guiding itself toward more detailed and accurate final answer.
Figure 13 compares the initial-round outputs of the baseline model and RePer, highlighting RePer's strong ability to generate accurate and detailed descriptions. While the baseline model frequently introduces hallucinated objects, RePer consistently produces more faithful image descriptions. Additionally, we present images generated by DALLE-3 (Betker et al., 2023) as part of the evaluation process in Section 4.4. The higher-quality captions from RePer lead to synthesized images that more closely resemble the original inputs, further demonstrating its effectiveness as a good captioner.
Figure 14 presents question-anseen answering cases that assess the model's 's image understanding capabilities. The responses are generated in single-turn answer format without iterative refinement.
17

---

Perception in Reflection
EXIT
Annotator Setttngs
Captions (Anorymous Models)
the Fac men unic meve Jccestt memer
sadge satica you
uic undin
mott uue coneenotee coreet diverie shght, becoe racoen seeaee cnee monion mose moveiig oppans
cose quctty coses indtuntinn suriigg
vad Insed nes selesiyy
Figure 11. Human annotation interface for caption quality assessment. Annotators ranked anonymized captions from three models per image using five quality criteria (authenticity, correctness, detail, coherence, completeness).
Ulmmnn: Orel detaited desolption
LLaVA e.::
adding roigee
buthoom faver
udditize to the toiks, thore sintk loated dawu kd skdz of the badarcee, Asssal tash can cas be sea on the right side soens and cup pluood ix
The Lotdl euroecas
and detadied Amerratinn oo cumruriu the raas elerantte auch sodlt with cakertal and the tranh
Overil the neeooeee masdlyy sctemn cnrld tngreeed with more poecte catalle and merrccions.
Score: no
deptins bathnoee with todlet main from. The reder atruned the mide room has culerist ickukee butlrsons and traa nat sett rigle add. cup plactd suutee naa sale roars buthaooen katueo mutree reflecting
located tosande
spacs.
Critic:
The reporoo proditeca biifyy wommo dwcrppttnndd of trage containc inwcrrade andennieiins Bleeoon print:
Thactokk indool the main Sezs and altunte docidll dis therronn.
calecia seat comocity untol.
The trak can rialt add cenadly manticned.
Aow mablle the irage, ani past of deveritionn innered.
invomeett trantions mimorr
Derledone:
cabiant with drovers aeatiseed.
DeDesabe
The LMD expprtac deas nut axaticn thee oubiact with deowas whid sanifmart part decsear..
Gaven
the
contaire inavern
Therrios Excred Sillowe
Seuse:
-The neponen comeerty iivesttiee the vote oberente the the milee, wirefi cosee and trddeen.
liooowe tnooreetty dearrhie the rast and mentoone mamer and mry that rewser.
LavAA.g
The innas fnsturo batiarons wisk u solks e dae anda deeas The tolls bu bnshundde ad more und las sdlerid cowe on Asink locded on the saDe: doantbee inage.
The
wheh
Crttle:
The 1304 gouse poovies greend deselptiss of the bathoon Hightghtiig the rain clevens as the noke with colertd sos ovey, skik trado oas and edrict sile the descrtption mentoone teaal trush the righ adlo wiech soamit,, bus mantivn the mill tode the the ner the bake: The vxmil deaciption teaomanchht makkdding and andinng
vidile sultionll,
Figure 12. Failure case L.La VA-1.5 in refinement. Despite receiving critic feedback the initial response, ILaVA-I 1.5 fails to correct the hallucinated objects (marked in red) in the second turn and further introduces the nonexistent "floor mat" in the third turn.
18

---

Perception in Reflection
Imagge
Original Image
LLaVA-La
(Ours)
oo leek oeloomnn cenddec me armone line
mim wide beting morir manuri
the hodguaad
with modes odeer and evry mugpiie
eomiaa receenent the deniiii bus agrurr prontadind
Original Image
LLaVA-i.5
(Ours)
Original Image
LLaVA
(Ours)
lerea Aooeen di bo colers oading siadl snlpoood flom perter our
bookk berigg roree then alou coklo vide dl ailooalions which de to cocherrnn yiul belggoaad
wnik, parile
Figure 13. Comparison of initial-round outputed image captions between the baseline model and RePer. RePer generates accurate and detailed descriptions, while the baseline model often introduces hallucinated objects. The corresponding DALLE-3 generated images further illustrate that RePer's captions lead faithful image reconstructions.
Question: Can you spot the zebra standing behind the boys?
L.aVA-155 (Baseline): Yes, there is seboa standing behind the three boys the Image.
RePer (Ours): there zebea this The features three standing front elephant, and they for
Question: kennlly any fumiture noir the window in the image.
LLaVA-15 (Baseline): There a wooden beacs near the window in the image.
(Ours): image,
placed window, boaquet on
Question: position bird's eys?
LLaVA-155 (Rascline): The bird's eye upen and lange, which charasteristic feature of birds of prey.
RePer (Ours): The bind has large and positioned on the side of bead.
Figure 14, Question-nnswering cases evaluating models' image understanding. Responses are generated a single-tur turn format without iterative refinement.
19


Reviews:
[{"review_id": "BUJYBAdxO1", "reviewer": "Reviewer_VmvG", "summary": "The paper proposes a reflective perception framework, named Reflective Perception (RePer), aimed at enhancing the capabilities of large vision-language models (LVLMs). By introducing dual-model interaction between policy and critic models, RePer seeks to enable LVLMs to iteratively refine their visual perceptions, akin to human observation processes. The core idea is to replace single-pass perception with an iterative feedback loop, allowing models to refine their understanding over multiple rounds. The authors also introduce Reflective Perceptual Learning (RPL), a training approach that fortifies the model's intrinsic reflective capabilities via a constructed visual reflection dataset (LLM-assisted) and reflective unlikelihood training. Experimentation demonstrates improvements in image understanding, captioning precision, and hallucination reduction, with RePer achieving alignment between model attention patterns and human visual focus. The methodology promises robustness in tasks requiring complex reasoning and multi-step manipulation.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for constructive feedbacks and for recognizing our presentation quality and experimental design.\nWe carefully address each of your concerns below.\n## Q1: Evaluation Reliability\nTo address concerns about potential bias from using the same LLM for both data construction and evaluation, we followed the suggested direction and conducted: **(1) a human evaluation** comparing LLM ranking with human preferences, and **(2) GAPE evaluation using alternative LLMs** not involved in data construction. These results collectively show:\n- **Our proposed GAPE evaluation aligns well with human judgment**, indicating minimal bias.\n- **RePer consistently achieves the highest preference**, both in human evaluations and in GAPE scores across different LLM evaluators.\n- The strong alignment confirms **GPT-4o as an effective discriminator**, which provides a “discrimination to generation” gradient that guides the model to learn accurate preferences.\n\n**1. Human Evaluation**\n\n- **Setup**: This human study uses captions generated by three 13B models on 119 GAPE samples. For each image, three anonymized captions were randomly ordered and shown to six annotators (PhD/Master’s students in CV), who ranked them based on the GAPE criteria (Fig. 7). The interface is shown in Re-Fig.1 at https://reper-vl.github.io/ICML-Rebuttal/.\n- **Metric**: \n  - Mean Rank: The average ranking position across all cases. \n  - Top-1 Rate: The percentage of times a model’s caption was ranked best by humans.\n-  **Results**: As shown below, both human evaluation metrics exhibit a consistent model ranking, aligning with the ranking from the GPT-4o-based GAPE benchmark. Notably, RePer is ranked as the top-1 in 63.87% of cases, validating its effectiveness.\n|Model|Mean Rank↓| Top-1↑|GAPE↑|\n|-|-|-|-|\n|LLaVA-1.5| 2.46|15.13%|77.37|\n|Volcano|2.01|35.29%|78.17|\n|RePer|**1.53**|**63.87%**|**82.54**|\n\n**2. Evaluation with Alternative LLMs**\n\n- To further rule out evaluator bias, we replaced GPT-4o with Claude and Gemini for GAPE scoring on 13B models. As shown below, RePer consistently outperforms baselines, reinforcing the reliability of observed improvements.\n|Model|Gemini↑|Claude↑|GPT-4o↑|\n|-|-|-|-|\n|LLaVA-SFT|78.55|74.64|74.88|\n|LLaVA-RLHF|78.89|74.18|75.36|\n|LLaVA-1.5|80.65|75.87|77.37|\n|Volcano|81.59|76.70|78.17|\n|RePer|**83.39** |**78.41**|**82.54**|\n## Q2: Latency & Cost\nWe justify the practicality of RePer from two angles:\n1. RePer already **outperforms baselines with its initial response** (Tab. 3), achieving better performance under the same inference cost. \n2. As suggested, we **evaluate RePer’s efficiency** on 100 samples from the DetailCaps, measuring latency (ms/token) and cost (total generation time).\nAs shown below, the added cost from extra turns is modest (e.g., +11 ms/token from 1 to 3 turns), and multi-turn refinement remains **optional** based on budget constraints or quality demands.\n|Model|#Turn|Gen. Token|Cost (ms)|Latency (ms/token)|CAPTURE|\n|-|-|-|-|-|-|\n|LLaVA-1.5|1|85.83|4750.0|55.34|51.23|\n|RePer|3|275.42|18377.5|66.73|55.55|\n## Q3: Reliance on Large-scale LLMs\nWe believe breakthroughs are possible without relying on >100B LLMs. \n\nIn RePer, core improvements stem from the reflective perception mechanism and RPL paradigm—not from reliance on large-scale LLMs. Both the reward scoring and the critic model can be implemented without large models: Fig. 2 (Step-3) illustrates rule-based scoring aligning vision and text, and Tab. 3 shows a smaller LLaVA-Critic-7B can be an effective critic. \n\nA promising direction beyond large LLMs is to build more reliable and efficient supervision systems. Large models are convenient for data generation but often introduce hallucinations that cap smaller models’ performance. Combining expert models for pre-/post-processing, or incorporating lightweight human-in-the-loop feedback [1] for corretion/validation, offers a cost-effective way to improve supervision quality.\n\nAnother direction is to pursue efficient modeling paradigms. Recent work such as OpenAI-O1/Deepseek-R1 employ RL methods like PPO/GRPO on verified data, enabling models to better exploit internal capabilities and self-generate high-quality reasoning trajectories.\n\n[1] Garg R et al. (2024). Imageinwords: Unlocking hyper-detailed image descriptions. Google.\n## Q4: Relations to Iterative Learning Strategies\nRePer shares conceptual ties with active and curriculum learning: it uses feedback to refine predictions and progressively improves responses from coarse to accurate. Unlike traditional approaches, this progression is self-structured within the RePer’s own decision loop.\nAs a future direction, integrating uncertainty-based reflection or organizing learning trajectories from simple to complex corrections could further enhance adaptability.\n## Q5: Reference & Typo\nThank you for the helpful references. We carefully considered them for designing our human evaluation, and will include the citations and fix the typo in the revision.", "claims_and_evidence": "The paper's claims are reasonably well-supported by experimental evidence. \n* (+) The authors assert that RePer improves image understanding and reduces hallucinations, which is backed by quantitative results across several benchmarks (i.e. MMHal-Bench and HallucinBench). \n* (+) The experiments demonstrate superior performance in capturing details and aligning model attention with human focus, which is supported by the results on DetailCaps.\n* (-) I am not fully convinced of the eval with GAVIE and GAPE: the evals seem to use the same LLMs involved in finetuning data generation? (sec 2.2, step-2, scoring). One may make the \"leveraging discrimination -> generation gradient\" argument here but that would need to be further supported empirically which the paper doesn't currently do.", "methods_and_evaluation_criteria": "The methods and evaluation metrics employed are appropriate (but not the strongest) for addressing the challenges in multimodal perception. \n* (+) RePer's dual-model architecture—using both policy and critic models in a feedback loop—adequately mimics iterative human perception, which seems fitting for tackling hallucinations and enhancing refined understanding. \n* (+) The use of the MMHal-Bench + HallucinBench + DetailCaps combo makes a good case for the improved capabilities with RePer. \n* (-) Again, not fully convinced of the results with GAVIE and GAPE unless empirical evidence is provided to show a) absence or minimal bias introduced with gen + eval with the same models; b) effective discrimination -> generation gradient at work (i.e. model judges better than it generates wrt. task at hand).\n* (-) Would be informative to have a A/B comparison on the latency & cost, since RePer likely generates a lot more tokens and many overlap across rounds of responses (e.g. see the example figure 9), so an efficiency eval helps understand the practicality aspect.\n\nOn additional comment on the negative point above on human evals -- generally the best replacement for GPT-eval-GPT here would be a medium-sized human evaluation. (Reference: see how this Google's text2image evaluation runs a convincing human evaluation)\nNot a practical fix now but should the work missed the acceptance this time, this would be the most ideal action before the next sub in my humble opinion.", "theoretical_claims": "The paper does not present formal theoretical proofs but does establish a conceptual framework grounded in reinforcement learning and imitation learning principles. The theoretical basis for Reflective Perceptual Learning aligns with existing reinforcement learning methodologies, yet does not delve deeply into formalizing the convergence properties or theoretical guarantees of the iterative reflective perception process. While the conceptual claims are plausible, they remain largely empirical rather than rigorous theoretical assertions.", "experimental_designs_or_analyses": "The experimental design appears robust, with comprehensive evaluation across multiple well-chosen benchmarks. The authors thoughtfully designed ablation studies to analyze the influence of reflection turns, scoring disparities, and unlikelihood loss weights. These provide strong evidence of RePer’s advantages and the importance of iterative refinement in perception. However, the experiments could be expanded to include more varied real-world scenarios to ensure generalizability. Additionally, again, because of the lack in strong-enough automated evals, including qualitative **human** feedback would strengthen the assessment of how well model improvements align with human expectations and perception nuances.\n\nOverall, the work looks very promising, but slightly incomplete. To echo some of the key points mentioned on the matter of \"what would have made a more complete experimentation\"\n* Efficiency analysis (with the added reflection budget);\n* Either empirically ground effectiveness of Discrimination -> Generation gradient with this design (GPT-eval-GPT), or human eval.", "relation_to_broader_scientific_literature": "The paper fits within the broader context of enhancing multimodal models by refining perception processes and reducing hallucinations. It draws on established concepts like Chain-of-Thought reasoning and imitates human-like iterative perception, enriching the current strategies used in vision-language models. RePer's approach contrasts with traditional single-pass perception methods by offering a cyclical refinement process. The empirical focus distinguishes it from theoretical studies and aligns with practical applications in multimodal research areas. However, a deeper engagement with similar iterative learning strategies used in adjacent fields, such as curriculum learning or active learning in machine learning, could enrich the discussion about RePer’s novel contributions and possible future extensions.", "essential_references_not_discussed": "On the method front, not that I know of. \nFor evaluation, perhaps mentioning below to contextualize better the evaluation of detailed text<->image alignment.\n* Image->Text: \n  - CLAIR: https://arxiv.org/abs/2310.12971\n  - DOCCI: https://arxiv.org/abs/2404.19753\n* Text->Image:\n  - Gecko: https://arxiv.org/abs/2404.16820v1\n  - DSG: https://arxiv.org/abs/2310.18235\n\nSince the core proposal with RePer is better image understanding on (fact-grounded) granularity, detailed semantic alignment is a natural topic to touch on.", "other_strengths_and_weaknesses": "Not much beyond what I'd discussed above. The paper has a clean presentation with well-versed prose; provides good empirical investigation (despite slightly flawed evaluation).", "other_comments_or_suggestions": "Reference typo: GAIVE -> GAVIE (https://arxiv.org/pdf/2306.14565)", "questions_for_authors": "RePer is a great attempt to move beyond the somewhat \"zeigeist\" of \"let's see if we could prompt LLMs this novel way\" with a well-executed (what I'd consider as) knowledge distillation investigation: LLM-assisted data curation -> model finetuning. \nHow much do you think the reliance on massive LLMs (>100B) form a semi-hard/hard barrier for smaller research groups to make breakthroughs on pushing the boundary of better reasoning? If not, what types of large step functions can we expect which are more free from the reliance.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1sLB9PktTa", "reviewer": "Reviewer_hDWb", "summary": "This paper proposes RePer, which teaches the VLM to iteratively revise and provide gradually better responses given a strong pre-built critic model.\n\nThe algorithm works by first collecting responses of varying quality, using these to construct an iteratively revised dataset, and then employing a “Reflective Unlikelihood Training” method that effectively teaches the model to follow this data.\n\nResults show some improvement across benchmarks.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive feedback, particularly for recognizing our work as a “well-motivated method” with solid experiments. Below, we carefully address each of your concerns and clarify potential misunderstandings.\n## Q1: LLaVA-1.5 BoN vs RePer\nWe appreciate the suggestion and clarify **a possible misunderstanding: the critic model is not reqired but optional** for RePer during inference. As discussed in Sec. 4.6 (Lines 370–381), RePer already outperforms LLaVA-1.5 in the first turn without any external feedback, demonstrating a **fair comparison** that reflects the effectiveness of our reflective perception training.\n\nTo further address the reviewer’s concern, we conducted another fair comparison on DetailCaps-4870 by implementing a **Best-of-N** (N=1,3,5,6) baseline for LLaVA-1.5, where we sample N responses and use the same critic (LLaVA-Critic-7B) to select the best one for computing the final score.\n\nAs shown in the table, even with a larger sampling budget, **LLaVA-1.5-13B Best-of-6 still falls short of RePer-13B’s performance using just a single-turn response**. This underscores RePer’s efficiency and effectiveness in answer generation: rather than relying on brute-force sampling, RePer produces strong initial responses through reflective perception learning mechinism, and further enhances them via feedback-driven refinement. \n\n| Model         | #BoN | #Reflection | Capture | Precision | Recall |\n|---------------|:------:|:-------------------:|---------|-----------|--------|\n| LLaVA1.5  |  1    |        -                 | 51.23   | 65.54     | 43.92  |\n| LLaVA1.5  |  3    | -                 | 51.28   | 65.73     | 43.81  |\n| LLaVA1.5  |  5    | -                 | 51.06   | 65.55     | 43.75  |\n| LLaVA1.5  |  6    | -                 | 51.36   | 66.11     | 43.95  |\n| RePer           | -    | 1                 | 54.29        |  **66.15**  | 47.66       |\n| RePer           | -    | 2                 | 54.68        |  65.24   |   48.68     |\n| RePer           | -    | 3                 | **54.73**        |  64.74         | **49.1**       |\n\nWe will clarify the fairness of the comparison more explicitly in the revision.\n\n## Q2: Alignment with Human Visual Focus\nTo address this concern, we conducted a human evaluation to further support the claim that RePer’s image attention better aligns with human visual focus.\n - **Setup**: We recruited 6 annotators (PhD/Master’s students in computer vision and NLP) and randomly sampled 100 images from the test set. For each image, we collected image attention maps (as shown in Fig. 4) generated by RePer and LLaVA-1.5 during inference. The attention maps were anonymized and randomly shuffled before being shown to annotators, who were asked: “Which attention map better reflects your own visual focus if you were to answer this question?” The annotation interface is shown in Re-Fig. 2 on our supplementary website: https://reper-vl.github.io/ICML-Rebuttal/.\n - **Metric**: We report the **win rate**, defined as the percentage of cases where RePer’s image attention map was preferred over LLaVA-1.5’s by annotators.\n - **Result**: Image attention maps from RePer was preferred in **70.27%** of cases over LLaVA-1.5, indicating **stronger alignment with human visual focus**.\n## Q3: Discussion about RISE\nPlease refer to **Lines 196–206** for our discussion on the connection and differences with RISE. While both methods leverage iterative supervision, RePer introduces a distinct imitation learning formulation based on **reward-weighted reflective unlikelihood training**. Moreover, it explicitly proposes a reflective perception mechanism aimed at addressing a key limitation of current LVLMs—the unrealistic assumption of perfect initial responses.\n\n## Q4: Correlation Between Feedback and Next-Round Response\nWe clarify that the next-round responses are **indeed correlated with the critic feedback**. As illustrated in Figure 8, during data construction, each successive response in the conversation is selected based on a higher GPT-4o score than the previous one. \nConcretely, the first response hallucinates non-existent objects like “bottle or vase” which the critic points out. The second response removes these errors and correctly identifies the image as a book. In the third turn, the model further corrects the authorship and improves conciseness, directly following the critic’s feedback. This step-by-step correction process demonstrates a strong alignment between the feedback and the revised responses, providing effective supervision for learning from reflection.", "claims_and_evidence": "1. The authors claim “RePer’s quantifiable improvements in image understanding, captioning precision, and hallucination reduction” compared to vanilla Llava 1.5. However, given that RePer requires an additional, pre-trained critic model, it is unclear whether the comparison with standalone Llava is fair. I think the paper would benefit from a Best-of-N baseline, which compares RePer’s N-step revision with sampling the solution from Llava-1.5 N times and using the critic to pick the best solution.\n\n2. The authors also claim, “The model achieves strong alignment between model attention patterns and human visual focus,” but the only evidence provided is a single example in Figure 4, which is not convincing. Although Figure 6a shows that average image token activations increase more with RePer, this does not necessarily imply a stronger alignment with human visual focus.", "methods_and_evaluation_criteria": "Yes, the method appears to be well-motivated, and the benchmarks are solid. However, one issue is that this approach assumes the existence of a robust critic model, which might not always be available.\nAdditionally see Claims And Evidence 1", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "See Claims And Evidence*", "relation_to_broader_scientific_literature": "It extends prior success of improving model's performance through revision, as shown in SCoRe and RISE, from math reasoning, text-based setting to vision-language models.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "The paper could benefit from deeper discussion with prior works, such as RISE.", "questions_for_authors": "1. How does RePer compare to Best-of-N against the same critic model?\n2. The model receives textual feedback from the critic model, while in training data, the next round of response isn't really correlated with the textual feedback, is this correct?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "F9Q5iPgQeA", "reviewer": "Reviewer_SX8u", "summary": "This paper introduces Reflective Perception (RePer), a system for improving VLMs through iterative self-reflection. It adopts a policy–critic framework, where a policy model generates outputs, and a critic model provides feedback to refine responses over multiple turns. The paper also proposes Reflective Perceptual Learning (RPL), a training strategy leveraging automatically generated reflection data and unlikelihood loss to improve factual accuracy and reduce hallucinations. Experiment results show RePer's quantifiable improvements in image understanding, captioning precision, and hallucination reduction", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the thoughtful feedback and for recognizing the value of our method and experiments. We carefully address the concerns and clarify potential misunderstandings as follows:\n## Q1: Alignment with Human Visual Focus\nWe address this concern by clearly defining “ground-truth human attention” and conducting human evaluations:\n\n**“Ground-truth human attention”** refers to the qualitative visual focus humans naturally adopt when answering questions—typically prioritizing the main subject over background elements. An attention map that better reflects human perception tends to activate more semantically relevant image tokens and highlight regions aligned with human focus patterns [1].\n\nTo further support this claim, we conducted a **human evaluation** to compare the image attention of RePer and LLaVA-1.5.\n - **Setup**: Six annotators (PhD/Master’s students in CV/NLP) were shown anonymized and shuffled attention maps from both models on 100 randomly sampled test images. They were asked to choose: “Which attention map better reflects your own visual focus if you were to answer this question?” The interface is shown in Re-Fig. 2 at https://reper-vl.github.io/ICML-Rebuttal/.\n - **Metric**: We report the **win rate**—the percentage of cases where RePer’s map was preferred over LLaVA-1.5’s.\n - **Result**: Image attention maps from RePer was preferred in **70.27%** of cases over LLaVA-1.5, indicating **stronger alignment with human visual focus**.\n\nWe will revise the paper to clarify the definition and more accurately scope this claim based on the supporting evidence.\n\n[1] Huang et al. OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation, CVPR 2024.\n\n## Q2: Advantages in Complex Reasoning\nWe appreciate the reviewer’s suggestion and would like to clarify **a potential misunderstanding: our work focuses on perception-oriented tasks, while complex reasoning is positioned as a future direction**. As noted at the end of the Introduction, we see strong perception as a foundation for advanced reasoning capabilities.\n\nTo further explore this potential, we evaluated 13B models on **three reasoning benchmarks**—ScienceQA, MMMU, and AI2D—using VLMEvalKit. As shown below, RePer consistently achieves the highest performance, demonstrating stronger reasoning capabilities and generalization.\n\n| Method  | ScienceQA | MMMU  | AI2D  |\n|-|-|-|-|\n| LLaVA-SFT      | 65.76| 34.67 | 47.67 |\n| LLaVA-RLHF     | 63.85| 34.89 | 44.62 |\n| VOLCANO        | 67.29     | 34.33 | 55.89 |\n| LLaVA-1.5      | 68.43 | 35.40 | 58.84 |\n| RePer          | **69.10** | **36.89** | **59.32** |\n\n## Q3: Novelty of RePer\n**1. Reflection in LVLMs vs. LLMs (Importance & Uniqueness)**: While reflection has been explored in LLMs, it holds distinct importance in the context of LVLMs. Unlike language models, where input tokens are discrete and semantically stable, visual perception involves high uncertainty. LVLMs are more prone to hallucinating nonexistent objects, misinterpreting visual cues, or overlooking salient regions—challenges that extend beyond typical language ambiguity. Our reflection-based method directly addresses these issues by enabling iterative refinement. Beyond improving output quality, it also enhances human-aligned visual attention and reduced hallucination, underscoring its unique value for multimodal perception.\n\n**2. Dual Methodological Contributions**:\n- First, we propose reflective perception learning to LVLMs. where image-grounded, step-wise corrections provide fine-grained rewards for aligning vision and language. This is fundamentally distinct from language-only feedback and proven highly effective.\n- Second, we introduce a reward-weighted unlikelihood training to reinforce high-quality responses while explicitly penalizing suboptimal ones. This mitigates the response collapse observed in prior works (e.g., RISE, SCoRe), where early-stage answers dominate learning regardless of quality, and enables stronger first-turn performance.\n\n**3. Data Construction innovations**:\nWhile not extensively emphasized in the main paper, our data construction pipeline introduces key innovations by combining VLM-based and rule-based reward signals to construct high-quality reflective conversations. This dual-reward structure improves the precision and interpretability of supervision, making a clear improvement over prior self-reflection datasets (e.g., RISE, SCoRe). Notably, it also resonates with both RM-based and RM-free RL paradigms.\n\nFinally, regarding the notion of novelty itself, we borrow a perspective from Novelty in Science [1]:\n> *If you hear a good idea, there is a moment of surprise and then, the better it is, the more obvious it may seem. If it is easy to explain and obvious in hindsight, this in no way diminishes the creativity (and novelty) of the idea.*\n\n[1] Black. Novelty in Science. Medium. https://medium.com/@black_51980/novelty-in-science-8f1fd1a0a143", "claims_and_evidence": "Most of the claims make sense to me; however, there are two worth discussing.\n\n1. \"RePer achieves strong alignment between model attention patterns and human visual focus\".\nFrom the discussion in Section 3.1, I can see the model shifts its attention after learning, but it is unclear to me how the model aligns with human perception. \n\n2. Advantages in complex reasoning tasks.\nThe paper emphasizes RePer's advantages in complex reasoning tasks, but the experimental dataset is not very convincing. The covered dataset includes image understanding, hallucination detection, and image captioning; none of the mentioned focuses on complex reasoning.", "methods_and_evaluation_criteria": "The overall method makes sense to me. I agree that the proposed method is a commendable way to address the targeted problem. The only concern about the method is the novelty (see Strengths And Weaknesses section). \n\nThe evaluation part is clear and reasonable, but there is potentially an overclaim concern, as mentioned in the Claims And Evidence section.", "theoretical_claims": "No serious flaws found", "experimental_designs_or_analyses": "See Claims And Evidence", "relation_to_broader_scientific_literature": "See Other Strengths And Weaknesses", "essential_references_not_discussed": "Related works are clear", "other_strengths_and_weaknesses": "The proposed method is both reasonable and meaningful, with experimental results across multiple datasets demonstrating its effectiveness. However, the novelty of this work is questionable, as the core concept of Reflective Perceptual Learning (RPL) does not introduce fundamentally new learning principles or new findings.\n\nSpecifically, the proposed learning framework closely resembles RLHF and self-reflection mechanisms, where a critic model provides iterative feedback to refine predictions. The underlying learning paradigm aligns with process supervision, and the process signal is derived from an LLM-as-a-Judge (plus rule-base scoring) approach. Additionally, the data construction strategy is similar to self-learning or self-distillation techniques. As a result, the overall framework does not present a fundamentally novel methodological contribution beyond its application to vision-language models.\n\nThat being said, expanding the success of language model training to VLM represents a meaningful step toward improving real-world applications. Given its potential impact, I am inclined to assign a positive score at this stage.", "other_comments_or_suggestions": "None", "questions_for_authors": "(Refer to the Claims And Evidence Section for more context on the following two questions.)\n\n\n1. While I appreciate and generally agree with the findings presented in Section 3.1, the evidence provided does not sufficiently demonstrate alignment with human perception. I would like to know if the authors have additional evidence to support this claim or if they would consider narrowing the scope of this claim for greater accuracy. To support this discussion, I am curious about the definition of \"ground-truth human attention\" in line 243. Is it from human annotation?\n\n2. I believe including datasets focusing on complex reasoning can greatly improve the soundness of the paper. Depending on the reasoning type of interest, the corresponding dataset may be tested, such as common sense reasoning (ScienceQA), complex multimodal reasoning (MultiModalQA), etc.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes a multi-step framework for visual perception based on a perception-feedback loop. Implementing such a system is challenging because using a model to self-correct it's own outputs has often been shown empirically to be ineffective.

Here, the authors begin by collecting a dataset of multi-turn reflective dialogs, by first generating initial responses to a question, then scoring these responses with both model-based and rule-based approaches, before finally ordering the responses from worst-to-best into a dialog (steadily improving responses interspersed with feedback).

Experiments are conducted finetuning finetuning LLaVA-1.5 on the collected dataset using Reflective Unlikelihood Training (RUT), a variation of unlikelihood training. Experiments show consistent improvements on image understanding, hallucination detection and detailed image captioning.

All reviewers voted to accept the paper and I agree. However, I found some of the emphasis in the paper (for example, on the perception-feedback loop) to be misaligned with the relative importance of the paper's contributions. According to Table 3, it seems like most of the benefit to the approach comes from the finetuning process, and not the multi-round perception-feedback loop. Further, the critic model is from a different paper and so isn't a contribution.

However, to my knowledge the RUT loss is novel. The same sequence appears in both the likelihood and the unlikelihood term, but with weightings determined by normalized reward. The advantage over weighted supervised finetuning appears to be that the unlikelihood term introduces explicit repulsion from bad responses. The author's acknowledge that "RPL is a Free-Form Preference Optimization" (Section 3.2). Personally, I think RUT begs for more time spent on the direct comparison to DPO (the collected dataset can be interpreted in terms of pairwise preferences). This is included in Table 4 but with insufficient details of the implementation. 

Overall, this is a solid paper. I encourage the authors to consider all feedback including tweaking the focus more towards reward-weighted reflective unlikelihood training (e.g. perhaps greater mention in the title/abstract, and improved details around Table 4).
Decision: Accept (poster)