Title: VisualQuality-R1: Reasoning-Induced Image Quality Assessment via Reinforcement Learning to Rank
Venue: NeurIPS 2025
Accepted: yes
Keywords: Image Quality Assessment

Reinforcement Learning

Reasoning-induced no-reference IQA model
Abstract: DeepSeek-R1 has demonstrated remarkable effectiveness in incentivizing reasoning and generalization capabilities of large language models (LLMs) through reinforcement learning. Nevertheless, the potential of reasoning-induced computation has not been thoroughly explored in the context of image quality assessment (IQA), a task depending critically on visual reasoning. In this paper, we introduce VisualQuality-R1, a reasoning-induced no-reference IQA (NR-IQA) model, and we train it with reinforcement learning to rank, a learning algorithm tailored to the intrinsically relative nature of visual quality. Specifically, for a pair of images, we employ group relative policy optimization to generate multiple quality scores for each image. These estimates are used to compute comparative probabilities of one image having higher quality than the other under the Thurstone model. Rewards for each quality estimate are defined using continuous fidelity measures rather than discretized binary labels. Extensive experiments show that the proposed VisualQuality-R1 consistently outperforms discriminative deep learning-based NR-IQA models as well as a recent reasoning-induced quality regression method. Moreover, VisualQuality-R1 is capable of generating contextually rich, human-aligned quality descriptions, and supports multi-dataset training without requiring perceptual scale realignment. These features make VisualQuality-R1 especially well-suited for reliably measuring progress in a wide range of image processing tasks like super-resolution and image generation.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes VisualQuality-R1, a no-reference IQA (NR-IQA) model that leverages reinforcement learning to rank (RL2R) with a novel integration of the Thurstone model and Group Relative Policy Optimization (GRPO). The core innovation lies in treating visual quality as a relative (rather than absolute) perceptual quantity.\n\nSTRENGTHS AND WEAKNESSES:\nAs far as I  am concerened, **Strengths** are:  1) Strong empirical results: Comprehensive experiments (zero-shot generalization, multi-dataset training, ablation studies) validate superiority in SRCC/PLCC metrics (e.g., **0.791** avg. SRCC vs. 0.759 for Q-Insight).  2) Interpretability: Human-aligned textual explanations (Figs. B, C in supplement) enhance transparency, a significant advance over \"black-box\" NR-IQA models.  3)Practical impact: Enables multi-dataset training without MOS realignment, addressing a key bottleneck in real-world deployment.  \n\n**Weaknesses are:**\n\n- Computational overhead: GRPO’s multiple generations per image (\\(K=6\\)) incur high inference costs. Efficiency trade-offs vs. gains are underexplored.  \n- Prompt rigidity: Reliance on a fixed text prompt (Table 1) limits adaptability to distortion-specific contexts (e.g., artistic vs. photographic images).  \n- Baseline depth: Comparisons with recent VLM-based methods (e.g., LLaVA-IQA) are missing. Q-Insight is the only RL-based baseline.\n- Societal impact: While claimed as minimal (§9), potential misuse for automated content moderation (e.g., suppressing \"low-quality\" images) is unaddressed.\n\nQUESTIONS:\n- The fixed prompt (Table 1) may hinder adaptation to context (e.g., synthetic art vs. natural images). Fig. C shows scores for stylized images lack nuance (e.g., rating digital art \"4.00\" without style sensitivity).  \n- GRPO’s \\(K=6\\) generations/image imply high costs. No latency/memory metrics are provided.  \n- Fig. B (supplement) shows an implausible score of **0.20** (below the 1–5 range)\n- The conclusion suggests extending to FR-IQA but offers no experiments.  \n\nFinally, if the authors can actively address my concerns, I will be happy to improve the rating score.\n\nLIMITATIONS:\nAdequately discussed (computational cost, prompt rigidity, error propagation in §9). The suggestion of \"sample-adaptive reasoning\" for efficiency is promising but untested.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-20"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces VisualQuality-R1, a reasoning-based framework for NR-IQA built upon LMM. The model is trained via reinforcement learning to rank images by visual quality, leveraging the GRPO algorithm used in DeepSeek-R1. A key innovation is the use of a continuous fidelity measure, grounded in the Thurstone model, rather than discretized binary labels as rewards. Experimental results across multiple benchmarks demonstrate that VisualQuality-R1 outperforms existing methods, delivering state-of-the-art performance and producing contextually rich, human-aligned quality explanations.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n- The proposed RL2R algorithm is novel and effectively captures the relative nature of image quality assessment. It enables multi-dataset training without requiring perceptual scale alignment.\n- The paper conducts extensive experiments, including comparisons with recent methods such as Q-Insight, and demonstrates the superior performance of VisualQuality-R1 across several benchmarks.\n- A comprehensive ablation study is presented, clearly analyzing the contributions of different components within the framework.\n- Figure 4 is impressive, illustrating the model's progressive improvement over the course of training and providing qualitative insight into its reasoning capabilities.\n\n## Weaknesses\n\n- Given that reasoning is central to the proposed framework, the paper would benefit from more qualitative comparisons with other methods (e.g., Q-Insight), to better showcase the advantages of VisualQuality-R1 in generating interpretable outputs.\n- Including typical failure cases would enhance the paper’s transparency and help readers better understand the limitations of the proposed method.\n- In Table 4, the performance gap between binary and continuous rewards is relatively small (e.g., 0.772 vs. 0.777 and 0.809 vs. 0.814), which may raise questions about the practical significance of adopting continuous rewards.\n- The multi-dataset training setup only includes two datasets. Incorporating a broader range of datasets could strengthen the generalization claims and provide a more thorough evaluation.\n\n## Conclusion\n\nThis paper presents a novel approach to NR-IQA through the integration of reasoning with reinforcement learning. The proposed RL2R algorithm  is innovative and effective. The experimental results, including strong baseline comparisons and detailed ablations, support the method’s contributions. Overall, this work represents a significant step forward in the field of IQA, enhances the understanding of IQA and provides a promising direction for future research.\n\nQUESTIONS:\nPlease see the weakness points above.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces VisualQuality-R1, a reasoning-induce, no-reference image quality assessment (NR-IQA) model that produces both scalar quality scores (1-5) and natural language justifications. The paper pointed out prior approaches using regression-based approaches which have their limitations and framed IQA as a ranking problem and trained the model using a custom Reinforcement Learning to Rank (RL2R) framework. \nThe model, built on a VLM (Qwen2.5-VL-7B) generates multiple reasoning-score pairs per image using Group Relative Policy Optimization (GRPO). These are used to compute probabilistic comparisons between image pairs via the Thurstone model, which accounts for both mean and variance of predicted scores. Training feedback comes from a continuous fidelity reward, which measures how well the model’s pairwise rankings align with human preferences (Mean Opinion Scores). \nExperiments across eight datasets show that it outperforms existing deep learning and VLM-based IQA methods. It is notable for its ability to generalize across dataset and for producing high-quality justifications for its predictions.\n\nSTRENGTHS AND WEAKNESSES:\nOverall the paper is a comprehensive paper that presents a significant advancement in the field of image quality assessment. The methodology is well motivated and the results are validated. The work is well written and makes a clear contribution.\n\nStrengths:\nNovel Approach: Combining GRPO with Thurstone-based quality comparisons and continuous fidelity reward is original within IQA.\nMultimodal Reasoning: Demonstrates a step beyond score-only predictions, offering contextual textual explanations.\nComprehensive Evaluation: Benchmarks across eight datasets and various distortion types.\nAblation and Analysis: Includes studies of GRPO response count, reward formulation, and score variance over training.\n\nOpportunities For Improvement:\nComputational Cost Analysis: The paper honestly acknowledges that the method is \"slow, expensive, and memory-hungry\" due to the need to generate multiple responses per image at inference time. While this limitation is stated, a quantitative comparison of the inference time and computational cost of VisualQuality-R1 versus other leading models would provide a clearer picture of the practical trade-offs involved.\nPrompt Robustness: The entire methodology relies on a single, fixed text prompt provided in the paper. The authors propose exploring dynamic prompting as future work, but a small-scale study on the current model's sensitivity to minor changes in the prompt wording could have added valuable insight into its robustness\nModel's Reasoning Lacks Empirical Substantiation: Although the model output rationales, there is no human evaluation or rubric to assess their alignment, usefulness or consistency. \nOverclaim Generalization: Although the paper claims that VisualQuality-R1 can train across datasets without perceptual scale realignment (i.e., without mapping all scores to a common scale), it actually relies on rescaling MOS scores to a common 1–5 range before training. The continuous reward also based on MOS differences assumes access to human scores during training, which limits scalability to unlabeled settings.\n\nQUESTIONS:\n1. The paper honestly acknowledges that the method is \"slow, expensive, and memory-hungry\" due to the need to generate multiple responses per image at inference time. Any plan to provide a quantitative comparison of the inference time and computational cost of VisualQuality-R1 versus other leading models?\n\n2. Although the model output rationales, there is no human evaluation or rubric to assess their alignment, usefulness or consistency. What is a plan to move forward with this?\n\n3. Although the paper claims that VisualQuality-R1 can train across datasets without perceptual scale realignment (i.e., without mapping all scores to a common scale), it actually relies on rescaling MOS scores to a common 1–5 range before training. The continuous reward also based on MOS differences assumes access to human scores during training, which limits scalability to unlabeled settings. Can you comment on this?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose VisualQuality-R1, a no-reference image quality assessment model that incorporates reasoning through reinforcement learning to rank. Specifically, the method employs GRPO to generate multiple quality estimates per image and leverages the Thurstone model to compute pairwise comparative probabilities. A continuous fidelity reward is introduced to replace traditional binary rewards, aligning better with human perceptual scores. Experimental results demonstrate that the proposed approach outperforms existing SOTA methods across multiple datasets.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n1. The proposed RL2R framework effectively captures the relative nature of human quality perception, as supported by the empirical results in Section 4. \n2. The introduction of a continuous fidelity reward, combining predicted probabilities and MOS labels, allows for more informative and stable policy updates in GRPO. \n3. Extensive experiments, including evaluations on multiple benchmark datasets and ablation studies, show the method achieves superior performance compared to prior approaches.\n\n### Weaknesses\n1. As shown in Table 4, purely relying on relative scoring does not yield significant performance gains. The improvements are mainly observed when enhanced with components like continuous rewards, adaptive variance estimation, and probability averaging, suggesting the base design alone is insufficient.\n2. Without joint training on multiple datasets, the proposed method achieves only a marginal improvement over Q-Insight, approximately a 1 percentage point gain, raising questions about its generalizability and robustness in single-dataset scenarios.\n3. Given the model's reliance on relative comparisons within a batch, the batch size and sample quality can significantly influence training. It is unclear whether the authors explored the impact of batch construction or optimized sampling strategies.\n4. The paper lacks analysis of computational cost, including training/inference time and memory consumption, making it difficult to assess the method's practicality in real-world deployment.\n5. The use of a single structured prompt for all distortion types may limit the model’s adaptability to diverse and context-specific quality cues.\n\nQUESTIONS:\nHave the authors conducted further prompt ablation studies, especially evaluating the effect of using different prompts for different distortion types?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}