Unlocking the Essence of Beauty: Advanced Aesthetic Reasoning with Relative-Absolute Policy Optimization
Authors: Boyang Liu
Yifan Hu
Senjie Jin
Shihan Dou
Gonglei Shi
Jie Shao
Tao Gui
Xuanjing Huang
Venue: ICLR 2026 Poster (2026)
Keywords: Reinforcement Learning，Image Aesthetic Assessment
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Multimodal large language models (MLLMs) are well suited to image aesthetic assessment, as they can capture high-level aesthetic features leveraging their cross-modal understanding capacity. However, the scarcity of multimodal aesthetic reasoning data and the inherently subjective nature of aesthetic judgment make it difficult for MLLMs to generate accurate aesthetic judgments with interpretable rationales. To this end, we propose Aes-R1, a comprehensive aesthetic reasoning framework with reinforcement learning (RL). Concretely, Aes-R1 integrates a pipeline, AesCoT, to construct and filter high-quality chain-of-thought aesthetic reasoning data used for cold-start. After teaching the model to generate structured explanations prior to scoring, we then employ the Relative-Absolute Policy Optimization (RAPO), a novel RL algorithm that jointly optimizes absolute score regression and relative ranking order, improving both per-image accuracy and cross-image preference judgments. Aes-R1 enables MLLMs to generate grounded explanations alongside faithful scores, thereby enhancing aesthetic scoring and reasoning in a unified framework. Extensive experiments demonstrate that Aes-R1 improves the backbone’s average PLCC/SRCC by 47.9%/34.8%, surpassing state-of-the-art baselines of similar size. More ablation studies validate Aes-R1's robust generalization under limited supervision and in out-of-distribution scenarios.

Published as a conference paper at ICLR 2026
UNLOCKING THE ESSENCE OF BEAUTY: ADVANCED AESTHETIC REASONING WITH RELATIVE- ABSOLUTE POLICY OPTIMIZATION
Boyang Liu Ax Yifan Hu Senjie Jin +., Shihan Dou'
Gonglei Shi'
Jie Shao Tao Gui Xuanjing Huang
Fudan University *Shanghai Collaborative Innovation Center of Intelligent Visual Computing "Tsinghua University "Bytedance
(boyangliu25, sjjin24) @m. fudan. edu, en tqui@fudan, edu cn
ABSTRACT
Advancements in multimodal large language models (MLLMs) demonstrate strong potential across various domains, with expanding applications in human- centered Image Aesthetic Assessment (IAA) through high-level cross modal un- derstanding capacity. Compared to data-intensive supervised fine-tuning (SFT), the efficient and scalable reinforcement learning (RL) promising alternative for IAA. However, directly applying RL to IAA fails to inspire reasoning patterns without prior aesthetic knowledge, and modeling an appropriate reward proxy to evaluate aesthetic scores is intricate. To this end, propose Aes-R1, compre- hensive IAA framework. Concretely, Aes-R1 integrates a data pipeline, AesCoT, construct and filter high-quality aesthetic reasoning data used for cold-start. After teaching the model to generate structured explanations before scoring, we employ the Relotive-Absolute Policy Optimization (RAPO), novel RL algorithm that jointly optimizes absolute score regression and relative ranking order, im- proving both per-image accuracy and cross-image preference judgments. Exten- sive experiments demonstrate that Aes-R1 improves the backbone's average PLCC- CISRCC by 47.9 47.9% surpassing state-of-the-arr baselines of similar train- ing size. More ablation studies validate Aes-RI's robust generalization in out-of- distribution scenarios with limited supervision
Predicted Scores (bansity)
True Scores (Density)
Predicted Scores (Count)
True Scores (Count)
(i) (ours)
VisuaiQuality-R1
(iii) Q-Align
(iv) Q- Q-Insight
(a) Distribution between ground truth and predicted score.
Ground Truth
0.77
Bank
Score 0.81 0.76
0.74 0.89 0.39 0.29
Rank
Q-insight
Soore
6
VisualQuality-Rt RL Scare: 0.90 0.96
0.90 0.68 0.90
0.10
Renk
3
9
Alin
Scane 0.48
(b) Comparison of various methods.
Figure I: (a) Distributions of ground truth and predictions on the AVA test set. Aes-RI achieves the closest alignment with the ground truth compared to baseline methods. (b) The case study shows that Aes-R1 surpasses other methods, which suffer from prediction errors and limited ordinal discernment, based on the relative-absolute reward design.
"Equal Contribution Project lead Corresponding author
Our code is available at httpp::/giiteee com/ wowp///sa////////aaa.....

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Recent advances in multimodal large language models (MLLMs) (Qwen et al. 2025 OpenAI ct al.
2024 Team et al.
2025) have shown remarkable potential in image assessment (Xu et al. To date, most efforts center on image quality assessment (IQA) (Wu et al. 2024
2023
Wu et al. 2023b).
2023a| You et al.
2025), which measures images' sharpness and clarity. As the development of creative design and social media recommendations, there is an emerging demand for reliable metrics that go far beyond pixel-level fidelity and some researchers have paid attention to image aesthetic assessment (IAA) (Huang et al.) 2024 Zhou et al. 2024), leveraging the cross-modal understanding capabilities of MLLMs to capture high-level features such as texture, color harmony and emotional impact, broadening vision tasks to the interpretable and human-centered aesthetic modeling-
Current IAA methods generally adopt supervised fine-tuning (SFT) on score-based targets (Wu et al.) 2023a Cao al. 2025). While such implicit reasoning restricts MLLMs from aligning visual ele- ments with multidimensional aesthetic criteria and lacks explainability in practice (Wu et al. 2025 Li al. 2025), some propose generating reasoning prior to the answer, supporting the integration of fine-grained descriptive feedback with aesthetic scores (Li et al. 2025). Nonetheless, existing resources (He al. 2022, Yang et al., 2022) primarily consist of score-based data, and artist-level rationales necessitate expensive expert annotation. Moreover, as presented in Tab. Bl extensive SFT causes the model to learn bias and overfit on curated dataset, rapidly reducing entropy and hindering further optimization, making it costly and difficult to scale.
Reinforcement learning (RL) is an effective alternative and has shown strong efficiency and gener- alization in goal-driven tasks (Chu et al. [025a]] Chen et al. 2025a). However, in our pilot study, directly applying RL to IAA exposes several challenges. 1) Although achieving high performance (Tab. B SFT 0 epoch, RAPO), the end-to-end RL fails to activate aesthetic reasoning patterns in the backbone due to the absence of corresponding corpus pre-training (Gandhi et al.) 2025 Ji et al.) 2025), resulting in the poor explanatory rationales shown in Fig. 6 (AesRI-Zero). 2) Modeling an appropriate reward proxy to evaluate aesthetic score is intricate since the absence of unified criteria for aesthetic preferences. We reproduce the related works on AVA (Murray et al.
2012) and find that,
although VisualQualit--RI (ii) (Li et al. (025) successfully separates high- and low- low-quality images
using the rank reward, it fails to calibrate scores, while Q-insight (iv) (Wu al.
2025) suffers from
peak mismatch when optimizing with scalar reward, as presented in Fig.
Integrating aesthetic priors and an effective reward mechanism is crucial for IAA in RL optimization. Inspired by Deepseek-RI R1 (Guo et al.) (2025a). we find that cold-start bootstrapping with thousands of high quality reasoning data effectively induces robust aesthetic analysis patterns (Gandhi et al, 2025 [Shao et al.] (025). Following this, RL transforms IAA into a data efficient self-learning process, primarily guided by the reward signal. Building on the results in Fig. [ we observe that human aesthetic judgment is inherently context-dependent besides absolute quality evaluation, as also noted by Immanuel Kant's assertion that "there can be no objective rule of taste which shall determine by means of concepts what is beautifful" (Kant) (Kan) Therefore, argue the IAA reward should comprise two complementary facets: a relative dimension that ranks images by comparative appeal and absolute dimension that assesses intrinsic aesthetic merit, thereby simultaneously enforces ordinal consistency and calibrates absolute aesthetic scores.
Building the above insights, we propose Aes-R1 framework. We first construct an automatic data pipeline AesCoT, which effectively generates high-quality aesthetic reasoning data along five dimensions based on original image-score pairs. Warming up on this data equips MLLMS with aes- thetic cognitive behaviors that generating reliable explanations before the aesthetic scores. We then utilize Relative-Absolute Policy Optimization (RAPO), a novel RL algorithm that optimizes both absolute and relative aesthetic preferences. Specifically, the model's gradient updates are simultane- ously steered by the absolute-error reward, calibrating aesthetic score constraints and relative-rank reward, aligning pairwise ranking consistency, which greatly improves effectiveness and stability in the exploration and exploitation process. Empirically, trained with only 15K data, Aes-RI sur- passes the state-of-the-art baselines of comparable size with the improvements on backbones by 47, 47.9% PLCC/SRCC. The distribution alignment experiment (see Fig. [ a) and reward ab- lations (see Sec. 4.3) demonstrate the effectiveness of our RAPO as well as generalization in the OOD setting (see Sec. 4.2). We further analyze the training recipe of SFT and RL from the entropy perspective (Cui al. 2025b) (see Sec.
2

---

Published as a conference paper at ICLR 2026
In summary, main contributions can be summarized follows:
We construct the effective and automatic data pipeline, AesCoT, which builds high-quality aes- thetic reasoning data along five dimensions based on original image-score pairs, significantly re- ducing labor cost. We will release our AesCoT-3K and AesCoT-10K to facilitate future research in IAA.
We develop a two-stage training pipeline, Aes-R1, based Selative-Absouuee Policy Optimiza- tion (RAPO) that jointly optimizes absolute score regression and relative ranking, enabling bal- anced modeling of intrinsic merit and comparative preference IAA.
We conducted extensive experiments to demonstrate the performance, robustness and general- ization of Aes-RI, which achieves fftte-of-the-ann aht performance across multiple IAA benchmarks with only 15K training samples. We also analyze the reward design and training recipe for SFT and RI, offering valuable insights.
RELATED WORK
Score-Based Image Aesthetic Assessment. Research on Image Aesthetic Assessment (IAA) has
progressed from rule-based and handcrafted pipelines (Bhattacharya al.) 2010; Mavridaki &
Mezaris 2015; Mittal et al., 2013 2012) to deep models supervised by human-annotated mean
opinion scores (MOSs). Deep learning IAA models (Talebi & Milanfar] 2018 Ke al. 2021
Ghosal & Smolic 2022 He et al. 2022 Behrad et al. 2025) became dominant after large-scale
aesthetic datasets Murray et al. 2012] He al.) 2022; Ren al. 2017 Yang et al.] 2022 Kong
et al. 2016) enabled direct pexel-level representation learning via powerful networks. More recently,
pretrained ML.MMs have demonstrated remarkable cross-modal aesthetic perception and evaluation
through visual- language pretraining (Ke et al.] (003)), prompt engineering (Zhu et al. [024 Sheng
ct al. 2025), and supervised fine-tuning (Zhang et al., 2023 Cao et al. 2025 Liao et al., 2025
Zhou et al. 2024). Nevertheless, score-only supervision without explicit linguistic rationales limits aesthetic understanding, weakens generalization and requires redundant dataset- training.
Reinforcement Learning- Enthanee- -odel-as---Juuge. Model-as-a- as----udde approach serves as novel and broadly applicable paradigm for addressing open-ended evaluation problems in the era of
Large Models (Gu et al. (025). Prior works on Model-as-a-Judge (Zheng et al. 2023 Zhu et al.)
2025
Chen et al. 2024 Lee et al. (024) as well as the Generative Reward Models (GRMs) (Mahan
al.
2024) highlight that reasoning step-by-step before verdict improves the stability and consis-
tency
Kim et al. 2024 Saha et al.
2025). Capitalizing on recent advances in post-training (Liu
et al.
2025 Yu al. 2025), various large reasoning judge models have emerged (Liu et al.) 2025
Chen et al. 2023b) Whitehouse et al. 2025) and have become popular fro automated assessment.
Extending image evaluation, researchers move beyond scalar scoring (Wu al. 2023a You al.)
2025 Zhou et al. 2024) by applying reinforcement learning to image quality assessment via step- by-step reasoning (Li et al. 2025) Wu et al., 2025). However, these methods are confined to either binary outcome supervision or rank supervision and have yet to be extended to aesthetic assessment, which demands finer-grained perceptual and semantic understanding capability.
3 METHODOLOGY
There is indeed an absence of a principled visual reasoning framework for IAA, which contributes the scarcity of high-quality aesthetic reasoning data and the misalignment of single-objective op- timization with nuanced human preferences. To address this, we propose our Aes-R1 framework. Specifically, we introduce (i) AesCoT, an aesthetic reasoning data construction pipeline that auto- matically curates diverse, reliable annotations with nulti-diimeniinal aesthetic rationales. (ii) A two-stage training pipeline based on Relative-Absolute Policy Optimization (RAPO) that jointly optimizes pairwise preferences and calibrated absolute scores under unified objective.
3.1 PROBLEM FORMULATION
Given image dataset that contains M image-score pairs D = {(I, 8;), [1, M]). considering pretrained MILM with parameter 0 and policy no(- P), aim to fine-tune the model for ideal
3

---

Published as a conference paper at ICLR 2026
Original Image-Scone Scorr--- Pairs
AcsCoT Data Construction
Acsthetic Reasoning Pairs
QA Pair Generation
Arswen shiikk This lusbicat plocngraph
Light Moral & Narnatvve
Aesthetic Amalysis
sisully stiking, ceptoring
grand mosetion mage
reficted serre.
Compesticn
iacustic
Color
Expesure
fareuiousl
dotas
paake, createg
ho
Great!
loading lone the converping olgo of the lake and the ove lice-atee adde a dyname quaity that keap the vice ai crasase. Lie interkey of auiight and shadan conantaro crlencts.... Thrse sageter
Quesion: The zestece unting patue G
Data Filtering 1C
why?
fivling
Continued
4 Information
babbrced... eningh signifinnlly arowagh miner imperteen maicng stasdozt euarple of lantippe photeggalll lough fills pod ahort of utter eneelience oue erail tones and lick inporativs persectivs vitheice
Senne Guidded
X
Wite thoughtfld es hige that explains the key vinsal stcnggh and walereno; unde with Jedgement of the image that matcho the balden sours.
Mismatch Reasoning
Factual Ervoes
Figure 2: Overview of AesCoT construction pipeline. Starting from original image-score pairs, we mask the continuous aesthetic score and prompt experts to produce CoT critiques along five aesthetic dimensions. Automated checks and human audits then remove any score leakage, reasoning score mismatch, or factual errors, yielding high- nigh-quaitt, interpretable multimodal aesthetic reasoning data.
aesthetic reasoning that provides the comprehensive explanation e and overall aesthetic score for every image in dataset I under prompt P:
T (c. 8) ~ no ()
We utilize policy gradient methods (Sutton et al. 1999) to optimize the MILM by maximiz-
ing the expected cumulative reward H()) on sampled aesthetic reasoning trajectories :: I(0) =
E (- [R(r)]. By performing gradient ascent on [():
VoJ(O) (
R(T)
Vo log (a+ 8e)
(2)
where 0 is selected from the vocabulary space V and st = (I, (I,P, is the state at time step t.
3.2 AESCOT
Most existing datasets (e.g., AVA (Murray et al. 20122) annotate image aesthetics using direct Mean Opinion Scores (MOSs). chain-of-Though (CoT) (Wei et al. (022) has been shown to substan- tially enhance the performance and interpretability by breaking down the reasoning process into intermediate steps (Bi et al., 2025). However, high-quality aesthetic reasoning trajectories require artist-level curation and iterative refinement, making such data rare and costly to scale. Distillation from strong model is an efficient and scalable approach (Guo et al., (025a) and was validated in domains from math reasoning @Jie al.] (023) to visual question answering (Shao et al.] 2024a). Based on these, we propose AesCoT, to our knowledge, the first automatic pipeline for aesthetics reasoning data construction.
We display the construction pipeline of AesCoT (see Fig. 2 For every collected original image- score pair (I,, 8:) in D, we first intentionally conceal the continuous score from outputs and enable the powerful close-sourced MLLM to give aesthetic analysis over five aesthetic dimensions (see Appendix E.Tp (light and shadow, mood and narrative, composition, color, exposure) that match the aesthetic levels, which are divided into bad (0-0.4), fair (0.4-0.7) and good (0.7-1.0) emphasize aesthetic differences. Then we concatenate the aesthetic analysis with the ground truth score, ob- taining Door = ((P,,,, G, si), i [L, M]} where e; is the reasoning trajectory towards s;. Finally, we carefully filter the outputs to eliminate score leakage, reasoning inconsistencies, and other fac- tual errors through automated checks and human audits, generating a reliable aesthetic reasoning dataset D DessCoo F(Dcor) = {(P, Iz, Ca, $2) ||& = 0}, where F is the data filter process and E, -e!cuk, ,e, e.lign elfat) represent different error types. Dataset details can be found in ??.
3.3 RELATIVE- POLICY OPTIMIZATION (RAPO)
Aesthetic evaluation is inherently subjective, shaped by individual preferences, cultural experience, and contextual cues. In real- worldwoldd settings, human judgments typically expressed through com-
4

---

Published as a conference paper at ICLR 2026
Scores
Relative Reward Reyt
Ground Truth
Question: What your
overall rating on the sesthetic of this picture Simage? 7
Ouigut
$11
Fro..
Gooad
Output
$1,2
Image 1
Output
Output
S21
Output
S22
+VD 500000 A.
T2
Image 2
Output
S
Absolute Reward Raas
Output
Sa,1
Policy Model
Output
Image
Output it
Relative-Absolate Policy Optimization
Reference
Model
Figure 3: The training pipeline of Relative Absolute Policy Optimization (RAPO). Given batch of training images, the policy model generates multiple outputs for aesthetic rating questions. RAPO computes both pairwise ranking based relative reward and score regression error based absolute reward for each output, then optimizes the policy model align with human aesthetic preferences.
parative assessments, addition to direct absolute scores, with judgments manifesting along two complementary relative preference (which image is favored) and absolute magnitude (by how much). In view of this, we introduce Relative Absolute Policy Optimization (RAPO), a novel re- inforcement learning algorithm that integrates relative assessment learning with absolute numerical optimization to better align model predictions with human aesthetic preferences.
Fig. 3 describes the training pipeline of RAPO. For a mini training batch {IZI, (Iv, &N)) where the training size is N, we follow GRPO (Shao et al. (202bb to sample group of K outputs {0.1, ..,, Oik for every prompt-image input (P.I, from the old policy =T"
A robust and reliable reward signal is significant for aligning MLLMs with human aesthetic prefer- ences. To this end, RAPO jointly optimizes the relative rank preference as well as the absolute score regression error.
Relative Rank Reward. Learning to rank is crucial paradigm for modeling human preference (Burges et al. 2005 Xia et al. 2008), and has proven effective for perceptual quality and aesthetic assessment Liu et al. [201]] Talebi & Milanfar 2018). Following (Tsai et al. 2007 Sun al. 2024 Wu et al. 2025) Zhang et al. 2023) You et al. 2025), our relative rank reward based on FRank is bounded, continuous, differentiable. and aligns directly with pairwise ranking consistency, and thus provides stable, precise signals for optimizing pairwise image aesthetic preference.
Specifically, we assume that the aesthetic score of an image follows the Gaussian distribution s ( d²). Consequently, the score difference 8; 8j follows N(4 My of of). Let og denote the k-th predicted score for image the pairwise comparison probability against image should be
Piu (I, = 2
Osk 14
)i
o3 +7
where ((.) denotes the standard normal cumulative distribution function. We estimate Ji as the mean of predicted scores of image j. and set small positive constant y to avoid division-by-zero.
Prank(O.)) N - 171
(Z, + (11 (1- (Z,,Z)) (Z),
(4)
where Pe is binary preference label based ground-truth MOSs of two images
11, 8: 2 8;
(I..I,)
(5)
0,
5

---

Published as a conference paper at ICLR 2026
Absolute Error Reward. Although the rank reward reinforces the model to order images rectly, leaves the calibrated scoring underconstrained. To fix this, we incorporate continu- ous absolute-error reward that explicitly calibrates predicted scores toward the ground-truth MOSs, where a is hyperparameter and is a small positive constant to avoid reward being zero.
Oack
8:1
2
Taba (o.a) eap
+
(6)
The total reward function of RAPO is combined with relative rank reward and absolute error reward
Trank Taba-
Training Aes-R1 for Aesthetic Reasoning. To fully inspire the aesthetic reasoning capability of the MILM, we first leverage the constructed AesCoT dataset to fine tune the model as a cold start stage, optimizing the negative log- likelihood of the aesthetic reasoning trajectory and the final score
(0) = (P P.a)) | log (e s|P,I)].
(8)
For the prompt-image pair PI). we calculate the rewards of all sampled outputs, yielding rewards R. = fri, . rx)) J. The advantage Av. for each token time step / in the k-th output given by subtracting the group average and dividing by the standard deviation
AR, = 7k (R(R))
(9)
(R.)
Following DAPO (Yu al. 2025), adopt higher clipping threshold and reduce the KL penalty terms. RAPO optimizes the policy by maximizing the following objective
(O)
= (p.I)) a,- (-1P.)
K
lal
(10)
Σ
min( (l) (lip(rk..(),, flow, Ebigh) As.e) (TL (reet)
K
Osk
where Trof denotes the reference policy of pretrained model, and the probability ratio is defined
Tok Il,, 0x,et)
Tk
nold
(11)
and the KL penalty is approximated by
Tref Trer(ok,d IPII, 0k,
Teat ( 01, TO (ox IP, I, 0x,ct)
DKI(TO |l Tres) =
log
(12)
To (or, P, I, 0c,ct)
EXPERIMENT
4.1 EXPERIMENT SETUP
Dataset and Metrics. We adopt five open-source general image aesthetic assessment datasets:
TAD66K (He et al., 2022), AVA (Murray al. 2012). FLICKR-AES (Ren et al. 2017). PARA Yang et al. 2022), and AADB (Kong et al. 2016). The model is trained on TAD66K, AVA, and Flickr-aes, with PARA and AADB serving as out-of-distribution (OOD) datasets. Mean Opinion Scores (MOSs) across these datasets are normalized into the range [0, 1]. For evaluation, model performance quantified using the Pearson linear correlation coefficient (PLCC) and the Spearman rank-order correlation coefficient (SRCC) between predicted scores and ground-trute annotations. The implementation details can be found in Appendix
Baselines and Implementation. We compare our method with (1) popular LLMs with various sizes, capacities and reasoning modes, including Qwen-2.5-VI. series (Qwen al. 2025) Seed-
1.5-WL-thinking (Guo et al.) 2025b), GPT4o (OpenAl et al.) 2024) and GPT4.1 (OpenAl] 2025):
(2) HandCrafted methods like NIQE MMittal et al. 2013) and BRISQUE (Mittal et al. 20 2); (3)
Deep Learning based methods including MUSIQ (Ke et al. 202 and NIMA (Talebi & Milanfar
2018: (4) MLLM based methods including VILA (Ke et al. 2023), Laion-Aes (Schuhmann et al,
2022). ) Q-Align (Wu al. 2023a), ArtiMuse (Cao et al. 2025),
DeQA-Score (You et al. 2025).
InInsttht (Li et al. 2025) and VisualQuality--R (Wu et al. 2025). The implementation details is
in Appendix

---

Published as a conference paper at ICLR 2026
Model
TAD66K
AVA
FLICKR
PARA
AADB
Average
PLCC
SRCC
PLCC
SRCC
PLCC
SRCC
PLCC
SRCC
PLCC
SRCC
PLCC
SRCC
Vouilla MLLM
Qwen2.5-VL-7B
0.2282
0.2242
0.3518
0.3684
0,5179
0.5696
0.5966
0.6252
0.4480
0.5076
0.4285
0,4589
Qwen2,5-VL-322
0,2300
0.2715
0.3893
0,4170
0.5800
0.6527
0.7176
0.7118
0.5060
0,5138
0.4846
0,5134
Seedl.--VL-thinking
0.2908
0.2974
0.4533
0.4871
0,5837
0.6374
0.6134
0.6364
0.5279
0.5577
0.4938
0.5232
GPT40-2024-08-00
0.2686
0.2979
0,4687
0.4939
0.5469
0.5507
0.6916
0.6719
0.5209
0,5314
0.4993
0,5092
GPT-441-2025.04.44
0.2707
0.3298
0,4846
0.5594
0.5909
0.6062
0.7290
0.7203
0.5102
0.5296
0.5171
0.5491
Hand-Crafted
NIQE [2013
0.1332
0.0811
0.1226
0.0448
0,1024
0.0487
0.3546
0.2478
0.0698
0.0566
0.1565
0.0958
BRISQUI 2012]
0.0294
0.0243
0,0310
0.0360
0,0830
0.0910
0.1288
0.0783
0.0565
0.0558
0.0657
0.0571
Deep Leap-nng Based
NIMA (2018)
0.3885
0.3654
0.6120
0.6361
0,5130
0.4796
0.5868
0,5709
0.3886
0.3904
0.4978
0.4885
MLL.M Rased
Q-Align 2023a
0.3498
0.3627
0,5215
0.5407
0.6231
0.6473
0.6181
0.6262
0.4474
0.4508
0.5120
0.5255
DeQA-Score" 2025
0.3985
0.3885
0.5718
0.5896
0.7057
0.7234
0.6796
0.4859
0.4948
0.5771
0.5663
Q Insight* 2025
0.3980
0.3886
0.5964
0.5898
0.7012
0.6769
0.7745
0.7428
0.5069
0,5184
0.5954
0,5813
VisualQualty-R 22025
0.3082
0.3915
0.4407
0.6195
0.5524
0.6769
0.5376
0.7507
0.3754
0.5363
0.4429
0,5930
Aes-R1
0.4511
0.4248
0.6702
0.6619
0.7243
0.6973
0.7842 0.7666 0.5386 0,5423 0.6337
0.6186
Table 1: PLCC and SRCC results compared to vanilla MILM, hand-crafted, deep-learning based and MLLM based methods. *Results are retrained in our 15K combined train set. (DeQA-Scon is trained on AVA and Flickr-aes in combination, owing the absence of per-image standard deviation data in TAD66K). The best results are highlighted in bold, and the runner-ups are underlined.
PARA
AVA
FLIOK
TAb68K
AADB
Average
1.15
C/M
379
6.0
1:40
6
FLICER
AADS
1.50
cat
1.40
Zear
4.55
vm
6/
MUSIQ
LATON
VILA
QuA4ign
ArttMuse
Ass
Figure 4: PLCC and SRCC results of models trained only on AVA dataset to compare the general- ization ability. Aes-RI is able to achieve performance comparable to models such as ArtiMuse (Cao ct al. 2025), which is pretrained on large-scale -anantan corpus.
4.2 QUANTITATIVE RESULT
Main Results. We adopt combined training set of 15K samples randomly collected from the official train set of AVA, TAD66K and FLICKR-AES in 2:2:1 ratio. Tab. lab.[[indicates several find- ings: 1) Aes-R1 significantly outperforms all baseline methods on in-domain datasets, achieving the highest average PLCC (0.6337) and SRCC (0.6186) scores across five benchmarks, with improve- ments of over 0.2 in both PLCC and SRCC compared to the backbone. 2) Compared to baTeb methods, the efficient and scalable RL-based methods demonstrate superior performance, highlight- trained on the
ing the promise of our Aes-RI framework in IAA. 3) Although VisualQuality- same data, successfully separates high- and low- low-quality images using rank reward (as shown in Fig. [ and achieves a high SRCC (0.5825), fails to calibrate scores to the empirical distribu- tion (PI.CC 0.4429). This highlights the rationale behind Aes-Rl's 's aelative-absolute reward design, which achieves the closest alignment with the ground-truth distribution, featuring a well-matched peak and reduced tail deviation. We further conduct reward ablations in Sec. 4.3
Generalization Ability. In Tab. I Aes-RI attains strong OOD performance on PARA and AADB trained on combined datasets compared to MLLM based baselines. We further validate Aes-R1's 's generalization in single dataset setting by training Qwen-2.5-VL Instruct on the AVA train set and compare it with the official baseline checkpoints on the test set. As shown in Fig. Aes-RI achieves the highest SRCC and the second-highest PLCC on average. Note that ArtiMuse is pretrained on a large-scale
corpus. The results indicate RL with appropriate feedback can achieve strong cross- task generalization without the need for manually annotated data. Detailed results can be found in Appendix D.I
7

---

Published as a conference paper at ICLR 2026
Binary reward
Emor
Rank reward
TAD66K
AVA
FLICKR
PARA
AADB
Average
reward
PLCC
SRCC
PLCC
SRCC
PLCC
SRCC
PLCC
SRCC
PLCC
SRCC
PLCC
SRCC
0.2286
0.2414
0.3872
0,4120
0.5020
0.5336
0.6109
0.6162
0.3989
0.4133
0,4255
0.4433
0.3981
0.3872
0.5928
0,5895
0,7019
0,6829
0.7034
0.7023
0,4313
0,4379
0.5655
0.5600
x
0.3170
0.3638
0.5098
0.5956
0.5550
0.6945
0.5171
0.2582
0.3719
0.5421
0.4542
0.5908
0.3884
0.3785
0.5963
0.5932
0.6996
0.6780
0.7694
0.7452
0.5284
0.5176
0,5964
0.5825
0,4147
0,3915 0.6294 0.6242
0.7422
0.7173
0.8192
0.7813
0.5428
0,5369
0.6297 0.6102
Table 2: Ablation studies of different reward combinations. Our RAPO proposed error-rank reward
significantly outperforms others.
SFT(epoch)
RL
Average Entropy
TAD66K
AVA
FLICKR
PARA
AADB
Average
PLOCC
SRCC
PI.CC
SRCC
P1,CC
SRCC
PLCCC
SRCC
PL.CC
SRCC
PLOCC.
SRCC
N/A
0.921
0.2282
0.2242
0,3518
0.3684
0.5179
0.5696
0.5966
0.6252
0,4480
0.5076
0.4285
0.5490
O
RAPO
0.961
0.4147
0.3915
0.6294
0.6242
0.7422
0.7173
0.8192
0.7813
0,5428
0,5369
0.6297
0.6102
N/A
1.609
0.2570
0.2548
0,3755
0,3734
0.5032
0,4731
0.5561
0.3416
0.4097
0.4084
RAPO
1.626
0.4513
0.4248
0.6702
0.6619
0.7243
0.6973
0.7842
0.7666
0.5386
0,5423
0.6337
0.6186
N/A
1.417
0.3215
0,3214
0,4652
0,4602
0.5209
0.5170
0.5941
0.4164
0.4624
0.4705
2
RAPO
1.391
0.4460
0.4326
0.6592
0.6588
0.6822
0.6667
0.7777
0.7290
0,4486
0.4663
0.6027
0.5903
N/A
0.705
0.3325
0.3248
0,4752
0,4730
0.5745
0.5594
0.6045
0.4354
0.4847
0,4903
10
RAPO
0.716
0.3215
0.3214
0,4652
0.4602
0.5209
0.5170
0.5941
0.6377
0.4104
0.4164
0.4624
0.4705
Table 3: The performance when RAPO is initialized from SFT checkpoints (0, 1, 2. 10 epochs), together with the average token entropy of the starting policy. Moderate SFT maximizes downstream performance, while excessive SFT declines entropy and RL gains, hindering OOD performance.
4.3 ABLATION STUDY
Reward Function Analysis. We conduct ablation studies on the reward function design, the key module of our RAPO algorithm. We investigate three kinds of reward designs: binary reward, relative rank reward (Bq. 4 and absolute error reward (Eq. (6). The binary reward was used in eLi al. 2025), which only supervises the outcome
f1 [Spred - Sgel <E
Tbinary = {o otherwise
(13)
We train the backbone on the 15K combined dataset via direct reinforcement learning without the cold-start stage to analyze the reward configurations. As summarized in Tab. [ single-objective rewards exhibit distinct behaviors. The error reward markedly outperforms the binary reward, as it provides a continuous and precise reward signal that aligns with the score distribution. Addition- ally, the rank-only reward optimizes relative ordering effectively, yielding high SRCC but depressed PLCC, which matches the results in Tab. [ reflecting the imbalance in predicting aesthetic scores.
Compared with single-objective rewards, the combining objectives improves the generalization pability, resulting in stronger performance in the -utto--distribution (OOD) datasets. Notably, the Aes-R1 configuration (Error- Rank) achieves best average PLCC and SRCC across all benchmarks.
Analysis of SFT and RL, We first attempt RAPO training without SFT on the AesCoT dataset (Aes-R1-Zero) Guo et al. (2025a). As illustrated in Fig. (6 Aes-R1-Zero achieves reasonable scoring accuracy but generates superficial and generic explanations, posing potential reward-hacking risks (Gao et al. 2023). This indicates the model lacks sufficient aesthetic prior, while recent studies (Cuu et al. 2025b) highlight the importance of SFT in providing a strong prior and stabilizing output formats. Consequently, Aes-R1 framework consists of two training stages, SFT and RL. To find the reasonable allocation between them, we conduct initialization experiments. As shown in Tab. the average PLCC/SRCC increased significantly from 0.4097/0.4084 to 0.4847/0.4903 while the average token entropy decreased from 1 1.609 to 0.716 after 1 epoch, 2 epochs and 10 epochs SFT. We then train RAPO based on above SFT checkpoints.
On the RL side, maintaining appropriately high entropy can foster exploration, stabilize training and mitigate entropy collapse (Cui et al., 22025a; Wang et al.) 2025a Cheng et al., 2025), which consis- tent with the RAPO results, where checkpoints with lower performance but higher entropy exhibit significant improvements. Excessive SFT leads to overfitting on the data and narrows the policy exploration, resulting in only marginal gains after RL. Therefore, in this paper. we set I epoch cold- start, which achieves the best scoring performance of 0.6337/0.6. 6166 during reinforcement learning, well expert-level aesthetic analysis (Aes-R1 in Fig. 6)
8

---

Published as a conference paper at ICLR 2026
Relative
Absolute
TAD66K
AVA
FLICKR
PARA
AADB
Average
PLCC
SRCC
PLCC
SRCC
PLCC
SRCC
PLCC
SRCC
PLCC
SRCC
PLCC
SRCC
1.0
0
0.3170
0,3638
0.5098
0.5956
0.555
0.6945
0.5171
0.7582
0.3719
0.5421
0.4542
0.5908
0.8
0.2
0,4059
0,4263
0.5596
0.6464
0.6318
0.7596
0.6752
0.8392
0.5041
0.5966
0.5553
0.6536
0.6
0.4
0.3986
0.4222
0.6274
0.6317
0.7075
0.7328
0.7750
0.7993
0.5305
0.5422
0.6078
0.6256
0.5
0.5
0,4147
0.3915
0,6294
0.6242
0.7422
0.7173
0.8192
0.7813
0.5428
0.5369
0.6297
0.6102
0.4
0.6
0.4282
0,4082
0.6478
0.6234
0.7254
0.6984
0.7914
0.7660
0.5432
0.5278
0.6272
0.6048
0.2
0.8
0.4251
0.3885
0.6492
0.6392
0.7280
0.6990
0.7764
0.7635
0.5291
0.5185
0.6216
0.6017
0
1.0
0.3981
0,3872
0,5928
0,5895 0,7019
0.6829 0.7034 0,7023 0,4313
0.4379 0.5655
0,5600
Table 4: The performance when changing weighting coefficient of the RAPO reward.
In contrast, extending SFT training to 2 up to 10 epochs leads to depressed entropy and limited exploration, and initializing RAPO results in worse performance and poorer generalization capabil- ity. Finally, we choose to train Aes-R1 with only epoch cold-start to cultivate aesthetic reasoning patterns, followed by RAPO training that inspires effective self-xxploration. Compared with meth- ods that rely on large volumes of constructed data, our Aes-R1 training pipeline achieves strong results while overcoming the prohibitive cost of data collection and curation. Detailed analysis of entropy can be found in Appendix D.2
Analysis of Weighting Coefficients in RAPO relative rank reward and the absolute error reward, we conduct ablation experiments. We change the weighting coefficients in the RAPO algorithm to explore the mutual relationship between the two reward components. The reward function can be written as:
To better understand the interplay between the
ArankTrank AuhxTubh.
(14)
As shown in Tab. we retrain the model using different reward configurations. More intuitively, visualize the trends of the PLCC and SRCC metrics as function of the weighting coefficients in
Fig. 5 The results reveal several compelling observations: When relying solely on the relative rank or absolute error reward (Arank = 1.0 or Xahs 1.0), the model achieves significant performance degradation. Besides, when both applying relative and absolute reward, it's worth noting and com- pelling that the two reward components exhibit complementary characteristics. As shown in Fig. [5] PLCC steadily increases as we shift weight from more relative reward (Arank = 0.8) toward more balanced configurations (Arank = 0.4 0.5), while SRCC peaks when Arank 0.8 and gradually declines as we increase Aabs- This inverse relationship likely stems from the distinct optimization objectives of the two metries: SRCC emphasizes preservation rank-order and thus benefits more from the relative ranking reward, whereas PLCC measures linear correlation and requires accurate absolute score calibration to minimize regression error. We find that the optimal performance is achieved at balanced weighting around Arank = 0.3 and Aubs = 0.5, where both PLCC (0.6297) and SRCC (0.6102) maintain strong performance without extreme trade-offs. This empirically vali- dates our design of RAPO and demonstrates that jointly optimizing relative preference and absolute calibration provides more comprehensive supervision than either component in isolation, enabling the model to obtain reliable and stable reward signals.
4.4 CASE STUDY
To further illustrate the advantages of Aes-RI, conduct case study on image of chick emerging from an egg with a ground-trute aesthetic score of 0.613. As shown in Fig. ( previous methods (e.g., 1. Qwen 2.5-7B-VL, Q-insight, and VisualQuality-RI consistently assign in- flated scores (0.82-0.91). They focus on surface-level attributes such as color vibrancy, balance, and soft lighting, while neglecting constructive critique. For instance, G commends the com- position and texture, yet briefly mentions that the focus could be clearer. Similarly, Qwen and VisualQuality-RR describe the image as welll-balanced' and 'appealing', without highlighting any significant weaknesses. This pattern reflects lack of nuanced, multidimensional judgment. In con- trast, A provides more balanced assessment, producing a score (0.59) closer to the ground truth, and offering both positive feedback and constructive criticism. Aes-R1 highlights although the image is technically competent, lacks a strong emotional impact or creative innovation. Fur- thermore, compared to Aes-RI-Zess Aes-RI further exhibits deeper explanatory capabilities and improved calibration, underscoring the effectiveness of cold-start bootstrapping. This case study highlights Aes-R1's 's closer alignment with human judgment and richer critique-awan aware explanations.

---

Published as a conference paper at ICLR 2026
PLcC/SRCC Performance with Weighted Coefficient
Relative Reward coefficient
0.2
0. 6.25
RLCC
SRCC
0.62
0.60
0.58
0.478
0.450
0.55
0.0
0.2
0.4
0.5
0.6
0.5
1.0
Absolute Reword coefficient
Figure 5: The trends of PLCC and SRCC metries with different weighting coefficients in RAPO reward. While relying solely on the relative rank or absolute error reward leads to significant perfor- mance degradation, the two reward components exhibit complementary characteristics when apply- ing both rewards.
GT Score: 0.613
GPT4.1
Score:: X
Qwen 2.5-7B-VL. Score: 0.85
Answer: This image cleverly justapeses a bright yellow mansmallow chick inside cracked white egpstell, depth
Anewer: The image captures cate and charminng moment yellow chick emerging shurp the chick,
whoie white egps The compueiion shallow
giving soft drawing background,
playful subjess and
with
background
acsbetic by making main subject pop.
contrax
emphasizes the subject. The are bright and sppealing, with the yellow
betwer the yellene chick and mired whites craates risual interest and whimsy, while the lighting is soft and plessent, highlighting texture. The photo is crcativs, well. composed, and visually engaging, though the focus could be slightly sharper. Overall this is highly aestbeti image just minor technical things improve.
chick contrasting nicely against the white of the egz. The compusitsos musiioon is well balunced, and the overall mood warm and inviting This image is visualy pleasing and captures deligletal scene.
Over-Apprnisal
Q-insight Score:: 0.82
AesR1-Zem @ Score: 0.65
VisualQuality-RI @ o'Score: 0.88
Answer The image features ckeo-sp a ycllow chick with sha depth of field sightly Murs backquound. The focus shapp the chick and qus, prominence. lighting gentle, warm line to the image. composition simple yes effective, drawing the viewer's attention to the chick and the egt, The colors are vibrant, with the yellow of the chick sanding out againa the white of the and the sighty muted backggrund.
Auswer: The image is close up of yellow chick emerging froms an The forus is sharp on the chick, with shallow 4epth of fild buuring the background and other eags The lighting soft and natural, highigghing the chick's uesture and the cat's smooth surface.
emerging from
energling bright
subject's
natural, adding a
composition simple
chiss, The
of compositonn simple effective, capturing chick vibrant. with yellow sanding out against white
life.
focus chick
egg Overall, image visualy appealing and well-composed.
nestral background
Q-Align @ Scoore: 0.72 X
Appraisal
AesRI (Ours) @ Score: 0.59
Crilique
Anower This fettures yellow chick
The compodition thoughtfuuly arranged, with draws viewer's atteation chick, the delicate Rothers the chick. The color palette harmonious, with the tright yellow of the chick sanding out againe the mured whoc and Huuc the backgmand This contrast mith the
sottly Huredd bickground
positoned
surrondiigg
while bodigrouad sutficieull Murred avoid distroctios
highightng
visual interest and emphasiees sujject Technically, the tocus The emotional impart gentle and whimiicll, evoking narmelie claxate further The wander new
well-executed,
where
partoularty the chick's head beak, could
the imagr pleasingy beneflt from
Howeerr perpective, eficeefeetive fools somrwhut starls, and the sorne lacks elear focal point beyond the chick. Addiiomally, the lightnng thaugh rof, could be more dramatic to add depth and drama. Ovrral, this phonograph demuratnics solid technical skill and appealing acehetis, but of emntionnl punch. la viuully pleasing image wish notuble strngghs, though leancs roum fir imprrveent in cruativity and impact.
dynamic angle stronger
just sbort of being truly exceptional due its sumew hut conventional compusition and lack
Figure 6: Beyond aver-appraisal: in comparison with existing models, Aes-RI excels through bal- anced and multi-diimnssonal aesthetic assessments
CONCLUSION
In this work, we introduce Aes-R1, an advanced framework for aesthetic assessment that combines quantitative evaluation with qualitative critique. Unlike existing approaches, which primarily focus on numerical prediction, Aes-RI captures multidimensional aesthetic judgments, incorporating af- firmative and critical perspectives. Extensive experiments across diverse datasets demonstrate that Aes-RI achieves superior performance in numerical assessment and provides interpretable, critique- aware evaluations. This dual capability highlights its potential as a foundation for future aesthetics judgment research, bridging the gap between objective metrics and human-like aesthetic reasoning.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENT
The authors wish to thank the amonymous reviewers for their helpful comments. This work was partially funded by National Natural Science Foundation of China (No. 62576106, 6247606 I, 62376061), Shanghai Rising-Star Program (23QA1400200), and Natural Science Foundation of Shanghai (23ZR1 1403500).
ETHICS STATEMENT
All authors have read and adhere to the ICLR Code Ethics and explicitly acknowledge compli- ance with its requirements during the submission process. Our implementation and data processing pipelines do not involve any collection, storage, or exposure of personal, sensitive, or proprietary information, and fully comply with applicable privacy and security regulations. All datasets used are either publicly available under appropriate licenses or have been obtained with explicit permission, ensuring legal and ethical compliance. Comprehensive bias and fairness assessments have been con- ducted, and no evidence of discriminatory behavior or disparate performance was observed across different demographic or attribute groups
REPRODUCIBILITY STATEMENT
To facilitate reproducibility, the complete source code is available in anonymous repository. De- tailed descriptions of dataset preprocessing, model hyperparameters, and training procedures are included in the appendix to support independent replication.
REFERENCES
Fatemeh Behrad, Tinne Tuytelaars, and Johan Wagemans. Charm: The missing piece in vit fine- tuning for image aesthetic assessment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7824-7824, 06 2025. doi: 10. 10.1109/CVPR52734
2025.00732.
Subhabrata Bhattacharya, Rahul Sukthankar, and Mubarak Shah. A framework for photo-quality assessment and enhancement based on visual aesthetics. Proceedings of the 18th ACM Inter- national Conference Multimedia, MM '10 pp. 271-280, New York, NY, USA, 2010. Associ- ation for Computing Machinery. ISBN 9781605589336. doi:
015511883351111879999 URL
https::/doi. /ttp////...//..//////// httpg//0.1115//110 73951. 1873990
Jing Bi, Susan Liang, Xiaofei Zhou, Pinxin Liu, Junjia Guo, Yunlong Tang, Luchuan Song, Chao Huang, Guangyu Sun, Jinxi He, et Why reasoning matters? a survey of advancements in multimodal reasoning (vl). arXiv preprint 25Xiv:2504.0311 2025.
Chris Burges, Tal Shaked, Erin Renshaw, Ari Lazier, Matt Deeds, Nicole Hamilton, and Greg Hullender. Learning rank using gradient descent. In Proceedings of the 22nd International Conference on Machine Learning, ICML '05, pp. 89-96, New York, NY, USA, 2005. Associ- ation for Computing Machinery. ISBN 1595931805. doi: 10.1145/110231.11100 https: //dol. www.oo..oo...o.oooooooooo.......
URL
Shuo Cao, Nan Ma, Jiayang Li, Xiaohui Li, Lihao Shao, Kaiwen Zhu, Yu Zhou, Yuandong Pu, Jiarui Wu, Jiaquan Wang, Bo Qu, Wenhai Wang. Yu Qiao, Dajuin Yao, and Yihao Liu. Artimuse: Fine- grained image aesthetics assessment with joint scoring and expert-level understanding. arXiv preprint arXiv:2507.1 0tt::2557..44.... 14533, 2025.
Dongping Chen, Ruoxi Chen, Shilin Zhang, Yaochen Wang, Yinuo Liu, Huichi Zhou, Qihui Zhang, Yao Wan, Pan Zhou, and Lichao Sun. Milm-as-ajjudge: assessing multimodal ilm-as-a-judge with vision-language benchmark. In Proceedings of the 41st International Conference on Machine Learning, JCML'2 224. IMLR.org, 2024.
Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. Sft or rl? an early investigation into training rl rl-like reasoning large rision-langueee models. arXiv preprint 2504 |468, 2025a.
11

---

Published as a conference paper at ICLR 2026
Nuo Chen, Zhiyuan Hu, Qingyun Zou, Jiaying Wu, Qian Wang. Bryan Hooi, and Bingsheng He. Judgelrm: Large reasoning models as a judge. arXiv preprint Trxiv:2504.00055 2025b.
Daixuan Cheng, Shaohan Huang, Xuekai Zhu, Bo Dai, Wayne Xin Zhao, Zhenliang Zhang, and Furu Wei. Reasoning with exploration: An entropy perspective on reinforcement learning for llms. arXiv preprint arXiv:2506. 14758, 2025.
Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma. Sft memorizes, generalizes: comparative study of foundation model post-training. arXiv preprint arXivv2501.17166 2025a.
Tianzhe Chu, Yuexiang Zhai, Jihan Yang. Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V. Le, Sergey Levine, and Yi Ma. Sft memorizes, rl generalizes: A comparative study of foundation model post-training arXiv preprint arXiv: 2025b.
Ganqu Cui, Yuchen Zhang, Jiacheng Chen, Lifan Yuan, Zhi Wang, Yuxin Zuo, Haozhan Li, Yuchen Fan, Huayu Chen, Weize Chen, Zhiyuan Liu, Hao Peng, Lei Bai, Wanli Ouyang, Yu Cheng, Bowen Zhou, and Ning Ding. The entropy mechanism of reinforcement learning for reasoning language models. arXiv preprint arXiv:2505.22611 2025a.
Ganqu Cui, Yuchen Zhang, Jiacheng Chen, Lifan Yuan, Zhi Wang, Yuxin Zuo, Haozhan Li, Yuchen Fan, Huayu Chen, Weize Chen, et al. The entropy mechanism of reinforcement learning for reasoning language models. arXiv preprint arXiv:2505. 2025b.
Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D Goodman. Cogni- tive behaviors that enable self-improving reasoners, or, four habits of highly effective stars. arXiv preprint ar 200i::20003030000 2025.
Leo Gao, John Schulman, and Jacob Hilton. Scaling laws for reward model overoptimization. In International Conference on Machine Learning, pp. 10885-10866. PMLR, 2023.
Koustav Ghosal and Aljosa Smolic. Image Aesthetics Assessment Using Graph Attention Network In 2022 26th International Conference on Puttern Recognition (ICPR), pp. 3160-3167, Los Alamitos, CA, USA, August 2022. IEEE Computer Society. doi: 10.1 10.1109//CPR3631...00 20022. 9956162. URL https: / /oi. 2022. 9956162
Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, Saizhuo Wang, Kun Zhang, Yuanzhuo Wang, Wen Gao, Lionel Ni, and Jian Guo. A survey on lmm-as-a-judge. arXiv preprint arXiv:2411.15508 2025.
Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhi- hong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang. Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Hanwei Xu, Honghui Ding. Huazuo Gao, Hui Qu Hui Li, Jianzhong Guo, Ji- ashi Li, Jingchang Chen, Jingyang Yuan, Jinhao Tu, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaichao You, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang, Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Mingxu Zhou, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang. Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du. Ruiqi Ge. Ruisong Zhang, Ruizhe Pan, Runji Wang. R. Chen, R. L. Jin. Ruyi Chen. Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye. Shiyu Wang. Shuip- ing Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Tao Yun, Tian Pei, Tianyu Sun, T. Wang. Wangding Zeng, Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang, W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang. Xiaokang Chen. Xi- aotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xiaosha Chen, Xiaowen Sun, Xiaox- iang Wang, Xinnan Song, Xinyi Zhou, Xianzu Wang, Xinxia Shan, Y. K. Li, Y. Q. Wang, Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan,
12

---

Published as a conference paper at ICLR 2026
Yiyang Ma, Yiyuan Liu, Yongqiang Guo. Yuan Ou, Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yumxian Ma. Ying Tang, Yukun Zha, Yut- ing Yan, Z. 7. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie. Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. DeepSeck-RI incentivizes reasoning in LLMs through reinforcement learning. Nature, 645(8081):6333 638, September 2025a. ISSN 1476-4687. 10.00...3338000 001///445566-000 025
doi: 10. (0.1038/s41586-00 586-025-09942--
URL wttpthtttp/oooooooo
Dong Guo, Faming Wu, Feida Zhu, Fuxing Leng. Guang Shi, Haobin Chen, Haoqi Fan, Jian Wang, Jianyu Jiang, Jiawei Wang. Jingji Chen, Jingjia Huang. Kang Lei, Liping Yuan, Lishu Luo, Pengfei Liu, Qinghao Ye, Rui Qian, Shen Yan, Shixiong Zhao, Shuai Peng, Shuangye Li, Si- hang Yuan, Sijin Wu, Tianheng Cheng, Weiwei Liu, Wenqian Wang, Xianhan Zeng. Xiao Liu, Xiaobo Qin, Xiaohan Ding, Xiaojun Xiao, Xiaoying Zhang, Xuanwei Zhang. Xuehan Xiong, Yanghua Peng. Yangrui Chen, Yanwei Li, Yanxu Hu, Yi Lin, Yiyuan Hu, Yiyuan Zhang, Youbin Wu, Yu Li. Yudong Liu, Yue Ling. Yujia Qin, Zanbo Wang. Zhiwu He, Aoxue Zhang. Bairen Yi, Bencheng Liao, Can Huang, Can Zhang. Chaorui Deng. Chaoyi Deng. Cheng Lin, Cheng Yuan, Chenggang Li, Chenhui Gou, Chenwei Lou, Chengzhi Wei, Chundian Liu, Chunyuan Li, Deyao Zhu, Donghong Zhong. Feng Li, Feng Zhang. Gang Wu, Guodong Li, Guohong Xiao, Haibin Lin, Haihua Yang. Haoming Wang, Heng Ji, Hongxiang Hao, Hui Shen, Huixia Li, Jiahao Li, Jialong Wu, Jianhua Zhu, Jianpeng Jiao, Jiashi Feng, Jiaze Chen, Jianhui Duan, Jihao Liu, Jin Zeng, Jingqun Tang, Jingyu Sun, Joya Chen, Jun Long, Junda Feng. Junfeng Zhan, Junjie Fang, Junting Lu, Kai Hua, Kai Liu, Kai Shen, Kaiyuan Zhang, Ke Shen. Ke Wang, Keyu Pan, Kun Zhang, Kunchang Li, Lanxin Li, Lei Li, Lei Shi, Li Han, Liang Xiang. Liangqiang Chen, Lin Chen, Lin Li, Lin Yan, Liying Chi, Longxiang Liu, Mengfei Du, Mingxuan Wang, Ningxin Pan, Peibin Chen, Pengfei Chen, Pengfei Wu, Qingqing Yuan, Qingyao Shuai, Qiuyan Tao, Renjie Zheng, Renrui Zhang, Ru Zhang, Rui Wang, Rui Yang, Rui Zhao, Shaogiang Xu, Shihao Liang, Shipeng Yan, Shu Zhong, Shuaishuai Cao, Shuangzhi Wu, Shufan Liu, Shuhan Chang, Songhua Cai, Tenglong Ao. Tianhao Yang, Tingting Zhang, Wanjun Zhong, Wei Jia, Wei Weng. Weihao Yu, Wenhao Huang, Wenjia Zhu, Wenli Yang. Wenzhi Wang. Xiang Long, XiangRui Yin, Xiao Li, Xiaolei Zhu, Xiaoying Jia, Xijin Zhang. Xin Liu, Xinchen Zhang. Xinyu Yang. Xiongcai Luo, Xiuli Chen, Xuantong Zhong, Xuefeng Xiao, Xujing Li, Yan Wu, Yawei Wen, Yifan Du, Yihao Zhang, Yining Ye, Yonghui Wu, Yu Liu, Yu Yue, Yufeng Zhou, Yufeng Yuan, Yuhang Xu, Yuhong Yang. Yun Zhang. Yunhao Fang. Yuntao Li, Yurui Ren, Yuwen Xiong. Zehua Hong, Zehua Wang, Zewei Sun, Zeyu Wang. Zhao Cai, Zhaoyue Zha, Zhecheng An. Zhehui Zhao, Zhengzhuo Xu, Zhipeng Chen, Zhiyong Wu, Zhuofan Zheng, Zihao Wang, Zilong Huang, Ziyu Zhu, and Zuquan Song. Seed1.5-v technical report. arXiv preprint 2505.:2000000022 2025b.
Shuai He, Yongchang Zhang, Rui Xie, Dongxiang Jiang, and Anlong Ming. Rethinking image aesthetics assessment: Models, datasets and benchmarks. IJCAI, 2022.
Yipo Huang, Xiangfei Sheng, Zhichao Yang. Quan Yuan. Zhichao Duan, Pengfei Chen, Leida Li, Weisi Lin, and Guangming Shi. Acsexpert: Towards multi-modality foundation model for image aesthetics perception. In Proceedings of the 32nd ACM International Conference on Multimedia, MM '24, pp. 5911-5920, New York, NY, USA, 2024. Association for Com- puting Machinery. ISBN 9798400706868. doi: 10.1 10.1145/366444733666666
URL https:
7/doi. 10. Sew...1115/3611111 3680644
Miaomiao Ji, Yanqiu Wu, Zhibin Wu, Shoujin Wang, Jian Yang, Mark Dras, and Usman Naseem. A survey on progress in I1m alignment from the perspective of reward design. arXiv preprint arXii:2505.02666. 2025.
Zhanming Jie, Trung Quoe Luong, Xinbo Zhang, Xiaoran Jin, and Hang Li. Design of chain-of- thought in math problem solving, arXiv preprint arXiv:2309, 0ett2299911104 11044, 2023.
Immanuel Kant. Critique of judgment, volume 10. Minerva Heritage Press, 1790.
Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar, and Feng Yang. Musiq: Multi-scale image quality transformer. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 2021. doi: 10. 10.11199/000 00.109//CCCVA9922222....05
13

---

Published as a conference paper at ICLR 2026
Junjie Ke, Keren Ye, Jiahui Yu, Yonghui Wu, Peyman Milanfar, and Feng Yang. Vila: Learn ing image aesthetics from user comments with lasoun-angguge pretraining. arXiv preprint arXiv:2303.14302 2023.
Seungone Kim, Juyoung Suk, Shayne Longpre. Bill Yuchen Lin, Jamin Shin, Sean Welleck, Gra- ham Neubig, Moontae Lee, Kyungjae Lee, and Minjoon Seo. Prometheus 2: An open source language model specialized in evaluating other language models. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Meth- ods Natural Language Processing, pp. 4334 4353, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10. 1..18633/1/222..mm//p mai.emmmmm............. URL
httip::/Eclaanthooooy..ooo wwtphhhwwwooo....//o/////////// wt2. emnlp ae--mainn 248/
Shu Kong, Xiaohui Shen, Zhe Lin, Radomir Mech, and Charless Fowlkes. Photo aesthetics ranking network with attributes and content adaptation. arXiv preprint arXiv: 1606.01121 2016.
Seongyun Lee, Seungone Kim, Sue Park, Geewook Kim, and Minjoon Seo. Prometheus- Vision-languagg model as a judge for eme-grained evaluation. In Findings of the association for computational linguistics ACL 2024, pp. 12286-1131,, 2024.
Weiqi Li, Xuanyu Zhang, Shijie Zhao, Yabin Zhang, Junlin Li, Li Zhang, and Jian Zhang. Q-insight: Understanding image quality via visual reinforcement learning. arXiv preprint arXhv:2503.2267, 2025.
Zhichao Liao, Xiaokun Liu, Wenyu Qin, Qingyu Li, Qiulin Wang, Pengfei Wan, Di Zhang, Long Zeng, and Pingfa Feng. Humanaesexpert: Advancing a multi-modality foundation model for human image aesthetic assessment. arXiv preprint arXiv:2503..23900 2025.
Xialei Liu, Joost Van De Weijer, and Andrew D. Bagdanov. Rankiqa: Learning from rankings for no-reference image quality assessment. In 2017 IEEE International Conference on Computer Vision (ICCV), pp. 1040-1049, 2017. doi: 10. 10.11/9C..2201..11188
Zijun Liu, Peiyi Wang, Runxin Xu, Shirong Ma, Chong Ruan, Peng Li, Yang Liu, and Yu Wu. Inference-time scaling for generalist reward modeling. arXiv preprins arXiv:2504.02444 024495, 2025.
Dakota Mahan, Duy Van Phung, Rafacl Rafailov, Chase Blagden, Nathan Lile, Louis Castricato, Jan-Philipp Fränken, Chelsea Finn, and Alon Albalak. Generative reward models. arXiv preprint 2rXi::2410.12832. 2024.
Eftichia Mavridaki and Vasileios Mezaris. A comprehensive aesthetic quality assessment method for natural images using basic rules of photography. In 2015 IEEE International Conference on Image Processing (ICIP), pp. 887-891, 2015. doi: N0.1109/CIIP22015533550
Anish Mittal, Anush Krishna Moorthy, and Alan Conrad Bovik. No-reference image quality assess- ment in the spatial domain. IEEE Transactions on Image Processing, 21(12):4695-4777 2012. doi: 10. 10.11009122201222214005
Anish Mittal, Rajiv Soundararajan, and Alan C. Bovik. Making a "completely blind" image qual- ity analyzer. IEEE Signal Processing Letters, 20(3):209-212, 2013. doi: 10.1 00.1109/1.P.2012. 2227726.
Naila Murray, Luca Marchesotti, and Florent Perronnin. Ava: large-scale database for aesthetic visual analysis. In 2012 IEEE Conference Computer Vision and Pattern Recognition, pp- 2408-2415, 2012.
doi: 10. 11110///CVPR.211222449999
OpenAl. Introducing GPT-4.1 in the api, April 2025. http::://ppeai.ccomm/ g-t-4-1/
OpenAl, :, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander Mądry, Alex Baker- Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, Alex Nichol, Alex Paino, Alex Renzin, Alex Tachard Passos, Alexander Kirillov, Alexi Christakis. Alexis Con- neau, Ali Kamali, Allan Jabri, Allison Moyer, Allison Tam, Amadou Crookes, Amin Tootoochian, Amin Tootoonchian, Ananya Kumar, Andrea Vallone, Andrej Karpathy, Andrew Braunstein,
14

---

Published as a conference paper at ICLR 2026
Andrew Cann, Andrew Codispoti, Andrew Galu, Andrew Kondrich, Andrew Tulloch, Andrey Mishchenko, Angela Back, Angela Jiang, Antoine Pelisse, Antonia Woodford, Anuj Gosalia, Arka Dhar, Ashley Pantuliano, Avi Nayak, Avital Oliver, Barret Zoph, Behrooz Ghorbani, Ben Leimberger, Ben Rossen, Ben Sokolowsky, Ben Wang, Benjamin Zweigg Beth Hoover, Blake Samie, Bob McGrew, Bobby Spero, Bogo Giertler, Bowen Cheng, Brad Lightcap, Brandon Walkin, Brendan Quinn, Brian Guarraci, Brian Hsu, Bright Kellogg, Brydon Eastman, Camillo Lugaresi, Carroll Wainwright, Cary Bassin, Cary Hudson, Casey Chu, Chad Nelson, Chak Li, Chan Jun Shern, Channing Conger, Charlotte Barette, Chelsea Voss, Chen Ding. Cheng Lu, Chong Zhang, Chris Beaumont, Chris Hallacy, Chris Koch, Christian Gibson, Christina Kim, Christine Choi, Christine Mcleavey, Christopher Hesse, Claudia Fischer, Clemens Winter, Coley Czarnecki, Colin Jarvis, Colin Wei, Constantin Koumouzelis, Dane Sherburn, Daniel Kappler, Daniel Levin, Daniel I Levy, David Carr, David Farhi, David Mely, David Robinson, David Sasaki, Denny Jin, Dev Valladares, Dimitris Tsipras, Doug Li, Duc Phong Nguyen, Duncan Findlay, Edede Oiwoh, Edmund Wong. Ehsan Asdar, Elizabeth Prochl, Elizabeth Yang, Eric Antonow, Eric Kramer, Eric Peterson, Eric Sigler, Eric Wallace, Eugene Brevdo, Evan Mays, Farzad Kho- rasani, Felipe Petroski Such, Filippo Raso, Francis Zhang, Fred von Lohmann, Freddie Sulit, Gabriel Goh, Gene Oden, Geoff Salmon, Giulio Starace, Greg Brockman, Hadi Salman, Haiming Bao, Haitang Hu, Hannah Wong, Haoyu Wang. Heather Schmidt, Heather Whitney, Heewoo Jun, Hendrik Kirchner, Henrique Ponde de Oliveira Pinto, Hongyu Ren, Huiwen Chang, Hyung Won Chung. Ian Kivlichan, Ian O' Connelll Ian O' O'Comnell, Ian Osband, Ian Silber, Ian Sohl, Ibrahim Okuyucu. Ikai Lan, Ilya Kostrikov, Ilya Sutskever, Ingmar Kanitscheider, Ishaan Gulrajani, Ja- cob Coxon, Jacob Menick, Jakub Pachocki, James Aung, James Betker, James Crooks, James Lennon, Jamie Kiros, Jan Leike, Jane Park, Jason Kwon, Jason Phang, Jason Teplitz, Jason Wei, Jason Wolfe, Jay Chen, Jeff Harris, Jenia Varavva, Jessica Gan Lee, Jessica Shieh, Ji Lin, Ji- ahui Yu, Jiayi Weng, Jie Tang, Jieqi Yu, Joanne Jang, Joaquin Quinonero Candela, Joe Beut- ler. Joe Landers, Joel Parish, Johannes Heidecke, John Schulman, Jonathan Lachman, Jonathan McKay, Jonathan Uesato, Jonathan Ward, Jong Wook Kim, Joost Huizinga, Jordan Sitkin, Jos Kraaijeveld, Josh Gross, Josh Kaplan, Josh Snyder. Joshua Achiam, Joy Jiao, Joyce Lee, Juntang Zhuang, Justyn Harriman, Kai Fricke, Kai Hayashi, Karan Singhal, Katy Shi, Kavin Karthik, Kayla Wood, Kendra Rimbach, Kenny Hsu, Kenny Nguyen, Keren Gu-Leeberrg, Kevin Button, Kevin Liu, Kiel Howe, Krithika Muthukumar, Kyle Luther, Lama Ahmad, Larry Kai, Lauren Itow, Lauren Workman, Leher Pathak, Leo Chen, Li Jing, Lia Guy, Liam Fedus, Liang Zhou, Lien Mamitsuka, Lilian Weng, Lindsay McCallum, Lindsey Held, Long Ouyang, Louis Feuvrier, Lu Zhang, Lukas Kondraciuk, Lukasz Kaiser, Luke Hewitt, Luke Metz, Lyric Doshi, Mada Aflak, Maddie Simens, Madelaine Boyd, Madeleine Thompson, Marat Dukhan, Mark Chen, Mark Gray, Mark Hudnall, Marvin Zhang. Marwan Aljubeh, Mateusz Litwin, Matthew Zeng, Max John- son, Maya Shetty, Mayank Gupta, Meghan Shah, Mehmet Yatbaz, Meng Jia Yang, Mengchao Zhong, Mia Glaese, Mianna Chen, Michael Janner, Michael Lampe, Michael Petrov, Michael Wu, Michele Wang, Michelle Fradin, Michelle Pokrass, Miguel Castro, Miguel Oom Temudo de Castro, Mikhail Pavlov, Miles Brundage, Miles Wang, Minal Khan, Mira Murati, Mo Bavar- ian, Molly Lin, Murat Yesildal, Nacho Soto, Natalia Gimelshein, Natalie Cone, Natalie Stau- dacher, Natalie Summers, Natan LaFontaine, Neil Chowdhury, Nick Ryder, Nick Stathas, Nick Turley, Nik Tezak, Niko Felix, Nithanth Kudige. Nitish Keskar, Noah Deutsch, Noel Bundick, Nora Puckett, Ofrr Nachum, Ola Okelola, Oleg Boiko, Oleg Murk, Oliver Jaffe, Olivia Watkins, Olivier Godement, Owen Campbell-Moore, Patrick Chao. Paul McMillan, Pavel Belov, Peng Su, Peter Bak, Peter Bakkum, Peter Deng, Peter Dolan, Peter Hoeschele, Peter Welinder, Phil Tillet, Philip Pronin, Philippe Tillet, Prafulla Dhariwal, Qiming Yuan, Rachel Dias, Rachel Lim, Rahul Arora, Rajan Troll, Randall Lin, Rapha Gontijo Lopes, Raul Puri, Reah Miyara, Reimar Leike, Renaud Gaubert, Reza Zamani, Ricky Wang, Rob Donnelly, Rob Honsby, Rocky Smith, Rohan Sahai, Rohit Ramchandani, Romain Huet, Rory Carmichael, Rowan Zellers, Roy Chen, Ruby Chen, Ruslan Nigmatullin, Ryan Cheu, Saachi Jain, Sam Altman, Sam Schoenholz, Sam Toizer, Samuel Miserendino, Sandhini Agarwal, Sara Culver, Scott Ethersmith, Scott Gray, Sean Grove, Sean Metzger, Shamez Hermani, Shantanu Jain, Shengjia Zhao, Sherwin Wu, Shino Jo- moto, Shirong Wu, Shuaiqi. Xia, Sonia Phene, Spencer Papay, Srinivas Narayanan, Steve Coffey, Steve Lee, Stewart Hall, Suchir Balaji, Tal Broda, Tal Stramer, Tao Xu, Tarun Gogineni, Taya Christianson, Ted Sanders, Tejal Patwardhan, Thomas Cunninghman, Thomas Degry, Thomas Dimson, Thomas Raoux, Thomas Shadwell, Tianhao Zheng, Todd Underwood, Todor Markov, Toki Sherbakov, Tom Rubin, Tom Stasi, Tomer Kaftan, Tristan Heywood, Troy Peterson, Tyce Walters, Tyna Eloundou, Valerie Qi, Veit Moeller, Vinnie Monaco, Vishal Kuo, Vlad Fomenko,
15

---

Published as a conference paper at ICLR 2026
Wayne Chang, Weiyi Zheng, Wenda Zhou, Wesam Manassra, Will Sheu, Wojciech Zaremba, Yash Patil, Yilei Qian, Yongjik Kim, Youlong Cheng, Yu Zhang. Yuchen Hc, Yuchen Zhang, Yu- jia Jin, Yunxing Dai, and Yury Malkov. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024.
Qwen, :, An Yang. Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang. Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report. arXiv preprint arXiv:2412.151 15115, 2025.
Jian Ren, Xiaohui Shen, Zhe Lin, Radomir Mech, and David J. Foran. Personalized image aesthetics. In The IEEE International Conference Computer Vision (ICCV), Oct 2017.
Swarnadeep Saha, Xian Li, Marjan Ghazvininejad, Jason E Weston, and Tianlu Wang. Learning In Forty-second International
to plan & reason for evaluation with Conference on Machine Learning, 2025. URL https: http:::/ http::::////ppeeieeieeee net/forummid
PNRZpmmÄP
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarezyk, and Jenia Jitsev. Laion-5b: an open large-scale dataset for training next generation image-text models. In Proceed- ings of the 36th International Conference on Neural Information Processing Systems, NIPS '22, Red Hook, NY, USA, 2022. Curran Associates Inc. ISBN 97817 1377778878.8
Hao Shao, Shengju Qian, Han Xiao, Guanglu Song, Zhuofan Zong, Letian Wang, Yu Liu, and Hongsheng Li. Visual cot: Advancing multi-modal language models with a comprehensive dataset and benchmark for chain-of-thought reasoning. In The Thihty-eight Conference Neu- ral Information Processing Systems Datasets and Benchmarks Track, 2024a. URL https: //openzeview.nee oet.://eeeeetoorrrmeaaaaeeeeeea
Rulin Shao, Shuyue Stella Li, Rui Xin, Scott Geng, Yiping Wang, Sewoong Oh, Simon Shaolei Du, Nathan Lambert, Sewon Min, Ranjay Krishna, et Spurious rewards: Rethinking training signals in rlvr. arXiv preprint arXiv:2506. 2025.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi. Haowei Zhang, Mingchuan Zhang, K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathe- matical reasoning in open language models. arXiv preprint arXiv:2402.033300 2024b.
Xiangfei Sheng, Leida Li, Pengfei Chen, Li Cai, and Giuseppe Valenzise. AesPrompt: Zero- shot Image Aesthetics Assessment with Multi-Granularity Aesthetic Prompt Learning. work- ing paper or preprint, July 2025. URL hal-05160413
wat.eehall..cc...... sal.nccence/
Wei Sun, Weixia Zhang, Yuqin Cao, Linhan Cao, Jun Jia, Zijian Chen, Zicheng Zhang, Xiongkuo Min, and Guangtao Zhai. Assessing uhd image quality from aesthetics, distortions, and saliency. arXiv preprint Xiv:2409.007449 2024.
Richard S Sutton, David McAllester, Satinder Singh, and Yishay Mansour. Policy gradient methods for reinforcement learning with function approximation. S. Solla, T. Leen, and K. Müller (eds.), Advances in Neural Information Processing Systems, volume 12. MIT Press, 1999. URL /tt./
File Flle/5565822350000ee/00aaaaaaaa3 http://s.....ppceee.eeeaea////pp
Hossein Talebi and Peyman Milanfar. Nima: Neural image assessment. IEEE Transactions on Image Processing, 27(8):399 4011, August 2018. ISSN 1941-0042. doi: 00.1100// 2018 28318999999
16

---

Published as a conference paper at ICLR 2026
Kimi Team, Angang Du, Bohong Yin, Bowei Xing, Bowen Qu, Bowen Wang, Cheng Chen, Chenlin Zhang, Chenzhuang Du, Chu Wei, Congcong Wang. Dehao Zhang, Dikang Du, Dongliang Wang, Enming Yuan, Enzhe Lu. Fang Li. Flood Sung, Guangda Wei, Guokun Lai, Han Zhu, Hao Ding, Hao Hu, Hao Yang, Hao Zhang, Haoning Wu, Haotian Yao, Haoyu Lu, Heng Wang. Hongcheng Gao, Huabin Zheng, Jiaming [i, Jianlin Su, Jianzhou Wang. Jiaqi Deng. Jiezhong Qiu, Jin Xie, Jinhong Wang, Jingyuan Liu, Junjie Yan, Kun Ouyang, Liang Chen, Lin Sui, Longhui Yu, Meng- fan Dong, Mengnan Dong, Nuo Xu, Pengyu Cheng, Qizheng Gu, Runjie Zhou, Shaowei Liu, Sihan Cao, Tao Yu, Tianhui Song, Tongtong Bai, Wei Song, Weiran He,, Weixiao Huang, Weixin Xu, Xiaokun Yuan, Xingeheng Yao, Xingzhe Wu, Xinhao Li, Xinxing Zu, Xinyu Zhou, Xinyuan Wang, Y. Charles, Yan Zhong, Yang Li, Yangyang Hu, Yanru Chen, Yejie Wang, Yibo Liu, Yibo Miao, Yidao Qin, Yimin Chen, Yiping Bao, Yiqin Wang, Yongsheng Kang, Yuanxin Liu, Yuhao Dong, Yulun Du, Yuxin Wu, Yuzhi Wang, Yuzi Yan, Zaida Zhou, Zhaowei Li, Zhejun Jiang, Zheng Zhang, Zhilin Yang. Zhiqi Huang, Zihao Huang, Zijia Zhao, Ziwei Chen, and Zongyu Lin. Kimi-vl technical report. arXiv preprint arXiv:2504.0744 2025.
Ming-Feng Tsai, Tie- Yan Liu, Tao Qin, Hsin-Hsi Chen, and Wei- WengYing Ma. Frank: ranking method with fidelity loss. In Proceedings of the 30th Annual International ACM SIGIR Con- ference Research and Development in Information Retrieval, SIGIR '07, pp. 383-390, New York, NY, USA, 2007. Association for Computing Machinery. ISBN 9781595935977. doi: 10.1145/127774..2277788 URL /tttps::/doi..rrr/
Shenzhi Wang. Le Yu, Chang Gao, Chujie Zheng, Shixuan Liu, Rui Lu, Kai Dang, Xionghui Chen, Jianxin Yang, Zhenru Zhang, Yuqiong Liu, An Yang. Andrew Zhao, Yang Yue, Shiji Song. Bowen Yu, Gao Huang, and Junyang Lin. Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcement learning for IIm reasoning. arXiv preprint arXiv:2506.0199 2025a.
Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, Zhaokai Wang. Zhe Chen, Hongjie Zhang, Ganlin Yang. Haomin Wang. Qi Wei, Jinhui Yin, Wenhao Li, Erfei Cui, Guanzhou Chen, Zichen Ding. Changyao Tian, Zhenyu Wu, Jingjing Xie, Zehao Li, Bowen Yang, Yuchen Duan, Xuchui Wang, Zhi Hou, Haoran Hao, Tianyi Zhang, Songze Li, Xiangyu Zhao, Haodong Duan, Nianchen Deng, Bin Fu, Yinan He, Yi Wang, Conghui He, Botian Shi, Junjun He, Yingtong Xiong, Han Lv, Lijun Wu, Wenqi Shao, Kaipeng Zhang, Huipeng Deng, Biqing Qi, Jiaye Ge, Qipeng Guo, Wenwei Zhang, Songyang Zhang, Maosong Cao, Junyao Lin, Kexian Tang, Jianfei Gao, Haian Huang, Yuzhe Gu, Chengqi Lyu. Huanze Tang, Rui Wang, Haijun Lv, Wanli Ouyang, Limin Wang, Min Dou, Xizhou Zhu, Tong Lu, Dahua Lin, Jifeng Dai, Weijie Su, Bowen Zhou, Kai Chen, Yu Qiao, Wenhai Wang, and Gen Luo. Internv13.5: 55: Advancing open open-source source multimodal models in versatil- ity, reasoning, and efficiency. arXiv preprint arXiv:2508. / 18265, 2025b.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian lchter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS '22, Red Hook, NY, USA, 2022. Curran Associates Inc. ISBN 978171387100 1088.
Chenxi Whitehouse, Tianlu Wang, Ping Yu, Xian Li, Jason Weston, Ilia Kulikov, and Swarnadeep Saha. J1: Incentivizing thinking ilm-as-a-judge via reinforcement learning. arXiv preprint arXiv:2505. 10320, 2025.
Haoning Wu, Zicheng Zhang, Weixia Zhang, Chaofeng Chen, Liang Liao, Chunyi Li, Yixuan Gao, Annan Wang, Erli Zhang, Wenxiu Sun, Qiong Yan, Xiongkuo Min, Guangtao Zhai, and Weisi Lin. Q-align: Teaching Imms for visual scoring via discrete text-defined levels. arXiv preprint arXiv:2312.17090, 2023a.
Haoning Wu, Zicheng Zhang, Erli Zhang, Chaofeng Chen, Liang Liao, Annan Wang. Kaixin Xu, Chunyi Li, Jingwen Hou, Guangtao Zhai, Geng Xue, Wenxiu Sun, Qiong Yan, and Weisi Lin. Q-instruct: Improving low-level visual abilities for multi-modality foundation models. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 25490-25500, 2024. doi: 10.1109/CVPR52733 2.2222222200..
Tianhe Wu, Jian Zou, Jie Liang, Lei Zhang, and Kede Ma. Visualquality-rl: Ceasoning-induced image quality assessment via reinforcement learning to rank. arXiv preprint arXiv:2505. 2025.
17

---

Published as a conference paper at ICLR 2026
Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to- image synthesis. arXiv preprint arXiv:2306. 09341, 2023b.
Fen Xia, Tin Liu, Jue Wang, Wensheng Zhang, and Hang Li. Listwise approach learning to rank: theory and algorithm. In Proceedings of the 25th International Conference on Machine Learning. ICML '08, pp. 1192-1199, New York. NY, USA, 2008. Association for Computing Machinery. ISBN 978 9781605522054. doi: 10.1 (4L145/1390156.1390003 URL https://doo /oii org/10. 1 45/ 390156. 56. 1390306
Jaazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: learning and evaluating human preferences for text-to-image generation. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS 23, Red Hook, NY, USA, 2023. Curran Associates Inc.
Yuzhe Yang, Liwu Xu, Leida Li, Nan Qie, Yaqian Li, Peng Zhang, and Yandong Guo. Personalized image aesthetics assessment with rich attributes. arXiv preprint 1rXiv:2203,1675: 2022.
Zhiyuan You, Xin Cai, Jinjin Gu, Tianfan Xue, and Chao Dong. Teaching large language models to regress accurate image quality scores using score distribution. arXiv preprini 1116) 2025.
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang. Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wang, Hongli Yu, Yuxuan Song, Xiangpeng Wei, Hao Zhou Jingjing Liu, Wei-Ying Ma, Ya-Qin Zhang, Lin Yan, Mu Qiao, Yonghui Wu, and Mingxuan Wang. Dapo: An open-source Ilm reinforcement learning system at scale. arXiv preprint arXiv:2503.1444 2025.
Weixia Zhang, Guangtao Zhai, Ying Wei, Xiaokang Yang, and Kede Ma. Blind Image Quality
Assessment via Vision-Language Correspondence: A Multitask Learning Perspective In 2023 JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 14071-14081
Los Alamitos, CA, USA, June 2023. IEEE Computer Society, doi: 10.1109/CVPR52222.2000 2023. 01352. URL
2023. 01352
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng. Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging llm-as-a-judge with mt-bench and chatbot arena. Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS '23, Red Hook, NY, USA, 2023.
Curran Associates Inc.
Zhaokun Zhou, Qiulin Wang, Bin Lin, Yiwei Su. Rui Chen, Xin Tao, Amin Zheng, Li Yuan, Pengfei Wan, and Di Zhang. Uniaa: A unified multi-modal image aesthetic assessment baseline and benchmark. arXiv preprint wrXiv:2404.09619 2024.
Hancheng Zhu, Ju Shi, Zhiwen Shao, Rui Yao, Yong Zhou, Jiaqi Zhao, and Leida Li. Attribute- driven multimodal hierarchical prompts for image aesthetic quality assessment. In Proceedings of the 32nd ACM International Conference on Multimedia, MM '24, pp. 2399-2408, New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400706868. doi: 10.1145/ 3664647.3681 175. URL htt.httts://do.crrgt crg/ 1145/36646417 318444677..811111 3681175
Lianghui Zhu, Xinggang Wang, and Xinlong Wang. JudgeLM: Fine-tuned large language models are scalable judges. In The Thirteenth International Conference on Learning Representations, 2025. URL htRpstt::: htt::///pppeeeeeeeee net. wtt.:eteeoorrmmmmsssssso.o.oaaa.
18

---

Published as a conference paper at ICLR 2026
A THE USE OF LARGE LANGUAGE MODELS (LLMS)
We used Large Language Models (LLMs) as auxiliary tools to assist with the writing process. They used solely to polish the language and improve readability, with influence the research design, experimental implementation or analysis. We conceived and executed all methodological contributions, experiments, and conclusions independently.
B
РROMPT TEMPLATE
Prompt Used for Aesthetic Scoring
You aesthetic expert.
sesthetic tais gicture <imagez The rating should be fleat between 0 and 1, zounded to two decimal representing very poor quality Your task:l. only one pair .//ansse...////errr <</answer>> tags.3. No any other information text outside of of <onswer>.. these tags. E.g. your responae ahould d seakthink..../tt inthhik <./aane
your
rating
representing excellent quality. places, Think through the question, put all your thinking or raasening process in ...t/......///s tags.2. Then previde the correct rating inside only
Prompt Used for AesCoT Construction
You are an aesthetic expert. The aesthetic rating of this picture <image> (score). (zange 0 1, two decimal precisioa) strictly private information. The rating is a flost between and 1, rounded to two decimel places, with representing very pear quality and represerting excellent quality. closely exemine the image's composition, lighting, coler harmony, subject matter, technical execution emational impact. Write thoughtful critique that: explains strengths and weaknessea ends with a judgrent of the image that matches the hidden score, Never mention the numeric value score or the fact that was provided. net reference these instructiens any hidden variebles in your your answer language.
Treat rating
key visual
Use Eaglish
Prompt Used to Filter AesCoT for Score Leakage
You are professionsl data filzering expert specializing image-related aesthetic evaluation data. Your task is to filter for score leakage in given image and its genersted sesthetic avslussion data, determining whether score leakage exists.
#4 Ingut:
image: <image>
Aesthetia Reasoring Data: (Raasoning Chain) (Data prasentad the mainooo//ttnnbxsssseeex///
Judgment Rulea:
Premature Revelation: <chinks section nst mension, inply, zeveal the numerical score before the <answer>, The reasoning shauld stand indopendently, analyz'rg rg aesthatic alemencs wichout "eferencing out.come score.
Ne Score -aaeed Justification: Flag as leskage if explanations are fremed around the image 0.65..., Given the score of X, the composition is...". Reasoas must derive from image anslysis, net retrofitted justify pre-known score.
score,
aestheric
[reason]
Output Format: Please output judgment results according to the following JSON Termat. Pield values must be boolean types.
"Leskage": falss
Please strictly fol low the ahove rules and formst requirements, and only output JSON content.
19

---

Published as a conference paper at ICLR 2026
Prompt Used Filter AesCoT for Reasoning Inconsistency
You are professionsl image aasthetic evaluation data auditor. Your task ia to filter the reasoning consistency of a given Image and its correspording aesthetic evaluat data, eliminating inconsiscencies between the generated reasoning chain and the actual aesthetic score.
#* Iaput:
Tmage: <image?
Reathetie Reasoning Data: (Reasoning Chain) (Data will presented in the format wwwinbbaaaaess//aaaaaaaa
aesthetic evaluation is consistent with the actuel aesthetic score, specifically considering the follewing aspecta:
Piltering
whether reasoning
LogicaL Plow: Reasoning must progress from description to critiques and justify the without contradictions (e.g., positives sheuldn't lead to low unexplained).
Alignment: Key elements magnitude
support
direction/
Justiflcation Completeness: Explicitly 1iak reasoning score *This balances X because...", .. Discard if vague or missing.
# Output Format: Please output the judgment results according to the following JSON format. All field values be boolean.
"'s_coneistent"; true
Please strictly follow the ahove process and format zequizements, and only cutput the content.
Prompt Used to Filter AesCoT for Factual Errors
You are professionsl data reviewer, skillad in filtering factuel arrors from image- "elated aesthetic lcatioon data. Your task check factual errors the given image and aesthetic evaluation data, specifically those that describe noa existent elements or misrepresent image festures.
# Inguu:
Image: <image>
Aeathetis Reasoring Data: (Reasoning Chain] (Data will preaented in the format canss///sshbbosssss////cc
Filtering Rules:
Accurate Deacription fon of Viaible Blements: The <think> section must only reference elements, sbjects, colors, shapes, or compositions are objectively present Inage. Flag as error if it deseribes non-exi stent features or fabricates details kike geople, animals, or backgrounds not visible.
No Haliveinations or invensions: Ensure ressoning avoids inventing astributes or irplying act mos//mmvmmnnts deplated.
Misrepresentationn Featurea: Incorsect colors sizes/propor.ions or spatial relationatips. Kinor interpretive leeway is allowed for subjective aesthetics, bat coze fects mast match.
diatortions actual contant, auch
Contextual Fidelity: The overall reasoning must align with the imago's s genre, style, or without external sasumpstons
Output Format: Please output
results the lowing format:
"is "is_factual"; "False"
strictly follow process format requirementa, only output JS0N content.
20

---

Published as a conference paper at ICLR 2026
C IMPLEMENTATION DETAILS
Dataset splits. We follow the official test splits for TAD66K, AVA, and PARA; for datasets lacking official splits, we randomly partition the data into train and test sets in a 9:1 ratio using a fixed seed ensure reproducibility.
Hyperparameters. We use Qwen2.5-VL--B--Isttruc (Qwen et al., 2025) as our backbone MLLM. In RAPO algorithm, the generation number K is set to and the reward hyperparam- eter a is set to 0.1, while KI. penalty coefficient / and thigh are set to 0.01 and 0.28, respectively. We first fine tune the backbone model AesCoT for only 1 epoch with a global batch size of 64 and an initial learning rate of 1 x 10-5 and then implement RAPO training on SFT model for another 10 epochs with an initial learning rate of 10 that linearly decay and mini-batch size of 32. We set the max completion length as 1024. All the training runs on NVIDIA A100 GPU using Deepspeed Zero 3 and Plash-Attention 2.
D MORE RESULTS
D.I RESULTS OF GENERALIZATION ABILITY
TAD66K
AVA
FLICKR
PARA
AADB
Average
Models
PLCCC
SRCC
PL.CC
SRCC
PLCCC
SRCC
PLCC
SRCC
PLCC
SRCC
PL.CC
SRCC
LAION 2022
0.3936
0.3775
0.6782
0.6678
0.5827
0.5504
0.6727
0.6197
0.4013
0.4018
0.5457
0.5234
MUSIQ
2021
0.4038
0.3818
0.8107
0.8096
0.5518
0.5111
0.4543
0.4841
0.3607
0.3504
0.5163
5074
Q-Align
20A5M
0.4370
0.4031
0.8170
0.8220
0.6485
0.6177
0.7181
0.6880
0.4879
0.4866
0.6217
0.6035
VILA
202
0.4239
0.3998
0,7704
0.7672
0.6410
0,6030
0.6576
0,6511
0.4468
0.4528
0,5879
0.5748
ArtiMuse
2025
0.4510
0.4190
0.8260
0.8270
0.6637
0.6244
0.7250
0.6790
0.4921
0.4873
0.6316
0.6073
Acs-RI
0.4495
0.4258
0,8103
0.8134
0.6624
0.6250
0.7411
0.7057
0.4743
0.4694
0.6275
0.6079
Table 5: Results of models trained only on AVA dataset
D.2 ANALYSIS OF ENTROPY IN RAPO
1.8
Quen2 5-VL T8--rehract
1.6
1.6
Kin--VL-AA0--1 Inetuutt
1.4
Estropy Voee
1.4
SFT epech RAPO spect RAPO
SeT 10 spach RAPO
Docibure RAPO
1.0
0.8
Train
(c) Comparison of Entropy SFT Training
(b) Comparison Entropy RL Training
Figure 7: Comparison of Average entropy in SFT and RL training. (a) Entropy peaks after a single SFT epoch and then declines in continuous training. (b) Under RAPO, entropy remains stable without collapse.
To analyze the entropy of our training pipeline, we conduct ablations on our backbone as well as on VL.3.5 (Wang et al. (0256) and Kimi-VL--A3B-Insteee ning heterogeneous model families, scales, and pretraining corpora. As shown in Fig. 7aa,, within AesCoT the average token entropy peaks after a single SFT epoch and then decreases with further SFT. We then initialize RAPO from multiple SFT checkpoints of the backbone and report results in Tab. 3 The comparison shows that moderate SFT improves performance, and starting RL from the high-entropy checkpoint yields the best results. Moreover, the slight entropy fluctuations observed in Fig. (b) demonstrate the stability and robustness of our RAPO algorithm, with no evidence of entropy collapse.
(Team et al., 2025) span-
21

---

Published as a conference paper at ICLR 2026
E AESCOT DATASET DETAILS
E. AESTHETIC DIMENSIONS
In this study, we select five key aesthetic dimensions- Light and Shadow, Mood and Narrative, Composition, Color, and Exposure- -aduureeaddddde guide the external experts generate systematic analysis of images based on these criteria. We believe these dimensions comprehensively cover the core elements of visual aesthetics: the Light and Shadow dimension assesses tonal contrast and atmo- sphere; the Mood and Narrative dimension evaluates thematic expression and emotional resonance; the Composition dimension examines spatial arrangement and visual guidance: the Color dimension focuses on chromatic coordination and harmony; and the Exposure dimension measures brightness control and detail clarity.
Dimension
Aesthetic explanation
Light and Shadow
Light and shadow establish volume, spatial depth, and visual focus. Directional- ity and contrast model form and texture; side/backlighting and cast shadows can evoke mystery drama. The color of light and time of day convey atmosphere. Strong light design supports the narrative while preserving highlight and shadow detail for legibility and dimensionality.
Mood and Narrative
The image's affective tenor and implied story emerge through subject pos- ture, environmental cues, pacing, and intentional ambiguity. The tension be- tween what is revealed and withheld invites inference and engagement. Symbol, metaphor, and gesture diversify readings and increase resonance and memora- bility.
Composition
Composition organizes elements determine balance, proportion, rhythm, and hierarchy. Eye paths, perspective, framing, and negative space create structural tension and unity between stability and dynamism. Effective composition clar- ifies information, isolates the subject, and avoids clutter dispersion.
Color
Hue, value, and chroma constitute color language that shapes emotion and spatial impression. Harmonies (complementary, analogous, triadic) and warm- cool contrasts set almosphere and depth; a dominant palette with accent colors establishes hierarchy. Color also carries symbolic and narrative functions; consistent grade and controlled contrasts sharpen stylistic identity.
Exposure
Exposure governs tonal distribution and the rendering of texture, where tech- nique meets intention. Choices among high key or low key, and whether to pro- lect highlights or shadows, should serve theme and mood while retaining critical detail within the available dynamic range. Deliberate over- or underexposure can be expressive, but information loss that weakens legibility or narrative should be avoided.
Table 6: Details of aesthetic dimensions
E.2 DATA COLLECTION DETAILS
We release two versions of our dataset: AesCoT-3K and AesCoT-10K. Initially, image-score pairs aggregated from publicly available sources. The state-of-the-art vision-language model GPT- 4.1 (OpenAl 2025) serves as an external expert. To ensure the model effectively distinguishes aesthetic quality during supervised fine-tuning, balanced sampling is applied across three rating intervals: bad (0.0-0.4), fair (07), and good (0.7-1.0). For consistency in our multi-dataset training experiments, sampling is restricted to images drawn from in-distribution benchmarks: AVA (Murry et al. 2012), TAD66K (He et al. 2022), and FLICKR-AES (Ren et al. 2017).
Moreover, carefully filter the expert trajectories mainly from three aspects: score leakage, rea- soning inconsistencies, and other factual errors.
Score leakage Since our pipeline provides the ground truth (GT) score to the model before gener- ating critiques, we expect the reasoning chain to flow naturally from image features to the final score,
22

---

Published as a conference paper at ICLR 2026
reflecting genuine aesthetic analysis. However, model-generated outputs may occasionally exhibit reverse reasoning patterns, such as "The aesthetic score of this image is 0.65..." which indicates the model is deriving explanations from the score rather than from visual observations inervaaions-coontaaaittnng contradicting our intended Chain-of-Thoughe direction. To eliminate such cases, we employ GPT-4.1 to evaluate every generated output, producing binary classification indicating whether score leakage is present.
Reasoning Inconsistency Even powerful closed-source models can occasionally produce judg- ments that misalign with image quality. This layer targets cases where the generated reasoning chain contradicts the GT score-for instance, when the model severely criticizes multiple aesthetic dimensions of an image with GT score of 0.9 (indicating high quality). To filter such inconsis- tent samples, leverage the strong comprehension capabilities of closed-source MLLMs: given both the generated reasoning chain and the GT score, the model returns a binary judgment on their consistency.
Factual Rrror Model-generated reasoning chains may suffer from hallucinations, describing non- existent elements or misinterpreting visual features, thereby undermining the validity and correct- ness of the explanations. To address this issue, we first use GPT-4.1 to generate detailed captions directly from the images. Subsequently, we perform an additional filtering round verify whether the content described in the reasoning chain aligns with the caption, ensuring factual grounding in actual image content.
Human Verification After these three automated filtering stages, recruited 5 highly educated students to conduct manual verification of flitered data according to the same three criteria.
The final composition of AesCoT is displayed in Appendix E.2
Image Aesthetic Level
Image Count
Image Aesthetic Level
Image Count
Bad (0-0.4)
1000
Bad (0-0.4)
2928
Fair (0.4-0.7)
1000
Fair (0.4-0.7)
3312
Good (0.7-1.0)
1000
Good (0.7-1.0)
3660
All
3000
All
9900
(a) AesCoT-3K
(b) AesCoT-10K
Table 7: Composition of AesCoT-3K and AesCoT-10K
F DATASET DISTRIBUTION
We visualize score histograms(Fig. 8) for each dataset. The score distributions are approxi- mately bell-shaped and close to normal distribution. We further assess the normality assumption by plotting the Q-Q plots (Fig. 9 and computing the RMSE (Tab. 8) between the empirical score distribution and its fitted Gaussian distribution, which consistently yields small errors, indicating good fit.
Dataset
AVA
TAD66K
FLICKR
PARA
AADB
Combined Train set
Num Samples
255502
52248
36449
28220
6958
15000
Mean
0.5383
0.5436
0.6029
0.6297
0.6081
0.5560
Std
0.0731
0.1447
0.1555
0.1088
0.1525
0.1275
RMSE
0.0028
0.0177
0.0185
0.0281
0.0213
0.0118
Table 8: Statistics of datasets used in our experiments.
23

---

Published as a conference paper at ICLR 2026
6000
50000
Density
Denshhy
Count
Dendhhy
2000
92
34
3.5
05
02
6.8
5,
0.
(o) AVA
(b) TAD66K
(c) FLICKR
6000
2000
7000
5000
1050
Denohhy
1800
08
0.4
0.6
0.4
0.5
(d) PARA
(e) AADB
(f) 15K Combined Train Set
Figure 8: Score histograms and distribution of different datasets.
(0) AVA
(b) TAD66K
(c) FLICKR
21
21
PARA
(e)
(f) Train
Figure 9: Q-Q plot of different datasets.
24


Reviews:
[{"review_id": "8is2FANhtb", "reviewer": "Reviewer_zj9p", "summary": "This work introduces two key contributions: the AesCoT data construction pipeline, which provides high-quality data for SFT training, and the RAPO algorithm, which features a novel dual-reward mechanism comprising a Relative Rank Reward and an Absolute Error Reward, specifically tailored for IAA. However, the overall novelty of this work is limited, and there are certain deficiencies in both the data construction pipeline and the reward design.", "questions": "Please refer to weakness", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. This work supplies high-quality data for aesthetic reasoning by constructing an automated pipeline.\n2. It explores the application of reinforcement learning to the IAA domain, demonstrating its viability.\n3. Through a series of comprehensive experiments, the study furnishes robust empirical validation for its claims.", "weaknesses": "1. The five evaluation dimensions proposed in the AESCOT process have certain limitations and lack support from relevant literature. In practical IAA, the assessment of image aesthetics is more flexible and cannot be comprehensively summarized by the five dimensions suggested by the authors. \n2. The filtering strategy mentioned in Section 3.2 and Appendix E is overly simplified, which raises considerable doubts about the credibility of the dataset.\n3. The method innovation is limited. Although the study focuses on applying RL to the IAA field, the overall pipeline closely resembles that of DeepSeek-R1, similarly employing high-quality data for SFT cold-start. Moreover, the proposed RAPO and DAPO are highly similar, with modifications to the reward function. The paper leans more toward application rather than introducing substantial innovations.\n4. The Relative Rank Reward relies on a critical assumption that the aesthetic score of an image follows a Gaussian distribution. All subsequent reward-rank calculations are based on this assumption, yet no empirical evidence is provided to validate that the scores indeed conform to a Gaussian distribution. The authors should substantiate the correctness of this assumption. \n5. Equation 3 essentially models the probability of which image is superior between two images. Why not directly employ the Bradley-Terry model for preference modeling instead?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "daNFDnI88A", "reviewer": "Reviewer_XmWM", "summary": "Authors proposed Aes-R1. It is a comprehensive framework for Image Aesthetic Assessment that combines an aesthetic reasoning data pipeline (AesCoT) with a reinforcement learning algorithm (RAPO). It jointly optimizes absolute aesthetic scores and relative ranking consistency, significantly improving both accuracy and generalization.", "questions": "1. Compared with models that directly predict scores, this reasoning-based MLLM approach requires significantly more inference tim, sometimes tens of times longer or even more—which poses clear challenges for large-scale inference. How do the authors consider this inefficiency issue?\n\n2. The presentation in Figure 5 is somewhat unclear. Given that score distributions can vary greatly across datasets, is it reasonable or meaningful to compare their absolute values directly?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The authors build AesCoT, an automatic pipeline that generates high-quality aesthetic reasoning data efficiently.\n2. They propose Aes-R1, a two-stage framework using RAPO to jointly optimize absolute scoring and relative ranking.\n3. Extensive experiments show Aes-R1 achieves state-of-the-art performance and generalization with limited training data.", "weaknesses": "1. The authors modify the reward design based on GRPO to fit the specific characteristics of the IAA task. However, since Q-Insight has adopted similar methods on the same task, the contribution and novelty of this paper appear somewhat diminished.\n\n2. Compared with models that directly predict scores, this reasoning-based MLLM approach requires significantly more inference time, sometimes tens of times longer or even more, which poses clear challenges for large-scale inference. How do the authors consider this inefficiency issue?\n\n3. The presentation in Figure 5 is somewhat unclear. Given that score distributions can vary greatly across datasets, is it reasonable or meaningful to compare their absolute values directly?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zYG1n6n1PB", "reviewer": "Reviewer_xmgr", "summary": "This paper introduces Aes-R1, a two-stage framework for advanced Image Aesthetic Assessment (IAA). It first \"cold-starts\" an MLLM by fine-tuning it on AesCoT, a new dataset of high-quality aesthetic critiques, to instill reasoning patterns. It then applies a novel reinforcement learning algorithm, Relative-Absolute Policy Optimization (RAPO), which uses a unique reward function that jointly optimizes for both absolute score accuracy and relative ranking preference. This combined approach allows Aes-R1 to significantly outperform existing baselines, improving its backbone model's average PLCC/SRCC scores by 47.9%/34.8% with minimal training data.", "questions": "1. Regarding the SFT cold-start phase: The appendix mentions both AesCoT-3K and AesCoT-10K, but the main paper does not specify which was used for the 1-epoch SFT. To ensure the reproducibility of the key results, could the authors please clarify which dataset version (3K or 10K) was used?\n\n**Suggestion:**\nThere might be a typo on Line 142 (\"fro\" -> \"for\").", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. Novel Relative-Absolute Policy Optimization (RAPO) Algorithm\nThe paper introduces RAPO, a novel RL algorithm with a joint reward function that optimizes for both \"absolute scores\" and \"relative ranking.\" This design better reflects complex human aesthetic judgment.\n\n2. Effective Two-Stage \"SFT Cold-Start + RL Finetuning\" Framework\nThe authors designed a two-stage process that first uses SFT to inject necessary aesthetic reasoning knowledge, then uses RL to align the model's scoring preferences.", "weaknesses": "1. **Reasoning Quality Is Not Quantitatively Evaluated.** \nThe paper's central claim—that SFT+RL training produces superior aesthetic reasoning compared to pure RL—is supported only by a single, qualitative case study. The work lacks any systematic, quantitative evaluation of the generated \"aesthetic analysis text.\" All metrics (PLCC/SRCC) focus exclusively on the final score's accuracy, failing to validate the \"advanced reasoning\" contribution mentioned in the title.\n2. The AesCoT reasoning data is generated by GPT-4.1, which the authors treat as a competent external source. However, GPT-4.1 performs poorly on direct aesthetic score prediction (as shown in Table 1), raising questions about its suitability as a source of high-quality aesthetic critique. While the paper does include a filtering process, a more thorough validation of AesCoT's explanation quality—either through human evaluation or comparison to expert-written analyses—would help justify the choice of GPT-4.1 as a reasoning teacher.\n3. The RAPO framework combines the relative and absolute rewards using a simple summation ($r = r_{rank} + r_{abs}$), which implies an equal 1:1 weighting. While this combined reward proves effective, the paper does not explore the relative influence of these two components. It would be valuable to include an ablation study on the weighting coefficients (e.g., $\\alpha \\cdot r_{rank} + \\beta \\cdot r_{abs}$) to better understand their interplay and to investigate whether a different balance could further optimize performance.\n4. The SFT cold-start phase is presented as critical, but its implementation lacks clarity. The authors mention both AesCoT-3K and AesCoT-10K datasets in the appendix but fail to specify which version was actually used for the 1-epoch SFT in the main experiments. This ambiguity makes it impossible to reproduce the key results or understand the precise data conditions for the model's \"cold start\".", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9riGwxpdAd", "reviewer": "Reviewer_FLJS", "summary": "This paper proposes Aes-R1, a novel framework for Image Aesthetic Assessment (IAA) using multimodal large language models (MLLMs). It introduces two key components: (1) AesCoT, an automatic data pipeline that generates high-quality chain-of-thought (CoT) aesthetic critiques along five dimensions (light, composition, color, exposure, mood), and (2) Relative-Absolute Policy Optimization (RAPO), a reinforcement learning (RL) algorithm that jointly optimizes absolute score regression and relative pairwise ranking. The method is trained on only 15K samples and achieves state-of-the-art performance across multiple IAA benchmarks, significantly outperforming existing baselines in both PLCC and SRCC metrics. The paper also provides insightful ablation studies on reward design and the interplay between supervised fine-tuning (SFT) and RL, emphasizing the importance of moderate SFT for maintaining policy entropy and enabling effective RL exploration.", "questions": "NA", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* S1: Novel and well-motivated RL framework: RAPO elegantly combines absolute and relative rewards to address the dual nature of human aesthetic judgment—calibrated scoring and ordinal preference—resolving limitations of prior rank-only or score-only RL approaches.\n\n* S2: Aes-R1 achieves significant gains (e.g., +47.9% PLCC) over strong baselines with limited training data and demonstrates robust out-of-distribution generalization.", "weaknesses": "* W1: AesCoT relies on GPT-4.1 for data generation, which may limit reproducibility and raise concerns about generalizability if weaker open-source models are used as the \"expert.\"\n\n* W2: It seems that the main contribution of this paper lies in the dataset, and there is not much innovation in terms of methodology.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)