DP²O-SR: Direct Perceptual Preference Optimization for Real-World Image Super-Resolution
Authors: Rongyuan Wu
Lingchen Sun
Zhengqiang ZHANG
Shihao Wang
Tianhe Wu
Qiaosi Yi
Shuai Li
Lei Zhang
Venue: NeurIPS 2025 poster ()
Keywords: Real-world Image Super-resolution
Diffusion Models
Direct Preference Optimization
Perceptual Quality
Preference Learning
Primary area: applications
TL;DR: DP²O-SR post-trains generative SR models to better match human perceptual preferences,   by optimizing over diverse outputs (sampled only via noise) using IQA-based rewards, without requiring human annotations during training.

Abstract: Benefiting from pre-trained text-to-image (T2I) diffusion models, real-world image super-resolution (Real-ISR) methods can synthesize rich and realistic details. However, due to the inherent stochasticity of T2I models, different noise inputs often lead to outputs with varying perceptual quality. Although this randomness is sometimes seen as a limitation, it also introduces a wider perceptual quality range, which can be exploited to improve Real-ISR performance. To this end, we introduce Direct Perceptual Preference Optimization for Real-ISR (DP²O-SR), a framework that aligns generative models with perceptual preferences without requiring costly human annotations. We construct a hybrid reward signal by combining full-reference and no-reference image quality assessment (IQA) models trained on large-scale human preference datasets. This reward encourages both structural fidelity and natural appearance. To better utilize perceptual diversity, we move beyond the standard best-vs-worst selection and construct multiple preference pairs from outputs of the same model. Our analysis reveals that the optimal selection ratio depends on model capacity: smaller models benefit from broader coverage, while larger models respond better to stronger contrast in supervision. Furthermore, we propose hierarchical preference optimization, which adaptively weights training pairs based on intra-group reward gaps and inter-group diversity, enabling more efficient and stable learning. Extensive experiments across both diffusion- and flow-based T2I backbones demonstrate that DP²O-SR significantly improves perceptual quality and generalizes well to real-world benchmarks.

DP'O-SR: Direct Perceptual Preference Optimization for Real-World Image Super-Resolution
Rongyuan Wu Lingehen Sun Zhengqiang 2 Wang',
Tianhe Wu Qiaosi , Shuai Li Lei Zhang!.2i
The Hong Kong Polytechnic University 2OPPO Research Institute
City University Hong Kong
'Corresponding author
https: : thub. com/ cswry/DP20-SR
Abstract
Benefiting from pre-trained text-to-image (T2I) diffusion models, real-world im- age super-resolution (Real-ISR) methods synthesize rich and realistic details. However, due to the inherent stochasticity of T2I models, different noise inputs often lead to outputs with varying perceptual quality. Although this randomness is sometimes seen as a limitation, it also introduces a wider perceptual quality range, which can be exploited to improve Real-ISR performance. To this end, we introduce Direct Perceptual Preference Optimization for Real-ISR (DP'O-SR), framework that aligns generative models with perceptual preferences without requiring costly human annotations. We construct a hybrid reward signal by com- bining full-reference and no-reference image quality assessment (IQA) models trained on large-scale human preference datasets. This reward encourages both structural fidelity and natural appearance. To better utilize perceptual diversity, we move beyond the standard best-vs-worst selection and construct multiple prefer- ence pairs from outputs of the same model. Our analysis reveals that the optimal selection ratio depends on model capacity: smaller models benefit from broader coverage, while larger models respond better to stronger contrast in supervision. Furthermore, we propose hierarchical preference optimization, which adaptively weights training pairs based on intra-group reward gaps and enter-group diversity, enabling more efficient and stable learning. Extensive experiments across both diffusion- and flow-based T2I backbones demonstrate that DP'O significantly improves perceptual quality and generalizes well to real-world benchmarks.
Introduction
Image super-resolution (ISR) [13, 60, 25, 59, 10, 23, 56, 16] aims to reconstruct high-resolution (HR) images from low-resolution (LR) inputs. Traditional methods emphasize pixcl-level accuracy but often produce over-smoothed results that lack realistic textures. To address this, recent approaches [22, 44, 43, 55, 24, 8] have shifted toward improving perceptual quality, which is particularly important for real-world ISR (Real-ISR) tasks where degradations are complex and typically unknown. Generative models [35, 17], especially large-scale pre-trained text-to-image (T2I) diffusion models such as Stable Diffusion (SD) [1] and FLUX [20], have demonstrated strong potential for Real-ISR [42, 52, 28, 48, 37, 33, 2] due to their capacity to synthesize plausible and diverse details. However, these models are inherently stochastic: different noise inputs can lead to significantly different output qualities. While this randomness is often considered as a drawback [37], it also introduces a broader perceptual quality range, which can be viewed source of supervision, enabling preference-driven optimization to better exploit T2I model's generative capability.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Only HR Roward
Only NRt Rowan!
Barh FR Nit Rloard
Gi
0.61
0.50
0.53
Desil suppoisxxss
Volteziraaal Araile
Over sateraiio
Rich and Narral diteil High fiieliy
Thatning
500
1000
(a) Effect of Pereeprual Reward Types.
(b) DPOOSR Generalization Performance.
Figure I: (a) Visual results of models trained with FR, NR, and hybrid rewards. FR reward suppresses detail, NR reward encourages hallucinations, while the hybrid reward preserves structure and improves realism. (b) DP2O-SR significantly boosts perceptual quality on the out-of-domain RealSR benchmark [5], improving both small and large generative Real-ISR models after only 500 steps. Larger models like C-FLUX benefit more from preference supervision.
To harness this diversity, we propose DP'O-SR--Drred Perceptual Preference Optimization for Real-ISR-a training framework that aligns generative ISR models with human-like perceptual preferences. Instead of relying on costly human annotations, we construct a perceptual reward using image quality assessment (IQA) models trained large-scale human preference data. This reward integrates both full-reference (FR) and no-reference (NR) metrics. FR metrics promote structural fidelity and help suppress hallucinated content, whereas NR metrics encourage realism and aesthetic coherence. By combining FR and NR IQA metrics, the hybrid reward provides a balanced signal that supports both accuracy and naturalness. As illustrated in Fig. (aa), models trained with only FR metrics tend to produce oversmoothed outputs, while those trained with NR metrics alone may generate hallucinated details. In contrast, the hybrid reward leads outputs with rich and natural details while remaining structurally consistent with ground-truth (GT).
Unlike Diff-DPO [40], which constructs single best-vs- worst pair from outputs of different models, we sample multiple outputs from a single model using different noise seeds. These outputs are ranked by perceptual reward, and preference pairs formed by sampling from the top-N and bottom-N candidates. This richer supervision captures finer perceptual distinctions and better utilizes the diversity inherent in stochastic generation.
We systematically investigate how the number of rollouts (M) and the selection ratio (N/M) influence learning across two representative backbones: a relatively samller diffusion model (ControlNet- SD2, denoted as C-SD2) and a larger flow-based model (ControlNet-FLUUX denoted as C-FLUX). Increasing M improves perceptual diversity and training stability, though with diminishing returns. The optimal N N/M varies by model capacity: smaller models prefer broader coverage (e.g., 1/4) for smoother gradients, while larger models benefit more from stronger contrast (e.g., 1/16), as their greater capacity enables them to learn more effectively from larger preference differences. These findings underscore the need to tailor data curation according model scale.
Even with well-chosen M and N/M, not all comparisons are equally informative some are ambigu- ous or redundant. This motivates more selective approach to learning. We propose Hierarchical Preference Optimization (HPO), which adaptively weights training pairs two levels: intra-group. by emphasizing comparisons with larger reward gaps; and inter-group. by prioritizing inputs with greater perceptual spread. By focusing on the most informative signals, HPO improves both training efficiency and perceptual alignment.
We evaluate DP"O-SR on the out-of-domain RealSR benchmark [5], which contains real-world LR-HB pairs captured under varying focal lengths, differing from the synthetic degradations used

---

during training. As shown in Fig. 1(b), both C-SD2 and C-FLUX achieve significant perceptual reward improvements within the first 500 training iterations of DPO-SR, surpassing strong baselines such as SeeSR [48] and OSEDiff [47]. Specifically, C-FLUX improves from approximately 0.51 to 0.65, while C-SD2 rises to 0.62, achieving top-2 performance early in training.
2 Related Work
Real-World Image Super-Resolution. Early deep learning- -baeed ISR methods [13, 27, 60, 12] primarily optimize pixel-level accuracy based on simple degradation assumptions, such as bicubic downsampling. As result, they often produce over-smoothed outputs in real-world scenarios. To enhance perceptual quality, researchers have introduced GAN-based techniques [22], including BSRGAN [55] and Real-ESRGAN [43], which use complex degradation models to better simulate complex real-world degradations. However, these GAN-based approaches often suffer from unstable training and visual artifacts [43, 24]. Recently, diffusion models [17. 35], especially large- scale pre-trained T2I models [1. 20J. have achieved strong results in Real-ISR tasks [42, 37, 48, 50J. These models excel at generating realistic details, but their outputs vary across different runs for the same LR input due to the inherent sampling randomness. Many existing works treat this randommess as a limitation. Some methods aim to stabilize the generation process [37], while others train one- step models [47, 36, 6, 53, 14] to reduce stochasticity. In contrast, we view this stochasticity as useful source of high-quality supervision and consequently develop a preference-based optimization framework to improve Real-ISR performance.
Preference Alignment. Aligning generative models with human preferences has become a key area of research, especially in the training of large language models (LLMs) using reinforcement learning with human feedback (RLHF) [11, 61, 32]. This process typically requires training a separate reward model, which adds significant computational cost [21]. DPO [34] provides a simpler alternative by directly optimizing the policy using preference data, without requiring an explicit reward model. Adapting these alignment techniques to diffusion models introduces additional challenges, mainly due to their iterative denoising process [29]. Recent work has extended DPO to diffusion models. For example, Diff-DPO [40] applies preference optimization across diffusion timesteps to improve visual aesthetics and prompt fidelity. Several follow-up works propose refinements, such as step-aware preference modeling [26, 18] and sample weighting based on score distributions [30]. Although DPO-based methods can be applied to Real-ISR, there lack a well-designed reward and carefully designed training strategies. We address these challenges by introducing a perceptual reward tailored to Real-ISR and by systematically exploring preference pair construction strategies beyond the conventional best-vs-worst selection. This enables more reliable learning across models with different capacities and under limited sampling budgets.
3 Background
Diffusion and Flow-based Models. Both diffusion and flow-based generative models define a stochastic interpolation between data x p(z) and noise e ~ (N, I) using the unified formulation I, = a,æ" + oee [31], where t is the timestep, and a, or are scalar scheduling coefficients. The goal is to learn a model that reverses this process to sample from the data distribution. Diffusion models [17] specify this path implicitly via a forward SDE and typically learn the score function (1,,, V= z log (2), with a standard constraint of + + = 1. In contrast, flow-based models [15] directly define the interpolation with a + o; = 1, enabling exact transport in finite time, and learn a velocity field (z t) dt 24. Different scheduling schemes lead distinct sampling trajectories.
d
From RLHF to Diff-DPO. Aligning generative models with human preferences traditionally relies on RLHF, which involves training reward model followed by policy optimization. However, this multi-stage process is complex and unstable. DPO [34] offers simpler alternative by directly optimizing the policy using preference data. Under the Traley-Terry preference model [4], DPÒ derives a loss that encourages higher likelihood ratios for preferred samples. As derived in the work [40], the training objective of Diff-DPO is formulated as follows:
LoPo = =- - ab ~q(ellaf ( (( log (-3((
|ee" - co(2"" t) ||e" cer(x))" 1)2 (Ile' co(ai, O)l ||e' car(ae), t))B)
(I)
where x
0xpx + - and x = axx + ofe are noisy versions of preferred and dispreferred samples at timestep t, c.(+,t) is the predicted noise, 33 controls the deviation between policy and
3

---

Reréraae
Polis To
Prefacnee
Pacereac Optiriation See 4.3
See 4.2
Nokaa Paol
Ronked by Peceptial Roward
Pve-tmiiing Beadts
Figure 2: Illustration of DP"O-SR framework. Given the same LR input and noise, frozen pre- trained SR model Tee (blue) generates diverse outputs by varying the random seed. These outputs are first evaluated using our proposed perceptual reward (Sec. 4.1), and then filtered through perceptual preference curation process (Sec. 4.2) to construct pairwise comparisons. A trainable policy model Ty (orange), initialized from Tet. subsequently optimized hicrarchical preference optimization (Sec. 4.3), which emphasizes informative comparisons and enhances perceptual alignment.
reference models, and a--) the sigmoid function normalizing the preference score. For flow-bass how-based models, we replace the predicted noise with velocity. This loss encourages better denoising of preferred samples, thereby aligning generation with human preferences.
Methodology
We propose OPRO-SR, a preference-driver optimization framework for Real-ISR using diffusion or flow-based models. Our key insight is to leverage the inherent stochasticity of these models-differes noise seeds naturally produce outputs with varying perceptual quality. By scoring these outputs with human-aligned perceptual reward and forming pairwise preferences, we enable direct supervision of perceptual quality without handcrafted losses or external reward networks. As illustrated Fig. 2, our framework comprises three components: (1) generating diverse ISR samples from frozen reference model, (2) ranking them with a perceptual reward to curate preference pairs, and (3) optimizing a trainable policy via uncertainty-ave preference learning. Our design aligns ISR outputs with human perceptual preferences and improves the perceptual quality both C-SD2 and C-FLUX.
4.1 Perceptual Reward Design
Given an LR input JRR E RAxxxx3 we use a Real-ISR model Ter to generate M ISR candidates S = (T....., JAs) by varying the noise seeds, which often exhibit certain perceptual variations. To rank the perceptual quality of these outputs, we introduce perceptual reward that aggregates multiple IQA metrics. Specifically, we consider two sets of metrics: set FR of FR metrics, which compare outputs against GT images, and set N/R of NR metrics, which assess quality without reference. Specifically, the FR set includes LPIPS [58], TOPIQ-FR [7], and AFINE-FR [9], and the NR set includes MANIQA [51], MUSIQ [19], CLIPIQ+ [41], TOPIQ-NR (7), AFINE-NR [9], and Q- A-Align [46]. Note that the commonly used distortion based metries PSNR and SSIM excluded, as they are not effective in describing perceptual quality.
For each candidate Jm, 1,... M and each metric o E FRUNR, we compute raw score st The scores are first firection-alignee such that higher values indicate better quality. Denote by 2 and d the maximum and minimum values among the score values of the M candidates, we then normalize each score S as s (s (s
iiin) ). To balance fidelity and perception, we define the perceptual reward of sample by averaging the normalized scores over the FR and NR sets, assigning them equal weight regardless of their sizes:
Rm
0.5
+
0.5 NR
Σ
5
(2)
FR
OETR
#EARR

---

where denotes the cardinality of a set. Based on this reward, we can identify the top-N and bottom-A candidates to construct preference pairs for training.
4.2 Perceptual Preference Data Curation
Given the reward signal, we construct preference pairs by sampling multiple ISR outputs per input using different noise seeds from a single stochastic model. In contrast to hiff-DPO [40]. which selects outputs from different models and constructs a single wers--errss--ors pair per input, our approach samples M outputs from the same model and ranks them using the reward function R..- We then select the top-N and bottom-N samples as positive and negative sets, respectively, and build N2 possible preference pairs. This allows construct richer training signals from single model without relying on external comparisons or ground-truth.
This design introduces two key control parameters: the number of samples per input (M) and the selection ratio (N/M). Varying these parameters enables us to control the contrast and diversity of the training signal. For example, smaller values of N/M are expected to yield stronger reward gaps (contrast) between positive and negative samples, potentially accelerating learning. Conversely, larger N/M ratios increase coverage and diversity but may reduce discriminative signal strength.
To explore the effect of these choices across architectures, we apply this curation strategy to two representative models of contrasting capacity: C-SD2 (a 0.8B UNet diffusion model) and C-FLUX (a 12B DiT flow model). We hypothesize that higher-capacity models may benefit from stronger contrast (low N/M), whereas smaller models may require more redundancy to ensure stable gradients. The experimental analysis of these hypotheses is provided Section 5.3.
4.3 Hierarchical Preference Optimization
We propose hierarchical preference optimization (HPO). an extension of Diff-DPO that improves preference alignment by adaptively weighting training pairs at two levels: intra-group and inter-group. While Diff-DPO treats all training pairs equally, HPO focuses on learning from more informative signals by leveraging both local and global variations.
Intra-group. Within each group of SR candidates generated from the same LR input, preference pairs (xl æd) may differ in informativeness. Intuitively, pairs with larger reward gaps ARt R., - R2 offer stronger supervision. We define the intra- grrra weight as Wataa xz) = R Rıl + (1 pgap). where Pyop is the average reward gap over all pairs within the group. This formulation ensures that high-contrast pairs receive greater emphasis while keeping the expected weight normalized.
Inter-group. Different LR inputs may yield SR candidate groups with varying levels of perceptual diversity. To prioritize groups that offer stronger supervision, we compute the standard deviation o of reward values } within group g. and assign gnter-group weight as Winer(g) = Jg + (1 - p where Po is the average standard deviation across all groups. This boosts the contribution of more informative groups while keeping the expected group weight close to
Final Loss Function. Each training pair (x a/)) is assigned total weight w = Wimrr (xl x/)) Winter (g). The final training objective LHrO = =< -(mmy (aw w f(x' ar:: f), where e(-) denotes the per- per-pair Diff-DPO loss defined in Eq. 1.
5 Experiment
5.1 Experimental Settings
Baselines. We adopt the ControlNet paradigm [57], where control branch is trained for the Real-ISR task, using pre-trained Stable Diffusion 2.0 (SD2) [1] and FLUX.I-Des (FLUX) [20] as backbones. For convenience, we refer to the resulting models as C-SD2 and C-FLUX. respectively, and their improved variants with our method as DP'O-SR (SD2) and DP'O-SR (FLUX). This setup enables us to evaluate DP*O-SR across models that differ substantially in capacity (0.8B 12B), architecture (UNet vs. MMDiT), and generative paradigm (diffusion vs. flow matching). The detailed network architecture of our C-SD2 and C-FLUX can be found in the Appendix.
Evaluation Metrics. We evaluate our method using a total of 14 IQA metrics, categorized into four groups: (1) Trained FR metrics: LPIPS [58], TOPIQ-FR [7], and AFINE-FR [9]; (2) Trained NR
5

---

Table 1: Performance comparison of different methods. Metric types are categorized into trained FR perceptual (in blue), trained NR perceptual (in green)), untrained NR perceptual (in yellow )),, and untrained FR fidelity (in purple)) based metries. Red bold values indicate better performance between the baseline C-SD2, C-FLUX and their boosted versions by OP*O-SR. Arrows indicate whether higher (f) or lower (L) values are better.
DP'SSRR
OP'O-SR
Datasets
Metrics
StahleSR DiMBIRv
SeeSR
CCSR
AddSR
OSEDif
C-SD2
G-HLUX
(SD2)
(FLUX)
LPIPS
0.4219
0.4471
0.4322
0,4080
0.4930
0.4043
0.4332
0.4268
0.4260
0.4187
TOPIQ-FR
0,4208
0,4108
0.4238
0.4357
0.3577
0,4375
0.4336
0.4396
0.4364
0.4489
AHINE-PR.
40.6309
-0.9339
-1,0931
-0.3396
-0,7704
-1.0567
-1.1433
-0,8962
-1.0164
-0.8341
MANIQA
0.5707
0.6528
0.6513
0,5956
0.7025
0.6327
0.6684
0.7165
0.6857
0.7199
MUSIQ
60.34
70.59
71.37
64.34
73.33
70.04
71.66
74.87
72.28
75.06
CLIPIQA+
0.6313
0.7491
0.7385
0.6421
0.7742
0.7164
0.7595
0.8124
0.7473
0.7993
TOPIQ-NR
0.5019
0.7084
0.7098
0.5941
0.7629
0,6341
0,7155
0.7611
0,7019
0.7645
Syn-Test
AFINE-NR
0.8693
-0.9683
-1,0483
-0.7937
-1.0751
-0.9879
-1,0097
-1,2263
-1,2026
-1,2764
QALIGNT
3.2196
4,1382
4.1614
3,3266
4,2086
3,9801
4.2481
4.5526
4.4266
4.7060
VQ-RIt
4.34
4.42
3.88
4.38
4.40
4.43
4.57
4.53
4.65
NIMA
4.8936
5.4096
5,3862
4.7765
5.5352
5.2029
5.3894
5.6417
5.4458
5.5986
TOPIQ-IAAT
4,7056
5,3929
5.3595
4.8338
5.6047
5.1199
5,4123
5,6106
5,3457
5,5292
PSNRT
22.75
22.43
22.41
23.54
21.00
22.61
22.46
21.48
21.26
21.27
SSIMI
0.5865
0.5355
0.5648
0,5928
0.4832
0.5775
0.5449
0.5259
0.5158
0.5143
LPIPS
0.3877
0.4288
0.3883
0,3645
0.4539
0.3729
0,4146
0.4045
0.4004
0.4024
TOPIQ-FR FR
0.4923
0.4747
0.4881
0.5367
0.4093
0,5059
0.4756
0.4656
0.4824
0.4867
AFINE-FR.
-0.7699
-0.8059
-0.7439
-0.9548
-0.1243
00.1144
-0.6578
-0.3331
-0.5916
-0.6097
MANIQAT
0.6230
0,6502
0.6451
0.6034
0.6810
0.6335
0.6629
0.7031
0.6632
0.6918
MUSIQU
65.88
69.28
69.82
63.57
71.39
69.09
70.44
73.16
69.60
72.77
CLIPIQA+
0.6501
0.7235
0.6910
0.6216
0.7438
0.6964
0,7295
0.7852
0.6798
0.7571
TOPIQ-NR
0.5748
0.6760
0.6891
0.5735
0.7262
0.6254
0.6828
0.7429
0.6522
0.7416
RealSR
AFINE-NR,
-1.0120
-0.9860
-1.0368
-0.9157
-1.1449
-1.0489
-L.0357
-1.1555
-L.0985
-1.0905
QALIGNT
3.2337
3,6866
3.6723
3.1317
3.7625
3.6399
3,6490
4.0206
3.6499
4.1492
VQ-RIT
3.78
4.12
3.98
3.80
4.08
4.09
4.08
4.21
4.02
4,32
NIMA1
4.8150
4,9190
4.9193
4,4545
5.1601
4,8952
4,9914
5.2324
4.9780
5,0740
TOPIQ-IAAT
4.5856
4.8981
4.8553
4.3509
5.0890
4.7545
4.9238
5.1144
4.7427
4.9798
PSNR1
24.64
24.83
25.15
26.21
23.31
25.15
23.61
22.49
23.58
23.49
SSIM
0.7077
0.6500
0.7213
0.7363
0.6404
0.7340
0.6566
0.6500
0.6594
0.6590
metrics: MANIQA [51], MUSIQ [19], CLIPIQA+ [41], TOPIQ-NR [7], AFINE-NR [9], and Q- Align [46]. which together form the perceptual reward used in training. These two groups constitute the sets FR and NR defined in Sec. 4.1 for reward computation. (3) Untrained NR perceptual metrics: VQ-RI [49], NIMA [38], and TOPIQ-IAA [7], used to assess perceptual generalization beyond training targets; (4) Untrained FR fidelity metrics: PSNR and SSIM [45], included for completeness. This comprehensive setting allows to evaluate both in-distribution performance and out-of-distribution generalization of Real-ISR models.
Finetuning, Post-Training and Testing Datasets. The C-FLUX model is finetuned for the Real-ISR task using approximately 1 million high-quality images. We use batch size of 32, learning rate of 1 x 10-4, and train the model for 45,000 steps. The C-SD2 variant follows a similar setup, but is trained with batch size of 256, a learning rate of 2 x 10-4, and 35,000 steps.
To support perceptual post-training, we curate a semantically diverse dataset from the Internet, containing 30,100 high-quality images spanning six major scene types and 266 sub-categories. Among them, 30,000 images are used for post-training, while the remaining 100 images form the Syn-Test set. We adopt the first-order degradation pipeline from ResShift [54], which better simulates real-world degradation than the second-order version used in RealESRGAN [43]. We further evaluate generalization performance on real-world benchmarks such as RealSR [5].

---

LR Image
StableSR
SecSR
DiABBRvv
C-SD2
GT Image
CCSR
AdeSR
OSEDiff
DPPOSRR (SD2)
Image
DiABIRvv
C-FLUX
GT Image
CCSR
AddR
OSEDifT
DPSO-SR (FLUN
Figure 3: Qualitative comparisons of different Real-ISR methods. Please zoom in for better view.
DP'O-SR Training Configuration. We train OP*O-SR with a batch size of 1024, learning rate of x 10-5, and set the preference weighting hyperparameter 3 to 5,000. The model is trained for 1,000 iterations. All experiments are conducted on 8 A800 GPUs. account for the stochasticity the generative SR models, we sample up to M = 64 outputs per LR image during training. Preference pairs are constructed from C-FLUX and C-SD2 outputs, each sampled using 25 and 50 inference steps, respectively. The corresponding classifier-free free guidance (CFG) scales are set to 2.5 and 3.5. These settings are kept consistent throughout both training and evaluation.
Offline Candidate Generation and IQA Labeling. For offline preference pair construction, we sample 64 outputs per LR image across 30,000 training images using the same 8xA800 GPU setup. This process requires approximately 168 hours for C-SD2 and 432 hours for C-FLUX. The resulting 1.92 million generated images are labeled using our suite of IQA models (Section 4), which takes an additional 72 hours. All configurations are held fixed to ensure reproducibility and fair comparison.
5.2 Results on Perceptual Preference Alignment
Improvement over Baselines. To evaluate the effectiveness of DP we compare it with the baseline models C-SD2 and C-FLUX using the four categories of metrics. As shown Tab. 1, our method consistently improves performance in almost all perceptual categories, demonstrating strong alignment with training signals and good generalization beyond them.
On the Syn-Test dataset, where the degradation process matches that used during training, we observe clear improvements on trained FR metries such as LPIPS (10.4332 - 0.4268) and TOPIQ- FR (10.4336 + 0.4396) for SD2, along with similar trends on C-FLUX. For trained NR metrics, performance is also enhanced across the board, including MANIQA (10.6684 - 0.7165), CLIP-IQA+ (10.7595 - 0.81 0.8124), and QALIGN (14.2481 - 4.5526). indicating that preference-guiie tuning effectively aligns outputs with perceptual quality signals. Importantly, our method generalizes well to perceptual metrics not used during training. For example, on the untrained metric VQ-R1, we observe consistent improvements over both baselines (e.g., 4.38 - 4.57 for SD2, 4.40 - 4.65 for FLUUX), suggesting that the learned preferences transfer beyond the supervised objectives. This aligns with the well-established perception-din disttroioonoooiiioonnn tradeoff [3], which suggests that improving perceptual quality often comes at the cost of lower PSNR SSIM.
fine-
7

---

Comparison with SOTA, Our O-R models outperform wide range of state- state-of-the-art Real- ISR methods on the challenging RealSR benchmark, including StableSR [42], DiffBIRv2 [28], SeeSR [48], CCSR [37], AddSR [50], and OSEDiff [47], especially in perceptual metrics. For instance, DP'O-SR (SD2) achieves the highest MANIQA (10.7031) and CLIPIQA+ (10.7852) among all methods, indicating strong perceptual quality and alignment with human preferences.
In untrained perceptual metrics such as VQ-RI and NIMA, OP'O-SR (FLUX) also exhibits strong generalization, achieving top-tier results (e.g., VQ-RI 14.32, QALIGN 14.1492). These results demonstrate that our method not only boosts perceptual alignment on seen metrics but also generalizes to diverse evaluation criteria beyond the training objectives. This highlights the ability of OP'O-SR to transform moderate diffusion backbones into highly competitive Real-ISR models.
Qualitative Comparisons. Fig. 3 presents visual comparisons of different Real-ISR methods. In the first example, the C-SD2 baseline produces dense, irregular stripe artifacts in the region of the staircase. After applying DP'O-SR, these artifacts are effectively removed and replaced with regular clean fence structures, achieving even better reconstruction quality than SeeSR and OSEDiff. In contrast, AddSR and CCSR partially erase the stair details. In the second example, C-FLUX generates grid-like artifacts in the background, which are largely eliminated after applying DP"O-SR. DiffBIRv2 and StableSR fail to reconstruct meaningful facial features, yielding smeared or indistinct results. AddSR distorts the beak geometry, while CCSR over-sharpens edges but lacks semantic accuracy. In addition, some methods tend to hallucinate unnatural chicken heads. In comparison, the results of O (C-FLUX) are visually clean and semantically faithful. It should be noted that both C-SD2 and C-FLUX use the same noise seed before and after applying OP"O-SR. This confirms that our method effectively suppresses semantic artifacts while preserving the overall structure and content of the original generation. Due to space limitations, more visualization results, as well as user study, can be found in the Appendix.
5.3 Effect of Sample Count and Selection Ratio
To validate the impact of sample count and selection ratio in our preference curation pipeline, we conduct experiments on two contrasting architectures: C-SD2 and C-FLUX. For each model, we vary the total number of rollouts M and the selection ratio N/M. The results reveal several consistent patterns and architecture-specife sensitivities:
Larger M generally improves performance, but with diminishing returns. For fixed N/M, increasing M consistently improves training stability and final reward, although the benefit decreases as M increases (e.g., M 64).
DP'O-SR (C-FLUX) is significantly more stable than OP'O-SR (C-SD2). C-SD2 exhibits reward collapse under low N or high N/M settings, likely due to overfitting sparse low-contrast supervision. In contrast, C-FLUX maintains stable and monotonic reward curves across most configurations, implying resilience noisy weak preference signals.
Both models exhibit architecture-speeffic optimal N/M regimes. For C-SD2, N/M = 1/4 yields the best trade-off between stability and reward contrast; lower ratios often lead to collapse, while higher ratios converge more slowly. C-FLUX, on the other hand, performs best at lower ratios such as 1/16, but suffers degraded performance at 1/2. This suggests that more capable models can effectively learn from stronger contrast signals, whereas smaller models benefit from greater redundancy and smoother gradients.
These results highlight the importance of tailoring preference curation strategies to model capacity. While moderate N/M ratios consistently perform well, the optimal configuration depends on the model's ability to generalize from noisy or low-contrast supervision. Our approach provides a scalable framework for systematically exploring these trade-offs. Based on these findings, we select N = 8 and M 32 for C-SD2, and N and M = 64 for C-FLUX in all experiments.
5.4 Stochasticity and the Effect of OP-R
Diffusion and flow- SR models generate diverse outputs due to their inherent stochasticity. To analyze this, we sample M outputs per input and compute perceptual reward statistics: Best@ M. Mean @ M, and Worst@ M. As shown in Fig. 5, for both C-SD2 and C-FLUUX baselines, Mean @ M remains relatively stable as M increases, suggesting that single sample is generally representative
8

---

Ewrzion
NN 1.10
1/M
SBM 1-16
NM 1100
SUM 1164
345
0.01
2.00
0.00
345
040
O-SR
O-SR (SD2)
DP' (FLUX)
Figure 4: Training curves of OP'O-SR on SD2 and FLUX under varying M and N/M configurations.
Larger M generally improves reward, while optimal N/M differs across model capacities.
pereet an Reverd
Pertaet mausk-alho Navard
0.60
6.55
0.55
8.50
0.50
0.45
Reig 120
0.45
Enggh J.RI0
1--00)
0.40
0.40
0.35
0.35
Number
Sample Number M
Perceptual reward statistics C-SD2
Pereeptual reward statistics C-FLUX
Figure Perceptual reward statistics by increasing sample count M on (a) C-SD2 and (b) C- FLUX backbones. In both cases, the baseline models (solid) show widening quality range with more samples, while DP"O-SR (dashed) consistently improves statistics especially Worst@ higghliiinn stronger robustness and reduced output variability.
of average performance. contrast, Best@ M increases and Worst@ decreases with larger M, indicating increased variability in output quality. DPRO-SR consistently improves all the three statistics across both backbones, with the most notable gain in Worst@ M. This indicates that our method not only improves average and best-case outcomes, but more importantly, raises the quality floor-leading to more consistent and perceptually robust outputs.
To further assess model stability, we follow the evaluation protocol of CCSR [37], randomly sampling 10 outputs per input and computing the mean and standard deviation of perceptual scores. As shown in Tab. 2, DPRO-SR achieves both better average performance and lower variance across various metries, confirming its superior robustness and reliability.
5.5 Global Reward, Local Refiner
Although our reward function composed entirely of global IQA metries, which assess overall image quality, we observe an intriguing behavior: localized refinement. As shown in Fig. 6, we compare the outputs of baseline model C-FLUX and its improved variant OP2O-FLUX across three random seeds, using the same L.R input. All inference hyperparameters, including the number of inference steps (25) and classifier-free guidance scale (CFG=2.55) are kept fixed ensure controlled comparison. The only varying factor is the random seed. Two noteworthy observations can be made.

---

Table 2: Performance comparison of different methods on RealSR bench [5].
Method
LPIPS)
TOPIQ-FRT
AFINE-FR)
MANIQAT
MUSIQU
CLIPIQA+1
TOPIQ-NR" AFINE-NR,
QALIGNT
C-SD2
0.41620.018
0,473+0.022
-06988002299
0.66410.019
70.34a1.79
0.730.0.025
-L.02200055
1630.0.187
OP'O-SR (SD2)
0.40540.009
0.46720.013
0,705400122 1.3329.0.150
73.24
0.78420.017
0.74520.015
-L1177000011 4.01700.117
C-FLUX
0.400a0.023
0.48060.027
00.3990.224
0.66540.025
69.70&2.15
0.68240.036
0.64420.046
-L.9660.070 1.644:0.211
DPOO.SR (FLUX)
0.40340.013
0.48550.016
17549920.176
0.89420.133
72.7840.33
0.75840.019
0.743=0.015
-1.1000.0047
4.143=0.113
LR Image
C-FLUX, Seed
C-FLIX,, Seed
C-FLLX, Seed
GT
DLUX, Seed
DPOO-FLUX, Seed
DP FLUX, Seed 3
Figure 6: Comparison between C-FLUX and DP*O-FLUX across three random seeds. Trained with global IQA rewards, OP"O-FLUX demonstrates localized refinement (e.g., wing structure), while keeping other regions (e.g., head reflections) almost unchanged.
Seed Sensitivity in Local Details. Even within the same model (C-FLUX or DP'O-FLUX), varying the random seed leads to noticeable differences in local structures, such as wing venation and insect head details. This highlights the stochastic nature of diffusion-based generation, especially with under-constrained guidance.
Local Enhancement from Global Reward. More surprisingly, under the same seed, DP'O-FLUX
often produces visibly sharper and more accurate local structures compared C-FLUX. For instance, the wing texture (red box) is significantly refined and more faithful to the ground truth, while other regions-such the specular highlight the insect's head (green box, white (rrow)-remane largely unchanged. This suggests that the model implicitly learns to prioritize perceptually salient regions, even though the reward is computed globally across the entire image.
This phenomenon implies that preference
lead to localized improvements without any explicit local supervision. training, when guided by global IQA rewards, can
The ablation study on the effectiveness of HPO can be found in the Appendix.
Conclusion
We introduced DP'O-SR, preference-driven framework for Real-ISR. We defined a perceptual reward that jointly considered naturalness and fidelity. Different from traditional best-vs-worst DPO sampling, we leveraged multiple pairwise comparisons within each group, leading to significantly improved optimization. We further analyzed how the optimal usage of preference samples varied across model scales and proposed a practical hyperparameter optimization strategy adaptively balance generalization and stability. Extensive experiments demonstrated that our approach brought notable gains over strong baselines, both perceptually and quantitatively.
Limitations. DP'O-SR has two main limitations. First, although the IQA-based reward correlates reasonably with human preference, it lacks interpretability and does not fully capture subjective perceptual quality. Designing more accurate and explainable reward models remains an important direction for future work. Second, current training pipeline is fully offline. Incorporating iterative or online preference optimization may further improve performance and adaptability.
10

---

References
[1] Stability.ai.
[2] Yuang Ai, Xiaoqiang Zhou, Huaibo Huang. Xiaotian Han, Zhengyu Chen, Quanzeng You, and Hongxia Yang. Dreamclear: High- capacity weald image restoration with privacy-safe dataset curation. Advances in Neural Information Processing Systems, 37:55443- 55469, 2024.
[3] Yochai Blau and Tomer Michaeli. The perception-distorion tradeoff. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 6228-6237, 2018.
[4] Ralph Allan Bradlcy and Milton E Terry. Rank analysis incomplete block designs: I. the method of paired comparisons. Biometrika, (343344:324-45555 1952.
[5] Jianrui Cai, Hui Zeng, Hongwei Yong. Zisheng Cao, and Lei Zhang. Toward real-world single image super-resolution A new benchmark and a new model. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1086-3095, 2019.
[6] Bin Chen, Gehui Li, Rongyuan Wu, Xindong Zhang, Jie Chen, Jian Zhang, and Lei Zhang. Adversarial diffusion compression for real-world image super-resolution. arXiv preprint
133383,, 2024.
[7] Chaofeng Chen, Jiadi Mo, Jingwen Hou, Haoning Wu, Liang Liao, Wenxiu Sun, Qiong Yan, and Weisi Lin. Topiq: A top-down approach from semantics to distortions for image quality assessment. IEEE Transactions on Image Procesing, 2024.
[8] Chaofeng Chen, Xinyu Shi, Yipeng Qin, Xiaoming Li. Xiaoguang Han. Tao Yang, and Shihui Guo. Real-world blind super-resolution via feature matching with implicit high-resolution priors. In Proceedings of the 30th ACM International Conference on Multimedia, pages 1329-1338, 2022.
[9] Chen, Tianhe Wu, Kede Ma, and Lei Zhang. Toward generalized image quality assessment: Relaxing the perfect reference quality assumption. arXiv preprint arXiv:2503. 15X.:22503.1121,, 2025.
[10] Xiangyu Chen, Xintao Wang, Jiantao Zhou, Yu Qiao, and Chao Dong. Activating more pixels in image super-resolution transformer. In Proceedings of the IEEF/CVE Conference on Computer Vision and Pattern Recognition, pages 22367-22377, 2023.
[11] Paul Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep rein- forcement learning from human preferences. Advances in neural information processing systems, 30, 2017.
[12] Tao Dai, Jianrui Cai, Yongbing Zhang, Shu-Tao Xia, and Lei Zhang, Second-order attention network for single image super-resolutio. In Proceedings of the IEEE/CVF conference on computer vision and puttern recognition, pages I 10055 1 11074, 2019.
[13] Chao Dong, Chen Change Loy, Kaiming Hc, and Xiaoou Tang. Learning deep convolutional network for image super-resolution. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6- 2014, 4, Proceedings, Part IV 13, pages 184-199. Springer, 2014.
[14] Linwei Dong, Qingnan Fan, Yihong Guo, Zhonghao Wang, Qi Zhang, Jinwei Chen, Yawei Luo, and Changqing Zou. Tsd-sr: One-step diffusion with target score distillation for real-world image super- resolution. arXiv preprint arXiv:241 I. 18263, 2024.
[15] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high- rightresouution image synthesis. arXiv preprint 2rXiv:2403.032206 2024.
[16] Hang Guo, Jimmin Li, Tao Dai, Zhihao Ouyang, Xudong Ren, and Shu-Tar Xia. Mambair: A simple bascline for image restoration with state-space model. In European conference on computer vision, pages 222 241. Springer, 2024.
[17] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
[18] Zijing Hu, Fengda Zhang, Long Chen, Kun Kuang, Jiahui Li, Kaifeng Gao, Jun Xiao, Xin Wang, and Wenwu Zhu. Towards better alignment: Training diffusion models with reinforcement learning against sparse rewards. arXiv preprint arviv:2003. /240, 2025.
[19] Junjie Ke, Qifei Wang, Yilin Wang, Peyman Milanfar, and Feng Yang. Musiq: Multi-scale image quality transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5148-5157, 2021.
11

---

[20] Black Forest Labs. Flux. https: httpe////gghhuub.oommbbbaaaeee
wwt.:/cccbbo//a......e.ee......a http//aaakkkoo soreck--oorett-absEEIEle 2024.
[21] Nathan Lambert, Louis Castricato, Leandro von Werra, and Alex Havrilla. Illustrating reinforcement learning from human feedback (rlhf). Hugging Face Blog, 2022. https//huugiingface.oo/ooooo/o/
[22] Christian Ledig, Lucas Theis, Ferene Huszár, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, ct al. Photo- photo-rcalistic single image super- resolution using generative adversarial network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4681-4690, 2017.
[23] Yawci Li, Yuchen Fan, Xiaoyu Xiang, Denis Demandolx, Rakesh Ranjan, Radu Timofte, and Luc Van Gool. Efficient and explicit modelling of image hierarchies for image restoration. In PuuPre Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18278-18289. 2023.
[24] Jie Liang, Hui Zeng, and Lei Zhang. Details artifacts: A locally discriminative learning approach realistic image super-resole resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 5657-5666, 2022.
[25] Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Lue Van Gool, and Radu Timofte. Swinir: Image restoration using swin transformer. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1833-1844, 2021.
[26] Zhanhao Liang, Yuhui Yuan, Shuyang Gu, Bohan Chen, Tiankai Hang, Ji Li, and Liang Zheng. Step-aware preference optimization: Aligning preference with denoising performance at each step. arXiv preprint arXiv:2406.04314 2(3), 2024.
[27] Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, and Kyoung Mu Lee. Enhanced deep residual networks for single image super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 136- 144, 2017.
[28] Xingi Lin, Jingwen He, Ziyan Chen, Zhaoyang Lyu. Bo Dai, Fanghua Yu, Yu Qiao, Wanli Ouyang. and Chao Dong. Diffbir: Toward blind image restoration with generative diffusion prior. In European Conference on Computer Vision, pages 430-448. Springer, 2024.
[29] Buhua Liu, Shitong Shao, Bao Li, Lichen Bai, Zhiqiang Xu, Haoyi Xiong, James Kwok, Sumi Helal, and Zeke Xie. Alignment of diffusion models: Fundamentals, challenges, and future. arXiv preprint vrXiv:2409.072533 2024.
[30] Runtao Liu, Haoyu Wu, Zheng Ziqiang. Chen Wei, Yingging He, Renjie Pi, and Qifeng Chen. Videodpo: Omni-preferennee alignment for video diffusion generation. arXiv preprint arXiv:2412. 14167, 2024.
[31] Nanye Ma. Mark Goldstein, Michael S Albergo, Nicholas M Botti, Eric Vanden-Eijnded and Saining Xie. Sit: Exploring flow and citlusion-based generative models with scalable interpolant transfommers. In European Conference on Computer Vision, pages 23-40. Springer, 2024.
[32] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730-277444 2022.
[33] Yunpeng Qu, Kun Yuan, Kai Zhao, Qizhi Xie, Jinhua Hao, Ming Sun, and Chao Zhou. Xpsr: Cross-modal priors for diffusion--bsed image super-resolution. In European Conference on Computer Vision, pages 285 3033 Springer, 2024.
[34] Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728-5533 53741,, 2023.
[35] Yang Song, Jascha Sohl-Dickstein, Diederík P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-hased generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456 2020.
[36] Lingchen Sun, Rongyuan Wu, Zhiyuan Ma, Shuaizheng Liu, Qiaosi Yi, and Lei Zhang. Pixel-level and semantic-level adjustable super-resolution: A dual-lora approach. arXiv preprint arXiv:2412.03017 2024.
[37] Lingchen Sun. Rongyuan Wu, Zhenggiang Zhang, Hongwei Yong, and Lei Zhang. Improving the stability of diffusion models for content consistent super-resolution. arXiv preprint arXiv:2401.0000 000877. 2023.
[38] Hossein Talebi and Peyman Milanfar. Nima: Neural image assessment. IEEE transactions on image processing, 27(8):3998 4011, 2018.
12

---

[39] Patrick von Platen. Suraj Patil, Anton Lozhkov. Pedro Cuenca, Nathan Lambert, Kashif Rasul, Mishig Davaadorj. Dhruv Nair, Sayak Paul, William Berman, Yiyi Xu, Steven Liu, and Thomas Wolf. Diffusers: State- diffusion models. httpp:///gtthii 2025-05-09.
wii.aaaeeed....rrrrrrrr 2022. Accessed:
[40] Bram Wallace, Meihua Dang, Rafael Rafailov, I Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong. Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 2228-823..22 2024.
[41] Jianyi Wang, Kelvin CK Chan, and Chen Change Loy. Exploring clip for assessing the look and feel of images. In Proceedings the AAAI Conference on Artificial Intelligence, volume 37, pages 2555-2563, 2023.
[42] Jianyi Wang. Zongsheng Yue, Shangehen Zhou, Kelvin CK Chan, and Chen Change Loy. Exploiting diffusion prior for real-world image supoutresolution arXiv preprint arXiv:2305. 2023.
[43] Xintao Wang. Liangbin Xie, Chao Dong. and Ying Shan. Real- Real-esrgan: Training weal-world blind super- resolution with pure synthetic data. In Proceedings of the IEEE/CVF international conference on computer vision. pages 1914, 2021.
[44] Xintao Wang. Ke Yu, Shixiang Wu, Jinjin Gu, Yihao Liu, Chao Dong, Yu Qiao, and Chen Change Loy. Esrgan: Enhanced super-resolution generative adversarial networks. In Proceedings of the European conference on computer vision (ECCV) workshops, pages 0-0, 2018.
[45] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing. 13(4)x600-612, -12, 2004.
[46] Haoning Wu, Zicheng Zhang, Weixia Zhang, Chaofeng Chen, Liang Liao, Chunyi Li, Yixuan Gao, Annan Wang, Erli Zhang, Wenxiu Sun, Q-aligm: Teaching Imms for visual scoring via discrete text-defined levels. arXiv preprint arXiv:2312.1700 2023.
[47] Rongyuan Wu, Lingchen Sun, Zhiyuan Ma, and Lei Zhang. One-step effective diffusion network for real-world image super-resolution. Advances in Neural Information Processing Systems, 37:92529-92553 2024.
[48] Rongyuan Wu, Tao Yang. Lingehen Sun, Zhengqiang Zhang, Shuai Li, and Lei Zhang. Seesr: Towards semantics-aware real-world image super-resolution. In Proceedings of the IEEF/CVF conference on computer vision and pallem recognition. pages 25467, 2024.
[49] Tianhe Wu, Jian Zou, Jie Liang, Lei Zhang, and Kede Ma. Visualquality-rl: Reasoning-induced induced image quality assessment via reinforcement learning to rank. arXiv preprint arXiv:2505. 14460, 2025.
[50] Rui Xie, Ying Tai, Kai Zhang, Zhenyu Zhang, Jun Zhou, and Jian Yang. Addsr: Accelerating diffusion- based blind super-resolution with adversarial diffusion distillation. arXiv preprint arXiv:2404.01711 2024.
[51] Sidi Yang, Tianhe Wu, Shuwei Shi, Shanshan Lao, Yuan Gong, Mingdeng Cao, Jiahao Wang, and Yujiu Yang. Maniqa: Multi-dimension attention network for no-reference image quality assessment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1191-1200, 2022.
[52] Tao Yang, Peiran Ren, Xuansong Xie, and Lei Zhang. Pixel-awarc stable diffusion for realistic image super-resolution and personalized stylization. arXiv preprint arXiv:2308. 14469, 2023.
[53] Zongsheng Yue, Kang Liao, and Chen Change Loy. Arbitrary-steps image resortienolution via diffusion inversion. arXiv preprint arXiv:2412.0900 2024.
[54] Zongsheng Yue, Jianyi Wang, and Chen Change Loy. Resshift: Efficient diffusion model for image super-resolution by residual shifting, arXiv preprint arXiv:2307. 12348, 2023.
[55] Kai Zhang, Jingyun Liang, Luc Van Gool, and Radu Timofte. Designing a practical degradation model for deep blind image super-resolution. Proceedings of the IEEE/CVF International Conference on Computer Vision pages 4791-4800, 2021.
[56] Leheng Zhang, Yawei Li, Xingyu Zhou, Xiaorui Zhao, and Shuhang Gu. Transcending the limit of local window: Advanced super- resolution transformer with adaptive token dictionary. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 2856-2866 2024.
13

---

[57] Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-lo-image diffusion models. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 3836-3847, 2023.
[58] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechuman, and Oliver Wang. The unreasonable effectiveness deep features perceptual metric. Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586-595, 2018.
[59] Xindong Zhang, Hui Zeng. Shi Guo, Lei Zhang. Efficient long-range attention network image super-resolution. In European Conference on Computer Vision, pages 649 667. Springer, 2022.
[60] Yulun Zhang. Kunpeng Kai Li, Lichen Wang, Bineng Zhong, and Yun Image super-resolution using very deep residual channel attention networks. In Proceedings of the European conference on computer vision (ECCV), pages 286-301, 2018
[61] Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences, arXiv preprint arXiv: / 19999989999 2019.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly state the main contributions, which propose new paradigm for enhancing diffusion-based Real-ISR performance. These claims are well-supported by experiments and empirical results throughout the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations in the "Conclusion and I. Limitation" section.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model cell-speciiiiatinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
15

---

Answer:
[NA]
Justification:
We do not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We have given the detailed experimental setting for reproducing in the "Exper- iment" section.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
16

---

reproducible, they Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
The code and data will released upon publication, but are not included in the supplementary materials they are still being organized.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (uttps://nips.cm cc/
Sewiiidee//odeSuumiiss
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: / httppsscccppbccccccceecce hewbblicguiidee CodeeSbbm ssi asionPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We have presented all the experimental settings the "Experiment" section.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [NA]
Justification: We follow the same experimental settings and comparison steps as previous work, which do not include such experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
17

---

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have reported the details of the system in the "Experiment" section.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: : /neurips..... cc wccppublic/Etiii ctw.diiieeiiiieieeneennenn
Answer: [Yes]
Justification:
The research fully complies with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consider- ation due laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is no socictal impact of this research.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [No]
Justification: We follow previous works that do not include such safeguards. The safeguards are valuable. We will consider this in future work.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We have cited all the referenced papers.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We will release the detailed document of codes and models.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/cooe/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The research does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
20

---

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to LLM policy (https: / wwt.e////oooeeeeeee////OO/O /LLM) for what should or should not be described.
21

---

A Appendix supplemental material
In this supplementary file, provide the following materials:
The detailed network architecture of C-SD2 and C-FLUX.
The ablation study on the effectiveness of HPO.
More visual comparisons.
Results of user study.
A.1 The Architecture of C-SD2 and C-FLUX
Fig. 7 illustrates the architectures of the two baseline models, C-SD2 [1] and ControlNet-FLUUX [20], both integrated with the proposed DP"O-SR module. The left subfigure shows C-SD2, which is built upon UNet-based architecture. Its control branch is initialized by duplicating the entire encoder of SD2. Following standard ControlNet paradigm [57], the low-resolution (LR) condition first encoded by image encoder and then added to the input of the frozen pre-trained backbone. The resulting combined features are then processed by the control branch. In C-SD2, the control features injected into the decoder blocks of the UNet.
The right subfigure presents C-FLUX, which adapts the MMDiT-based [20] architecture. reduce computational overhead, its control branch is initialized by copying only the first four Double Transformer Blocks from the pre-trained model. Similar C-SD2, the LR condition encoded and fused with the model's input before being passed to the control branch. However, in contrast to C-SD2, the control features C-FLUX are added to the intermediate representations within the Double Transformer blocks, rather than the decoder. This architectural difference reflects the underlying backbone discrepancy between UNet and MMDiT. All pretraining procedures follow official diffusers [39] training recipes ensure consistency.
A.2 Effectiveness of HPO
To assess the contributions of the two components in our hierarchical preference optimization (HPO), conduct an ab- lation study by separating the effects of intra-group and inter-group weighting. As shown in Tab. 3, applying only intra-group Brighting-ffcussie on local contrast within each SR candidate group-alres already provides noticeable improvement over the baseline. Similarly, using only inter-group group weighting, which emphasizes groups with higher reward dispersion, also leads to better performance. Combining both strategies yields the highest perceptual reward, confirming that the two weighting mechanisms are complementary.
Table 3: Ablation study on HPO.
base
intra
inter
Perceptual Reward 0.645 0.648 0.649 0.651
LR
Condition
Control Part
Control
Part
Initiation
Initiation
Transforme
Ennender
UNet
UNet Decoder
Double
Single
Imput
Ingut
Prompt
Prompt
Time
Time
Figure 7: The illustration of baselines (C-SD2 and C-FLUX) equipped with DP2O-SR.
22

---

A.3 More Visual Comparisons
We present additional qualitative comparisons to demonstrate the effectiveness of our proposed method O-SO over the two strong baselines: C-SD2 and C-FLUX. For each baseline, we show visual comparisons between it and its improved versions, i.e., C-SD2 vs. OP"O-SR (SD2) and C-FLUX vs. DP2O-SR (FLUX), alongside results from several state-of-the-ar methods, including StableSR [42], DifIBIRv2 [28], SeeSR [48], CCSR [37], AddSR [50], and OSEDiff [47].
Below, first present four representative cases based the C-SD2 baseline, shown in Fig. 8. These comparisons highlight the improvements made by O-SR (SD2) in terms of structural fidelity and visual detail, well as its advantages over existing SOTA methods.
Text Restoration. In this example, our method demonstrates clear superiority in restoring corrupted text. The baseline C-SD2 fails to recover the correct structure of the letters, especially the letter "m", which is severely distorted. In contrast, DP"O-SR (SD2) produces accurate and sharp letterforms, closely resembling the ground-truth. Notably, the structure of the letter "c" is also cleaner and more distinguishable. While methods like OSEDiff and SeeSR partially recover "e", their outputs remain ambiguous, often appearing between "e" and "o".
Case 1:
Case 2: Roof Tile Reconstruction. The second case highlights the reconstruction of architectural details, specifically roof tiles. Although C-SD2 manages to recover tile-like structure, the spacing and arrangement are overly dense compared to the ground-truth. DP"O-SR (SD2) significantly improves the regularity and spacing of the tiles, resulting in pattern much more faithful to the original. Other SOTA methods struggle in this case, with most failing to recover the tile layout.
Case 3: Road Surface Texture. In the third example, we examine the restoration of road textures and surface patterns. C-SD2 restores the general layout of the path but incorrectly renders the gaps between the bricks as white. Our improved model, OP"O-SR (SD2), not only recovers the brick layout more accurately but also generates clearer and more structurally consistent gap lines. Competing methods mostly fail to reconstruct the pattern or texture meaningfully.
Case 4: Building Structure Recovery. The fourth case focuses complex architectural structures. While C-SD2 recovers a reasonable amount of building details, DP"O-SR (SD2) significantly enriches the reconstruction with finer structural elements and sharper contours. Compared other methods, the output of DP2O-SR (SD2) shows clear advantages in both structural fidelity and visual realism. Although methods like OSEDiff and DiftBIRv2 generate plausible building forms, their results often include distorted or wavy line structures.
We then present visual comparisons between C-FLUX and DP'O-SR (FLUX), as shown in Fig. 9, to further validate the generality and robustness of our method.
Case 1: Dice Number Restoration. In this example, most competing methods fail to reconstruct the dice numbers accurately. While C-FLUX is able to recover the correct "5" on the lower-left white dice, it mistakenly generates "9" on the upper blue and red dice, deviating from the ground truth. In contrast, our OPRO-SR (FLUX) restores the correct "6" patterns with sharp and well-aligned dots, achieving the highest structural fidelity among all methods.
Case 2: Parrot Eye and Beak Details. In this case, C-FLUX fails to recover the black eye of the red parrot, resulting in an unnatural facial appearance. DPPO-SR (FLUX) successfully reconstructs the eye region while maintaining a coherent and realistic texture. Although AddSR also restores the eye, it introduces an unnaturally rigid beak that does not exist in the ground-truth. Overall, DP*O-SR (FLUX) provides more natural and visually consistent reconstruction.
Case 3: Artifact Removal and Structural Clarity. For the third example, C-FLUX produces noticeable vertical artifacts that degrade the overall visual quality. Our DPRO-SR (FLUX) effectively suppresses these artifacts while preserving local details and edge sharpness. In comparison, methods such as StableSR, D and OSEDiff exhibit strong "oil-painting" effect, leading to flatter textures and less realistic structures.
Case 4: Fence Reconstruction. In the final case, C-FLUX fails to reconstruct the fence structure, resulting missing geometric elements. DP"O-SR (FLUX) not only restores the complete fence but also aligns well with the ground-truth. Although DiffBIRv2 and OSEDiff generate fence-like patterns, their results appear noisy and irregular. These comparisons further demonstrate that DP2O-SR (FLUX) achieves superior structural accuracy and perceptual realism across diverse scenes.
23

---

A.4 User Study
In the user study, we conduct two categories of experiments. In the first category, we evaluate the effectiveness of DP²O through pairwise comparisons between the DP'O-enhanced model and its original (non-enhanced) counterpart. In the second category, we perform multi-way way comparison between the DP O-enhancde model and existing state-of-the-ar methods.
In each trial of the experiments, participants presented with low-quality (I-Q) image along with either two images (for pairwise comparisons) or seven images (for multi-way comparison), and are asked the following question:
Given the LQ image, which one of the candidate images is the best high-quality version of it?
For each backbone model, we invited 10 participants to evaluate 20 distinct scenarios. In each scenario, the participants completed two pairwise comparisons and multi way comparison, resulting in 3 evaluations per scenario. This yielded a total of 10 20 x = 400 evaluations per backhone. Since the experiments were conducted 2 x 400 = 800 evaluation outputs.
backbones, the entire user study comprised
The results of our user study are summarized in Fig. 10. In the pairwise comparisons (top row), participants consistently preferred the DP O-enhannadeed models over the original pre-trained models. Specifically, for C-SD2 (Fig. 10a), 67.5% of selections favor DP over the pre-trained model. For C-FLUX (Fig. 10b), D outperforms the pre-trained with preference rates 69.5%.
In the multi-way comparison (bottom row), where participants selected the most plausible high- resolution candidate among seven methods, DP again performs the best. For C-SD2, DP accounts for 28.0% of total votes, ahead of OSEDiff (19.5%) and SceSR (18.5%). For C-FLUX, OP receives a dominant 44.0% of the votes, significantly outperforming the second-best method, OSEDiff (12%).
These results collectively demonstrate the consistent superiority of OP"O-SR across diverse backbone models and evaluation settings, reflecting its robustness and perceptual quality gains over baselines and state-of-the-art alternatives.
24

---

(())
(DM
(i)
eduroam
odaroom
o
oderoam
oduroam
edaroaa
LR Image
StableSR
SceSR
DimBiRy2
C-SD2
aderoom
odorooms
oduroxm
((R))
(()))))
((M
eduroam
odaronir
odoroon
couroam
eduroam
GT Image
CCSR
AddSR
OSEDif
DP SR (SD2)
LR Image
StableSR
SceSR
DiMBBRRv
C-SD2
mw
mtt
Jadshh
GT Image
CCSR
AddSR
OSEDiT
OPSR (SD2)
Image
StableSR
DifBIRv2
C-SD2
GT Image
CCSR
AddSR
OSEDif
(SD2)
LR Image
StableSR
SccSR
DitBBRRy2
C-SD2
Image
AddSR
OSEDII
DPOSSR (8D22
Figure 8: Qualitative comparison between C-SD2, DP*O-SR (SD2), and other Real-ISR methods.
Zoom in for visual details.
25

---

LR Image
StableSR
SecSR
DiABIRv2
C-FLUX
28
01
GT Image
CCSR
AddSR
OSEDIT
DPPO-SR (FLUX)
LR Image
StableSR
SceSR
DifBBIRv
C-FLUX
GT Image
CCSR
AddSR
OSEDim
DP'O-SR (FLUX)
IR Image
StableSR
SeeSR
DiMBRvv
C-FLXX
GT Image
CCSR
AddSR
OSEDif
DPO-SR FLUX
LR Image
StableSR
SecSR
DiABiRRv
C-FLUX
GT Imuge
CCSR
AddSR
OSEDift
DPOSS FLUX
Figure 9: Qualitative comparison between C-FLUX, DPO--R (FLUX), and other Real-ISR methods. Zoom in for visual details.
26

---

DPO-SR (SD2)
C-SD2
32.5%
DPO-SR (FLUX)
C-FLUX
30.5%
AARAR
ABISEL
GUI
IL LLS
arla
1991
OFF
Swcttt
12.%%
GRI
(a) User study results on C-SD2. Top: pair- wise comparison between DP'O and its base- line. Bottom: multi-way preference distribu- tion among seven methods.
(b) User study results on C-FLUX. Top: pair- wise comparison between DP and its basc- line. Bottom: multi-way preference distribu- among seven methods.
Figure 10: Summary of user study results C-SD2 and C-FLUX. In both pairwise and multi-way way comparisons, OP consistently outperforms baselines and state-of-ther oh-the-rrt methods, indicating its strong perceptual advantage in recovering high-quality images from low-resolution inputs.
27


Reviews:
[{"review_id": "edSMmrBe80", "reviewer": "Reviewer_iowj", "summary": "The authors propose a novel model for Real-World Image Super-Resolution. The approach utilizes a pretrained model as a reference for generating super-resolution (SR) candidates from a given low-resolution input. These candidates are then evaluated using a composite metric called CPS, which ranks them from top to bottom. In the subsequent training phase, one top-ranked and one bottom-ranked candidate are selected and combined for optimization using Direct Preference Optimization (DPO).", "questions": "* What is the impact of using only FR or NR metrics in the model?\n\n* Which IQA metric is most similar to CPS?. This analysis could provide deeper insight into why CPS performs well for both ranking and significance weighting.\n\n* Why are FR and NR metrics given equal importance, considering NR includes six metrics while FR reports only three?\n\n* What is the performance gap when using random pair selection instead of hierarchical significance weighting?. This experiment can help establish a lower bound for comparison with the proposed approach.\n\n* Can the intra-pair strategy be implemented by simply selecting the top- or bottom-1 candidate and selecting all counterparts' samples (i.e., selecting top-1 and then selecting several samples of the bottom part? Or is an additional condition required to maximize variance?\n\n* Is there any negative impact if the dataset contains an imbalance between positive and negative samples?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "While the model demonstrates superior performance compared to all baselines, several concerns remain regarding the clarity and justification of key components. First, CPS is introduced as a composite metric, however, there is no clear motivation for using all IQA metrics. A more principled justification or ablation study would strengthen the argument. Second, lines 135–136 state that “CPS is a relative score computed within the compared set S,” yet Tables 1, 2, and 3 report CPS scores across different models. This contradicts the definition and raises questions about the validity and interpretability of these comparisons between models. Third (minor), the intended message of Figure 3a-b is unclear. Both images appear visually similar, and the role of the downward arrow is not explained, reducing the figure’s impact. Fourth, Section 3.2 describes a core component of the model, but is only briefly described in the main paper. Key algorithmic steps and equations are presented in the supplementary material, which may hinder comprehension for readers.", "was_revised": true, "final_justification": "The authors did an excellent job clearing my concerns. I believe that the paper will be improved with the new details and results.  I raised my vote to weak accept", "rebuttal": "We thank the reviewer iowj  for his/her valuable feedback. For brevity, we use `W#` for weaknesses and `Q#` for questions (e.g., W1, Q1) throughout our response.\n\n---\n\n**[W1, Q1 and Q3]: Why use all IQA metrics? Why not only use FR or NR metrics as reward? Why are FR and NR metrics given equal importance in CPS formulation?**\n\nWe thank the reviewer for the insightful questions.\n\nWe do not use all IQA metrics indiscriminately. Instead, we carefully select 3 full-reference (FR) and 6 no-reference (NR) metrics commonly used in perceptual quality assessment.\n\nFR metrics assess fidelity and structural similarity to ground truth, but relying solely on them can produce less vivid outputs. NR metrics promote perceptual quality without reference, encouraging more appealing visuals. However, NR metrics alone may lead to reward hacking or favor unrealistic generations—a concern also raised by Reviewer Jm8D (Q2) regarding CLIP-based NR metrics.\n\nTo investigate further, we conducted ablation studies using only FR or only NR metrics in CPS. As shown below, FR-only rewards improve fidelity but hurt perceptual quality, while NR-only rewards do the opposite. Combining both gives a better overall balance:\n\n| Method        | LPIPS ↓ | TOPIQ-FR ↑ | AFINE-FR ↓ | MANIQA ↑ | MUSIQ ↑ | CLIPIQA+ ↑ | TOPIQ-NR ↑ | AFINE-NR ↓ | QALIGN ↑ |\n|---------------|---------|-------------|-------------|-----------|----------|--------------|--------------|--------------|------------|\n|               |   FR    |     FR      |     FR      |    NR     |   NR     |     NR       |     NR       |     NR       |    NR      |\n| Only FR       | 0.428   | 0.442       | -1.144      | 0.631     | 70.5     | 0.724        | 0.689        | -1.003       | 4.083      |\n| Only NR       | 0.476   | 0.389       | -0.745      | 0.722     | 74.6     | 0.795        | 0.762        | -1.142       | 4.289      |\n| FR + NR (ours)| 0.447   | 0.418       | -1.008      | 0.687     | 73.8     | 0.776        | 0.745        | -1.110       | 4.342      |\n\nTo avoid imbalance from the unequal number of FR and NR metrics, we assign equal total weight to each group in the CPS formulation. This prevents overemphasizing the NR group and ensures that both fidelity and perceptual quality are equally represented during optimization.\n\nWe agree that visual examples would help illustrate these trade-offs. Due to rebuttal constraints, we will include qualitative comparisons in the final version.\n\n---\n\n**[W2]: Why are CPS scores compared across models if CPS is defined as relative within a set?**\n\nThank you for the thoughtful comment and for highlighting this potential confusion.\n\nTo clarify, CPS is not computed by normalizing each model’s outputs independently. Instead, for each input image, we collect outputs from *all models under comparison* (e.g., those in the same table), and apply min-max normalization jointly across this full set.\n\nThis ensures CPS reflects the relative perceptual quality across models, making scores directly comparable within the same evaluation group. Thus, CPS values in Tables 1–3 are valid for intra-table comparison, but not across different tables or experiments.\n\nWe appreciate the reviewer’s attention and will clarify this computation procedure in the revised manuscript.\n\n---\n\n**[W3]: HSW lacks detail in the main paper, relying on the supplement.**\n\nThank you for the suggestion. We will add the key HSW steps and equations to the main paper to improve clarity and self-containment.\n\n---\n\n**[W4]: The purpose of Figure 3a-b is unclear, and the downward arrow is unexplained.**\n\nThank you for the feedback. We will revise Figure 3 to clarify its message: (a) shows that increasing sampling times during preference data construction improves performance with best/worst-of-1 pairs; (b) illustrates a trade-off between data quality and quantity when using best/worst-of-N, with optimal performance at a moderate N. \n\nThe downward arrow indicates increased sampling during the preference data collection stage.\n\n---\n\n### **[Q2]: Which IQA metric aligns most closely with CPS?**\n\nThank you for the thoughtful question. We analyzed which IQA metrics align most closely with CPS and whether a reduced metric set could suffice for reward design.\n\nCPS is computed as the average of normalized FR and NR metrics. To ensure equal group contributions, each FR metric is weighted 1/6 and each NR metric 1/12. The table below shows the normalized scores of our model (DP²O-SD2), the weights, and each metric’s contribution:\n\n| Metric     | Group | Normalized Score | Weight | Contribution |\n|------------|--------|------------------|--------|--------------|\n| LPIPS      | FR     | 0.5015           | 1/6    | 0.0836       |\n| TOPIQ-FR   | FR     | 0.4777           | 1/6    | 0.0796       |\n| AFINE-FR   | FR     | 0.7072           | 1/6    | 0.1179       |\n| MANIQA     | NR     | 0.6693           | 1/12   | 0.0558       |\n| MUSIQ      | NR     | 0.8318           | 1/12   | 0.0693       |\n| CLIP-IQA   | NR     | 0.7228           | 1/12   | 0.0602       |\n| TOPIQ-NR   | NR     | 0.8167           | 1/12   | 0.0681       |\n| AFINE-NR   | NR     | 0.5725           | 1/12   | 0.0477       |\n| QALIGN     | NR     | 0.7632           | 1/12   | 0.0636       |\n| **Total**  | —      | —                | 1      | 0.6458       |\n\nAmong FR metrics, AFINE-FR, LPIPS, and TOPIQ-FR contribute most to CPS. For NR metrics, MUSIQ, TOPIQ-NR, and QALIGN are the top contributors.\n\nTo test if a reduced set suffices, we conducted ablations using only the top-k metrics from each group. This reflects the RealSR task’s need to balance **fidelity** (FR) and **perception** (NR). Results are shown below:\n\n| Metric     | Top-1 FR+NR (AFINE-FR, MUSIQ) | Top-2 FR+NR (+LPIPS, TOPIQ-NR) | Top-3 FR+NR (+TOPIQ-FR, QALIGN) | All metrics (ours) |\n|------------|------------------|-------------------|--------------------|------------------|\n| LPIPS      | 0.4694           | 0.4513            | 0.4461             | 0.4466           |\n| TOPIQ-FR   | 0.3974           | 0.4122            | 0.4216             | 0.4184           |\n| AFINE-FR   | -0.9426          | -0.9759           | -1.0110            | -1.0084          |\n| MANIQA     | 0.7011           | 0.6890            | 0.6806             | 0.6865           |\n| MUSIQ      | 74.2999          | 73.7008           | 73.5275            | 73.7686          |\n| CLIP-IQA   | 0.7719           | 0.7761            | 0.7755             | 0.7755           |\n| TOPIQ-NR   | 0.7418           | 0.7509            | 0.7466             | 0.7446           |\n| AFINE-NR   | -1.0903          | -1.0808           | -1.0927            | -1.1095          |\n| QALIGN     | 4.2324           | 4.2779            | 4.3241             | 4.3424           |\n| CPS↑       | 0.6223           | 0.6330            | 0.6536             | 0.6909           |\n\nWhile CPS is most influenced by a few strong metrics, using only top-k metrics consistently underperforms compared to using all. This suggests that the additional metrics provide **complementary, non-redundant signals**, supporting the need for a diverse set in reward design.\n\nWe will include this analysis and tables in the revised manuscript.\n\n---\n\n**[Q4]: What's the performance drop when replacing HSW with random pairing?**\n\nThank you for the helpful question.\n\nOur HSW strategy builds on the PPC sampling framework:\n\n- The **baseline** uses a fixed top-1 vs. bottom-1 pairing.\n- **PPC** introduces diversity by randomly pairing samples within top-N and bottom-N candidates.\n- **HSW** further improves this by assigning higher weights to more informative pairs during training.\n\nAs shown in Table 2(c), PPC improves CPS from 0.626 (baseline) to 0.632, and adding HSW further raises it to 0.641.\n\nWe do not perform fully random pairing (i.e., sampling any two outputs arbitrarily), as our method relies on reward-ranked candidates to ensure meaningful preference signals.\n\nWe will clarify this distinction and the ablation structure in the revised manuscript.\n\n---\n\n**[Q5 & Q6]: How do intra-pair strategies and class imbalance affect model performance? Is a simple top-1 vs. bottom-N pairing sufficient?**\n\nThank you for the insightful questions.\n\n**For Q5:**  \nWe tested a top-1 vs. bottom-N pairing strategy, where the best sample is paired with multiple negatives. As shown in the table (e.g., *Top1+Bottom10*, CPS = 0.685), this setup performs reasonably well but consistently lags behind our full method (CPS = 0.703).\n\nThis suggests that fixing the anchor to top-1 limits intra-pair diversity. In contrast, sampling from top-N and bottom-N introduces richer variation, which benefits preference learning.\n\n**For Q6:**  \nWe agree that imbalance can bias training. To mitigate this, we maintain symmetry between positive and negative pools (top-N vs. bottom-N). While exhaustive testing is infeasible, our experiments with asymmetric settings (e.g., Top1+Bottom4/7/10) show diminishing gains, supporting the effectiveness of balanced sampling.\n\nOverall, our strategy balances informativeness and robustness, which we find crucial for stable optimization. We will clarify these points in the revised version.\n\n| Method           | LPIPS↓ | TOPIQ-FR↑ | AFINE-FR↓ | MANIQA↑ | MUSIQ↑ | CLIPIQA↑ | TOPIQ-NR↑ | AFINE-NR↓ | QALIGN↑ | CPS↑  |\n|------------------|--------|-----------|-----------|---------|--------|----------|-----------|------------|---------|--------|\n| Top1+Bottom4     | 0.460  | 0.403     | -0.909    | 0.686   | 74.12  | 0.765    | 0.750     | -1.140     | 4.239   | 0.669  |\n| Top1+Bottom7     | 0.454  | 0.411     | -0.896    | 0.684   | 74.11  | 0.773    | 0.753     | -1.152     | 4.224   | 0.689  |\n| Top1+Bottom10    | 0.453  | 0.409     | -0.874    | 0.681   | 74.15  | 0.769    | 0.755     | -1.145     | 4.256   | 0.685  |\n| Ours             | 0.447  | 0.418     | -1.008    | 0.687   | 73.80  | 0.776    | 0.745     | -1.110     | 4.342   | 0.703  |"}, {"review_id": "JOOdulux4L", "reviewer": "Reviewer_Jm8D", "summary": "The paper proposes DP2O-SR (Direct Perceptual Preference Optimization for Super-Resolution), a preference-alignment framework that fine-tunes diffusion-based Real-ISR models toward higher perceptual quality. The method fine-tunes diffusion-based Real-ISR models for perceptual quality by (1) Perceptual Preference Curation (PPC), which generates M SR candidates per LR image and ranks them with a combined perceptual score (CPS), forming top/bottom N^2 preference pairs; (2) Hierarchical Significance Weighting (HSW), which scales each pair’s DPO loss by its CPS gap and cross-pair variance to focus learning on informative examples; and (3) a weighted Diffusion-DPO fine-tuning stage that updates the policy model starting from the frozen reference backbone.", "questions": "1. CPS is both the training signal and the primary benchmark. Please provide human opinion scores or at least FR metrics that were not part of CPS to show real perceptual gains. Under what conditions would the method still outperform when evaluated with MOS or FR metrics?\n\n2. Most practical SR systems return a single output. How does DP2O-SR perform when M = 1 at test time? Please report CPS/LPIPS/PSNR/SSIM for the best-of-1 setting.\n\n3. Many CPS components (e.g., CLIPIQA+, Q-Align) rely on CLIP embeddings that are vulnerable to adversarial textures. Have you observed failure cases where CPS prefers oversharpened or hallucinated details?\n\n4. HSW combines two factors. Provide separate ablations for (i) intra-pair only and (ii) inter-group only to isolate their contributions.\n\nScore could increase if (i) unbiased human-rated perceptual improvements are demonstrated; (ii) single-sample inference results remain competitive; (iii) clearer justification of CPS composition and normalisation is provided.", "limitations": "The authors acknowledge limited candidate sampling and small training set. They do not discuss risks of optimising against learned IQA metrics that may drift from human perception or be gamed. Please add discussion of CPS brittleness and potential misuse (e.g., hallucination of realistic but incorrect content).", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths:**\n1. This work bridges RLHF-style preference alignment and super-resolution, potentially inspiring wider adoption of preference learning in image restoration.\n2.  Experiments on a synthetic Syn-Test set and the RealSR benchmark show consistent CPS improvements over two ControlNet backbones (SD-2.0 and FLUX) and competitive/perceptually stronger results than prior Real-ISR methods (StableSR, DiffBIR, SeeSR, OSEDiff, etc.).\n3.  Ablation studies demonstrating the effect of M, N, and HSW provided\n\n**Weaknesses:**\n1. Main metric (CPS) is also the optimization target, risking circular evaluation; limited external metrics (NIMA, TOPIQ-IAA) are not enough to rule out over-fitting.\n2.  Real-world benchmark evaluation still averages over 20 stochastic samples which is unrealistic at inference time; single-sample quality is not reported.\n3.  CPS is a straightforward average of existing metrics; hierarchy weighting resembles curriculum/importance sampling that is incremental novelty.\n4. Preference-based alignment for diffusion models already explored (Diff-DPO [1], VideoDPO [2]); adaptation here is incremental.\n5. No statistical significance tests or variance across random seeds provided.\n\n[1] Wallace, Bram, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. \"Diffusion model alignment using direct preference optimization.\" In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8228-8238. 2024.\n\n[2] Liu, Runtao, Haoyu Wu, Ziqiang Zheng, Chen Wei, Yingqing He, Renjie Pi, and Qifeng Chen. \"Videodpo: Omni-preference alignment for video diffusion generation.\" In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 8009-8019. 2025.", "was_revised": true, "final_justification": "The rebuttal addresses many of the key concerns. The authors add public benchmark results (PIPAL), report statistical significance for gains, and provide details of a user study showing perceptual preference alignment. These significantly strengthen the empirical validity of the approach.\n\nSome limitations remain: the method is evaluated on only two diffusion-based models, and the CPS metric's robustness to CLIP-induced hallucination is not fully quantified. Still, the method represents a promising adaptation of preference learning to SR, and the presented results are compelling.\n\nI will revise my score acknowledging the paper’s technical merit and growing relevance of preference-driven optimization in perceptual image restoration.", "rebuttal": "We thank the reviewer Jm8D for his/her valuable feedback. For brevity, we use `W#` for weaknesses and `Q#` for questions (e.g., W1, Q1) throughout our response.\n\n---\n\n**[W1 & Q1]: Can you provide MOS or external FR metrics to validate gains?**\n\nThank you for the helpful comment. To mitigate the risk of circular evaluation with CPS, we include extensive qualitative comparisons (Appendix Sec. 4) and a user study (Appendix Sec. 5) based on human perceptual preferences. The study confirms that our method consistently outperforms baselines in terms of visual quality, providing strong evidence beyond the optimization metric.\n\nWe will clarify this point in the main paper to better highlight the role of human evaluation in validating perceptual improvements.\n\n---\n\n**[W2 & Q2]: Report CPS, LPIPS, PSNR, SSIM for M=1.**\n\nThank you for the valuable suggestion. To evaluate the model under realistic single-sample conditions, we report CPS, LPIPS, PSNR, and SSIM for the M=1 setting (i.e., one stochastic sample at inference). As shown in the table below, the results are very close to the M=20 averages reported in the main paper, indicating that our model maintains stable performance even without sampling multiple outputs.\n\nThis also aligns with our analysis in Fig. 4(a), where increasing M mainly expands perceptual diversity, while average quality remains consistent.\n\n| Method                    | PSNR ↑ | SSIM ↑ | LPIPS ↓ | CPS ↑ |\n|---------------------------|--------|--------|----------|------------------|\n| **M = 1**                 | 21.35  | 0.53   | 0.447    | 0.649            |\n| **M = 20 (paper report)** | 21.38  | 0.53   | 0.447    | 0.648            |\n\nWe will clarify this finding in the revised paper to emphasize the practicality of our method under single-sample inference.\n\n---\n\n**[W3]: CPS and HSW are incremental novelty.**\n\nThank you for the comment. While CPS builds on existing IQA metrics, its novelty lies in unifying full- and no-reference scores into a normalized preference signal tailored for Real-ISR—a setting where such integration has not been explored. This allows leveraging complementary strengths of multiple IQA metrics.\n\nHSW is inspired by importance sampling, but we design it specifically to balance intra-pair contrast and inter-group diversity within limited candidate sets—an essential factor for stable and effective DPO training in Real-ISR tasks.\n\nAlthough both components draw from existing ideas, their integration into a unified preference-driven framework, along with demonstrated consistent improvements across models and benchmarks, represents a practical and impactful contribution. We will clarify this in the revised version.\n\n---\n\n**[W4]: Preference alignment in diffusion has been explored (e.g., Diff-DPO, VideoDPO); this adaptation seems incremental.**\n\nThank you for the thoughtful comment.\n\nWhile our work is inspired by preference-based alignment frameworks like Diff-DPO and VideoDPO, we believe our contributions go beyond a direct adaptation and address challenges unique to RealSR:\n\n1. **Task-specific reward design:** RealSR demands a careful balance between perceptual quality and fidelity. We propose CPS, a unified reward that combines full- and no-reference IQA metrics—unlike prior work that focuses on semantic alignment (e.g., text-image relevance).\n\n2. **Systematic study of preference data construction:** We analyze how preference quality varies with sampling count and selection strategies (e.g., top-N vs. bottom-N), which is largely unexplored in prior diffusion-based alignment literature.\n\n3. **Hierarchical Significance Weighting (HSW):** Our weighting scheme accounts for both intra-pair contrast and inter-group variance, offering a more structured and informative signal than the binary win/loss formulation in standard DPO.\n\nTogether, these elements constitute a meaningful extension of preference-based alignment methods to low-level vision tasks. We will clarify these distinctions in the revised manuscript and better highlight the novelty relative to prior work.\n\n---\n\n**[W5]: No significance tests or seed variance reported.**\n\nThank you for pointing this out.\n\nTo address the concern about variability and robustness, we report the mean and standard deviation (± std) of 13 IQA metrics across 20 random seeds. The table below provides a direct measure of output stability under stochastic sampling. All models use the **same set of seeds** for fair comparison.\n\nWhile we did not conduct formal significance tests, we believe that reporting **seed-based variance** serves a similar purpose—demonstrating that the improvements are consistent and robust rather than due to random chance. We will clarify this intent in the revised manuscript.\n\n| Dataset   | Method             | PSNR ↑     | SSIM ↑    | LPIPS ↓   | TOPIQ-FR ↑ | AFINE-FR ↓ | MANIQA ↑  | MUSIQ ↑   | CLIPIQA ↑ | TOPIQ-NR ↑ | AFINE-NR ↓ | QALIGN ↑  | NIMA ↑    | TOPIQ-IAA ↑ |\n|-----------|--------------------|------------|-----------|------------|-------------|--------------|------------|------------|-------------|--------------|--------------|------------|------------|----------------|\n| Syn-Test | C-SD2              | **22.20±0.47** | 0.53±0.023 | 0.458±0.014 | 0.413±0.016 | -0.812±0.289 | 0.656±0.018 | 70.699±1.69 | 0.726±0.029 | 0.695±0.024 | -0.924±0.059 | 4.050±0.177 | 5.312±0.171 | 5.410±0.149 |\n| Syn-Test | DP²O-SR (SD2)      | 21.38±0.43 | 0.53±0.017 | **0.447±0.010** | **0.418±0.014** | **-1.008±0.185** | **0.687±0.015** | **73.769±0.95** | **0.776±0.021** | **0.745±0.015** | **-1.110±0.053** | **4.342±0.130** | **5.499±0.143** | **5.524±0.126** |\n| Syn-Test | C-FLUX             | **21.72±0.51** | 0.52±0.029 | 0.426±0.016 | 0.446±0.019 | **-1.085±0.223** | 0.688±0.020 | 72.349±1.41 | 0.748±0.022 | 0.717±0.022 | -1.158±0.077 | 4.443±0.121 | 5.374±0.162 | 5.325±0.141 |\n| Syn-Test | DP²O-SR (FLUX)     | 21.27±0.39 | 0.52±0.022 | **0.420±0.012** | **0.448±0.017** | -0.947±0.196 | **0.712±0.014** | **74.252±0.83** | **0.787±0.016** | **0.758±0.013** | **-1.210±0.065** | **4.577±0.083** | **5.462±0.136** | **5.443±0.117** |\n| RealSR   | C-SD2              | **23.38±0.79** | 0.62±0.036 | 0.467±0.019 | **0.441±0.019** | **-0.595±0.296** | 0.666±0.020 | 71.060±1.58 | 0.724±0.027 | 0.704±0.025 | -0.949±0.061 | 3.597±0.203 | 5.046±0.146 | 5.067±0.138 |\n| RealSR   | DP²O-SR (SD2)      | 22.73±0.62 | **0.64±0.026** | **0.440±0.012** | 0.436±0.016 | -0.356±0.183 | **0.700±0.014** | **72.710±0.96** | **0.758±0.021** | **0.747±0.015** | **-1.111±0.046** | **3.808±0.155** | **5.169±0.127** | **5.088±0.128** |\n| RealSR   | C-FLUX             | **24.18±0.81** | 0.66±0.037 | 0.410±0.021 | **0.481±0.026** | **-0.698±0.256** | 0.660±0.023 | 70.690±1.76 | 0.692±0.035 | 0.693±0.031 | -1.038±0.063 | 3.630±0.220 | 4.932±0.148 | 4.783±0.159 |\n| RealSR   | DP²O-SR (FLUX)     | 23.90±0.59 | 0.66±0.030 | **0.407±0.015** | 0.477±0.019 | -0.587±0.199 | **0.689±0.015** | **72.680±0.99** | **0.744±0.023** | **0.757±0.013** | **-1.088±0.051** | **3.903±0.143** | **5.021±0.119** | **4.948±0.120** |\n\n---\n\n**[Q3]: Do CLIP-based CPS components favor oversharpened or hallucinated details?**\n\nThank you for the insightful question.\n\nWe have observed that CLIP-based components in CPS (e.g., CLIPIQA+, Q-Align) can indeed favor oversharpened or hallucinated textures when used in isolation. \n\nTo address this, CPS integrates both FR and NR IQA metrics. The FR components serve to anchor the reward signal in ground-truth consistency, effectively mitigating the tendency toward hallucinated or overly sharp artifacts.\n\nWe will include visual examples in the revised version to illustrate how CPS achieves a better balance between perceptual quality and fidelity.\n\n---\n\n**[Q4]: Ablate HSW into intra-pair and inter-group to isolate contributions.**\n\nThank you for the helpful suggestion.\n\nWe have conducted separate ablation studies to isolate the effects of the two components in HSW:\n\n- (i) Intra-pair only (IP)\n- (ii) Inter-group only (IG)\n\nThe table below shows the CPS score under different configurations:\n\n| Method            | CPS Score |\n|-------------------|-----------|\n| Baseline          | 0.588     |\n| + PPC             | 0.630     |\n| + PPC + IP only   | 0.634     |\n| + PPC + IG only   | 0.636     |\n| + PPC + IP + IG   | **0.642** |\n\nWe find that both components individually contribute to performance gains, and their combination yields the best result.\n\nWe will include this ablation in the revised manuscript to clarify the individual contributions of intra-pair and inter-group weighting.\n\n---\n\n**[Q5]: Please provide justification of CPS composition and normalization.**\n\nThank you for the helpful question.\n\nCPS is designed to balance **fidelity** and **perceptual quality** by combining both FR and NR IQA metrics:\n\n- **FR metrics** (e.g., LPIPS, TOPIQ-FR, AFINE-FR) emphasize structural and content fidelity.\n- **NR metrics** (e.g., CLIPIQA+, MUSIQ, Q-Align) encourage perceptual realism and aesthetics.\n\nUsing only FR tends to suppress fine details, while NR-only optimization may lead to hallucinated or oversharpened artifacts. The combination helps avoid these extremes.\n\nTo ensure fair contribution from each metric, we apply **min-max normalization** across each comparison set, aligning their numeric ranges. Additionally, we assign **equal weights to the FR and NR groups**, rather than individual metrics, to avoid imbalance due to differing group sizes.\n\nMetrics such as PSNR and SSIM are excluded from CPS due to their known poor correlation with human perception.\n\nDue to rebuttal format constraints, we are unable to include illustrative examples here, but we will provide detailed visual comparisons in the revised version to further justify the design of CPS."}, {"review_id": "dMdRdo2O8f", "reviewer": "Reviewer_Rx2M", "summary": "To better align with human perceptual preferences and enhance the perceptual quality of super-resolved outputs, the paper extends the DPO algorithm and proposes DP²O-SR. By introducing a combined perceptual score (CPS) and hierarchical significance weighting, the method achieves strong performance across several benchmarks and evaluation metrics.", "questions": "Applicability to one-step methods. One-step diffusion SR methods are increasingly favored in practical scenarios and typically do not rely on noisy inputs. Could the proposed RL strategy be adapted to such settings?", "limitations": "Yes.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n1. The motivation to improve the generation stability of diffusion models is well-founded. The proposed RL solution effectively addresses this issue and demonstrates notable performance gains.\n2. The enhancements over DPO are meaningful. Through both visual and quantitative analysis of candidate quantity and hierarchical weighting, the method mitigates the challenge of small perceptual differences between SR samples that often diminish RL effectiveness.\n3. The method yields impressive quantitative improvements on Syn-Test and RealSR benchmarks, validating its effectiveness.\n4. The paper is clearly written and easy to follow.\n\n### Weaknesses\n1. Efficiency of candidate generation. The paper should elaborate on the computational cost and time required to generate offline candidates. Information about hardware resources and runtime would be valuable. Additionally, DPO's optimization may cause the learned policy to deviate from the reference model, but the training samples are provided by the reference model. Have the authors considered online RL alternatives such as GRPO?\n2. Metric justification. Since the model is explicitly optimized for the self-defined CPS metric, improved CPS is expected. How is CPS validated as a reliable proxy for human perceptual alignment? A user study or correlation analysis with human judgments would strengthen the claim.\n3. Fidelity vs. perceptual quality. Diffusion-based methods are known to enhance perceptual quality at the cost of fidelity. Reporting PSNR and SSIM results is strongly recommended. Including DRealSR results would further support the generalizability of the method.\n4. Visual variation illustration. The claim that different noise inputs can lead to perceptually diverse outputs from the same LR input (lines 35–37) would benefit from more visual examples to clearly illustrate this phenomenon.", "was_revised": true, "final_justification": "The rebuttal has addressed my concerns. Rating maintained.", "rebuttal": "We thank the reviewer Rx2M for his/her valuable feedback. For brevity, we use `W#` for weaknesses and `Q#` for questions (e.g., W1, Q1) throughout our response.\n\n---\n\n**[W1]: Report generation cost; consider online RL (e.g., GRPO) to address DPO drift.**\n\nThank you for the valuable comments.\n\nFor offline candidate generation, we use 8×A100 GPUs in parallel. Generating 20 samples per LR image over 4,000 images takes approximately **7 hours with C-SD2** and **18 hours with C-FLUX**. After sampling, we apply IQA models to label the 80,000 generated images, which takes an additional **3 hours** using the same GPU setup.\n\nWe acknowledge that DPO may introduce distribution mismatch, as the policy can deviate from the reference model used to generate training samples. While online RL methods such as GRPO may alleviate this drift, they often come with trade-offs in terms of **training stability, data efficiency, and implementation complexity**. We opt for an offline approach due to its simplicity and scalability, but view online RL as a promising future direction.\n\n---\n\n**[W2]: Provide user study or correlation analysis with human judgments.**\n\nThank you for the important question.\n\nWe have conducted a user study to validate whether CPS aligns with human perceptual preferences. As described in Appendix Sec. 5, participants were asked to compare model outputs across several methods. The results show that CPS is strongly correlated with human judgments, and our method is consistently preferred over the baselines.\n\nThis supports CPS as a reliable proxy for perceptual quality.\n\n---\n\n**[W3]: Report PSNR/SSIM; include DRealSR to test generalization.**\n\nThank you for the thoughtful suggestion.\n\nWe have included **PSNR and SSIM results** in the table below to examine the fidelity aspect of our method. \n\nTo assess generalization, we also report results on the **DRealSR dataset**. We observe that **DP²O-SR consistently outperforms the base models (C-SD2 and C-FLUX)** across most perceptual and fidelity metrics, demonstrating the robustness of our method.\n\nThe table below summarizes results on three datasets: Syn-Test, RealSR, and DRealSR. We will include this expanded evaluation in the revised manuscript.\n\n| Dataset   | Method             | PSNR ↑ | SSIM ↑ | LPIPS ↓ | TOPIQ-FR ↑ | AFINE-FR ↓ | MANIQA ↑ | MUSIQ ↑ | CLIPIQA ↑ | TOPIQ-NR ↑ | AFINE-NR ↓ | QALIGN ↑ | NIMA ↑ | TOPIQ-IAA ↑ |\n|-----------|--------------------|--------|--------|----------|--------------|----------------|------------|------------|--------------|---------------|----------------|------------|---------|----------------|\n| Syn-Test | C-SD2              | **22.20** | 0.53   | 0.458   | 0.413       | -0.812         | 0.656     | 70.699     | 0.726        | 0.695         | -0.924         | 4.050     | 5.312  | 5.410          |\n| Syn-Test | DP²O-SR (SD2)      | 21.38  | 0.53   | **0.447** | **0.418**   | **-1.008**     | **0.687** | **73.769** | **0.776**    | **0.745**     | **-1.110**     | **4.342** | **5.499** | **5.524**      |\n| Syn-Test | C-FLUX             | **21.72** | 0.52   | 0.426   | 0.446       | **-1.085**     | 0.688     | 72.349     | 0.748        | 0.717         | -1.158         | 4.443     | 5.374  | 5.325          |\n| Syn-Test | DP²O-SR (FLUX)     | 21.27  | 0.52   | **0.420** | **0.448**   | -0.947         | **0.712** | **74.252** | **0.787**    | **0.758**     | **-1.210**     | **4.577** | **5.462** | **5.443**      |\n| RealSR   | C-SD2              | **23.38** | 0.62   | 0.467   | **0.441**   | **-0.595**     | 0.666     | 71.060     | 0.724        | 0.704         | -0.949         | 3.597     | 5.046  | 5.067          |\n| RealSR   | DP²O-SR (SD2)      | 22.73  | **0.64** | **0.440** | 0.436       | -0.356         | **0.700** | **72.710** | **0.758**    | **0.747**     | **-1.111**     | **3.808** | **5.169** | **5.088**      |\n| RealSR   | C-FLUX             | **24.18** | 0.66   | 0.410   | **0.481**   | **-0.698**     | 0.660     | 70.690     | 0.692        | 0.693         | -1.038         | 3.630     | 4.932  | 4.783          |\n| RealSR   | DP²O-SR (FLUX)     | 23.90  | 0.66   | **0.407** | 0.477       | -0.587         | **0.689** | **72.680** | **0.744**    | **0.747**     | **-1.088**     | **3.903** | **5.021** | **4.948**      |\n| DrealSR  | C-SD2              | **26.000** | 0.665  | 0.488   | **0.431**   | **-0.368**     | 0.626     | 66.631     | 0.697        | 0.672         | -0.882         | 3.440     | 4.922  | 4.912          |\n| DrealSR  | DP²O-SR (SD2)      | 25.306 | **0.675** | **0.464** | 0.419       | 0.150          | **0.660** | **69.187** | **0.737**    | **0.721**     | **-1.042**     | **3.708** | **5.000** | **5.005**      |\n| DrealSR  | C-FLUX             | **26.648** | **0.682** | 0.453   | 0.432       | -0.008         | 0.628     | 67.210     | 0.680        | 0.668         | -0.932         | 3.522     | 4.863  | 4.688          |\n| DrealSR  | DP²O-SR (FLUX)     | 26.265 | 0.674  | **0.446** | **0.444**   | **-0.023**     | **0.663** | **69.809** | **0.735**    | **0.724**     | **-0.993**     | **3.814** | **4.969** | **4.855**      |\n\n---\n\n**[W4]: Provide more visuals to support perceptual diversity from noise variation.**\n\nThank you for the suggestion. We will include more visual results in the next revision to better support the motivation regarding perceptual diversity from noise variation.\n\n---\n\n**[Q1]: Can the RL strategy extend to one-step SR methods without noise inputs?**\n\nThank you for the insightful question.\n\nOur current method relies on stochastic generation (via noise inputs) to produce diverse candidates for offline reward modeling. Therefore, it is not directly applicable to deterministic one-step SR methods that lack inherent randomness.\n\nHowever, if controlled randomness is introduced—e.g., by injecting noise during training—our RL framework could potentially be adapted to such settings. Exploring such extensions remains an interesting direction for future work."}, {"review_id": "1SpaePjLOM", "reviewer": "Reviewer_4d1m", "summary": "This paper introduces a novel adaptation of Direct Preference Optimization (DPO) for Real Super-resolution, DP2O-SR. The core methodology involves generating a comprehensive quality score (CPS) for super-resolved images by normalizing and averaging multiple reference and non-reference metrics. For a given low-resolution image, N different super-resolution results are produced and ranked according to their CPS scores. Instead of simply comparing the best and worst outcomes, DiffusionDPO leverages all possible pairs from these N results, labeling them positive or negative based on their relative quality scores. The training process incorporates two key weighting mechanisms: an \"importance\" weight assigned to each LR image based on the variance of the quality across its N generated results, and another weight that increases with the quality score difference between positive and negative samples. The paper reports quantitative improvements, and a limited number of visual results.", "questions": "The main weakness with the proposed work is the evaluation of the methodology. The metrics used in the ranking are also the ones reported. There's no human evaluation which would be the ultimate goal. This makes it harder to conclude on the claims. I would like the authors to share some comments on this point.\n\nI also have a few questions:\n\n  1. After applying the proposed method, how sensitive are the results with the noise seed? It would be good to show some standard deviations for the metrics (before and after the fine-tuning)\n  2. For completeness, even if not meaningful from the point of view of perceptual quality, it would be good to show PSNR values. Is the average distortion changing with this method?\n  3. Why have you limited the selection to samples generated under the same configuration but just with a different random seed. Why not also include different CFG guidelines? Is there a conceptual reason for this? (some of the metrics being sensitive or bias to larger guidance values so the ranking will be disrupted?)", "limitations": "The authors discuss the limitation (L298-L300).", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strong points of the paper:\n\n  - Tackles a key challenge: The paper focuses on a very important practical problem: how to effectively train and fine-tune diffusion models for real-world super-resolution.\n  - Introduces a clever quality metric: The central idea of creating a single, combined quality score to rank and measure the difference between results is quite interesting.\n  - Shows measurable improvements: The evaluation demonstrates better quantitative results on several standard metrics when tested against public benchmarks.\n\nAreas where the paper could be stronger:\n\n  - Metrics overlap: The same metrics used to create the combined quality score are also used to evaluate the method. This raises the question of whether the method truly improves how images look or just performs better on those specific metrics.\n  - Lacks human evaluation: The paper doesn't include any tests with human observers, which is the ultimate goal for judging image quality. This makes it tough to tell the real impact of the method. Also, it's unclear how well the new combined quality score matches what people prefer. Few visual results.\n  - Limited evidence for technical novelty: While the combined score and the use of weights for image pairs are interesting technical innovations, the results aren't strong enough to fully prove their significance.", "was_revised": true, "final_justification": "After careful consideration, my main concern remains with the decision to relegate a critical discussion of the paper's core weakness—specifically, the potential for metric overlap to bias evaluation, as noted in W1 and W2—to the appendix. I believe the paper would be significantly stronger and more convincing if this topic were addressed directly within the main text. However, given that the evidence and analysis are already present in the supplementary material, I will be updating my score to borderline accept.", "rebuttal": "We thank the reviewer 4d1m for his/her valuable feedback. For brevity, we use `W#` for weaknesses and `Q#` for questions (e.g., W1, Q1) throughout our response.\n\n---\n\n**[W1 and W2]: Does metric overlap bias evaluation toward the optimized score rather than real perceptual gain? Lacks human evaluation.**\n\nThank you for the valuable comments.\n\nWe acknowledge the concern that using the same metrics for both optimization (via CPS) and evaluation may introduce bias. To address this, we conducted a user study based on human perceptual judgments (Appendix Sec. 5). The results show that **CPS aligns well with human preferences**, and our method is **consistently preferred over baselines** in terms of perceived visual quality.\n\nIn addition, we have already included **extensive visual comparisons** in the supplementary material (Appendix Sec. 4), which further illustrate the perceptual improvements of our method.\n\n---\n\n**[W3]: Technical novelty is interesting but lacks strong evidence.**\n\nThank you for the insightful feedback.\n\nWhile CPS and our weighting strategy are conceptually simple, they are carefully designed to address two key challenges in RealSR: (1) balancing perceptual quality and fidelity, and (2) modeling the strength of human preferences more accurately.\n\nBeyond these, we introduce a systematic framework for **Preference Pair Curation (PPC)**, where we study how increasing noise samples and selecting top-$N$/bottom-$N$ pairs affects the quality of offline supervision—a direction rarely explored in prior work.\n\nTo evaluate the practical impact of each component, we conducted ablation studies with Table 2 (c) (also below). The results show that **each component—PPC, HSW—brings consistent CPS improvements**, supporting the effectiveness of our design.\n\nWe will revise the paper to better clarify the motivation and significance of these contributions within the context of preference-based learning for low-level vision.\n\n|        | baseline | w / PPC | w / PPC+HSW |\n|--------|----------|---------|-------------|\n| CPS    | 0.591    | 0.632   | 0.641       |\n\n---\n\n**[Q1 and Q2]: Please report metric std deviations before and after fine-tuning. Miss PSNR results.**\n\nThank you for the helpful suggestions.\n\n**1. Sensitivity to noise seed (standard deviation):**  \nWe have added a table reporting the **mean and standard deviation** of 13 FR and NR IQA metrics across multiple noise seeds, both **before and after fine-tuning** on the Syn-Test and RealSR datasets.\n\nWe observe that the **standard deviations consistently decrease** after applying our DP²O-SR method. This indicates that our method not only improves average perceptual quality, but also yields **more stable outputs** under varying noise conditions—a desirable trait for practical deployment.\n\n**2. PSNR reporting (distortion):**  \nAlthough PSNR is a widely used fidelity metric, it primarily reflects pixel-level similarity and often favors overly smooth outputs. In contrast, we place greater emphasis on perceptual fidelity metrics such as **LPIPS**, which better capture human-perceived similarity and perceptual realism.\n\nIn response to your suggestion, we have now included **PSNR and SSIM results in the table below** for completeness. We believe this trade-off is reasonable: while our method may yield slightly lower PSNR, it significantly improves perceptual quality, as reflected in LPIPS, and other no-reference metrics.\n\nThese results, along with the standard deviation analysis, will be included in the revised manuscript.\n\n| Dataset   | Method             | PSNR ↑       | SSIM ↑      | LPIPS ↓     | TOPIQ-FR ↑   | AFINE-FR ↓    | MANIQA ↑    | MUSIQ ↑     | CLIPIQA ↑   | TOPIQ-NR ↑   | AFINE-NR ↓   | QALIGN ↑    | NIMA ↑     | TOPIQ-IAA ↑ |\n|-----------|--------------------|--------------|-------------|-------------|---------------|----------------|--------------|--------------|--------------|----------------|----------------|--------------|-------------|----------------|\n| Syn-Test | C-SD2              | **22.20±0.47** | 0.53±0.023 | 0.458±0.014 | 0.413±0.016  | -0.812±0.289  | 0.656±0.018  | 70.699±1.69  | 0.726±0.029  | 0.695±0.024   | -0.924±0.059  | 4.050±0.177  | 5.312±0.171 | 5.410±0.149  |\n| Syn-Test | DP²O-SR (SD2)      | 21.38±0.43   | 0.53±0.017 | **0.447±0.010** | **0.418±0.014** | **-1.008±0.185** | **0.687±0.015** | **73.769±0.95** | **0.776±0.021** | **0.745±0.015** | **-1.110±0.053** | **4.342±0.130** | **5.499±0.143** | **5.524±0.126** |\n| Syn-Test | C-FLUX             | **21.72±0.51** | 0.52±0.029 | 0.426±0.016 | 0.446±0.019  | **-1.085±0.223** | 0.688±0.020  | 72.349±1.41  | 0.748±0.022  | 0.717±0.022   | -1.158±0.077  | 4.443±0.121  | 5.374±0.162 | 5.325±0.141  |\n| Syn-Test | DP²O-SR (FLUX)     | 21.27±0.39   | 0.52±0.022 | **0.420±0.012** | **0.448±0.017** | -0.947±0.196  | **0.712±0.014** | **74.252±0.83** | **0.787±0.016** | **0.758±0.013** | **-1.210±0.065** | **4.577±0.083** | **5.462±0.136** | **5.443±0.117** |\n| RealSR   | C-SD2              | **23.38±0.79** | 0.62±0.036 | 0.467±0.019 | **0.441±0.019** | **-0.595±0.296** | 0.666±0.020  | 71.060±1.58  | 0.724±0.027  | 0.704±0.025   | -0.949±0.061  | 3.597±0.203  | 5.046±0.146 | 5.067±0.138  |\n| RealSR   | DP²O-SR (SD2)      | 22.73±0.62   | **0.64±0.026** | **0.440±0.012** | 0.436±0.016  | -0.356±0.183  | **0.700±0.014** | **72.710±0.96** | **0.758±0.021** | **0.747±0.015** | **-1.111±0.046** | **3.808±0.155** | **5.169±0.127** | **5.088±0.128** |\n| RealSR   | C-FLUX             | **24.18±0.81** | 0.66±0.037 | 0.410±0.021 | **0.481±0.026** | **-0.698±0.256** | 0.660±0.023  | 70.690±1.76  | 0.692±0.035  | 0.693±0.031   | -1.038±0.063  | 3.630±0.220  | 4.932±0.148 | 4.783±0.159  |\n| RealSR   | DP²O-SR (FLUX)     | 23.90±0.59   | 0.66±0.030 | **0.407±0.015** | 0.477±0.019  | -0.587±0.199  | **0.689±0.015** | **72.680±0.99** | **0.744±0.023** | **0.747±0.015** | **-1.088±0.051** | **3.903±0.143** | **5.021±0.119** | **4.948±0.120** |\n\n---\n\n**[Q3]: Why not vary CFG?**\n\nThank you for the insightful question. We agree that varying CFG is a natural way to generate diverse outputs.\n\nHowever, we found that changing CFG introduces a **predictable trade-off**: higher CFG values tend to improve perceptual quality (as measured by no-reference metrics), but degrade fidelity (as measured by full-reference metrics), and vice versa. This makes it difficult to identify samples that are simultaneously strong in both aspects.\n\nIn contrast, sampling multiple outputs under the **same CFG but different noise seeds** allows us to capture a richer range of natural variations, including rare but valuable samples that are high in both fidelity and perceptual quality. These “lucky” samples help define the upper bound of model capability.\n\nSince our goal is to guide the model toward outputs that balance both perceptual quality and fidelity, we fix CFG and vary the noise seed to build a more expressive and informative preference space for reward learning.\n\n**Empirically**, this design is supported by the results in Table 1 (main paper), where our preference-tuned models consistently outperform baselines on both full-reference and no-reference metrics.\n\nWe will clarify this rationale in the revised manuscript."}]

Author rebuttal:

Meta-review: This submission presents a method called DP²O-SR for real-world image super-resolution that leverages Direct Preference Optimization to align diffusion model outputs with human perceptual preferences. The key contributions include the Combined Perceptual Score (CPS) for quantifying perceptual quality and Hierarchical Significance Weighting (HSW) to enhance training efficiency. Reviewers acknowledged the submission's motivation, methodological innovation, and empirical improvements over baselines, particularly in perceptual quality and stability. While initial concerns were raised about metric circularity (CPS serving as both training signal and evaluation criterion), the authors addressed these through additional experiments. The rebuttal further clarified computational costs, failure modes, and the balance between fidelity and perceptual quality. Although some limitations remain, the reviewers unanimously upgraded their scores, recognizing the submission's technical rigor, practical impact. The submission is thus suggested for acceptance.
Decision: Accept (poster)