LGDM: Latent Guidance in Diffusion Models for Perceptual Evaluations
Authors: Shreshth Saini
Ru-Ling Liao
Yan Ye
Alan Bovik
Venue: ICML 2025 poster (2025)
Keywords: Perceptual Quality
Latent Diffusion Models
Training-free
Gen-AI
IQA
Primary area: deep_learning->generative_models_and_autoencoders
TL;DR: We propose Perceptual Manifold Guidance in Latent Diffusion Models to extract multi-scale, multi-time denoising U-Net features for zero-shot No-Reference Image Quality Assessment, marking the first-ever application of pretrained LDMs to NR-IQA.

Abstract: Despite recent advancements in latent diffusion models that generate high-dimensional image data and perform various downstream tasks, there has been little exploration into perceptual consistency within these models on the task of No-Reference Image Quality Assessment (NR-IQA). In this paper, we hypothesize that latent diffusion models implicitly exhibit perceptually consistent local regions within the data manifold. We leverage this insight to guide on-manifold sampling using perceptual features and input measurements. Specifically, we propose Perceptual Manifold Guidance (PMG), an algorithm that utilizes pretrained latent diffusion models and perceptual quality features to obtain perceptually consistent multi-scale and multi-timestep feature maps from the denoising U-Net. We empirically demonstrate that these hyperfeatures exhibit high correlation with human perception in IQA tasks. Our method can be applied to any existing pretrained latent diffusion model and is straightforward to integrate. To the best of our knowledge, this paper is the first work on guiding diffusion model with perceptual features for NR-IQA. Extensive experiments on IQA datasets show that our method, LGDM, achieves state-of-the-art performance, underscoring the superior generalization capabilities of diffusion models for NR-IQA tasks.

LGDM: Latent Guidance in Diffusion Models for Perceptual Evaluations
Shreshth Saini' Ru-Ling Liao?
Yan Ye? Alan C. Bovik
Abstract
Despite recent advancements in latent diffusion models that generate high-dimensional image data and perform various downstream tasks, there has been little exploration into perceptual consistency within these models on the task of No-Reference Image Quality Assessment (NR-IQA). In this pa- per, we hypothesize that latent diffusion models implicitly exhibit perceptually consistent local re- gions within the data manifold. We leverage this insight to guide on-manifold sampling using per- ceptual features and input measurements. Specifi- cally, we propose Perceptual Manifold Guidance (PMG), algorithm that utilizes pretrained latent diffusion models and perceptual quality features to obtain perceptually consistent multi-scale and multi-timmestepppe feature maps from the denoising U- Net. We empirically demonstrate that these hyper- features exhibit high correlation with human per- ception in IQA tasks. Our method can be applied to any existing pretrained latent diffusion model and is straightforward to integrate. To the best of our knowledge, this paper is the first work on guid- ing diffusion model with perceptual features for NR-IQA. Extensive experiments on IQA datasets show that our method, LGDM, achieves state-of- the-art performance, underscoring the superior generalization capabilities of diffusion models for NR-IQA tasks.
2
x D(Z.)
Aipe
20
Content Hine
(A)
(
Figure I: An overview of our proposed approach: (a) shows the transition of latent samples across latent manifolds, high- lighting the steps of DDIM and our PMG. (b) depicts the content bias (green) on the manifold MMo- ~ D(Z)). show- ing that the guidance term in red (PMG) pushes a data sample (1 ~ D())j)) towards the perceptually consistent region (yellow) on the manifold. Here D is decoder of VAE.
2019; Song et al., 2021b: Zhang et al., 2023a). Latent Diffu- sion Models (LDMss, which embed data into a compressed latent space, enhance computational efficiency (Rombach et al., 2022). Diffusion models provide strong data priors that effectively capture the intricacies of high-dimensional data distributions, making them powerful for generative tasks. Conditional generation using posterior sampling has become crucial for solving various real-world low-level vi- sion problems (Kawar et al., 2022; Chung et al., 2024; Rout et al., 2024; Song et al., 2024). Additionally, several meth- ods leverage the rich internal representations of diffusion models by extracting either send-selected single or subsets of features from denoising U-Net for downstream tasks (Tumanyan et al., 2023; Ye et al., 2023; Xu et al., 2023; Baranchuk et al., 2022). Despite these advancements in addressing tasks like inverse problems, segmentation, and semantic keypoint correspondence, there has been little ex- ploration into perceptual consistency of diffusion models for Ro-RRefrrnnce Image Quality Assessment (NR-IQA). NR-IQA aims to evaluate image quality in line with human perception without high-quality reference image (Wang & Bovik, 2006). It plays a crucial role in optimizing parame- ters for image processing tasks, such as resizing, compres- sion (Feng et al., 2023; Liu et al., 2023), and enhancement (Hou et al., 2024; Fei et al., 2023; Zhang et al., 2024). Fig. illustrate a diverse spectrum of distortions frequently en-
Introduction
Score-based diffusion models have advanced significantly in recent years and have achieved remarkable success at syn- thesizing high-quality images across diverse scenes, views, and lighting conditions (Ho et al., 2020; Song & Ermon,
Work done during internship at Alibaba Group. 'Laboratory for Image and Video Engineering (LIVE), The University of Texas Austin, Texas, USA. AAliaab Group, Sunnyvale, USA. Corre- spondence to: Shreshth Saini <saini 2 uduxxax.eddu>..
Proceedings of the nnnd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Latent Guidance in Diffusion Models for Perceptual Evaluations
MARO
(a)
(b)
(c)
(d)
Figure 2: Exemplar Authentic and Synthetic distorted im- ages. Here (a) shows Camera Motion Blur, (b) shows com- pression artifacts, (c) shows noise, and (d) shows a night scene with grains. Best viewed when zoomed in.
FLAC
Adhwet: Ditarie Calaeb
Figure 3: Median SRCC scores of NR-IQA methods across authentic distortion IQA datasets, demonstrating the supe- nior performance of our method.
countered in real-world datasets. Early NR-IQA methods used hand-crafted natural scene statistics features (Zhang et al., 2015; Mittal et al., 2012; Saad et al., 2012), and have evolved into booming-based quality metrics (Madhusudana et al., 2022; Tu et al., 2021; Ke et al., 2021; Saini et al., methods show promise, they
diffusion hyperfeatures (Luo al., 2024), from denoising U-Net for NR-IQA. As shown by (Ghildyal et al., 2024), intermediate features of foundation models outperform state- of-the-art learned metrics based on final feature layers, Our method, Latent Guidance in Diffusion Model (LGDM), is a framework for extracting perceptually consistent diffu- sion hyperfeatures from unconditionally pretrained latent diffusion models:
2024). While learning-ban often lack generalizability. With the advent of generative models, some authors have explored the use of pixel dif- fusion models for NR-IQA tasks (Fu al., 2024; Li al., 2024b; Babnik et al., 2024; Wang et al., 2025). While these approaches show impressive progress, they are often ad hoc, focusing on tasks like quality feature denoising and image restoration by converting NR-IQA problems into Full-Reference IQA (FR-IQA) ones. Additionally, train- ing on specific IQA datasets limits their generalizability. By contrast, our goal is to utilize pretrained latent diffu- sion models without fine-tuning, leveraging perceptual guid- ance to extract intermediate multi-scale and multi-time fea- tures, termed diffusion hyperfeatures (Luo et al., 2024), for NR-IQA. Our proposed method robustly handles breadth of degradations, capturing both local texture artifacts and larger-scale structural inconsistencies as shown in Fig. 2.
We introduce a novel approach for leveraging uncondi- tional latent diffusion models to tackle the challenging task of NR-IQA without any fine-tuning or additional training of feature extraction model.
We design a manifold guidance scheme (PMG) that en- sures the sampling process remains on the manifold and close to perceptually consistent region. We provide exten- sive theoretical proof that our perceptual guidance keeps gradient updates on the tangent spaces of the data mani- fold, maintaining proximity to the local perceptually con- sistent manifold. We also utilize intermediate multi-scale and multi-time features from the denoising U-Net, result- ing in high correlation with human perceptual judgments.
At the core of our method is the manifold hypothesis: real data does not occupy the entire pixel space but instead lies on smaller underlying manifold. Previous works (Chung et al., 2022; He et al., 2024; Sun et al., 2023) have used the manifold concept for guided sample generation and solving inverse problems. In IQA, deep models aim to learn distor- tion manifolds that correlate highly with human perceptual quality (Agnolucci et al., 2024; Su et al., 2023; Guan al., 2018; Gao et al., 2024). These manifolds represent regions within the data manifold that contain perceptually consistent samples, with content bias further narrowing these regions (Fig. 1).
Extensive experiments on both authentic and synthetic IQA benchmarks demonstrate that our method achieves state-of-the-at performance. To the best of our knowl- edge, this is the first approach to introduce perceptual guidance in latent diffusion models for NR-IQA.
We evaluate LGDM against both supervised and unsuper- vised of methods on ten IQA datasets, consis- tently achieving superior results (see Fig. 3).
Towards further advancing progress in this direction, we propose Perceptual Manifold Guidance (PMG) to ensure perceptually consistent m--manioold sampling, conditioned on weak perceptual quality features. Fig. provides an overview and conceptual visualization of our approach. Un- like previous state-of- of-te-of-the-art CNN or transformer- based IQA models that only utilize the final feature layer, we extract intermediate multi scate and multi-time features, termed
2 Background
2.1 NR-IQA
With reference to NR-IQA, to better capture the complex relationship between image content and perceived qual-
2

---

Latent Guidance in Diffusion Models for Perceptual Evaluations
ity, manifold learning techniques have been explored (Ag- nolucci et al., 2024; Su et al., 2023; Guan et al., 2018; Gao et al., 2024). These approaches aim to uncover intrinsic low-dimensional structures within high-dimensional data, thereby aligning more closely with human visual perception. They generally rely on the following hypothesis:
using unconditional diffusion models (Song et al., 202 1b; Chung al., 2022; Yu et al., 2023), common approach is to replace the score function in DDPM equation A-7 with a conditional score function Vz, log p(z; ly), where is the conditioning variable. Using Bayes' rule, the conditional score function can be decomposed into the unconditional score function and a likelihood term: Va, log p(rzly) = Vx, log p(z,) + Vx, log p(y)2;), Incorporating this into the reverse SDE yields:
Assumption 1: (Strong Manifold Hypothesis). For given data distribution X E ID, the actual data points are con- centrated on a k-dimensional locally linear subspace mani- fold M C RD, such that l < D.
dx
[
Br -/
Be Bi( log
2
Latent diffusion models have demonstrated strong repre- sentation learning capabilities when trained on large-scald scale datasets containing wide range of authentic and synthetic distortions (Rombach et al., 2022; Zhang et al., 2023a), but their application to NR-IQA problems remains underex- plored (De et al., 2024; Babnik et al., 2024; Wang et al., 2025).
+ Vv, log p(y)(=)))) dt + v Brdw
(3)
The above SDE can be treated as a two-step process, the first getting an unconditional denoised sample I;-1, fol- lowed by the gradient update with respect to x2. Since, the likelihood term Zz, log (ggp())))))) is generally intractable, the second term approximate a gradient update to minimizing the guidance loss around the denoised sample It-1-
We demonstrate that latent diffusion models implicitly learn perceptually consistent subspace manifolds due to their extensive training data and ability to capture data priors through score matching. By leveraging the learned score function so with perceptual guidance from a features of the perceptual model t'p. and extracting diffusion hyper- features H = - h, = - so (Ix, l], where T represents the total sampling steps and is a subset of intermediate layers, align features from the denoising network su at different time steps with human perceptual judgments.
x/-11 = vār- + V ā, (2).
+ ofE
(4)
xz-1 = - VVx,G(ropp v)
where ( is a tunable step size. Here, Tweedie's 's estimate Tooe is used since the guidance term is defined on the clean data Zo. i.e., Gz( y) = Eptaro m.) ( y)] ~ G(zoje y). The guidance term is optimized over a neighborhood around 2t E RR.
2.2 Diffusion Models
Many methods use equation 5 for conditional generation and various vision tasks (Chung et al., 2022: Kawar et al., 2022; Yu et al., 2023; Song et al., 2023). For example, (Chung et al., 2022) define an l2 loss term as |y A(zoj)) where A represents known differentiable forward degradation model, effectively guiding the generated sample to match the condition y.
We begin by reviewing the DDIM followed by conditional diffusion models.
Denoising Diffusion Implicit Models (DDIM). To address the slow generation of DDPM, (Song et al., 2021a) proposed Denoising Diffusion Implicit Models (DDIMs), which de- fine a mon-Markovian diffusion process for faster sampling. The DDIM sampling update is:
3 Latent Guidance in Diffusion Model
Tt-1 = V + V1 - -f o0 totE
As discussed in Section 1, diffusion models implicitly learn perceptually consistent region. The diffusion process in LDMs naturally generates on- on-manifold perceptually consis- tent samples without requiring additional models to estimate tangent spaces of the data manifold (Srinivas et al., 2023; Bordt et al., 2023; He et al., 2024), as we will demonstrate in this section. We exploit the same, and aim to generate per- ceptually consistent sample (input measurement) by guiding the sampling process on manifold. In the process, U-Net features align well with perceptual quality of measurement for relatively less noisy steps (discussed in 4).
t T 0,
(L)
where 0+ 1 8, ā. = Ti az or corresponds
/(1- (1- (11)--)))(11)))) to DDPM sampling, and when or = O sampling becomes deterministic, where e ~ NOOO I). The term dop is direct estimation of the clean data xi from noisy data Tf, calculated using Tweedie's formula (Efron, 2011):
- ās/āu-1
Pojt= = Vāu (ot + vI σ4, so(2,,)))
(2)
In LDMs, the diffusion process operates within the latent space, training score function so (z,, t). Let x E D
Conditional Diffusion Models. For conditional generation
3

---

Latent Guidance in Diffusion Models for Perceptual Evaluations
as B(M;r) E d(x, < <r}. Consider the distribution of noisy data given by p(æ:) f (o)(o))) p(ze|zo)= := W Xo, (1-a,)1) represents the Gaussian perturbation of the data at time 6, and 0r II._1 Ots is the cumulative product of the noise schedule ag. Under the Assumption I, the distribution pr (2x4 is concentrated on a (D - 1)-dim manifold M, := y R D d(y, VT,M) = r1 == v(1 -ax) -k).
SA
PMG
P3y Up
Most posterior sampling methods (Chung et al., 2023) opti- mize the guidance term C( y) over dú E IRD whereas the score function se is trained only with samples on M,, as indicated by Proposition 1. This discrepancy implies that the solution 27 (leading to zōje via Tweedie's 's formula 2) may not reside on Mc, resulting in a suboptimal solution (Yu et al., 2023). To overcome this limitation, we propose a solution over Mr. From Assumption 1, the manifold M, coincides with its tangent space Mz, i.e., Tz. = R with k < D (Park et al., 2023). Practically, we optimize the guidance term G(Io es y) over x, E T Mr This new compact solution space ensures consistent on-manifold sam- pling throughout the process.
Hlyerteatuurs
Figure 4: Illustrates aggregation of multi-scale and multi- timestep feature maps (Hyperfeatures) from denoising U- Net (se) for NR-IQA. In the image, 2z is intermediate noisy latent, D and Vip are decoder and perceptual quality features, respectively. PMG is our proposed algorithm for estimating the noisy sample 2
The latent space of a well-trained autoencoder implicitly captures the lower-dimensional structure of the data mani- fold, which can be leveraged for tangent space projection (Srinivas et al., 2023; Bordt et al., 2023). The latent pro- cessing of LDMs aids this as the samples already lie in the lower-dimensional space R Formally (proof follows (He et al., 2024), see Appendix B):
represent the original high-dimensional data, and let E :
RD + R* be an encoder and D: IR* - RD be a decoder, where k < D. The embeddings in the latent space are given
by z = (x) E R²
To guide the sampling process towards the perceptually consistent region on the manifold and ensure perceptually consistent hyperfeature extraction from the denoising score function, we propose the PMG framework. An overview of our proposed sampling process is depicted in Fig. 1, which illustrates the step-by-step guidance for extracting perceptually aligned features in the latent space. During the sampling process, our PMG guidance term push estimate on-manifold to reduce the error term.
Proposition 2 (On-manifold sample with LDM) Given perfect autoencoder, i.c. V zaje G( v) E Ta, Z then D(V 2011 G(F E M.
D(E(x)), and gradient
For LDMs, the minimization of the guidance term occurs within the tangent space of the clean data manifold. This guarantees that the generated sample remains close to the real data, without deviations. Although authors of (Rout et al., 2024) do not explicitly discuss on-manifold sampling in LDMs, their results empirically suggest the inherent man- ifold consistency of LDMs.
3.1 Perceptual Manifold Guidance
We propose using perceptual features from an input measure- ment y derived via perceptual quality model thp in the con- ditional score function, leading to <., log p(ze| (y))), y). The choice of v), is detailed in Section 4 and Appendix D.
Having defined consistent on-manifold sampling, we fi- nally present our Perceptual Manifold Guidance (PMG) for perceptually consistent on-manifold sampling. Using Bayes' theorem on our new conditional score function V., log ((ee(()(pp)))) y) (see Appendix B for details):
Before redefining the sampling steps, let's first consider the noisy sample manifolds.
Given Assumption 1, (Chung et al., 2023; 2022) show that noisy data 1, is probabilistically concentrated on (D - 1)- dimensional manifold Mr, which encapsulates the clean data manifold M. Formally (see Appendix B for detailed proof):
Vx, logp(zulv,
Vz, log p(z))
+V s. log (v)l + V., log p(yl())
(6)
From (Rout et al., 2024), LDM's intractable terms can be approximated as:
Proposition 1 (Noisy Data Manifold) Let the distance function be defined as M) := inf I|æ - v/l2- and define the neighborhood around the manifold M
V2, log logp(v;(v) = V., log = D(zg))) (7)

---

Latent Guidance in Diffusion Models for Perceptual Evaluations
2x, logr(ulz)) V2 logr(y/zoj= (=
(8)
Algorithm L.DM: Latent Guidance in Diffusion Models
Require: Input image x, encoder E()) decoder D(-), score function 88 (). perceptual metric vpp()) regression model Ie. time steps T. guidance weights Çi. C2 Output: Predicted quality score qp
Based on Assumption 1, Propositions 1 and 2, Equations 6-8, and Lemma in Appendix B, we derive the follow- ing theorem for Perceptual Manifold Guidance (proof in Appendix B):
I: 20 + E(x)
Theorem 1 (Perceptual Manifold Guidance) Given As-
2: H h 0
sumption 1, given perfect encoder E, decoder D. and an efficient score function sp(ze, t), let the gradient
3: for 1 = -T, 1,...,.1 do
4:
<~ 10
(v)) re- side on the tangent space Tzgje 2 of the latent manifold 2. Throughout the diffusson process, all update terms 2, remain on noisy latent manifolds 2,, with 201; lying in a perceptu- ally consistent manifold locality.
V zoj1 G1 (D( (2(2)))) v) and zon (e(())(((2)))))))
5:
4o h =0((),t)
6:
H - HU he
7:
oip: t 1 (zt vI ar 62)
val
2 t 201t S1 V za11 G, (D (D (20je);
9: Dit t 401e - $2 Vo G2 (v. (201)))) vp(x))
Discretized steps based on Theorem can be written as:
10: z|-1 t vār- 2011 v 1 ā= - -0 + orE
11: end for
Z01e f 20/e= - S1 V.Van, z01s
(i(((zo))) v)
(9)
12: Up 4 (H)
13: return 9p
of + 2/1= <2 saje (P(= (y))
(10)
ai-1- f v a 1 201: V a (z), () t)+ (11)
intermediate layers of the denoising U-Net across multiple diffusion timesteps. This enables us to capture both coarse and fine-grained image features crucial for assessing per- ceptual quality (Ghildyal et al., 2024). Recent studies (Xu et al., 2023; Wu et al., 2023; Luo et al., 2024) have shown that intermediate representations within diffusion models exhibit reliable semantic correspondences, although they have mostly been used for tasks such as data augmentation, generation, and segmentation. We hypothesize that these intermediate features also correlate strongly with human perceptual judgments of image quality, motivated by the diffusion models' ability to generate perceptually appeal- ing images and their robust representational capabilities for various downstream tasks (Zhao et al., 2023).
Equation 9, 10, 11 are posterior sampling, perceptual con- sistency step, and DDIM update step respectively. We use G; as l2 functions. The perceptual consistency step in PMG (Equation 10), guides the sample to be close to the percep- tual quality of the input measurement. Specifically, during sampling, the perceptual guidance term adjusts the earlier estimate of the clean latent sample Z/12 toward perceptu- ally consistent locality on the tangent space of the clean latent manifold, T Tujo Z. From Theorem 1, all update terms 2,, including 20je, are on the manifold Z. This ensures the sampling process remains close to a perceptually consistent region on the manifold, with D(="j) closely aligned with the perceptual quality of the input measurement (see Fig. 1). We use internal representations from the denoising U-Net sy measure this perceptual consistency, detailed in Sec- tion 3.2. The effectiveness of this approach is demonstrated empirically in Section 4, where the absence of the percep- tual guidance term (1b)) in LGDM results in suboptimal performance.
To extract these diffusion hyperfeatures, we gather inter- mediate feature maps from all upsampling layers of the denoising U-Net across multiple diffusion timesteps dur- ing the sampling process, see Fig. 4. These feature maps inherently contain shared representations that capture dif- ferent image characteristics, such semantic content, various scales and levels of abstraction. Since these features are distributed over both the network layers and diffusion timesteps, we aggregate 89 layers and timesteps as diffu- sion hyperfcatures for NR-IQA. Specifically, the set of all extracted features denoted as:
3.2 Diffusion Hyperfeatures & NR-IQA
We propose to use diffusion syperfeatures-mmltiiccoo and multi-timestep feature maps extracted from the denoising U-Net (sg) of pretrained latent diffusion model.
Previous NR-IQA methods typically rely on features ex- tracted from fine-tuned models (Ke et al., 2021; Madhusu- dana et al., 2022; Liu et al., 2022; Saini et al., 2024). How- ever, these methods often use features from the final layer or a single scale, limiting their ability to capture the com- plete spectrum of image characteristics. In contrast, we harness the rich hierarchical representations available in the
T H = U
H
(12)
where (x,,) represents the feature map from layer at timestep t, L is the set of layers from which we extract features, and 7 is the total number of timesteps considered.
5

---

Latent Guidance in Diffusion Models for Perceptual Evaluations
Table 1: Comparison of our proposed LGDM with SOTA NR-IQA methods on PLCC and SRCC Scores for authentic IQA
datasets. The best results are red, and the second- best results are blue.
METHODS
LIVEC
KoNIQ
FLIVE
SPAQ
PLCC1
SRCC1
PLCC1
SRCC1
PLCC1
SRCC1
PLCC1
SRCC1
ILNIQQ (ZHANG ET AL., 2015)
0,508
0.508
0.537
0,523
0.712
0.713
BRISQUE (MITTAL ET AL., 2012)
0.629
0.629
0.685
0.681
0.341
0.303
0.817
0.809
WADIQAM (BOSSE ET AL., 2018)
0.671
0.682
0.807
0.804
0.467
0.455
DBCNN (ZHANG ET AL., 2020)
0.869
0.851
0.884
0.875
0.551
0.545
0.915
0.911
TIQA (STEPIEN & OSZUST, 2023)
0.861
0.845
0.903
0.892
0.581
0.541
METAIQA (ZnU ET AL.,, 2020)
0.802
0.835
0.856
0.887
0.507
0.540
P2P-BM (YING Al.,, 2020)
0.842
0.844
0.885
0.872
0.598
0.526
HYPERIQA (SU BT AL., 2020)
0.882
0.859
0.917
0.906
0.602
0.544
0.915
0.911
TRES (GOLESTANEH
Al.., 2022)
0.877
0.846
0.928
0.915
0.625
0.554
MUSIQ (KE ET AL., 2021)
0.746
0.702
0.928
0.916
0.661
0.566
0.921
0.918
RE-IQA (SAHA ET AL., 2023)
0.854
0.840
0.923
0.914
0.733
0.645
0.925
0.918
LODA (XU ET AL., 2024)
0.899
0.876
0.944
0.932
0.679
0.578
0.928
0.925
LIQECZIANG ET AL., 2023B)
0.866
0.865
0.913
0.898
ARNIQA(AGNOLL
ET 2024)
0.823
0.797
0.883
0.869
0.670
0.595
0,909
0.904
Q-ALIGN WU AL., 2024)
0,853
0.860
0.941
0,940
0.933
0.930
GENZIQA(DE
2024)
0.897
0.873
0.932
0,916
0.718
0.613
DP-IQA (Fu ET AL., 2024)
0.913
0,893
0.951
0.942
0.683
0.579
0.926
0.923
LGDM-1%
0.853
0.842
0.929
0,921
0.751
0.672
0.912
0.901
LGDM-VERISQUR LGDM GGDM-MMUSTO LGDM-(1- 0, #8DeL5
0,852
0.840
0.924
0,919
0.691
0.598
0,917
0.908
0.869
0,858
0.939
0,928
0.747
0.672
0,922
0.920
0.901
0,893
0.952
0.941
0.799
0.683
0.931
0.929
IQA LGDM-Uspel.6
0.903
0.891
0.952
0.944
0.761
0.679
0.929
0.924
0.940
0.908
0.972
0.967
0.812
0.705
0.948
0.947
Our experiments show that perceptual quality is built pro- gressively during reverse diffusion (later timesteps), making an appropriate range of sampling timestep to be in (0- (0-100), where denotes completely clean image. Finally, with the aggregated diffusion hyperfcatures H, we employ lightweight regression network Is parameterized by ø following standard NR-IQA practice (Madhusudana al., 2022; Saha et al., 2023; Saini et al., 2024) to predict the perceptual quality score:
licly available and well-recognized IQA datasets, covering synthetic distortions, authentic distortions, and the latest AI- generated content (AIGC). These datasets are summarized in Table 8 (Appendix D). Many previous methods focused only on synthetic distortions, because of the difficulty gen- eralizing to real-wor real-world distortions. By contrast, diverse dataset that includes both synthetic and authentic distortions, allowing for fair comparison across all types of IQA datasets, including recent AIGC datasets.
LDM is
pretrained
9p 96 H
(13)
For LDM, we use the widely adopted Stable Diffusion v1.5 (Rombach et al., 2022), pretrained the LAION-5B dataset (Schuhmann et al., 2022). For text conditioning, use empty string as prompt. We run 10 DDIM steps, with t within the range (0, 100] and set the hyperparameters Çi and {2 to and 0.2, respectively. For regression, we use small neural network with two hidden layers. We use Pear- son Linear Correlation Coefficient (PLCC) and Spearman's Rank Order Correlation Coefficient (SRCC) as evaluation metrics. The impact of the choice of 1i, is discussed in detail in the ablation study and Appendix D. All experi- ments were conducted on an NVIDIA A100 GPU using PyTorch. Additional implementation details are provided in the Appendix
where 4p the predicted quality score. Importantly, the diffusion model 89 remains fixed and is not fine-tuned at any stage, preserving its zero-shot generalization capabil- ities. The use of multi-scale and multi-timestep features enables the model to be sensitive to different types of dis- tortions and image artifacts, which might not be captured when using single-scale feature. Our experiments show that LGDM with PMG gives better performance across all NR-IQA benchmarks (see Section 4).
Experiments
4.1 Experimental Settings
To thoroughly evaluate the effectiveness of our proposed method, we conducted extensive experiments on ten pub-
6

---

Latent Guidance in Diffusion Models for Perceptual Evaluations
Table 2: Comparison of our proposed LGDM with SOTA NR-IQA methods on PLCC and SRCC Scores for Synthetic IQA datasets.
METHODS
LIVE
CSIQ
TID2013
KADID
PLCC1
SRCC
PLCC1
SRCCI
PLCC1
SRCC1
PLCC1
SRCC1
ILNIQR (ZHANG AI.,, 2015)
0.906
0.902
0.865
0.822
0.648
0.521
0.558
0.534
BRISQUE (MITTAL ET AL., 2012)
0.944
0.929
0.748
0.812
0.571
0.626
0.567
0.528
WADIQAM (BOSSE ET AL., 2018)
0.955
0.960
0.844
0.852
0.855
0.835
0.752
0.739
DBCNN (ZHANG ET AL., 2020)
0.971
0.968
0.959
0.946
0.865
0.816
0.856
0.851
TIQA (STEPIEN & OSZUST, 2023)
0.965
0.949
0.838
0.825
0,858
0.846
0.855
0.850
METAIQA (ZHU AL.,, 2020)
0.959
0.960
0.908
0.899
0.868
0.856
0.775
0.762
P2P-BM (YING ET AL.,, 2020)
0.958
0.959
0.902
0.899
0.856
0.862
0.849
0.840
HYPERIQA (SU ET AL., 2020)
0.966
0,962
0,942
0,923
0.858
0.840
0.845
0.852
TRES (GOLESTANEH ET AL., 2022)
0.968
0.969
0,942
0.922
0.883
0.863
0.859
0.859
MUSIQ (KE ET AL., 2021)
0.911
0.940
0.893
0.871
0,815
0.773
0.872
0.875
RE-IQA (SAHA ET AT.,, 2023)
0.971
0.970
0.960
0.947
0.861
0.804
0.885
0.872
LoDA (XU ET AL., 2024)
0.979
0.975
0.901
0.869
0.936
0.931
LGDM-
0.980
0.978
0.971
0.952
0.871
0.814
0.892
0.885
LGDM EBRISQUE
0.980
0.977
0.970
0.951
0.814
0.771
0.841
0.838
LGDM MUMSIQ
0.981
0,979
0.969
0,950
0.869
0.811
0.898
0.887
LGDM-UR- RE-M-RRe-IQ..
0.983
0.981
0.972
0.952
0.904
0.882
0.932
0.930
LGDM (1 0 @SDDe..5 L.GDM-05De15
0.981
0.978
0.971
0.958
0.908
0.876
0.935
0.931
0.988
0.986
0.981
0.964
0.921
0.883
0.961
0.958
4.2 Experimental Results & Comparisons
Table PLCC and SRCC comparison of LGDM on AI-
Generated Datasets for IQA. The best results are in red, and the second-best results are in blue.
We evaluate LGDM ten datasets. Table presents the performance of LGDM four authentic distortion ("In the Wild") datasets, with LGDM-$sDOL. achieving the best results across all datasets. Here d'sDv1.5 denote that we use SDv1-5 as perceptual metric, and feed its intermediate feature during sampling. Specifically, on the LIVEC (Ghadi- yaram & Bovik, 2015) dataset, LGDM-USDDI.. attained PLCC of 0.940 and an SRCC of 0.908, significantly surpass- ing the previous best method, LoDA (Xu et al., 2024). On the FLIVE dataset (Ying et al., 2020), which contains the largest collection of human-labeled authentically distorted images emulating social media content (UGC), our method achieves a ofate-of-the-art PLCC of 0.812 and an SRCC of 0.705, demonstrating its robustness handling diverse and complex real-worke distortions.
AGIQA-IK IK
AGIQA-3K
PLCC:
SRCC
PLCC|
SRCC
CONTRIQUE (Madhusudana al., 2022)
0.708
0.670
0.804
RE-IQA (Sala 2023)
0.670
0.614
0.845
0.785
GenZIQA (De 2024) 2024)
0.861
0.840
0.892
0.832
LGDM-U
0.903
0.891
0.929
0.83
TID2013 (Ponomarenko al., 2013), and KADID (Lin et al., 2019). The results demonstrate that proposed ap- proach outperforms other methods across almost all datasets. Similar to the performance authentic distortion dataset in Table 1, LGDM-UsDeLL. achieves the best performance across all synthetic datasets, indicating a strong alignment with human perceptual judgments. The results also sug- gest that perceptual guidance using v "'te-Q.A and d's consistently enhances the model's generalization capabili- ties. The superior performance of LGDM- TID2013 (Ponomarenko et al., 201 3)nd KADID (Lin et al., 2019) datasets, with PLCC and SRCC scores of 0.921/0.Am 883 and 0.961/0.955 respectively, underscores the value of uti- lizing diffusion hyperfcatures. Additional results are pro- vided in Appendix D.
We also evaluated our method on AIGC datasets to assess its ability to handle Al-generated images, which often present unique challenges. As shown in Table 3, LGDM-UsDDLL outperformed previous methods on both AGIQA-IK (Li al., 2023) and AGIQA-3K (Li al., 2024a) datasets, achieving PLCC scores of 0.903 and 0.929, respectively. As compared to GenZIQA (De et al., 2024), the previous best- performing method, our approach demonstrates significant improvements, highlighting its strong prior for Al-generated content, which is often lacking in previous methods.
on the
Table 2 reports the PLCC and SRCC scores of LGDM and existing NR-IQA methods on four synthetic datasets: LIVE (Sheikh et al., 2006), CSIQ (Larson & Chandler, 2010),
4.3 Ablation Study
Cross-Dataset Generalization. We conducted cross- dataset evaluations to assess the generalization capability

---

Latent Guidance in Diffusion Models for Perceptual Evaluations
Table 4: SRCC Scores for Cross Dataset Evaluations. The best results are in red, and the second-bor -estt results are in blue.
Table 6: Inference Speed Comparison. LGDM times are for sampling only with half precision.
Method
Est. Time (s)
Note
Train
Test
Methods
QCN (Shin et al., 2024)
0.15
Geometric Ordering LORA Finetuning
Q-Align (Wu et al., 2024)
0.1
REIQA
DEIQT
LoDA
L.DM--'DD+13
DP-IQA (Fu
2024)
0.023
Distilled/Finetueed DM Finetuned DM (8 steps avg)
FLIVE KonlQ
0.764
0.733
0.763
0.802
GenzIQA (De et al., 2024)
1.4
LGDM step)
1.1
Single Step SOTA Accuracy
FLIVE LIVEC
0.699
0.781
0.805
0.849
LGDM (10 steps)
9.7
KonIQ LIVEC
LIVEC
0.791
0.794
0.811
0.853
KonlQ
0,769
0.744
0.745
0.794
Time Step Variations and Performance. Table 5 shows the impact of varying the number of timesteps during sampling on FLIVE (Ying et al., 2020) using LGDM--SDDLLs. We observe a convex trend in the SRCC scores perfos--erfoorancce improves with an increase in the number of timesteps up to 50, but further increments result in diminishing returns, with increased computational cost (time taken). For practical use, trade-off between SRCC and computational efficiency is required. We choose the optimum value of 10 time steps that gives us very close to highest performance with five times less compute time.
of our method. Table presents the results of inter-dataset evaluations. Our LGIDM-//sDeLs consistently achieved the highest SRCC scores across all cross-dataset combinations, demonstrating the robustness and strong generalization ca- pabilities of I.GDM's perceptual feature maps. Interestingly, when trained on smaller datasets such as LIVEC and eval- uated on x10 larger dataset KonIQA, LGDM-USDwIL still outperforms other methods by large margin. This demon- strates the strong perceptual features of LGDM and less dependence on regression model.
Table 6 compares the inference speed. Single-step variant of LGDM (1 step) offers a competitive inference runtime with a trade-off in accuracy. It is important to consider that L.DDM utilizes zero-shot backhone, which significantly reduces computational costs during the training or adap- tation phase compared to methods requiring finetuning or distillation. This presents a balance where some inference speed is exchanged for substantially higher accuracy and the adaptability of a frozen foundation model.
Different Choices of Vp. We evaluated different perceptual model features 9i, to analyze their impact on perceptual guidance during the sampling process. We list experimental results for all variants of ti, in Table 1 and 2. Specifically, we use onno perceptual guidance), BRISQUE, MUSIQ, RE- IQA, and SDv1-5. Ç1 = O implies that during sampling we do not perform update in Equation 9, i.c. gradient for 12 loss of actual image, and just rely on perceptual fea- tures' loss (Equation 10). Given large pretrained diffusion model, they have strong bias for perceptual features that provide the best guidance, hence V/DDe.. 5 gives the best performance across datasets. It should be noted that LGDM with perceptual features generally boosts the performance of underlying perceptual metric with strong priors from Diffu- sion models. For instance, RE-IQA demonstrating a strong perceptual correlation, when used with LGDM gives around 3% performance boost, While models with worse hu- man judgment correlation (e.g., (BrISQQE) tend to reduce performance by deviating samples away from the percep- tually consistent regions as compared when no perceptual guidance is applied (1+). Similar experiments were also conducted on synthetic datasets, please see Appendix D.
Table 7: PLCC and SRCC for Different Versions of SD LIVEC (Ghadiyaram & Bovik, 2015) and FLIVE (Ying et al., 2020). The best results are highlighted in red, and the second- best results are highlighted in blue.
SD Version
LIVEC
FLIVE
PLCC
SRCC1
PLCC1
SRCC
1.3
0.932
0.901
0.804
0.695
1.4
0.938
0,903
0,807
0.698
1.5
0.940
0.908
0,812
0.705
2
0,910
0.882
0.781
0.674
2.1
0.917
0.886
0.788
0.681
Effect of Different Versions of SD. We also compared the effect of using different versions of Stable Diffusion. Table 7 shows that SD v1.5 consistently outperforms other versions, achieving the highest PLCC and SRCC scores. Specifi- cally, SD v1.5 reaches an SRCC of 0.908 LIVEC and 0.705 on FLIVE, outperforming newer versions like v2.0 and 12. which exhibit lower correlation values. The de- cline in performance in newer versions may be attributed to architectural changes or training modifications that diverge from the characteristics required for effective NR-IQA, i.e.
Table 5: SRCC and time taken for different number of timesteps on FLIVE (Ying et al., 2020) by LGDM-
Time Steps
SRCC
Time Taken (s)
0,624
3.27
5
0.673
9.89
10
0.705
21.30
50
0.711
110.45
8

---

Latent Guidance in Diffusion Models for Perceptual Evaluations
focus on the generation of high quality aesthetic image, rather than broader coverage of image quality.
reported the corresponding SRCC values for each decoder layer in Fig. 6.
5 Conclusion
216
3.40
316
3.55
In this work, we introduced the Latent Guidance in Diffu- sion Models (LGDM) for No-Reference Image Quality As- sessment (NR-IQA). Leveraging the strong representation capabilities of pretrained latent diffusion models (LDMs), we proposed Perceptual Manifold Guidance (PMG) to direct the sampling process toward perceptually consistent regions on the data manifold. We demonstrated the value of ex- tracting multi-scale and multi-tine malti--immesep features-diffusioon diffusion hyperfeatures from the denoising U-Net, providing a rich representation for quality assessment. To our knowledge, this is the first work utilizing pretrained LDMs directly for NR-IQA without any fine-tuning of diffusion backbone.
3.6
0.00
3.9
378
3.5
0.00
366
3.88
Figure 5: Effect of (2 on the hyperfeatures. Figure on the left and right, shows the PLCC and SRCC values respec- tively for FLIVE and LIVEC datasets. Hyperfeatures were collected for single time step.
Impact of Weights of Perceptual Guidance Term (O). We conducted experiments to evaluate the effect of different values for the hyperparameters Çi and (2. LGDM- in Table 1 represents the case where only the first term in PMG is used. The first term provides a strong baseline due to content bias effects from data consistency. We set Ç1 = following (Rout et al., 2024). In Fig. 5, we show SRCC scores for different values of &2 on FLIVE. We observe that values too small or large for Ç lead to poor perceptual features by pushing the samples away from the perceptually consistent region on M.
6 Acknowledgment
This work was supported by the National Science Foun- dation Al Institute for Foundations of Machine Learning (IFML) under Grant 2019844. The authors thank the Texas Advanced Computing Center (TACC) at The University of Texas Austin for providing compute infrastructure that contributed to the part of research outcomes in this paper.
7 Impact Statement
LONG
This paper presents work whose goal is to advance the field of Image quality assessment. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here.
References
Agnolucci, L.,, Galteri, I.., Bertini, M., and Del Bimbo, A. Amiqa: Learning distortion manifold for image quality assessment. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 189- 198, 2024.
Undt DootDoodeet.o Layer
Figure Contribution of individual layer of SDv1.5 to- wards SRCC for FLIVE, LIVEC, and TID-2013. A consis- tent pattern of contribution of layers across synthetic and authentic datasets.
Babnik, 2. Peer, P., and Struc, V. ediffiqa: Towards effi- cient face image quality assessment based on denoising diffusion probabilistic models. IEEE Transactions on Biometrics, Behavior, and Identity Science, 2024.
Contribution of Individual SD Layers We also investigate the contribution of individual layer in SDv1-5 U-Net. It may be observed that different layers contribute differently to the overall performance. Given the high computational cost of running SD, we limited this experiment to single time sampling. Notably, layer 4 contributes significantly more to final quality prediction than other layers. Initial layers seems to develop strong perceptual correlation, we have
Baranchuk, D., Voynov, A., Rubachev, I., Khrulkov, V.. and Babenko, A. Label-efficient semantic segmenta- tion with diffusion models. In International Confer- ence on Learning Representations, 2022. URL https: / // /ooenrevieww net htt//forrmmmddce wttp:rummiiiiixxxxxxxxxxxx
Bordt, S., Upadhyay. U., Akata, Z., and von Luxburg, U. The manifold hypothesis for gradient- explanations.

---

Latent Guidance in Diffusion Models for Perceptual Evaluations
In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pp. 3697-3702, 2023.
Gao, T., Pan, W. Zhang, Y., Zhao, S., Zhang, S., Zheng, X., Li, K., Cao, L.., and Ji, R. Local manifold learning for no-reference image quality assessment. arXiv preprint arXi::2406.19243 2024.
Bosse, S., Maniry, D., Müller, K.-R., Wiegand, T., and Samek, W. Deep neural networks for no-reference and full-reference image quality assessment. IEEE Transac- tions on Image Processing, 27(1):20--219 2018. doi: 00.1099/IP2201.2766551
Ghadiyaram, and Bovik, A. C. Massive online crowd- sourced study of subjective and objective picture quality. IEEE Transactions Image Processing, 25(1):372-387, 2015.
Ghildyal, A., Barman, N., and Zadtootaghaj.. S. Founda- tion models boost low-level perceptual similarity metrics. arXiv preprint wrXiv:24499.07600 2024.
Chung, H., Sim, B., Ryu, D., and Ye, J. C. Improving diffusion models for inverse problems using manifold constraints. Advances in Neural Information Processing Systems, 13-25683-25696 2022.
Golestaneh, S. A., Dadsetan, S., and Kitani, K. M. No- reference image quality assessment via transformers, rel- ative ranking, and self-consisteney. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp- 1 1220-1230, 2022.
Chung, H., Kim, J., Mccann, M. T., Klasky, M. L., and Ye, C. Diffusion posterior sampling for general noisy inverse problems. In The Eleventh International Confer- ence on Learning Representations, 2023. URL https: // //openreview.oo net.
net forum? On 9GGAGTOk.
Guan, F., Jiang, G., Song, Y., Yu, M., Peng, Z.,, and Chen, F. No-reference setb-dynamie-rangee image quality as- sessment based on tensor decomposition and manifold learning. Applied Optics, 57(4))839-844 2018
Chung. H., Lee, S., and Ye, J. C. Decomposed diffusion sampler for accelerating large-scale inverse problems. In The Twelfth International Conference on Learning Representations, 2024. URL https: // http://eeeeeeeeee//////eeee net/ forum? i i DsEgtt FAG.
Guan, J., Yi, S., Zeng. X., Cham, W.-.., and Wang, X. Vi- sual importance and distortion guided deep image quality assessment framework. IEEE Transactions Multime- dia, 19(11):2505-257) 2017.
De, D., Mitra, S., and Soundararajan, R. Genziqa: Gen- eralized image quality assessment using prompt-guided latent diffusion models. arXiv preprint arXiv:2406.04654 2024.
He, Y. Murata, N., Lai, C.-H., Takida, Y., Uesaka, T., Kim. D., Liao, W.-H., Mitsufuji, Y., Kolter, Z., Salakhutdinov, R., and Ermon, S. Manifold preserving guided diffusion. In The Twelfth International Confer- ence Learning Representations, 2024. URL https: //openreview, net / for worum? BBx0Lod
Efron, B. Tweedie's formula and selection bias. Journal of the American Statistical Association, 106(4 1614, 2011.
Fang, Y., Zhu, H., Zeng, Y., Ma, K., and Wang, Z. Percep- tual quality assessment of smartphone photography. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 3677-3686, 2020.
Ho, J., Jain, A., and Abbeel, Denoising diffusion proba- bilistic models. Advances in neural information process- ing systems, 33:6840-6851, 2020.
Hosu, V., Lin, H., Sziranyi, T., and Saupe, D. Koniq- 10k: An ecologically valid database for deep learning of blind image quality assessment. IEEE Transactions on Image Processing, 29:4041-4P0 2020.
Fei, B., Lyu, Z., Pan, L., Zhang, J., Yang. W., Luo, T., Zhang, B., and Dai, B. Generative diffusion prior for unified image restoration and enhancement. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern 2023.
Hou, J., Zhu, Z., Hou, J., Liu, H., Zeng, H., and Yuan, Global arreeuue-awaare aware diffusion process for low-light image enhancement. Advances in Neural Information Processing Systems, 36, 2024.
Recognition, pp.
Feng, R., Gao, Y., Jin, X., Feng, R., and Chen, Z. Semanti- cally structured image compression via irregular group- based decoupling. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision, pp. 17237- 17247,200 2023.
ITU-T RECOMMENDATION P. Subjective video quality assessment methods for multimedia applications. 1999.
Jiang, Q. Shao, F., Gao, W., Chen, Z., Jiang, G., and Ho, Y.-S. Unified no-reference quality assessment of singly and multiply distorted stercoscopic images. IEEE Trans- actions on Image Processing, 166-1881, 2018
Fu, H., Wang, Y., Yang, W., and Wen, B. Dp-iqa: Utilizing diffuusion prior for blind image quality assessment in the wild. arXiv preprint 4405 19996, 2024.
10

---

Latent Guidance in Diffusion Models for Perceptual Evaluations
Kawar, B., Elad, M., Ermon, S., and Song, J. Denoising diffusion restoration models. Advances in Neural Infor- mation Processing Systems, 35:23593-23606, 2022.
Moorthy, A. K. and Bovik, A. C. Blind image quality assessment: From natural scene statistics perceptual quality. IEEE transactions Image Processing, 20(1)): 3350-3364, 2011.
Ke, Wang, Q.. Wang, Y. Milanfar, P., and Yang. Musiq: Multi-scale image quality transformer. Proceedings of the IEEE/CVF international conference on computer vision, pp. 5148-5157, 2021.
Park, Y.-H., Kwon, M., Choi, J., Jo. J., and Uh Y. Un- derstanding the latent space of diffusion models through the lens of riemannian geometry. Advances in Neural Information Processing Systems, 36:22129- -41422, 2023.
Larson, E. C. and Chandler, D. M. Most apparent distortion: full-reference image quality assessment and the role of strategy. Journal of electronic imaging, 19(1):011006 011006, 2010.
Ponomarenko, N., Ieremeiev, O., Lukin, V., Egiazarian, K., Jin, L.,, Astola, J., Vozel, B., Chehdi, K., Carli, M., Battisti, F., et al. Color image database tid2013: Pecu- liarities and preliminary results. In European workshop on visual information processing (EUVIP), pp. 106-111. IEEE, 2013.
Li, C., Zhang, Z., Wu, H., Sun, W., Min, X., Liu, X., Zhai, G., and Lin, W. Agiqa-3k: An open database for ai- generated image quality assessment. IEEE Transactions Circuits and Systems for Video Technology. 2023.
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pp. 10684-10695, 2022.
Li, C., Kou, T., Gao, Y., Cao, Y., Sun, W., Zhang, Z., Zhou, Y., Zhang, 7.., Zhang, W., Wu, H., et al. Aigiqa-20k: A large database for ai-generated image quality assessment. In Proceedings of the IEEE/CVF Conference Com- puter Vision and Puttern Recognition, pp. 6327-6336, 2024a.
Rout, L., Raoof, N., Daras, G., Caramanis, C., Dimakis, A., and Shakkottai. S. Solving linear inverse problems prov- ably via posterior sampling with latent diffusion models. Advances in Neural Information Processing Systems, 36, 2024.
Li, X., Zheng, J., Hu, R., Zhang, Y., Li, K., Shen, Y., Zheng, X., Liu, Y., Zhang, S., Dai, P., et al. Feature denoising diffusion model for blind image quality assessment. arXiv preprint ar arXiv:2401. 24.1. 11949, 2024b.
Saad, M. A., Bovik, A. C., and Charrier, C. Blind image quality assessment: A natural scene statistics approach in the dct domain. IEEE transactions on Image Processing, 21(8):3339-3352, 2012.
Lin, H., Hosu, V., and Saupe, D. Kadid-10k A large- scale artificially distorted iqa database. In 2019 Eleventh International Conference on Quality of Multimedia Expe- rience (QoMEX), pp. 1-3. IEEE, 2019.
Liu, J., Zhou, W., Li, X., Xu, J., and Chen, Z. Liqa: Lifelong blind image quality assessment. IEEE Transactions 2022.
Saha, A., Mishra, S., and Bovik, A. C. Re-iqa: Unsupervised learning for image quality assessment in the wild. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 5846-5855, 2023.
Multimedia, 25:5358-533
Liu, J., Sun, H., and Katto,1 J. Learned image compression with mixed transformer-cnn architectures. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 14388-14397, 2023.
Saini, S., Saha, A., and Bovik, A. C. Hidro-vqa: High dynamic range oracle for video quality assessment. In Proceedings of the IEEE/CVF Winter Conference on Ap- plications of Computer Vision, pp. 469 409, 2024.
Luo, G., Dunlap, L., Park, D. H., Holynski,
A., and Darrell, T. Diffusion hyperfeatures: Searching through time and space for semantic correspondence. Advances in Neural Information Processing Systemx, 36, 2024.
Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 35: 2022.
Madhusudana, P. C., Birkbeck, N., Wang, Y., Adsumilli, B., and Bovik, A. C. Image quality assessment using contrastive learning. IEEE Transactions on Image Pro- cessing, 31:4149-4161, 2022.
Shao, H., Kumar, A., and Thomas Fletcher, P. The rieman- nian geometry of deep generative models. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 315-323, 2018.
Mittal, A., Soundararajan, R., and Bovik, A. C. Making a "completely blind" image quality analyzer. IEEE Signal processing letters, 20(3):209-21 2012.

---

Latent Guidance in Diffusion Models for Perceptual Evaluations
Sheikh, H. R., Sabir, M. F., and Bovik, A. C. A statistical evaluation of recent full reference image quality assess- ment algorithms. IEEE Transactions image processing, 15(11))3440-33) 2006.
Su, S., Yan, Q. Zhu, Y., Sun, J., and Zhang, Y. From distor- tion manifold perceptual quality: data efficient blind image quality assessment approach. Pattern Recognition, 133:109047, 2023.
Shin, N.-H., Lee, S.-H., and Kim, C.-S. Blind image qual- ity assessment based on geometric order learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 12799-12808, 2024.
Sun, S., Wei, L., Xing, J., Jia, J., and Tian, Q. Sddm: score- decomposed diffusion models on manifolds for unpaired image-to-image translation. In International Conference Machine Learning, pp. 53115-33133 PMLR, 2023.
Song. B., Kwon, M., Zhang, Z., Hu, X., Qu, Q., and Shen, I.. Solving inverse problems with latent diffu- sion models via hard data consistency. In The Twelfth International Conference on Learning Representations, 2024. URL https: pestps://openreeee http:///pppeeeeeeeeeeeeeeeeeeeee wwwprrevviennne//oooeeeee net//erunrrr id= (b==jhhdRqOhh
Tu, Z., Yu, X., Wang, Y., Birkbeck, N., Adsumilli, B., and Bovik, A. C. Rapique: Rapid and accurate video quality prediction of user generated content. IEEE Open Journal of Signal Processing, 2:425-440, 2021.
Tumanyan, N., Geyer, M., Bagon, S., and Dekel, T. Plug- and-play diffusion features for text-driven image-to- image translation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pp. 1921-1930, 2023.
Song. J., Meng, C., and Ermon, S. Denoising diffu- sion implicit models. International Conference on Learning Representations, 2021a. URL https:// openreview net/ wtttt/oorummi-sttliaaaate
Song, J., Zhang, Q., Yin, H., Mardani, M., Liu, M.-Y., Kautz, J., Chen, Y., and Vahdat, A. Loss-guided diffu- sion models for plug-and and-pay controllable generation. In International Conference on Machine Learning, pp- 32483-32498. PMLR, 2023.
Vincent, P. connection between score matching and de- noising autoencoders. Neural computation, 23(7):1661- 1674, 2011.
Wang, Z. and Bovik, A. C. Modern image quality assess- ment. PhD thesis, Springer, 2006.
Song, Y. and Ermon, S. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
Wang. Z., Hu, B., Zhang. M., J., Li, L., Gong, M., and Gao, X. Diffusion model-based visual compensation guidance and visual difference analysis for no-reference image quality assessment. IEEE Transactions on Image Processing, 34:263-278, 2025. doi: 10.1 109/ 3523800.
Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., and Poole, B. Score-based generative mod- cling through stochastic differential equations. In In- ternational Conference on Learning Representations, net/
2021b. URL
Wu, H., Zhang, Z., Zhang, W., Chen, C., Liao, L., Li, C., Gao, Y., Wang, A., Zhang, E., Sun, W., Yan, Q.. Min, X., Zhai, G., and Lin, W. Q-align: Teaching LMMs for visual scoring via discrete text-defined lev- els. In Forty-firs International Conference on Machine Learning, 2024. URL https: openreview net/ forum? id PHjj-jj7j8....
id=PxTIG12RRHS
Srinath, S., Mitra, S., Rao, S., and Soundararajan, Learn- ing generalizable perceptual representations for data- efficient no-reference image quality assessment. In Pro- ceedings of the IEEE/CVF Winter Conference on Appli- cations of Computer Vision, pp. 22-31, 2024.
Wu, W., Zhao, Y., Chen, H., Gu, Y., Zhao, R., He, Y., Zhou, H., Shou, M. Z., and Shen, C. Datasetdm: Synthesizing data with perception annotations using diffusion models. Advances in Neural Information Processing Systems, 36: 54683-54695 2023.
Srinivas, S., Bordt, S., and Lakkaraju, H. Which models have siceeptually-allined gradients? an explanation via off-manifold robustness. Advances in neural information processing systems, 36:21172-21195 2023.
Stepien, I. and Oszust, M. Tiqa-psi: Toolbox for percep- tual image quality assessment of pan-sharpened images. SoftwareX, 23:101494 2023.
Xu, J., Liu, S., Vahdat, A., Byeon, W., Wang, X., and De Mello, S. Open-vocabulary panoptic segmentation with text-to-image diffusson models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- ter Recognition, pp. 2955- 2023.
Su, S., Yan, Q. Zhu, Y., Zhang, C., Ge, X., Sun, J., and Zhang, Y. Blindly assess image quality in the wild guided by a adlf-adaptive hyper network. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 3667-3676, 2020.
Xu, K., Liao, L., Xiao, J., Chen, C., Wu. H., Yan, Q., and Lin, W. Boosting image quality assessment through
12

---

Latent Guidance in Diffusion Models for Perceptual Evaluations
efficient transformer adaptation with local feature hancement. In Proceedings of the LEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 2662- 2672, 2024.
Ye, J., Wang, N., and Wang, X. Featurenerf: Learning generalizable nerfs by distilling foundation models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8962 8962-8973, 2023.
Ying, Z., Niu, H., Gupta, P., Mahajan, D., Ghadiyaram, D., and Bovik, A. From patches pictures (paq-2-piq): Mapping the perceptual space of picture quality. In Pro- ceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 3575-3585, 2020.
You, and Korhonen, Transformer for image quality assessment. In 2021 IEEE international conference image processing (ICIP), pp. 1389-1393, IEEE, 2021.
Yu. J., Wang. Y., Zhao, C., Ghanem, B., and Zhang, J. Free- dom: Training- friee energy-guides conditional diffusion model. In Proceedings the IEEE/CVF International Conference Computer Vision, pp. 23174-23188 234 2023.
Zhang, C., Zhang, C., Zhang, M., and Kweon, I. S. Text-to- image diffusion models in generative ai: A survey, arXiv preprint ar 23Xi::2333007909 2023a.
Zhang, L., Zhang, L., and Bovik, A. C. A feature-enriched completely blind image quality evaluator. IEEE Transac- tions on Image Processing, 24(8):2579-259) 2015.
Zhang, W. Ma, K., Yan, J., Deng. D., and Wang. Z. Blind image quality assessment using a deep bilincar convolu- tional neural network. IEEE Transactions on Circuits and Systems for Video Technology, 30(1):36-47, 2020.
Zhang, W., Zhai, G., Wei, Y., Yang, X., and Ma, K. Blind image quality assessment via vision- vision-languee language correspon- dence: A multitask learning perspective. In Proceedings of the IEEE/CVF conference on computer vision and pat- tern recognition, pp. 14071-14081, 2023b.
Zhang, Y., Shi, X., Li, D., Wang. X., Wang. J., and Li, H. A unified conditional framework for diffusion- image restoration. Advances in Neural Information Processing Systems, 36, 2024.
Zhao, W., Rao, Y., Liu, 2., Liu, B., Zhou, J., and Lu, Unleashing text-to-image diffusion models for visual per- ception. Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5729-5739, 2023.
Zhu, H., Li, Ju, J., Dong, W., and Shi, G. Metaiqa: Deep meta-learning for no-reference image quality assessment. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 14143-14152, 2020.
13

---

Latent Guidance in Diffusion Models for Perceptual Evaluations
Appendix
Here, we provide additional theoretical proof, implementation details, and experimental results to complement those in the main paper. Specifically, Section A discusses more related work and background diffusion models and NR-IQA, Section B presents detailed theoretical proofs and supporting discussionn, Section C describes the implementation details, Section D includes further quantitative analyses to demonstrate the performance of LGDM, Finally in Section E we discuss the main limitation of our proposed method and possible extensions.
A Related Work
A.1 NR-IQA
No-Reference Image Quality Assessment (NR-IQA) has been a focal point of research over the past two decades, aiming to evaluate image quality based on human perception without relying on reference images. Early approaches predominantly utilized handcrafted features derived from natural scene statistics (NSS), with models such as BRISQUE (Mittal et al., 2012), DIIVINE (Moorthy & Bovik, 201 1), BLIIDD (Saad al., 2012), and NIQE (Mittal et al., 2012). While these methods effectively leveraged statistical regularities in natural images, their performance often suffered when dealing with complex or unseen distortions due to their reliance on specific statistical models.
The emergence of deep learning introduced convolutional neural networks (CNNs) into NR-IQA, enabling models to learn hierarchical feature representations directly from data. Transformer-base architectures further advanced the field by capturing long-range dependencies and contextual information, with models such as MUSIQ (Ke et al., 2021), TReS (Golestaneh et al., 2022), and TRIQ (You & Korhonen, 2021) demonstrating significant improvements performance. Despite these advancements, a major limitation persists: the lack of large-scale, diverse datasets encompassing the full spectrum of real-world distortions. This scarcity hampers the generalization capabilities of NR-IQA models, as they trained on datasets that do not adequately represent all possible image degradation scenarios.
To tackle the complexity of image distortions, the concept of perceptual or distortion manifolds has been explored in image quality assessment models. Manifold learning techniques aim to uncover the intrinsic low-dimensional structures within high-dimensional data, which better align with human visual perception. For instance, Jiang et al. (Jiang et al., 2018) applied manifold learning to reduce the dimensionality of RGB images, constructing low-dimensional representations for stereoscopic image quality assessment. Similarly, Guan et al. (Guan al., 2017) employed manifold learning on feature maps to capture the intrinsic geometric structures of high-dimensional data in low-dimensional space, thereby enhancing prediction accuracy for High-Dynamic-Ran Range (HDR) images. These approaches highlight the potential of manifold learning in modeling the complex relationships between image content and perceived quality.
Although diffusion models (DMs) have demonstrated remarkable efficacy in generating high-dimensionn data and capturing rich feature representations within their intermediate layers (Ho et al., 2020; Song et al., 202 la), their application to NR-IQA has been minimal. Existing works incorporating DMs often use them for specific tasks, such as quality feature denoising or image restoration, effectively converting NR-IQA into full-reference IQA (FR-IQA) problems (Li et al., 2024b; Babnik et al., 2024). Typically, these methods involve training on specific IQA datasets, limiting their generalizability to diverse distortions.
In our work, we demonstrate that since diffusion models are trained on large-scale datasets containing user-generated content (UGC) images- with a wide range of authentic and synthetic distortions-thar inherently learn perceptually consistent manifolds. Although these models are not specifically trained for IQA tasks, they are designed to capture data priors by learning score functions, enabling them to model complex data distributions and capture both high-level and low-level features. This capability allows them to generate a diverse set of images with fine details. We believe that, with appropriate perceptual guidance, it is possible to extract features from diffusion models that correlate highly with human perception, in zero-shot setting.
Diffusion models have also demonstrated the ability to learn meaningful representations within their U-Net architectures, as evidenced by studies that leverage intermediate features for various vision tasks (Zhao et al., 2023; Wu et al., 2023). This suggests an opportunity to harness these models for NR-IQA, which has so far remained underexplored. Our
14

---

Latent Guidance in Diffusion Models for Perceptual Evaluations
work aims address this gap by utilizing pretrained diffusion models without any fine-tuning, thereby preserving their inherent generalization capabilities. By extracting multi-scale and multi-time-step features-referreee to as diffusion hyperfeatures-andd incorporating perceptual guidance, we propose a method that overcomes the limitations of current NR-IQA approaches. This strategy leverages the rich representations within diffusion models to improve generalization across diverse image distortions, aligning more closely with human perceptual judgments.
A.2 Diffusion Models
Diffusion models consist of a forward noise process and a backward denoising process. In the discrete formulation (Song et al., b021b; Ho et al., 2020), the forward process manifests as a Markov chain described by:
qXxIN xo) II (x* xk-1), g(>a xx-1) ((1
(A-1)
The coefficients { k= and [= mamually set and may differ depending various diffusion formulations (Song et al., 2021b). Given that each Markov step g(xx: | xx-1) 1) is linear Gaussian model, the resultant marginal distribution gXx, Xo) assumes a Gaussian form, (ccxoo /) The parameters {cu]Em and {dx ) can be derived from {ax and {bx 1R-1- For sample generation, we train a neural network, so sx(X,, tx), to estimate the score function Vxu log (Xx, xo). The backward process, which we assume to be a Markov chain, is typically represented as:
(x xx) (x)x((xx)) + (x w
(A-2)
where xx(xx) = X5 + (xx 4 /c) Cr is the predicted xa obtained from the Tweedie's formula. Here {ux] { J and fur)k== can be computed from the forward process coefficients {a and {bx ) The formulation Equation A- 2 encompasses many stochastic samplers of diffusion models, including the ancestral sampler in DDPM (Ho et al., 2020),
and the DDIM sampler in (Song et al., 2021b). For variance-presert diffusion models (Ho al., 2020), we have:
ak Vak,
by = V Bk,
v akk.
ds =
v1
(A-3)
where 0g := - Bk, ā. = IT aj, and age 3h follow the notations in (Ho et al., 2020). DDPM sampling:
uk Va--1- Uk -ax āx-1), wk / Bx V
(A-4)
and for DDIM sampling (Song et al., 2021b), have:
Uk V 1i-aa-1-- of vI aks
Uk
wk dkk
(A-5)
where the conditional variance sequence {ax 1 can be arbitrary. And depedning on the value of of. it can become DDPM or DDIM sampling, i.e. With 3
(1-ng 1). it become DDPM.
(I-6a.)
Score Based Diffusion Models.Let 2o pl (X) represent samples from the data distribution. Diffusion models define the generative process the reverse of noising process, which can be represented by the pariance-preservve stochastic differential equation (VP-SDE) (Song et al., 2021b) z(t), [0, TY:
B dr rdl + BBdw
(A-6)
where 3r E (0, 1) is the noise schedule of the process, a monotonically increasing function of t, and w is d-dimensional standard Wiener process. This SDE is defined such that on ~ p(X) when t = 0, and as - T, the distribution approaches a standard Gaussian, i.e., Øт NO, I). Our goal is to learn the reverse-time SDE corresponding to equation (A-6):
3r B.V., logp(z;) di + VBdw
dr =
(A-7)
2
15

---

Latent Guidance in Diffusion Models for Perceptual Evaluations
where dw is reverse-time Wiener process and di runs backward, and V., logp(a;) is the score function (Song al., 2021b). We approximate the score function using neural network so(2,, parameterized by 0, trained via denoising score matching (Vincent, 2011):
l- arg min ( (20)
(150 Vx, logp(rs/
(A-8)
Once is learned, we approximate the reverse-time SDE and generate clean data by iteratively solving Equation A-7 from noisy samples (Song & Ermon, 2019).
B Theoretical Proofs
B.1 Lemma (Tweedie's Formula for Exponential Family)
Let p(2/n) belong to the exponential family distribution:
p(= 1)) po(z) exp (17 T(x) - -(()))
(B-1)
where is the natural or canonical parameter of the family, (()) is the cumulant generating function (cfg) (which makes (2) integrate to 1), and po(2) is the density when 17 = 0. Then, the posterior mean == 18/7 2] should satisfy:
(v.((z)) = V logp(((> - V. Logpu(())
(B-2)
Proof. The marginal distribution p(2) can be expressed as:
plz) / Py (n())(nn)d
(B-3)
which, using the form of (2), becomes:
p(z) po(2) / exp (1) T(2) ((n) d(n)
(B-4)
Taking the derivative of p(2) with respect to
V=p(=) = V=po(z) / exp (n T(x) +(n)) d(
(B-5)
/ (V+T(2)) mpo())) (n T()) (n)) p(n)dn
Rearranging, we get:
V V=p(=)= [=po(z)
Po(2) p(=) +(=)+(((0))) (V=T(=)) (p)(())(
(B-6)
which simplifies to:
Vep(z) Vzpo(2)
po(z) p(2) (V.T(=)) / (p)(T)dn.
(B-7)
Thus:
<2p(2) V<po(z)
+ +V.T(())) Bin
(B-8)
p(=) po(z)
Finally:
(V.T(:)) Ein x] = V. logg(=)) V. log
(B-9)
This concludes the proof.
16

---

Latent Guidance in Diffusion Models for Perceptual Evaluations
B.2 Proposition 3 (Tweedie's formula for SDE)
For the case of VP-SDE, we can estimate p(=0 (x)) as:
20/2 == E|20 2x va(t)
(2e + (1 V(t)) (z)))
(B-10)
Proof. For the case VP-SDE, we have
p(2(|20)= (2 a()))) exp
(B-11)
2(1 - a(t))
Gaussian distribution. We can get the canonical decomposition as:
p(zz|=o) = -oo(=)) exp (z) (
- -((0))))))))))))))())())))))--
(B-12)
And,
Po(2)) = (2T(1 a(t))))/22 exp (- |
1
(B-13)
(2(1-a(t))
Vā(t)
T(z)) = - ā(t) 4
(B-14)
@00)]|20|1
((20) = 2(1 a(t))
(B-15)
Therefore, from Lemma 1:
V aa(t)
50 V (zg + 1 a(l) 2t
(B-16)
1 1-a(1)
Giving us:
20/2 va(l) (4+ (1 vá()) log (2())
(B-17)
This concludes the proof.
B.3 Conditional Score functions
As mentioned the main paper, conditional score function can be written as (Equation 6 ):
log
&s v., p(z)(2))
+V log log caje D(zot)) V., (ogp(t)/()) zooe D(=o)t))
(B-18)
Proof. From Baye's theorem we can write the conditional distribution as:
p(zlv(y), y) = ((((w))=))((>> (())))
(B-19)
Note, y is conditionally independent of (z) given 2, for later timesteps in diffusion process as 2 gives more structural information for image. Therefore:
p(zl((y).v) p(ll))a)))(laa)ppaaa
(B-20)
Our score function becomes:
Vz, log = Vz, logp(z)) + Vz, log (o))(() + Vz, log
(B-21)
We can write the posterior as:
p(ylz)) / pl
(B-22)
17

---

Latent Guidance in Diffusion Models for Perceptual Evaluations
Following (Chung et al., 2023) and Proposition 3, can have the posterior
p(ulz)) = p(ylzoj:)
(B-23)
Therefore:
v., (og)( v) Ri 2, logg(zz) Vz, log (z)) + Vz, log (ygp(y)=0)) Following (Rout et al., 2024), we can approximately write the conditional probability for LDM given decoder D:
(B-24)
ply|z)) ply [ro = (zaot))
(B-25)
Note that ignore the gluing term proposed by (Rout et al., 2024) it depends the forward degradation model only valid for inverse problems. Our final conditional score function becomes:
V x logp(zaltpp(). v) 2., log p(2)) + v., logl(y/zoj D(zoje)) + V., log (()p(()))0 D(zoje))
(B-26)
B.4 Proposition 1 (Noisy Data Manifold)
Let the distance function be defined d(x, M) yeM || yl12- and define the neighborhood around the manifold MM as r) {i D M) r). Consider the distribution of noisy data given by p(xx) f p 2z (2) )oro
pla, (m)) = N (v Xo, (1 - -x) represents the Gaussian perturbation of the data at time 4, and a, = II ar the cumulative product of the noise schedule oz,, Under the Assumption the distribution pe(2,) is concentrated on (D dim manifold M, = y E D d(y, VT,M) TI v(1 (D)(D-M)
Proof. (Mainly follow (Chung al., 2022)):
We begin by defining the manifold M as .MM : f== E IK+1:D 0} which represents subspace where the last D - k coordinates are zero. Essentially, this means that M lies within lower-dimensional subspace of IR Let be a x² random variable with n degrees of freedom. We use the following concentration bounds:
P(X 2// 2r) SeTT,
(B-27)
P(X- - n - 22 Se
(B-28)
Now, consider the quantity ΣΦ E1=441 T-nse concentration bounds and setting
which follows a x² distribution with & degrees of freedom. Using the k)'', we can express the following bound:
(D
PP 22 - kIve
Σ
a (D -) 2 2 H)(e+()) 2 21
(B-29)
The above inequality gives range for the summation of the squared components of x; beyond the first A: dimensions. We can now rewrite this in terms of the Euclidean norm of these components:
P
Σ 27 E 1 imk+1
2Ve). 2VE + 22²))
2 8.
(B-30)
where we have defined:
r = v(1 (0))))(D k).
(B-31)
To ensure that the probability holds for a given confidence level 1 8, define:
60--=== = k
log
(B-32)
2
We then use d define:
1 2 / + 26,
Er. min
1, V max
/
(B-33)
v1 1-a(( -k) kk)
18

---

Latent Guidance in Diffusion Models for Perceptual Evaluations
which ensures - 1. This value Dr helps in determining the size of the neighborhood around the manifold Mr. such that:
P P(z, E B(MAG,D- v(1 (1-- (1- - 21
(B-34)
Thus, we have shown that the noisy data distribution p(x+)) is concentrated within a certain neighborhood around the manifold M,, with high probability. The parameter Cr.D- decreasing with respect 5 and D k, because <, is also decreasing in these parameters, and D- is an increasing function of e ''
This concludes the proof.
In pixel space, as discussed by (Chung et al., 2023), the optimization of the guidance term occurs in the entire space R However, from Proposition 1, we know that xy actually lies in much smaller subspace of T. specifically in TR To prevent sampling from deviating from the content-bias region on the manifold, one obvious way to improve the sampling process to restrict the optimization space to M,. specifically the tangent space T M, Previous literature has suggested using autoencoders to approximate this tangent space T Mr (Shao et al., 2018). However, since autoencoders are not trained on intermediate noisy samples, their practical effectiveness is limited. We instead use Latent Diffusion Models (LDM), where the entire sampling process occurs in the latent space M. This approach ensures overall data consistency, but it may still not fully achieve perceptual consistency within the content-bing bias region on the manifold M (see Fig, 1 for an intuitive illustration).
B.5 Proposition 2 (On-manifold sample with LDM)
Given perfect autoencoder, i.e. z = (x)),),, and gradient V soo G( v) E Tz, then D(V zo11 G(zojz: v)) E M.
Proof. We begin by considering a perfect autoencoder, consisting of an encoder & and a decoder D, which satisfies the property æ = D(E(x)). for any data point x X C C. Let zq = E(ro) be the latent representation zo. Since the autoencoder is perfect, we have xg D(z)).
To understand how the encoder and decoder interact in terms of their mappings, consider their Jacobians. The Jacobian of the encoder aE maps changes in the data space Rº to changes in the latent space R&* The Jacobian of the decoder aD d2g maps changes in the latent space R* back to the data space RD,, Since the autoencoder is perfect, encoding and then decoding must recover the original input exactly. This implies that the composition of the encoder and decoder Jacobians must yield the identity mapping:
O0 OD
Oup Ozo = 1,
(B-35)
where I is the identity matrix. This property ensures that the encoder and decoder are exact inverses of each other in terms of their linear mappings at ro and 2p.
Consider a gradient =o G(zoje= y) E Te Z, where Tz 2 is the tangent space of the latent space E at z0. We want to determine the behavior of this gradient when mapped back to the data space using the decoder. The decoder Jacobian 8.g maps vectors from the latent space to the data space. Since V.o. G(zaje y) in the tangent space Te, 2, applying the decoder Jacobian gives:
BD
8D
D(V Puje G(zop. v)) = azo
V Sojt G(zoje. v)-
(B-36)
Since the Jacobian S5e maps changes in the latent space to corresponding changes in the data space, and the latent space Z designed to represent the underlying data manifold M, it follows that:
8D
0D
Tz, Z - Tag
(B-37)
Ozo
Thus, the vector D(V=,. Zale G(zojt= 1)) lies in the tangent space Tz, M of the data manifold at To. This implies that the gradient update, when mapped back to the data space, remains on the data manifold, ensuring consistency in the sampling process.
This concludes the proof.
B.6 Lemma (Distribution Concentration)
Consider the optimality of the diffusion model, i.e., (Voiz + /1 - ax4,,) for 2 E Z. For some c ~ NN T), the sum of noise components v1 - ât-1 - ozco(z1, + 044 DDIM sampling can be expressed as:
19

---

Latent Guidance in Diffusion Models for Perceptual Evaluations
V -1-1 1) +o. 0r6 vI â
(B-38)
where ~ (0, f). Since /1 1111p 02-1 of (2+, and ofe, are independent, their sum is also a Gaussian random variable with mean of O and a variance (1 - - - of + of = (1 ā;-)).
Furthermore, let the latent data distribution p(z) probability distribution with support on the linear manifold M that satisfies Assumption 1. For any p(=), consider
2t = vāc-12 + / ā - - (2,, ozEL.
(B-39)
Then, the marginal distribution Pt- (25-1), which is defined as:
( ( + v ā - of 0f p(=al=))(2)) dz dzy.
(B-40)
probabilistically concentrated on 2,-1 for 6 N),
Proof. Since E0(2, () is independent of e,, their sum is the sum of independent Gaussian random variables, resulting in Gaussian distribution with a variance (1 - ā.-1). By this result, the multivariate normal distribution has mean /ā,-iz and a covariance matrix (1 - T.-1)). Consequently, the marginal distribution of the target can be represented as:
Pe-1(2(-1)= = / N
(B-41)
which matches the marginal distribution defined in Proposition 1. Therefore, in accordance with Proposition 1, the probability distribution (=t-() probabilistically concentrates M
B.7 Theorem 1 (Perceptual Manifold Guidance)
Given Assumption 1, for perfect encoder 8, decoder D, and efficient score function s0 (), let gradient V Soo G1 (D((o))) y) and V Vani (D((())))))))) (y)) reside the tangent space T Z latent manifold Z. Throughout .!l the diffusion process, all update terms 2, remain on noisy latent manifolds 2,, with =ojr in perceptually consistent manifold
locality.
Proof. We begin by establishing that both the gradients for data and perceptual consistency are constrained to the tangent space of the latent manifold, ensuring that updates remain on the manifold during the diffusion process. At = T, we consider the noisy sample 2T generated from a Gaussian distribution. Noisy sample is expressed as:
2T / + VI- VT-āTer, eT NO.
(B-42)
where =o (xro) represents the latent variable corresponding the clean sample zg- The support of the distribution p(20) lies the manifold 2, ensuring that 2α Z. Assume that for all t Tı, there exists 20 E Z such that:
Vū,zo + VI Ove, 10,11)
(B-43)
We aim to prove that this also holds for / Ti - 1. At timestep l = Ti, the two gradients V Zi (for data consistency) and V %o G2 (for perceptual consistency) lie in the tangent space Tz, Z. These gradients contribute to the update of the latent representation. The overall gradient update becomes:
< = 20/7; - (GI V soIry G1 + C2
V 20 G2)
(B-44)
Where ( are scalars. Since both gradients reside in the tangent space T Z, and Assumption 1, the updated term 2//7 remains on the latent manifold 2. Using the update rule for 27,-1, similar to the diffusion update step, we have:
2T - V /or -1007, + /1 or ii, ~'~ N(0,
(B-45)
20

---

Latent Guidance in Diffusion Models for Perceptual Evaluations
Table 8: Summary of the IQA datasets used in our experiments.
Dataset
Type
Images
Description
LIVE IQA (Sheikh al., 2006)
Synthetic
779
29 reference images; distortions at levels
CSIQ-IQA (Larson & Chandler, 2010)
Synthetic
866
30 reference images; distortions
TID2013 ( (Ponomarenko et al.. 2013)
Synthetic
3,000
25 reference images: 24 distortions at 5 levels
KADID-IOK (Lin et al., 2019)
Synthetic
10,125
81 reference images; 25 distortions 5 levels
LIVEC (Ghadiyaram
Bovik, 2015)
Authentic
1,162
Mobile images with real-world distortions
KonlQ-10k ( (Hosu et al., 2020)
Authentic
10,073
Diverse images from YFCC100M dataset
FLIVE ( (Ying ct al., 2020)
Authentic
39,810
Emulates social media content
SPAQ ( (Fang et al., 2020)
Authentic
11,000
Mobile images with annotations
AGIQA-3K (Li et al., 2023)
AIGC
3,000
Al Al-generated images for IQA
AGIQA-IK (Li et al., 2023)
AIGC
1,000
Al Al-generated images IQA
Thus, the updated latent variable remains on the manifold, as the noise component e' is Gaussian, and the mean update is based on 2 E Z. Applying Lemma 2, give us p(21,-1). that is probabilistically concentrated on Zr-1-
The perceptual manifold, subspace of the content-bias manifold defined by the data-consistency gradient Fajie G1. In other words, data consistency keeps the sample within a region where the structural content is retained, and perceptual consistency term ensures that the sample moves toward regions of the manifold Z that are perceptually meaningful. The second gradient (y)) represents a movement within this subspace align with human perception. Since V Zoje G2 resides in the tangent space Tz Z and also influenced by the perceptual features ppp. the update ensures that the latent variable moves towards more perceptually consistent locality within the overall content-bias region.
term V =ojs (D)
Formally, let Meontent be the subspace of manifold corresponding to content consistency based on G1. and let Mppercepuual C Meontent be the sub-manifold that represents regions of perceptual consistency. The update using Vzoji G2 effectively ensures:
201
M perceptual,
(B-46)
where Mperceprual is a more constrained subspace within the content-consistent manifold, ensuring perceptual quality. By induction, we have shown that for all I, there exists a 2 E Z such that 2 remains on the latent manifold throughout the diffusion process. Furthermore, the inclusion of the perceptual consistency gradient ensures that z0j= is updated towards a perceptually consistent region on the manifold. Thus, the final updated latent variable =0js is not only data-consistent but also perceptually consistent within the manifold Z, as required.
This concludes the proof.
C Implementation Details
C.1 Datasets and Evaluation Protocol
The datasets used in our study (Table 8) contain images labeled with Mean Opinion Scores (MOS) following ITU-T P.910 guidelines (ITU-T RECOMMENDATION, 1999). We train the regressor Is using la loss on MOS ground truth values. Evaluation metrics include Pearson Linear Correlation Coefficient (PLCC) and Spearman's Rank Order Correlation Coefficient (SRCC), ranging from 0 to I, with higher values indicating better correlation.
Following (Saha et al., 2023; Madhusudana et al., 2022), we split each dataset into training, validation, and test sets (70%, 10%, and 20%, respectively), using source image- based splits to prevent content overlap. The process repeated 10 times, and median performance is reported to ensure robustness.
C.2 Implementation Details
Model Configuration For text conditioning, we use an empty string " as prompt. We adopt the SDv1.5 and VQ- from the official Stable Diffuuson vl.5, with default settings from GitHub' and Hugging Face?. VQ-VAE is used with 8x
'https://gi http::////gthhub.cc/////////// /ttpv////seee sttbbl e- ffusion Phttps::/huggiiggfaee ctteeaacccccc//ooo ccop/om coostables e- fus fonvoo 5/ S/stable-dae fiussioonn
21

---

Latent Guidance in Diffusion Models for Perceptual Evaluations
Table 9: Performance Comparison on Different Test Datasets when trained on FLIVE (Ying et al., 2020). SRCC scores are reported.
Method
Test: LIVEC
Test: KonIQ
Test: SPAQ
CONTRIQUE (Madhusudana et al., 2022)
0.734
0.777
0,820
RE-IQA (Saha al., 2023)
0.690
0.796
0.825
ARNIQA (Agnolucei et al., 2024)
0.699
0.798
0.837
LIQE (Zhang et al., 2023b)
0.743
0.813
0.713
GRepQ (Srinath et al., 2024)
0,751
0,810
0,829
LGDMM-ssee.5
0.849
0.802
0.838
downsampling for 512 x 512 resolution, which matches the typical resolution of IQA datasets like LIVE (Sheikh et al., 2006) and CSIQ (Larson & Chandler, 2010).
Sampling and Perceptual Features We use 10 DDIM steps for sampling, balancing efficiency and quality, The choice of perceptual metric d crucial using cell-correlated metrics such as RE-IQA MUSIQ improves model performance. Poor metrics can degrade results, shown in our ablation studies.
Guidance Weights The weights for perceptual guidance, Çi and (2, are set to I and 0.2 based on empirical evaluations. This setup provides sufficient guidance, which enhances prediction quality.
Calculating u We simply pass the input image x through their respective perceptual quality model (feature extractors) before starting the main diffusion loop in Algorithm 1. The resulting features are stored as the target (z)). In the case of De1.5, special case where we leverage the SD... model to pre-compute the perceptual features. Crucially, this calculation is done separately and beforehand, and the resulting features are distinct from the H collected later. One can think of these features as having similar structure H. and calculated in similar way, but without line 9 in Algorithm 1.
C.3 Autoencoder
Though a perfect autoencoder is ideal for maintaining samples on the manifold M, the Stable Diffusion v1.5 VAE yields effective results despite minor imperfections. As shown in Table 7, it provides the best performance across configurations.
D Additional Results and Ablation Study
In this section, we provide further empirical evaluations of our proposed Latent Guidance in Diffusion Models (LGDM) by presenting additional experimental results, ablation studies, and analyses to supplement the findings the main paper. We also evaluate the impact of model hyperparameters and different configurations, including the version of Stable Diffusion (SD), the number of time steps, and the weighting of perceptual guidance terms. Lastly, we discuss the impact of various layers of UNet towards NR-IQA performance.
D.1 Impact of Time Step Range Sampling Process
Figure shows the effect of varying the range of time steps used during the sampling process. We observe general decline in SRCC as we increase the time range, specifically when more noisy samples involved. For larger time step ranges, the model relies on noisier intermediate representations, which reduces its ability to accurately predict image quality. This is an expected behaviour since, details in the images are generated towards the later time steps in diffusion process,i.e. less noise. This suggests that optimizing the range of time steps used for feature extraction critical to maintaining high-quality predictions.
D.2 Effect of Time Steps on Quality and Computation Time
Table 5 provides an analysis of the SRCC scores and the computation time for different numbers time steps on the FLIVE (Ying et al., 2020) dataset using LGDM- score, with the highest value of 0.711 obtained 50 time steps. However, this comes the cost of increased computational
As expected, increasing the number of time steps improves the SRCC
22

---

Latent Guidance in Diffusion Models for Perceptual Evaluations
FLIVE SRCC for Ranges of Time Steps
0.7
FLIVE-SRCC
0.5
SRCC
0.5
0.4
0.3
0-100
100-200
200-300
300-400
400 500
500-800
600-700
800-900
100-1000
700
Time Step Range
We report the behaviour of LGDM change the range of timesteps in the sampling process. As move towards larger timestep range buckets, where there are more noisy samples, SRCC on FLIVE (Ying et al., 2020) decreases.
Figure 7:
time, with significant jump from 21.30 seconds for 10 time steps to 110.45 seconds for 50 time steps. This trade-off suggests that while more time steps can yield better performance, it is essential to balance quality with computational efficiency, especially for real-time applications.
Table 10: PLCC and SRCC Scores for varying the weights of second term in PMG (equation 10) on LIVEC (Ghadiyaram & Bovik, 2015) and FLIVE (Ying al., 2020). The best results highlighted in bold, and the second-best results underlined.
62
LIVEC
FLIVE
PLCC1
SRCC1
PLCC1
SRCC1
0.853
0.842
0.751
0.672
0.2
0.940
0.908
0.812
0.705
0.5
0,931
0.901
0.802
0.691
0.7
0.918
0.882
0.780
0.675
0.904
0.868
0.772
0.664
D.3 Effect of Perceptual Guidance Weighting
Table 10 presents the PLCC and SRCC scores for varying the perceptual guidance weight, Ç2. in the Perceptual Manifold Guidance (PMG) term for the LIVEC (Ghadiyaram & Bovik, 2015) and FLIVE (Ying et al., 2020) datasets. The results indicate that an optimal value of (2 = 0.2 yields the best PICCC and SRCC scores all ten datasets. Specifically, an SRCC of 0.908 is achieved on LIVEC (Ghadiyaram & Bovik, 2015) and 0.705 on FLIVE (Ying et al., 2020) at this weight, When &2 is set too high (e.g.. S2 1), the model's performance deteriorates, suggesting that samples move away from the perceptually consistent region manifold. Conversely, setting & too low results in underutilization of the perceptual guidance, which leads to suboptimal quality predictions. With moderate weighting, superior performanc can be achieved across all benchmarks.
D.4 Summary and Insights
Our extended experiments validate the effectiveness of LGDM across various IQA datasets, demonstrating its robustness against both synthetic and real-world distortions. The ablation studies provide valuable insights into the factors that impact model performance:
Model Version Selection: SD v1.5 emerged as the best-performing version for NR-IQA, emphasizing the importance of selecting the appropriate diffusion model.
Time Step Range: If sampling time steps from higher ranges are taken, diminishes the performance significantly.
Total Time steps: While increasing time steps improves prediction quality, it also significantly increases computation time, highlighting trade-off between accuracy and efficiency.
23

---

Latent Guidance in Diffusion Models for Perceptual Evaluations
Layer Importance: Intermediate layers of the diffusion model were found contribute the most towards perceptual quality, suggesting the possibility of optimizing feature extraction by focusing specific layers.
Perceptual Guidance Weighting: The optimal perceptual guidance weight strikes a balance between content and perceptual terms, which crucial for maintaining nigh-quality predictions.
Overall, these results underscore the capability of pretrained diffusion models serve effective feature extractors for NR-IQA tasks, provided that the appropriates guidance is provided. Our method, which exploits the inherent generalization capabilities of diffusion models, successfully advances the statc-of-the-ar in NR-IQA, offering promising approach for future developments in perceptually consistent no-reference image quality assessment.
E Limitations & Extension
Our proposed LGDM framework shows strong performance in NR-IQA; however, there are few limitations and potential extensions worth noting.
The computational cost of our approach, particularly due to the iterative diffusion model sampling, can be high, which might limit real-time or resource-constrained applications. Also, the scope of our work is limited to NR-IQA. We did not extend our evaluation to other low-level vision tasks or explore the use perceptual control image generation due to time constraints and the focus on a single task in this paper. Throughout the experiments and proofs, we assumed that VAE ideal, whereas in real world they introduce notable reconstruction errors.
Limitations:
The general framework of LGDM, particularly its ability extract perceptual features, has potential applications beyond NR-IQA. It can be extended to other low-level vision tasks that are sensitive to perceptual features, such image denoising, super-resolution, and enhancement, where quality assessment is crucial. Moreover, since posterior sampling known to be limiting due to its clean sample estimation from intermediate time step f, one can explore using more advanced techniques such as Bayesian filtering to directly approximate posterior sampling in intermediate time step t, that computationally more efficient as it avoids taking derivatives on the estimated score function. Our method can also be adapted for perceptually controllable image generation. Exploring these directions can help expand the impact of our approach and leverage its strengths across broader range of vision tasks.
Extensions:
In future work, we plan to explore these extensions, allowing LGDM to contribute more broadly to the field of low-level computer vision.
24


Reviews:
[{"review_id": "mbxnls1CBz", "reviewer": "Reviewer_P6XD", "summary": "This paper introduces Latent Guidance in Diffusion Models (LGDM) for No-Reference Image Quality Assessment (NR-IQA), leveraging the powerful representation capabilities of pretrained Latent Diffusion Models (LDMs). Specifically, the authors propose Perceptual Manifold Guidance (PMG) to steer the sampling process toward perceptually consistent regions on the data manifold. Additionally, intermediate multi-scale and multi-time features from the denoising U-Net are utilized to estimate image quality through a lightweight network. The evaluation is conducted on both real and synthetic IQA datasets.\n\n## update after rebuttal\nI appreciate the authors’ detailed response. After re-evaluating the paper in light of their reply, most of my concerns have been addressed. However, the overall performance heavily relies on ψp, which is an existing NR-IQA model. In particular, the case where ψ = ∅ yields results that are inferior the comparison methods. On the other hand, the approach exhibits some innovation, and the experimental outcomes suggest that it leads to improved performance. Taking into account the feedback from other reviewers as well, I have increased my score to 3.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive feedback. We would like to address your questions and concerns below.\n\n1. **Multi-Timestep Features (Weakness 3):** Thank you for bringing up this question. In our method, Perceptual Manifold Guidance (PMG), operates iteratively during the DDIM sampling process. At each step t, PMG applies guidance (Eqs 9, 10) to refine the estimated clean latent $\\hat{z}_{0|t}$ towards the input image $x$ and its perceptual features $\\psi_p(x)$. Using only the final step (t=1) allows for only one application of this guidance, which may not guarantee a convergence on perceptually relevant sub-manifold (Theorem-1). In contrast, performing multiple steps (we use 10 steps) allows PMG to iteratively steer the latent trajectory more effectively onto the perceptually relevant submanifold. This multi-step refinement process is crucial for generating perceptually aligned hyperfeatures **H** that lead to SOTA performance. As shown in Table 4 and Figure-7(Appendix D), using only a single step yields lower performance (SRCC 0.624) compared to using 10 steps (SRCC 0.705). While the 1-step features have the least intrinsic noise, the lack of sufficient PMG iterations makes them less perceptually aligned with the target quality. Aggregating features $h_t$ across time captures the evolution of the image representation during guided reconstruction, providing richer information about quality-related aspects (e.g., texture refinement, artifact removal) than a static snapshot at the final time step. In summary, using single-step, can still produce comparable results, and we encourage the use of fast inference and trade-off with accuracy. \n\n2. **Dependence on $\\psi_p$ (Weakness 1):** We would like to clarify that while the choice of $\\psi_p$ influences the degree of performance boost, LGDM's success is not heavily dependent on it. As shown in Tables 1 (Authentic) and 7 (Synthetic), our method LGDM-$\\psi_{\\phi}$ (where $\\psi_p$ is disabled by setting $\\zeta_2$ = 0) already achieves strong results, often matching or exceeding previous SOTA. This baseline performance relies only on the pretrained LDM's features **H** extracted using data consistency guidance (Eq. 9). This demonstrates the inherent perceptual capabilities captured by the LDM with appropriate sampling and our hyperfeature approach. Furthermore, the $\\psi_p$ term (Eq. 10) acts as a refinement step, guiding the sampling towards a specific perceptual submanifold defined by $\\psi_p(x)$. Its effectiveness depends on how well $\\psi_p(x)$ correlates with human perception. Optimal Guidance is Internal ($\\psi_{SDv1.5}$ Crucially, our best results across all datasets are achieved with LGDM-$\\psi_{SDv1.5}$. This inherently reduces the over-reliance on external quality models.  \n\n3. **Time Complexity (Weakness 2):**   We acknowledge that LGDM’s inference time is higher than that of non-diffusion methods. However, our approach leverages a zero-shot pretrained LDM backbone, avoiding the expensive training or fine-tuning required by methods like DP-IQA or GenZIQA. Furthermore, since our PMG is adaptable, it can be used with any other efficient DM model with strong priors to potentially reduce the inference cost. Below, we compare the inference time with LGDM(sampling only with half precision):\n| Method | Est. Time (s) | Notes |\n| :----------------- | :------------ | :------------------------------ |\n| QCN | ~0.15 | Geometric ordering |\n| Q-Align | ~0.1 | LORA Finetuning |\n| DP-IQA (Teacher) | ~0.023 | Distilled/Finetuned DM |\n| GenzIQA | ~1.4 | Finetuned DM (8 steps avg) |\n| LGDM (1 step) | ~1.1 | Single Step |\n| LGDM (10 steps)| ~9.7 | SOTA Accuracy |\n\n4. **Missing Reference (DP-IQA [Fu et al., arXiv 2024]):** Thank you for pointing out DP-IQA [Fu et al., arXiv 2024]. We have added it to our related work discussion and included it in our comparison tables in the updated manuscript. \n| Method                                     | LIVEC (PLCC)  | LIVEC (SRCC)  | KonIQ (PLCC)  | KonIQ (SRCC)  | FLIVE (PLCC)  | FLIVE (SRCC)  | SPAQ (PLCC)   | SPAQ (SRCC)   |\n| :---------------------- | :-----------: | :-----------: | :-----------: | :-----------: | :-----------: | :-----------: | :-----------: | :-----------: |\n| DP-IQA (Teacher) |     0.913     |     0.893     |     0.951     |     0.942     |     0.683     |     0.579     |     0.926     |     0.923     |\n| **LGDM-$\\psi_{SDv1.5}$ (Ours)**           | **0.940**   | **0.908**   | **0.972**   | **0.967**   | **0.812**   | **0.705**   | **0.948**   | **0.947**   |\n\n5. **Supplementary Material Clarification:** The Appendix included at the end of the main PDF serves as our supplementary material.\n\n[De et al., arXiv 2024] \"GenzIQA: Generalized Image Quality Assessment using Prompt-Guided Latent Diffusion Models.\" arXiv preprint arXiv:2406.04654 (2024).\n\n[Fu et al., arXiv 2024] Fu, Honghao, et al. \"DP-IQA: Utilizing Diffusion Prior for Blind Image Quality Assessment in the Wild.\" arXiv preprint arXiv:2405.19996 (2024).", "claims_and_evidence": "No, why use multi-timestep feature maps instead of just the features from the final step? The authors need to provide more evidence.", "methods_and_evaluation_criteria": "Yes, the method was evaluated on public datasets with standard metrics.", "theoretical_claims": "I did not thoroughly check the correctness of some proofs.", "experimental_designs_or_analyses": "Yes, I have reviewed the validity of the experiment design and analysis.", "relation_to_broader_scientific_literature": "A novel approach for using pretrained unconditional latent diffusion models for NR-IQA.", "essential_references_not_discussed": "Some diffusion model based NR-IQA methods are not including and discussed.\nDP-IQA: Utilizing Diffusion Prior for Blind Image Quality Assessment in the Wild.", "other_strengths_and_weaknesses": "Strengths:\n1. Using pre-trained diffusion models for no-reference image quality assessment is interesting.\n2. The proposed method achieves state-of-the-art performance on public datasets.\n\nWeaknesses:\n1. The performance of the proposed method heavily depends on ψp. ψp is an existing NR-IQA method.\n2. The time complexity of the proposed method is high.\n3. Why use multi-timestep feature maps instead of just the features from the final step? In the initial stages, the features may contain excessive noise, which could potentially affect the accuracy of the evaluation.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "Please see the weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tkLXBuY1Ua", "reviewer": "Reviewer_rraS", "summary": "This paper is the first to propose using a pre-trained latent diffusion model as a perceptual model to extract perception features aligned with human perception for NR-IQA. The authors introduce a novel sampling method that adjusts samples to align with human perception while preserving the manifold. Additionally, they leverage multi-level, multi-timestep features from the diffusion model to jointly estimate the final quality score. The proposed method achieves SOTA performance across all benchmarks and demonstrates superior generalization capability.\n\n## update after rebuttal\nAfter the author explained the settings of previous methods and their own method in the generalization experiments, I believe that this paper is indeed very meaningful for the development of the field. Therefore, I give it a score of 3.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your valuable insights and acknowledgment of our efforts. Below, we respond to your questions and discuss the issues you have raised.\n1. **Clarification on the calculation of $\\psi_p$:** We apologize for the lack of clarity regarding the calculation of the perceptual features $\\psi_p$ used in our Perceptual Manifold Guidance (PMG). In our framework (specifically in Eq. 10 and Algorithm 1, Line 9), $\\psi_p$ represents a feature representation (e.g., a vector or set of feature maps) extracted from the perceptual quality model for the original input image $x$ (the image being evaluated). It serves as a fixed target during the guided diffusion sampling process (think of it as features from models like RE-IQA). We simply pass the input image $x$ through their respective perceptual quality model (feature extractors) before starting the main diffusion loop in Algorithm 1. The resulting features are stored as the target $\\psi_p(x)$. In the case of $\\psi_{SD_{v1.5}}$, a special case where we leverage the $SD_{v1.5}$ model to pre-compute the perceptual features. Crucially, this calculation is done separately and beforehand, and the resulting features are distinct from the H collected later. One can think of these features as having a similar structure as **H**, and calculated in a similar way, but without line 9 in Algorithm-1. We hope this clarifies that $\\psi_p(x)$ is a pre-computed target feature set derived from the input image $x$, guiding the sampling process, while **H** is the set of features collected throughout this guided process used for the final prediction. We have revised the manuscript (Section 3) to make this distinction explicit and more detailed. \n\n\n2. **Scope of Generalization Experiments (Cross-Dataset Evaluation):** We presented results for key representative pairs in Table 3, demonstrating strong performance, particularly when training on smaller datasets (LIVEC) and testing on larger ones (KonIQ), which aligns with common practices in literature [Saha et al., CVPR 2023; Agnolucci, Lorenzo, et al. WACV 2024]. We believe these results, combined with LGDM's core design, provide strong evidence for its generalization capabilities. A key strength of LGDM is that the powerful LDM feature extractor (SDv1.5) remains frozen and is not finetuned for IQA. Only an extremely lightweight regression head ($g_{\\theta}$) is trained on IQA labels for each specific training dataset. This reliance on the general-purpose priors of the foundation model inherently promotes generalization across different datasets and distortion types. Achieving SOTA performance within each of the diverse datasets (Tables 1, 2, 3, and 7) using this approach strongly suggests the robustness and general applicability of the extracted hyperfeatures. Many methods require significant finetuning or adaptation of their entire backbone on large aggregated datasets for good cross-dataset performance. A direct \"train once, test all\" comparison might unfairly disadvantage our approach, which explicitly avoids large-scale backbone tuning to maintain adaptability and leverage zero-shot priors. The strong results in Table 3 already show LGDM generalizes better than competing methods under the standard protocol. Despite this, below, we show an extension of our cross-data validation where models are trained on FLIVE and evaluated on the rest (we include the extended version of this in our updated manuscript).\n\n| Method                    | Test: LIVEC | Test: KonIQ | Test: SPAQ  |\n| :------------------------ | :---------: | :---------: | :---------: |\n| CONTRIQUE |    0.734    |    0.777    |    0.820    |\n| RE-IQA         |    0.690    |    0.796    |    0.825    |\n| ARNIQA    |    0.699    |    0.798    |    0.837    |\n| QCN            |    0.653    |    0.728    |    0.815    |\n| LIQE          |    0.743    |    0.813    |    0.713    |\n| GRepQ       |    0.751    |    0.810    |    0.829    |\n| QualiCLIP+    |    0.725    |  0.817  |  0.841  |\n| LGDM$\\psi_{SDv1.5}$ (Ours) |  0.849  |    0.802    |    0.838    |\n\n\n[Saha et. al., CVPR 2023]. \"Re-iqa: Unsupervised learning for image quality assessment in the wild.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023.\n\n[Agnolucci, Lorenzo, et al. WACV 2024] \"Arniqa: Learning distortion manifold for image quality assessment.\" Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2024.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "I find some parts of the authors' explanation unclear, and my biggest question concerns the input measurement. The paper does not specify how this is calculated at any point, only mentioning that methods like MUSIQ and RE-IQA can be used, which I can understand. However, it is unclear why SD1.5 can also be directly used as a measurement. According to Algorithm 1, the multi-scale features **H** have not yet been fully collected at the stage where the final quality score is supposed to be computed. So, what does $\\psi_{SD_{v1.5}}$ represent? How is it calculated?", "theoretical_claims": "I'm not an expert in this field the I could not fully understand the theoretical claims, I will read the comments of other reviewers and try to understand it.", "experimental_designs_or_analyses": "The experiment is complete.", "relation_to_broader_scientific_literature": "NR-IQA is particularly important in the current era when image generation and related fields are gaining significant popularity. This is because real-world data often lacks ground truth, necessitating evaluation metrics that are more aligned with human perception to better assess the quality of generated images. Similarly, this approach will also be crucial for future advancements in video generation and restoration tasks.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "None.", "other_comments_or_suggestions": "None.", "questions_for_authors": "1. At this stage, assuming that the theoretical derivations and proofs are correct, I believe this paper is highly significant. However, I have some concerns about the generalization experiments. In my understanding, different benchmarks in Table 1 require different score predictors. The results in Table 3 only demonstrate one domain shift, but to establish the method as a truly usable evaluation metric, it is essential to ensure that after training on one domain, the model can achieve SOTA performance across all other domains. Therefore, I hope the authors can supplement this experiment.\n2. I find some parts of the authors' explanation unclear, and my biggest question concerns the input measurement. The paper does not specify how this is calculated at any point, only mentioning that methods like MUSIQ and RE-IQA can be used, which I can understand. However, it is unclear why SD1.5 can also be directly used as a measurement. According to Algorithm 1, the multi-scale features **H** have not yet been fully collected at the stage where the final quality score is supposed to be computed. So, what does $\\psi_{SD_{v1.5}}$ represent? How is it calculated?\n3. For the theoretical derivations and proofs, I will read other reviewers' comments and try to understand them, then make my judgment.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0uYDxGoNJ0", "reviewer": "Reviewer_GzVH", "summary": "This work proposes an NR-IQA model by leveraging latent diffusion models (LDMs). The core idea is that diffusion models inherently learn perceptually consistent regions within their data manifold. The authors introduce Perceptual Manifold Guidance (PMG), a technique that extracts multi-scale and multi-timestep feature maps (diffusion hyperfeatures) from pretrained latent diffusion models to assess image quality. Experiments on multiple datasets demonstrate that pretrained latent diffusion models inherently capture perceptually meaningful features, making them effective for NR-IQA without explicit fine-tuning.\n\n## update after rebuttal\n\nThe additional expriments address my conerns about the choice of SD. I will keep my positive rating.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive feedback and your recognition of our work. We would like to answer your questions and address your concerns below.\n1. **Reliance on Theoretical Assumptions:** We agree with the reviewer that our theoretical proofs rely on assumptions, which are idealizations of real-world scenarios. However, we argue for their practical applicability and justification within our framework. \n\nFor manifold concentration (Assumption 1, Proposition 1), the assumption that high-dimensional data like images lies near a lower-dimensional manifold has been used in a lot of work in machine learning and generative modeling ([Chung et al., 2023]). While global linearity might not hold, data often exhibits significant local low-dimensional structure ([Turk & Pentland, JoCN 1991]), which is relevant as our guidance steps operate locally around the current sample estimate. Furthermore, Proposition 1 (building on [Chung et al. 2023]) discusses probabilistic concentration near the manifold, acknowledging deviations rather than strict adherence, which is more realistic. \n\nPerfect Autoencoder (Proposition 2), while practical VAEs have reconstruction errors, high-quality autoencoders like the one used in SD are trained for perceptual fidelity. That is, a well-trained autoencoder also have similar effects for mapping the guidance to the data manifold ([He, Yutong, et al., 2023]). Most importantly, the strong SOTA performance of LGDM consistently demonstrated throughout the paper serves as robust empirical validation. It suggests that despite being approximations, these assumptions provide a useful theoretical framework and that our PMG mechanism functions effectively with the practical VAE and data distributions encountered.\n\n2. **Dependency on SD and Hyperparameter sensitivity:** We understand the concern about potential sensitivity to the specific SDv1.5 model. We selected SDv1.5 as it is a widely used, powerful, and publicly available baseline, and it empirically yielded the best results among the versions tested (Table 5). We agree that testing on more datasets would strengthen the comparison. However, we chose LIVEC and FLIVE as they represent challenging and diverse authentic distortion benchmarks (largest). The consistent trend observed across these two datasets (v1.5 > v1.4/1.3 > v2.x) suggests the finding hold more broadly. Crucially, Table 5 also shows that LGDM performs very well even with other SD versions (e.g., v1.4 achieves 0.938/0.903 on LIVEC, still outperforming most prior SOTA). This demonstrates that our core contribution – the PMG framework and hyperfeature extraction – is robust and provides significant benefits across different underlying LDM versions, even if v1.5 offers the peak performance. The method's success is not solely hinged on v1.5. LGDM is designed to be modular, the PMG mechanism can be applied to guide feature extraction from other pretrained DMs, leveraging their respective priors. We discuss the impact of $\\zeta_1$ and $\\zeta_2$ in section-4.3. Note that even though, appropriate values of these parameters gives us SOTA performance, the maximum deviation is well under 5\\%, suggesting that sensitivity is limited. \n\n3. **Comparison with Recent NR-IQA Models:** We acknowledge the rapid progress in NR-IQA and agree that comparison with the latest methods is important. We have updated our related work section and comparison tables to include the suggested models (ARNIQA, LIQE, QualiCLIP+, Q-align) and other relevant recent works like DP-IQA, GenZIQA, QCN, and GRepQ. We provide an extended comparison table below:\n\n| Method | LIVEC         | KonIQ         | FLIVE         | SPAQ          |\n| :----------- | :-----------: | :-----------: | :-----------: | :-----------: |\n| LIQE         | 0.866/0.865   | 0.913/0.898   | -             | -             |\n| ARNIQA                        | 0.823/0.797   | 0.883/0.869   | 0.670/0.595   | 0.909/0.904   |\n| QualiCLIP+                    | 0.867/0.821   | 0.898/0.889   | 0.665/0.618   | 0.914/0.911   |\n| Q-Align                       | 0.853/0.860   | 0.941/0.940   | -             | 0.933/0.930   |\n| QCN                           | 0.893/0.875   | 0.945/0.934   | 0.741/0.644   | 0.928/0.923   |\n| GRepQ                         | 0.867/0.859   | 0.916/0.908   | 0.582/0.531   | -             |\n| GenZIQA                       | 0.897/0.873   | 0.932/0.916   | 0.718/0.613   | -             |\n| DP-IQA (Teacher)              | 0.913/0.893   | 0.951/0.942   | 0.683/0.579   | 0.926/0.923   |\n| **LGDM$\\psi_{Sdv1.5}$(Ours)** | **0.940/0.908** | **0.972/0.967** | **0.812/0.705** | **0.948/0.947** |\n\n\n\n[Chung et al., 2023] “Diffusion posterior sampling for general noisy inverse problems”, In Proc. ICLR, 2023. arXiv:2209.14687 (2022).\n\n[He, Yutong, et al., 2023] \"Manifold preserving guided diffusion.\" arXiv preprint arXiv:2311.16424 (2023).\n\n[Turk & Pentland, JoCN 1991] \"Eigenfaces for recognition.\" Journal of cognitive neuroscience 3.1 (1991): 71-86.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "The proposed methods LGDM and PMG make sense because they align well with the core challenges of NR-IQA (lack of reference images, perceptual consistency). The evaluation criteria (benchmark datasets & metrics) are comprehensive, covering a wide range of real-world and AI-generated distortions, ensuring fair and robust assessment.", "theoretical_claims": "The proofs for Propositions 1 and 2 and Theorem 1 are generally correct. However, the proofs depend heavily on assumptions such as the existence of a perfect autoencoder and a noise model that perfectly concentrates on a lower-dimensional manifold. In practice, these conditions are approximations, there may be some gaps when applying them to real-world applications.", "experimental_designs_or_analyses": "The experimental design and analyses are well conducted and align with standard practices in the IQA community.  The primary concern is the choice of SD model. The method is built on Stable Diffusion v1.5, which consistently shows high performance. While Table 5 compares different versions of Stable Diffusion, the overall design might be sensitive to the particular pretrained model used. The conclusions are drawn from the results on LIVE-C and FLIVE, and may change when tested on other datasets.", "relation_to_broader_scientific_literature": "This work extends established ideas from diffusion model research and manifold learning to tackle the specific challenges of NR-IQA.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Overall, the paper stands out for its innovative integration of diffusion models into NR-IQA, its rigorous theoretical underpinnings, and its extensive experimental validation. Addressing the noted weaknesses (reliance on Idealized Assumptions, dependency on a specific pretrained model, hyperparameter sensitivity) could further enhance its impact and ease of adoption in practical scenarios.", "other_comments_or_suggestions": "It's better to compare this method with more recent NR-IQA models published after 2023, e.g., ARNIQA, LIQE, QualiCLIP+, Q-align, etc.", "questions_for_authors": "NO.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HivOn61hoj", "reviewer": "Reviewer_Smqd", "summary": "In this paper, an algorithm named LGDM (latent guidance in diffusion models) is proposed for NR-IQA. It utilizes the pretrained latent diffusion models for sampling process toward perceptually consistent regions on the manifold. Specifically, it extracts diffusion hyperfeatures, which are multi-scale and multi-timestep feature maps, from the denoising U-Net of a pretrained LDM. Then, it aggregates the features to predict the quality score by using a lightweight regression network. Experimental results on various IQA benchmarks show that the proposed algorithm achieves better results than conventional methods.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Your feedback for our work is greatly appreciated. In the following, we will address your questions and concerns in detail.\n1. **Missing References:** Thank you for pointing out the missing recent papers, GRepQ and QCN. We have already included both methods in our updated manuscript. See point-3 in our response to Reviewer GzVH (3).\n\n2. **Comparison with Other LDM-based Methods:** We had included GenzIQA in our AIGC comparison (Table 2). In our updated manuscript, we have included the GenzIQA and DP-IQA Table 1 and discussed them in detail.\n\n| Method | LIVEC (PLCC)  | LIVEC (SRCC)  | KonIQ (PLCC)  | KonIQ (SRCC)  | FLIVE (PLCC)  | FLIVE (SRCC)  | SPAQ (PLCC)   | SPAQ (SRCC)   |\n| :- | :-----------: | :-----------: | :-----------: | :-----------: | :-----------: | :-----------: | :-----------: | :-----------: |\n| GenzIQA  |     0.897     |     0.873     |     0.932     |     0.916     |     0.718     |     0.613     |      -        |      -        |\n| DP-IQA (Teacher)  |     0.913     |     0.893     |     0.951     |     0.942     |     0.683     |     0.579     |     0.926     |     0.923     |\n| **LGDM$\\psi_{SDv1.5}$ (Ours)**        | **0.940**   | **0.908**   | **0.972**   | **0.967**   | **0.812**   | **0.705**   | **0.948**   | **0.947**   |\n\nWe attribute LGDM’s superior performance over methods like GenzIQA to several key technical differences. In terms of feature extraction, LGDM directly extracts diffusion hyperfeatures, which are multi-scale and multi-timestep. This approach captures a rich spectrum of visual information—including texture, structure, and semantic cues across both spatial scales and temporal steps—providing a more robust and nuanced representation than GenzIQA’s reliance on cross-attention maps between image features and learned text prompts that are subsequently pooled. Regarding the guidance mechanism, LGDM employs Perceptual Manifold Guidance (PMG) as described in Equations (9) and (10) to actively steer the DDIM sampling trajectory at each reverse step. This dynamic, step-wise guidance ensures that the intermediate latent states—and therefore the extracted hyperfeatures H—are continuously aligned with both the input image $x$ and its perceptual representation $\\psi_p(x)$. In contrast, GenzIQA depends on extensive prompt tuning and cross-attention weights to adapt the attention mechanism for quality assessment, which lacks the direct, iterative adjustment provided by our PMG. Finally, LGDM leverages a completely frozen pretrained LDM backbone (can and be adapted by any DM) and only trains a lightweight external regression network $g_{\\phi}$. This zero-shot use of the core model enables LGDM to fully exploit the broad, general-purpose priors learned from massive datasets, thereby enhancing both robustness and generalization. GenzIQA, by comparison, requires extensive finetuning of cross-attention weights and prompt tuning, which may risk overfitting and reduce adaptability. In essence, LGDM’s combination of direct multi-step hyperfeatures extraction with active, guided sampling ensures that its features are optimally tailored for perceptual quality assessment, leading to state-of-the-art performance without complex finetuning. We have extended the discussion for Table 1, and comparison with other methods (see our point-3 in response to Reviewer-3), in our updated manuscript.  \n3. **Inference Speed Comparison:** We acknowledge the importance of comparing inference speed. Below, we compare the inference time with LGDM(sampling only with half precision):\n\n| Method | Est. Time (s) | Notes |\n| :----------------- | :------------ | :------------------------------ |\n| QCN | ~0.15 | Geometric ordering |\n| Q-Align | ~0.1 | LORA Finetuning |\n| DP-IQA (Teacher) | ~0.023 | Distilled/Finetuned DM |\n| GenzIQA | ~1.4 | Finetuned DM (8 steps avg) |\n| LGDM (1 step) | ~1.1 | Single Step |\n| LGDM (10 steps)| ~9.7 | SOTA Accuracy |\n\nWhile slower than distilled/finetuned competitors, LGDM achieves higher accuracy. LGDM (10 steps) is indeed slower than highly optimized/distilled methods like DP-IQA and GenzIQA. However. this runtime reflects the cost of our multi-step PMG guidance, which is crucial for achieving SOTA accuracy, despite this our single-step version can provide comparative inference runtime with accuracy trade-off. The speed should be viewed in the context of the zero-shot backbone, which saves enormous computational resources during the training/adaptation phase compared to methods requiring finetuning or distillation. It represents a trade-off, sacrificing some inference speed for significantly higher accuracy and the flexibility of using a frozen foundation model. We have updated the table 3 in our updated manuscript to incorporate the suggested changes. \n\nWe sincerely apologize for the minor formatting errors. We appreciate them being pointed out. We have incorporated suggested changes and thoroughly checked the manuscript to remove such formatting oversight.", "claims_and_evidence": "Yes, this paper provides the propositions and theorem with proper explanation to support their claims. Also, it shows extensive experimental results to support their claims.", "methods_and_evaluation_criteria": "Yes, the proposed algorithm is technically reasonable for me and the evaluation process seems fair.", "theoretical_claims": "Yes, theoretical claims seem to be correct for me.", "experimental_designs_or_analyses": "Yes, the proposed algorithm follows the standard evaluation protocol in this field.", "relation_to_broader_scientific_literature": "The proposed algorithm exploits the pretrained LDM for better NR-IQA. Specifically, it extracts diffusion hyperfeatures from denoising U-Net. Using the pretrained LDMs for better performances is not new for data augmentation, segmentation, and generation. However, for IQA, there are only few methods, which uses LDM, such as GenzIQA. Hence, I think the proposed algorithm provides some contribution to the field as a reasonable and simple approach to use LDMs for NR-IQA.", "essential_references_not_discussed": "Some recent papers are not addressed and compared. It would be better to compare with these algorithms as well. \n[1] Learning generalizable perceptual representations for data-efficient no-reference image quality assessment. WACV24\n[2] Blind image quality assessment based on geometric order learning. CVPR24", "other_strengths_and_weaknesses": "The proposed algorithm shows good results on various IQA datasets. However, since the proposed algorithm exploits the pretrained LDM, which is trained on large scale datasets, the proposed algorithm has some advantages compared to conventional IQA methods. Therefore, I believe it should be compared more extensively with IQA methods using LDMs. At least, it would be better to provide a more detailed explanation of Table 2—for example, how the proposed algorithm fundamentally differs from other methods and why it performs better in Table 2.\n\nAlso, even though Table 4 provides the time test, it would be better to compare the inference speed with other algorithms.", "other_comments_or_suggestions": "- L129: D -> \\mathbb D\n- eq (3): missing ], missing )\n- L146 (right side): x0 -> x_0\n- L373-374: editing comment?\n- \"\": incorrectly oriented quotation marks. It frequently appears in the paper.", "questions_for_authors": "In overall, I think this paper has some technical contribution for this field. However, the paper does not seem to be fully revised yet. It still contains many typos and some editing comments that should have been removed. Also, I have some questions listed in other strengths and weaknesses section. I would like to see the response from authors. Then, I will decide my final rating.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "N/A"}}]

Author rebuttal:

Meta-review: This paper proposes to utilize pre-trained latent diffusion models to obtain perceptually consistent features for NR-IQA. The paper originally received 2xWeakReject and 2xWeakAccept. The main concerns include hyperparameter sensitivity, time complexity, scope of generalization experiments, reliance on theoretical assumptions, insufficient evaluations, etc. The authors have provided rebuttals and addressed most concerns of reviewers. Afterward, two reviewers raise the rating and all reviewers lean to accept this paper. The authors are suggested to carefully revise the paper and incorporate newly conducted experiments according to the comments and discussions.
Decision: Accept (poster)