Semi-Supervised Blind Quality Assessment with Confidence-quantifiable Pseudo-label Learning for Authentic Images
Authors: Yan Zhong
Chenxi Yang
Suyuan Zhao
Tingting Jiang
Venue: ICML 2025 poster (2025)
Keywords: BIQA
Semi-supervised Learning
Pseudo-labels
Confidence Learning
Real-world Distorted Images
Primary area: applications->computer_vision
TL;DR: This paper introduces a novel semi-supervised framework for blind image quality assessment based on label propagation, utilizing confidence-quantifiable pseudo-labels to enhance performance.

Abstract: This paper presents CPL-IQA, a novel semi-supervised blind image quality assessment (BIQA) framework for authentic distortion scenarios. To address the challenge of limited labeled data in IQA area, our approach leverages confidence-quantifiable pseudo-label learning to effectively utilize unlabeled authentically distorted images. The framework operates through a preprocessing stage and two training phases: first converting MOS labels to vector labels via entropy minimization, followed by an iterative process that alternates between model training and label optimization. The key innovations of CPL-IQA include a manifold assumption-based label optimization strategy and a confidence learning method for pseudo-labels, which enhance reliability and mitigate outlier effects. Experimental results demonstrate the framework's superior performance on real-world distorted image datasets, offering a more standardized semi-supervised learning paradigm without requiring additional supervision or network complexity.

Semi-Supervised Blind Quality Assessment with Confidence-quantifilll Pseudo-label Learning for Authentic Images
Yan Zhong 12
Chenxi Yang 12
Suyuan Zhao
Tingting Jiang" 24 6a
Abstract
This paper presents CPL-IQA, a novel semi- supervised blind image quality assessment (BIQA) framework for authentic distortion narios. To address the challenge of limited la- beled data in IQA area, our approach leverages seaiiddenne-quaniiiibbbll pseudo-label learning to effectively utilize unlabeled authentically dis- torted images. The framework operates through a preprocessing stage and two training phases: first converting MOS labels vector labels via en- tropy minimization, followed by an iterative pro- cess that alternates between model training and label optimization. The key innovations of CPL- IQA include a manifold assumption-basee based label optimization strategy and a confidence learning method for pseudo-labels which enhance reliabil- ity and mitigate outlier effects. Experimental re- sults demonstrate the framework's 's superior perfor- mance real-world distorted image datasets, of- fering a more standardized semi- cemi-supervised learn- ing paradigm without requiring additional super- vision or network complexity.
Homn Faing SUDTIS
H1ame Tarin
(A3
(1)
MDS 2:95 uSE8.5
103
100
Figure I. Real-world images with similar MOS values may differ greatly in score distribution and standard deviation (std), such as images (A) and (C), as well as images (B) and (D). These four images all taken from the authentically distorted image
database KonIQ-1OK (Hosu et al., 2020).
of the processed image. Therefore, plentiful image qual- ity assessment (IQA) methods have emerged. According to the amount of available reference images, existing ob- jective IQA methods can be categorized as Full-Reference approaches(FR IQA), Reduced--Reference approaches(RRR- IQA) and No-Reference approaches (NR-IQA) (Zhai & Min, 2020). Since it is difficult to obtain the undistorted version of authentically distorted images in practical applications, growing attention has been given to NR-IQA (also termed as Blind IQA, i.e. BIQA) (Su et al., 2020), which can be applied in various domains, such as image denoising (Tian et al., 2020), restoration (Li et al., 2023) and generation (Elasri et al., 2022). this work, we focus on BIQA methods for authentically distorted images. Traditional BIQA methods aim to predict the image quality scores (i.e. Mean Opin- ion Scores, MOS) by manually extracting features from distorted images (Liu et al., 2020; Zhou et al., 2017: Jiang et al., 2017). While effective for synthesized distortions, they struggle with authentic distortions. Therefore, there have sprung up numerous studies about Deep Learning (DL) basedd BIQA methods for authentically distorted images, due its strong ability in learning and the success of fusing discriminative features in visual domains (Fang et al., 2020; Su et al., 2020; Talebi & Milanfar, 2018; Ying et al., 2020). However, the training process of these methods requires a large amount of data to prevent overfitting, while there exists no large database of authentically distorted images nowadays since the annotation process is expensive and
1. Introduction
Recent years have witnessed the proliferation of rorldworld image datasets, which is the cornerstone of computer vi- sion research and application (Yue al., 2022). However, there may occur distortions during the process of image acquisition, transmission, and processing (Prabhakaran & Swamy, 2023), leading to degradation in the visual quality
Sccooo of Mathematical Seiences, Peking University, Bei- jing, China SState Key Laboratory of Multimedia Information Processing, School Computer Science, Peking University, Bei- jing, China 'Department of Computer Science and Technology, Tsinghua University, Beijing, China "Naionna Biomedical Imag- ing Center, Peking University, Beijing, China. Correspondence to: Yan Zhong <zhongyan@e stu, wtw.puu.edu.nn.. Tingting Jiang <ttjiang @ pku.edu.cn>>.
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Semi-Supervised Blind Quality Assessment with Confidencc-quantffable Pscudo-label Learning for Authentic Images
time-consuming (Yue al., 2022; Prabhakaran & Swamy, 2023; Zhang et al., 2022c).
This fact motivates us to convert MOS label to the vector label that can reasonably simulate the MOS distribution for each sample. To achieve this, we propose preprocessing procedure named Label Conversion in CPL-IQA, which is implemented based on entropy minimization.
To solve this problem in authentically distorted scenar- ios, some researchers try to explore unsupervised or semi- supervised BIQA methods (Prabhakaran & Swamy, 2023; Yue et al., 2022: Saha et al., 2023), which can utilize unlabeled images to boost the quality prediction perfor- mance. For unsupervised strategy, the intuitive idea is to train the BIQA model on large synthetically distorted image databases first based on Contrastive Learning (Madhusu- dana et al., 2022; Saha et al., 2023), then the pre-trained model is fine-tuned on specific authentically distorted im- age database (Prabhakaran & Swamy, 2023). Although this approach performs well in synthetically distorted scenarios, the performances on authentic images are not satisfactory. For semi remi-supervised strategy, some works try to train BIQA networks using both labeled and unlabeled samples based on knowledge distillation ( (Yue et al., 2022), which requires the score distribution of each image. However, only MOS labels are available in most authentically distorted image datasets (such as SPAQ (Fang et al., 2020)), while images variety of score distributions (as shown in Figure 1). And the latest semi-supervised BIQA methods SSLIQA (Yue et al., 2022) and SS-IQA (Pan et al., 2024) both require additional net- work branches and extra data during training, leading to higher training costs. In general, the applicability of exist- ing DL-based semi-supervised BIQA methods is limited due to the rigorous training requirements and conditions.
For Goal (2), firstly, we predict the pseudo-labels of unla- beled images following the process of label propagation. Then, in order to eliminate the effects of some inaccurate pseudo-labels, we propose
entropy-based confidence learning method to distinguish the quality of these predicted pseudo-labels, which can effectively improve the perfor- mance of CPL-IQA. It is worth noting that no auxiliary datasets and multiple network branches are required during the training process of CPI.-IQA. The main contributions are summarized as follows:
In this paper, we propose a novel and high-applicabillty semi-supervised BIQA method named CPL.IIQA for authentic images according to learning the proper pseudo-labels for unlabeled images based on label propagation.
with similar MOS values may correspond
We propose an effective strategy of Label Conversion to address the obstacle to applying label propagation effectively in the BIQA field, which is implemented based on the technique of entropy minimization.
We propose an effective confidence learning method, which can eliminate outliers of pseudo-labels and en- hance the generalization ability of CPL-IQA.
Therefore, we propose a novel semi-supervised BIQA frame- work named CPL-IQA' based on label propagation (LP), which can be effectively trained end-to-end only on a sin- gle branch network without extra inputs. The idea of IP is to first construct a nearest neighbor graph in the feature space based on the manifold hypothesis (Zhou et al., 2003), then the pseudo-labels of unlabeled samples are obtained by the neighbor graph and labels of limited labeled samples. However, there exists a severe obstacle to applying LP effec- tively in the BIQA field: The approach of label propagation is skilled in handling multi-label (vector-label) data (Zhong et al., 2021), whereas MOS label in IQA datasets is a scalar. Therefore, to train BIQA model using labeled and unlabeled data simultaneously based on LP, CPL-IQA is required to achieve two goals: (1) Reasonably convert the scalar MOS labels of labeled samples into vector labels to meet the re- quirement of I.. (2) Effectively predict the pseudo-labels of unlabeled images and corresponding confidence levels, which will be fed into the BIQA model for training.
Extensive experiments on authentically distorted image databases are conducted to validate the applicability and effectiveness of the proposed method.
2. Related Works
2.1. Blind Image Quality Assessment (BIQA)
Traditional BIQA methods focused on manually construct- ing statistical features and predicting MOS labels of images by linear mapping (Liu et al., 2018; Wang et al., 2002: Saad et al., 2012; Moorthy & Bovik, 2010). With the de- velopment and wide applications of deep learning (DL), DL-based BIQA methods have become the mainstream di- rection of BIQA research, while these BIQA methods are "data hungry". Therefore, some unsupervised BIQA meth- ods have sprung up (Madhusudana et al., 2022; Prabhakaran & Swamy. 2023; Saha et al., 2023; Zhao al., 2023). which are designed based on Contrastive Learning (Le-Khac et al., 2020), and more complex BIQA network structures are proposed to improve the generalization of BIQA (Liu et al., 2017; Lin et al., 2020; Ma et al., 2017b; Zhou et al., 2022). However, these methods are trained on syntheti- cally distorted databases, leading to limited performance on images with authentic distortions. More recently, some
For Goal (1), according to NIMA (Talebi & Milanfar, 2018), training BIQA models with the MOS distribution of each image outperforms training with the scalar label of MOS.
'CLL short of confidence-uuaniffabble pseudo-label learning.
2

---

Semi-Supervised Blind Quality Assessment with Confidence-quantffabbe Pscudo-label Learning for Authentic Images
The GT MDS labels labeled Images
Pacudo unlabeled images
The vector labels of labeled images
Unlabeled
Labeled images
Festure extmetor 4o
Full conmected predictor foz
images
GT MOS labele
Backéoone
(1.)
Pecude-MO labeb
Fearure daa
MD5+1
Step 2: model training
LFF Noce labell
Vecter lsbuls
Nearest neighhor graph constroction
Full connected Prodictor fe,
Iterated atternately
MOS 1005-100
Eq/12)
Label Optinnizing with confidence
Label Convension
Vector Schils
Step 1: pseudo label learning
Images with pseudiolabbll
Preprocessing Procedure
Stage
Stage
Figure 2. Overview CPL-IQA. The overall training process is described in Section 3.2,
works (Zhang et al., 2024; Zhong et al., 2024a; Wang et al., 2023; Yang et al., 2024; Zhang et al., 2022a;b) try to improve the generalization of BIQA models by combining BIQA net- works with specific learning paradigm, such as Continual Learning, Causal Learning, Curriculum Learning, Adver- sarial Learning and Semi-supervised Learning. However, these methods require much higher training costs, including the latest semi- supervipee BIQA methods SSLIQA (Yue et al., 2022) and SS-IQA (Pan et al., 2024), which both require training with multiple network branches and addi- tional datasets. Therefore, the motivation of this paper is to design semi-supervised BIQA method that can be effec- tively trained end-to-end only on single branch network without extra inputs in authentically distorted scenarios. See Appendix B.1 for more reviews.
3. Methods
We first outline the preliminaries and the overview of CPL- IQA, followed by detailed introductions of each module.
3.1. Preliminaries
We assume X := (I1,..., 2I41...,,....................... the authen- tically distorted image dataset in the semi-supervised BIQA domain, where the first / images are labeled and the remain- ing - I are unlabeled images. Y, == (V1,-.., :.) are the MOS labels of the first / samples. The goal of our semi- supervised BIQA method is to train an effective MOS predic- tion model with unlabeled images X == (.r Inn,, labeled images Xy == (F1,..., 21) and their labels Yz.
3.2. Overview of CPL-IQQA
2.2. Deep Semi-supervised Learning (D-SSL)
By designing different unsupervised losses for training with unlabeled samples, existing D-SSL methods can effectively deal with the scenarios of data scarcity in practical applica- tions. One type of representative D-SSL method is known as Transductive Semi-Supervised Learning (TSSL) (Wang et al., 2016; Shi et al., 2018). Another type is designed based on label propagation (Iscen et al., 2019; Douze et al., 2018: Haeusser et al., 2017). More recently, some active learning- based supe-supe methods have been proposed (Fan al., 2024). Regrettably, these approaches are only suit- able for classification problems and cannot be implemented in regression tasks such as IQA. Therefore, some BIQA works (Prabhakaran & Swamy, 2023; Yue et al., 2022) try to construct the unsupervised contrastive loss that can be trained with unlabeled samples, which inevitably require additional training costs with lower applicability. Differ- from these unsupervised methods, our proposed BIQA method can achieve both high applicability and low com- plexity with a more standard semi-supervised paradigm. For a detailed literature review, please refer to Appendix B.2.
As shown in Figure 2, the basic idea of our approach is learn- ing the manfidence-uuantifaate pseudo-labels for unlabeled samples Xy deal with the insufficient label problem. The whole framework consists of a preprocessing procedure named Label Conversion and two training stages. Stage 2 includes two steps: Model Training and Label Optimizing, which are iterated alternately until convergence.
In the Label Conversion procedure before training, due to the superior values of score distributions of images in training, transform MOS labels into vector representa- tions by entropy minimization, which can be regarded the simulation of score distributions. In Stage 1, the feature extractor Φe, and label predictor fe, are trained together through labeled images XL, YL and Eq. 10, ResNet101 (He et al., 2016) chosen as the backbone. In the Label Op- timizing step of Stage 2, the features of all the samples X can be obtained according to the current feature extrac- tor de, Then nearest neighbor graph structure is con- structed in the feature space based on the manifold assump- tion (Zhou et al., 2003), through which the latent semantic
3

---

Semi-Supervised Blind Quality Assessment with Confidence-quantffable Pscudo-label Learning for Authentic Images
representations of these samples can be created to learn the proper pseudo-labels Yu for unlabeled images. In the Model Training step of Stage 2, the feature extractor Φ8, and predictor fe, are further trained with Xz. Xu and the predicted pseudo-labels Ve. In order to improve the utilization of pseudo-labels, we allot a confidence weight for each pseudo-label during this training step in Stage 2.
score set M, so each element of the vector is positive and the sum of that is 1.
Since the distribution of MOS labels is unavailable in many authentically distorted image databases, we can regard the entropy of e as the degree of uncertainty of vector labels. Therefore, the MOS labels of labeled samples are converted to vector labels by entropy minimization.
3.3. Modules
arg min H(O:), H(p) = - --Er ps log Pi, 1e.=11qfo,
02
In this section, we will introduce the details of each module about our proposed CPL-IQA method, including modules of Label Conversion, Nearest Neighbor Graph Construction, Label Optimizing, Confidence of Labels, Loss Functions, Alternate Iterative Training, Analysis and Discussion about the proposed method CPL-IQA.
(2)
where 1 1= E E q= 100]" E ER 100x1 V; is the label vector converted from the MOS label 8; of the i-th sample, with all elements being positive. We de- note all the converted labels as V2 [11 vi] E 1000 (p) is information entropy (Greven et al., 2014) for an arbitrary random vector p. also applying to the label vector v;- According to Eq. 2, converting the GT MOS value to a vector representation is a one-to-one problem without any prior information. For example, an MOS value 2.4 and can uniquely be converted to vector label [0; 0.6; 0.4; O;..; 0; 0).
3.3.1. 1. LABEL CONVERSION.
The motivations of this module lie in the two main aspects: (1) The core idea of our method is to learn the pseudo- label and corresponding confidence level for each unlabeled training sample based on label propagation, which is par- ticularly adept at handling multi-label or vector-label data. (2) According to NIMA (Talebi & Milanfar, 2018), train- ing with the vector label representing the MOS distribution has demonstrated more advanced effectiveness compared to training with the scalar label of MOS. Therefore, we as- sume that the confidence levels of labels in labeled samples are high enough, and the MOS labels of labeled samples converted into vector representations of simulated MOS distribution labels for each sample by entropy minimization. Although we admit that the conversion of MOS labels to vec- tor labels of MOS distributions could be a one-to-many map- ping without any prior information, entropy minimization is reasonable since it results in a more concentrated simulated label distribution, suggesting lower variance and higher con- fidence, which fits well with the above assumption of high confidence level, leading to one-to-one mapping.
3.3.2. NEAREST NEIGHBOR GRAPH CONSTRUCTION.
In Stage 2 of CPL-IQA, features are extracted according to the current parameters 0,, including the parameters of the backbone and the following Fully Connected (FC) layer. The role of FC is to realize the low dimensional repre- sentation of extracted feature information and reduce the computation complexity of graph construction, inspired by low-rank learning (Hu et al., 2021). In our settings, the backbone is ResNet101 (He et al., 2016) and the subse- quent FC layer maps the learned 2048-dimensional features extracted by ResNet101 into 256-dimensional features, de- noted as Q [gt, 02...........................0.. 9m] E dg (d = 256). We use the ANN(Belkin & Niyogi, 2003) to construct the weight matrix G, so sample point gi is linked to qj if and only if 4 is the nearest neighbor of qj in kNN graphs. &NN graph G is defined as:
Concretely speaking, in order to press close to the artificially subjective BIQA method (Fang et al., 2020), we uniformly consider MOS labels as the average result of score sets M 2,, 3, ..., 99, 1003. hence the value range of the specified MOS is the closed interval [1, 100]. For authen- tically distorted image databases with different ranges of MOS labels, we firstly normalize the MOS range to [1, 100] by Max-min Normalization:
Gig = { exp (- 11 (1ll-g))l()) if qi is linked to fj (3)
otherwise
where a is the prree-parammter.
Note that qj is not necessarily one of the & nearest neighbors of qi when qi is that of g, Therefore, for the sake of the sub- construct the normalized symmetrical
sequent learning, graph matrix Ö in our method:
min( (z)
8; = 1+ max (z) min (z)
x 99,
(1)
Ō= "G+ /2, G-D-112GD-12 (4)
where z = [21, 22, ..., J. and 8; is the normalized MOS label of the i-th sample. Then, the MOS value is replaced by a 100-dimensional vector v, where each element can be viewed as a proportion of the corresponding score in the
The D in Eq. 4 is a diagonal matrix, where the element in the i-th row and i-th column the sum of the i-th row of G.
4

---

Semi-Supervised Blind Quality Assessment with Confidencc-quantffable Pseudo-label Learning for Authentic Images
3.3.3. LABEL OPTIMIZING.
The setting of Eq. 9 echoes the operation of Label Conver- sion for labeled samples.
Following the format of the converted vector labels, we initialize the vector labels of the unlabeled images the zero vectors with 100 dimensions, denoted as Vu [M==1: 11:2; ...; vn]. According to the manifold hypothe- sis (Zhou et al., 2003), the latent semantic representations P E RTx100 of all samples can be learned through the following iterative process, where t denotes the number of iterations and all elements of P(O) are initialized zero.
3.3.5. Loss FUNCTIONS.
As shown in Figure 2, our proposed CPL-IQA method in- cludes two training stages.
Stage of CPL.IIQA a supervised learning process trained with labeled images Xr, and their labels Yz, the loss func- tion of which can be formulated as:
P(t 1) =(((() + (1 - V,V,
(5)
Lu (Xr, Yr, 02) = Σ loss, (foz 1dn, (M.)))
in which V=[Vi; Vo] (V2 is defined after Eq. 2), and is a hyper- parrmm-rreettrr between 0 and 1, which is set as 0.99 in our method. To save computational costs, we can obtain pa (i.e. the convergent P) via the following assertion:
(10)
where EM(() denotes the operation of Label Conversion with entropy minimization, ó and f are feature extractor and label predictor with parameters @1 and 8g respectively, and loss, represents the EMD (earth mover's distance) loss (Hou et al., 2016) with = 2.
Assertion 3.1. Define sequence (P(() as Eq. 5 with & obtained by Eq. 4, then {P(t)} converges to pa in Eq. 6:
Pr lim P(t) (1 ( (1-->)(->0>----
(6)
Stage 2 of CPL-IQA is trained with both labeled samples XL. YL and unlabeled samples X,, and the loss function is:
See Appendix A.I for the proof. Finally, we can normalize P and obtain the predicted pseudo-labels Yg:
L (x,YL, Vu: 02) = Σ
nr loss2 (Fo, 10o, (z:)), ve)
P /11P w= = PT.'.. Yu = PoW, (7)
i=1
where P represents the pseudo-inverse of Pt. since it is not a square matrix. The intuition behind Eq. 7 is follows: After computing P and P [Pr: Pul. we have obtained the latent semantic representations for both labeled and unlabeled samples within the same feature space. Then, labels Y can be predicted using a weight matrix W as Y = PW. Since the ground-truth labels Yú for labeled samples are known, we can calculate W from Yz and Pz, and then use W to predict the labels Yu for unlabeled samples. Based on Eq. 7, our label optimizing method can handle regression tasks such as IQA. While traditional label propagation is only suitable for classification tasks (Iscen et al., 2019).
+
Σ 2=I+1
7y loss2 (foz (do, (z))
(11)
where Ye == (0i-1...., (n) denote the learned pseudo- labels for unlabeled images Xy in the Label Optimizing step of Stage 2. And lossı represents loss function designed based on L2 loss (MAE loss).
3.3.6. ALTERNATE ITERATIVE TRAINING.
After Label Optimizing in Eq. 7 and confidence learning in Eq. 8, we can further train the feature extractor Φu, and FC predictor for with Eq. 11. Since the predicted results of fe, (with normalization) are m-dimensional vector labels, the learned pseudo-labels are real numbers range in interval [1, 100). Therefore, loss in Eq. 11 can be measured by:
3.3.4. CONFIDENCE OF LABELS.
According to the normalized P in Eq. 7, we can use infor- mation entropy (-) estimate the confidence 17 of the pseudo-labels learned in the module of Label Optimizing, then the impact of pseudo-labels with high uncertainty can be reduced for model training. For the j-th unlabeled sam- ple, the confidence ?) can be defined as:
loss (dk, ykigg) L1 (d2 * * 9K) (k= I,..., n),, (12) where de = normalize (fe, [0,, (æx)]} and g [1, 1,,,...,m m] is the vector of label levels, m = 100 in our method. And L1(-) denotes the L1 loss (MAE loss).
7 = 1 H (P:) log(m), (j = 1+ n),.,,),
Therefore, the process of alternate training in Stage 2 can be iterated through the two steps: Step 1: Label Optimiz- ing. According to image features extracted by the latest updated model, a new LNN graph is constructed by Eqs. 3-4. Then the lacudo-labels of unlabeled samples are learned by Eqs. 6-7. Step 2: Model Training. After calculating the confidence of pseudo-labed labels by Eq. 8, the current model is further trained with Eqs. 1 11-12.
where m = |M is the size of set M. Thus, Ni ranges be- tween 0 and 1, with higher ny indicating greater confidence. On the other hand, we assign high confidence to the ground truth MOS labels of labeled samples, that is:
m = 1, (i = 1,....1 1).
5

---

Semi-Supervised Blind Quality Assessment with Confidence-quantffabl Pscudo-label Learning for Authentic Images
The pseudo-code of CPL-IQA is summarized in Algorithm in Appendix C, where line 1, lines 1-1. lines 1-1 and lines 1- I are the processes of Label Conversion, Stage 1, Step I and Step 2 of Stage 2. respectively.
Table 1. Performance comparison KonlQ-10K with the propor- tion 1:3:1 of samples division. The best results are highlighted in bold (same the later tables).
Methods
PLCC
SRCC
NIQE
0.300
0.276
3.3.7. ANALYSIS AND DISCUSSION.
Traditional BIQA
BRISQUE
0,581
0.541
Supervised
GWH--LBP
0,581
0.541
In the process of label propagation (Zhou et al., 2003), fea- tures are normalized to construct the manifold structure of samples (Iscen et al., 2019) by cosine similarity, which is not suitable for regression and IQA tasks, since this kind of manifold structure may wreak havoc the distribution of regression labels (see Appendix E.3 for details). Instead of cosine similarity, we construct the nearest neighbor graph by the original extracted feature information, which is more compatible with IQA tasks. An interesting finding is that the limiting value Eq. 6 in the iterative process of Eq. 5 can be regarded as the optimal solution of specific regularization framework, which describes the process of label propaga- tion from the perspective of optimization. Here we have the following assertion.
SSEQ
0.326
0.303
CNNIQA
0.654
0.635
WaDIQaM
0.665
0.644
PAQ-2-PIQ
0.728
0.718
NSSADNN
0.665
0,549
Supervised
GraphIQA
0,862
0.845
DL-based BIQA
MB-CNN
0,609
0.600
MUSIQ
0.864
0.838
Causal-IQA
0.853
0.837
CONTRIQUE
0.768
0.775
Unsupervised
Re-IQA
0.782
0.804
SSLIQA
0.867
0.841
SS-1QA
0.854
0.829
Semi- Supervsiee
Semi-IQA
0.864
0.839
Ours
0.873
0.845
cient (SRCC), Kendall Rank-order Correlation Coefficient (KRCC), and Root Mean Squared Error (RMSE).
Assertion 3.2. In the iterative process of Eq. 5, the limiting value Eq. 6 can be regarded as the optimal solution of the regularization framework Eq. 13 (with P > O in Eq. 14).
1.2. IMPLEMENTATION DETAILS
D = arg min H(P), PEP
(13)
In the CPL-IQA, we choose ResNet101 as the backbone for compelling comparison with SSL-IQA (Yue et al.. 2022). which uses ResNet101 (He al., 2016) as one branch. In La- bel Conversion, we take score set M 2, 3, ..., 99, 100} and m = 100 is the cardinality of M. k is set to 10 and σ = 500 in the Nearest Neighbor Graph Construction with ANN, and the graph G in Eq. 3 is computed with the FAISS library (Johnson et al., 2019). We let 0.99 in the pro- cess of Label Optimizing. The dimension d of features extracted by the FC layer after the backbone is 256, which can improve computing efficiency in graph construction.
H(P)=
Σ Gi 1,11 VDa
P.
1
P3
H
P. ViII
VDi
(14)
See Appendix A.2 for proof. The first and second terms of Eq. 14 can be regarded as the smoothness constraint and fitting constraint (Zhou al., 2003), respectively.
4. Experiments
Our CPL-IQA is trained with Pytorch library on two Intel Xeon E5-2609 v4 CPUs and four NVIDIA RTX 2080Ti GPUs. The batch size B = 64 in Stage 1, and Stage is performed with B B₁ + By, where B, = and By = 56 denote the number of labeled and unlabeled images in one batch. The training is conducted for just 10 epochs in total with SGD optimization, including 5 epochs in Stage and 5 epochs in Stage 2. Meanwhile, we resize all the images into 256 x 256 and randomly crop 10 sub-images to the size of 224 x 224, and we initialize the backbone by the pre-training weights obtained by the classification task on ImageNet (Deng et al., 2009) before training in Stage 1.
In this section, we conduct experiments on authentically distorted images to validate the superiority of CPL-IQA.
4.1. Experimental Settings
4.1.1. DATASETS AND EVALUATION METRICS
We perform the main experiments on four representative authentically distorted image databases, including KonIQ- 10K (Hosu et al., 2020). LIVE-C (Ghadiyaram & Bovik, 2015), NNID (Xiang et al., 2019) and SPAQ (Fang et al., 2020). More details of these datasets are shown in Appendix D. In addition to the main experiments, more databases in Table 9 are used in more extra experiments in Appendix F, including BID (Ciancio et al., 2010) and KADID-IOK (Lin et al., 2019). We evaluate BIQA models by four typi- cal metrics, including Pearson Linear Correlation Coeffi- cient (PI.CC), Spearman Rank-order Correlation Coeffi-
4.2. Performance Comparison
1.2. MAIN RESULTS
We divide the image set of KonlQ-10K by 1:3:1, which corresponds to the ratio of the number of training im-
6

---

Semi-Supervised Blind Quality Assessment with Confidenceqqanniffabe Pscudo-label Learning for Authentic Images
Table 2. The test results of cross-data experiments conducted on LIVE-C and NNID. All the CNN-based methods are trained with 20% labeled images and 60% unlabeled images sampling from KonIQ-1OK.
Table 3. Impacts of Label Conversion, the weight of Labell Confi dence and Label Optimizing in Stage 2.
KonlQ-10K
SPAQ (1:8:1)
SPAQ
Methods
Metries
(1:3:1)
(2:7:1)
LIVE-C
NNID
Methods
PLCC
SRCC
PLOC
SRCC
PLCC
0.713
0.812
0.823
SRCC
0.719
0,813
0.818
CNNIQA
0.513
0.485
0.597
0.584
Label-only
KRCC
0.502
0.589
0.616
WaDIQaM
0.538
0.535
0.704
0.702
PAQ-2-PIQ
0.528
0.506
0.715
0.712
RMSE
9.979
13.267
11.665
NSSADNN
0.439
0.426
0.733
0.731
Supervised
GruphIQA
0.619
0.591
0.728
0.727
PLCC
0.850
0.873
0.888
MB-CNN
0,481
0.459
0.553
0.548
Ours (Stage 1)
SRCC
0.822
0,869
0.887
MUSIQ
0,655
0.672
0,701
0.723
KRCC
0.619
0.672
0.686
Causal-IQu
0.742
0.705
0.748
0.767
RMSE
7.288
10.284
9.466
Unsupervised
CONTRIQUE
Re-IQA
0.598
0,612
0.593
0.612
0.584
0.624
0.770
0.679
Ours
PLCC
0.862
0.881
0.898
SRCC
0.829
0,883
0.897
SSLIQA
0.706
0.695
0.771
0.770
(Full)
KRCC
0.630
0.689
0.699
Semi-Supervised
Semi-IQA
SS-IQA
0.731
0,718
0.705
0.684
0.723
0.743
0.759
0.735
(weights x)
RMSE
7.068
10.009
8.953
Ours
0.777
0,721
0.772
0,773
PLCC
0.873
0.896
0.903
Ours
SRCC
0.845
0.893
0.902
(Full)
ages with labels, training images without labels and test images, respectively, to conduct the comparative experi- ments to analyze the effectiveness of CPL-IQA. Sixteen advanced BIQA methods are used to compare with CPL- IQA, which consist of four traditional BIQA methods (in- cluding NIQE (Mittal et al., 2012b), BRISQUE (Mittal et al., 2012a), GWH-GLBP (Li et al., 2016) and SSEQ (Liu et al., 2014)) and twelve DL-e BIQA methods (includ- ing PAQ-2-PIQ (Ying et al., 2020), CNNIQA (Kang al., 2014), WaDIQaM (Bosse et al., 2017), NSSADNN (Yan et al., 2019), GraphIQA (Sun et al., 2023a), MB-CNN (Pan et al., 2022), CONTRIQUE (Madhusudana et al., 2022),
(wcights
KRCC
0.652
0.703
0.709
RMSE
6.728
9.220
8.911
models trained on KonIQ- (sample proportion is 1:3:1)
on two unseen authentically distorted image databases: LIVE-C and NNID. To echo models trained on KonIQ- 10K, the MOS labels of images in LIVE-C and NNID are normalized to 1-100 by Eq. and these images are randomly cropped to the size of 224 x 224 before testing. And the results are displayed in Table 2, which shows that CPL-IQA can perform stably unseen databases with one-up gener- alization ability. Peculiarly, the PLCC value of CPL-IQA on LIVE-C is more than 10% of the second place, which indicates its advantage in generalization for the authentically distorted IQA tasks.
MUSIQ (Ke et al., 2021), Causal-IQA (Zhong et al., 2024a),
Re-IQA (Saha et al., 2023), SSLIQA (Yue et al., 2022), SS-IQA (Pan et al., 2024)), and Semi-IQA (Li et al., 2024). Semi-IQA, SS-IQA and Re-IQA are the SOTA semi- supervised BIQA methods, CONTRIQUE and Re-IQA are the SOTA unsupervised BIQA methods2. Other supervised methods are trained with the corresponding 20% labeled images for the sake of fairness. Finally, the comparison results between CPL-IQA and other methods are displayed in Table 1. We can observe that, compared with Das-based methods, the general performances of traditional methods are fairly unsatisfactory. And our CPL-IQA has achieved better results than other competing methods. Besides, CPL- IQA with only one branch performs better than SSLIQA and SS-IQA (consisting of two network branches), which illustrates the effectiveness of CPL-IQA.
4.3. Ablation Study
We investigate the impacts of different components of CPL- IQA. Firstly, we record the impact of each stage on the final results on KonIQ-10k and SPAQ with different split ratios, which are shown in Table 3. The Label-Onll (Fang et al., 2020) method is trained by ResNet101 and L, loss only on labeled samples, and Ours (Stage 1) and Ours (Full) denote the training results of CPL-IQA in Stage I and Stage 2, respectively. The "weights x" denotes the confidence of all samples set as in Stage 2, and "weights v means confidence is calculated by Eq. 8. Looking down from the top in Table 3, the performances are gradually improving, illustrating the effectiveness of Label Conversion, the weight of Label Confidence, and Label Optimizing in Stage 2.
4.2.2. CROSS-DATA EXPERIMENTS
In addition, we further perform a set of cross-data exper- iments examine the generalization ability of CPL-IQA. Specifically, we directly test above mentioned CNN-based
In addition, we study the impact of cardinal number of the score set M mentioned in Section 3.3 and the split ratio of image samples, the results of which are shown in Table 4 and Table 5. Looking from left to right, we can observe that the larger the ratio of labeled training samples, the better
Unsupervised model is firstly trained on unlabeled training set, then fine-tuned on a labeled training set from the same dataset.
7

---

Semi-Supervised Blind Quality Assessment with Confidence-quantffabbe Pscudo-label Learning for Authentic Images
Table 4. Impact of the cardinality m of score M.
0.875
0.810
KonlQ-10K(1:3:))
SPAQ(1:8:1) 1)
00,005
m
10
20
100
10
20
100
0.800
5.6330
0.5275
PLCC
SRCC
0,850
0.821
0.837
0.872
0.873
0.841
0.884
0.880
0.888
0.892
0.896
0.893
0.550
0.55
Es (1 hrsubb-00
0.8225
ledre Sea.doiiaa
(1//hcbilde)
Ezochs
Epocts
KRCC
0.627
0.644
0.652
0.686
0.697
0.703
RMSE
7.081
6.788
6.728
9.673
9.345
9.220
PLCC
(b) SRCC
Table 5. Impact of the split ratio datasets (fixed m = 20).
0.555
0.850
7.2-
MetPis Froudd Adbels
m=
KonIQ-10K
SPAQ
0.45
0.640
Ratio
1:7:2
2:6:2
3:5:2
1:8:1
2:7:1
3:6:1
8 0.33
7.0
0.800
PLCC
SRCC
0.845
0.813
0.872
0.837
0.875
0.848
0.888
0.892
0.901
0,902
0.903
0.905
0.225
0.620
Netire
6.0
KRCC
0.620
0.644
0.661
0.697
0.706
0.716
Roochs
Epochs
RMSE
7.284
6.788
6.700
9.345
8.935
8.746
(c) KRCC
(d) RMSE
performance, and the same goes for cardinal number m.
Figure 3. Comparison of performances between pseudo-labels learned by Eq. 7 (blue lines) and network prediction (yellow lines)
4.4. Visualized Analysis
according to ground-truth MOS labels on KonIQ-10K (1:3:1) in each epoch of Stage 2.
We further investigate the quality of pseudo-labels learned by Eq. 7 during each iteration in Stage 2. The whole visual analysis experiments are conducted on KonlQ-10K (1:3:1)
250
200
150-
150-
with m 100. On the one hand, we record the perfor- mances of pseudo-labels learned by Eq. 7 in each epoch of Stage 2, including PLCC, SRCC. KRCC and RMSE according to ground-truth (GT) labels. Meanwhile, the per-
125
125
100
75-
75-
so
formances of directly predicted labels by the model in the current epoch recorded. The above results are shown in Figure 3, from which we observe that: (i) pscudo- labels learned by Eq. 7 are almost always more effective than that predicted by the network. (ii) the performance of pseudo-labels predicted by the network is gradually improv- ing. Therefore, the strategy of Alternate Iterative Training and the method of Label Optimizing can be proven to be effective. On the other hand, we make comparison be- tween the distribution of pseudo-labels learned by Eq. 7 and that of ground -ruth (GT) MOS labels, which are shown in Figure 4. It can be observed that the distribution of pseudo-labels is almost consistent with that of the GT which proves the effectiveness of Label Optimizing once again. Note that achieving the consistency between the distributions of pseudo-labels and GT labels is not trivial. Traditional label propagation methods, tailored for classi- fication perform poorly (Appendix F.3), highlighting the superior adaptability of our proposed methods to IQA tasks.
90
100
800
85
Score lavels
Spore sooel
(a) Pscudo-labels via Eq. 7
(b) Ground-truth (GT) labels
Figure 4. The distribution contrast of pseudo-labels obtained by Eq. for unlabeled training samples and the corresponding GT- labels, validating the accuracy of the predicted pseudo-labets
trained with labeled and unlabeled samples from different datasets, and (6) Evaluating CPL.IIQA with unlabeled train- ing data from multiple sources. More details are shown in Appendixes E.1, E.2, E.3, E.4, E.5, and E.6 respectively.
5. Conclusions
This paper proposes a novel semi-supervised BIQA method CPL-IQA for real-world images with authentic distortions based
pseudo-label learning, which includes a preprocessing procedure (Label Conver- sion) and a two-stage training process. In the preprocessing procedure Label Conversion, MOS labels are transformed into vector labels according to entropy minimization, then a basic feature extractor is trained with limited labeled sam- ples in Stage 1, based on which the proper pseudo-labels are learned for unlabeled images in the Label Optimizing step of Stage 2, which is executed alternately and iteratively with the step of Model Updating for continuous training until convergence. Extensive experiments are conducted to prove the superiority of the proposed CPL-IQA.
mnnfidence-quantffaale
4.5. More Experimental Results
We also conducted additional experiments to further plore several aspects, including (1) the Results with Dif- ferent Backbones, (2) the Visualization of Label Confi- dence, (3) the impact of Cosine Similarity-based Manifold Structure on the experimental outcomes, (4) the Impact on CPL-IQA of & in Eq. 3, (5) performances of CPL-IQA
8

---

Semi-Supervised Blind Quality Assessment with Confidence-quantffable Pscudo-label Learning for Authentic Images
Acknowledgements
Douze, M., Szlam, A., Hariharan, B., and Jégou, H. Low- shot learning with large-scale diffusion. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 0349-3358, 2018.
This work is partially supported by Sino-German Center (M 0187) and the NSFC under contract 62088102. We also acknowledge High-Performance Computing Platform of Peking University for providing computational resources.
Elasri, M., Elharrouss, O., Al-Maadeed, S., and Tairi, H. Image generation: A review. Neural Processing Letters, 54(5)440094446, 2022.
Impact Statement
This paper presents work whose goal is to advance the field of Machine Learning and Image Quality Assessment. There may exist some potential societal consequences of our work, none of which must be specifically highlighted here.
Fan, C., Wu, Q.. Zhao, Y., and Mo, L. Integrating active learning and semi-supervised learning for improved data- driven hvac fault diagnosis performance. Applied Energy, 156:122356, 2024.
Fang, Y., Zhu, H., Zeng. Y., Ma, K., and Wang. Z. Percep- tual quality assessment of smartphone photography. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3677-3686, 2020.
References
Belkin, M. and Niyogi, P. Laplacian eigenmaps for di- mensionality reduction and data representation. Neural Computation, 15(6):1373-1
2003.
Ghadiyaram, and Bovik, A. C. Massive online crowd- sourced study of subjective and objective picture quality. IEEE Transactions Image Processing, 25(1):372-387, 2015.
Bosse, S., Maniry, D., Wiegand, T., and Samek, W. A deep neural network for image quality assessment. In 2016 IEEE International Conference on Image Processing, pp. 3773-3777. IEEE, 2016.
Greven, A., Keller, G., and Warnecke G. Entropy, vol- ume 47. Princeton University Press, 2014.
Bosse, S., Maniry, D., Müller, K.-R., Wiegand, T., and Samek, W. Deep neural networks for no-reference and full-reference image quality assessment. IEEE Transac- tions on Image Processing, 27(1):206-219, 2017.
Haeusser, P., Mordvintsev, A., and Cremers, D. Learning by association-a versatile semi-supervised training method for neural networks. In Proceedings of the IEEE Confer- ence on Computer Vision and Pattern Recognition, pp. 89-98, 2017.
Chen, W. Wu, Y., Zhang, Z., Zhuang, F., He, Z., Xie, R., and Xia, Fairgap: Faimess-aware recommendation via generating counterfactual graph. ACM Transactions Information Systems, 42(4):1-25, 2024.
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learn- ing for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, 2016.
Chen, W., Ye, G., Wang. Y., Zhang. Z., Zhang, L., Wang, D., Zhang, Z., and Zhuang, F. Smoothness really matters: A simple yet effective approach for unsupervised graph do- main adaptation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 15875-15883, 2025.
Hosu, V., Lin, H., Sziranyi, T., and Saupe, D. Koniq- 10k: An ecologically valid database for deep learning of blind image quality assessment. IEEE Transactions on Image Processing, 29:4041-4055 2020.
Ciancio, A., da Silva, E. A., Said, A., Samadani, R., Obrador, P., al. No-reference blur assessment of digital pictures based multifeature classifiers. IEEE Transactions Image Processing, 20(1):644-7) 2010.
Hou, L., Yu, C.-P., and Samaras, D. Squared earth mover's distance- based loss for training deep neural networks. 2016.
arXiv preprint 1oXiv:1611.000
Deng, J. Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on Computer Vision and Pattern Recognition, pp. 248-255. Ieee, 2009.
Hu, E. J., Shen, Y., Wallis, P.. Allen-Zhu, Z., Li, Y.. Wang, S., Wang, I.., and Chen, W. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106 106.:22006006885, 2021.
Ding, K., Liu, Y., Zou, X., Wang, S., and Ma, K. Locally adaptive structure and texture similarity for image quality assessment. In Proceedings of the 29th ACM Interna- tional Conference on Multimedia, pp. 2483-2491 2021.
Iscen, A., Tolias, G., Avrithis, Y., and Chum, O. Label propa- gation for deep semi-supervised learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 5070-5079, 2019.

---

Semi-Supervised Blind Quality Assessment with Confidence-quantffable Pscudo-label Learning for Authentic Images
Jiang, Q. Shao, F., Lin, W., Gu, K., Jiang, G., and Sun, H. Optimizing multistage discriminative dictionaries for blind image quality assessment. IEEE Transactions Multimedia, 2038):2035-22) 2048, 2017.
Liu, L.,, Liu, B., Huang, H., and Bovik, A. C. No-reference image quality assessment based spatial and spectral entropies. Signal Processing: Image Communication, 29 (8):856-86) 2014.
Johnson, J., Douze, M., and Jégou, H. Billion-scale similar- ity search with gpus. IEEE Transactions on Big Data, 7 (3):535-547, 2019.
Liu, X., Van De Weijer, J., and Bagdanov, A. D. Rankiqa: Learning from rankings for no-reference image quality assessment. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1040-1049, 2017.
Kang, L., Ye, P., Li, Y., and Doermann, D. Convolutional neural networks for no-reference image quality assess- ment. In Proceedings of the IEEE Conference on Com- puter Vision and Puttern Recognition, pp. 1733-1740, 2014.
Liu, Y., Gu, K., Wang, S., Zhao, D., and Gao, W. Blind quality assessment of camera images hased on low-level and high-level statistical features. IEEE Transactions on Multimedia, 21(1):135-145 2018.
Liu, Y., Gu, K., Li, X., and Zhang, Y. Blind image quality assessment by natural scene statistics and perceptual char- acteristics. ACM Transactions on Multimedia Computing, Communications, and Applications, 16(3):1-91, 2020.
Ke, J., Wang, Q.. Wang, Y., Milanfar, P., and Yang, Musiq: Multi-scale image quality transformer. Proceedings of the IEFF/CVF International Conference on Computer Vision, pp. 5148-5157, 2021.
Ma, H., Sima, K., Vo, T. V., Fu, D., and Leong, T.-Y. Re- ward shaping for reinforcement learning with an assistant reward agent. In Forty-first International Conference on Machine Learning, 2024a.
Le-Khac, H., Healy, G., and Smeaton, A. F. Contrastive representation learning: A framework and review. Ieee Access, 8: 193997--939933 2020.
Li, A., Wu, J., Tian, S., Li, L., Dong, W. and Shi, G. Blind image quality assessment based on progressive multi-task learning. Neurocomputing, 500:307-318, 2022.
Ma, H., Vo, T. V., and Leong, T.-.. Mixed-initiative bayesian -oba optimization in hierarchical reinforce- ment learning. Proceedings of the 23rd international conference on autonomous agents and multiagent sys- lems, pp. 1328- 1336, 2024b.
Li, D., Jiang, T., and Jiang, M. Norm-in-norm loss with faster convergence and better performance for image qual- ity assessment. In Proceedings of the 28th ACM Interna- tional Conference on Multimedia, pp. 789-797, 2020.
Ma. H., Luo, Z., Vo. T. V., Sima, K., and Leong, T.-Y. Highly efficient self-adaptive reward shaping for rein- forcement learning. In Thirteenth International Confer- ence on Leurning Representations, 2025.
Li, Q.. Lin, W., and Fang. Y. No-reference quality assess- ment for multiply-distorter images in gradient domain. IEEE Signal Processing Letters, 23(4):541-544 2016.
Ma, K., Liu, W., Liu, T., Wang, Z., and Tao, D. dipiq: Blind image quality assessment by Barming-to-rame dis- criminable image pairs. IEEE Transactions on Image Processing, 2618):39513394, 2017a.
X., Li, B., Jin, X., Lan, C., and Chen, Z. Learning distor- tion invariant representation for image restoration from causality perspective. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 1714-17244 2023.
Ma, K., Liu, W., Zhang, K., Duanmu, Z., Wang, Z., and Zuo, W. End-to-end blind image quality assessment using deep neural networks. IEEE Transactions Image Processing, 27(3):1202-1215 12111 2017b.
Li, X., Wang, 7., and Xu, B. Blind image quality assess- ment with semi-supervised learning. Journal of Visual Communication and Image Representution, 100:104100 2024.
Ma, X., Feng, C., Wang, J., Lin, Q., Zhang, S., Zhu, J., Chen, X., Liu, C., and Yu, D. A model-agnostic semantic- quality compatible framework based on self-supervised semantic decoupling. In Proceedings of the 3 ACM International Conference Multimedia, pp. 6774-6784,
Lin, H., Hosu, V., and Saupe, D. Kadid-10k- A large- scale artificially distorted iqa database. In 2019 Eleventh International Conference on Quality of Multimedia Expe- rience, pp. 1-3. IEEE, 2019.
2023.
Madhusudana, P. C., Birkbeck, N., Wang, Y., Adsumilli, B., and Bovik, A. C. Image quality assessment using contrastive learning. IEEE Transactions on Image Pro- cessing, 41611 2022.
Lin, H., Hosu, V., and Saupe, D. Deepfl-iqa: Weak su- pervision for deep iqa feature learning. arXiv preprint arXiv:2001.00 08/ 13, 2020.
10

---

Semi-Supervised Blind Quality Assessment with Confidence-quantffabbe Pscudo-label Learning for Authentic Images
Mittal, A., Moorthy, A. K., and Bovik, A. C. No-reference image quality assessment in the spatial domain. IEEE Transactions on Image Processing, 21(12):4695-4008 2012a.
Su, S., Yan, Q. Zhu, Y., Zhang, C., Ge, X., Sun, J., and Zhang, Y. Blindly assess image quality in the wild guided by self-adaptive hyper network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3667-3676, 2020.
Mittal, A., Soundararajan, R., and Bovik, A. C. Making "completely blind" image quality analyzer. IEEE Signal Processing Letters, 20(3):209-212, 2012b.
Sun, S., Yu, T., Xu, J., Zhou, W., and Chen, Z. Graphiqa: Learning distortion graph representations for blind image quality assessment. IEEE Transactions on Multimedia, 23-2912-2925, 2023a.
Moorthy, A. K. and Bovik, A. C. A two-step framework for constructing blind image quality indices. IEEE Signal Processing Letters, 17(5):113-516, 2010.
Sun, W. Min, X., Tu, D., Ma, S., and Zhai, G. Blind quality assessment for in-the-wild images via hierarchical feature fusion and iterative mixed datahase training. IEEE Journal of Selected Topics in Signal Processing, 2023b.
Pan, W., Gao, T., Zhang, Y.. Zheng, X., Shen, Y., Li, K., Hu, R., Liu, Y., and Dai, P. Semi-supervised blind image quality assessment through knowledge distillation and incremental learning. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, volume 38, pp. 4388-4396, 2024.
Talebi, H. and Milanfar, P. Nima: Neural image assess- ment. IEEE transactions Image Processing, 27(8): 3998 ,011, 2018.
Pan, Z., Yuan, F., Wang, Xu, L., Shao, X., and Kwong, S. No-reference image quality assessment via multibranch convolutional neural networks. IEEK Transactions on Artificial Intelligence, 4(1): 160, 2022.
Thomee, B., Shamma, D. A., Friedland, G., Elizalde, B., Ni, K., Poland, D., Borth, D., and Li, L.-J. 100m: The new data in multimedia research. Communications of the ACM, 59(2): 73, 2016
Prabhakaran, V. and Swamy, G. Image quality assessment using semi-supervised representation learning. In Pro- ceedings of the IEEE/CVE Winter Conference on Appli- cations of Computer Vision, pp. 538-547, 2023.
Tian, C., Fei, L., Zheng, W., Xu, Y., Zuo, W., and Lin, C.-W. Deep learning on image denoising: An overview. Neural Networks, 131:251-275, 2020.
Wang, J., Chen, Z., Yuan, C., Li, B., Ma, W., and Hu, W. Hierarchical curriculum learning for no-reference image quality assessment. International Journal of Computer Vision, 131(11):30743399) 2023.
Qin, G., Hu, R., Liu, Y.. Zheng, X., Liu, H., Li, X., and Zhang, Y. Data-efficient image quality assessment with attention-panel decoder. In Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 37, pp. 2091- 2100, 2023.
Wang, Z. and Ma, K. Active fine-tuning from gmad ex- amples improves blind image quality assessment. IEEE Transactions on Pattern Analysis and Machine Intelli- gence, 44(9):577744900 2021.
Saad, M. A., Bovik, A. C., and Charrier, C. Blind image quality assessment: A natural scene statistics approach in the dct domain. IEEE Transactions on Image Processing, 21(8):3339-3352, 2012.
Wang. Z.. Bovik, A. C., and Lu, L. Why is image quality assessment so difficult? In 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing, volume pp. IV-3313. IEEE, 2002.
Saha, A., Mishra, S., and Bovik, A. C. Re-iqa: Unsupervised learning for image quality assessment in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5846-5855, 2023.
Wang. Z., Zhu, X., Adeli, E., Zhu, Y., Zu, C., Nic, F., Shen, graph-based based transductive learning for multi-modal classification of brain disorder disease. In Medical Image Computing and Computer- Assisted Intervention 2016: 19th International Conference, Athens, Greece, October 17-21, 2016, Pro- ceedings, Part 1 19, pp. 291-299. Springer, 2016.
D., and Wu, G. Progressive graph-
Shi, W., Gong, Y., Ding, C., Tao, Z. M.. and Zheng, N. Transductive semi-supervised deep learning using min- max features. In Proceedings of the European Conference on Computer Vision, pp. 299-315, 2018
Srinath, S., Mitra, S., Rao, S., and Soundararajan, R. Learn- ing generalizable perceptual representations for data- efficient no-reference image quality assessment. In Pro- ceedings of the IEEE/CVF Winter Conference on Appli- cations of Computer Vision, pp. 22-31, 2024.
Wang, Z., Wang, H., Chen, T., Wang, Z., and Ma, K. Trou- bleshooting blind image quality models the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16256-16265, 2021.

---

Semi-Supervised Blind Quality Assessment with Confidence-quantffbble Pseudo-label Learning for Authentic Images
Xiang, T., Yang, Y., and Guo, S. Blind night-time image quality assessment: Subjective and objective approaches. IEEE Transactions on Multimedia, 125517125- 2019.
Zhang, W., Li, D., Ma, C.,, Zhai, G., Yang. X., and Ma, K. Continual learning for blind image quality assessment. IEEE Transactions on Pattern Analysis and Machine In- telligence, 45(3):2864-2877 2022a.
Xu, J., Ye, Li, Du, H., Liu, Y., and Doermann, D. Blind image quality assessment based on high order statis- tics aggregation. IEEE Transactions on Image Processing, 25(9):4444-4535 2016.
Zhang, W., Li, D., Min, X., Zhai, G., Guo, G., Yang, X., and Ma, K. Perceptual attacks of no-reference image quality models with human-in-the-llow
Advances in Neural Information Processing Systems, 35:2916 2922999992,, 2022b.
Yan, B., Bare, B., and Tan, W. Naturalness-aware deep no- reference image quality assessment. IEEE Transactions on Multimedia, 21(10):2603 2615, 2019.
Zhang. W. Zhai, G., Wei, Y., Yang. X., and Ma, K. Blind image quality assessment via vision-language correspon- dence: A multitask learning perspective. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14071-1408) 2023.
Yang, C., Liu, Y., Li, D., et al. Exploring vulnerabil- ities of no-reference image quality assessment mod- els: A query-based black-box method. arXiv preprint M.Xi:22001.05217, 2024.
Zhang, W., Ma, K., Zhai, G., and Yang, X. Task-specific nor- malization for continual learning of blind image quality models. IEFE Transactions on Image Processing, 2024.
Ye, P., Kumar, J., Kang, L.,, and Doermann, D. Unsuper- vised feature learning framework for no-reference image quality assessment. In 2012 IEEE Conference on Com- puter Vision and Puttern Recognition, pp. 1098-1105. IEEE, 2012.
Zhang, X., Zhang, X., Xiao, Y., and Liu, G. Theme- aware semi-supervised image aesthetic quality assess- ment. Mathematics, 10(15):2009, 2022c.
Zhao, K., Yuan, K., Sun, M., Li, M., and Wen, X. Quality- aware pre-trained models for blind image quality assess- ment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22302- 22313, 2023.
Ying, Z., Niu, H., Gupta, P., Mahajan, D., Ghadiyaram, D., and Bovik, A. From patches to pictures (paq-2-piq): Map- ping the perceptual space of picture quality. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3575-3585, 2020.
Zhao, S., Zhang, J., Wu, Y., Luo, Y., and Nie, Z. Langcell: Language- pre-training for cell identity understanding. arXiv preprint arXiv:2405.0677
Yue, G., Cheng, D., Li, L., Zhou, T., Liu, H., and Wang, T. Semi-supervised authentically distorted image qual- ity assessment with consistency-preservvii dual-branch convolutional neural network. IEEE Transactions on Multimedia, 2022.
2024.
Zhong, Y., Wu, X., Jiang, B., and Chen, H. Multi-label local-to- local-to-global feature selection. 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1-8. IEFE, 2021.
Zeng, H., Zhang, L., and Bovik, A. C. Blind image quality assessment with a probabilistic quality representation. In 2018 25th IEEE International Conference on Image Processing (ICIP), pp. 609-613. IEEE, 2018
Zhong, Y., Wu, X., Zhang, L., Yang, C., and Jiang, T. Causal-iqa: Towards the generalization of image qual- ity assessment based on causal inference. In Forty-first International Conference on Machine Learning. 2024a.
Zhai, G. and Min, X. Perceptual image quality assessment: survey. Science China Information Sciences, 63:1-52. 2020.
Zhong, Y., Zhao, R., Wang, C., Guo, Q. Zhang, J., Lu, Z., and Leng. L. Spike-ssm: A sparse, precise, and efficient spiking state space model for long sequences learning. arXiv preprint arXiv:2410. 17268, 2024b.
Zhang, L., Zhang, L., and Bovik, A. C. A feature-enriched completely blind image quality evaluator. JEEE Transac- tions on Image Processing. 24(8):2579-2591 2015.
Zhang, W., Ma, K., Yan, J., Deng, D., and Wang, Z. Blind image quality assessment using a deep bilinear convolu- tional neural network. IEEE Transactions on Circuits and Systems for Video Technology, 30(1):36 2018.
Zhong, Y., Wu, X., Zhao, X., Zhang, L., Song, X., Shi, L., and Jiang, B. Semi-supereised multi- feature selection with consistent sparse graph learning. arXiv preprint arXiv:2505 2025a.
Zhang, W., Ma, K., Zhai, G., and Yang, X. Uncertainty- aware blind image quality assessment in the laboratory and wild. IEEE Transactions on Image Processing, 30: 3474-3486, 2021.
Zhong, Y., Zhao, X., Zhao, G., Chen, B., Hao, F., Zhao, R., He, J.., Shi, L.,, and Zhang, L. Ctd-inpaintingg: Towards the coherence of text-driven inpainting with blended dif- fusion. Information Fusion, pp. 103163, 2025b.
12

---

Semi-Supervised Blind Quality Assessment with Confidence-quantffable Pscudo-label Learning for Authentic Images
Zhou, D., Bousquet, O., Lal, T., Weston, J., and Schölkopf, B. Learning with local and global consistency. Advances in Neural Information Processing Systems, 16, 2003.
A. Proofs
A.1. The Proof of Assertion 3.1
Proof. Without loss of generality, let P(0) V, according to Eq. have:
Zhou, W., Yu, L.., Qiu, W., Zhou, Y., and Wu, M. Local gra- dient patterns (gg)): An effective local-statistical-eeul extraction scheme for no-reference image quality assess- ment. Information Sciences, 397: 1-1,, 2017. 2017.
P(() = (V (1-7) > GnyV. (15)
Thou, Z., Xu, Y., Xu, R., and Quan, Y. No-reference image quality assessment using dynamic complex-valued neural model. In Proceedings of the 30th ACM International Conference Multimedia, pp. 1006-1015, 2022.
Note that 0 < 1, and the eigenvalues of & are in the
interval [-1, then
Zhu, H., Li, L., Wu, J., Dong, W., and Shi, G. Metaiqa: Deep meta-learning for no-reference image quality as- sessment. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 14143- 14152, 2020.
lim (7G)-1= =, t
(16)
lim Σ = =(I-
(17)
8-000
==
Zhu, H., Chen, B., Zhu, L., Wang, S., and Lin, W. Deepde: Deep distance correlation perceptual image quality evaluator. arXiv preprint arXiv:22 1. 1.04927, 2022.
Hence we can obtain:
= lim P(() = (1 (1 (1-))
(18)
Zhu, H., Sui, X., Chen, B., Liu, X., Chen, P., Fang, Y., and Wang, S. 2afe prompting of large multimodal models for image quality assessment, IEEE Transactions on Circuits and Systems for Video Technology, 2024a.
which proves up Assertion 3.1.
A.2. The Proof of Assertion 3.2
Proof. We set the derivative of H(P) w.r.t equal to zero, to obtain the optimal P (denoted as P*);
Zhu, H., Wu, H., Li, Y., Zhang, Z., Chen, B., Zhu, L., Fang. Y., Zhai, G., Lin, W., and Wang, S. Adaptive image quality assessment via teaching large multimodal model to compare. In Advances in Neural Information Processing Systems, pp. 22611-32629 2024b.
OH OP
= - GP + -V) (19)
P=P*
which equivalent to:
p GP
p V 0.
(20)
1 1
1 p
Hence we have:
P V. (1 G)P- = 1+ 1+
(21)
where I is the identity matrix. Note that symmetric and invertible matrix, then we have:
Ô is
1+p
# (I 1+ μ I+
1 GV
pe
(22)
Note that μı > 0, so we can regard 1/(1 + p)) and // (1+ p)) as 7 and 1 7 respectively, then Eq. 22 can be rewritten as:
p = - -
(23)
which is the same as Eq. 6 in the submitted version, and proves up Assertion 3.2.
13

---

Semi-Supervised Blind Quality Assessment with Confidencc-quantifabbll Pscudo-label Learning for Authentic Images
B. More detailed Related Works
BIQA methods SSLIQA (Yue et al., 2022) and SS-IQA (Pan et al., 2024), which both require training with multiple net- work branches and additional datasets. Therefore, the mo- tivation of this paper is to design a semi supervise BIQA method that can be effectively trained end-to-end only on a single branch network without extra inputs in authentically distorted scenarios.
B.1. Blind Image Quality Assessment
Blind Image Quality Assessment (BIQA) has attracted wide attention in recent year since reference images are unavail- able in authentically distorted image datasets (Wang al., 2002). Traditional BIQA methods focused on manually constructing statistical features and predicting MOS labels of images by linear mapping (Wang et al., 2002; Liu et al., 2018; Saad et al., 2012; Moorthy & Bovik, 2010). With the development and wide applications of convolutional neural networks (CNNs) and deep learning (DL) (Chen et al., 2024; 2025; Zhong et al., 2025b; Ma et al., 2025; 2024b;a; Zhao et al., 2024; Zhong et al., 2024b), DL-based or CNN-based BIQA methods have become the mainstream direction of BIQA research (Zhu et al., 2024a;b; 2022), while these BIQA methods are "data hungry". Therefore, some works (Kang et al., 2014; Bosse et al., 2016; Ding et al., 2021) try to cope with the insufficient labeled data by dividing the image into several patches with the same MOS label. Although different weighting strategies can be utilized to integrate local scores, this operation still leads to the loss of global information. So, more complex networks are proposed to improve the generalization of BIQA (Liu et al., 2017; Lin et al., 2020; Ma et al., 201 Zhou et al., 2022). For example, RankIQA (Liu et al., 2017) proposed a Siamese Network to rank image pairs that are syntheti- cally distorted, and MEON (Ma et al., 2017b) is pre-trained for distortion identification tasks. However, these methods are trained on symthetically distorted databases, leading to limited performance on images with authentic distortions.
It is noteworthy to mention that the methodology proposed in this paper represents the current SOTA BIQA method. Although methods such as DEIQT (Qin et al., 2023), Re- IQA (Saha et al., 2023), CONTRIQUE (Madhusudana et al., 2022), and GRepQ (Srinath et al., 2024) utilize a limited labeled dataset for training, their core approach involves pre- training on a large-scale dataset followed by fine-tuning on a smaller dataset. Consequently, these methods are inherently not comparable to semi semi-supervisee IQA approaches. Nev- ertheless, in the experiments detailed in the main text, we have compared our method with SOTA unsupervised IQA methods, including Re-IQA and CONTRIQUE, thereby thoroughly validating the efficacy of the proposed method. Moreover, although the work presented in (Zeng et al., 2018) also suggests transforming scalar Mean Opinion Score (MOS) labels into vector labels for the training of IQA models, it achieves this transformation into normal distribution through a pre-defined variance. This approach not only is exclusively suitable for fully supervised training but also may lead to suboptimal performance due to the potential inaccuracy of the pre-set variance. In contrast, our proposed method, based on entropy minimization, is effec- tively applicable to semi-supervised IQA learning, offering more adaptable and potentially more accurate framework for quality assessment.
To train the BIQA model with enough distorted image samples, some unsupervised BIQA methods have sprung up (Madhusudana et al., 2022: Prabhakaran & Swamy, 2023; Saha al., 2023; Zhao et al., 2023), which signed based on Contrastive Learning (Le-Khac et al., 2020), among which CONTRIQUE (Madhusudana et al., 2022) and (Prabhakaran & Swamy, 2023) can only be trained on the synthetically distorted databases, while Re-IQA (Saha et al., 2023) and (Zhao et al., 2023) can be trained directly on images with authentic distortions. Additionally, some of them try to make the most of existing supervisory sig- nals, such multi-task learning (Li et al., 2022; Ma al., 2023), rank learning (Li et al., 2020), training with mixed- dataset (Sun et al., 2023b) and Multimode signals (Zhang et al., 2023). More recently, some works (Zhang et al., 2024; Wang & Ma, 2021; Wang et al., 2021; 2023; Yang et al., 2024; Zhang et al., 2022a;b) try to improve the generaliza- tion and robustness of BIQA models by combining BIQA networks with specific learning paradigm, such as Continual Learning, Active Learning, Curriculum Learning and Ad- versarial Learning. However, these methods require much higher training costs, including the latest semi-supervised
B.2. Deep Semi-supervised Learning
de-
According to designing different unsupervised losses for training with unlabeled samples, existing Deep Semi- Supervised Learning (D-SSL) methods can effectively deal with the scenarios of data scarcity in practical applications. One type of represenattive D-SSL method is known as Trans- ductive Semi Supervisdd Learning (TSSL), which regards the labels of unlabeled samples optimization variables, and iteratively updates them in the training process (Wang et al., 2016; Shi et al., 2018 8; Zhong et al., 2025a). Another type of D-SSL method is designed based on label propaga- tion, which propagates labels of labeled data to nearby unla- beled data by constructing the Manifold Graph in the sample space (Iscen et al., 2019; Douze et al., 2018; Haeusser et al., 2017). More recently, some active learning- based semi- supervised methods have been proposed (Fan et al., 2024). Regrettably, these approaches are only suitable for classifi- cation problems and cannot be implemented in regression tasks such as IQA. Therefore, some BIQA works (Prab- hakaran & Swamy, 2023; Yue et al., 2022) try to construct
14

---

Semi-Supervised Blind Quality Assessment with Confidence-uuaniffabl Pscudo-label Learning for Authentic Images
Algorithm 1 Training Process of CPL-IQA
Table 6. The detailed attributes of four authentically distorted
Input:
Parameters o, Y:
image databases used in experiments.
The dimension number of score sets m;
Databases
Number
MOS Range
Distribution
Nearest neighbor parameter I: in kNN;
the vector of label levels
[1, 2, m]];
KonIQ-10K
10,073
[1,5]
Yes
Unlabeled images Xu == (#1-1. 21+2 Zn);
LIVE-C
1,162
[0,100]
No
Labeled images X == (11, ,2....,, mz), and their MOS labels Yz == (y1, -2,...... W1)
NNID SPAQ
2,240
[0.1]
No
11,125
[0,100]
No
Output: Feature extractor ov,, MOS label predictor fo..
1: Initialize 8 and 02;
2: Vz = (YL);
of image content and quality in terms of brightness, color, contrast, and sharpness. And its quality is re- ported by MOS with the range of [1, 5].
3: For epoch E [I,... Ti] do:
4: J1 loss| (X,,, Vz; 01, 02);
5: 0 - 0 - aVJ1 V0, (0- = 02);
LIVE-C (Ghadiyaram & Bovik, 2015). LIVE-C con- sists of 1162 authentically distorted images captured from many diverse mobile devices. Each image was assessed on a continuous quality scale by an average of 175 unique subjects, and the MOS labels range in [0, 100).
6: End for.
7: Repeat:
8: Extract the features of all images: Q = So. (X);
9:
Construct nearest neighbor graph by Eqs. 3-4;
10:
Learn pseudo-labele labels Yu by Eqs. 6-7;
11:
Compute confidence vi of pseudo-labels by Eq. 8:
12:
J2 = loss2 (Xz., Yr; 0,90 00 los82 (Xo, 8o; 0, n.9))
NNID (Xiang al., 2019). NNID contains 2240 im- ages with 448 different image contents captured by different photographic equipment in real-world scenar- ios. And the MOS labels of NNID range in [o, 1].
loss2 defined in Eq. 12.
13:
0 - a J2/VO;
14:
Until convergence.
SPAQ (Fang et al., 2020). SPAQ includes 11,125 im- ages taken by 66 mobile phones, which contain wide range of distortions during shooting, such as sensor noise, blurring due to out-of-focus, motion blurring, over- or under-exposure, color shift, and contrast re- duction. And the MOS labels range in [0, 100].
the unsupervised contrastive loss that can be trained with unlabeled samples. However, the training processes require additional supervisory information or network branches in these methods, which inevitably leads to additional training costs and lower applicability. Different from these unsu- pervised methods designed by Contrastive Learning, our proposed BIQA method can achieve both high applicability and low complexity with more standard semi-supervised paradigm.
The details of these datasets are shown in Table 6, where Distribution denotes whether the database provides the dis- tribution information of the opinion scores for each image. In fact, there is no Distribution information in most authen- tically distorted image databases
C. Algorithm
E. More Experimental Results
The pseudo-code of CPL-IQA is summarized in Algo- rithm 1, where line 2, lines 3-6. lines 8-11 and lines 12-13 are the processes of Label Conversion, Stage Step 1 and Step 2 of Stage 2, respectively.
E. Results of Different Backbones
In our experimental settings in Section 4, the backbone of CPL-IQA is set as ResNet101. And we make comparisons among the performances of CPL-IQA trained with different backbones, including ResNet18, 18, ResNet50, ResNet101, and Vision Transformer-base (ViT-base).
D. Datasets
In this paper, we perform experiments on four representative authentically distorted image databases:
As shown in Table 7, we can observe that these CPL-IQA models with different backbones all perform well, and the deeper the backbone (the more parameters), the better the experimental results. The results of ViT-base are just slightly better than that of ResNet101, but not significantly so, pos- sibly due to the large number of parameters in ViT and the limited amount of training data, which led to overfitting.
K0K (Hosu et al., 2020). It includes 10,073 images with authentic distortions chosen from YFCC100M (Thomee et al., 2016). Eight depth feature- based content or quality metrics are used in the sam- pling process to ensure a wide and uniform distribution
15

---

Semi-Supervised Blind Quality Assessment with Confidence-quantffablee Pscudo-label Learning for Authentic Images
Full connected predictor fo₂
Table Comparison of the performances of CPL-IQA trained with
Feature extractor d6,
different backbones.
slandardize
Methods
Metrics
KonIQ-I00 10K (1:3:1)
SPAQ (1:8:1)
Backbone
Ours
PLCC
0,841
0.869
(Full)
SRCC
0.812
0.868
(weights )
KRCC
0.605
0.673
(AlexNet)
RMSE
7.287
10.101
Ours
PI.CC
0.852
0.886
(Full)
SRCC
0.810
0.884
Figure 6. The schematic diagram of standardized feature extrac- tor designed for Cosine similarity (CS)-based manifold structure, where the extracted features are standardized.
(weights )
KRCC
0.621
0.694
(ResNet18)
RMSE
7.126
9.892
Ours
PLCC
0,870
0.892
(Full)
SRCC
0.842
0,886
(weights ))
KRCC
0.649
0.695
700
rumber of 20
(ResNet50)
RMSE
6.734
9.445
6000
150-
500
325
Ours
PLCC
0.873
0.896
400
300-
(Full)
SRCC
0.845
0.893
300
(weights V
KRCC
0.652
0.703
(ResNet101)
RMSE
6.728
9,220
100
Ours
PI.CC
0.871
0.895
20
45
Score lovel
0
Score level
1.00
(Full)
SRCC
0.852
0.898
(a)
(b)
(weights V)
KRCC
0,646
0.715
(ViI-base)
RMSE
6.692
9.004
Figure 7. Comparison of distributions between Pseudo-labels learned based on the CS based manifold and (b). Ground truth
(GT) MOS labels the 60% unlabeled training images sampled from KonIQ-10K (1:3:1).
E.3. Cosine Similarity Manifold Structure
In the module of Analysis and Discussion of Section 3.3 in the submitted version, we claimed that the distribution of the predicted MOS labels will be damaged if the sample features standardized to construct the manifold struc- ture by cosine similarity (abbreviated to CS-based manifold structure for convenience), Here we conduct the following experiments to verify this point.
(a)
(b)
Figure 5. Comparison (a). confidence weights of pseudo-labels and (b). L1 loss between pscudo-labels and corresponding GT labels in feature space.
E.3.1. EXPERIMENT SETTINGS.
In order to make clear comparison with the results of kNN-based manifold structure in Figure in the submit- ted version, train model KonIQ-10K (1:3:1) with
E.2. Visualization of Label Confidence
In this subsection, visualize the confidence of pseudo- labels learned by the module of Label Optimizing in Sec- tion 3.3 for the 60% unlabeled training images sampled from KonIQ-10K (1:3:1). To be specific, through feature dimensionality reduction by t-SNE, we visualize these 60% unlabeled samples in a two-dimensional space, and make comparison between the confidence coefficients of their pseudo-labels and the differences from corresponding GT MOS labels (measured by L loss).
100 according to standardized features ex- tracted by ResNet101 and its subsequent FC layer (mapping 2048-dimensional vectors to 256-dimensional features). As shown in Figure 6, different from the feature extractor used in Figure 2 in the submitted version, an operation feature standardization embedded the feature extractor dæ.-
Note that standardization is different from the normalization after predictor fa, in Figure 6. Standardization means stan- dardizing to unit vectors, and normalization means the op- eration of Softmax. After training the standardized feature extractor, the image quality features can be obtained with norms equal Therefore, we can define the CS-based
According to Figure 5, the higher the confidence (left is smaller than right), the smaller the corresponding L, loss (left is greater than right) on the whole, which proves the validity of Confidence of Labels illustrated in Section 3.3.
16

---

Semi-Supervised Blind Quality Assessment with Confidence-quantifabble Pscudo-label Learning for Authentic Images
Table
The impact on CPL-IQA the number nearest neighbors is in Eq.
k
2
4
8
9
10
11
12
14
PLCC
0.812
0.847
0.838
0.859
0,865
0.873
0.870
0.864
0.868
SRCC
0.785
0.823
0.840
0.836
0.840
0.845
0.837
0.832
0.851
manifold structure based on the cosine similarity between pairwise sample features:
pare the performance of our method against some advanced baselines under this cross-dataset configuration. Specifi- cally, we select 80% from BID (Ciancio et al.. 2010), 70% from KonIQ- (Hosu et al., 2020), 10% from KonlQ- 10k, and the remaining 20% from two datasets as labeled, unlabeled, validation, and test sets, respectively. Among them, the labeled dataset contains 469 images, which also much smaller than the unlabeled dataset, including 7051 images. The details of dataset BID are shown in Table 9, In this experimental setup, we compared our approach with the following baseline methods: BRISQUE (Mittal et al., 2012a), CORNIA (Ye et al., 2012). NIQE (Mittal et al., 2012b), ILNIQE (Zhang et al., 2015), HOSA (Xu et al.,
94 9) if q is linked to q
Gy
(24)
0
otherwise,
where yi linked to qj means yi is the k nearest neighbor of qj in this manifold graph. Everything else stays the same except that Eq. 24 replaces Eq. 2 in the submitted version.
E.3.2. EXPERIMENT RESULT.
As shown in Figure 7, the distribution of the predicted pseudo-labels becomes high on both sides and low in the middle, which is a far cry from the GT distribution. This supports our earlier mention the main text that during label propagation (Zhou et al., 2003), features are normal- ized to construct the manifold structure of samples (Iscen et al., 2019) using cosine similarity, which is not suitable for regression and IQA tasks.
2016), dipIQ (Ma et al., 2017a), DB-CNN (Zhang et al.,
2018), Meta-IQA (Zhu et al., 2020), HyperIQA (Su et al.,
2020), UNIQUE (Zhang al., 2021), and Semi-IQA (Li
et al., 2024). The results are shown in Table 10.
From Table 10 we can observe that: (1) Comprehensive experimental results demonstrate the superior performance and robustness of our proposed method in cross-dataset scenarios, where labeled and unlabeled training samples originate from different datasets. (2) The comparative anal- ysis reveals that CPL-IQA achieves more significant per- formance improvements on KonIQ-10K compared to Semi- IQA, which substantiates the effectiveness of our proposed label propagation-based semi-supervised IQA framework. The relatively marginal improvement on BID can be at- tributed the substantial disparity between the number of labeled and unlabeled samples. Unlike conventional methods that directly predict pseudo-labels using the ini- tial model, CPL-IQA's label propagation mechanism en- ables more accurate pseudo-label estimation. This charac- teristic makes CPL-IQA particularly effective in leveraging large-scale unlabeled datasets like KonlQ- where can fully exploit the abundant unlabeled samples through its propagation- based learning paradigm.
E.4. The Impact on CPL-IQA of The Number of Nearest Neighbors l: in Eq. 3
In Section 3.3.2, the number of nearest neighbors k in Eq. 3 is by default set to 10 in the proposed CPL-IQA. To validate its effectiveness, we conduct experiments with different val- ues of & on the KonIQ-IOK dataset. Except for the variation in ls the experimental setup remains consistent with that in Table 1. The PLCC and SRCC results with different settings of l are summarized in Table 8.
According to Table 8, it can be observed that when & exceeds 10, there is little to no improvement in PLCC and SRCC. Therefore, we choose to set = 10 as the default value.
E.5. Performances of CPL-IQA Trained with Labeled and Unlabeled Samples from Different Datasets
In a standard semi-supervised learning framework, the train- ing dataset typically comprises both labeled and unlabeled samples. In our experiments in the main text, we exclusively investigated the scenario where both labeled and unlabeled samples were drawn from the same dataset. To further vali- date the effectiveness of our proposed semi-supervised CPL- IQA approach, we extended our investigation to a more chal- lenging drtssed setting, where labeled and unlabeled training samples are sourced from different datasets. Specif- ically, we conducted comprehensive experiments to com-
E.6. Evaluating CPL-IQA Performance with Unlabeled Training Data from Multiple Sources
To systematically investigate the impact of different sources of unlabeled training samples on the performance of the CPL-IQQA model, we conducted comparative experiments by adjusting the source of unlabeled training samples while maintaining the experimental setup of Table 1. Specifically, during the training process, we consistently used 20% of the KonIQ-1Ok dataset as labeled training samples and an
17

---

Semi-Supervised Blind Quality Assessment with Confidence-uanntffabl Pscudo-label Learning for Authentic Images
Table The detailed attributes of three image databases used in experiments in Appendix E.5 and Appendix E.6. Same to the experiments in the main text, MOS labels are firslly normalized to the MOS range! [1, 100] by Max-min Normalization Eq. before training.
Table I.. PI.CC and SRCC Results of CPL.-IQA 20% KonlQ 10k Dataset with Different Unlabeled Training Samples. During the training process, labeled samples were consistendy maintained at 20% of the KonIQ-1OK dataset across all three experimental settings. The unlabeled training samples were respectively derived from: (1) 60% of KonlQ- 10k (6044 samples), (2) an equivalent number of samples from SPAQ dataset, and (3) an equivalent number of samples from KADID-IOK dataset.
Datahases
Number
MOS Range
Distortion Type
KonIQ-10K
10,073
[1,5]
Authentic
BID
586
[0.5]
Authentic
SPAQ 11,125
[0,100]
Authentic
Unlabeled Training Sets
KonIQ-10k
SPAQ
KADID-IOK
KADID-IOK
10,125
[1,5]
Synthetic
PLCC SRCC
0,873
0.844
0.832
0.845
0.833
0.817
Table 10. Correlation between model predictions MOSs on test sets of BID and KonIQ-1Ok, respectively. Top two correlations are highlighted in boldface.
belongs to the category of authentically distorted datasets. These findings collectively suggest that the model's perfor- mance on the test set (from the same dataset as the labeled training samples) improves when the unlabeled training set exhibits closer similarity to the labeled training samples.
BID
KonlQ- 10k
Methods
PLCC
SRCC
PLCC
SRCC
BRISQUE
0.602
0.581
0.079
0,020
CORNIA
0.619
0.591
0.534
0.418
NIQE
0.453
0.440
0.531
0.524
ILNIQE
0.526
0.502
0.530
0.505
HOSA
0.561
0.501
0.652
0.633
dipIQ
0.152
0.018
0.434
0.228
DB-CNN
0.705
0.683
0.688
0.635
MetalQA
0.686
0.653
0.623
0.570
HyperlQA
0.827
0.797
0.715
0.661
UNIQUE
0,666
0,662
0.749
0.713
Semi-IQA
0.811
0.801
0.775
0.750
Ours
0.819
0.784
0.780
0.772
additional 20% as test samples. For the unlabeled training samples, we designed three experimental scenarios (with consistent numbers of unlabeled samples across scenarios): (1) Same-source dataset scenario: Unlabeled samples were drawn from the same dataset labeled samples, utiliz- ing the remaining 60% (6044 samples) of the KonIQ-10k dataset; (2) Same-type distortion dataset scenario: Both la- beled and unlabeled samples were from authentic distortion datasets but different sources, with 6044 samples randomly selected from the SPAQ dataset (Fang et al., 2020); (3) Different-type distortion dataset scenario: Unlabeled sam- ples were from synthetic distortion dataset, with 6044 sam- ples randomly selected from the KADID-IOK dataset (Lin et al., 2019).
The detailed characteristics of the SPAQ and KADID- datasets are presented Table 9. The experimental results are documented in Table 11, revealing the following key observations: (I) The CPL-IQA model achieves optimal per- formance when the unlabeled training set shares the same dataset source with both the test samples and labeled train- ing samples. (2) The model demonstrates superior perfor- mance with SPAQ as the unlabeled training set compared to KADID- 10K, which can be attributed to the fact that SPAQ, similar to the test samples and labeled training samples,
18


Reviews:
[{"review_id": "KS3CucfqPp", "reviewer": "Reviewer_R7vR", "summary": "In this paper, an algorithm named CPL-IQA is proposed for semi-supervised BIQA task. The proposed algorithm leverages confidence-quantifiable pseudo label learning to utilize the unlabeled images for training. Specifically, it first converts MOS labels to vector labels via entropy minimization. Then, during training, it predicts the pseudo label of unlabeled images by NN graph construction. CPL-IQA alternates model training and label optimization. Extensive experimental results show that the proposed algorithm achieves the better performances than existing algorithms.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the affirmation and valuable suggestions. We have addressed each point below in detail.\n# For Essential References Not Discussed\nIn fact, our method is not directly comparable with [1], [2], or [3], as they follow fundamentally different paradigms. Specifically:\n* **Ours: A semi-supervised BIQA** method that leverages confidence-quantifiable pseudo-label learning to utilize unlabeled data.\n* **[1]: A fully supervised LLM-based** approach trained on multiple joint datasets, requiring massive computational resources.\n* **[2]: A fully supervised** method that adapts **large-scale pretrained models** with minimal trainable parameters.\n* **[3]: A fully supervised** approach using **multiple comparison transformers** and score pivots to construct an embedding space.\n\nThus, our method is entirely incomparable with [1] and, strictly speaking, also with [2] and [3]. Nevertheless, for a fair comparison under as equal training conditions and model sizes as possible, we conducted experiments following the settings in Table 2 of our paper:\n|  | KonIQ-10K | KonIQ-10K |LIVE-C | LIVE-C | NNID | NNID |\n|-|-|-|-|-|-|-|\n|  | PLCC | SRCC | PLCC | SRCC | PLCC |SRCC |\n|  [2] | 0.868 | 0.836  |0.742 | 0.685 |0.760 | 0.737 |\n| [3] | 0.852 | 0.831 | 0.759 |0.703 |0.746 | 0.758 |\n| **Ours**   | **0.873** | **0.845** | **0.777** | **0.721** | **0.772** | **0.773** |\n\nThe results confirm our method's superiority. **In final version, we will cite [1], [2], [3] and expand** the experimental section to include more detailed discussions and comparisons with them, further validating the effectiveness of our approach.\n# For Question 1\nIn fact, as mentioned in Left Lines 423-427, Figure 4 primarily serves to demonstrate that the distribution of predicted pseudo-labels closely aligns with that of the ground truth, thereby validating the effectiveness of our Label Optimizing module. \n\nFor quantitative evaluation, we have computed the two metrics between pseudo-labels and GT labels, including MAE (5.124) and RMSE (6.687). We will incorporate these results into final version for quantitative analysis.\n\n# For Question 2\nThe pseudo-labeling process does not significantly increase training time, primarily due to two key innovations:\n* While the label propagation is an iterative process (Eq. 5), we prove in Assertion 3.1 and Eq. 6 that its convergent solution can be obtained through simple matrix operations (Eqs. 6-7), eliminating the need for actual iterations. \n* For constructing the nearest neighbor graph matrix (Sec 3.3.2), we employ FAISS (as stated in Right Line 303) - an efficient dense vector similarity search library developed by Facebook AI Research, which dramatically accelerates the pseudo-label learning process.\n\nTo further validate the efficiency of pseudo-label learning, we tested on a single 2080 Ti GPU using SPAQ with the same experimental setup as in Table 3 (SPAQ 1:8:1). The results show that: (1) Stage 1 training required 63.24 seconds per epoch (5 epochs in total); (2) In Stage 2, pseudo-label generation took 19.38 seconds while model training required 918.56 seconds per epoch (5 epochs in total). This demonstrates that the pseudo-labeling process accounts for only 2.1% of Stage 2 training time and 1.9% of the combined runtime for both stages.\n\n# For Question 3\nIndeed, incorporating additional unlabeled data enhances model performance, and the same holds true for labeled data. While the results in Tables 5 & 10 partially support this observation, we conducted the following supplementary experiments to further validate the conclusion:\n\n**On the one hand**, to systematically validate this effect on KonIQ-10k, we varied the proportion of unlabeled data ($N_u$) following the settings in Table 1:\n\n| $N_u$ | PLCC | SRCC | \n|-|-|-|\n|  0% | 0.850 | 0.822  |\n| 30% | 0.865 | 0.831 | \n| 60%   | **0.873** | **0.845** |\n\n**On the other hand**,  following Table 11's settings, we only increased the labeled training data proportion ($N_l$) from KonIQ-10k while using SPAQ as unlabeled training data:\n\n| $N_l$ | 20% | 50% | 80% |\n|-|-|-|-|\n| PLCC | 0.844 | 0.870  | **0.892** |\n| SRCC | 0.833 | 0.858 | **0.883** |\n\nThe above results and Tables 5 & 10 can confirm that expanding labeled or unlabeled datasets can both improve model performance.\n# For Question 4\nThe key limitation of our algorithm stems from the label conversion module based on entropy minimization, which inherently presumes high confidence in the annotated quality score distributions. In practice, annotation biases (such as significant labeling noise or uneven MOS distributions) may compromise pseudo-label reliability. For example, when adding Gaussian noise (μ=0.5, σ=0.5) to the MOS labels of training samples while keeping the Table 1 configuration, PLCC and SRCC decreased to 0.632 and 0.607 respectively.\n\nTherefore, our future work will focus on developing noise-robust BIQA techniques. We will further add and analyze these limitations in **final version**.", "claims_and_evidence": "Yes, the claims are supported by extensive experimental results including ablation studies and proofs in appendix.", "methods_and_evaluation_criteria": "Yes, the proposed algorithm technically sounds and the evaluation process seems fair.", "theoretical_claims": "Yes, theoretical claims seem to be correct for me.", "experimental_designs_or_analyses": "Yes, experimental designs and analysis process are valid and fair.", "relation_to_broader_scientific_literature": "Most of algorithms for BIQA have been proposed for supervised learning scenario. However, the proposed algorithm is proposed for semi-supervised IQA to perform better on real-world distorted images, which are not well handled in existing IQA datasets for supervised learning scenarios. To this end, the proposed algorithm exploits pseudo label learning approach. This deep semi-supervised learning mechanism is well-studied but not common in BIQA field.", "essential_references_not_discussed": "Some recent papers are not addressed and compared. It would be better to compare with these algorithms as well.\n\n- [1] Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels, ICML24\n- [2] Boosting Image Quality Assessment through Efficient Transformer Adaptation with Local Feature Enhancement, CVPR24\n- [3] Blind image quality assessment based on geometric order learning. CVPR24", "other_strengths_and_weaknesses": "Please see questions for authors section for weakness.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "- Figure 4 compares the distribution of pseudo labels and that of corresponding GT labels. However, it would be good to have quantitative measurement such as MAE between pseudo and GT labels.\n\n- Does the pseudo labeling process increase the training time significantly? How fast it is? \n\n- I am curious whether adding more unlabeled data would improve performance. For example, if the KonIQ dataset with labels and the LIVE-C dataset without labels are trained together, how does the performance change when tested on the KonIQ dataset? Table 11 provides similar experimental results. However, it only uses 20% of KonIQ dataset as labeled samples.\n\n- What is the limitation of the proposed algorithm? For example, it would be good to have failure cases.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "N/A"}}, {"review_id": "5KwGOR3dgE", "reviewer": "Reviewer_mdhL", "summary": "This work focuses on semi-supervised blind image quality assessment (BIQA). The method first converts MOS labels to vector labels via entropy minimization, then constructs nearest neighbor graph to help label optimization with confidence. The pseudo labels are then combined with ground truth to guide the model retraining. The experiments show a good performance of the proposed method.\n\n------------------------- ## update after rebuttal ##-------------------------------------\n\ncomments after rebuttal:\n\nThe authors have partially addressed my concerns, but some confusions still exist.\n\n(1) The performance of different conversion methods given in the rebuttal  shows an evident fluctuation, which is somewhat weird. From a natural sense, the performance should not be that sensitive, especially for the N-D simulation with fixed variance, which is approximately a smoothed version of the proposed entropy maximization.\n\n(2) The authors claims that \"our approach maintains compatibility with any single-branch backbone architecture\" in the rebuttal, which is not supported by current version of experiments.\n\nOverall, I would keep my score unchanged.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the valuable comments and will improve in final version.\n# For Claims & Evidence\nThis is a misunderstanding. We clarify that:\n\n1. **The purpose of Fig. 1 is to demonstrate the limitations of some existing semi-supervised BIQA methods that require full score distributions (Left Lines 61-68)**. As shown, images with similar MOS values can have divergent score distributions, making direct MOS-to-distribution conversion unreliable. Unfortunately, many datasets lack distribution information, limiting these semi-supervised methods' applicability.\n\n2. While some datasets provide variance information, simulating a normal distribution solely based on variance is often inaccurate. Moreover, in real-world scenarios, variance information is frequently unavailable. These limitations restrict the broad applicability of these semi-supervised methods. \nIn addition, as shown in Left Line 95, it is confirmed that **models trained with vectorized distribution labels outperform those using scalar MOS labels**. \nTherefore, the two conflict findings motivated us to **develop a more flexible solution that converts MOS labels into vector labels for training without requiring distribution and variance**.\n\n3. Therefore, **the vectorized conversion process in our method does not depend on variance information**. To achieve this, we propse the entropy minimization (Eq.2) under the **high-confidence MOS assumption** (Left Lines 182-201), leading both better performance than scalar MOS training and broader applicability than distribution-dependent methods.\n\nThus, Fig. 1 and our method are logically consistent. **We will clarify this point more explicitly in final version to prevent any potential misunderstanding.**\n\n# For Weakness 1\nWe acknowledge that label conversion is relatively common in IQA. However, **existing methods are designed specifically for either fully supervised or LLM-based BIQA**, including:\n1. [1] discretizes the score range into five equal intervals, converting continuous scores into one-hot encoded rating levels.\n2. [2] converts MOS labels into vector representations by assuming each MOS score follows a normal distribution, which is simulated using a predetermined fixed variance.\n3. [3] extends [2] by incorporating sample-specific variance values.\n\nWe conduct experiments comparing our method with: (1) the one-hot conversion in [1], and (2) the normal distribution (N-D) simulation used in [2-3], using the same setting as Tab. 1.\n| method | One-hot | N-D | Ours |\n|-|-|-|-|\n| PLCC | 0.803 | 0.832  | **0.873** |\n| SRCC | 0.796 | 0.816 | **0.845** |\n\n**Evidently, existing methods are unsuitable for semi-supervised BIQA scenarios due tofailing reliable pseudo-label confidence assessment**. In contrast, our method innovatively addresses this gap.\n\n[1] Q-align. ICML'24\n\n[2] BIQA with Probabilistic Representation. ICIP'18\n\n[3] A fuzzy neural network for opinion score distribution prediction for BIQA. TCSVT'24\n\n# For Weakness 2\nWhile our method shows marginal improvement over SSLIQA in Tab 1, it demonstrates significantly superior performance in cross-dataset experiments on LIVE-C in Table 2, highlighting its exceptional generalization capability. Moreover, as detailed in Line 715, SSLIQA requires multi-branch network training, whereas our approach maintains compatibility with any single-branch backbone architecture, offering reduced parameters and simpler structure. These advantages substantiate the significance of our method.\n\n# For Weakness 3\nThanks for your suggestion. In fact, the setting $B_L=8, B_U=56$ in Stage 2 is to match Stage 1's batch size $64$, i.e., $B_L+B_U=64$.\nMoreover, the significantly larger size of $B_U$ compared to $B_L$ reflects the practical scenario where labeled samples are substantially fewer than unlabeled samples.\n\nTo investigate the impact of ratio $B_L / B_U$, under the settings of Tab. 1, we obtained the following results:\n\n| $(B_L,B_U)$ | (4,60) | (8,56) | (16,48) |\n|-|-|-|-|\n| PLCC | 0.869 | 0.873  | 0.871 |\n| SRCC | 0.841 | 0.845 | 0.848 |\n\nThe results show that while an excessively small \n$B_L$ may degrade performance; **the model exhibits relative insensitivity** to variations in the ratio of $B_u / B_L$. \n\n# For Weakness 4\nThis is a misunderstanding. The two-step training in our method **does not put a burden on the implementation**.\n\nAs stated in Sec. 3.2, our framework employs a two-stage approach:\n**Stage 1** utilizes only labeled data for training, introducing no additional computational overhead;\n**Stage 2** consists of two steps, with the pseudo-labeling process requiring merely 2.1% of the total training time **(see Response to Reviewer R7vR, Question 2)**.\n\nMoreover, training on SPAQ, our method requires only 16.69 minutes (1 epoch in Stage 1 + 1 epoch in Stage 2) on a single 2080Ti GPU, versus SSLIQA's 18.31 minutes for 2 epochs. This efficiency gain stems from our single-branch design versus SSLIQA's dual-branch architecture, demonstrating our superior efficiency.", "claims_and_evidence": "The authors claim that  images with similar MOS values may correspond to a variety of score distributions (it is true) as shown in Fig.1, but in the label conversion, the variation is not considered, which looks like paradoxical.", "methods_and_evaluation_criteria": "It does make sense.", "theoretical_claims": "Theoretically correct.", "experimental_designs_or_analyses": "Almost sound but inadequate. It would be better to give more experiments. For example, how does the method perform if B_L and B_u varies. Further, the method requires at least two-step training, which puts more burden on the implementation, and the training cost is also questionable.", "relation_to_broader_scientific_literature": "Increamental. The label conversion is not uncommon, which has been tried in various work in quality assessment.", "essential_references_not_discussed": "The references are almost adequate in a constrained page space.", "other_strengths_and_weaknesses": "[strength]\n1) The work designs a semi-supervised BIQA method to address the limited labeled data. \n2) The method introduces a nearest neighbor graph and label propagation strategy to construct pseudo labels. \n3) The method is sound and easy to understand.\n\n[weakness]\n1) The label conversion is somewhat common in IQA/VQA. For example, Q-Align adopts a similar conversion method. It is also good to compare the proposed label conversion with existing MOS discretization methods to demonstrate the distinction and effectiveness.\n2) The experimental result is not impressive comparing to current methods such as SSLIQA. \n3) It would be better to give more experiments. For example, how does the method perform if B_L and B_u varies. I'm wondering if the performance is sensitive to the ratio of B_u/B_L, and which value is more reasonable.\n4) The method requires at least two-step training, which puts more burden on the implementation, and the training cost is also questionable.", "other_comments_or_suggestions": "I have put all my comments and questions in the above.", "questions_for_authors": "I have put all my comments and questions in the above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BqMVyryhgJ", "reviewer": "Reviewer_s6Ro", "summary": "This paper presents a novel semi-supervised blind image quality assessment framework, named CPL-IQA, for assessing the quality of real distorted images. The method effectively utilizes a large number of unlabeled real distorted images through confidence-quantifiable pseudo-label learning, addressing the challenge of limited labeled data in the field of image quality assessment.CPL-IQA consists of two phases, namely, label transformation preprocessing and alternating model training and label optimization. Its core innovation lies in the label optimization strategy based on the streaming assumption and the confidence learning method for pseudo-labels. Experimental results show that the framework performs well on real distorted image datasets, provides a more standard semi-supervised learning paradigm, and does not require additional supervisory information or complex network structure.\n\n## update after rebuttal\nThe author's response answered my questions and I will keep my score. In addition, I hope the author will finally make the code public and promote the community.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your affirmation and the valuable comments that help us improve our work. The following is a careful response and explanation about the weaknesses and questions.\n\n# For Weakness 1\nWe sincerely appreciate the constructive feedback. We would like to clarify that Sections 3.3.1 to 3.3.6 (pages 4–6) of the manuscript have already provided a comprehensive and step-by-step explanation of the algorithm’s implementation. \n\nTo further enhance clarity and ensure readers can better understand the implementation details, **in the final version**, we will include more illustrative examples or pseudocode snippets within the main text to complement the existing Algorithm 1 in Appendix C, which can further improve the readability and transparency of the method while maintaining the rigor of the technical content.\n\n# For Weakness 2\nThank you for the insightful suggestion. **In the final version, we will add a concise discussion in Section 3.3.7 for a more concise exposition of the intuitive understanding of these theoretical linkages**, including the following two key insights:\n\n**On the one hand**, the manifold assumption serves as the theoretical foundation for label propagation, positing that samples with similar features (i.e., neighboring nodes in the graph) should share similar labels. This fundamental assumption enables the label propagation process, where labels from annotated samples are effectively disseminated to unlabeled ones through the constructed nearest neighbor graph (Eq.4), thereby facilitating pseudo-label learning.\n**On the other hand**, for the optimization perspective, the iterative propagation (Eq. 5) implicitly solves the regularization problem (Eqs. 13-14), where the first term enforces smoothness (labels vary slowly over the constructed nearest neighbor graph), and the second term fits the initial labels. The hyperparameter $\\mu$ in Eq.14 balances the above two objectives.\n\n# For Questions\nThank you for the valuable comments that help us improve our work. In fact, **both of the proposed model and label conversion strategy exhibit strong robustness across diverse datasets with varying MOS distributions**, which have been comprehensively demonstrated in our paper. Specifically:\n\n**On the one hand**, we have conducted experiments on multiple datasets (KonIQ-10K, SPAQ, LIVE-C, and NNID) with significantly different MOS ranges and distribution characteristics (e.g., KonIQ-10K: [1,5], SPAQ: [0,100]). The consistent performance improvement (Tables 1–3) across these datasets demonstrates the generalization ability of our label conversion strategy.\n\nAdditionally, in Appendix E.5 and E.6, we explicitly tested scenarios where unlabeled training data came from different distortion types (authentic vs. synthetic) and sources (e.g., training with labeled dataset BID and unlabeled datasets KonIQ-10K in Appendix E.5, and training with labeled dataset KonIQ-10K and unlabeled datasets SPAQ/KADID-10K in Appendix E.6). Results in Tables 10 & 11 show that our method remains effective even when MOS distributions differ, as the entropy minimization inherently adapts to the input label range (Eq. 1 normalizes MOS to [1,100]) and preserves relative quality relationships. The confidence-weighted training (Eq. 8–9) further mitigates potential outliers from label conversion.\n\nTo further validate the robustness of our proposed strategy, we conducted additional experiments: we split KonIQ-10k into labeled training and test sets (8:2 ratio), trained the model on the labeled KonIQ-10k training set and the unlabeled LIVE-C dataset, and evaluated it on the remaining 20% KonIQ-10k test set. The results are as follows:\n\n| training sets | 80% labeled KonIQ-10k | 80% labeled KonIQ-10k + unlabeled LIVE-C |\n|-|-|-|\n| PLCC | 0.879 | **0.891**  | \n| SRCC | 0.874 | **0.885** | \n\nThe above results show that our method is robust to different datasets. \n\n**On the other hand**,  **Although the label conversion process in Eq. (2) cannot be directly applied to predict MOS distributions** (since our method operates under the reasonable assumption that the labeled samples have sufficiently high confidence levels (Lines 190-192)), **the well-trained CPL-IQA model can robustly predict label distributions.** For example, using the model trained as described above, we randomly selected 10 samples each from KonIQ-10k and LIVE-C, and computed the average JS divergence and Wasserstein distance between the predicted MOS distributions and the variance-simulated normal distributions (approximating ground truth). The results are as follows:\n\n| Samples | KonIQ-10k |  LIVE-C |\n|-|-|-|\n| JS | 0.065 | 0.097  | \n| W-Dist | 0.042 | 0.086 | \n\nThe results demonstrate that both the JS divergence and Wasserstein distance remain below 0.1, confirming the accuracy of the predicted MOS distributions and validating the robustness of our method across diverse datasets and MOS distributions.\n\n**In final revision, we will further highlight the robustness.**", "claims_and_evidence": "1) The main innovations of CPL-IQA include a label optimization strategy based on the streaming assumption and a confidence learning method for pseudo-labeling that enhances reliability and mitigates the effects of outliers. Section 3.3.3 of the paper describes the label optimization process in detail, and Section 3.3.4 describes the confidence learning method and demonstrates the effectiveness of these methods in the experimental section through ablation studies and visual analysis.\n2) Experimental results show that the framework exhibits superior performance on real-world distorted image datasets. Table 1 shows the performance comparison of CPL-IQA with other traditional and deep learning BIQA methods on the KonIQ-10K dataset, and the results show that CPL-IQA achieves better performance metrics. The cross-dataset experiments in Table 2 also further validate the generalization ability of CPL-IQA.\n3) The proposed confidence learning approach eliminates pseudo-label outliers and enhances the generalization capability of CPL-IQA. Section 3.3.4 describes the principle of confidence learning and the relationship between confidence and pseudo-label accuracy is supported by the visual analysis in Figure 5. The ablation study in Table 3 also shows the performance enhancement of using confidence weights.", "methods_and_evaluation_criteria": "1) The paper clearly states that the problem to be addressed is semi-supervised blind image quality assessment (BIQA) in real distortion scenarios since it is difficult to obtain distortion-free reference images in practical applications and the cost of labeling real distortion images is high. The CPL-IQA framework is precisely designed to utilize limited labeled data and a large number of unlabeled real distorted images. The core idea is to enhance the performance of the model in the data sparse case through confidence quantized pseudo-label learning.\n2) The paper uses a series of representative real distortion image databases as benchmark datasets, including KonIQ-10K, LIVE-C, NNID, and SPAQ. These datasets are commonly used in the field of BIQA, and in particular, KonIQ-10K and SPAQ are considered to be the most important datasets containing real-world distortions. The use of these datasets can effectively evaluate the performance of models in real-world application scenarios.", "theoretical_claims": "- Assertion 3.1 claims that the sequence ${P^{(t)}}$ defined by Eq. (5) converges to $P^*$ in Eq. (6) via $\\tilde{G}$ obtained by Eq. (4). The proof of this statement is given in Appendix A.1.\n\n- Assertion 3.2 claims that the limit value of the iterative process of Eq. (5), Eq. (6), can be considered as an optimal solution of the regularization framework Eq. (13) (with $\\mu > 0$ in Eq. (14)). The proof of this statement is given in Appendix A.2.", "experimental_designs_or_analyses": "- CPL-IQA was compared with sixteen state-of-the-art BIQA methods, both traditional and deep learning-based, on the KonIQ-10K dataset. The experimental setup uses a 1:3:1 dataset division ratio (training labeled data: training unlabeled data: test data). The results are presented in Table 1.\n\n- The effects of the different components of CPL-IQA were analyzed experimentally. Specifically, the effects of Label Conversion, the weight of Label Confidence, and Label Optimization in Stage 2 are investigated, and the results are presented in Table 3. In addition, the effects of the base m of the score set M and the proportion of dataset partitioning on the performance are also investigated, and the results are presented in Tables 4 and 5, respectively.\n\n- The quality of the pseudo-labels learned by Eq. (7) in each iteration of Stage 2 was analyzed by visual means. The change in performance between pseudo-labeling and model-directly predicted labeling was compared (Fig. 3), as well as the pseudo-labeling distribution versus the true MOS labeling distribution (Fig. 4).", "relation_to_broader_scientific_literature": "- Semi-supervised Blind Image Quality Assessment (BIQA) Framework: The core of the CPL-IQA proposal lies in its semi-supervised learning paradigm, which aims to address the problem of scarcity of labeled data in real-world scenarios. This is in line with the recent trend of research on semi-supervised and unsupervised BIQA methods. Existing deep learning (DL)-driven BIQA methods are usually “data starved”, and thus utilizing unlabeled data to improve performance has become an important research direction.CPL-IQA efficiently utilizes unlabeled real distorted images through pseudo-label learning with quantifiable confidence. This is in line with some previous semi-supervised BIQA works such as SSLIQA (Yue et al., 2022) and SS-IQA (Pan et al., 2024), which all aim to utilize unlabeled data, but CPL-IQA differs in that it uses only a single-branch network and does not rely on additional inputs or datasets, which reduces the training cost and improves applicability, whereas SSLIQA and SS-IQA are more suitable to utilize unlabeled data, whereas CPL-IQA and SS-IQA are more suitable to utilize unlabeled data. SSLIQA and SS-IQA require multi-branch networks and additional data.\n\n- Confidence learning method for pseudo-labeling: In order to mitigate the negative impact of inaccurate pseudo-labeling on model training, CPL-IQA proposes an entropy-based confidence learning method. The uncertainty of the predicted pseudo-labels is estimated by calculating their entropy; the higher the entropy, the lower the confidence level, and a lower weight will be assigned in subsequent model training. This is consistent with the common strategy of handling noisy pseudo-labels in semi-supervised learning.", "essential_references_not_discussed": "The relevant work cited in the paper is relatively comprehensive and does not suffer from missing key literature.", "other_strengths_and_weaknesses": "Strenghts:\n\n- The originality lies in its novel semi-supervised BIQA framework CPL-IQA.\nOriginality lies in its novel semi-supervised BIQA framework, CPL-IQA, which efficiently utilizes unlabeled real distorted images through confidence-quantifiable pseudo-label learning, which directly addresses the key challenge of scarcity of labeled data in the BIQA domain.\n- Uniqueness of Label Transformation Strategy: The entropy minimization-based label transformation method proposed by CPL-IQA is a key innovation in transforming scalar MOS labels into vector labels. This paves the way for applying label propagation techniques, which excel in handling vector labels, to regression tasks like MOS prediction, and draws on the idea of utilizing MOS distributions for training in NIMA, but with innovative adaptations.\n- Successful application of label propagation to regression tasks: traditional label propagation methods are mainly applied to classification problems. cPL-IQA successfully extends the idea of label propagation to regression tasks (i.e., image quality assessment) and proposes a corresponding implementation via Eq. (7), which is in itself of some originality and importance.\n\nWeaknesses:\n\n- Although Appendix C provides the algorithm pseudo-code, the key steps of the training process, such as label transformation, graph construction, label optimization, and the iterative process of model training, can be described in more detail in the main method section, which helps readers better understand the implementation details of the algorithm.\n\n-  While the paper provides proofs of label propagation convergence and equivalence to the regularization framework (in the Appendix), a more concise exposition of the intuitive understanding of these theoretical linkages in the main body might help a broader audience understand the theoretical underpinnings of the approach. For example, it could be briefly explained how the flow shape assumption plays a role in label propagation and how the regularization framework explains the process of label propagation from an optimization perspective.", "other_comments_or_suggestions": "I hope that in the future the authors will make the code public and promote the community.", "questions_for_authors": "The entropy minimization-based label transformation method proposed in the paper is a key step in transforming scalar MOS labels into vector labels. Although it is explained in the paper that this is done to better model the MOS distribution and obtain higher confidence, is this conversion strategy still robust in the face of different datasets or large differences in the characteristics of the MOS distribution? Have the authors conducted relevant experiments to verify the generalization ability of this label transformation strategy, e.g., analyzing the quality of the transformed vector labels on datasets with significantly different MOS distributions? I would consider the method more practical if the authors could provide an analysis of the label transformation effect across datasets or comparative experiments to demonstrate the robustness of the strategy. If the label transformation is very sensitive to the data characteristics, it will limit the scope of its application in different scenarios in the real world.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KcC4QcZUu1", "reviewer": "Reviewer_cfTH", "summary": "This work aims at addressing the quality assessment task for authentically disorted images from a semi-supervised learning perspective. The key idea is to leverage confidence quantifiable pseudo-label learning to confront the data insufficiency challenge. The proposed CPL-IQA can be trained on unlabeled images with an entropy-based confidence learning to mitigate the negative effects from noisy labels.\n\n\n## update after rebuttal\nIn the rebuttal, the authors have futher verified the value of this work by leveraging unlabeled data for boost IQA model. I will keep my rating.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your affirmation and the valuable comments that help us improve our work. The following is a careful response and explanation about the weaknesses and questions.\n\n# For Experimental Designs Or Analyses\nWe sincerely appreciate your recognition of our experimental design and analysis, as well as this valuable suggestion.\n\nIn fact, we have already applied the proposed method to train an IQA model using a combination of existing annotated IQA datasets and additional unlabeled images under standard experimental settings. Specifically, as shown in Appendix Table 10, we select 80% from BID,70% from KonIQ-10k, 10% from KonIQ10k,and the remaining 20% from two datasets as labeled, unlabeled, validation, and testsets, respectively. The results clearly demonstrate the superiority of our approach.\n\nTo further validate the effectiveness of our method, as suggested by the reviewer, we split KonIQ-10k into labeled training and test sets (8:2 ratio), trained the model on the labeled KonIQ-10k training set and the unlabeled LIVE-C dataset, and evaluated it on the remaining 20% KonIQ-10k test set. The results are as follows:\n\n| training sets | 80% labeled KonIQ-10k | 80% labeled KonIQ-10k + unlabeled LIVE-C |\n|-|-|-|\n| PLCC | 0.879 | **0.891**  | \n| SRCC | 0.874 | **0.885** | \n\nFrom the above results, we can observe that, after training with the extra unlabeled dataset LIVE-C, the performance of the model is improved on the test set of KonIQ-10k compared to the counterpart which is trained only on training set of KonIQ-10k. These results further validate the effectiveness of our proposed method.\n\nIn the camera-ready version, we will incorporate these additional experimental findings to further strengthen our work.\n\n# For Other Strengths And Weaknesses\nThanks for this valuable comment. We acknowledge that resizing could theoretically alter distortion characteristics and affect quality assessment accuracy. In fact, the resize-and-crop strategy in our model is actually another choice that is widely used in many works, which can balance computational efficiency and practical applicability. Below, we provide a detailed justification for this design choice in our model from three perspectives:\n1. **Preservation of Global Context:** Since the required final cropped size during training is 224×224, resizing images to 256×256 before cropping in our method ensures that the model captures relatively global regions of the image while retaining fine-grained distortion patterns. This strategy has been widely adopted in classic IQA works such as DB-CNN, NIMA, and SSL-IQA, and our approach follows this well-established practice.\n2. **Trade-off Between Performance and Efficiency:** We agree with the reviewer that cropping multiple patches from original-resolution images (without resizing) is theoretically preferable. However, this requires extracting multiple patches per image, significantly increasing training overhead and computational complexity. This is because, for large original images, a small number of cropped patches may fail to cover sufficient global regions of the image. Therefore, existing models adopting this strategy (e.g., LIQE and HyperIQA) typically address this by cropping a large number of patches, which further escalates computational costs.\n3. **Empirical Validation:** Following the experimental setup in Table 1, we conducted additional tests by randomly cropping 10 patches per original-resolution image (denoted as Strategy 2) and compared its performance with our default resize-and-crop approach (Strategy 1). The results show that Strategy 2 only improves PLCC and SRCC by less than 1%, but at the cost of 4× longer runtime. The cost-performance ratio is unfavorable.\n\n| Pre-processing methods | Stategy 1 | Stategy 2 |\n|-|-|-|\n| PLCC | 0.873 | 0.878  | \n| SRCC | 0.845 | 0.851 | \n| Running time in one epoch of Stage 1 (s) | 63.24 | 238.57 | \n\nWe will emphasize in camera-ready version that our resizing-and-cropping strategy balances efficiency and performance, ensuring scalability for real-world deployment.\n\n# For Questions\nThank you for your careful review and valuable feedback. **We are sorry for this oversight and will correct it in the camera-ready version** by bolding the SRCC result of HyperIQA in the table.\n\nNotably, this oversight does not impact Appendix E.5's analysis, as Table 10 primarily compares our method with Semi-IQA, where ours shows superior performance. While HyperIQA achieves marginally better results on BID through computationally intensive supervised training, it underperforms significantly on KonIQ-10k compared to our method, further validating our approach's effectiveness.\n\nIn camera-ready version, we will conduct a more thorough review to eliminate any remaining typos and ensure the rigor and high standard of our research.", "claims_and_evidence": "Yes,", "methods_and_evaluation_criteria": "Yes. This work propose to leverage unlabeled data for IQA model training, which includes an neccessary module to mitigate the inevitable label noise issue. The proposed alternate learning between model training and label optimization is reasonable since the reliability of pseudo-labels progressively increase with the correctness of the model predictions.", "theoretical_claims": "The proofs in the appendix is correct to the Reviewer.", "experimental_designs_or_analyses": "In the experiments, existing IQA datasets with full quality annotations are manually splited into three parts: training images with labels, training images without labels (exists but not used), and test images. The experiments setup is reasonable to verify the effectiveness of the proposed semi-superivsed learning scheme. Compared with supervised learning, unsupervised learning, and other semi-supervised learning methods in the filed of IQA, the proposed method presents better performance. To the Reviewer, this work can be further improved if the authors can use the proposed method to train an IQA model on the combination of existing annotated IQA datasets and extra unlabeled images following regular setup, e.g., split KonIQ-10k with a ratio of 8:2 (training/test), training the model on the training set of KonIQ-10k and an extra unlabeled dataset, verifying that the the performance of model is improved on the test set of KonIQ-10k compared to the counterpart which is trained only on training set of KonIQ-10k.", "relation_to_broader_scientific_literature": "In the era of deep learning, data plays a key role in developing powerful artificial intelligence models across various tasks. However, the scale of data will be limited when the cost of annotations is high, where IQA is one of such case. This work is closely related to the important topic of training artificial intelligence models using unlabeled data.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Resizing all images into 256x256 may deteriorate IQA performance, it's better to crop multiple image patches from images with their original resolution.", "other_comments_or_suggestions": "No.", "questions_for_authors": "In Table 10, the SRCC result of HyperIQA is higher than the proposed method but not marked with bold. Is there any typo?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper proposes CPL-IQA, a semi-supervised BIQA framework that uses confidence-aware pseudo-labels and entropy-based label conversion to leverage unlabeled data. It achieves strong results on real-world distorted images without added supervision or complexity. Three out of four reviewers are positive. The remaining reviewer raised a few unresolved concerns, stating that (1) the performance of different label conversion methods fluctuates unexpectedly, particularly for the N-D simulation, and (2) the claimed compatibility with single-branch architectures is not fully supported by experimental evidence. Nonetheless, the AC believes the strengths of the paper outweigh these issues and recommends acceptance. The authors are expected to address the above concerns and include the rebuttal content in the camera-ready version and release the code as promised.
Decision: Accept (poster)