<figure><img src="image_1.png" /></figure>

HAL
open science

# Lightweight Image Quality Prediction Guided by Perceptual Ranking Feedback

Aymen Sekhri, Mohamed-Chaker Larabi, Seyed Ali Amirshahi

► To cite this version:

Aymen Sekhri, Mohamed-Chaker Larabi, Seyed Ali Amirshahi. Lightweight Image Quality Prediction Guided by Perceptual Ranking Feedback. 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2025), IEEE, Apr 2025, Hyderabad, India. 10.1109/ICASSP49660.2025.10888173. [hal-05120240]

HAL Id: hal-05120240

https://hal.science/hal-05120240v1

Submitted on 19 Jun 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Lightweight Image Quality Prediction Guided by Perceptual Ranking Feedback

Aymen Sekhri¹,², Mohamed-Chaker Larabi¹, Seyed Ali Amirshahi²

¹CNRS, Université de Poitiers, XLIM, Poitiers, France

²Norwegian University of Science and Technology, Gjøvik, Norway

*Abstract*—Automatic Image Quality Assessment (IQA) remains a difficult challenge due to the complexity of mimicking the Human Visual System (HVS) and the limitations of traditional objective Image Quality Metrics (IQM). Existing learnable methods often involve high computational costs and fail to adequately capture the nuanced perceptual characteristics of the HVS, including the human ability to rank image quality and human sensitivity to differences in areas with high-frequency. In this study, we propose an effective approach that addresses these challenges by incorporating the characteristics of HVS and the perceptual classification into a lightweight IQM framework based on the transformer architecture. This allows our method to capture long-range dependencies effectively. Our approach leverages Objective Error Maps (OEMs) to enhance sensitivity to visual errors and employs a ranking module as an objective function, providing feedback on the perceptual quality at the feature level. Experimental results demonstrate that our approach not only achieves competitive performance compared to state-of-the-art IQMs but also significantly reduces computational complexity.

*Index Terms*—Image Quality Assessment, Vision Transformers, Human Visual System

## I. INTRODUCTION

Evaluating image quality is crucial in image-processing applications such as denoising, super-resolution, and artifact reduction, where perceptual image quality is vital [1]–[6]. Although subjective human evaluation is highly accurate, it is often impractical due to time and cost constraints. Thus, Image Quality Metrics (IQMs) aim to develop algorithms that can automatically mimic the subjective scores given by observers [7], assessing images distorted by processes such as compression and post-processing.

Image Quality Assessment (IQAs) is a challenging task for machines despite being straightforward for observers. Existing IQMs are categorized into Full-Reference (FR), Reduced-Reference (RR), and No-Reference (NR) metrics. FR-IQMs compare the quality of a given image with a reference (pristine) image using methods such as the Mean Squared Error (MSE), Peak Signal-to-Noise Ratio (PSNR), and Structural SIMilarity index (SSIM) [8], or the Feature SIMilarity index (FSIM) [9] which focuses on comparing structural features. NR-IQMs, such as DIIVINE [10] and BRISQUE [11], evaluate the quality of an image without having access to the reference image. It is important to note that NR-IQMs often struggle with different distortions. This is why some NR-IQMs like BPRI [12] and BMPRI [13] use what is referred to Pseudo-Reference Images (PRI) to estimate the image quality based on blockiness, sharpness, and the level of noise in the image. The advent of deep learning in computer vision has led to new FR-IQMs that address the limitations of classical techniques. As an example, [14]–[16] use deep features from pre-trained deep neural networks to evaluate the quality of an image, while WaDIQaM [17] and PieAPP [18] use end-to-end neural networks for image ranking. Recent methods, such as IQT [19], employ transformer architectures for IQA. Other approaches incorporate HVS modeling through CNNs, as seen in [20]. Finally, another study used a CNN-based top-down strategy, leveraging high-

level semantics to direct the network's attention towards semantically significant areas of distortion [21].

A drawback of using attention mechanisms to capture global information and incorporate the characteristics of the HVS is the rather high computational cost that comes with increased model complexity. To address this issue, we propose a lightweight IQM that incorporates OEMs and the human ability to rank distorted images without relying on predicting continuous quality scores, all within a lightweight transformer-based architecture. Below you can find a list of our contributions in this study:

* The use of only half the small Vision Transformer (ViT) trained in a self-supervised manner and only a one-layer decoder makes the proposed IQM significantly lighter compared to other transformer-based IQMs.

* By incorporating a Quality Feature Ranking (QFR) module at the quality representation level, the model can rank images based on their features.

* OEMs are used in two different manners: either using them as input to boost the model's ability to capture differences in high-frequency bands or using them as ground truth to generate error maps from the quality features.

* The results demonstrate performance that is on par with or superior to other methods when tested on benchmark FR-IQA datasets.

The paper is structured as follows: in Section II we provide details of the proposed method followed by a discussion on the training implementation in Section III. The proposed IQM is then compared to other state-of-the-art IQMs in Section IV. Finally, Section V concludes with final remarks and future research directions.

## II. PROPOSED METHOD

The proposed FR-IQM is based on a lightweight transformer architecture [22]. The architecture of our network (Figure 1) consists of multiple sub-modules. The Content-Aware Encoders (CAEs), Intermediate Layers (ILs), Quality-Aware Decoder (QAD), Quality Feature Ranking (QFR) Module, Regressor Module, and finally the OEM Generator. Training the entire architecture involves two main steps. First, the CAE, QAD, and QFR modules are trained on a combination of all training datasets without accessing the test sets. In the second step, the weights of the pre-trained QFR Module are frozen to serve as a ranking loss while training the remaining framework components on each dataset individually, and the performance is tested on the unseen test sets. In this study, we proposed three variants of this process depending on the use of the OEMs.

### A. Content Aware Encoders

CAEs transform input images into abstract representations containing semantic information using a Self-Supervised (SSL) pre-trained ViT. The shift caused by the distortions is then computed by
<figure><img src="image_3.png" /><figcaption>Fig. 1. Architecture of our proposed model variants: LT-QFR, LT-QFR-a, and LT-QFR-g. LT-QFR includes CAE, IL, QAD, and QFR loss. LT-QFR-a extends LT-QFR by incorporating OEM information through module (1). LT-QFR-g further includes the OEM Generator as shown in (2).</figcaption></figure>

comparing the reference and distorted embeddings. We first introduce the ViT framework and then explain how this shift is calculated.

1) **Vision Transformer:** Dosovitskiy et al. [23] marked a shift in computer vision by using transformers instead of CNNs, leveraging self-attention to capture long-range dependencies between image patches. ViT processes a 2D image $X \in \mathbb{R}^{H \times W \times 3}$ by dividing it into N patches, flattening them into 1D vectors, and embedding them. A [CLS] token is added to the sequence to later represent the entire image, followed by adding the positional embeddings to maintain the spatial information of the image. The self-attention mechanism operates by generating queries $q$, keys $k$, and values $v$ from the input embeddings and computing attention weights

$$ \text{Attention}(q, k, v) = \text{softmax}\left(\frac{q k^T}{\sqrt{d_k}}\right) v, \quad (1) $$

where $d_k$ is the dimension of the key vectors. This mechanism captures long-range dependencies, providing superior contextual understanding compared to CNNs, which are limited by their receptive field [24].

In this study, we use a pre-trained ViT model from the DINO approach [25], which distills knowledge to produce consistent representations from different views of the same image. We employ the ViT-S/16 variant with a patch size of 16 × 16 pixels. However, to prevent overfitting, we use only the first six layers instead of the 12 layers. The distorted image and its reference are encoded by $\mathcal{F}(\cdot)$, resulting in feature vectors $f_r = \mathcal{F}(I_r)$ and $f_d = \mathcal{F}(I_d)$. The encoder's frozen weights ensure that it functions deterministically, where identical images produce the same representation, and deviations due to noise are reflected in the feature space as the shift representation caused by the distortion.

2) **Shift Representation:** The next step is to quantify the noise-induced shift at the representation level between the distorted and reference features. This shift is calculated by

$$ f_n^i = f_r^i - f_d^i, \quad \text{where } i = 0, 1, \dots, N \quad (2) $$

which corresponds to the difference between the i-th patch in the reference test images. Since SSL-trained encoders focus on capturing semantic information rather than distortions, the resulting shift representations, $f_n$, tend to be small, potentially diminishing the effectiveness of the framework. To address this, we insert intermediate linear layers that project $f_n \in \mathbb{R}^{N \times C}$ to a higher dimension, 2C,

followed by a hyperbolic tangent activation function, which scales the values between [-1, 1]. The embeddings are then brought back to the original dimension C through another linear layer, resulting in the projected noise information $f_{np}$. This approach helps prevent the shift representations from having near-zero mean and variance. For symmetry, the same transformation is applied to the reference representations $f_r$ to obtain $f_{rp}$. Both $f_{rp}$ and $f_{np}$ serve as inputs to the QAD $\mathcal{G}(\cdot)$, as detailed in the next section.

## B. Quality-Aware Decoders

In this section, we employ the transformer decoder [22] used for machine translation, excluding the masked self-attention part. Instead, we retain the core idea of using the cross-attention mechanism. In our setup, queries originate from the projected reference representations $f_{rp}$, while keys and values are derived from the projected shift representations $f_{np}$. The cross-attention process facilitates the generation of quality features by aligning pertinent information from the reference with relevant information from the shift features that reflects the effect of the applied noise. As depicted in Figure 1, $\mathcal{G}(\cdot)$ represents the QAD containing the cross-attention mechanism

$$ \text{CA}(q_r, k_n, v_n) = \text{softmax}\left(\frac{q_r k_n^T}{\sqrt{d_k}}\right) v_n, \quad (3) $$

where $q_r$ denotes the queries from $f_{rp}$. Keys $k_n$ and values $v_n$ are calculated from $f_{np}$, and $d_k$ represents the dimension of the key vectors. A linear layer is then applied to the output embedding vectors with a skip connection to preserve the noise information. The output is then projected using a feed-forward neural network, producing a sequence of embedding vectors $g = \mathcal{G}(f_{rp}, f_{np})$ containing quality features. By adopting this approach to discern quality features, attention scores serve as crucial indicators. High attention scores between the corresponding vectors indicate high similarity between the reference and the distorted patch, suggesting a minimal noise effect on the content, where $I_d = D(I_r)$ and $D(\cdot)$ represents the applied noise. In contrast, low attention scores denote significant distortion at specific positions, indicating pronounced content alteration.

## C. Quality Feature Ranking Module

To enhance the learning ability, we incorporate an additional layer for learning to rank image quality without directly predicting the quality itself. This layer is designed to mimic the HVS, which is particularly capable of determining the correct order of quality even when assigning precise numerical values is challenging. We introduce a ranking module (Figure 1), that acts as a loss function, providing feedback on the relative quality ranks at the feature level. Given a batch of M distorted images $I_d^i$ with scores $y^i$, the network generates quality tokens $g_q^i$. We create all possible pairs of quality tokens $(g_q^i, g_q^j)$ where $i \neq j$ and then subtract the pairs to calculate the quality distance between two images at the feature level,

$$ g_q^{ij} = g_q^i - g_q^j. \quad (4) $$

The goal is to learn a function that predicts which image is perceptually better based on the extracted features. The output from Equation 4 is passed through a linear layer $S_{\theta}(\cdot)$ with learnable weights $\theta$, followed by a sigmoid function $\sigma$ to produce the predicted ranking score

$$ \hat{r}^{ij} = \sigma(S_{\theta}(g_q^{ij})). \quad (5) $$

The ground-truth ranking $r^{ij}$ is defined as

$$ r^{ij} = \begin{cases} 1 & \text{if } y^i > y^j \\ 0 & \text{otherwise} \end{cases}, \quad (6) $$
<figure><img src="image_4.png" /><figcaption>Fig. 2. The OEM generator, including reshaping, two convolutional layers with batch normalization, ReLU activation, upsampling, and a final convolutional layer.</figcaption></figure>

and the ranking module is trained using a Binary Cross-Entropy (BCE) loss defined as

$$ \mathcal{L}_{\text{rank}} = -(r^{ij} \log(\hat{r}^{ij}) + (1-r^{ij}) \log(1-\hat{r}^{ij})). \quad (7) $$

The QFR module is first trained on the combined train datasets to improve the robustness of the ranking prediction. The learnable parameters $\theta$ are then frozen on individual datasets, acting as a deterministic differentiable objective function. This approach ensures effective propagation of ranking feedback during back-propagation, enhancing the network's ability to learn quality features for predicting MOS.

## D. Objective Error Map Prediction Model

Since the HVS is sensitive to differences in the high-frequency band [20], [26], regions with high spatial variance are more critical than those with low-frequency. Saliency prediction is often used to guide IQMs towards the most important areas of the image. However, since our encoder, based on DINO pre-trained weights, can automatically detect and segment objects in an image in an unsupervised manner and specific segmentation features [25], we opt to use the OEM as a handcrafted low-level feature to guide the decoder towards regions with high spatial variation.

In order to produce the OEM, we first compute $\hat{I}_r$ and $\hat{I}_d$, which represent the high-frequency components of the reference image $I_r$ and distorted image $I_d$, respectively. This is achieved by subtracting a low-pass filtered version of each image from the original

$$ \hat{I}_r = I_r - \text{LPF}(I_r), \quad \hat{I}_d = I_d - \text{LPF}(I_d). \quad (8) $$

Here, LPF denotes a Low-Pass Filter implemented by downsampling the input image and then upsampling it back to its original size using linear interpolation. The resulting high-passed images are then normalized to the range [-1, 1] after subtraction and scaled by (1/255). The OEM is defined similar to [20] to quantify perceptual error

$$ E = \frac{1}{\log\left(\frac{255^2}{\varepsilon}\right)} \log\left(\frac{1}{\left(\hat{I}_d - \hat{I}_r\right)^2 + \frac{\varepsilon}{255^2}}\right). \quad (9) $$

Here, $\varepsilon$ is a small constant set to 1 and used to avoid division by zero in the denominator. As shown in Figure 3, the first row illustrates OEM examples. We use this OEM in two different scenarios. Let $I_e$ be the OEM produced using $I_r$ and $I_d$. In the first scenario, we generate long-range dependency information by feeding it to the CAE noted as $\mathcal{F}(\cdot)$, producing OEM semantic features $f_e$. These features are then added element-wise to the noise information, resulting in $f_{ne} = f_n + f_e$ (see Section II-A2). Then it is projected using the IL (Figure 1). This addition acts as an amplification of the noise features where the high-frequency band exists. Importantly, this scenario does not increase the number of learnable parameters.

In the second scenario, we use another module after the QAD (Figure 1), which is responsible for generating $I_e$ from the set of

<figure><img src="image_5.png" /><figcaption>Fig. 3. Comparison of the generated OEMs with the ground truth. The first row shows the ground truth OEMs, while the second row illustrates the corresponding generated OEMs. This demonstrates the accuracy and effectiveness of OEM Generator at focusing on differences in high-frequency regions.</figcaption></figure>

quality features $g$ with a 2D shape of $N \times C$. The process begins by reshaping the 2D features into a 3D tensor of dimensions $\frac{H}{P} \times \frac{W}{P} \times C$ (Figure 2). Following this initial transformation, a 3 × 3 convolutional layer is applied, accompanied by batch normalization and ReLU activation. The resultant feature maps undergo upsampling by a factor of 4 using bilinear interpolation to augment their spatial resolution. This process happens twice. Subsequently, a final convolutional layer processes these upsampled features before generating the output. The final output maintains the original OEM's resolution of $H \times W \times 1$. The predicted OEM $\hat{I}_e$ is then compared to the ground-truth OEM $I_e$ using the Mean Absolute Error (MAE) Loss. This helps the QAD in capturing quality features that contains HVS characteristics by focusing on the interested areas (Figure 3).

We present three variants of our proposed approach for IQA, a Lightweight Transformer-based IQA metric with Quality Feature Ranking (LT-QFR); LT-QFR-a, which integrates OEM content features; and LT-QFR-g, focusing on generating OEM from quality features.

## III. TRAINING IMPLEMENTATION

The training process involves preparing the datasets and selecting the appropriate objective functions also, incorporating elastic net regularization [27] for mitigating overfitting.

### A. Datasets

Three FR benchmark datasets are used to train and test our methods. The CSIQ dataset [28] contains 30 source images distorted by six types of impairments, yielding to 866 images, with quality assessments provided as Differential Mean Opinion Scores (DMOS) on a scale of [0, 1]. TID2013 [29] contains 24 images and 25 distortion types, totaling 3000 images, with quality scores given as MOS. KADID-10k [30] includes 10,125 images from 81 pristine images, each subjected to five levels of distortion, rated using a 5-point Degradation Category Ratings (DCR) method. Due to the lack of data we consider only the training and test sets, so, we partitioned the datasets into training (80%) and test (20%) sets based on reference images, ensuring the test set remains unseen during training. For pre-processing, images retained their original sizes with random horizontal and vertical flipping. Quality scores were scaled to [1,5] for consistency across datasets using

$$ q' = 1 + \frac{4 \times (q - q_{\min})}{q_{\max} - q_{\min}} \quad (10) $$

where $q'$ denotes the scaled subjective quality score.
## B. Objective Function

We use the Huber loss as an objective function to assess the predicted quality score. This loss function is commonly employed in machine learning and especially in regression problems. It combines the properties of MSE and MAE, offering a balanced approach between robustness to outliers and sensitivity to small errors [31]. In addition to the Huber loss, we use the QFR as an objective function for training on individual datasets since the weights learned by $S_{\theta}(\cdot)$ are frozen in this case. This is done to enhance the learned quality representation $g$ by providing the QAD a perceptual ranking within a batch of distorted images, as described in Section II-C. To address the overfitting issue, we employ a regularization technique known as elastic net [27]. This method combines Lasso and Ridge regularization approaches, effectively preventing overfitting and serving as a feature selection mechanism in regression models. The regularization loss function is defined as

$$ \mathcal{L}_{reg} = \alpha \|\theta_{\mathcal{R}}\|_{1} + (1 - \alpha) \|\theta_{\mathcal{R}}\|_{2}^{2} \quad (11) $$

where $\theta_{\mathcal{R}}$ represents the learnable parameters of the regressor modules responsible for predicting the MOS, and $\alpha$ is a hyper-parameter set empirically to 0.7 for controlling the strength of the $l_1$ and $l_2$, respectively. By combining all the losses, we formulate a comprehensive loss function

$$ \mathcal{L} = \lambda_0 \mathcal{L}_H + \lambda_1 \mathcal{L}_{rank} + \lambda_2 \mathcal{L}_{reg} \quad (12) $$

with $\mathcal{L}_H$ representing the Huber loss and $\lambda_0, \lambda_1, \lambda_2$ corresponding to the hyper-parameters that signify the influence of each loss term on the overall objective function. We set $\lambda_0$ and $\lambda_1$ to five and $\lambda_2$ to one empirically. This combined loss is used for our two variants LT-QFR and LT-QFR-a. However, for LT-QFR-g, we include an additional term to assess how far the generated OEM $\hat{I}_e$ is from the ground truth $I_e$. To do this, we compute the MAE as

$$ \mathcal{L}_{gen} = \lambda_3 \frac{1}{HW} \| \hat{I}_e - I_e \|_1 \quad (13) $$

with hyper-parameter $\lambda_3$ set to one.

## C. Configuration details

To train the model, we employed the AdamW optimizer [32] with a learning rate set to $5 \times 10^{-4}$ for the QAD $G(\cdot)$ and the regressor $\mathcal{R}(\cdot)$, as well as the intermediate layers. Training was conducted with a batch size of 128 images for 300 epochs. Furthermore, we incorporated a learning rate scheduler, specifically Cosine Annealing [33]. The implementation was carried out using PyTorch on an NVIDIA Tesla V100S-PCIE-32GB GPU.

# IV. EXPERIMENTAL RESULTS

We evaluated the performance of our three model variants—LT-QFR, LT-QFR-a, and LT-QFR-g—across the three datasets using Pearson's Linear Correlation Coefficient (PLCC) and Spearman's Rank Order Correlation Coefficient (SROCC) (Table I), we have also included the base model in which the QFR loss was not used. Our models exhibit competitive performance when compared to other state-of-the-art FR-IQA methods. Specifically, LT-QFR achieved a PLCC of 0.960 and an SROCC of 0.957 on the KADID10k dataset, demonstrating its effectiveness on large-scale datasets and outperforming all previous methods, whether classical, CNN-based, or hybrid learnable methods. The LT-QFR-a variant excelled on the CSIQ dataset, achieving a PLCC of 0.985 and an SROCC of 0.977. Although its performance on the KADID-10k dataset was slightly lower compared to LT-QFR, it still outperformed previous methods. On the TID2013 dataset, both variants achieved performance on- par to the AHIQ [34] and TOPIQ [21], with our models achieving

<table><caption>TABLE I
QUANTITATIVE COMPARISON BETWEEN OUR METHODS AND OTHER FR METHODS IN THE FIELD. OUR MODEL ACHIEVED ON-PAR PERFORMANCE WITH STATE-OF-THE-ART FR-IQMS. RESULTS ARE BORROWED FROM [21], [34], [35]. THE BOLD VALUES REPRESENT THE BEST RESULTS ACROSS ALL MODELS, WHILE THE UNDERLINED VALUES INDICATE THE BEST PERFORMANCE AMONG OUR PROPOSED VARIANTS.</caption><thead><tr><th rowspan="2">Methods</th><th colspan="2">CSIQ</th><th colspan="2">TID2013</th><th colspan="2">KADID10K</th></tr><tr><th>PLCC</th><th>SROCC</th><th>PLCC</th><th>SROCC</th><th>PLCC</th><th>SROCC</th></tr></thead><tbody><tr><td>SSIM [8]</td><td>0.852</td><td>0.865</td><td>0.777</td><td>0.727</td><td>0.645</td><td>0.718</td></tr><tr><td>MS-SSIM [36]</td><td>0.889</td><td>0.906</td><td>0.830</td><td>0.786</td><td>0.686</td><td>0.802</td></tr><tr><td>VSI [37]</td><td>0.928</td><td>0.942</td><td>0.900</td><td>0.897</td><td>0.804</td><td>0.861</td></tr><tr><td>MAD [28]</td><td>0.950</td><td>0.947</td><td>0.827</td><td>0.781</td><td>0.586</td><td>0.724</td></tr><tr><td>GMSD [38]</td><td>0.945</td><td>0.950</td><td>0.855</td><td>0.804</td><td>0.805</td><td>0.847</td></tr><tr><td>SCQI [39]</td><td>0.927</td><td>0.943</td><td>0.907</td><td>0.905</td><td>0.853</td><td>0.854</td></tr><tr><td>LPIPS-VGG [16]</td><td>0.970</td><td>0.967</td><td>0.944</td><td>0.936</td><td>-</td><td>-</td></tr><tr><td>DISTS [40]</td><td>0.973</td><td>0.965</td><td>0.947</td><td>0.943</td><td>-</td><td>-</td></tr><tr><td>DOG [41]</td><td>0.943</td><td>0.954</td><td>0.934</td><td>0.926</td><td>-</td><td>-</td></tr><tr><td>DeepQA [20]</td><td>0.965</td><td>0.961</td><td>0.947</td><td>0.939</td><td>0.891</td><td>0.897</td></tr><tr><td>DualCNN [35]</td><td>-</td><td>-</td><td>0.924</td><td>0.926</td><td>0.949</td><td>0.941</td></tr><tr><td>WaDIQaM [17]</td><td>-</td><td>-</td><td>0.946</td><td>0.940</td><td>0.889</td><td>0.896</td></tr><tr><td>PieAPP [18]</td><td>0.975</td><td>0.973</td><td>0.946</td><td>0.945</td><td>-</td><td>-</td></tr><tr><td>AHIQ [34]</td><td>0.978</td><td>0.975</td><td><strong>0.968</strong></td><td><strong>0.962</strong></td><td>-</td><td>-</td></tr><tr><td>TOPIQ [21]</td><td>0.980</td><td><strong>0.978</strong></td><td>0.958</td><td>0.954</td><td>-</td><td>-</td></tr><tr><td>Base</td><td>0.948</td><td>0.944</td><td>0.942</td><td>0.937</td><td>0.954</td><td>0.951</td></tr><tr><td>LT-QFR</td><td>0.965</td><td>0.966</td><td>0.959</td><td>0.952</td><td><strong>0.960</strong></td><td><strong>0.957</strong></td></tr><tr><td>LT-QFR-a</td><td><strong>0.985</strong></td><td>0.977</td><td>0.957</td><td>0.949</td><td>0.955</td><td>0.952</td></tr><tr><td>LT-QFR-g</td><td>0.977</td><td>0.975</td><td><strong>0.964</strong></td><td><strong>0.960</strong></td><td>0.952</td><td>0.950</td></tr></tbody></table>

this with only 13.98 million parameters compared to AHIQ's 139.3 million parameters. For instance, LT-QFR and LT-QFR-a have a parameter size of 53.36 MB with an inference runtime of 1.26 ms, making it faster (Table II). The third variant, LT-QFR-g, achieved the highest performance among our variants with a PLCC of 0.964 and an SROCC of 0.960 on the TID2013 dataset. On the KADID-10k dataset, LT-QFR-g maintained strong performance with a PLCC of 0.952 and an SROCC of 0.950 as well as in the CSIQ dataset.

<table><caption>TABLE II
MODEL SIZE AND INFERENCE TIME PER IMAGE (PERCENTAGE IN BRACKETS INDICATES THE COMPLEXITY REDUCTION RATE W.R.T. AHIQ).</caption><thead><tr><th>Method</th><th>Total params [M]</th><th>Params size [MB]</th><th>Runtime [MS]</th></tr></thead><tbody><tr><td>AHIQ [34]</td><td>139.3</td><td>557.2</td><td>35.16</td></tr><tr><td>LT-QFR (ours)</td><td>13.98 (89.96%)</td><td>53.36 (90.42%)</td><td>1.26 (96.42%)</td></tr><tr><td>LT-QFR-a (ours)</td><td>13.98 (89.96%)</td><td>53.36 (90.42%)</td><td>4.60 (86.92%)</td></tr><tr><td>LT-QFR-g (ours)</td><td>19.29 (86.15%)</td><td>73.65 (96.59%)</td><td>1.26 (96.42%)</td></tr></tbody></table>

# V. CONCLUSION AND FUTURE WORK

In this paper, we introduced our novel models—LT-QFR, LT-QFR-a, and LT-QFR-g—which demonstrated strong performance across multiple datasets, achieving high PLCC and SROCC scores while significantly reducing the number of parameters compared to the state-of-the-art AHIQ model. Our findings confirm that these models are both accurate and computationally efficient. However, gaining a deeper understanding of the HVS characteristics through subjective tests and exploring the behavior of Transformers for IQA will be essential for further enhancing model robustness. Future work will focus on incorporating more advanced HVS characteristics, such as JND maps, and continuing to improve the effectiveness of IQMs.

# ACKNOWLEDGMENT

This work is partially funded by the Nouvelle-Aquitaine Research Council under project REALISME.
REFERENCES

[1] G. Zhai and X. Min, "Perceptual image quality assessment: a survey," *Sci. China Inf. Sci.*, vol. 63, no. 11, 2020. [Online]. Available: https://doi.org/10.1007/s11432-019-2757-1

[2] X. Min, K. Gu, G. Zhai, X. Yang, W. Zhang, P. Le Callet, and C. W. Chen, "Screen content quality assessment: Overview, benchmark, and beyond," *ACM Comput. Surv.*, vol. 54, no. 9, oct 2021. [Online]. Available: https://doi.orgremotexs.ntu.edu.sg/10.1145/3470970

[3] Z. Zhu, D. Liang, S. Zhang, X. Huang, B. Li, and S. Hu, "Traffic-sign detection and classification in the wild," in *2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016*, Las Vegas, NV, USA, June 27-30, 2016. IEEE Computer Society, 2016, pp. 2110–2118. [Online]. Available: https://doi.org/10.1109/CVPR.2016.232

[4] Y. Lou, Y. Bai, J. Liu, S. Wang, and L. Duan, "Veri-wild: A large dataset and a new method for vehicle re-identification in the wild," in *IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019*, Long Beach, CA, USA, June 16-20, 2019. Computer Vision Foundation / IEEE, 2019, pp. 3235–3243.

[5] S. A. Golestaneh, S. Dadsetan, and K. M. Kitani, "No-reference image quality assessment via transformers, relative ranking, and self-consistency," in *IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2022*, Waikoloa, HI, USA, January 3-8, 2022. IEEE, 2022, pp. 3989–3999. [Online]. Available: https://doi.org/10.1109/WACV51458.2022.00404

[6] M. Tliba, A. Sekhri, M. A. Kerkouri, and A. Chetouani, "Deep-based quality assessment of medical images through domain adaptation," in *2022 IEEE International Conference on Image Processing (ICIP)*. IEEE, 2022, pp. 3692–3696.

[7] Y. W. Lee, D. M. Strong, B. K. Kahn, and R. Y. Wang, "AIMQ: a methodology for information quality assessment," *Inf. Manag.*, vol. 40, no. 2, pp. 133–146, 2002. [Online]. Available: https://doi.org/10.1016/S0378-7206(02)00043-5

[8] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: from error visibility to structural similarity," *IEEE TIP*, 2004.

[9] L. Zhang, L. Zhang, X. Mou, and D. Zhang, "Fsim: A feature similarity index for image quality assessment," *IEEE Transactions on Image Processing (TIP)*, vol. 20, no. 8, pp. 2378–2386, 2011.

[10] M. A. Saad, A. C. Bovik, and C. Charrier, "Blind image quality assessment: A natural scene statistics approach in the DCT domain," *IEEE Trans. Image Process.*, vol. 21, no. 8, pp. 3339–3352, 2012. [Online]. Available: https://doi.org/10.1109/TIP.2012.2191563

[11] A. Mittal, A. K. Moorthy, and A. C. Bovik, "No-reference image quality assessment in the spatial domain," *IEEE Trans. Image Process.*, vol. 21, no. 12, pp. 4695–4708, 2012. [Online]. Available: https://doi.org/10.1109/TIP.2012.2214050

[12] X. Min, K. Gu, G. Zhai, J. Liu, X. Yang, and C. W. Chen, "Blind quality assessment based on pseudo-reference image," *IEEE Transactions on Multimedia*, vol. 20, no. 8, pp. 2049–2062, 2018.

[13] X. Min, G. Zhai, K. Gu, Y. Liu, and X. Yang, "Blind image quality estimation via distortion aggravation," *IEEE Transactions on Broadcasting*, vol. 64, no. 2, pp. 508–517, 2018.

[14] S. A. Amirshahi, M. Pedersen, and S. X. Yu, "Image quality assessment by comparing cnn features between images," *Journal of Imaging Science and Technology*, vol. 60, no. 6, pp. 60 410–1, 2016.

[15] S. A. Amirshahi, M. Pedersen, and A. Beghdadi, "Reviving traditional image quality metrics using cnns," in *Color and Imaging Conference*, vol. 2018, no. 1. Society for Imaging Science and Technology, 2018, pp. 241–246.

[16] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, "The unreasonable effectiveness of deep features as a perceptual metric," in *CVPR*, 2018.

[17] S. Bosse, D. Maniry, K.-R. Müller, T. Wiegand, and W. Samek, "Deep neural networks for no-reference and full-reference image quality assessment," *IEEE TIP*, 2017.

[18] E. Prashnani, H. Cai, Y. Mostofi, and P. Sen, "Pieapp: Perceptual image-error assessment through pairwise preference," in *CVPR*, 2018.

[19] M. Cheon, S.-J. Yoon, B. Kang, and J. Lee, "Perceptual image quality assessment with transformers," in *CVPR*, 2021.

[20] J. Kim and S. Lee, "Deep learning of human visual sensitivity in image quality assessment framework," in *2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017*, Honolulu, HI, USA, July 21-26, 2017. IEEE Computer Society, 2017, pp. 1969–1977. [Online]. Available: https://doi.org/10.1109/CVPR.2017.213

[21] C. Chen, J. Mo, J. Hou, H. Wu, L. Liao, W. Sun, Q. Yan, and W. Lin, "Topiq: A top-down approach from semantics to distortions for image quality assessment," *IEEE Transactions on Image Processing*, 2024.

[22] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, "Attention is all you need," *Advances in neural information processing systems*, vol. 30, 2017.

[23] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn et al., "An image is worth 16x16 words: Transformers for image recognition at scale," *arXiv preprint arXiv:2010.11929*, 2020.

[24] M. Raghu, T. Unterthiner, S. Kornblith, C. Zhang, and A. Dosovitskiy, "Do vision transformers see like convolutional neural networks?" *Advances in Neural Information Processing Systems*, vol. 34, pp. 12116–12128, 2021.

[25] M. Caron, H. Touvron, I. Misra, H. Jégou, J. Mairal, P. Bojanowski, and A. Joulin, "Emerging properties in self-supervised vision transformers," in *Proceedings of the IEEE/CVF international conference on computer vision*, 2021, pp. 9650–9660.

[26] M. Liu, L.-M. Po, X. Xu, K. W. Cheung, Y. Zhao, K. W. Lau, and C. Zhou, "Long-range dependencies and high-order spatial pooling for deep model-based full-reference image quality assessment," *IEEE Access*, vol. 8, pp. 72 007–72 020, 2020.

[27] H. Zou and T. Hastie, "Regularization and variable selection via the elastic net," *Journal of the Royal Statistical Society Series B: Statistical Methodology*, vol. 67, no. 2, pp. 301–320, 2005.

[28] E. C. Larson and D. M. Chandler, "Most apparent distortion: full-reference image quality assessment and the role of strategy," *J Electron Imaging*, 2010.

[29] N. Ponomarenko, L. Jin, O. Ieremeiev, V. Lukin, K. Egiazarian, J. Astola, B. Vozel, K. Chehdi, M. Carli, F. Battisti et al., "Image database tid2013: Peculiarities, results and perspectives," *Signal Process Image Commun*, 2015.

[30] H. Lin, V. Hosu, and D. Saupe, "Kadid-10k: A large-scale artificially distorted iqa database," in *QoMEX*, 2019.

[31] P. Huber, "Robust estimation of a location parameter," in *Breakthroughs in statistics: Methodology and distribution*. Springer, 1992, pp. 492–518.

[32] I. Loshchilov and F. Hutter, "Decoupled weight decay regularization," *arXiv preprint arXiv:1711.05101*, 2017.

[33] ——, "Sgdr: Stochastic gradient descent with warm restarts," *arXiv preprint arXiv:1608.03983*, 2016.

[34] S. Lao, Y. Gong, S. Shi, S. Yang, T. Wu, J. Wang, W. Xia, and Y. Yang, "Attentions help cnns see better: Attention-based hybrid image quality assessment network," in *IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2022*, New Orleans, LA, USA, June 19-20, 2022. IEEE, 2022, pp. 1139–1148. [Online]. Available: https://doi.org/10.1109/CVPRW56347.2022.00123

[35] D. Varga, "Composition-preserving deep approach to full-reference image quality assessment," *Signal Image Video Process.*, vol. 14, no. 6, pp. 1265–1272, 2020. [Online]. Available: https://doi.org/10.1007/s11760-020-01664-w

[36] Z. Wang, E. P. Simoncelli, and A. C. Bovik, "Multiscale structural similarity for image quality assessment," in *ACSSC*, 2003.

[37] L. Zhang, Y. Shen, and H. Li, "Vsi: A visual saliency-induced index for perceptual image quality assessment," *IEEE TIP*, 2014.

[38] W. Xue, L. Zhang, X. Mou, and A. C. Bovik, "Gradient magnitude similarity deviation: A highly efficient perceptual image quality index," *IEEE TIP*, 2013.

[39] S.-H. Bae and M. Kim, "A novel image quality assessment with globally and locally consistent visual quality perception," *IEEE TIP*, 2016.

[40] K. Ding, K. Ma, S. Wang, and E. P. Simoncelli, "Image quality assessment: Unifying structure and texture similarity," *IEEE TPAMI*, 2022.

[41] S. Pei and L. Chen, "Image quality assessment using human visual DOG model fused with random forest," *IEEE Trans. Image Process.*, vol. 24, no. 11, pp. 3282–3292, 2015. [Online]. Available: https://doi.org/10.1109/TIP.2015.2440172