CPDINet: Blind image quality assessment via a content perception and distortion inference network

IET Image ProcessingEarly View ORIGINAL RESEARCH PAPEROpen Access CPDINet: Blind image quality assessment via a content perception and distortion inference network Xiao Shao, Xiao Shao orcid.org/0000-0003-1194-240X School of Computer Science, Nanjing University of Information Science and Technology, Nanjing, People's Republic of ChinaSearch for more papers by this authorMengqing Liu, Mengqing Liu School of Information Engineering, Nanchang Hangkong University, Nanchang, People's Republic of ChinaSearch for more papers by this authorZihan Li, Zihan Li School of Computer Science, Nanjing University of Information Science and Technology, Nanjing, People's Republic of ChinaSearch for more papers by this authorPeiyun Zhang, Corresponding Author Peiyun Zhang zpy@nuist.edu.cn orcid.org/0000-0002-4864-6279 School of Computer Science, Nanjing University of Information Science and Technology, Nanjing, People's Republic of China Correspondence Peiyun Zhang, School of Computer Science, Nanjing University of Information Science and Technology, Nanjing 210044, People's Republic of China. Email: zpy@nuist.edu.cnSearch for more papers by this author Xiao Shao, Xiao Shao orcid.org/0000-0003-1194-240X School of Computer Science, Nanjing University of Information Science and Technology, Nanjing, People's Republic of ChinaSearch for more papers by this authorMengqing Liu, Mengqing Liu School of Information Engineering, Nanchang Hangkong University, Nanchang, People's Republic of ChinaSearch for more papers by this authorZihan Li, Zihan Li School of Computer Science, Nanjing University of Information Science and Technology, Nanjing, People's Republic of ChinaSearch for more papers by this authorPeiyun Zhang, Corresponding Author Peiyun Zhang zpy@nuist.edu.cn orcid.org/0000-0002-4864-6279 School of Computer Science, Nanjing University of Information Science and Technology, Nanjing, People's Republic of China Correspondence Peiyun Zhang, School of Computer Science, Nanjing University of Information Science and Technology, Nanjing 210044, People's Republic of China. Email: zpy@nuist.edu.cnSearch for more papers by this author First published: 15 March 2022 https://doi.org/10.1049/ipr2.12463AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinked InRedditWechat Abstract Nowadays, it is still challenging for the blind image quality assessment (BIQA) to accurately predict quality scores of distorted images, since distorted images have rich content information and complex distortions. To solve these problems, a content perception and distortion inference network for BIQA is proposed, which divides IQA task into content perception and distortion inference processes. Since humans try to understand image content before perceiving quality scores, a content feature extractor is designed to explore content information in an image to deal with the content variation problem. To handle the distortion diversity problem, a distortion feature extractor is proposed to capture distortion features in images. Because extracted content features and distortion ones have different characteristics, attention-based fusion blocks to fuse multi-scale content features and distortion ones as guidance to selectively enhance important features based on calculated weight scores are proposed. With fused features, a quality prediction module is designed to regress multi-scale features to quality scores. Experiments are performed on six public IQA datasets, including LIVE, CSIQ, TID2013, LIVEC, KonIQ-10k, and SPAQ. Experimental results show that the proposed method can effectively predict quality scores for both synthetically and authentically distorted images than its peers, including the state-of-the-art methods. 1 INTRODUCTION 1.1 Background Objective image quality assessment (IQA) is a fundamental computer vision task that aims to automatically predict image quality scores without human intervention. With the development of social media and smart phones in recent years, a huge number of digital images need to be processed. However, distortions may occur in an image during the procedure of acquisition, transmission, compression, and shooting process of camera devices. In general, images suffer from two types of distortions: synthetic distortions and authentic ones. The former is generated in post-processing operations, such as white noise, JPEG, and JP2K. The latter is introduced during the processing of camera devices, such as motion blur, over-exposure, and compression error. Although subjective image quality assessment methods are the most reliable way to measure image quality, it is time-consuming and labourious, which makes it impractical in real application scenarios. Hence, it is urgent to develop an effective computer algorithm to automatically predict image quality scores, which is known as the objective IQA method. According to the use of reference information, the objective IQA can be divided into three categories: full-reference IQA (FR-IQA) [1-5], reduce-rReference IQA (RR-IQA) [6], and no-reference IQA (NR-IQA) [7-10]. With full or partial reference information for comparison, these FR-IQA and RR-IQA methods have achieved great prediction accuracy over the past decades. NR-IQA, i.e. blind IQA (BIQA), has attracted the attention of a growing number of researchers. Generally, BIQA methods can be divided into hand-crafted-based BIQA methods [11]–[14] and learning-based ones [15]-[31]. The former aims to extract well-designed hand-crafted features, in which a shallow regression model (e.g. support vector machine) is applied to regress these features to a quality score. However, these hand-crafted features are shallow and incapable of representing complex distortions in the real world, which makes them have unsatisfactory prediction accuracy. To solve the problem, many learning-based BIQA methods are proposed by utilising the powerful feature representation ability of convolutional neural networks (CNNs). Benefiting from large training samples, learning-based features automatically learned by CNNs can represent more abundant distortions than hand-crafted-based features. However, distorted images have rich content information and complex distortions, it is still a challenging task for existing BIQA methods to accurately predict quality scores. 1.2 Motivations 1.2.1 To handle the content variation and distortion diversity problem Current BIQA methods [15-21, 26, 28, 30] aim to map the relationship between distortion features and quality scores by using a simple end-to-end CNN. These methods focus on extracting distortion features and ignore content information in distorted images, which degrades the accuracy of image quality prediction. For the human visual system (HVS), when a human receives a distorted image, the perceptive quality is not only affected by distortions (e.g. white noise and JPEG compression) in an image, but also by content information (e.g. salient objects and object shape). For example, humans may consider an image with blue sky has better quality than one with a blurry animal [38]. As shown in Figure 1, although the left image has more serious distortions than the right one, its MOS value is higher than the right one. The reason is that content information in the left image may be pleasing to more people than the right one. To effectively extract content information, a content feature extractor is designed to encode distorted images to multi-scale content features. Since semantic features of pre-trained image classification networks contain rich discriminative image content information, we adopt a pre-trained ResNet-50 [40] to extract image content features. FIGURE 1Open in figure viewerPowerPoint Illustration of the effect of different image content. A higher MOS value denotes a higher image quality Since the content feature extractor is designed to extract content features in images, but it is not sensitive to synthetic distortions (e.g. white Gaussian noise, Gaussian blur, and JPEG compression). As shown in Figure 2, images may suffer from various distortions, e.g. from JP2K to CC distortions. To solve the problem, a distortion feature extractor is proposed to effectively learn distortion information in images, which is firstly pre-trained on a synthetic distortion dataset to fully learn distortion features. FIGURE 2Open in figure viewerPowerPoint Example of six distortion types of distorted images in CSIQ dataset 1.2.2 To handle the fusing problem of different features As mentioned before, humans perceive quality scores depend on content information and distortion one. To explore the effect of different content information and distortion one in quality prediction, the statistical results are shown in Figure 3. FIGURE 3Open in figure viewerPowerPoint The 24 distortion types against image MOS variance under the change of reference images in TID2013. “distortion type” denotes 24 distortion types in TID2013. “MOS variance” represents image MOS variance of 24 distortion types under the change of reference images To verify content information and distortion one play different roles in quality process, MOS variances of 24 distortion types under the change of different image content are calculated to measure the dispersion of MOS values. Let s ¯ $\overline{s}$ and μ $\mu$ denote the mean and variance value of all MOS values, respectively: s ¯ = 1 n ( s 1 + s 2 + ⋯ + s n ) , μ = 1 n ∑ i = 1 n ( s i − s ¯ ) 2 , \begin{equation} {\left\lbrace \def\eqcellsep{&}\begin{array}{l}\overline{s}=\dfrac{1}{n}(s_{1}+s_{2}+\cdots +s_{n}),\\[3pt] \mu =\dfrac{1}{n}\displaystyle\sum _{i=1}^{n}(s_{i}-\overline{s})^{2}, \end{array} \right.} \end{equation} (1)where s i $s_{i}$ ( i ∈ { 1 , … , n } $i \in \lbrace 1,\ldots ,n\rbrace$ ) denote MOS value of n $n$ images with different content information under the same distortion type. From Figure 3, it can be observed that for different distortion types, different reference images with content variation have different effects on MOS values. For some distortion types (e.g. JP2K and SSR), content information has a great influence on MOS values, which results in a higher MOS variance than others. On the contrary, for partial distortions (e.g. IN and Block), content information has a small influence on MOS values. These observations verify that content information and distortion one have different influence on MOS values. If the extracted content features and distortion one are directly fused without adjustment, networks can fail to focus on important features based on different images. To solve the problem, attention-based fusion blocks are proposed to adaptively fuse multi-scale content features and distortion ones according to their weights. 1.3 Contributions This work has the following contributions: To effectively predict quality scores for synthetically and authentically distorted images, a universal BIQA method is proposed, which divides the IQA process into content perception and distortion inference processes. To understand the image content information, a content feature extractor is applied to extract multi-scale content features. To fully learn abundant distortion information, a distortion feature extractor is designed to extract multi-level distortion features, which hierarchical perception mechanism is more consistent with human perception. To effectively fuse extracted content features and distortion ones, attention-based fusion blocks are proposed to calculate the weight scores of content features and distortion ones. In each attention-based fusion block, a multi-scale spatial attention module (MSSAM) is proposed to adaptively extract attention features at different scales. By adaptively combining content features and distortion ones, it can achieve a further improvement in prediction accuracy. For synthetically and authentically distorted images, the proposed method can achieve satisfactory prediction accuracy. Experimental results demonstrate its superiority. The rest of the paper is organised as follows. Section 2 introduces the related work. Section 3 describes the proposed method in details. Section 4 presents the experimental results. Finally, Section 5 concludes the paper. 2 RELATED WORK In this section, we briefly review the recent BIQA methods, including classic hand-crafted-based BIQA methods and learning-based ones. 2.1 The classic hand-crafted-based BIQA methods The hand-crafted-based BIQA methods aim to design hand-crafted features to reflect image property (e.g. texture, colour, and structure) and detect quality degradation of images. These well-designed features are commonly inspired by the human visual system (HVS) and natural scene statistics (NSS). Then, a shallow regression model (e.g. support vector machine) is applied to regress these features to a quality score. Moorthy et al. [8] propose a BIQA method to extract natural scene statistics from the wavelet transform domain, which first identifies distortion types. Li et al. [11] propose a BIQA method based on the discrete moments, which is based on the gradient information of distorted images. Wu et al. [12] propose a KNN-based BIQA model, in which a novel feature fusing scheme is proposed by fusing the information from multiple domains and colour channels. To accurately predict image quality for graphic and screen content images, Min et al. [13] propose a unified BIQA method, in which an adaptive multi-scale weighting mechanism is proposed to adaptively fuse multi-scale features to a quality score. To effectively extract contrast distortions in a distorted image, Yue et al. [14] propose a BIQA method by fusing entropy information and K-L divergence into quality prediction process. Although these hand-crafted-based BIQA methods can achieve well predictions for specific distortions, it is difficult to design hand-crafted features. In addition, the regression model is shallow. These hand-crafted-based BIQA methods are hard to represent complex distortions (e.g. motion blur and ghosting) in the real world, which result in insufficient prediction accuracy. 2.2 The learning-based BIQA methods Recently, deep learning has achieved more attention in many image processing fields [41, 42]. The advantage of deep learning benefits from the powerful representation ability of convolutional neural networks (CNN). Due to the strong representation capability of CNNs, many learning-based BIQA methods are proposed to automatically extract features in an end-to-end manner by using convolutional operations. These features learned by convolutional layers can represent more complex distortions. Kang et al. [15] firstly apply the CNN to IQA task, which has a shallow network with pooling layers to predict quality scores. Kim et al. [16] propose an efficient BIQA method that first learns a local quality map from an FR-IQA method. Then, the pre-trained network is fine-tuned to predict quality scores. To enhance the learning ability of quality prediction, Ma et al. [17] propose a multi-task CNN to identify distortion types and predict image quality simultaneously. In WaDIQaM [18], Bosse et al. propose a deep CNN with ten convolutional layers to extract distortion features and apply fully connected (FC) layers to regress these features to quality scores. In addition, to solve the problem of unbalance scores between patches, a weight module with FC layers is proposed to calculate the weight score for each image patch. To augment the training samples, Liu et al. [19] generate plenty of synthetically distorted images to augment the training data. Then, a CNN is applied to learn to rank these generated distorted images. Finally, the pre-trained CNN is fine-tuned for quality prediction process. Similar to ref. [19], Gu et al. [20] augment the training samples with generated synthetically distorted images, in which the reinforcement learning is used to train the network. To measure the difference between distorted images and reference ones, Pan et al. [21] first predict a similar quality map by using a fully CNN. Then, the predicted quality map is used to guide image quality prediction. Instead of regressing extracted features to quality scores, Talebi et al. [22] predict the distribution of human opinion scores by using an end-to-end CNN. To compensate for the lack of reference images, Ren et al. [23] and Lin et al. [24] use a generative adversarial network (GAN) to generate distortion-free images. The generated images and distorted ones are used to measure quality degradation caused by distortions. Bongini et al. [25] also propose a GAN-based BIQA method, in which a GAN is applied to generate distorted images to augment the training samples. Kim et al. [26] propose a CNN-based BIQA by predicting the error map to enhance the learning ability of quality prediction. To solve the problem for single scalar fails to reveal the subjective diversity, Jiang et al. [27] propose a model to predict the empirical score distribution and quality scores simultaneously. To detect structural degradation caused by distortions, Yan et al. [28] propose a two-stream CNN to extract features from the spatial and gradient domains. Yang et al. [29] propose a two-tasks saliency-guided BIQA method, which jointly optimises visual saliency prediction and quality prediction with a shared network. Inspired by the hierarchical perception mechanism of HVS, Wu et al. [30] propose a lightweight CNN to extract multi-level degradation features. Finally, these learned multi-level features are regressed to a quality score in an end-to-end manner. To effectively predict quality scores for both synthetic and authentic distortions, Zhang et al. [32] train the model by using both synthetically and authentically distorted images. The fidelity loss is applied to optimise the network. To enhance the generalisation ability, Zhu et al. [33] propose a meta-learning-based BIQA method, in which the meta-learning is used to learn a quality prior model. Then the network is fine-tuned for BIQA of unknown distortions. Inspired by human learning, Zhang et al. [34],[35] utilise continual learning for BIQA task to effectively handle the problem of dataset expansion. The IQA model is trained by a stream of datasets, and the model is built on the previously learned data. Although these BIQA methods achieve satisfactory prediction accuracy, they lack the ability to effectively extract content information in images, which is critical to quality prediction. To solve this problem, methods [38, 39] use a pre-trained image classification network to understand image content information first before predicting a quality score. Although these methods are consistent with human perception, the image classification network lacks sensitivity to distortion information, which allows these methods to be further enhanced. In this paper, we propose a different way to perceive image quality by dividing the IQA process into content perception and distortion inference processes. Previous methods use pre-trained image classification networks to extract content information, which lacks the extraction of synthetic distortion. We design a content feature extractor and a distortion one to fully learn both content information and synthetic distortion one. Considering that extracted content features and distortion ones play different roles in quality prediction, we propose an attention-based fusion block to adaptively weigh different content features and distortion ones. By combining adaptively weighted content features and distortion ones, the proposed method can improve prediction accuracy. 3 PROPOSED METHOD To achieve a universal BIQA method, we propose a content perception and distortion inference network for blind image quality assessment, which can effectively predict quality scores for synthetically and authentically distorted images. The architecture of the proposed method is shown in Figure 4, which consists of four parts: A content feature extractor, a distortion feature extractor, an adaptive fusion module, and a quality prediction module. The content feature extractor and distortion one are adopted to extract content features and distortion ones in an image. The adaptive fusion module aims to adaptively fuse captured content features and distortion ones, which consists of four attention-based fusion blocks. The quality prediction module is designed to learn the relationship between fused features and quality scores. The details of the proposed method are introduced as follows. FIGURE 4Open in figure viewerPowerPoint The network architecture of the proposed method We firstly feed a distorted image into the content feature extractor (CFE) to capture multi-scale content-aware features. To compensate for the insensitivity to distortions, we also feed input into the distortion feature extractor (DFE) to extract multi-scale distortion-aware features. Let C i $C_{i}$ and D i $D_{i}$ ( i ∈ { 1 , 2 , 3 , 4 } $i \in \lbrace 1,2,3,4\rbrace$ ) denote multi-scale content features and distortion ones, respectively: C i = G C ( I d ) , \begin{align} C_{i}=G_{C}(I_{d}), \end{align} (2) D i = G D ( I d ) , \begin{align} D_{i}=G_{D}(I_{d}), \end{align} (3)where I d $I_{d}$ , G C ( · ) $G_{C}(\cdot )$ , and G D ( · ) $G_{D}(\cdot )$ represent a distorted image, CFE, and DFE, respectively. For the attention-based fusion block (AFB), it aims to calculate the weights of C i $C_{i}$ and D i $D_{i}$ ( i ∈ { 1 , 2 , 3 , 4 } $i \in \lbrace 1,2,3,4\rbrace$ ). The pooling operation in AFB would perform average-pooling and max-pooling operations along the channel axis to obtain two efficient feature descriptors with size H × $\,{\times}\,$ W × $\,{\times}\,$ 1. However, directly mapping the C i $C_{i}$ and D i $D_{i}$ ( i ∈ { 1 , 2 , 3 , 4 } $i \in \lbrace 1,2,3,4\rbrace$ ) with channels {256, 512, 1024, 2048} to two channels results in a sharp reduction in the channel dimension, which causes a lot of information loss and affects the performance of the AFB. Since the 1 × $\times$ 1 convolutional layer can effectively extract non-sparse global features with high-intensity information, features compressed by 1 × $\times$ 1 convolutional layer can retain more useful image information. To effectively compress the feature dimension and retain most of the relevant features, eight 1 × $\times$ 1 convolutional layers are applied for C i $C_{i}$ and D i $D_{i}$ to extract more valuable features. Let C i ′ $C_{i}^{^{\prime }}$ and D i ′ $D_{i}^{^{\prime }}$ ( i ∈ { 1 , 2 , 3 , 4 } ) $(i \in \lbrace 1,2,3,4\rbrace )$ be content features and distortion ones processed by 1 × $\times$ 1 convolutional layers, respectively: C i ′ = f s ( C i ) , \begin{gather} C_{i}^{^{\prime }}=f_{\text{s}}(C_{i}), \vspace{4.67502pt}\end{gather} (4) D i ′ = f s ( D i ) , \begin{gather} D_{i}^{^{\prime }}=f_{\text{s}}(D_{i}), \end{gather} (5)where f s ( · ) $f_{\text{s}}(\cdot )$ denotes 1 × $\times$ 1 convolutional layer. Since C i ′ $C_{i}^{^{\prime }}$ and D i ′ $D_{i}^{^{\prime }}$ are different, directly fusing these features can decrease the network's learning ability. To solve the problem, the adaptive fusion module uses four proposed attention-based fusion blocks (AFB) to adaptively fuse these features. Let F m i $F_{m}^{i}$ and F i ′ $F_{i}^{^{\prime }}$ ( i ∈ { 1 , 2 , 3 , 4 } ) $(i \in \lbrace 1,2,3,4\rbrace )$ denote concatenated features and AFB's output features, respectively: F m i = C i ′ ⊗ D i ′ , \begin{gather} F_{m}^{i}=C_{i}^{^{\prime }}\otimes D_{i}^{^{\prime }}, \end{gather} (6) F i ′ = G F ( C i ′ , D i ′ , F m i ) , \begin{gather} F_{i}^{^{\prime }}=G_{F}(C_{i}^{^{\prime }}, D_{i}^{^{\prime }}, F_{m}^{i}), \end{gather} (7)where G F ( · ) $G_{\text{F}}(\cdot )$ and ⊗ $\otimes$ denote the adaptive fusion module and a concatenation operator, respectively. Finally, the quality prediction module takes fused features as input and regresses them to a quality score. Let y d $y_{d}$ represent a quality score: y d = G R ( F 1 ′ , F 2 ′ , F 3 ′ , F 4 ′ ) , \begin{align} y_{d}=G_{\text{R}}(F_{1}^{^{\prime }},F_{2}^{^{\prime }},F_{3}^{^{\prime }},F_{4}^{^{\prime }}), \end{align} (8)where G R ( · ) $G_{\text{R}}(\cdot )$ denotes the quality prediction module. 3.1 Content feature extractor For HVS, the top-down perception mechanism verifies that humans can first understand image content before perceiving image quality [38]. To mimic the working mechanism of HVS, Li et al. [38] adopt a pre-trained image classification network to extract semantic information, in which features from the last convolutional layer are regressed to a quality score. The underlying assumption is that image classification networks are designed and trained to discriminate the objects in an image, which makes the semantic features of pre-trained image classification networks contain abundant content information. The high-level features from the last convolutional layers only represent global information. The early layers of CNN contain rich detail information, e.g. local details and texture information. Neglecting to extract local detail information can degrade the accuracy of quality predictions. To effectively extract content aware features, the ResNet-50 [40] is adopted as the content feature extractor (CFE), which is pre-trained on ImageNet [44]. To capture both local and global features, we extract multi-level features from conv2_10, conv3_12, conv4_18, and conv5_9 in ResNet-50 instead of only using high-level semantic features from the last convolutional layer. 3.2 Distortion feature extractor The CFE is applied to extract content aware features from images for its inherent property, but it lacks the ability to extract synthetic distortions. To effectively learn distortion information in images, a distortion feature extractor (DFE) is proposed. To make the DFE sensitive to distortion information, we pre-train the DFE on the LSQA [30] to learn abundant distortion features. Since HVS is the hierarchical perception mechanism, we extract multi-level features to measure quality degradation. The pre-training process of the proposed DFE is shown in Figure 5. It consists of a ResNet-50 backbone module to extract features at different levels, a quality perception module to measure hierarchical quality degradation, and a quality regression module to regress fused features to a quality score. The backbone module firstly extracts basic features from a distorted image I d $I_{d}$ from low-level to high-level. Let f i $f_{i}$ ( i ∈ { 1 , 2 , 3 , 4 } ) $(i \in \lbrace 1,2,3,4\rbrace )$ denote multi-level features extracted from the ResNet-50: f i = F D ( I d ) , \begin{equation} f_{i}=F_{\text{D}}(I_{d}),\vspace{4.67502pt} \end{equation} (9)where F D ( · ) $F_{\text{D}}(\cdot )$ represents the ResNet-50 backbone module. Then, the quality perception module, which consists of four branches of 3 × $\times$ 3 convolutional layers and pooling layers, is applied to extract features from multi-level features f 1 $f_{1}$ , f 2 $f_{2}$ , f 3 $f_{3}$ , and f 4 $f_{4}$ . Let M i $M_{i}$ ( i ∈ { 1 , 2 , 3 , 4 } ) $(i \in \lbrace 1,2,3,4\rbrace )$ denote output features of the quality perception module: M i = F P ( f i ) , \begin{equation} M_{i}=F_{\text{P}}(f_{i}), \vspace{4.67502pt} \end{equation} (10)where F P ( · ) $F_{\text{P}}(\cdot )$ represents the quality perception module. Finally, to obtain a predicted quality score, the quality regression module is applied to regress fused features to a quality score. Let M f $M_{f}$ and q $q$ represent fused features and a quality score, respectively: M f = M 1 ⊗ M 2 ⊗ M 3 ⊗ M 4 , \begin{equation} M_{f}=M_{1}\otimes M_{2}\otimes M_{3}\otimes M_{4}, \end{equation} (11) q = F R ( M f ) , \begin{equation} q=F_{\text{R}}(M_{f}), \end{equation} (12)where F R ( · ) $F_{\text{R}}(\cdot )$ and ⊗ $\otimes$ represent the quality regression module and a concatenation operator, respectively. In order to learn the relationship between distortion features and ground truth quality scores, the accuracy of prediction performance is improved by minimising the loss function. Let l d $l_{d}$ denote the loss function: l d = 1 N ∑ i = 1 N ∥ G D I d ; θ − y g ∥ l 1 , \begin{equation} l_{d}=\frac{1}{N}\sum _{i=1}^{N}\parallel G_{D}{\left(I_{d};\theta \right)}-y_{g}\parallel _{l_{1}}, \end{equation} (13)where G D ( · ) $G_{\text{D}}(\cdot )$ , θ $\theta$ , y g $y_{g}$ , and l 1 $l_{1}$ denote the DFE, the set of network parameters in DFE, the ground truth score of the distorted image I d $I_{d}$ , and the l 1 $l_{1}$ -norm, respectively. After the pre-training process of the proposed DFE is completed, the quality perception module and the quality regre