# No-Reference Image Quality Assessment Based on a Multitask Image Restoration Network
## 3 Proposed Method
### Multitask Image Restoration Sub-Network  
The quality of pseudo-reference images is crucial for the performance of the model. MT-IRN utilizes a multitask image restoration network to generate higher-quality pseudo-reference images, thereby improving the model's performance. Inspired by the BPSQM, the multitask image restoration sub-network takes the generation of pseudo-reference images as the main task and the generation of SSIM maps as an auxiliary task, generating higher-quality pseudo-reference images through the mutual promotion of the two goals. The SSIM map provides rich structural information for the reference image, generating the auxiliary task that allows the image restoration sub-network to learn the structural information of the reference image, and thus improving the structural similarity.  
MT-IRN employs a U-Net architecture [34, 35, 36] to construct the image restoration sub-network. U-Net is an encoder-decoder structure widely used in the fields of image segmentation and restoration. Its skip connection structure concatenates low-level features with high-level features, allowing the decoder to retain more detailed information during the up-sampling process. The structure of the multitask image restoration sub-network is illustrated in Figure 2. The multitask learning mode adopts a hard sharing mode [37, 38], in which the two tasks share the parameters of a shallow layer, and then different convolutional layers are set for each task to achieve their respective goals.  
The multitask image restoration sub-network accepts the distorted image as input and generates the pseudo-reference image, the SSIM map, and the quality restoration features F1, F2, and F3 as its outputs. The encoder consists of six convolutional modules, E1-E6. E1 is a single-layer convolutional layer composed of 3 \(\times\) 3 convolution kernels with a stride of 1. To avoid gradient vanishing [39] when deepening the model and to reuse low-level features, E2-E6 use residual convolutional blocks for downsampling. The specific structures of two types of residual blocks are shown in Figure 3. E2 employs a residual block 1 architecture, comprising two 3 \(\times\) 3 convolutional layers with stride 1. Notably, this architecture preserves the size of the input. For E3-E6, they employ the  
Figure 1: The architecture of the MT-IRN.  
Figure 2: The architecture of the multitask image restoration sub-network.  
residual block 2 architecture, which comprises a 3 \(\times\) 3 convolutional layer with stride 2 and another 3 \(\times\) 3 convolutional layer with stride 1. Consequently, the output size is reduced by half compared to the input size Since the sizes of the input and output feature maps are inconsistent, a 1 \(\times\) 1 convolution with a stride of 2 is performed on the input during the residual connection to match their sizes.  
The decoder consists of six deconvolutional layers, D1-D6, which perform upsampling on the high-level features of distorted images to generate the pseudo-reference image and SSIM map. Through multi-level skip connections, the decoder effectively preserves the detailed information in the input image while avoiding the loss of feature details caused by pooling layers, thus improving the image restoration performance. D1-D6 are all composed of 3 \(\times\) 3 deconvolutional layers, where the stride of D1-D4 is 2, and the stride of D5 and D6 is 1. The specific structural parameters of the image restoration sub-network are summarized in Table 1.