# A Combined Full-Reference Image Quality Assessment Method Based on Convolutional Activation Maps
## 1 Introduction
### Related Work
Moreover, the presented network had one output that predicted the image quality score. First, the input distorted and reference images were decomposed into \(224\times 224\)-sized image patches and the quality of each image pair was predicted independently of each other. Finally, the overall image quality was determined by averaging the scores of the image pairs. [PERSON] and [PERSON] [25] introduced a similar dual-path CNN but their model accepts a distorted image and an error map calculated from the reference and the distorted image as inputs. Furthermore, it generates a visual sensitivity map which is multiplied by an error map to predict perceptual image quality. Similarly to the previous algorithm, the inputs are also decomposed into smaller image patches and the overall image quality is determined by the averaging of the scores of distorted patch-error map pairs. Recently, generic features extracted from different pretrained CNNs, such as AlexNet [4] or GoogLeNet [2], have been proven very powerful for a wide range of image processing tasks. [PERSON] et al. [6] applied feature vectors extracted from the OverFeat [26] network, which was trained for object classification on ImageNet ILSVRC 2013 [5], to carry out image classification, scene recognition, fine-grained recognition, attribute detection, and content-based image retrieval. The authors reported on superior results compared to those of traditional algorithms. Later, [PERSON] et al. [27] pointed out that feature vectors extracted from pretrained CNNs outperform traditional image quality metrics. Motivated by the above-mentioned results, a number of FR-IQA algorithms have been proposed relying on different deep features and pretrained CNNs. [PERSON] et al. [28] compared different activation maps of the reference and the distorted image extracted from AlexNet [4] CNN. Specifically, the similarity of the activation maps was measured to produce quality sub-scores. Finally, these sub-scores were aggregated to produce an overall quality value of the distorted image. In contrast, [PERSON] et al. [29] extracted deep features with the help of a VGG16 [30] network from reference and distorted image patches. Subsequently, the distorted and the reference deep feature vectors were fused together and mapped onto patch-wise quality scores. Finally, the patch-wise scores were pooled, supplementing with a patch weight estimation procedure to obtain the overall perceptual quality. In our previous work [31], we introduced a composition preserving deep architecture for FR-IQA relying on a Siamese layout of pretrained CNNs, feature pooling, and a feedforward neural network. Another line of works focuses on creating combined metrics where existing FR-IQA algorithms are combined to achieve strong correlation with the subjective ground-truth scores. In [32], [PERSON] examined the properties of three FR-IQA metrics (MS-SSIM [14], VIF [33], and R-SVD [34]), and proposed a combined quality metric based on the arithmetical product and power of these metrics. Later, this approach was further developed using optimization techniques [35; 36]. Similarly, [PERSON] [37] selected 16 FR-IQA metrics and applied their scores as predictor variables in a lasso regression model to obtain a combined metric. [PERSON] et al. [38] took a similar approach, but kernel ridge regression was utilized to fuse the scores of the IQA metrics. In contrast, [PERSON] et al. [39] fused the results of six metrics with the help of a neural network. [PERSON] [40] carried out a decision fusion based on 16 FR-IQA measures by minimizing the root mean square error of prediction performance with a genetic algorithm.