# A Combined Full-Reference Image Quality Assessment Method Based on Convolutional Activation Maps
## 3 Experimental Results
### Comparison to the State-of-the-Art  
Our proposed algorithm was compared to several state-of-the-art FR-IQA metrics, including SSIM [12], MS-SSIM [14], MAD [49], GSM [56], HaarPSI [20], MDSI [57], CSV [58], GMSD [19], DSS [59], VSI [60], PerSIM [61], BLESS-SR-SIM [62], BLESS-FSIM [62], BLESS-FSIMc [62], LCSIM1 [40], ReSIFT [63], IQ(\(\mathcal{L}_{T}\)) [28], MS-UNIQUE [64], RVSIM [65], 2 stepQA [66], SUMMER [67], CEQI [68], CEQIc [68], VCGS [69], and DISTS [70], whose original source code are available online. Moreover, we reimplemented SSIM CNN [41] in MATLAB R2019a (Available : [[https://github.com/Skythianos/SSIM-CNN](https://github.com/Skythianos/SSIM-CNN)]([https://github.com/Skythianos/SSIM-CNN](https://github.com/Skythianos/SSIM-CNN))). For learning-based approaches, we retrained the models using exactly the same database partition (approx. 80% for training and 20% for testing with respect to the reference images to avoid semantic overlap) that we used for our method. Since the feature extraction part of LCSIM1 [40] was not given, we can report the results of LCSIM1 on TID2013 [53], CSIQ [49], and TID2008 [46] databases. Furthermore, mean PLCC, SROCC, and KROCC values were measured over 100 random train-test splits for machine learning-based algorithms. In contrast, traditional FR-IQA metrics are tested on the whole database, and we report on the PLCC, SROCC, and KROCC values. Besides FR-IQA methods, some recently published deep learning-based NR-IQA algorithms, including DeepFL-IQA [42], BLINDER [43], RankIQA [71], BPSOM-MD [72], and NSSADNN [73], have been added to our comparison. Due to the difficulty of reimplementation of deep NR-IQA methods, the performance numbers in the corresponding papers are reported in this study. The results of the performance comparison to the state-of-the-art on KADID-10k [44], TID2013 [53], VCL-FER [47], TID2008 [46], MDID [48], and CSIQ [49] are summarized in Tables 4-6, respectively. It can be observed that the performance of the examined state-of-the-art FR-IQA algorithms are far from perfect on KADID-10k [44]. In contrast, our method was able to produce PLCC and SROCC values over 0.95. Furthermore, our KROCC value is about 0.09 higher than the second the best one. On the smaller TID2013 [53], TID2008 [46], MDID [48], and VCL-FER [47] IQA databases, the performances of the examined state-of-the-art approaches significantly improve. In spite of this, our method also gives the best results on these databases in terms of PLCC, SROCC, and KROCC, as well. Similarly, our method achieves the best results on CSIQ [49]. On the other hand, the difference between the proposed method and other state-of-the-art methods is observably less than those on the other IQA benchmark databases. Significance tests were also carried out to prove that the achieved improvements on benchmark data sets are significant. More precisely, the ITU (International Telecommunication  
Figure 6: Plots of mean PLCC, SROCC, and KROCC between ground-truth and predicted MOS values measured on KADID-10k [44] over 100 random train-test splits as a function of the percentage of the dataset used for training. Mean PLCC, SROCC, and KROCC are plotted as red lines, while the standard deviations are depicted as shaded areas. (**a**) Mean PLCC as a function of the dataset used for training. (**b**) Mean SROCC as a function of the dataset used for training. (**c**) Mean KROCC as a function of the dataset used for training. Union) guidelines [74] for evaluating quality models were followed. The \(H_{0}\) hypothesis for a given correlation coefficient (PLCC, SROCC, or KROCC) was that a rival state-of-the-art method produces not significantly different values with \(p<0.05\). Moreover, the variances of the \(z\)-transforms were determined as \(1.06/(N-3)\), where \(N\) stands for the number of images in a given IQA database. In Tables 4-6, the green background color stands for the fact that the correlation is lower than those of the proposed method and the difference is statistically.