# No-Reference Image Quality Assessment with Convolutional Neural Networks and Decision Fusion
## 3 Experimental Results and Analysis
### Comparison to the State of the Art
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**CLIVE [50]**} & \multicolumn{3}{c}{**KonIQ-10k [4]**} \\ \hline
**Method** & **PLCC** & **SROCC** & **KROCC** & **PLCC** & **SROCC** & **KROCC** \\ \hline BLIINDER [66] & 0.782 & 0.763 & 0.576 & 0.876 & 0.864 & 0.668 \\ DeepRN [67] & 0.784 & 0.753 & 0.579 & 0.866 & 0.880 & 0.666 \\ BLIINDS-II [114] & 0.473 & 0.442 & 0.291 & 0.574 & 0.575 & 0.414 \\ BMPRI [68] & 0.541 & 0.487 & 0.333 & 0.637 & 0.619 & 0.421 \\ BRISQUE [15] & 0.524 & 0.497 & 0.345 & 0.707 & 0.677 & 0.494 \\ CurveletQA [11] & 0.636 & 0.621 & 0.421 & 0.730 & 0.718 & 0.495 \\ DIVINE [9] & 0.617 & 0.580 & 0.405 & 0.709 & 0.693 & 0.471 \\ ENIQA [69] & 0.596 & 0.564 & 0.376 & 0.761 & 0.745 & 0.544 \\ GRAD-LOG-CP [70] & 0.607 & 0.604 & 0.383 & 0.705 & 0.696 & 0.501 \\ MultiGAP- & 0.841 & 0.813 & 0.626 & 0.915 & 0.911 & 0.732 \\ NRIQA [36] & 0.831 & 0.801 & 0.607 & 0.901 & 0.885 & 0.703 \\ MSDF-IQA [71] & 0.629 & 0.604 & 0.427 & 0.771 & 0.749 & 0.515 \\ PIQE [72] & 0.172 & 0.108 & 0.081 & 0.208 & 0.246 & 0.172 \\ OG-IQA [73] & 0.545 & 0.505 & 0.364 & 0.652 & 0.635 & 0.447 \\ SSEQ [74] & 0.487 & 0.436 & 0.309 & 0.589 & 0.572 & 0.423 \\ UNIQUE [75] & **0.991** & 0.855 & **0.633** & 0.900 & 0.897 & 0.664 \\ \hline CONTRIQUE [76] & 0.857 & 0.845 & - & 0.906 & 0.894 & - \\ DB-CNN [77] & 0.869 & 0.851 & - & 0.884 & 0.875 & - \\ DeepFL-IQA [78] & 0.769 & 0.734 & - & 0.887 & 0.877 & - \\ KonCept512 [79;82] & 0.848 & 0.825 & - & 0.937 & 0.921 & - \\ MLP [78;80] & 0.769 & 0.734 & - & 0.887 & 0.877 & - \\ PAQ-2-PiQ [46] & 0.850 & 0.840 & - & 0.880 & 0.870 & - \\ PQR [81] & 0.882 & **0.857** & - & 0.884 & 0.880 & - \\ WSP [43] & - & - & - & 0.931 & 0.918 & - \\ \hline DF-CNN-IQA & 0.859 & 0.849 & 0.630 & **0.949** & **0.931** & **0.738** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison of _DF-CNN-IQA_ to the state of the art on CLIVE [50] and KonIQ-10k [4]. Median PLCC, SROCC and KROCC values were measured over 100 random train-validation-test splits. The best results are typed in **bold**, and the second-best results are underlined. We denoted by ‘-’ if the data was not available. all the other considered NR-IQA algorithms on SPAQ [5], while it performs weaker than the best state-of-the-art method on TID2013 [51], which in contrast to SPAQ [5] contains artificial distortions. The direct and weighted averages obtained on these IQA databases are summarized in Table 6. It can be seen that the direct average PLCC, SROCC and KROCC values of the proposed method are larger than those of the majority of the considered state-of-the-art methods. If we take a look at the weighted average by the number of images found in the applied IQA benchmark databases, the difference between the proposed method and the other methods is even more significant. Specifically, DF-CNN-IQA can outperform all the other algorithms in this regard. Moreover, it can be concluded that the proposed method tends to give a better performance on larger IQA benchmark databases with authentic distortions. To ascertain whether the achieved results are significant or not, one-sided \(t\)-tests [83] were carried out between the 100 SROCC values produced by our and the other state-of-the-art methods tested by us. Namely, the null hypothesis was that the mean SROCCs of two sets containing 100-100 correlation values are equal with a confidence of 95%. The results of the one-sided \(t\)-tests are summarized in Table 7. In this table, symbol 1 \((-1)\) stands for that the proposed method is statistically better (worse) at a confidence interval of 95% than the other examined NR-IQA method. From these results, it can be concluded that the proposed method is able to significantly outperform the state of the art on large databases with authentic distortions (KonIQ-10k [4] and SPAQ [5]), while it is outperformed by two methods on artificial distortions (TID2013 [51]).