# No-Reference Image Quality Assessment with Convolutional Neural Networks and Decision Fusion
## 2 Materials and Methods
### Materials
#### 2.1.2 Evaluation Protocol and Implementation Details  
The evaluation of NR-IQA algorithms is based on the correlation strength measured between predicted and ground-truth scores. To characterize the correlation strength, the Pearson linear correlation coefficient (PLCC), Spearman rank order correlation coefficient (SROCC) and Kendall rank order correlation coefficient (KROCC) are widely applied in the literature [8]. Moreover, before the calculation of PLCC, a non-linear mapping is carried out between the predicted and ground-truth scores using a logistic function with 5 parameters, as recommended by [PERSON] et al. [53]. Formally, this non-linear mapping can be given as  
\[Q=\beta_{1}\bigg{(}\frac{1}{2}-\frac{1}{e^{-\beta_{2}(Q_{p}-\beta_{3}}}\bigg{)} +\beta_{4}Q_{p}+\beta_{5}, \tag{1}\]  
where \(\beta_{i}\)'s \((i=1,\dots,5)\) are the fitting parameters. Moreover, \(Q_{p}\) and \(Q\) denote the predicted and mapped quality scores, respectively. Between vectors \(\mathbf{x}\) and \(\mathbf{y}\) of the same length \(N\), PLCC is defined as  
\[PLCC(\mathbf{x},\mathbf{y})=\frac{\sum_{i=1}^{N}(x_{i}-\bar{x})(y_{i}-\bar{y}) )}{\sqrt{\sum_{i=1}^{N}(x_{i}-\bar{x})^{2}\sum_{i=1}^{N}(y_{i}-\bar{y})^{2}}}, \tag{2}\]  
where \(x_{i}\), \(y_{i}\) denote the \(i\)th element of \(\mathbf{x}\), \(\mathbf{y}\), respectively. Moreover, \(\bar{x}=\frac{1}{N}\sum_{i=1}^{N}x_{i}\) and \(\bar{y}=\frac{1}{N}\sum_{i=1}^{N}y_{i}\). Between these vectors, SROCC is defined as  
\[SROCC(\mathbf{x},\mathbf{y})=1-\frac{6\sum_{i=1}^{N}d_{i}^{2}}{N(N^{2}-1))}, \tag{3}\]  
Figure 3: Sample images from TID2013 [51] with their MOS values: (**a**) pristine reference image, (**b**) spatially correlated noise, \(MOS=3.6923\), (**c**) contrast change, \(MOS=6.6500\) and (**d**) multiplicative Gaussian noise, \(MOS=3.3750\).  
where \(d_{i}\) stands for the difference between the two ranks of each \(x_{i}\) and \(y_{i}\). Moreover, the following formula is used to determine KROCC  
\[KROCC(\mathbf{x},\mathbf{y})=\frac{C-D}{\binom{N}{2}}, \tag{4}\]  
where \(C\) is the number of concordant pairs between \(\mathbf{x}\) and \(\mathbf{y}\), while \(D\) is the number of discordant pairs.  
As is common in the literature [8], we report on median PLCC, SROCC and KROCC values measured over 100 random train-validation-test splits. Since the median is more robust to outliers, it is preferred in the literature to arithmetic mean. More precisely, an IQA benchmark database containing authentic distortions (CLIVE [50], KonIQ-10k [4] and SPAQ [5]) was randomly divided into a training set with approximately 70% of images, a validation set with 10% of images, and a test set with the remaining 20%. Since TID2013 [51] consists of images with artificial distortions, it was divided with respect to the reference images to avoid any semantic content overlap between the train, validation and test sets.  
The proposed method was implemented in Python 3.6.1 programming language using the Keras deep learning library [54] and the Keras utilities (Kuti) library. Moreover, the proposed method was trained on two NVidia Titan Xp GPUs.