# No-Reference Image Quality Assessment with Convolutional Neural Networks and Decision Fusion
## 2 Materials and Methods
### Materials  
#### 2.1.1 Databases  
As already mentioned, a number of benchmark IQA databases have been made publicly available in the recent decades [8; 47; 48; 49]. In this study, we deal with the quality assessment of natural images containing authentic distortions. To evaluate and compare our proposed method to the state of the art, three large databases--LIVE In the Wild (CLIVE) [50], KonIQ-10k [4] and SPAQ [5]--with authentic distortions have been chosen. Moreover, one database--TID2013 [51]--with artificial distortions was also utilized in our experiments. In the following, a brief overview is given about the previously mentioned IQA benchmark databases.  
[PERSON] et al. [50] collected 1162 images with unique scenes captured by a variety of mobile device cameras. Subsequently, these images were evaluated in a crowdsourcing experiment to obtain MOS values (ranging from 0 to 100). Similarly to CLIVE, the images of KonIQ-10k [4] were evaluated in a crowdsourcing experiment. An important difference between CLIVE [50] and KonIQ-10k [4] is that the images of KonIQ-10k were collected from the YFCC-100m [52] multimedia database using a two-stage filtering process. First, those images were discarded which did not have a creative commons license. Second, such a filtering process was applied that enforces the relative uniform distribution of eight image attributes, i.e., bitrate, resolution, JPEG quality, brightness, colorfulness, contrast, sharpness and deep feature. The MOS range in KonIQ-10k [4] spans from 1 to 5. [PERSON] et al. [5] collected 11,125 high-resolution digital images captured by 66 smartphones. Subsequently, the collected images were evaluated in a laboratory environment involving 600 subjects. Moreover, the MOS values of each image have been determined based on at least 15 quality scores. In contrast to the previously mentioned databases, TID2013 [51] contains images with artificial distortions. Specifically, it contains a set of 25 pristine reference images whose perceptual quality are considered perfect. From these reference images, 3000 distorted images are derived using 24 different types of distortions at five distortion levels.  
The empirical MOS distributions of CLIVE [50], KonIQ-10k [4], SPAQ [5] and TID2013 [51] are depicted in Figure 1. The main characteristics of the applied IQA databases are summarized in Table 1. Figures 2 and 3 depict several sample images from KonIQ-10k and TID2013 databases, respectively.  
Figure 1: MOS distributions in the applied benchmark IQA databases: (**a**) CLIVE [50], (**b**) KonIQ-10k [4], (**c**) SPAQ [5] and (**d**) TID2013 [51].  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Attribute** & **CLIVE [50]** & **KonIQ-10k [4]** & **SPAQ [5]** & **TID2013 [51]** \\ \hline Year & 2015 & 2018 & 2020 & 2013 \\ Number of images & 1169 & 10,073 & 11,125 & 3000 \\ Number of scenes & 1169 & 10,073 & 11,125 & 25 \\ Distortion type & authentic & authentic & authentic & artificial \\ Subjective framework & crowd-sourcing & crowd-sourcing & laboratory & laboratory \\ Number of annotators & 8000 & 350,000 & 600 & 540 \\ Number of annotations & 1400 & 1,200,000 & 186,400 & 524,340 \\ Resolution & \(500\times 500\) & \(1024\times 768\) & \(\sim\)\(4000\times 4000\) & \(512\times 384\) \\ MOS range & 0–100 & 1–5 & 0–100 & 0–9 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Overview about the applied publicly available IQA benchmark databases. In this table, we denoted by ‘-’ if the data was not available.  
Figure 2: Sample images from KonIQ-10k [4] with their MOS values: (**a**) \(MOS=3.4673\), (**b**) \(MOS=3.6040\), (**c**) \(MOS=2.2883\) and (**d**) \(MOS=2.1981\).