# No-Reference Image Quality Assessment with Convolutional Neural Networks and Decision Fusion
## 2 Materials and Methods
### Methods  
The high-level overview of the proposed end-to-end method is depicted in Figure 4. An input image to be evaluated is passed through six in-parallel state-of-the-art CNN bodies pretrained on ImageNet [55] database for image classification, such as VGG16 [56], ResNet50 [57], InceptionV3 [58], InceptionResNetV2 [59], DenseNet201 [60] and NASNet-Mobile [61], succeeded by global average pooling layers and regression heads. The main characteristics of the applied CNNs are summarized in Table 2. It is important to note that the fully connected output layers of these models were not utilized in this work, allowing us to attach new layers to them and train them for NR-IQA. Moreover, the pretrained CNN body term is reserved for a deep model without the fully connected output layers in our terminology. Figure 4: High-level overview of the proposed method. An input image is passed through six in-parallel CNN bodies followed by global average pooling layers and regression heads. The six branches are trained using the same train and validation datasets but independently from each other. To obtain an overall quality score for the input image, the quality predictions of the six branches are fused. The main idea behind global average pooling (illustrated in Figure 5) is to enforce correspondences between feature maps and the number of predicted semantic categories by taking the average of each feature map and compiling a feature vector. Thus, arbitrary sized images can be put to the input of a CNN. Other advantages are that overfitting can be avoided with this layer and robustness can be increased against spatial translations of the input [63]. The outputs of the global average pooling layers are connected to six regression heads containing four separately fully connected layers. The inner structure of the regression heads is depicted in Figure 6. Specifically, the first three layers consists of 2048, 1024 and 256 nodes, respectively. To avoid overfitting, these layers possess dropout rates [64] of 0.25, 0.25 and 0.5, respectively. To ensure better gradient propagation, rectified linear units (ReLU) were applied as activation functions in the first three layers [62]. Since the last layer predicts the quality, it contains only one neuron and applies a simple linear activation function. Let \((\mathbf{I},q)\) denote the training dataset, where \(\mathbf{I}\) is an RGB image of arbitrary size and \(q\) is the ground-truth quality score belonging to \(\mathbf{I}\). Moreover, we denote by \(\hat{q}\) the quality score predicted by one regression head. Since image quality prediction is formulated as a regression task here, mean squared error (MSE) loss is applied, which--as the name suggests--takes the mean of squared differences between the ground-truth and predicted values. Formally, this can be written as: \(L_{MSE}=(q-\hat{q})^{2}\). The six branches of the proposed fused architecture are trained using the same training and validation datasets but independently from each other. Specifically, a pretrained CNN \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**CNN** & **Size (MByte)** & **Top-1 Accuracy** & **Top-5 Accuracy** & **Parameters** & **Depth** \\ \hline VGG16 [56] & 528 & 0.713 & 0.901 & 138,357,544 & 23 \\ ResNet50 [57] & 98 & 0.749 & 0.921 & 25,636,712 & - \\ InceptionV3 [58] & 92 & 0.779 & 0.937 & 23,851,784 & 159 \\ InceptionResNetV2 [59] & 215 & 0.803 & 0.953 & 55,873,736 & 572 \\ DenseNet201 [60] & 80 & 0.773 & 0.936 & 20,242,984 & 201 \\ NASNetMobile [61] & 23 & 0.774 & 0.919 & 5,326,716 & - \\ \hline \hline \end{tabular}
\end{table} 
Table 2: On ImageNet [62] database pretrained CNNs applied in this study. In this table, Top-1 and Top-5 accuracy refers to the model’s performance on the ImageNet [55] validation dataset. Figure 5: Illustration of global average pooling. It takes the average of each feature map to compile a new feature vector. Figure 6: Structure of the regression head. The first three layers contains 2048, 1024 and 256 nodes with dropout rates of 0.25, 0.25 and 0.5, respectively.