# Improved Combined Metric for Automatic Quality Assessment of Stitched Images
## 4 The Experimental Results  
In the original paper [20], the ISIQA database was randomly divided into 80% training and 20% testing sets (21 and 5 scenes, respectively) to train the SIQE so it led to an obvious increase in correlation due to the smaller testing set. Significantly lower values were obtained by testing the SIQE metric on the whole dataset, which is a typical approach in the general-purpose IQA. Hence, the results of the experiments obtained for all 264 images from the ISIQA dataset using the proposed approach and a comparison of these results with the previously proposed methods are presented in Table 1. To illustrate the necessity and influence of each feature and each part of the model on the obtained performance, the results of the ablation study are presented in Table 2.  
As can be seen in the results presented in Table 2, due to the choice of the PLCC as the optimization criterion, in some cases, the removal of individual features led to a small increase in the rank-order correlation coefficients. This was partially caused by the repetition of the optimization procedure during the ablation study for each configuration of features. An additional illustration of the linearity of the correlation of the proposed metric with the subjective MOS values provided for all images from the ISIQA database is presented in Figure 4. It contains the scatter plot obtained for the proposed CombSIQE metric, presented in comparison to the original SIQE metric. Each dot visible in the scatter plot represents a single image and in an ideal case (PLCC equal to 1), all these dots should form a straight line. As can be clearly observed, the fundamental role of the SIQE metric in the proposed approach is certain. Comparing the results obtained for the weighted sum of the nine features and the weighted product of the same nine features, some advantages of the weighted sum can be observed. These result from the higher flexibility of this model, with 18 coefficients subjected to optimization: 9 weights (\(w_{1}\)-\(w_{9}\)) and 9 exponents (\(a_{1}\)-\(a_{9}\)). Nevertheless, only the joint application of both models aggregated into the proposed CombSIQE metric makes it possible to achieve much better performance with PLCC and  
\begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Metric**} & \multicolumn{3}{c}{**Correlation with MOS**} \\ \cline{2-4}  & **PLCC** & **SROCC** & **KROCC** \\ \hline CombSIQE (proposed) & **0.8684** & **0.8665** & **0.6810** \\ \hline only weighted sum & 0.8387 & 0.8363 & 0.6439 \\ only weighted product & 0.8362 & 0.8338 & 0.6431 \\ \hline without \(f_{1}\) (SIQE) & 0.3963 & 0.4177 & 0.2862 \\ without \(  
Figure 4: Scatter plots for SIQE (**a**) and the proposed metric (**b**) obtained for the whole ISIQA dataset. Some of the clearly visible outliers on the SIQE scatter plot removed using the proposed method can be observed in Figure 4, e.g., for images no. 177, 182, and 193 (MOS from 20.5896 to 24.6378 and SIQE from 57.5504 to 60.0336), images no. 168, 178, and 194 (MOS from 31.2171 to 32.8717 and SIQE from 63.1936 to 66.3105), or images no. 61 (MOS = 60.7633, SIQE = 40.8954) and 229 (MOS = 65.4083 and SIQE = 37.2225). The last two images (shown in Figure 5g,h) have many small details, whereas the six low-quality images with MOS < 35 (Figure 5a-f) contain some clearly visible ghosting artifacts, which may be easier to detect using the additionally proposed features, e.g., based on local entropy and detection of edges. In the presence of ghosting artifacts, more edges may be detected in the images and the local entropy also increases so a combination of features based on them and the SIQE leads to a significantly better correlation with the MOS values provided in the ISIQA dataset.