# A Multi-Feature Automatic Evaluation of the Aesthetics of 3D Printed Surfaces
## 4 Discussion  
### Analysis of Experimental Results  
The analysis of the correlation obtained for the proposed combined metrics should be preceded by the presentation of the correlation values achieved for elementary metrics and features. Top ten values, including also rank-order correlations, are presented in Table 1, whereas the results achieved for the five \"best\" combinations of two metrics are shown in Table 2.  
As presented in Table 1, only two SSIM-like elementary metrics (FSIM and CW-SSIM) may be found among those leading to the highest correlation with subjective quality scores, whereas the others utilize various variants of entropy-based calculations. Nevertheless, the entropy values strongly depend on image contrast; therefore, local calculations are conducted to reduce the direct influence of contrast on the evaluation results, as well as preprocessing operations, e.g., before the calculation of the HOG features. As may be noted, although the highest correlation for elementary metrics is obtained for the metric based on the entropy of depth maps, which requires the use of the 3D scanner, the \"best\" combinations of two elementary metrics are based on the mutual calculations of the IQA metrics or their combination with standard deviation computed for the HOG \begin{table}
\begin{tabular}{l c c c} \hline \hline
**Metric** & **PLCC** & **SROCC** & **KROCC** \\ \hline FV (16 blocks) 1 & 0.6936 & 0.6673 & 0.4811 \\ FSIM (9 blocks) & 0.6820 & 0.6845 & 0.5185 \\ FSIM (4 blocks) & 0.6780 & 0.6826 & 0.5114 \\ FSIM (16 blocks) & 0.6756 & 0.6865 & 0.5195 \\ CW-SSIM (9 blocks) & 0.6323 & 0.6098 & 0.4232 \\ CW-SSIM (16 blocks) & 0.5929 & 0.5823 & 0.4027 \\ CW-SSIM (4 blocks) & 0.5807 & 0.5633 & 0.3981 \\ kurtosis of HOG & 0.5075 & 0.5177 & 0.3874 \\ MV (RGB+hue 256 blocks) 2 & 0.4816 & 0.4920 & 0.3480 \\ EVAR (16 blocks) 3 & 0.4462 & 0.5300 & 0.3660 \\ \hline \hline \end{tabular} 1
[FOOTNOTE:1]Footnote 1: EV is the average local entropy obtained for the depth maps. 2
[FOOTNOTE:2]Footnote 2: MV is the product of the mean of the local entropy and its variance calculated separately for each RGB channel and hue channel. MV values calculated for three RGB channels are then averaged and multiplied by MV obtained for the hue channel. 3  
\end{table} 
Table 1: The correlation values of 10 “best” elementary metrics with subjective scores obtained for 107 images from the considered dataset. features. The increase in [PERSON]'s correlation from below 0.7 to over 0.87 fully confirms the validity of the proposed approach. It is also worth noting that even the use of two metrics with the proposed model lead to better results than presented in earlier papers where only the weighted product has been used (0.8353 in the paper [26], and 0.8537 in the paper [21], obtained for the combinations of four and five metrics, respectively). Nevertheless, even better results may be achieved using the combination of three or four metrics, using the \"best\" combination of two metrics as the starting point and adding each of the remaining metrics or features to the model with further optimization of its parameters. The results obtained for five \"best\" combinations of three and four metrics are presented in Table 3, where only the additional metrics used together with FSIM and SR-SIM (both computed using 16 blocks) are listed. Analyzing the results presented in Table 3, the advantage of the use of the proposed approach may be easily observed as even the use of three metrics makes it possible to exceed PLCC = 0.9, and the use of four mutually calculated IQA metrics leads to PLCC over 0.91. Further optimization of five metrics does not lead to significantly better results; hence, considering the increased computational complexity, the only reasonable combination of five metrics is the use of FSIM (for 16 blocks), SR-SIM (for 16 blocks), MCSD (for 4 blocks), CSSIM4 (for 9 blocks), and MV calculated for red channel using 256 blocks.