# Quality Assessment of 3D Synthesized Images Based on Textural and Structural Distortion Estimation
## 4 Experiments and Results
### Statistical Significance Test  
The statistical significance test [16,72] helps to determine whether a quality metric is statistically better than the other metric. We conducted this test to statistically verify the performance of the proposed metric. In this experiment, we considered only the 3D-IQA metrics as the previous evaluations have shown that the 2D-IQA algorithms perform rather poorly compared to 3D-IQA approaches in assessing the quality of DIBR-synthesized images. The F-test procedure was used to test the significance of the difference between two quality assessment metrics. In the F-test, we compared the variances of residuals (Equation (18)) of two metrics \(i\) and \(j\) with the F-ratio threshold to find the statistical significance. The F-ratio threshold was obtained from the F-distribution look-up table with \(\alpha=0.05\).  
\[\frac{\sigma_{j}^{2}}{\sigma_{i}^{2}}>F-ratio \tag{19}\]  
If the \(\frac{\sigma_{j}^{2}}{\sigma_{i}^{2}}\) ratio is greater than the F-ratio then the metric \(i\) is said to be significantly superior to the metric \(j\). Similarly, the metric \(i\) is said to be significantly inferior to the metric \(j\) if this ratio is less than the p-value. The two metrics are said to be statistically indistinguishable if this ratio lies between the p-value and the F-ratio threshold. The F-ratio is called the right-tail critical value and the p-value is called the left-tail critical value. Both of these values were obtained from the F-distribution look-up table at a 95% significance level. The results of the test are presented in Table 6. Each entry in the table is a codeword of 6 characters corresponding to symbols A, G, J, j, S, and T, which represent the distortions AWN, Gauss, JP2K, JPEG, Sample, and Transloss, respectively. In the codeword, the value '1' means that the performance of metric in the row is significantly superior to the metric in the column, the value '0' means that the metric in the row is significantly inferior to  
\begin{table}
\begin{tabular}{c c c c} \hline \hline \multicolumn{3}{c}{**2D-IQA Metrics**} & \multicolumn{3}{c}{**3D-IQA Metrics**} \\ \hline
**Metric** & **Variance** & **Metric** & **Variance** \\ \hline
3 DSwiM & 3.9174 & PSNR & 2.1919 \\ SStD & 3.4625 & SSIM & 2.8056 \\ Chen & 1.7483 & VSNR & 2.1232 \\ Benoit & 3.0423 & IFC & 5.0145 \\ Campisi & 2.8056 & MSSIM & 1.6881 \\ Ryu & 1.5860 & VIF & 2.4588 \\ PQM & 1.7518 & UQI & 2.9281 \\ Gorley & 3.3626 & & \\ You\({}_{i}\) & 2.9615 & & \\ You\({}_{g}\) & 5.8762 & & \\ SIQM & 2.7138 & & \\ ST\_SIAQ & 3.3297 & & \\ NIQSV & 3.6606 & & \\ \hline Ours & **1.3983** & & \\ \hline \hline \end{tabular}
\end{table}
Table 5: Variance of the residuals of subjective ratings and the mapped objective scores of the proposed and the compared 2D and 3D quality metrics. The best results are highlighted in bold.  
the metric in the column, and \(-\)' means that the performance of metric in the row and column is equivalent or statistically indistinguishable. The results demonstrate that except AWN and Transloss, the performance of the proposed metric is significantly superior or equivalent to all the compared 3D-IQA metrics in all distortion types.  
The experimental evaluations performed on the benchmark DIBR synthesized image dataset showed that the performance of the proposed 3D-IQA metric is appreciably better than the compared 2D and 3D-IQA algorithms. Moreover, the variance and the statistical significance tests also showed that our metric is significantly superior or equivalent to most of the compared 3D-IQA metrics. All these performance analyses reveal that the proposed metric is reliable and accurate in estimating the quality of the DIBR-synthesized views.