# Hierarchical Gradient Similarity Based Video Quality Assessment Metric
## 3 Experimental Results
Table 2 shows the PCC of state-of-the-art metrics performed on four kinds of distortion in the LIVE database, and Figure 3 shows the performance comparison between these metrics as bar charts. For three of the four distortion types, the proposed metric performs in the top three. For the IP distortion \begin{table}
\begin{tabular}{c c c} \hline \hline
**Methods** & **Pearson CC** & **Spearman CC** \\ \hline VQM [24] & 0.702 & 0.723 \\ MOVIE [25] & 0.786 & 0.810 \\ ST-MAD [26] & 0.830 & 0.824 \\ STAQ [27] & 0.719 & 0.5665 \\ STSI [13] & 0.779 & 0.778 \\ SW-SSIM [8] & 0.585 & 0.596 \\ MC-SSIM [9] & 0.679 & 0.698 \\ PSNR [13] & 0.368 & 0.404 \\ PQR (by PQA500) [13] & 0.695 & 0.712 \\ DMOS (by PQA500) [13] & 0.695 & 0.711 \\ Proposed (DP) & 0.775 & 0.769 \\ Proposed (VP) & 0.736 & 0.740 \\ Proposed (VA) & 0.759 & 0.761 \\ Proposed (DP&PkVP) & 0.810 & 0.807 \\ Proposed (VP&VA) & 0.804 & 0.811 \\ Proposed (DP&VA) & 0.817 & 0.816 \\ Proposed (HVQA) & **0.832** & **0.833** \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Performance comparison on the LIVE database. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Methods** & **Wireless** & **IP** & **H.264** & **MPEG2** & **All Data** \\ \hline PSNR [13] & 0.4675 & 0.4108 & 0.4385 & 0.3856 & 0.368 \\ VQM [24] & 0.7325 & 0.6480 & 0.6459 & **0.7860** & 0.702 \\ STSI [13] & 0.7544 & **0.8072** & 0.8298 & 0.6624 & 0.779 \\ SW-SSIM [8] & 0.5867 & 0.5587 & 0.7206 & 0.6270 & 0.585 \\ PQR (PQA500) [13] & 0.6464 & 0.7300 & 0.7455 & 0.6456 & 0.695 \\ DMOS (PQA500) [13] & 0.6426 & 0.7295 & 0.7427 & 0.6445 & 0.695 \\ MOVIE [25] & **0.8386** & 0.7622 & 0.7902 & 0.7595 & **0.8106** \\ STAQ [27] & 0.5684 & 0.7080 & **0.8778** & **0.7988** & 0.7192 \\ VRF [28] & 0.7708 & 0.7453 & 0.7062 & 0.6019 & 0.6983 \\ ST-MAD [26] & **0.8123** & **0.7900** & **0.9097** & **0.8422** & **0.8299** \\ Proposed (HVQA) & **0.8109** & **0.8264** & **0.8445** & 0.7654 & **0.8324** \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Pearson correlation coefficient ( PCC) scores of video quality assessment (VQA) metrics on each kind of distortion in live database. In saliency calculation, pixels with the high spatio-temporal gradient magnitudes are determined to be saliency pixels. The threshold is defined to be the average of the largest \(k\) gradient magnitudes. When \(k\) is set to 1, all the pixels are the saliency pixels (i.e., the visual attention is not included in the proposed metrics). As shown in Table 1, the metric evaluating visual attention significantly (denoted as Proposed (HVQA)) outperformed the metric without evaluating visual attention (denoted as Proposed (DP&VP)). It should be noted that the parameter \(k\) is set to 0.35 through exhaustive experiments, which were performed on the LIVE database. Figure 4 shows the correlation coefficients of the proposed metrics when the parameter \(k\) is set to be different values. The values range from 0.2 to 0.5 with an interval of 0.05. It can be observed that when \(k\) is set to 0.35, both the SROCC and PCC are the largest. For \(k\) set from 0.2 to 0.4, the correlation coefficients are higher than 0.8. However, when \(k\) is larger than 0.4, the correlation coefficients become small (i.e., evaluating visual attention can improve the performance of the proposed metric). The robustness is also validated by evaluating the proposed metric on the EPFL-PoliMI database in the following paragraph. In order to further investigate the robustness of the proposed metric, it is also evaluated on the EPFL-PoliMI database. Table 3 shows the results of the proposed HVQA metrics as well as the comparison metrics (the reference values are copied from [29]). It can be observed that the PCC and SROCC of the HVQA metric are both higher than 0.965, which indicates high consistence with human perception. Furthermore, the proposed HVQA metric significantly outperforms the comparison results, including the MOVIE and MS-SSIM. Therefore, the proposed hierarchical gradient-based metric is robust and efficient.