# Quality Assessment of 3D Synthesized Images Based on Textural and Structural Distortion Estimation
## 3 The Proposed Technique
### Estimating the Structural Distortion  
The study presented in [14] shows that the human visual system (HVS) is highly adapted for extracting structural information from the image. The inaccuracies and compression artifacts in the depth map adversely affect the structural details of the image during the process of DIBR, generally distorting the edges and gradients in the images [55; 60]. The depth compression may cause the pixels to be lost or wrongly projected in the synthesized view. Similarly, the estimation inaccuracies in the depth cause ghost artifacts, inconsistent object shift, and distortion of edges in the synthesized view. These distortions in the image affect both the texture and the structure of the image. Therefore it is equally important to compute the structural dissimilarities in the image to assess its quality. Several methods are proposed to compute the structural similarity in 2D images, e.g., [29; 60; 62; 63; 64; 65].  
We used the Hausdorff distance [66] to compute the structural similarity score. The Hausdorff distance measures the degree of mismatch between two sets [66; 67]. Similar to a texture distortion metric, this mismatch is also computed locally. The Hausdorff distance can be computed for grayscale images, e.g., [68], and for binary images [66; 67]. In the proposed metric, since we want to estimate the distortion in the structural details in the warped image compared to the reference image, the edges in the two images are detected and these edge images are used to estimate the degree of mismatch. Any edge detector can be used for this purpose, however, similar to [66], in our study we used the Canny edge detector [69] to compute the edge maps. The Hausdorff distance between two image blocks \(x_{ij}\) and \(y_{ij}\) of size \(m\times n\) centered at location \((i,j)\) in image \(I\) and \(R\), respectively, as defined in the preceding section, is computed as follows:  
\[HD(x_{ij},y_{ij})=\max(hd(x_{ij},y_{ij}),hd(y_{ij},x_{ij})) \tag{6}\]  
The function \(hd(x_{ij},y_{ij})\) is called the directed Hausdorff distance from \(x_{ij}\) to \(y_{ij}\) and it can be defined as  
\[hd(x_{ij},y_{ij})=\max_{a\in x_{ij}}\min_{b\in y_{ij}}||a-b||. \tag{7}\]  
Equation (7) identifies the point \(a\) in \(x_{ij}\) that is the farthest from any point in \(y_{ij}\) and measures its distance from the nearest neighboring point in \(y_{ij}\). The function \(hD(x_{ij},y_{ij})\) then ranks each point of \(x_{ij}\) according to its distance from the nearest neighbor in \(y_{ij}\) and picks the largest distant point from these ranked distances because it is the most mismatched point between the reference and distorted image blocks. Similarly, the directed Hausdorff distance from \(y_{ij}\) to \(x_{ij}\) is computed. In \(hd(x_{ij},y_{ij})\) and \(hd(y_{ij},x_{ij})\), the former represents the degree of mismatch between the synthesized and original image block and the latter represents the degree of mismatch between the original and the synthesized image blocks. Then the largest of the two is chosen as the mismatch score (Equation (6)). The obtained value is normalized.  
\[HD_{ij}=\frac{HD(x_{ij},y_{ij})}{255\times\sqrt{mn}} \tag{8}\]The value of \(HD_{ij}\) falls in the interval \([0,1]\). Recall that \(HD_{ij}\) is the degree of mismatch, and the structural similarity score is computed by subtracting this normalized Hausdorff score (\(HD_{ij}\)) from \(1\).  
\[\eta_{ij}=1-HD_{ij} \tag{9}\]  
The structural scores for all \(k\) blocks are computed and averaged to obtain a single score \(S\).  
\[S=\frac{1}{MN}\sum_{i=1}^{M}\sum_{i=1}^{N}\eta_{ij} \tag{10}\]