# Virtual View Synthesis Based on Asymmetric Bidirectional DIBR for 3D Video and Free Viewpoint Video
## 2 Proposed Asymmetrical Bidirectional Rendering Approach
### Asymmetrical Bidirectional Rendering
In the virtual image, the length of each disocclusion is recorded and the edge pixels of the disocclusion are classified based on the depth information. By applying the Laplacian operator to the primary depth image [30], the foreground edges of the disocclusion can be obtained and labeled as follows:  
\[F(x,y)=\left\{ \begin{array}{ll}1,&\text{if}(\Delta d)_{w}(x,y)<0\\ 0,&\text{if}(\Delta d)_{w}(x,y)>0\end{array} \right.,\text{for}(x,y)\in\delta \Omega_{d}, \tag{2}\]  
where \(\Delta d\) denotes the Laplacian of the depth image and \(\left(\Delta d\right)_{w}\) denotes the warped Laplacian image. \(\delta\Omega_{d}\) represents the boundary of disocclusion. Based on the depth information, the edge pixels of the disocclusions and OFRs are back-warped into the auxiliary image, so the associated pixels can be located. Based on the marked pixels through inverse 3D warping, the regions for hole filling are extracted in the auxiliary image, as shown in Figure 5c. For the disocclusion, starting from the right background edge, we extend to the left in the horizontal direction and mark the adjacent pixels as the selected pixels for warping. The length of the extension depends on the category of the left edge pixel. For BG-BG disocclusion, all the pixels between the left and right edges are selected. But for FG-BG disocclusion, the pixels of the hole length are selected for warping. As for the OFRs, since the edge pixels are located in the same depth layer, the selected pixels can be determined by extracting the pixels inside the contour. In this process, convex or concave hull extraction is optional. The result in Figure 6 shows that, for certain applications, the convex hull does not represent well the boundaries of a given set of points. The concept of alpha-shape is introduced as a solution to this problem, and other solutions, such as crust algorithms are also proposed. However, most of the proposed approaches address the reconstruction of surfaces from sets of points belonging to that surface and, therefore, are not optimized for the referred problem. Compared with the convex hull, the concave hull can better reflect the accurate shape of the contour and contain fewer selected pixels, thus reducing the computational cost. Hence, a concave hull extraction method based on K nearest neighbors [31] is  
Figure 5: Extraction of the region for hole filling: (**a**) primary image; (**b**) original virtual image; and (**c**) extraction of the warping region in the auxiliary image. applied in this paper to select the pixels for OFR filling. The K nearest neighbors extraction approach is able to deal with arbitrary sets of points by taking care of a few special cases. The \"smoothness\" of the computed hull can also be controlled by the user through the k parameter. Unlike the alpha-shape method, the modified K nearest neighbors approach can automatically adjust the value of k according to the distribution of the point set, instead of manually setting it based on the scene. In our experiment, the value of k starts from 3, and its value can be increased recursively in the extraction process until the given points are inside the computed polygon. It is worth mentioning that most of the virtual views are synthesized through parallel rendering, that is, the virtual view is on the baseline formed by the primary and auxiliary views. In this way, the positional relationship between the left and right edges of the hole can be maintained in the auxiliary image, and almost all the OFRS can find the corresponding content in the auxiliary image. If there is a rotation relationship between the virtual view and reference views, the positional relationship between the edge pixels is changed after inverse 3D warping, and the extension approach mentioned above is no longer appropriate. In this case, the concave hull method is used to extract the warping regions for all types of holes to ensure that all helpful pixels are selected.