# Virtual View Synthesis Based on Asymmetric Bidirectional DIBR for 3D Video and Free Viewpoint Video
## 2 Proposed Asymmetrical Bidirectional Rendering Approach
### Depth-Guided Postprocessing  
This part is mainly to deal with the remaining holes after image merging. As mentioned in Section 1, [PERSON]'s method [16] is not very suitable for filling these holes, because some foreground textures might be sampled to fill the disocclusions. For this task, a depth-guided inpainting method is proposed to fill the remaining holes. As the depth value can reflect that the object belongs to the foreground or background to some extent, depth information is introduced to guide the filling process.  
Like [PERSON]'s method [16], the depth-guided inpainting method includes three core steps: priority computation, patch matching, and priority update. The priority computation is performed on all the edge pixels of holes. For pixel \(p\) on the hole edge \(\delta\Omega\), \(\Psi_{p}\) denotes the square template centered at \(p\). The priority of pixel \(p\) is computed as follows:  
\[P(p)=[C(p){\cdot}D(p){\cdot}Z(p)]B(p)+\varepsilon, \tag{10}\]  
where \(C(p)\) and \(D(p)\) are the confidence term and data term as defined in [16]. \(Z(p)\) is depth term, which reflects the depth value of \(p\). \(B(p)\) is background term, which is used to identify foreground pixels on the disocclusion edges. The two newly introduced terms are defined as follows:  
\[Z(p)=\frac{d_{\max}-\frac{\sum_{q\
eq\Psi_{p}\cap\Phi}d(q)}{\|\Psi_{p}\cap\Phi \|}}{d_{\max}-d_{\min}}, \tag{11}\]  
\[B(p)=\left\{\begin{array}{ll}0,&\text{for}\in\delta\Omega_{d}\&\&\&F(p)=1\\ 1,&\text{otherwise}\end{array}\right., \tag{12}\]  
where \(\Phi\) denotes the source region. \(d_{\max}\) and \(d_{\min}\) are the highest and lowest non-zero depth values in the depth image. The depth term tends to give a higher priority to background pixels with lower depth value. The background term can prevent the disocclusion filling from starting from the foreground edge, even if the relevant pixels have higher confidence and texture complexity. In addition, in order to avoid the situation where the priority is equal to 0, a little parameter \(\varepsilon\) is added to Equation (10) and is set as 0.00001.  
Figure 8: Result of depth guided postprocessing: (**a**) merging result; and (**b**) postprocessing result.  
Figure 7: Magnified results of the direct merging and color correction-based merging: (**a**) direct merging; and (**b**) color correction-based merging.  
After all priorities on \(\delta\Omega\) are computed, the patch \(\Psi_{\beta}\) with the highest priority would be filled first. Considering that content with similar depth value is more suitable for filling holes in the patch, depth information is used to search for the best matching patch. The candidate patch with the smallest matching cost is represented as:  
\[\Psi_{\hat{q}}=\arg\min_{\Psi_{q}\in\Phi^{\prime}}\big{[}\text{SSD}_{\text{ color}}\Big{(}\Psi_{\beta},\Psi_{q}\Big{)}+\text{SSD}_{\text{depth}}\Big{(}\Psi_{ \beta},\Psi_{q}\Big{)}\big{]}, \tag{13}\]  
where \(\Phi^{\prime}\) is the search region, SSD is the sum of squared differences of the valid pixels in two patches. The content in \(\Psi_{\hat{q}}\) is copied to the empty region in \(\Psi_{\beta}\). After each iteration, the priority items are updated to accommodate the new valid information. The above steps are repeated until all remaining holes are filled. The filling result is shown in Figure 8b, which does not introduce the blurring effect.