# Virtual View Synthesis Based on Asymmetric Bidirectional DIBR for 3D Video and Free Viewpoint Video
## 2 Proposed Asymmetrical Bidirectional Rendering Approach
### Color Correction-Based Merging  
Unlike single view rendering, asymmetric bidirectional rendering uses the two reference images to synthesize the target virtual image. Considering that there may be differences in brightness and color between the two images, some color-correction methods are proposed [24; 32]. In these methods, two whole images are warped to the virtual view, and the ratio image is created for each virtual image. For pixels which are projected from both views, the color value of them is computed by blending the colors of two corresponding pixels. Then the ratio of these pixels are computed and inserted in the ratio image. For pixels only warped from the left or right reference image, their ratio values are obtained by computing the locally averaged ratio of the adjacent pixels. Then the color correction is performed based on the ratio image. However, in our method, only the occlusion layer in the auxiliary view is warped to the virtual view, the color correction process mentioned above is no longer applicable. Therefore, a modified merging approach based on color correction is proposed in this section. Since the virtual view is located between the left and right reference views, it can be assumed that the color difference among them is gradual. In our method, color correction is performed on the two synthesized images first, and then the contents of the auxiliary virtual image are copied to holes in the primary virtual image. Figure 6: Comparison of circumscribed polygon extraction methods: (**a**) hole edge pixels; (**b**) result of convex hull; and (**c**) result of concave hull. The color-correction process is performed in the Lab color space because it has a wider color gamut and better perceived uniformity. First we convert the two virtual images from Red-Green-Blue (RGB) color space to Lab color space. Compute the average \(L\), \(a\), and \(b\) of each image, as follows:  
\[\overline{L}_{i}=\frac{1}{\text{num}_{i}}\sum L_{i}(x,y),\text{for}(x,y)\
otin \Omega_{i}, \tag{3}\]  
\[\overline{a}_{i}=\frac{1}{\text{num}_{i}}\sum a_{i}(x,y),\text{for}(x,y)\
otin \Omega_{i}, \tag{4}\]  
\[\overline{b}_{i}=\frac{1}{\text{num}_{i}}\sum b_{i}(x,y),\text{for}(x,y)\
otin \Omega_{i}, \tag{5}\]  
where \(i=p,a\) represents the primary and auxiliary virtual image respectively, and \(\text{num}_{i}\) denotes the number of valid pixels in the image. Then the overall mean of each channel is computed as:  
\[\overline{L}=\frac{\overline{L}_{p}+\overline{L}_{a}}{2},\overline{a}=\frac{ \overline{a}_{p}+\overline{a}_{a}}{2},\overline{b}=\frac{\overline{b}_{p}+ \overline{b}_{a}}{2}, \tag{6}\]  
In the correction process, taking the \(L\) channel for example, the corrected result can be expressed as:  
\[L^{\prime}{}_{i}(x,y)=\frac{\overline{L}}{\overline{L}_{i}}L_{i}(x,y),\text{ for}(x,y)\
otin\Omega_{i},i=p,a, \tag{7}\]  
For the \(a\) and \(b\) channels, the same computation is performed. The processed images are converted from Lab color space to RGB color space. After color correction, the contents in the auxiliary virtual image are used to fill the holes. The merging process can be expressed as follows:  
\[I(x,y)=\left\{ \begin{array}{l}I_{p}(x,y),I_{p}(x,y)\
otin\Omega\\ I_{a}(x,y),I_{p}(x,y)\in\Omega\end{array} \right., \tag{8}\]  
where \(I_{p}(x,y)\) and \(I_{a}(x,y)\) are the corresponding pixels in the primary and auxiliary images, respectively, and \(I(x,y)\) represents the merging result. During the merging, in order to prevent the ghosts, the pixels on the background edge of the disocclusions are replaced with corresponding valid pixels in the auxiliary image, as:  
\[I(x,y)=I_{a}(x,y),\text{for}(x,y)\in\delta\Omega_{d}\&\&F(x,y)=0, \tag{9}\]  
The local comparison results of direct merging and color correction-based merging are shown in Figure 7. Compared with direct merging, the proposed merging approach reduces the abrupt changes in brightness, and makes the texture transition more natural. The overall result is shown in Figure 8a.