# Reliability-Based View Synthesis for Free Viewpoint Video
## 2 Related Work  
Depth image-based rendering is one of the most common methods of generating virtual viewpoints at arbitrary locations [3; 5]. Hence the accuracy of the depth map is very important for the directly warped image quality. Meanwhile, the associated depth maps are coarse because of the deficient stereo matching algorithm or low-precision depth acquisition instrument, and synthesized images are not capable of satisfying the visual perception. Initially, the common approach to eliminate pinholes, cracks, and other artifacts was to preprocess the depth maps before DIBR, aiming to reduce the disparity along the boundary between foreground and background. In [6], [PERSON] and [PERSON] used a symmetric Gaussian filter to smooth the whole depth map, and in [7] they used an asymmetric filter to reduce the vertical edge artifacts. In [8], [PERSON] used the background information to cover the holes after using a bilateral filter to preprocess the depth maps. In [9], an edge-dependent Gaussian filter, which is capable of smoothing the depth map while preserving the boundary along the foreground and background, was presented. Based on the experimental results and subjective performance outcome, all of the filters in [6; 7; 8; 9] easily caused geometrically distorted foreground objects, especially when the distance between the reference view and virtual view was large, and the artifacts and other unwanted effects could be seen as unsatisfactory. In this paper, depth consistency cross-checking is used to check whether the depth value of each pixel is reliable or unreliable. For the left reference view, each pixel in the texture image is warped to the right reference viewpoint to verify its similarity to the corresponding depth map. A pixel is marked as unreliable if it fails to match the corresponding texture pixel in another reference view. The cross-checking for the right view is similar. In the second period, there are two methods to solve large-hole region problems, spatial correlation and temporal correlation [5]. In the spatial domain, some methods utilize texture information in the neighboring regions of the frame that are selected at the same moment. [PERSON] et al. [10] proposed an exemplar-based method that iteratively fills the disocclusions using the neighboring information; nowadays, this method denotes a classical inpainting algorithm. Experimental results show that inpainting obtains good performance when holes appear as narrow gaps, but the texture information is easily lost when holes are large. In fact, experiments indicate that large holes are always caused by disocclusions. In [11], [PERSON] and [PERSON] presented a novel virtual view synthesis method with a single viewpoint plus depth (SVD) format, which can cover disoccluded regions in a visually plausible manner. This improved texture synthesis is realized by exemplar-based texture synthesis, including a new priority term and a new best exemplar selection based on both texture and depth information. Actually, these spatial methods always focus on filling the narrow gaps or small holes in the texture image, while texture synthesis can fill the large-scale holes [12; 13; 14]. Since large holes are observed when areas that are occluded by foreground objects in the reference view become exposed in the synthesized view, view-blending approaches can be used to alleviate this problem, as two adjacent cameras can cover a relatively wider viewing angle [15; 16; 17]. As to exploiting the temporal correlation, [PERSON] and [PERSON] [18] tried to determine the background information using a background subtraction method, but this approach relies on good performance of the foreground segmentation method, so it cannot be adopted in complex circumstances. [PERSON] explored the motion vector of H.264/AVC bit stream to render disocclusions in the virtual view [19].