# Reliability-Based View Synthesis for Free Viewpoint Video
## 4 Experimental Results  
In this section, the proposed framework is implemented in C++ based on OpenCV, and the tested multiview video plus depth sequences include two Microsoft datasets: _Ballet_ and _Breakdancers_. In all video sequences, the size of each frame is 1024 \(\times\) 768 pixels, and each video contains 100 frames with an unmoved background. The baseline between two adjacent cameras is 20 cm for both _Ballet_ and _Breakdancers_. The associated depth maps and camera parameters are provided with the sequences. The format of all video sequences is avi, while texture images contain three channels (RGB). To evaluate the performance of the proposed method, we implemented two state-of-the-art methods and my previous work in [26], in order to compare this with the proposed approach. One of these two methods is a commonly used reference software, VSRS 3.5 [27], which mainly contains a simple DIBR method [3] and a classical inpainting technique [10]. The other is a hole-filling method exploiting temporal correlations based on GMM [5]. These two methods [5; 27] represent the exploitation of spatial correlation and temporal correlation, respectively. In each experiment, the test sequence was composed of three real video sequences from three reference viewpoints. The coded left and right views with their associated depth videos were projected to interpolate the virtual video in the target viewpoint between them. The rendered sequence was compared with the actual video on the target viewpoints to measure the peak signal-noise ratio (PSNR) and structural similarity index (SSIM). In order to show wide practical applicability of the proposed synthesis algorithm, each view synthesis method was performed on both small baseline and large baseline instances. Tables 1 and 2 show the average PSNR and SSIM values for 100 frames. Figure 4: Examples of depth map processing method: (**a,b**) enlarged integrated texture image and its associated depth map before depth map processing method (DMPM); (**c,d**) image and its associated depth map after DMPM. In the PSNR evaluation, the proposed approach obtained 4-10 dB better results than VSRS 3.5 on _Ballet_ for a large baseline instance. In the case of a small baseline, the results for both _Ballet_ and _Breakdancers_ were also better. The proposed method also showed better results beyond GMM-based disocclusion filling method and my previous work. Inpainting is an effective algorithm to fill narrow gaps and other small empty regions when the baseline is small, however, it is not practical for fill large empty regions. Moreover, my previous work did not perform well for both _Ballet_ and _Breakdancers_ sequences. This is due to the fact that simple GMM is not capable to deal with the scenes which foreground objects are with reciprocating motion. Consequently, the proposed approach yielded better results on both tested sequences. The larger the baseline, the better the results. In addition to the objective measurements, Figure 5 shows a subjective comparison. Figure 5 presents the synthesized results generated by a simple DIBR technology, where the disocclusion regions and pinholes remain to be filled. Figure 5 shows the performance of VSRS 3.5, where large empty regions are filled based on neighboring texture information. Blurring effects are observed, in contrast to our proposed method in Figure 5. This improvement comes from our idea of avoiding global processing for every pixel to handle the noise. Hence, our method shows desirable results in reducing errors and removing unwanted effects, while texture remains sharp and clear. Figure 5 shows an enlarged part of the synthesis result produced by the GMM-based disocclusion filling method; the temporal correlation method shows better performance in filling large empty areas beyond the inpainting method. Depth refinement and weighted blending lead to much more satisfactory interpolation results, as shown in Figure 5.