## 4 Results and Discussions
### Quantitative Comparisons  
During the inpainting task, as there were no datasets for video inpainting before, the evaluation is always based on qualitative comparisons. Although the inpainting result is not unique, which means that every result is acceptable as long as it is natural and realistic, the gap between the inpainting result, and the ground truth can still be used as a standard to evaluate the quality of the algorithms. To evaluate our algorithm quantitatively, we compare our model with a typical patch-based algorithm [11]. This approach is considered canonical in terms of versatility, enabling us to confirm the versatility of our algorithm, while recent models are incapable of handling both inpainting and SR tasks. In addition, the code for [11] is publicly available online, so we can compare it with our synthetic test data. However, the recent models lack universality, have limited improvement in PSNR and SSIM index and have little improvement in visual effect. Thus, our work is focused on introducing a universal model capable of restoring both images and videos. Since the method used in [11] is a canonical method in terms of versatility, we select it as the baseline model for the inpainting task. We employ five assessment criteria, namely peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), mean L1 error, mean L2 error, and perceptual loss. The L1 and L2 errors intuitively reflect the gap between two images at the pixel level. PSNR is based on error sensitivity, and SSIM evaluates the image in terms of brightness, contrast, and structure. Perceptual loss, which is widely used in super-resolution tasks, is a function that evaluates the similarity between two images based on high-level features. Here, we use the feature of the second full connection layer of the VGG16 model as the standard. For L1 error, L2 error, and perceptual loss, a smaller value means better image quality. However, the opposite is true for PSNR and SSIM. The average results of the five indexes on the synthetic test datasets are shown in Table 5. Our model performs better on PSNR, SSIM, L1 loss, and L2 loss, while the performance on perceptual loss is similar. This is mainly due to the non-local architecture, which considers mostly the spatio-temporal features in each frame. Moreover, as the recent models [47; 48; 49; 50; 51] have limited improvement in PSNR and SSIM index and little improvement in visual effect, the results of our model can already meet the application requirements of most scenarios, and our approach has greater universality. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Methods** & **PSNR** & **SSIM** & **L1** & **L2** & **VGG** \\ \hline
[11] & 40.2829 & 0.9917 & 0.6276 & 3.6390 & 0.1750 \\ Ours & 40.5042 & 0.99473 & 0.5484 & 2.9696 & 0.2411 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: The comparison of inpainting results between [11] and our model. Long computing times are an always-standing problem in video inpainting tasks. This is because a large amount of time is spent on patch seeking, matching, and any other options. For the deep learning algorithms, the parameters have been confirmed in training stage. Correspondingly, the calculation becomes extremely simple in the testing step. When inpainting a 32-frame video with a size of 160 \(\times\) 240, the patch-based algorithm [11], when implemented on Matlab, takes about 6.5 min, but our model only consumes about 1.5 s on TensorFlow. Furthermore, the time of inpainting an incomplete sample frame cost about 0.05 s on our proposed model, while that of [11] costs about 12 s. Although the implement platform is different, the operation time is cut down greatly. During the SR task, we conduct quantitative comparisons to demonstrate the advantages of our proposed model. All of the LR images are generated by bicubic downscaling with a scaling factor of \(\times\)4 from the HR images according to the standard protocol. We conduct the comparisons on benchmark datasets using the quality metrics of PSNR and SSIM.