Abstract. This paper presents a new video restoration scheme based on the joint sparse and lowrank matrix approximation. By grouping similar patches in the spatiotemporal domain, we formulate the video restoration problem as a joint sparse and low-rank matrix approximation problem. The resulted nuclear norm and 1 norm related minimization problem can also be efficiently solved by many recently developed numerical methods. The efficiency of the proposed video restoration scheme is illustrated on two applications: video denoising in the presence of random-valued noise and video in-painting for archived films. The numerical experiments indicated the proposed video restoration method is compared favorably against many existing algorithms. Key words. nuclear norm, low-rank matrix, sparse matrix, denoising, in-painting AMS subject classifications. 68U10, 65J22, 90C25, 65K05 1. Introduction. Even with today's advances in camera and digital sensor technology, video data collected in practice often suffers from many types of annoying degradations, e.g., noise contamination, image blurring and missing data. For example, video data can be quite noisy when captured at high sensitivities, such as low lighting condition, high ISO setting or high capture rate. The frames of video data can be blurred when there are fast moving objects in the scene or when there are camera shakes. Some parts of video data can be missing or invisible from a user perspective due to occlusions, scratches, or errors in data conversion/communication. The goal of video restoration is then to recover the original one from the obtained degraded video data. With the prevalence of webcams and camera phones, the problem of video restoration has become even more important than before. In recent years, patch-based image restoration scheme has emerged as one promising approach for various image restoration tasks, e.g. denoising and in-painting ([1, 2, 3]). The basic idea of these approaches is to regularize the restoration process by utilizing the spatial redundancy of original static images. Compared to static image data, video data tends to be of lower quality due to the high speed capturing rate of video camera, e.g. lower signal-to-noise ratio and lower image resolution. Meanwhile, owing to its significant temporal redundancy, video data usually provides much richer information about the scene than static image data does. Thus, the efficiency of restoring degraded video data largely depends on how efficient the temporal redundancy is utilized in the methods. Some of patched based image restoration methods have been extended to the case of video denoising ([4, 5]). Although differing from details, these video restoration methods are built upon the same methodology that exploits the self-similarity in video by examining the the similarity among different image patches. The aforementioned patch-based image/video restoration methods showed impressive results on suppressing image noise when the noise is mostly Gaussian white noise. In practice, the existence of outliers in the image/video data is not rare and can be caused by many factors, e.g. electronic noise and errors in analog-to-digital converter. Also, scratches, blotches and dust are prevalent in archived VHS video
In REF , Ji et al. approximated the patch stack by reformulating the problem as a low rank matrix completion problem.
Robust video restoration by joint sparse and low rank matrix approximation
SIAM Journal on Imaging Sciences
Mathematics Computer Science