# Object-Wise Video Editing
## 1 Introduction  
Video editing at the object level is a challenging task due to the difficulties of dynamic object region segmentation and novel view video synthesis. Inserting a new object or removing an unexpected object from a video are challenging but interesting tasks. In both tasks, background inpainting for filling a missing region after object removal is a critical part of realistic new video synthesis. The missing region has to be appropriately inferred from given background information in the video. One of the familiar inpainting algorithms from single images is the exemplar-based approach introduced in References [1; 2]. The exemplar-based approach generates quick inpainting outputs when the hole is comparatively small. On the other hand, it suffers from insufficient visual coherence globally for bigger holes. The most critical and important aspect of the video inpainting method is how consistently random holes are filled with appropriate information in both space and time dimensions. It can be performed by the single image inpainting approach which considers patches over the neighbour frame, taking into account the flow of motion information [3; 4; 5; 6] or global-space time consistency optimized by the global minimization energy function [7]. The existing inpainting methods are not suitable for dealing with a video including dynamic objects and complex video scenes taken from a moving camera, which is common in many video clips captured by daily used devices like smartphones. To overcome such problems, inter-frame registration and respective pixel-wise calibration throughout the frames in a video are needed. Motion-based alignment techniques for inpainting are proposed in References [8; 9; 10; 11]. These methods try to compute the motion information of each pixel or patch to find the flow of camera movement throughout the video. A robust andfast alignment between the cameras can be performed using Homography transformation calculated from sparse key-points. Homography-based approaches are usually suitable for video editing tools to handle the problem of camera movement [10; 12]. However, the single Homography transformation matrix is not capable of estimating dense corresponding points of the missing region in video inpainting task. In 3D video editing, inpainting methods show a reasonable performance when the measurement of 3D information is accurate. Inpainting in 3D space can be performed when accurate 3D translation matrix between frames is possible to be extracted. However, calculation of accurate transformation matrix with moving camera is not a trivial task, where scene space relationship among the frames benefits diverse 3D video processing [13]. A Homography based inpainting method introduced in Reference [10] considers 3D scene information having piece-wise planner geometry to remove a moving object from video taken with moving camera. To fill the background of moving object, they use a single source frame based information maintaining the consistency of background color. A region-based Homography approach is applied in video inpainting in Reference [12] with static and moving cameras. Considering the computational complexity of their methods, they consider the alignment between groups of neighboring frames. The alignment among the neighboring groups is calculated by minimizing an energy function of both spatial and temporal coherence. A sampling-based approach proposed in Reference [14] which allows to perform high quality video processing like denoising, super-resolution, inpainting, and deblurring. They consider the utilization of redundant scene information over the frames of video instead of improving a separate 3D scene information producing rather blurry inpainting results. And it requires human intervention to remove the foreground object point cloud from the scene to get the background depth. Another important task in video editing is novel view synthesis. Reconstructing a different view image or video in computer vision is a challenging task.