Block-based motion estimation (ME) and motion compensation (MC) techniques are widely used in modern video processing algorithms and compression systems. The great variety of video applications and devices results in diverse compression specifications, such as frame rates and bit rates. In this paper, we study the effect of frame rate and compression bit rate on block-based ME and MC as commonly utilized in inter-frame coding and frame rate up-conversion (FRUC). This joint examination yields a theoretical foundation for comparing MC procedures in coding and FRUC. First, the video signal is locally modeled as a noisy translational motion of an image. Then, we theoretically model the motion-compensated prediction of available and absent frames as in coding and FRUC applications, respectively. The theoretic MC-prediction error is studied further and its autocorrelation function is calculated, yielding useful separable-simplifications for the coding application. We argue that a linear relation exists between the variance of the MC-prediction error and temporal distance. While the relevant distance in MC coding is between the predicted and reference frames, MC-FRUC is affected by the distance between the frames available for interpolation. We compare our estimates with experimental results and show that the theory explains qualitatively the empirical behavior. Then, we use the models proposed to analyze a system for improving of video coding at low bit rates, using a spatiotemporal scaling. Although this concept is practically employed in various forms, so far it lacked a theoretical justification. We here harness the proposed MC models and present a comprehensive analysis of the system, to qualitatively predict the experimental results. Yehuda Dar received the B.Sc. degree in computer engineering and the M.Sc. degree in electrical engineering from the Technion-Israel Institute of Technology, Haifa, in 2008 and 2014, respectively, where he is currently pursuing the Ph.D. degree with the Department of Computer Science. His research interests are in image and video processing, in particular, signal compression and video coding. Alfred M. Bruckstein received the B.Sc. and M.Sc. degrees from the Technion-Israel Institute of Technology (Technion), Haifa, in 1976 and 1980, respectively, and the Ph.D. degree in electrical engineering from Stanford University, CA. Since 1984, he has been with Technion, where he holds the Ollendorff Chair in Science. His current research interests are in swarm/ant robotics, image and signal processing, analysis and synthesis, pattern recognition, and various aspects of applied geometry.
The great variety of video applications and devices results in diverse compression specifications, such as frame rates and bit rates REF .
Motion-Compensated Coding and Frame Rate Up-Conversion: Models and Analysis
IEEE Transactions on Image Processing
IEEE Transactions on Image Processing
Computer Science Mathematics Medicine