Apparatus and method for detecting motion vectors to half-pixel accuracy

Abstract: Motion vectors between a current frame and its preceding frame of video signals are optimally determined to a half-pixel accuracy by finding a plurality of full-pixel resolution motion vectors corresponding to a plurality of best matching blocks in a search region; and, thereafter, deriving a half-pixel resolution motion vector by selecting an half-pixel resolution block which yields the minimum error function out of all possible half-pixel resolution candidate blocks included in half-pixel resolution search regions, each of which corresponding to one of the full-pixel motion vectors.

Description: Field of the Invention The present invention relates to a method and an apparatus for determining motion vectors between a current frame and a preceding frame of video signals to a half-pixel accuracy. Description of the Prior Art As is well known, transmission of digitized video signals can attain video images of a much higher quality than the transmission of analog signals. When an image signal comprising a sequence of image "frames" is expressed in a digital form, a substantial amount of data is generated for transmission, especially in the case of a high definition television system. Since, however, the available frequency bandwidth of a conventional transmission channel is limited, in order to transmit the substantial amounts of digital data therethrough, it is inevitable to compress or reduce the volume of the transmission data. Among various video compression techniques, the so-called hybrid coding technique, which combines temporal and spatial compression techniques together with a statistical coding technique, is known to be most effective. Most hybrid coding techniques employ a motion compensated DPCM(differential pulse code modulation), two-dimensional DCT(discrete cosine transform), quantization of DCT coefficients, and VLC(variable length coding). The motion compensated DPCM is a process of determining the movement of an object between a current frame and its previous frame, and predicting the current frame according to the motion flow of the object to produce a differential signal representing the difference between the current frame and its prediction. This method is described, for example, in Staffan Ericsson, "Fixed and Adaptive Predictors for Hybrid Predictive/Transform Coding", IEEE Transactions on Communications , COM-33 , No. 12(December 1985); and in Ninomiya and Ohtsuka, "A Motion-Compensated Interframe Coding Scheme for Television Pictures", IEEE Transactions on Communications , COM-30 , No. 1 (January 1982). The two-dimensional DCT, which reduces or removes spatial redundancies between image data, converts a block of digital image data, for example, a block of 8x8 pixels, into a set of transform coefficient data. This technique is described in Chen and Pratt, "Scene Adaptive Coder", IEEE Transactions on Communications , COM-32 , No. 3(March 1984). By processing such transform coefficient data with a quantizer, zigzag scanning, and VLC, the amount of data to be transmitted can be effectively compressed. Specifically, in the motion compensated DPCM, current frame data is predicted from the corresponding previous frame data based on an estimation of the motion between the current and the previous frames. Such estimated motion may be described in terms of two dimensional motion vectors representing the displacement of pixels between the previous and the current frames. Several methods for estimating the displacement of an object in a video sequence have been proposed. Generally, they can be classified into two types: pixel recursive algorithm; and block matching algorithm( see , e.g., J.R.Jain et al., "Displacement Measurement and Its Application in Interframe Image Coding", IEEE Transactions of Communications COM-29 , No. 12 (December 1981)). The present invention is primarily concerned with the block matching algorithm. According to the block matching algorithm, a current frame is divided into a multiplicity of search blocks. The size of a search block typically ranges between 8x8 and 32x32 pixels. To determine a motion vector for a search block in the current frame, a similarity calculation is performed between the search block of the current frame and each of a plurality of equal-sized candidate blocks included in a generally larger search region within a previous frame. An error function such as the mean absolute error or mean square error is used to carry our the similarity measurement between the search block of the current frame and each of the candidate blocks in the search region. And a motion vector, by definition, represents the displacement between the search block and a candidate block which yields a minimum error function. In a motion compensated coding method, the coding performance depends heavily on the accuracy of the motion vector. In the current available schemes such as ISO/IEC MPEG standards, a motion vector may be detected to a half-pixel accuracy ( see MPEG Video Simulation Model Three, International Organization for Standardization, Coded Representation of Picture and Audio Information, 1990, ISO-IEC/JTC1/SC2/WG8 MPEG 90/041). In a conventional half-pixel resolution motion estimator, motion estimation is performed in two steps. First, an integer-valued full-pixel resolution motion vector is derived for each search block in the current frame. Then, the full-pixel resolution motion vector is refined to a half-pixel accuracy, to thereby determine a final half-pixel resolution motion vector of a block. The half-pixel resolution motion vector may have a non-integer value and point to a location between full-pixel locations. The candidate blocks for estimation and compensation in the half-pixel resolution, i.e., half-pixel resolution candidate blocks, are spatially interpolated blocks of the previous frame and also included in a corresponding search region, which is called a half-pixel resolution search region ( see , e.g., EP Pat. No. 0560577A2 issued to Lucas et al.). The search for the half-pixel resolution motion vector usually follows the full-pixel resolution motion estimation such that the half-pixel resolution motion vector is searched around a location indicated by the full-pixel resolution motion vector. In case the half-pixel resolution candidate block which yields the minimum error function were not included in the half-pixel resolution search region resulting from the full-pixel resolution motion vector of the block, however the final half-pixel resolution motion vector which will be derived from the half-pixel resolution search region would not be optimal inasmuch as the other half-pixel resolution displacement would yield a smaller error function. In the US-A-4937666 there is described a circuit for determining a motion vector representing the displacement between a current and a preceding frame of a video signal with fractional pixel precision. For this purpose, at first one single full-pixel resolution motion vector is determined. Then to obtain a correction to this vector a fractional-pixel displacement vector is determined by comparing a search block with fractional-pixel resolution candidate blocks placed in the surrounding of the full-pixel resolution vector. The document EP-A-0 468 279 refers to a method for determining motion vectors whereby the accuracy of a full-pixel resolution motion vector is increased to half-pixel resolution by interpolating the full-pixel resolution motion vector and adjacent full-pixel resolution vectors. The article "A Fast Feature Matching Algorithm of Motion Compensation for Hierarchical Video CODEC" by X. Lee, Proceedings SPIE Visual Communications and Image Processing '92, vol. 1818, pages 1462 - 1474 describes a method for decreasing the computational effort when determining full-pixel resolution motion vectors. This is achieved by employing special reduced resolution video frames instead of the original full-pixel frames. Summary of the Invention It is, therefore, a primary object of the present invention to provide a method and an apparatus for deciding an accurate half-pixel resolution motion vector of a block, which is not restricted to a half-pixel resolution search region defined by a full-pixel resolution motion vector of the block, thereby improving the overall compression efficiency. The above and other objects are achieved by a method as claimed in claim 1, and an apparatus as claimed in claim 3. Further advantageous features of the invention are defined in the subclaims. Brief Description of the Drawings The above and other objects and features of the present invention will become apparent from the following description of preferred embodiments given in conjunction with the accompanying drawings, in which: Fig. 1 is a block diagram of a prior art video encoder that employs the motion estimator of the present invention; Fig. 2 shows a block diagram of the motion estimator of the present invention; Fig. 3 schematically illustrates the full-pixel resolution motion vector detector shown in Fig. 2; Fig. 4 represents a flow chart explaining the procedure carried out in the motion vector comparator of Fig. 3; and Fig. 5 depicts a block diagram of the half-pixel resolution motion vector detector of Fig. 2. Detailed Description of the Preferred Embodiments With reference to Fig. 1, there is shown a block diagram of a video encoder which employs a motion estimator 110 of the present invention. Current frame data 101 is fed to a subtracter 102 and the motion estimator 110. Actually, the current frame data is bead on a block-by-block basis, from an input memory(not shown) wherein each frame is stored as successive blocks of pixel data for the processing thereof on a block-by-block basis. The block size typically ranges between 8x8 and 32x32 pixels. A block of predicted frame data 103 from a motion compensated predictor 122 is subtracted from the search, block of the current frame data at the subtracter 102; and the resultant data, i.e., differential pixel data, is dispatched to a transform coder 105, wherein a block of differential pixel data is encoded into a set of transform coefficients by using, e.g., a discrete cosine transform. The transform coefficients are then quantized at a quantizer 106. Thereafter, the quantized transform coefficients are transmitted via two signal paths: one that leads toward an entropy coder 107, wherein the quantized transform coefficients are coded together with the motion vector 119 by using, for example, a combination of run-length and variable length coding; and the other that leads to an inverse quantizer 113 and then to an inverse transform coder 114, wherein the quantized transform coefficients are converted back into reconstructed differential pixel data by inverse quantization and inverse transformation, respectively. The reconstructed differential pixel data and predicted data from the motion compensated predictor 122 are combined at an adder 115 to provide reconstructed current frame data to be written onto a frame memory 121. The current frame data 101 and the reconstructed previous frame data 118 from the frame memory 121 are processed to determine motion vectors by using the block-matching algorithm at the motion estimator 110 of the present invention, which will be described hereinafter with reference to Figs. 2 to 5. The motion vector 119 is applied to the motion compensated predictor 122, wherein the predicted data, i.e., the candidate block data, corresponding to the motion vector 119 is retrieved from the frame memory 121 on a block-by-block basis and provided to the subtracter 102 and the adder 115 for further processing. And also the motion vector 119 is fed to the entropy coder 107. In the motion estimator 110 of the present invention, a motion vector is determined in two steps with different resolution sizes as described hereinafter. Fig. 2 illustrates a block diagram of the motion estimator 110 of the present invention shown in Fig. 1. The current frame data 101 and the reconstructed previous frame data 118 are applied to a full-pixel resolution motion vector detector 160 and to a half-pixel resolution motion vector detector 170. The full-pixel resolution motion vector detector 160 determines a plurality of full-pixel resolution motion vectors corresponding to the plurality of best matching blocks in a search region. These full-pixel resolution motion vectors are dispatched to a half-pixel resolution motion vector detector 170, wherein a half-pixel resolution motion vector is determined by selecting a half-pixel resolution block which yields a minimum error function out of all half-pixel resolution candidate blocks included in the plurality of half-pixel resolution search regions, each of which corresponding to one of the full-pixel motion vectors. With reference to Fig. 3, there is shown a block diagram of the full-pixel resolution motion vector detector 160 shown in Fig. 2. Current frame data 101 and the previous frame data 118 are inputted to a block selector 220. At the block selector 220, a group of full-pixel resolution candidate blocks in a search region corresponding to a search block of the current frame are selected from the previous frame. These selected candidate blocks on line 222 and the search block on line 224 are inputted to a block comparator 230. The block comparator 230 calculates the error functions, each of the error functions representing the difference between the search block in the current frame and each of the candidate blocks with an integer valued displacement, included in the search region of the previous frame. These error functions on line 236 and corresponding displacements on line 233 from the block selector 220 are inputted to a motion vector comparator 240. The motion vector comparator 240 compares the error functions and selects a plurality of displacements which yield a plurality of smallest error functions in accordance with the procedure described in Fig. 4. Fig. 4 shows a procedure to select a predetermined number, e.g., N, of full-pixel resolution motion vectors at the motion vector comparator 240. SE i 's and SD i 's are two groups of variables to store N selected error functions and corresponding displacements respectively, wherein i is a positive integer value ranging from 1 to N. In step S1, SE i 's are initialized to a value larger than the maximum error function value. The initial value of j is 1. Therefore, in step S2, an error function E 1 and a displacement D 1 of a first candidate block are read from the block selector 220 and the block comparator 230 shown in Fig. 3. Then, E 1 is compared with all the SE i values in step S3. In step S4, if one of SE i values is larger than E 1 , the largest SE i value is discarded and the remaining SE i values and E 1 are rearranged, e.g., in an ascending order with the smaller value of i being related to the smaller error function value, and stored back in SE i 's for the processing of a next candidate block. If all SE i values are smaller than E 1 , SE i 's remain the same. Displacements are also stored in SD i 's in accordance with the rearranged SE i 's in step S4. Actually, all SE i values are initialized to an identical value and are larger than E 1 . In case that multiple SE i 's have the same largest value and the value is larger than E 1 , the value in any the SE i 's is discarded and replaced by E 1 . Steps S2 through S4 are repeated for the next candidate block, i.e., second candadate block in the search region. Specifically, in step S2, an error function E 2 and a displacement D 2 of a second candidate block are read. Then, E 2 is compared with all the SE i values in step S3. In step S4, if one of SE i values is larger than E 2 , the largest SE i value is discarded and the remaining SE i values and E 2 are rearranged and stored back in SE i 's for the processing of a next candidate block. Displacements are also stored in SD i 's in accordance with the rearranged SE i 's in step S4. Steps S2 through S4 are repeated for the remaining candidate blocks until all the candidate blocks in the search region are processed. After all the candidate blocks have been processed, i.e., j equals the number of candidate blocks in the search region, the displacements remaining in SD i 's are determined as the full-pixel resolution motion vectors with respect to the search block. Fig. 5 is a block diagram of the half-pixel motion vector detector 170 shown in Fig. 2. Interpolated frame data, i.e., the previous frame data in a half-pixel position, are calculated at a frame interpolator 410. A block selector 420 receives the interpolated frame data and the current frame data 101 together with the full-pixel resolution motion vectors 245 from the motion vector comparator 240 shown in Fig. 3. At the block selector 420, half-pixel resolution candidate blocks in a half-pixel resolution search region corresponding to one of the full-pixel resolution motion vectors are selected from the interpolated frame. An end-of-search signal is also produced at the block selector 420 upon the completion of processing each search block. One of the half-pixel resolution candidate blocks on line 422 and the search block of the current frame on line 423 are dispatched to a block comparator 430, to thereby calculate an error function between the two blocks therein. The corresponding half-pixel resolution displacement between the two blocks on line 421 is fed to a latch 460. The error function A is dispatched to a comparator 440 and to a latch 450. At the start of each search block processing, B in the comparator 440 is initialized to a certain large value, to be replaced with error functions as explained hereinafter. The comparator 440 compares the error function A with the stored value B. If A < B, then the comparator outputs a pulse to the latch 450 which latches or stores the error function A in place of B, thereby replacing the previously stored value with the new error function. In this case, the displacement of the half-pixel resolution block is also latched by the latch 460. If A ≥ B, the value B remains unchanged, and the displacement in latch 460 remains the same. After the block selector 420 outputs all the half-pixel resolution candidate blocks associated with the full-pixel resolution motion vector, the process of calculating and comparing error functions are repeated for a next search region indicated by a next full-pixel resolution motion vector. After all the half-pixel resolution search regions corresponding to each of the full-pixel resolution motion vectors have been processed, the half-pixel resolution displacement which yields the minimum error function in all the search regions corresponding to each of the full-pixel resolution motion vectors remains in the latch 460. And the end-of-search signal, produced at the block selector 420, latches the displacement at the latch 460 into the latch 470, wherein the value on the output lead represents the final half-pixel resolution motion vector associated with the search block of the current frame. In accordance with this method, the half-pixel resolution motion estimation is performed for a plurality of search regions corresponding to each of the full-pixel resolution motion vectors, providing a more correct half-pixel resolution motion vector. Although the present invention has been described with respect to a half-pixel resolution motion vector, motion vectors, which are calculated in terms of various sub-pixel resolutions, e.g., a 1/4 pixel resolution, can be used in encoding a motion picture, and such motion vectors can be calculated in a similar manner as presented herein.

Claims: A method for determining motion vectors between a current frame (101) and a preceding frame (118) of video signals to a half-pixel accuracy, wherein the current frame (101) is divided into a multiplicity of search blocks of an identical size and the preceding frame (118) is made to include a corresponding number of search regions, each of the search regions having M candidate blocks of said identical size, M being an integer greater than 1, which comprises the steps of: (a) calculating M first error values, each of the M first error values representing a difference between a search block in the current frame (101) and a full-pixel resolution candidate block, included in a corresponding search region of the preceding frame (118); (b) selecting K full-pixel resolution motion vectors (245) for the search block which yield K smallest first error values among said M first error values, K being an integer greater than 1 and smaller than M; (c) obtaining M half-pixel resolution candidate blocks in a half-pixel resolution search region corresponding to one of the K full-pixel resolution motion vectors (245); (d) calculating M second error values, each of the M second error values representing a difference between the search block in the current frame (101) and a corresponding half-pixel resolution candidate block; (e) repeating steps (c) to (d) for the remaining (K-1) full-pixel resolution motion vectors (245) obtained in step (b), thereby providing KxM second error values; (f) selecting a half-pixel resolution motion vector for the search block, said half-pixel resolution motion vector yielding a second error value which is smallest among the KxM second error values; and (g) repeating steps (a) to (f) until all the search blocks in the current frame (101) are processed. The method according to claim 1, wherein said step (b) includes the steps of: (b1) initializing a first group of variables, SE i 's, which stores K first error values, wherein i is a positive integer ranging from 1 to K; (b2) reading a first error value E and a displacement D corresponding to a candidate block in a search region of the preceding frame (118); (b3) comparing said E with the values stored in said SE i 's; (b4) arranging the values stored in said SE i 's and said E in an ascending order and storing K arranged values in said SE i 's except the largest one; (b5) storing displacements, each of said displacements corresponding to one of the values stored in SE i 's, in a second group of variables, SD i ' s, wherein i is the positive integer ranging from 1 to K; and (b6) repeating steps (b1) to (b5) until all of the candidate blocks in the search region are processed, wherein the displacements stored in SD i 's, in the moment the repeating operation is finished, are determined as the K full-pixel resolution motion vectors (245). An apparatus for determining motion vectors between a current frame (101) and a preceding frame (118) of video signals to a half-pixel accuracy, wherein the current frame (101) is divided into a multiplicity of search blocks of an identical size and the preceding frame (118) is made to include a corresponding number of search regions, each of the search regions having M candidate blocks of said identical size, M being an integer greater than 1, which comprises: means for calculating M first error values, each of the M first error values representing a difference between a search block in the current frame (101) and a full-pixel resolution candidate block, included in a corresponding search region of the preceding frame (118); means selecting K full-pixel resolution motion vectors (245) for the search block which yield K smallest first error values among said M first error values, K being an integer greater than 1 and smaller than M; means for obtaining M half-pixel resolution candidate blocks in a half-pixel resolution search region corresponding to each of the K full-pixel resolution motion vectors (245) to thereby provide KxM half-pixel resolution candidate blocks; means for calculating KxM second error values, each of the KxM second error values representing a difference between the search block in the current frame (101) and a corresponding half-pixel resolution candidate block; and means for determining a half-pixel resolution motion vector for the search block, said half-pixel resolution motion vector yielding a second error value which is smallest among the KxM second error values. The apparatus as recited in claim 3, wherein the K full-pixel resolution motion vectors (245) selecting means includes: means for reserving K predetermined values arranged in an ascending order; means for sequentially receiving each of the M first error values and its corresponding displacement, comparing a received first error value and the K predetermined values to select a largest one among them, and storing displacements corresponding to the remainder of the largest one; means for rearranging the remainder of the largest one in an ascending order and providing the rearranged values to the reserving means; and means for repeating the reserving, the storing, and the rearranging processes for a next received first error value until all of the received M first error values are processed, thereby providing the stored displacements as the K full-pixel resolution motion yectors (245).

Inventors: KIM SANG-HO, KIM, SANG-HO
Applicants: DAEWOO ELECTRONICS CO LTD, DAEWOO ELECTRONICS CO., LTD
Application year: 1994
Main classification: H04N7
IPC codes: H04N7, G06T7