A motion estimating device for producing variable length coded motion vector

Abstract: A motion estimating device produces a variable length code motion vector (MV) which signifies the amount of movement of a block of pixel data. The motion estimating device (Figure 3) includes a mean error calculating circuit (100) that successively calculates the mean error between a small (N₁ X N₂) block of pixel data in a current picture (10) and a series of corresponding size blocks of data from a preceding reference picture (30), the reference picture blocks being shifted successively within a larger (M₁ X M₂) block (20) under the control of position coordinate signals (k,1). Each mean error is computed by unit (40) and corresponding variable length code data (D1) is produced by unit (60). Data (D2) based on the difference between the previously computed vector (MV) from delay (70) and the current position coordinates (k,1) is computed by unit (80) in variable length code form. Unit (50) seeks and outputs a minimum value of the thus computed error signals to produce a variable length code vector (MV) with minimised data content.

Description: The present invention relates to a motion estimating device comprising: data amount estimating means for estimating the amount of data required for variable length coding the output of the mean error calculating means; coded length operational means for calculating the length of code required to represent the displacement between said block of data in the reference signal and the block of data in the picture signal for each calculation when coded according to a variable length coding scheme; adding means for adding the amount of data required for variable length coding the output of the mean error calculating means and the length of code required to represent the displacement between the block of data in the reference signal and the block of data in the picture signal for each calculation; and minimum value seeking means for determining the minimum output of the adder for each picture block and outputting a motion vector signal representing the displacement between reference and picture signal blocks producing the minimum output from the adding means. The present invention finds application in high definition television (HDTV), high definition video tape recorders (HD-VCR), digital VCRs, digital camcorders, multimedia displays, videophones, video conferencing and the like. Interest has been shown recently in a data compression method which digitally encodes digital video and audio signals, for storage or transmission, and thereafter decodes the encoded signals for reproduction. In the aforementioned devices such as HDTV, HD-VCR, digital VCRs, digital camcorders, multimedia displays, videophones, video conferencing and the like, international efforts are being made to formulate standardised signal formats and data transmission protocols. A known system for data compression is shown in Figure 1 which illustrates a block diagram of a conventional motion compensated Differential Pulse Code Modulation (DPCM) coded transmitting device, which reduces the amount of data to be transmitted, for a video signal. According to Figure 1, an N X N transform unit 1 divides a picture into a gathering of small blocks of N X N (for example, N=8) in order to perform a data compression within the picture and performs a transform for respective blocks thereby to transform a video signal VS in the time domain into a signal in the frequency domain. The signal energy is gathered mainly toward low frequency areas by the said transform. The transform performed may be a Discrete Cosine Transform (DCT), a Walsh-Hadamard Transform (WHT), a Discrete Fourier Transform (DFT), a discrete Sine Transform (DST) or the like, although the DCT is most widely used. A quantizer 3 quantizes transform coefficients outputted from the N X N transform unit 1 in accordance with a quantization step size q determined by reference to the contents of a buffer 6. In this way, the transform coefficients are optimised such that the human eye can hardly perceive the effects of quantisation and data compression. The quantization step size q changes in accordance with the buffer state, and controls the quantization thereby to adjust the bit rate of the transmitted signal CVS, which is transmitted to a receiving terminal. A variable length coder 5 makes the most of statistical characteristics of the representative values quantized from the quantizer 3 and performs variable length coding thereby to compress the data. Reference is directed to GB 9300180.0 (GB-A-2264605) for a fuller description of the coding. The buffer 6 thus adjusts the transmitting bit rate of the data inputted from the variable length coder 5 thereby to output a substantially constant rate signal CVS, and at the same time, feedbacks the quantization step size q according to the degree to which the buffer is filled with data thereby to adjust the rate of output data produced by the quantizer 3, so that an overflow or underflow should not occur. The data outputted from the quantizer 3 can be further compressed by a motion compensation method. More particularly, the data can be compressed by utilization of the similarity between successive frames of the picture data; if there is no change between pictures in the case of still pictures, the data can be compressed further. A motion vector can be obtained from an estimation of motion for the blocks within the picture, and the block motions are compensated by utilization of a motion vector and because the difference between successive frames is very small, data compression can be achieved. A motion compensated DPCM method that compensates for motion between successive pictures is shown in Figure 1. The motion compensation is performed by a DPCM loop comprised of a frame memory 7, a motion estimation unit 8 and a motion compensation unit 9. An inverse quantizer 4 inversely quantizes the quantized data prior to the performance of the motion compensation and an N X N inverse transform unit 2 transforms a signal in the frequency domain into a video signal in the time domain. A motion estimating unit 8 receives video data for a prior picture stored in the frame memory 7 and also the current input video signal VS, and estimates motion as to from which place in the prior picture a currently-to-be-coded-block comes, and a motion vector MV thereof is obtained, thereby to be transmitted to a (remote) decoding unit (not shown) and simultaneously to be outputted to the motion compensation unit 9. The motion compensating unit 9 reads video data from a corresponding position of the frame memory 7 according to the motion vector MV obtained from the motion estimating unit 8 thereby to compensate for the motion. The compensated motion is added to an output of the N X N inverse transform unit 2 to thereafter be inputted to the frame memory 7. The picture compensated from the motion compensating unit 9 is further inputted to a first mixer Al to thereby produce an error signal by being subtracted from a current picture signal VS. At this moment, a first and a second switch SW1 and SW2 cut off an output signal from the motion compensation unit 9 either by block or by frame in order to prevent errors from enlarging on a picture where quantized noises generating from the DPCM loop are accumulated and restored, thereby to result in a realization of a role of intra frame coding. A block matching method is widely utilized as the method of obtaining the motion vector from the aforementioned motion estimating unit 8. Firstly, the present picture is partitioned into a motion block of N 1 X N 2 pixels, and the position from which the said block has come in the reference picture is sought within a seek domain of M 1 X M 2 in the reference picture (where M n > N n ). The place where a Mean Absolute Error MAE or Mean Square Error MSE is minimized, is determined as the value of the motion vector. Figure 2 is a block diagram of a conventional motion estimating unit for determining the motion vector. In Figure 2, a first N 1 X N 2 block forming unit 10, partitions the presently incoming picture into a block of size N 1 X N 2 . An M 1 X M 2 block forming unit 20 establishes a seek domain on a reference picture (a picture prior to a current picture or a prior picture reproduced from the DPCM loop) stored in the frame memory 7. Thus, in order to discriminate from which position of the reference picture the said N 1 X N 2 block has moved, a seek block of M 1 X M 2 which is larger than the block of N 1 X N 2 currently to be coded is established. A second N 1 X N 2 block forming unit 30 takes a data of N 1 X N 2 block size according to a horizontal/vertical displacement (k,l). An adder A10 obtains a data error outputted from the first and second N 1 X N 2 block forming units 10 and 30. A mean error calculating unit 40 calculates an absolute mean (or a square mean) of the errors outputted from the adder A10. A minimum value seek unit 50 selectively outputs as the motion vector MV the horizontal/vertical displacement (k,l) when the same possesses the smallest error value out of the error values calculated from the mean error calculating unit 40. In other words, the error value is calculated while the horizontal/vertical displacement (k,l) is successively changed, and a block is recognized as a moved block when the calculated error value is the smallest so that the horizonal/vertical displacement (k, l) at this moment is recognized as the motion vector MV to thereby be outputted. Methods such as the Mean Absolute Error or Mean Square Error among pixels of respective blocks and the like are typically utilized to obtain the mean error from the mean error calculating unit 40. However, although the conventional technique according to the aforesaid motion vector estimation is effective when a motion vector is transmitted as a code of fixed length, the final data amount is not minimized at all times when the motion vector is variable length coded. Thus, there is a problem in that the motion vector tends to be increased if the code length of the motion vector is lengthened when the motion vector is variable length coded, even though the data amount of image signal is minimised. This problem is addressed in US-A-4562468 and US-A-4460923. These documents disclose selecting a motion vector on the basis of the comparison between a current image block and a previously imput image block that produces the shortest motion vector code for transmission. A device according to the present invention is characterised by mean error calculating means for successively calculating a mean error signal as a function of a mean error between respective blocks of data in reference and picture signals, and in that the data amount estimating means is arranged to estimate said amount of data, in dependence on an input quantization step size, and a mean error signal calculated by the mean error calculating means. Preferably, the data amount estimating means calculates an estimate for said amount of data according to the formula:- D 1= k 1 × E ( k,l ) × q - k 2 where D1 is the estimated data amount, E(k,l) is the means error, q is the quantization step size and k 1 and k 2 are empirically derived constants. A device according to the present invention preferably includes delaying means for delaying the motion vector from the minimum value seeking means, means for determining the difference between the delayed motion vector and the displacement between the present picture and reference signal block, and the coded length operational means operating on the basis of the output of the means for determining the difference between the delayed motion vector and the displacement between the present picture and reference signal blocks. A device according to the present invention preferably includes means for selectively applying the delayed motion vector to the means for determining the difference between the delayed motion vector and the displacement between the present picture and reference signal block to thereby switch between pulse code modulation and differential pulse code modulation representation of the motion vector. A device according to the present invention preferably includes means for producing data compressed signals corresponding to successive ones of said picture signals utilising a variable length code having an adjustable quantization step of a value which is varied to control the data transmission rate, said parameter determining means being responsive to the value of said step. Preferably, said mean error calculating means comprises first means for forming a first block of data from said picture signal; second means for forming a block of data from the reference picture signal larger than said second block; means for successively producing from data in the second block, blocks of data of the same dimensions as the first block but dimensionally shifted within the second block in response to applied coordinate signals. An embodiment of the present invention will now be described, by way of example, with reference to Figure 3 of the accompanying drawings, in which: Figure 1 is a block diagram of a conventional motion compensated coding device; Figure 2 is a block diagram of a conventional motion estimating device; and Figure 3 is a block diagram of a motion estimating device for a system in accordance with the present invention. With reference to Figure 3, the motion estimating device comprises: a mean error calculating means 100 for calculating a mean error between pixel data of a reference picture and a current picture, for estimating motion; and a motion vector setting means 200 which, in response to receipt of the mean error outputted from the mean error calculating means 100 and also the quantized step size q generated according to the degree to which the buffer 6, (see Figure 1) is filled with data, establishes as a motion vector MV as a horizontal/vertical displacement (k,l) wherein data to be variable length coded for transmission is minimized. The mean error calculating means 100 constructed as above comprises: a first N 1 X N 2 block forming unit 10 for partitioning the currently incoming picture into a pixel block of N 1 X N 2 size; an M 1 X M 2 block forming unit 20 for establishing a seek domain on a reference picture (a picture preceding to a current picture, or a preceding picture reproduced from the DPCM loop) stored in the frame memory 7; a second N 1 X N 2 block forming unit 30 for taking a pixel data of N 1 X N 2 block size in accordance with the horizontal/vertical displacement (k,l) among the pixel data established in the M 1 X M 2 block forming unit 20; an adder A10 for obtaining a data error outputted from the first and second N 1 X N 2 block forming units 10 and 30; and a mean error calculating unit 40 for calculating an absolute means (or mean square) of errors output from the adder A10. The motion vector setting means 200 comprises: a variable length coded data amount estimating unit 60 for estimating the mean error outputted from the mean error calculating unit 100 and the data amount of the video signal variable length coded in accordance with the quantized step size q; a coded length operational unit 80 for determining the coded length of the variable length coding when the current horizontal/vertical displacement (k, l) is selected as a motion vector; a second adder A20 for adding the output data D1 from the variable length coded data amount estimating unit 60 to output data D2 from the coded length operational unit 80; a minimum value seek unit 50 for receiving the data D1 + D2 added at the second adder A20 and for further receiving the horizontal/vertical displacement (k,l) to thereafter seek the horizontal/vertical displacement (k, l) where the adder data D1 + D2 is minimized, so that the sought horizontal/vertical displacement (k,l) can be selected as the motion vector MV to thereby be outputted; a delayer 70 for delaying the motion vector MV outputted from the minimum value seek unit 50 for a predetermined period of time; and a third adder A30 for calculating an error between the motion vector MV delayed at the delayer 70 and the current horizontal/vertical displacement (k, l) to thereafter output the same to the coded length operational unit 80. Next, operation of the motion estimating device will be described in more detail. When frame data VS of the current picture is inputted, the first N 1 X N 2 block forming unit 10 partitions the current picture into a pixel block of N 1 X N 2 size to thereafter output the pixel data of corresponding blocks to the first adder A10. The M 1 X M 2 block forming unit 20 partitions a frame data of the reference picture inputted from the frame memory 7 as illustrated in Figure 1 into a seek domain block of M 1 X M 2 size, where M > N. The reference picture denotes a picture preceding a current picture or a preceding picture reproduced from the DPCM loop but in this embodiment of the present invention, a simply inputted video signal can be stored on a separate frame memory, delayed and taken as a reference picture. In other words, in order to discriminate which position of the reference picture the N 1 X N 2 blocks has moved to form the current picture, a seek block of M 1 X M 2 which is larger than a block of N 1 X N 2 currently calling for encoding is established. The second N 1 X N 2 block forming unit 30 partitions a pixel block of M1 X M2 seek domain in the M 1 X M 2 block forming unit 20 into a pixel block of N 1 X N 2 size to thereby output the pixel data in accordance with the horizontal/vertical displacement (k, l). The first adder A10 adds the data outputted from the first N 1 X N 2 block forming unit 10 to the data outputted from the second N 1 X N 2 block forming unit 30 in accordance with the horizontal/vertical displacement (k, l), so that an error resulting out of these two data can be outputted. The mean error calculating unit 40, when the error outputted from the first adder A10 is inputted, calculates a mean error of the pixel data. As previously stated, methods such as a Mean Absolute Error MAE or Mean Square Error MSE can be used to obtain the mean error in the mean error calculating unit 40. An explanation employing the Mean Absolute Error MAE method will now be given as an example. A calculating formula for the Mean Absolute Error method can be defined as follows: The variable length coded data amount estimating unit 60, when the mean error E (k, l) calculated from Formula 1 and the quantized step size q are inputted, estimates a data amount generated when the error signal {x(i, j)-y(i+k,j+1)} is variable length-coded from the mean error E(k, l) and quantized step size q. At this moment, the estimated data amount (D1) is approximately proportionate to the mean error E (k, l) and is inversely proportional to the step size q, which can be expressed in the following formula D1 = k 1 X E(k, l) X q -k2 From the foregoing Formula 2, k 1 and k 2 are any integers which can be determined experimentally to achieve optimum values. The data amount D1 estimated by the formula 2 in the variable length coded data amount estimating unit 60 is inputted to the second adder A20. Meanwhile, at the coded length operational unit 80, when the horizontal/vertical displacement (k, l) is selected as a motion vector, the displacement is directly transformed to Pulse Code Modulation PCM or to a DPCM which due to the action of adder A30 seeks a difference between a motion vector of preceding block and a motion vector of present block, to perform a variable length coding on this difference, and to output the resulting code (data amount : D2). At this moment, the coded length (D2) has already been determined in accordance with a statistical characteristic. The second adder A20 adds the coded length D2 outputted from the coded length operational unit 80 to a video signal coded data amount D1 outputted from the variable length coded data amount estimating unit 60. A final data amount D1 + D2 added at the second adder A20 is inputted to a minimum value seek unit 50 which is responsive to changes in the horizontal/vertical displacement (k, l) and detects a horizontal/vertical displacement where the final data amount D1 + D2 is minimized, to thereby output the same as a motion vector MV. The delayer 70 delays the motion vector MV outputted from the minimum value seek unit 50 to thereafter output the same. In other words, the motion vector MV detected from the foregoing process is stored on the delayer 70 for the DPCM of a motion vector in the next block. A switch SW selects PCM/DPCM in order to encode the motion vector. As seen from the foregoing, the motion estimating device, having regard to the variable length coding, seeks an optimum motion vector, generating the minimum data amount needed to make the motion vector variable-length encoded, so that the problem of increased transmission data can be solved. It will be apparent to one skilled in the art that, although the foregoing description of the embodiment has been presented for the purpose of illustration and description, various changes and modifications can be made. In particular, in the embodiment in Figure 3, an example of a block matching according to a full search was given, but it will be apparent that a three step search, hierarchical search and motion vector estimating methods other than the full search can be applied to the present invention.

Claims: A motion estimating device comprising; data amount estimating means (60) for estimating the amount of data (D1) required for variable length coding the output of mean error calculating means (100); coded length operational means (80) for calculating the length of code (D2) required to represent the displacement between a block of data in a reference signal and the corresponding block of data in a picture signal for each calculation when coded according to a variable length coding scheme; adding means (A20) for adding the amount of data (D1) required for variable length coding the output of the mean error calculating means (100) and the length of code (D2) required to represent the displacement between the block of data in the reference signal and the block of data in the picture signal for each calculation; and minimum value seeking means (50) for determining the minimum output of the adder for each picture block and outputting a motion vector signal (MV) representing the displacement between reference and picture signal blocks producing the minimum output from the adding means; characterised by mean error calculating means (100) for successively calculating a mean error signal (E(k, l)) as a function of a mean error between respective blocks of data in reference and picture signals, and in that the data amount estimating means (60) is arranged to estimate said amount of data, in dependence on an input quantization step size (q), and a mean error signal calculated by the mean error calculating means (100). A device according to claim 1, wherein the data amount estimating means (60) calculates an estimate for said amount of data according to the formula:- D 1= k 1 × E ( k,l ) × q - k 2 where D1 is the estimated data amount, E(k,l) is the mean error, q is the quantization step size and k 1 and k 2 are empirically derived constants. A device according to claim 1 or 2, including delaying means for delaying the motion vector (MV) from the minimum value seeking means (50), means (A30) for determining the difference between the delayed motion vector (MV) and the displacement between the present picture and reference signal block, and the coded length operational means (80) operating on the basis of the output of the means (A30) for determining the difference between the delayed motion vector and the displacement between the present picture and reference signal blocks. A device according to claim 3, including means (S/W) for selectively applying the delayed motion vector (MV) to the means (A30) for determining the difference between the delayed motion vector (MV) and the displacement between the present picture and reference signal block to thereby switch between pulse code modulation and differential pulse code modulation representation of the motion vector (MV). A device according to any preceding claim, including means for producing data compressed signals (CVS) corresponding to successive ones of said picture signals utilising a variable length code having an adjustable quantization step (q) of a value which is varied to control the data transmission rate, said parameter determining means being responsive to the value of said step. A device according to any preceding claim, wherein said mean error calculating means (100) comprises first means (10) for forming a first block of data from said picture signal; second means (20) for forming a block of data from the reference picture signal larger than said second block; means for successively producing from data in the second block, blocks of data of the same dimensions as the first block but dimensionally shifted within the second block in response to applied coordinate signals (k, l).

Inventors: JEONG JE-CHANG, JEONG, JE-CHANG
Applicants: SAMSUNG ELECTRONICS CO LTD, SAMSUNG ELECTRONICS CO., LTD.
Application year: 1993
Main classification: H04N7
IPC codes: H04N7, H04N11, H03M7, G06T7, G06T9, H04N5