Motion detection method and its apparatus

Abstract: A motion detecting method and its apparatus are provided in which during block matching procedure for detecting the movement, error data in displacement between reference two-dimensional blocks and target two-dimensional blocks to be examined of a frame are calculated by a known manner and also, a movement throughout the two-dimensional blocks is detected by summing up a series of the error data at each displacement point of the blocks and selecting the smallest of the error data sum.

Description: The present invention relates to a method of detecting a movement of an object in a moving picture and an apparatus for performing the method. A method of detecting a movement of an object in a series of time consecutive moving pictures is commonly used for motion-compensated interframe coding in the technical field of motion picture coding. One of the conventional motion detecting apparatuses will be explained referring to the drawings. Fig.5 is a block diagram of a conventional motion detecting apparatus employing the known block matching method. Fig.6 is an explanatory diagram showing this motion detecting method. Provided in Fig.5 are a buffer memory 1, an error calculating circuit 2, an error comparing circuit 3, and an address generating circuit 4. In Fig.6, the numeral 61 represents a two-dimensional block in a t-th frame picture, 62 is a block in a (t-1)th frame picture for which errors from the block 61 are calculated, and 63 is a motion vector calculated by motion detection. It is assumed that the t-th frame is a reference frame and a motion vector between the (t-1)th frame and the t-th frame is calculated for each two-dimensional block consisted of mxn pixels. The picture data of each frame are first stored into the buffer memory 1 and then, read out on a block by block basis. The two-dimensional block 61 shown in Fig.6 is a reference block in the t-th frame having a center coordinate point S(x,y,t) and its displacement from the (t-1)th frame is calculated. The buffer memory 1 stores the picture data of the (t-1)th frame as well as the picture data of the t-th frame. First, data of a two-dimensional block having a center coordinate point (x,y,t-1) is retrieved from the memory 1. Then, the error calculating circuit 2 calculates a sum of the square errors between the picture data amplitudes of the two blocks as expressed by: and transmits it to the error comparing circuit 3. Then, the address generating circuit 4 produces an address signal for a succeeding block shifted by one horizontal pixel from the reference block in the (t-1)th frame and having a center coordinate point S(x+1, y, t-1). Also, the error calculating circuit 2 calculates σ0(+1,0) and transmits it to the error comparing circuit 3. The error comparing circuit 3 then compares σ0(0,0) and σ 0(+1,0) and selects a smaller one. The sums of errors σ0(xh,yv) are calculated in the same manner as above in the range of -h<xh<h and -v<yv<v. These operations calculate differences between the reference block in the t-th frame and the blocks in the (t-1)th frame having the center coordinate points within a area (-h<xh<h,-v<yv<v) shown by the dotted line in Fig. 6. The error comparing circuit 3 selects a smallest σ0(mx,my) of σ0(xh,yv) and outputs an address (mx,my). Hence, the motion vector 63 of S(x,y,t) is expressed by mv(mx,my). Similarly, the motion vectors of the blocks in the t-th frame can be calculated by repeating the foregoing operations. (For example, see "Digital pictures" by Natravali, Haskell, PLENUM 1988, pp.334-340.) Such a conventional motion detecting apparatus can detect the movement of an object in each block of a picture but fails to detect a movement of the entire picture. The movement of the entire picture may be calculated by averaging the motion vectors of all blocks. However, the averaged motion vector is inaccurate when there are may detected vectors which mismatch with the real motion of the object or magnitude of the mismatching is large. Also, there is no theoretical solution for accurately processing the round caused due to averaging calculation. An alternative method of using motion vectors of blocks to predict the motion of a whole image is disclosed in EP-A-0181215. This patent discloses a motion vector calculating circuit which calculates motion vectors for several areas of an image, and then tests these motion vectors for the whole image, selecting the motion vector for the whole image which gives the least error. However, this method will only achieve the best motion vector if one of the blocks also has this vector as its motion vector. It is an object of the present invention to provide a method of detecting a movement of a whole picture such as panning movement with a high accuracy, and an apparatus for performing the method. For achievement of the object of the present invention, an improved method comprises a first step of dividing a picture supplied at a time t into K two-dimensional blocks each composed of m x n pixels, where K, m and n are integers greater than one; a second step of selecting as a kth reference block a kth one of said K two-dimensional blocks having a center coordinate pixel point Sk(x, y, t), where k is an integer satisfying 1≤k≤K; a third step of calculating an error σk(xh, yv) between the kth reference block ad each of J two-dimensional blocks within a picture supplied at a time t', where J is a integer greater than one, one of said J two-dimensional blocks having a center coordinate pixel points Sk(x, y, t') and each of the others of said J two-dimensional blocks having a center coordinate pixel points Sk(x+xh, y+yv, t') which is shifted by xh in the horizontal direction and yv in the vertical direction from the center coordinate pixel points Sk(x, y, t') in a rage of -h<xh<h and -v<yv<v, to obtain J said errors σk(xh, yv) for the respective J two-dimensional blocks; a fourth step of selecting at least one of said J errors σk(xh, yv) according to a predetermined criteria; a fifth step of detecting a movement of the kth reference block in accordance with the at least one of said J errors σk(xh,yv) selected in said fourth step; wherein said second through fifth steps are repeated for each of k equals 1, 2, . . ., K, a sixth step of adding said errors σk(xh, yv) throughout said K two-dimensional blocks with respect to each of J errors σk(xh,yv) to obtain J error sums σ(xh, yv) each expressed as: where -h<xh<h and -v<yv<v; a seventh step of selecting at least one of said J error sums σ(xh, yv) according to a predetermined criteria; and an eighth step of detecting a overall movement of said K two-dimensional blocks in accordance with the at least one of said J error sums σ(xh, yv) selected in the seventh step. Fig. 1 is a block diagram of a motion detecting apparatus showing a first embodiment of the present invention; Fig. 2 is a block diagram showing an error adding circuit and a second error comparing circuit of the apparatus; Fig. 3 is a block diagram of another motion detecting apparatus showing a second embodiment of the present invention; Fig. 4 is an explanatory view showing an interpolation method of the second embodiment; Fig. 5 is a block diagram of a conventional motion detecting apparatus; and Fig.6 is an explanatory view showing a conventional motion detecting method. Preferred embodiments of the present invention will be described in the form of a motion detecting method and its apparatus referring to the accompanying drawings. Fig.1 is a block diagram of a motion vector detecting apparatus showing a first embodiment of the present invention. As shown in Fig.1, there are provided a buffer memory 1, an error calculating circuit 2, an error comparing circuit 3, and an address generating circuit 4 which are arranged in the same manner as of the prior art. The error calculating circuit 2 may be a square error calculating circuit like that of the prior art or absolute value calculating circuit. Also, provided are an error adding circuit 5 and a second error adding circuit 6 which both are illustrated in more detail in Fig.2. The error adding circuit 5 comprises an adder 51, a memory 52, and an address generator circuit 53. The error comparing circuit 6 comprises a comparator 61 and a selector 62. The operation of the motion detecting apparatus having the foregoing arrangement will now be explained referring to Figs.1 and 2. As apparent from Fig.1, the motion vector of each block of a picture can be detected in the same manner as of the prior art. In particular, the apparatus of the present invention contains the error adding circuit 5 and the second error comparing circuit 6 for detection of the movement throughout the picture. It is assumed that the movements throughout a t-th frame and a t′-th frame are to be detected. Like the prior art, the center coordinate point of a reference two-dimensional block of the t-th frame is denoted by Sk(x,y,t). While the center coordinate point Sk(x,y,t′) of a like block of the t′-th frame is displaced by ±h pixels in the horizontal direction and ±v pixels in the vertical direction, its error σ k(xh,yv) (where -h<xh<h and -v<yv<v) from the reference block is calculated by the error calculating circuit 2. Then, the error σ k(xh,yv) is fed to the error adding circuit 5 where it is added to an error data sum σ(xh,yv) given by summing errors from 0 to k-1 and stored in the memory 52. As the result, the error data sum σ(xh,yv) from 0 to k is obtained. Then, the memory 52 fetches this error data sum σ(xh,yv) for updating. The address generator circuit 53 produces an address signal for supplying to the memory 52 in synchronism with the operation of the address generating circuit 4. When the total number of two-dimensional blocks of a frame is K, the error data sum σ(xh,yv) (where -h<xh<h and -v<yv<v) for the all blocks of the frame is calculated by repeating the foregoing operation K times for the 1st to the K-th block, as expressed as a function of xh and yh: The error data sum σ(xh,yv) is transferred to the second error comparing circuit 6 where the smallest error σ(mxt,myt) of σ(xh,yv) (where -h<xh<h and -v<yv<v) is selected with the comparator 61 and subsequently, the selector 62 delivers an address (mxt,myt). Hence, the motion vector of movement throughout the t-th frame is obtained as mv(mxt,myt). As set forth above, the error data sum σ(xh,yv) as a function of xh,yv is equivalent to the error generated when the motion detection is performed by the conventional method for the entire frame as one single block. Thus, the first embodiment allows the movement throughout the frame of a picture to be detected with accuracy. Fig.3 is a block diagram of another motion vector detecting apparatus showing a second embodiment of the present invention, which contains an interpolating circuit 7 unlike the first embodiment. The operation of the motion vector detecting apparatus of the second embodiment will now be explained referring to Figs.3 and 4. Fig.4 is an explanatory diagram showing a technique of calculating a motion vector through interpolation, in which the X axis denotes the displacement xh in the horizontal direction and the Y axis represents the error . The squares of errors detected fall on their respective points x. The graph of Fig.4 exhibits no vertical displacement for ease of explanation. Although the smallest error data is regarded as a motion vector in the first embodiment, it is more accurate to calculate the motion vector as follows: (1) Select b number of smallest squared error from σ(xh,yv). (2) Interpolate more quantities between them. (3) Select the minimum value as the motion vector. Fig.6 shows a parabola in which the minimum value represents a motion vector of the displacement throughout the frame. The interpolation may also be carried out by a common manner using e.g. Newton's difference division formula, Lagrange's interpolation formula, Gregory-Newton interpolation formula, Newton-Gauss interpolation formula, or Stirling's interpolation formula. In practice, the interpolation is conducted at the interpolating circuit 7 coupled to the output of the second error comparing circuit 6 as shown in Fig.3. The second error comparing circuit 6 performs the operation described in the first embodiment and also, delivers b number of the smallest errors to the interpolating circuit 7. Although the motion vector of displacement throughout a frame picture is calculated in the foregoing embodiments, it may be detected across any portion of the picture. For example, a motion vector for a quarter of the picture can be calculated without change of the arrangement of the present invention. Although error data between two adjacent blocks are summed up in the embodiments, they may be calculated from any groups of blocks. For example, a group of blocks of which motion vector magnitudes are approximately equivalent can be extracted and examined for detecting the movement throughout the blocks extracted as above. The present invention is not limited to the second embodiment in which no vertical movement is involved as assumed for ease of description. A motion vector of displacement throughout the picture may be calculated from the lowermost point of an interpolated curve which extends on a two-dimensional space with both vertical and horizontal movements accounted respectively or using a parameter of their combination movement.

Claims: A motion detecting method for detecting a movement within a moving picture comprising: a first step of dividing a picture supplied at a time t into K two-dimensional blocks each composed of m x n pixels, where K, m and n are integers greater than one; a second step of selecting as a kth reference block a kth one of said K two-dimensional blocks having a center coordinate pixel point Sk(x, y, t), where k is an integer satisfying 1 ≤k≤K; a third step of calculating an error σk(xh, yv) between the kth reference block and each of J two-dimensional blocks within a picture supplied at a time t', where J is an integer greater than one, one of said J two-dimensional blocks having a center coordinate pixel points Sk(x, y, t') and each of the others of said J two-dimensional blocks having a center coordinate pixel points Sk(x+xh, y+yv, t') which is shifted by xh in the horizontal direction and yv in the vertical direction from the center coordinate pixel points Sk(x, y, t') in a range of -h<xh<h and -v<yv<v, to obtain J said errors σk(xh, yv) for the respective J two-dimensional blocks; a fourth step of selecting at least one of said J errors σk(xh, yv) according to a predetermined criteria; a fifth step of detecting a movement of the kth reference block in accordance with the at least one of said J errors σk(xh,yv) selected in said fourth step; wherein said second through fifth steps are repeated for each of k equals 1, 2, . . ., K, characterized by further comprising: a sixth step of adding said errors σk(xh, yv) throughout said K two-dimensional blocks with respect to each of J errors σk(xh,yv) to obtain J error sums σ(xh, yv) each expressed as: where -h<xh<h and -v<yv<v; a seventh step of selecting at least one of said J error sums σ(xh, yv) according to a predetermined criteria; and an eighth step of detecting an overall movement of said K two-dimensional blocks in accordance with the at least one of said J error sums σ(xh, yv) selected in the seventh step. A method according to claim 1, wherein said seventh step determines a minimum one of said J error sums σ(xh, yv), and said eighth step detects the overall movement of said K two-dimensional blocks in accordance with the thus determined minimum one of said J error sums σ(xh, yv). A method according to claim 1, wherein said seventh step selects at least two of said J error sums σ(xh, yv) having a smallest value, and said eighth step interpolates an error sum from the thus selected at least two error sum σ(xh, yv), and detects the overall movement of said K two-dimensional blocks in accordance with the thus interpolated error sum. A data processing apparatus for detecting a movement within a moving picture comprising: a buffer memory (1) having an input receiving said motion picture data; an error calculation circuit (2) operatively coupled to said buffer memory for (a) retrieving from said buffer memory a kth reference block having a center coordinate pixel point Sk from among K two-dimensional blocks contained within a picture supplied at a time t, each of said K two-dimensional blocks being composed of m x n pixels, where K, m and n are positive integers greater than one, and k is an integer satisfying 1 ≤k≤K, (b) retrieving from said buffer memory J two-dimensional blocks each having a center coordinate point Sj contained within a corresponding picture supplied at a time t', each of said J two-dimensional blocks composed of m x n pixels, where J is a positive integer, and (c) calculating an error σk (j) between the kth reference block ad each of the J two-dimensional blocks to obtain J said errors σk (j)) for the respective J two-dimensional blocks; and a comparator circuit (3) operatively coupled to said error calculation circuit for comparing the calculated J errors σk(j) with each other and for generating data indicative of a detected movement of the kth reference block in accordance with the compared J errors σk(j); characterized by further comprising: an adding circuit (5) operatively coupled to said error calculation circuit (2) for summing the errors σk(j) throughout the K two-dimensional blocks with respect to each of said J errors σk(j) to obtain J error sums σ(j) each expressed as: where 1≤j≤J; and another comparator circuit (6) operatively coupled to said another adding circuit (6) for comparing said J error sums σ(j) with each other and for generating data indicative of a detected overall movement of the K two-dimensional blocks in accordance with the compared J error sums σ(j). An apparatus according to claim 4, wherein said another comparator circuit (6) includes means (62) for selecting a minimum one of said J error sums σ(j) as representative of the overall movement of said K two-dimensional blocks. An apparatus according to claim 4, wherein said another comparator circuit (6) includes means (62) for selecting at least two of said J error sums σ(j) having a smallest value, and wherein said apparatus further comprises an interpolating circuit (7) for interpolating an error sum from the selected at least two error sums σ(j), wherein the interpolated result is representative of the overall movement of said K two-dimensional blocks.

Inventors: TAKAHASHI TOSHIYA, TAKAHASHI, TOSHIYA
Applicants: MATSUSHITA ELECTRIC IND CO LTD, MATSUSHITA ELECTRIC INDUSTRIAL CO., LTD.
Application year: 1991
Main classification: G06T7
IPC codes: H04N7, H04N11, G06T7