Patent Publication Number: US-7221390-B1

Title: Computer-assisted motion compensation of a digitized image

Description:
CROSS REFERENCE TO RELATED APPLICATIONS 
     This application is based on and hereby claims priority to German Application No. 19921253.8 filed on May 7, 1999, the contents of both of which are hereby incorporated by reference. 
     BACKGROUND OF THE INVENTION 
     1. Field of the Invention 
     The invention relates to methods and to computer-aided motion compensation in a digitized image. 
     2. Description of the Related Art 
     Such a method and system are known from K Uomori, A. Morimura, H. Ishii, Electronic image stabilization system for video cameras and VCRs, SMPTE Journal, Vol. 101, No. 2, pages 66–75, 1992. In this known method, an image is recorded by a camera, is digitized and is stored. The recorded digitized image has pixels which have associated coding information. The expression coding information means brightness information (luminance information) or color information (chrominance information). 
     The stored image is enlarged, and the coding information is interpolated between the original pixels in the unenlarged image. “Global” motion estimation, that is to say estimation of the motion of the entire image, and “global” motion compensation are applied to the entire enlarged image. Global motion estimation results in the formation of an image motion vector, which is associated with the enlarged image. A motion-compensated image detail is read from the enlarged image. The expression motion-compensated image detail means an image detail which has been shifted by the determined image motion vector within the enlarged image. 
     The motion compensation is carried out in order to compensate in particular for a jittering motion of a person when recording an image using a handheld camera. 
     The procedure known from Uomori, et al. has the particular disadvantage that the enlargement of the image reduces the image resolution of the camera, and this is perceptible as blurring in the motion-compensated image. 
     It is also known for mechanical means to be used for motion compensation. In an arrangement such as this, mechanical sensors are provided in a camera and measure any motion, for example a jittering motion, to which the camera is subject. A moving lens system is also provided which is controlled as a function of the determined motion such that the motion is compensated for, thus ensuring a stable, motion-compensated image before the image is actually recorded. 
     However, this technology has the disadvantage that it is considerably heavier due to the additional sensors and due to the lens system. Furthermore, the production of such system is expensive. This technology is therefore unsuitable, especially for small appliances, for example a mobile video telephone. 
     A block-based image coding method is known in accordance with the H.263 Standard from ITU-T, International Telecommunication Union, Telecommunications Sector of ITU, Draft ITU-T Recommendation H.263, Videocoding for low bitrate communication, May 2, 1996. Further methods for global motion estimation are known from S. B. Balakirsky, R. Chellappa, Performance characterization of image stabilization algorithms, Proc. Int. Conf. On Image Processing, Vol. 2, 3, pages 413–416, 1996, R. Gupta, M. C. Theys, H. J. Siegel, Background compensation and an active-camera motion tracking algorithm, Proc. Of the 1997 Int. Conf on Parallel Processing, pages 431–440, 1997 and D. Wang, L. Wang, Global Motion Parameter Estimation Using a Fast and Robust Algorithm, IEEE Trans. On Circuits and Systems for Video Tech., Vol. 7, No. 5, pages 823–826, 1997. 
     SUMMARY OF THE INVENTION 
     The invention is thus based on the problem of specifying methods and systems for motion compensation in a digitized image, which do not require any complex additional lens system and which avoid any reduction in the image resolution of the recorded image. 
     In one method for computer-aided motion compensation in a digitized image, motion estimation is in each case carried out for the entire image, for that image and for a predetermined number of previous images, with an image motion vector being determined. An image motion vector is thus determined in each case. The images are stored in a memory having been compensated for motion, using the respective image motion vector. A predetermined area of the memory is read in order to form the motion-compensated image. 
     In a further method for computer-aided motion compensation in a digitized image, the image is stored in a memory. Motion estimation is carried out for the entire image, and this results in an image motion vector. Furthermore, an area of the memory is read which comprises an image detail of the entire image. The position of the area in the image is found on the basis of a predetermined basic position, compensated for motion as a function of the determined image motion vector. The area of the memory at the determined position is read in order to form the motion-compensated image. 
     A system for computer-aided motion compensation in a digitized image has a processor which is set up such that the following steps can be carried out:
         motion estimation is in each case carried out for the entire image, for that image and for a predetermined number of previous images, with an image motion vector being determined in each case,   the images are stored in a memory having been compensated for motion, using the respective image motion vector, and   a predetermined area of the memory is read in order to form the motion-compensated image.       

     A further arrangement for computer-aided motion compensation in a digitized image comprises a processor which is set up such that the following steps can be carried out:
         the image is stored in a memory,   motion estimation is carried out for the entire image, with an image motion vector being determined,   an area of the memory which includes an image detail of the entire image is read,   the position of the area in the image is found on the basis of a predetermined basic position, compensated for motion as a function of the determined image motion vector, and   the area of the memory at the determined position is read in order to form the motion-compensated image.       

     A program is stored in a computer-legible storage medium and, once this program has been loaded in a memory in the computer, it allows a computer to carry out the following steps for motion compensation of a digitized image:
         motion estimation is in each case carried out for the entire image, for that image and for a predetermined number of previous images, with an image motion vector being determined in each case,   the images are stored in a memory having been compensated for motion, using the respective image motion vector, and   a predetermined area of the memory is read in order to form the motion-compensated image.       

     A program is stored in a further computer-legible storage medium and, once this program has been loaded in a memory in the computer, it allows a computer to carry out the following steps for motion compensation in a digitized image:
         the image is stored in a memory,   motion estimation is carried out for the entire image, with an image motion vector being determined,   an area of the memory which includes an image detail of the entire image is read,   the position of the area in the image is found on the basis of a predetermined basic position, compensated for motion as a function of the determined image motion vector, and   the area of the memory at the determined position is read in order to form the motion-compensated image.       

     A computer program product contains a computer-legible storage medium, in which a program is stored which allows a computer to carry out the following steps for motion compensation in a digitized image once said program has been loaded in a memory in the computer:
         motion estimation is in each case carried out for the entire image, for that image and for a predetermined number of previous images, with an image motion vector being determined in each case,   the images are stored in a memory having been compensated for motion, using the respective image motion vector, and   a predetermined area of the memory is read in order to form the motion-compensated image.       

     A further computer program product comprises a computer-legible storage medium, in which a program is stored which allows a computer to carry out the following steps for motion compensation in a digitized image once said program has been loaded in a memory in the computer:
         the image is stored in a memory,   motion estimation is carried out for the entire image, with an image motion vector being determined,   an area of the memory which includes an image detail of the entire image is read,   the position of the area in the image is found on the basis of a predetermined basic position, compensated for motion as a function of the determined image motion vector, and   the area of the memory at the determined position is read in order to form the motion-compensated image.       

     The invention specifies a cost-effective solution for motion compensation in a digitized image, which avoids any reduction in the image resolution of the camera. The motion-compensated image thus has better clarity than that produced by the method disclosed in Uomori, et al. 
     The developments described in the following text relate not only to the methods, the systems and the computer program products, but also to computer-readable storage media. The invention can be implemented both in software and in hardware, for example using a special electrical circuit. A development provides that each previous image is overwritten with motion compensation by the subsequent image in the memory, in overlapping areas of the images. 
     The invention is preferably used for coding and/or decoding a digitized image. 
     The invention is particularly suitable for use in a mobile telecommunications terminal, for example a mobile video telephone. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       These and other objects and advantages of the present invention will become more apparent and more readily appreciated from the following description of the preferred embodiments, taken in conjunction with the accompanying drawings of an exemplary embodiment of the invention, described in more detail in the following text, in which: 
         FIG. 1  is a block diagram illustrating the specific principle of the exemplary embodiment; 
         FIG. 2  is a block diagram of a system having a camera and a coding unit for coding the image sequence recorded by the camera, and for decoding the coded image sequence; 
         FIG. 3  is a detailed block diagram of a system for image coding and for global motion compensation; 
         FIGS. 4   a  to  4   c  each show an image in which a motion vector field is established for the image, showing a comparison to a previous image with a predetermined area ( FIG. 4   a ) from which the motion vectors are in each case determined in order to form parameters for a motion model, an image with all the motion vectors ( FIG. 4   b ), and an image with motion vectors after one iteration of the method with the predetermined area illustrated in  FIG. 4   a  ( FIG. 4   c ); 
         FIG. 5  is a flow chart illustrating the method steps in the method for determining the image motion vector for an image; 
         FIG. 6  is a sketch illustrating the storage of previous images according to a first exemplary embodiment; and 
         FIG. 7  is a sketch illustrating the selection of an area for selection of an image detail for motion compensation in an image according to a second exemplary embodiment. 
     
    
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 
     Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like elements throughout. 
     First Exemplary Embodiment 
       FIG. 2  shows a system which comprises two computers  202 ,  208  and a camera  201 , illustrating image coding, transmission of the image data and image decoding. 
     A camera  201  is connected to a first computer  202  via a line  219 . The camera  201  transmits recorded images  204  to the first computer  202 . The first computer  202  has a first processor  203  which is connected via a bus  218  to an image memory  205 . The first processor  203  in the first computer  202  carries out a method for image coding. In this way, coded image data  206  is transmitted from the first computer  202  via a communication link  207 , preferably a line or a radio path, to a second computer  208 . The second computer  208  contains a second processor  209 , which is connected via a bus  210  to an image memory  211 . The second processor  209  carries out a method for image decoding. 
     Both the first computer  202  and the second computer  208  have a respective screen  212  and  213 , on which the image data  204  is visualized. Input units, preferably a keyboard  214  or  215 , respectively, and a computer mouse  216  or  217 , respectively, are provided for both the first computer  202  and the second computer  208 . 
     The image data  204 , which is transmitted from the camera  201  via the line  219  to the first computer  202 , is data in the time domain, while the data  206  which is transmitted from the first computer  202  to the second computer  208  via the communication link  207  is image data in the frequency domain. 
     The decoded image data is displayed on a screen  220 . 
       FIG. 3  is a block diagram of a system for carrying out the method according to a first exemplary embodiment, within which a block-based image coding method is carried out. 
     A video data stream to be coded with successive digitized images is supplied to an image coding unit  301 . The digitized images are subdivided into macroblocks  302 , with each macroblock containing 16×16 pixels. The macroblock  302  comprises 4 image blocks  303 ,  304 ,  305  and  306 , with each image block containing 8×8 pixels, which have associated luminance values (brightness values). Furthermore, each macroblock  302  comprises two chrominance blocks  307  and  308  with chrominance values (color information, color saturation) associated with the pixels. 
     The block of an image contains a luminance value (=brightness), a first chrominance value (=hue) and a second chrominance value (=color saturation). In this case, the luminance value, the first chrominance value and the second chrominance value are referred to as color values. 
     The image blocks are supplied to a transformation coding unit  309 . In the case of difference image coding, values to be coded of image blocks from previous images are subtracted from the image blocks to be coded at that time, and only the subtraction information  310  is supplied to the transformation coding unit (discrete cosine transformation DCT)  309 . To this end, the present macroblock  302  is signaled to a motion estimation unit  329  via a link  334 . In the transformation coding unit  309 , spectral coefficients  311  are formed for the image blocks or difference image blocks to be coded, and are supplied to a quantization unit  312 . 
     Quantized spectral coefficients  313  are supplied both to a scanning unit  314  and to an inverse quantization unit  315 , in a reverse path. After a scanning method has been carried out, for example a zigzag scanning method, entropy coding is applied to the scanned spectral coefficients  332  in an entropy coding unit  316  provided for this purpose. The entropy-coded spectral coefficients are transmitted to a decoder as coded image data  317  via a channel, preferably a line or a radio path. 
     Inverse quantization of the quantized spectral coefficients  313  is carried out in the inverse quantization unit  315 . Spectral coefficients  318  obtained in this way are supplied to an inverse transformation coding unit  319  (inverse discrete cosine transformation IDCT). Reconstructed coding values (also difference coding values)  320  are supplied in a difference image mode to an adder  321 . The adder  321  furthermore receives coding values for an image block, which are obtained from a previous image on the basis of motion compensation that has already been carried out. The adder  321  is used to form reconstructed image blocks  322 , which are stored in an image memory  323 . 
     Chrominance values  324  of the reconstructed image blocks  322  are supplied from the image memory  323  to a motion compensation unit  325 . Interpolation for brightness values  326  is carried out in an interpolation unit  327  provided for this purpose. The interpolation process is preferably used to double the number of brightness values contained in the respective image block. All the brightness values  328  are supplied both to the motion compensation unit  325  and to the motion estimation unit  329 . The motion estimation unit  329  also receives the image blocks of the respective macroblock (16×16 pixels) to be coded, via the link  334 . The motion estimation process is carried out in the motion estimation unit  329  taking account of the interpolated brightness values (“motion estimation on a half-pixel basis”). 
     The motion estimation process results in a motion vector  330  which expresses any shift in the position of the selected macroblock from the previous image to the macroblock  302  to be coded. 
     Both brightness information and chrominance information related to the macroblock determined by the motion estimation unit  329  are shifted by the motion vector  330 , and are subtracted from the coding values of the macroblock  302  (see data path  231 ). 
     The motion estimation process is carried out by determining an error E, for each image block for which motion estimation is carried out, with respect to a region of the same shape and size as the image block in a previous image, for example using the following rule: 
                     E   =         ∑     i   =   1     n     ⁢       ∑     j   =   1     m     ⁢            x     i   ,   j       -     xd     i   ,   j                  →     min   ⁢           ⁢     ∀     d   ∈   S             ,           (   1   )               
where
         i, j are each indices,   n, m represents a number (n) of pixels along a first direction x and, respectively, a number (m) of pixels along a second direction y, which are contained in the image block,   xi, j is coding information which is associated with a pixel at the relative position denoted by the indices i, j in the image block,   xdi, j is coding information which is associated with the respective pixel denoted by i, j in the region of the previous image, shifted by a value d which can be predetermined,   S is a search area of predetermined shape and size in the previous image.       
     The error E is calculated for each image block for various shifts within the search area S. The image block in the previous image whose error E is the minimum is selected as being the most similar to that image block for which the motion estimation is carried out. 
     The motion estimation process thus results in the motion vector  330  which has two motion vector components, a first motion vector component BV x  and a second motion vector component Bv y  along the first direction x and the second direction y: 
     
       
         
           
             
               
                 
                   BV 
                   = 
                   
                     
                       ( 
                       
                         
                           
                             
                               BV 
                               x 
                             
                           
                         
                         
                           
                             
                               BV 
                               y 
                             
                           
                         
                       
                       ) 
                     
                     . 
                   
                 
               
               
                 
                   ( 
                   2 
                   ) 
                 
               
             
           
         
       
     
     The motion vector  330  is associated with the image block. 
     The image coding unit from  FIG. 3  thus produces a motion vector  330  for all the image blocks or macro image blocks. 
     The motion vectors  330  are supplied to a unit for selection and/or weighting of the motion vectors  330 . In the unit for selection of the motion vectors  335 , the motion vectors  330  which are selected and/or are provided with high weightings are those which are associated with image blocks which are located in a predetermined area  401  (see  FIG. 4   a ). Furthermore, the unit  335  is used to select and/or to provide high weightings for those motion vectors which have been estimated as being reliable ( 342 ). 
     The selected motion vectors  336  are supplied to a unit for determining the parameters for the motion model  337 . The motion model as described in the following text and as shown in  FIG. 1  is determined from the selected motion vectors in the unit for determining the parameters for the motion model  337 . 
     The determined motion model  338  is supplied to a unit for compensation  339  for the motion between the camera and the recorded image. The motion is compensated for in the unit for compensation  339  using a motion model which is described in the following text, so that an image  340 , whose motion is compensated for, is stored once again, after the processing in the unit for compensation  339 , in the image memory  323  in which the image which had not previously been processed but whose motion is to be compensated for is stored. 
       FIG. 1  shows the principle on which the global motion determination is based, in the form of a block diagram. 
     The parameters for the motion model  338  described in the following text are calculated (step  103 ) on the basis of a motion vector field  101 , the predetermined area or a weighting mask  102 , and a weighting mask of reliability factors  106 . 
     The expression motion vector field  101  is intended to mean a set of all the determined motion vectors  330  relating to an image. In  FIG. 4   b , the motion vector field  101  is represented ( 402 ) by dashed lines, which each describe a motion vector  330  for one image block. The motion vector field  402  is sketched on the digitized image  400 . The image  400  comprises a moving object  403  in the form of a person, and an image background  404 . 
       FIG. 4   a  shows a predetermined area  401 . The predetermined area  401  indicates a region in which the image blocks must lie in order that the motion vectors which are associated with these image blocks are selected. 
     The predetermined area  401  is obtained from an edge area  405  which is formed by image blocks which are located at a predetermined first distance of  406  from one edge  407  of the digitized image  400 . This means that image blocks directly adjacent to the edge  407  of the image  400  are ignored when determining the parameters for the motion model  338 . Furthermore, the predetermined area  401  is formed by image blocks which are located at a predetermined second distance  408  from the center  409  of the digitized image  400 . 
     The predetermined area or the weighting mask is changed using an iterative method with the following steps, in order to form a new area for the subsequent iteration (step  104 ). 
     A vector difference value VU is in each case determined for each image block in the predetermined area  401  and is used to describe the difference between the determined motion model  338  and the motion vector  330  with which the respective image block is associated. The vector difference value VU is formed, for example, using the following rule:
 
 VU=|BV   X   −MBV   X   |+|BV   Y   −MBV   Y |,  (3)
 
where MBV x  and MBV y  each denote the components of an image motion vector MBV calculated on the basis of the motion model.
 
     The process for determining the model-based image motion vector is explained in more detail in the following text. 
     When using a binary mask, an image block is contained in the new area of the further iteration when the respective vector difference value VU is less than a threshold value ε which can be predetermined. However, if the vector difference value VU is greater than the threshold value ε, then the image block with which the respective motion vector is associated is no longer considered in the new predetermined area. 
     When using a weighting mask, the weighting factors for the blocks are specified in the opposite ratio to this vector difference value VU. 
     This procedure results in those motion vectors which differ considerably from the image motion vectors MBV calculated from the determined motion model not being considered, or being considered only to a minor extent, in the calculation of the parameters for the motion model in a further iteration. 
     Once the new area or the new weighting mask has been formed, the motion vectors which are associated with image blocks which are contained in the new area are used, possibly in addition to the weighting mask, to determine a new parameter set for the motion model. 
     The method described above is carried out in a number (which can be predetermined) of iterations or until a termination criterion is satisfied, for example the number of blocks eliminated in an iteration step falls below a specific number. 
     In this case, the new area are in each case used as the predetermined area or the new weighting mask in addition to the old motion vectors, as input variables for the next iteration. 
     The global motion is determined by determining parameters in a model for the global camera motion. 
     In order to explain the motion model, the following text contains a detailed derivation of the motion model: 
     It is assumed that a natural, three-dimensional scene is imaged, using the camera, onto a two-dimensional projection plane. An image of a point
 
   p     0 =( x   0   , y   0   , z   0 ) T   (4)
 
is formed using the following rule:
 
                         (         X           Y         )     =       F     z   0       ⁢       (           x   0               y   0           )     ⋀     z   0           &gt;&gt;   F     ,           (   5   )               
where F is a focal length and X, Y are coordinates of the imaged point p 0  on the image plane.
 
     If the camera is now moved, then the imaging rule remains synchronized to the coordinate system, which moves with the camera, but the coordinates of the object points must be transformed to this coordinate system. Since all camera motions can be regarded as an accumulation of rotation and translation, the transformation from the fixed coordinate system (x, y, z) to the jointly moving coordinate system ({tilde over (x)} 0 , {tilde over (y)} 0 , {tilde over (z)} 0 ) can be formulated on the basis of the following rule: 
     
       
         
           
             
               
                 
                   
                     ( 
                     
                       
                         
                           
                             
                               x 
                               ~ 
                             
                             0 
                           
                         
                       
                       
                         
                           
                             
                               y 
                               ~ 
                             
                             0 
                           
                         
                       
                       
                         
                           
                             
                               z 
                               ~ 
                             
                             0 
                           
                         
                       
                     
                     ) 
                   
                   = 
                   
                     
                       
                         ( 
                         
                           
                             
                               
                                 r 
                                 11 
                               
                             
                             
                               
                                 r 
                                 12 
                               
                             
                             
                               
                                 r 
                                 13 
                               
                             
                           
                           
                             
                               
                                 r 
                                 21 
                               
                             
                             
                               
                                 r 
                                 22 
                               
                             
                             
                               
                                 r 
                                 23 
                               
                             
                           
                           
                             
                               
                                 r 
                                 31 
                               
                             
                             
                               
                                 r 
                                 32 
                               
                             
                             
                               
                                 r 
                                 33 
                               
                             
                           
                         
                         ) 
                       
                       · 
                       
                         ( 
                         
                           
                             
                               
                                 x 
                                 0 
                               
                             
                           
                           
                             
                               
                                 y 
                                 0 
                               
                             
                           
                           
                             
                               
                                 z 
                                 0 
                               
                             
                           
                         
                         ) 
                       
                     
                     + 
                     
                       
                         ( 
                         
                           
                             
                               
                                 t 
                                 1 
                               
                             
                           
                           
                             
                               
                                 t 
                                 2 
                               
                             
                           
                           
                             
                               
                                 t 
                                 3 
                               
                             
                           
                         
                         ) 
                       
                       . 
                     
                   
                 
               
               
                 
                   ( 
                   6 
                   ) 
                 
               
             
           
         
       
     
     Based on rule (6), any image change caused by camera motion is modeled in accordance with the following rule: 
                       (           Δ   ⁢           ⁢   X               Δ   ⁢           ⁢   Y           )     =         (               C   F     ⁢           ⁢     cos   ⁡     (     φ   z     )         -   1             -     C   F       ⁢           ⁢     sin   ⁡     (     φ   z     )                     C   F     ⁢           ⁢     sin   ⁡     (     φ   z     )                   C   F     ⁢           ⁢     cos   ⁡     (     φ   z     )         -   1           )     ·     (         X           Y         )       +     (           t   X               t   Y           )         ,           (   7   )               
where ΔX, ΔY is a coordinate change of the pixels, resulting from the described camera motion in a time interval Δt, and φz is the angle through which the camera has been rotated about a z-axis in this time interval Δt. A predetermined factor CF denotes any focal length change or translation along the z-axis.
 
     The equation system described in rule (7) is non-linear, and it is therefore impossible to determine the parameters in the equation system directly. 
     For this reason, a simplified motion model is used for faster calculation, in which the camera motion in the imaging plane is used by a motion model with 6 parameters, which are formed using the following rule: 
     
       
         
           
             
               
                 
                   
                     ( 
                     
                       
                         
                           
                             
                               X 
                               ~ 
                             
                             0 
                           
                         
                       
                       
                         
                           
                             
                               Y 
                               ~ 
                             
                             0 
                           
                         
                       
                     
                     ) 
                   
                   = 
                   
                     
                       
                         ( 
                         
                           
                             
                               
                                 r 
                                 11 
                                 ′ 
                               
                             
                             
                               
                                 r 
                                 12 
                                 ′ 
                               
                             
                           
                           
                             
                               
                                 r 
                                 21 
                                 ′ 
                               
                             
                             
                               
                                 r 
                                 22 
                                 ′ 
                               
                             
                           
                         
                         ) 
                       
                       · 
                       
                         ( 
                         
                           
                             
                               
                                 X 
                                 0 
                               
                             
                           
                           
                             
                               
                                 Y 
                                 0 
                               
                             
                           
                         
                         ) 
                       
                     
                     + 
                     
                       
                         ( 
                         
                           
                             
                               
                                 t 
                                 X 
                                 ′ 
                               
                             
                           
                           
                             
                               
                                 t 
                                 Y 
                                 ′ 
                               
                             
                           
                         
                         ) 
                       
                       . 
                     
                   
                 
               
               
                 
                   ( 
                   8 
                   ) 
                 
               
             
           
         
       
     
     The equation system obtained from this with the data for the motion vector field is now solved by linear regression, with the complexity corresponding to inversion of a symmetrical 3×3 matrix. 
     After determining the parameters r′ 11 , r′ 12 , r′ 21 , r′ 22 , t′ X , and t′ Y , the parameters in rule (7) are approximated using the following rules: 
     
       
         
           
             
               
                 
                   
                     
                       T 
                       _ 
                     
                     = 
                     
                       
                         T 
                         ′ 
                       
                       _ 
                     
                   
                   , 
                 
               
               
                 
                   ( 
                   9 
                   ) 
                 
               
             
             
               
                 
                   
                     
                       C 
                       F 
                     
                     = 
                     
                       
                          
                         
                           det 
                           ⁡ 
                           
                             ( 
                             
                               
                                 
                                   
                                     r 
                                     11 
                                     ′ 
                                   
                                 
                                 
                                   
                                     r 
                                     12 
                                     ′ 
                                   
                                 
                               
                               
                                 
                                   
                                     r 
                                     21 
                                     ′ 
                                   
                                 
                                 
                                   
                                     r 
                                     22 
                                     ′ 
                                   
                                 
                               
                             
                             ) 
                           
                         
                          
                       
                     
                   
                   , 
                 
               
               
                 
                   ( 
                   10 
                   ) 
                 
               
             
             
               
                 
                   
                     ρ 
                     z 
                   
                   = 
                   
                     arcsin 
                     ⁢ 
                     
                       1 
                       2 
                     
                     ⁢ 
                     
                       
                         ( 
                         
                           
                             r 
                             21 
                             ′ 
                           
                           - 
                           
                             r 
                             12 
                             ′ 
                           
                         
                         ) 
                       
                       . 
                     
                   
                 
               
               
                 
                   ( 
                   11 
                   ) 
                 
               
             
           
         
       
     
       FIG. 4   c  shows the motion vectors which are associated with those image blocks which are located in the predetermined area  401 . In this case, the predetermined area  401  is changed from the predetermined area  401  from  FIG. 4   a  by one iteration (step  104 ). 
     These parameters are used to determine the image motion vector for the image which describes the motion to which the image is subject relative to the camera recording the image. 
     Each image in the chronological sequence of images is stored in the image memory  323 , shifted by the respective image motion vector which has been determined for that image. The image memory  323  is larger than the camera format used to record the sequence of images. Thus, specifically that section of the camera, represented by the reconstructed image, within the image memory  323  is shifted by the respective image motion vector, and is then stored, compensated for that motion. When a current, movement-compensating image is being stored, areas in the image memory to which previous images have already been written are overwritten by the new image. 
     This procedure is illustrated symbolically in  FIG. 6  by four images  601 ,  602 ,  603 ,  604 . Each of these four images  601 ,  602 ,  603 ,  604  is shifted from a basic position  600  by a vector corresponding to the image motion vector  611 ,  612 ,  613 ,  614  associated with the respective image  601 ,  602 ,  603 ,  604 . 
     The image information within the image memory  323  and which is contained in an area  610  of the image memory  323  which contains the basic position  600  at the upper right-hand edge of the area  610  is read as the motion-compensated image for the image that is currently to be coded or displayed. 
     The read image is denoted by the reference symbol  620  in  FIG. 6 . 
     If the background of the recorded image sequence is a fixed object, for example a landscape or the rear wall of a video conference room, then a relatively large cohesive area of the actual image background builds up over time after coding and motion-compensated storage of a number of images in the image memory  323 , caused in particular by different camera viewing points during recording of the image sequence. 
     The same area  620  is always read from the image memory  323 , that is to say for each image. 
     The individual method steps of the method will be described once again with reference to  FIG. 5 : 
     After the start of the method (step  501 ), an image block or macro image block is selected (step  502 ). A motion vector is determined for the selected image block or macro image block (step  503 ), and a check is carried out in a further step (step  504 ) to determine whether all the image blocks or macro image blocks for the image have been processed. 
     If this is not the case, then a further image block or macro image block which has not yet been processed is selected in a further step (step  505 ). 
     If, however, all the image blocks or macro image blocks have been processed, then those motion vectors are selected which are associated with an image block or a macro image block and which lie in the predetermined area (step  506 ). 
     The parameters for the motion model are determined from the selected motion vectors (step  507 ). If a further iteration needs to be carried out, that is to say the stated number of iterations has not yet been reached or the termination criterion is not yet satisfied, then a new area is defined in a further step (step  509 ), and the weighting mask for the next iteration is calculated (step  510 ) as a function of the vector difference values VU. 
     This is followed by compensation for the motion of the image using the determined motion model (step  508 ). 
     A number of alternatives to the exemplary embodiment described above are explained in the following text: 
     The shape of the area is in principle undefined and is preferably dependent on prior knowledge of a scene. Those image areas which are known to differ considerably from the global motion should not be used to determine the motion model. 
     The area should contain only motion vectors of image areas which have been found to be reliable on the basis of the reliability values  342  in the motion estimation method. 
     In general, the motion estimation can be carried out using any desired method, and is in no way restricted to the principle of block matching. Thus, for example, motion estimation can also be carried out using dynamic programming. 
     The nature of the motion estimation and hence the way in which a motion vector is determined for an image block are thus not relevant to the invention. 
     Alternatively, in order to determine the parameters in the equation system (7) approximately, it is possible to linearize the sine terms and cosine terms in the rule (7). 
     This therefore results in the following rule for small angles ρZ 
                     (           Δ   ⁢           ⁢   X               Δ   ⁢           ⁢   Y           )     =           (             C   F     -   1             -     C   F       ⁢     ω   z                   C   F     ⁢     ω   z               C   F     -   1           )     ·     (         X           Y         )       +     (           t   X               t   Y           )       =         (           R   1           -     R   2                 R   2           R   1           )     .     (         X           Y         )       +       (           t   X               t   Y           )     .                 (   12   )               
Since the equations for ΔX and ΔY cannot be optimized independently mutually, the least sum of the squares is formed, that is to say using the following rule:
 
     
       
         
           
             
               
                 
                   
                     
                       ∑ 
                       
                         V 
                         _ 
                       
                     
                     ⁢ 
                     
                       [ 
                       
                         
                           
                             ( 
                             
                               
                                 Δ 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   X 
                                   η 
                                 
                               
                               - 
                               
                                 
                                   R 
                                   1 
                                 
                                 ⁢ 
                                 
                                   X 
                                   η 
                                 
                               
                               + 
                               
                                 
                                   R 
                                   2 
                                 
                                 ⁢ 
                                 
                                   Y 
                                   η 
                                 
                               
                               - 
                               
                                 t 
                                 x 
                               
                             
                             ) 
                           
                           2 
                         
                         + 
                         
                           
                             ( 
                             
                               
                                 Δ 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   Y 
                                   η 
                                 
                               
                               - 
                               
                                 
                                   R 
                                   2 
                                 
                                 ⁢ 
                                 
                                   X 
                                   η 
                                 
                               
                               + 
                               
                                 
                                   R 
                                   1 
                                 
                                 ⁢ 
                                 
                                   Y 
                                   η 
                                 
                               
                               - 
                               
                                 t 
                                 y 
                               
                             
                             ) 
                           
                           2 
                         
                       
                       ] 
                     
                   
                   → 
                   min 
                 
               
               
                 
                   ( 
                   13 
                   ) 
                 
               
             
           
         
       
     
     In this case, ΔX η , ΔY η  denote the X and Y components of the motion vector of the image block η at the position X η , Y η  of the predetermined area V of the image. 
     According to equation (12), R1, R2, tx and ty are the motion model parameters to be determined. 
     Once the optimization process has been carried out, the associated model-based motion vector MBV (ΔX ΔY) is established on the basis of the determined equation system (12) by substitution of the X and Y components in the respective macroblock. 
     Instead of the areas mentioned above, weighting masks Ax, Ay, which separately represent the reliability of the motion vectors, the a priori knowledge and the conclusions from the VUs in iterative procedures for the X and Y components of the motion vectors are used in the calculation of the parameters for the motion model in accordance with the following optimization rule: 
     
       
         
           
             
               
                 
                   
                     
                       ∑ 
                       
                         V 
                         _ 
                       
                     
                     ⁢ 
                     
                       [ 
                       
                         
                           
                             
                               
                                 
                                   ( 
                                   
                                     
                                       α 
                                       
                                         X 
                                         η 
                                       
                                     
                                     · 
                                     
                                       ( 
                                       
                                         
                                           Δ 
                                           ⁢ 
                                           
                                               
                                           
                                           ⁢ 
                                           
                                             X 
                                             η 
                                           
                                         
                                         - 
                                         
                                           
                                             R 
                                             1 
                                           
                                           ⁢ 
                                           
                                             X 
                                             η 
                                           
                                         
                                         + 
                                         
                                           
                                             R 
                                             2 
                                           
                                           ⁢ 
                                           
                                             Y 
                                             η 
                                           
                                         
                                         - 
                                         
                                           t 
                                           x 
                                         
                                       
                                       ) 
                                     
                                   
                                   ) 
                                 
                                 2 
                               
                               + 
                             
                           
                         
                         
                           
                             
                               
                                 ( 
                                 
                                   
                                     α 
                                     
                                       Y 
                                       η 
                                     
                                   
                                   · 
                                   
                                     ( 
                                     
                                       
                                         Δ 
                                         ⁢ 
                                         
                                             
                                         
                                         ⁢ 
                                         
                                           Y 
                                           η 
                                         
                                       
                                       - 
                                       
                                         
                                           R 
                                           2 
                                         
                                         ⁢ 
                                         
                                           X 
                                           η 
                                         
                                       
                                       + 
                                       
                                         
                                           R 
                                           1 
                                         
                                         ⁢ 
                                         
                                           Y 
                                           η 
                                         
                                       
                                       - 
                                       
                                         t 
                                         x 
                                       
                                     
                                     ) 
                                   
                                 
                                 ) 
                               
                               2 
                             
                           
                         
                       
                       ] 
                     
                   
                   → 
                   min 
                 
               
               
                 
                   ( 
                   14 
                   ) 
                 
               
             
             
               
                 
                   
                     
                       α 
                       
                         X 
                         η 
                       
                     
                     ∈ 
                     
                       A 
                       X 
                     
                   
                   , 
                   
                     
                       α 
                       
                         Y 
                         η 
                       
                     
                     ∈ 
                     
                       
                         A 
                         y 
                       
                       . 
                     
                   
                 
               
               
                 
                     
                 
               
             
           
         
       
     
     A weighting mask Ax, Ay, for the reliability of the motion vectors ( 105 ) can be calculated, for example, by calculating the values αx, αy for an image block during block matching, as follows: 
                       α   x     =       1     SAD   match       ·       ∑   N     ⁢              SAD   η     -     SAD   match                     x   η     -     x   match                    ,           (   15   )                   α   y     =       1     SAD   match       ·       ∑   N     ⁢              SAD   η     -     SAD   match                     y   η     -     y   match                    ,           (   16   )               
where SAD η , is the sum of the pixel differences in a block for the η-th shift (x η , y η ) in the block matching process, SADmatch represents the same for the best, finally chosen region (x match , y match ), and N is the total number of search positions which have been investigated. If this value is calculated taking account, for example, of only the 16 best regions, then the block matching process can be carried out as a “spiral search”, with the SAD of the poorest of the 16 chosen regions as the termination criterion.
 
     A further option for calculating a weighting mask A x =A y =A for the reliability of the motion vectors is provided by: 
                     α   =     Σ   ⁢       SAD   -     SAD   match       N         ,           (   17   )               
where α=α x =α y  is the weighting factor of an image block or of its motion vector.
 
     The invention can be used, for example, to compensate for motion of a moving camera or else to provide motion compensation for a camera which is integrated in a mobile communications appliance (portable video telephone). 
     Second Exemplary Embodiment 
       FIG. 7  shows the image memory  323  with a symbolic representation of the storage and reading of the image, according to a second exemplary embodiment. 
     For the purposes of this exemplary embodiment, it is assumed that the camera is recording an image using a larger image format than is required for the image output. 
     The method corresponds essentially to the first exemplary embodiment, with the difference that an image detail  700  is now read from the image memory  323  after the motion estimation process has been carried out, and is shifted by the determined image motion vector  701  starting from a basic position  702 . 
     It should be noted that the motion compensation for the image is carried out before the transformation coding of the individual image blocks, in order that it is possible to use a coder with which the image format of the image to be output is processed. This clearly means that the motion compensation is in this case carried out “upstream” of the coder. 
     Alternatives to the exemplary embodiments described above are described in the following text: 
     The method for motion compensation is not dependent on the chosen method for motion estimation, that is to say any desired methods can be used to determine the image motion vector. 
     The invention has been described in detail with particular reference to preferred embodiments thereof and examples, but it will be understood that variations and modifications can be effected within the spirit and scope of the invention.