Patent Publication Number: US-8526680-B2

Title: Image processing method, image processing program, image processing device, and imaging device

Description:
CROSS-REFERENCE TO RELATED APPLICATION 
     This application is a U.S. National Stage application claiming the benefit of prior filed International Application Number PCT/JP2006/321904, filed Nov. 1, 2006, in which the International Application claims a priority date of Nov. 30, 2005 based on prior filed Japanese Application Number 2005-345715, the entire contents of which are incorporated herein by reference. 
     TECHNICAL FIELD 
     The present invention relates to an image-processing method and an image-processing program which are applied to a digital camera, image-processing software, or the like. Further, the present invention relates to an image-processing apparatus which is applied to a digital camera and to an imaging apparatus such as a digital camera. 
     BACKGROUND ART 
     Image stabilization by digital processing detects a motion vector between plural images which are photographed continuously, and position-adjusts and composes the images according to the motion vector. A method of detecting the motion vector is disclosed in the Patent Document 1 or the like, which detects, for example, a motion vector (local motion vector) from each block of plural images and obtains one motion vector of the entire image based on the local motion vector of each block.
     Patent Document 1: Japanese Unexamined Patent Application Publication No. H07-38800   

     DISCLOSURE 
     Problems to be Solved 
     However, a local motion vector with low reliability is mixed in the local motion vectors. Thus, the number of blocks needs to be increased so as to improve accuracy of detecting the motion vector by this method, and the operation amount increases by the quantity of increased blocks. Moreover, depending on a manner of movement between images, there may be a case that the motion vector does not represent the motion precisely. 
     Accordingly, an object of the present invention is to provide an image-processing method, an image-processing program, an image-processing apparatus, and an imaging apparatus which are capable of detecting motion between plural images with high precision without increasing the number of blocks. 
     Means for Solving the Problem 
     A first image-processing method according to the present invention is an image-processing method which detects motion between plural images, and is characterized by including a searching operation of searching for a local motion vector from each of blocks in the plural images, an electing operation of electing a representative motion vector from the local motion vectors of the respective blocks, and an approximating operation of approximating a motion vector distribution on the images based on the representative motion vector. 
     In addition, it is preferable that at least three of the representative motion vectors are elected, and the motion vector distribution is approximated by a linear function of a position on the images. 
     Further, it is preferable that in searches of the respective blocks, two types of projected data made by adding in two directions feature quantities of reference areas in the blocks are used. 
     Further, it is preferable that in searches of the respective blocks, two types of cumulative added images are created in advance, which are made by cumulatively adding in the two directions the feature quantities of the blocks, and the two types of projected data are created from a part of the two types of cumulative added images. 
     Further, a calculating processing of the motion vector distribution by the searching operation, the electing operation, and the approximating operation may be performed on plural coarse images made by performing resolution-reducing conversion of the plural images. 
     Furthermore, using the motion vector distribution calculated from the plural coarse images, motion between plural detailed images before being subjected to the resolution-reducing conversion may be detected. 
     Further, the plural detailed images may be position-adjusted and configured based on the detected motion. 
     Further, a first image-processing program according to the present invention is characterized by being configured to cause a computer to execute any one of the first image-processing methods according to the present invention. 
     Further, a first image-processing apparatus according to the present invention is characterized by including a unit which executes any one of the first image-processing methods according to the present invention. 
     Further, a first imaging apparatus according to the present invention is characterized by including an imaging unit capable of continuously imaging a subject to obtain plural images, and any one of the first image-processing apparatuses according to the present invention. 
     (Second Invention) 
     A second image-processing method according to the present invention is an image-processing method which searches for a motion vector between plural images, and is characterized by including a calculating operation of calculating a correlation between reference areas in the plural images while shifting positions of the reference areas between the images, in which a correlation between two types of projected data made by adding in two directions feature quantities of the reference areas is calculated as the correlation between the reference areas. 
     In addition, it is preferable that in the calculating operation, two types of cumulative added images are created in advance, which are made by cumulatively adding in the two directions the feature quantities of the reference areas, and the two types of projected data are created from a part of the two types of cumulative added images. 
     Further, it is preferable that the feature quantities of the reference areas are extracted by performing two or more processings on the reference areas. 
     Further, it is preferable that one of the two or more processings is an edge-detecting filter processing. 
     Further, the motion vector may be a motion vector of a block made by dividing the images. 
     Further, the motion vector may be a motion vector of the entire images. 
     Further, a second image-processing program according to the present invention is characterized by being configured to cause a computer to execute any one of the second image-processing methods according to the present invention. 
     Further, a second image-processing apparatus according to the present invention is characterized by including a unit which executes any one of the second image-processing methods according to the present invention. 
     Further, a second imaging apparatus according to the present invention is characterized by including an imaging unit capable of continuously imaging a subject to obtain plural images, and the second image-processing apparatus according to the present invention. 
     Effects 
     According to the present invention, there are realized an image-processing method, an image-processing program, an image-processing apparatus, and an imaging apparatus which are capable of detecting motion between plural images with high precision. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a block diagram showing a structure of an electronic still camera of a first embodiment. 
         FIG. 2  is a flowchart showing a flow of an image stabilization processing. 
         FIG. 3  is a view explaining the principle of step S 3 . 
         FIG. 4  is a view explaining the step S 3  when using projected data. 
         FIG. 5  is a view explaining the step S 3  when using cumulative added images. 
         FIG. 6  is a view explaining step S 4 . 
         FIG. 7  is a view showing representative motion vectors V a , V b , V c . 
         FIG. 8  is a view showing an approximate plane Sx of Δx. 
         FIG. 9  is a view showing an approximate plane Sy of Δy. 
         FIG. 10  is a view showing an example of local motion vectors V i  after replacement. 
         FIG. 11  is a flowchart showing a flow of a detection processing in a second embodiment. 
         FIG. 12  is a view explaining a concept of a method of calculating a correlation value d of the second embodiment (and the first embodiment). 
     
    
    
     DETAILED DESCRIPTION OF THE EMBODIMENTS 
     First Embodiment 
     Hereinafter, a first embodiment of the present invention will be explained. This embodiment is an embodiment of an electronic still camera having an image-stabilizing function. 
       FIG. 1  is a block diagram showing the structure of an electronic still camera. As shown in  FIG. 1 , in the electronic still camera  10 , there are arranged an imaging sensor  12 , a signal-processing circuit  13 , a memory  14 , a companding circuit  15 , a recording part  16 , and an image-processing circuit  17 . 
     A shooting lens  11  attached to the electronic still camera  10  forms an optical image of a subject on the imaging sensor  12 . The imaging sensor  12  images the optical image of the subject to obtain an image. This image is processed in the signal-processing circuit  13  and thereafter stored temporarily in the memory  14 . The image-processing circuit  17  performs an image-processing on the image, and the image after the image-processing is subjected to a compression processing in the companding circuit  15  and thereafter recorded in the recording part  16 . 
     Particularly, when the electronic still camera  10  is set to an image stabilization mode, the imaging sensor  12  obtains 2×n number of images at infinitesimal time intervals with underexposure of n levels relative to a proper exposure time or an exposure time set in advance. The image-processing circuit  17  inputs the plural images and executes an image stabilization processing to obtain a stabilized image. The stabilized image is recorded in the recording part  16  via the companding circuit  15 . 
     Next, an overall flow of the image stabilization processing by the image-processing circuit  17  will be explained. 
       FIG. 2  is a flowchart showing a flow of the image stabilization processing. As shown in  FIG. 2 , when plural images are inputted (Step S 1 ), the image-processing circuit  17  performs resolution-reducing conversion of the plural images to obtain plural coarse images (step S 2 ), and searches for a local motion vector from each block of the coarse images (step S 3 ). 
     Thereafter, the image-processing circuit  17  elects representative motion vectors (step S 4 ) from the local motion vectors, and approximates a motion vector distribution on images based on the representative motion vectors (step S 5 ). The processing of the aforementioned steps S 2  to S 5  is detection based on coarse images, and hence corresponds to “coarse detection”. 
     Then, the image-processing circuit  17  uses the motion vector distribution obtained by the coarse detection to search for a local motion vector freshly from each block of plural detailed images before being subjected to the resolution-reducing conversion (step S 6 ). The processing of this step S 6  is based on detailed images, and hence corresponds to “detail detection”. 
     Furthermore, the image-processing circuit  17  position-adjusts and composes the plural detailed images based on the local motion vector obtained in the step S 6  (step S 7 ), and outputs the image after the composition as the stabilized image (step S 8 ). 
     Next, steps S 3  to S 7  will be explained in detail. Here, for simplicity, it is assumed that the number of images is two. 
     (Step S 3 ) 
       FIG. 3  is a view explaining the principle of step S 3 . 
     As shown in  FIG. 3 , in this step, an edge-detecting filter processing is performed on each of the coarse images f 1 , f 2  to obtain edge images g 1 , g 2 . Then, each of the edge images g 1 , g 2  is divided in plural blocks R. The number of blocks R is 24 in total for example, that is, 6 blocks in a horizontal direction of an image and 4 blocks in a vertical direction of an image. Hereinafter, the horizontal direction of an image (left and right direction in the drawings) will be called an “x-direction”, and the vertical direction of an image (upward and downward direction in the drawings) will be called a “y-direction”. A search for a local motion vector from the blocks R is performed as follows. 
     First, as shown by enlargement in  FIG. 3 , reference areas E 1 , E 2  in blocks R of the edge images g 1 , g 2  are referred respectively, and the absolute value of a difference between the reference areas E 1 , E 2  is calculated as a correlation value d between the reference areas E 1 , E 2 . The calculation of the correlation value d is repeated while varying a position shift amount (x-direction: α, y-direction: β) between the reference areas E 1 , E 2 . 
     A variation range (search range) of this position shift amount (α, β) is set sufficiently wide. For example, when the size of each of the reference areas E 1 , E 2  is 75 pixels×75 pixels, the search range is set to α: ±15 pixels, β: ±15 pixels. 
     Then, in this search range, values (Δx, Δy) of the position shift amount (α, β) with which the correlation value d becomes a minimum are found out. These values (Δx, Δy) are considered as a local motion vector V of the blocks R. 
     However, in the above search, the operation amount increases when the reference areas E 1 , E 2 , which are two-dimensional data, are used as they are in the operation, and thus it is preferable to use, instead of the reference areas E 1 , E 2 , one-dimensional projected data created by adding the reference areas E 1 , E 2  in the y-direction and one-dimensional projected data created by adding the reference areas E 1 , E 2  in the x-direction. 
       FIG. 4  is a view explaining the step S 3  when using the projected data. 
     As shown in  FIG. 4 , in this case, the edge-detecting filter processing in the x-direction is performed on the coarse images f 1 , f 2 , to obtain edge images g 1 v, g 2 v, and the edge-detecting filter processing in the y-direction is performed on the coarse images f 1 , f 2  to obtain edge images g 1 h, g 2 h. Note that edge components of the y-direction appear in the edge images g 1 v, g 2 v, and edge components of the x-direction appear in the edge images g 1 h, g 2 h. 
     Among them, as shown by enlargement in  FIG. 4 , a reference area E 1  of the edge image g 1 v is added in the y-direction to create projected data L 1 h. 
     Further, a reference area E 2  of the edge image g 2 v is added in the y-direction to create projected data L 2 h. 
     Further, a reference area E 1  of the edge image g 1 h is added in the x-direction to create projected data L 1 v. 
     Further, a reference area E 2  of the edge image g 2 h is added in the x-direction to create projected data L 2 v. 
     In this case, the correlation value d between the reference areas E 1 , E 2  is represented by the sum of the absolute value of a difference between the projected data L 1 h, L 2 h and the absolute value of a difference between the projected data L 1 v, L 2 v. 
     Here, the projected data L 1 h, L 2 h, L 1 v, L 2 v need to be created every time the position shift amount (α, β) of the reference areas E 1 , E 2  changes. However, the operation amount becomes large when the reference areas E 1 , E 2  are added every time the position shift amount (α, β) changes, and hence cumulative added images of the edge images g 1 v, g 2 v, g 1 h, g 2 h may be created in advance. 
       FIG. 5  is a view explaining the step S 3  when the cumulative added images are used. 
     As shown in  FIG. 5 , also in this case, the edge images g 1 h, g 2 h, g 1 v, g 2 v are obtained from the coarse images f 1 , f 2 , similarly to the case in  FIG. 4 . 
     Among the edge images, the edge image g 1 v is cumulatively added in the y-direction to create a cumulative added image h 1 . 
     Further, the edge image g 2 v is cumulatively added in the y-direction to create a cumulative added image h 2 . 
     Further, the edge image g 1 h is cumulatively added in the x-direction to create a cumulative added image v 1 . 
     Further, the edge image g 2 h is cumulatively added in the x-direction to create a cumulative added image v 2 . 
     To denote specifically, calculation equations for these cumulative added images h 1 , h 2 , v 1 , v 2  are as follows. 
     
       
         
           
             
               
                 
                   
                     
                       
                         
                           
                             h 
                             1 
                           
                           ⁡ 
                           
                             ( 
                             
                               x 
                               , 
                               y 
                             
                             ) 
                           
                         
                         = 
                         
                           
                             ∑ 
                             
                               
                                 x 
                                 ′ 
                               
                               = 
                               0 
                             
                             x 
                           
                           ⁢ 
                           
                             
                               g 
                               
                                 1 
                                 ⁢ 
                                 v 
                               
                             
                             ⁡ 
                             
                               ( 
                               
                                 
                                   x 
                                   ′ 
                                 
                                 , 
                                 y 
                               
                               ) 
                             
                           
                         
                       
                     
                     
                       
                         
                           
                             v 
                             1 
                           
                           ⁡ 
                           
                             ( 
                             
                               x 
                               , 
                               y 
                             
                             ) 
                           
                         
                         = 
                         
                           
                             ∑ 
                             
                               
                                 y 
                                 ′ 
                               
                               = 
                               0 
                             
                             y 
                           
                           ⁢ 
                           
                             
                               g 
                               
                                 1 
                                 ⁢ 
                                 h 
                               
                             
                             ⁡ 
                             
                               ( 
                               
                                 x 
                                 , 
                                 
                                   y 
                                   ′ 
                                 
                               
                               ) 
                             
                           
                         
                       
                     
                   
                   
                     
                       
                         
                           
                             h 
                             2 
                           
                           ⁡ 
                           
                             ( 
                             
                               x 
                               , 
                               y 
                             
                             ) 
                           
                         
                         = 
                         
                           
                             ∑ 
                             
                               
                                 x 
                                 ′ 
                               
                               = 
                               0 
                             
                             x 
                           
                           ⁢ 
                           
                             
                               g 
                               
                                 2 
                                 ⁢ 
                                 v 
                               
                             
                             ⁡ 
                             
                               ( 
                               
                                 
                                   x 
                                   ′ 
                                 
                                 , 
                                 y 
                               
                               ) 
                             
                           
                         
                       
                     
                     
                       
                         
                           
                             v 
                             2 
                           
                           ⁡ 
                           
                             ( 
                             
                               x 
                               , 
                               y 
                             
                             ) 
                           
                         
                         = 
                         
                           
                             ∑ 
                             
                               
                                 y 
                                 ′ 
                               
                               = 
                               0 
                             
                             y 
                           
                           ⁢ 
                           
                             
                               g 
                               
                                 2 
                                 ⁢ 
                                 h 
                               
                             
                             ⁡ 
                             
                               ( 
                               
                                 x 
                                 , 
                                 
                                   y 
                                   ′ 
                                 
                               
                               ) 
                             
                           
                         
                       
                     
                   
                 
               
               
                 
                   [ 
                   
                     Equation 
                     ⁢ 
                     
                         
                     
                     ⁢ 
                     1 
                   
                   ] 
                 
               
             
           
         
       
     
     Then, as shown by enlargement in  FIG. 5 , the aforementioned projected data L 1 h are created by taking a difference between two top and bottom lines in a reference area E 1  of the cumulative added image h 1 . 
     Further, the aforementioned projected data L 2 h are created by taking a difference between two top and bottom lines in a reference area E 2  of the cumulative added image h 2 . 
     Further, the aforementioned projected data L 1 v are created by taking a difference between two left and right lines in a reference area E 1  of the cumulative added image v 1 . 
     Further, the aforementioned projected data L 2 v are created by taking a difference between two left and right lines in a reference area E 2  of the cumulative added image v 2 . 
     Therefore, in this case, the correlation value d between the reference areas E 1 , E 2  is calculated by a small operation amount with the following equation. Here, B is the block size. 
     
       
         
           
             
               
                 
                   
                     d 
                     ⁡ 
                     
                       ( 
                       
                         α 
                         , 
                         β 
                       
                       ) 
                     
                   
                   = 
                   
                     
                       
                         ∑ 
                         
                           i 
                           = 
                           α 
                         
                         
                           α 
                           + 
                           B 
                         
                       
                       ⁢ 
                       
                          
                         
                           
                             { 
                             
                               
                                 
                                   h 
                                   1 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   
                                     i 
                                     , 
                                     B 
                                   
                                   ) 
                                 
                               
                               - 
                               
                                 
                                   h 
                                   1 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   
                                     i 
                                     , 
                                     0 
                                   
                                   ) 
                                 
                               
                             
                             } 
                           
                           - 
                           
                             { 
                             
                               
                                 
                                   h 
                                   2 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   
                                     i 
                                     , 
                                     
                                       B 
                                       + 
                                       β 
                                     
                                   
                                   ) 
                                 
                               
                               - 
                               
                                 
                                   h 
                                   2 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   
                                     i 
                                     , 
                                     β 
                                   
                                   ) 
                                 
                               
                             
                             } 
                           
                         
                          
                       
                     
                     + 
                     
                         
                     
                     ⁢ 
                     
                       
                         ∑ 
                         
                           j 
                           = 
                           β 
                         
                         
                           β 
                           + 
                           B 
                         
                       
                       ⁢ 
                       
                          
                         
                           
                             { 
                             
                               
                                 
                                   v 
                                   1 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   
                                     B 
                                     , 
                                     j 
                                   
                                   ) 
                                 
                               
                               - 
                               
                                 
                                   v 
                                   1 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   
                                     0 
                                     , 
                                     j 
                                   
                                   ) 
                                 
                               
                             
                             } 
                           
                           - 
                           
                             { 
                             
                               
                                 
                                   v 
                                   2 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   
                                     
                                       B 
                                       + 
                                       α 
                                     
                                     , 
                                     j 
                                   
                                   ) 
                                 
                               
                               - 
                               
                                 
                                   v 
                                   2 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   
                                     α 
                                     , 
                                     j 
                                   
                                   ) 
                                 
                               
                             
                             } 
                           
                         
                          
                       
                     
                   
                 
               
               
                 
                   [ 
                   
                     Equation 
                     ⁢ 
                     
                         
                     
                     ⁢ 
                     2 
                   
                   ] 
                 
               
             
           
         
       
     
     (Step S 4 ) 
       FIG. 6  is a view explaining step S 4 . In  FIG. 6 , denoted by reference letters R 1 , R 2 , . . . , R 24  are respective blocks in an image, and denoted by reference letters V 1 , V 2 , . . . , V 24  are images of local motion vectors searched individually from the blocks R 1 , R 2 , . . . , R 24 . In this manner, hereinafter, the explanation will be given with the number of blocks R being 24, and a subscript denoting a block number being attached to each of amounts related to each of the blocks. 
     In this step, first the local motion vectors V 1 , V 2 , . . . , V 24  are evaluated. As evaluated values of each local motion vector V i , for example, the following two types of evaluated values ΔE i , e i  are used: 
     First Evaluated Value ΔE i    
     This value is the sum of correlation values d 1 , d 2 , . . . , d 24  of the respective blocks R 1 , R 2 , . . . , R 24  when a position shift amount substantially equivalent to values (Δx i , Δy i ) of the local motion vector v i  are applied tentatively to all the blocks R 1 , R 2 , . . . , R 24 . It can be considered that the smaller the first evaluated value ΔE i , the higher the reliability of the local motion vector V i . 
     However, when obtaining a correlated value d k  of blocks R k , it is desirable to perform the following search. Specifically, the correlation value d is calculated while varying the position shift amount (α, β) of the blocks R k  in a narrow range in the vicinity of the values (Δx i , Δy i ) equivalent to the local motion vector V i , and the smallest value of the correlation value d at this time is assumed as the correlation value d k  of the blocks R k . A variation range (search range) thereof is, for example, α: Δx i ±2 pixels, β: Δy i ±2 pixels. 
     Note that to denote specifically, a calculation equation for the first evaluated value ΔE i  is as follows (however, the number of blocks is 24). 
     
       
         
           
             
               
                 
                   
                     Δ 
                     ⁢ 
                     
                         
                     
                     ⁢ 
                     
                       E 
                       i 
                     
                   
                   = 
                   
                     
                       ∑ 
                       
                         k 
                         = 
                         1 
                       
                       24 
                     
                     ⁢ 
                     
                       
                         min 
                         
                           
                             α 
                             = 
                             
                               
                                 Δ 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   x 
                                   i 
                                 
                               
                               ± 
                               2 
                             
                           
                           , 
                           
                             β 
                             = 
                             
                               
                                 Δ 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   y 
                                   i 
                                 
                               
                               ± 
                               2 
                             
                           
                         
                       
                       ⁢ 
                       
                         [ 
                         
                           
                             d 
                             k 
                           
                           ⁡ 
                           
                             ( 
                             
                               α 
                               , 
                               β 
                             
                             ) 
                           
                         
                         ] 
                       
                     
                   
                 
               
               
                 
                   [ 
                   
                     Equation 
                     ⁢ 
                     
                         
                     
                     ⁢ 
                     3 
                   
                   ] 
                 
               
             
           
         
       
     
     Second Evaluated Value e i    
     This value is an edge amount of the block R i  which is the source for search of the local motion vector V i . For example, this value is the mean value of the absolute values of two edge components of the block R i  obtained by the edge-detecting filter processing in the x-direction and the edge-detecting filter processing in the y-direction. It can be considered that the larger the second evaluated value e i , the higher the reliability of the local motion vector V i . 
     Now, in this step, the first evaluated value ΔE i  is generated for each of the local motion vectors V 1 , V 2 , . . . , V 24 , and the local motion vectors V 1 , V 2 , . . . , V 24  are sorted in ascending numeric order of the first evaluated values ΔE i . However, regarding local motion vectors having the same first evaluated values ΔE i  as each other, second evaluated values e i  are generated further, and the local motion vectors are sorted in descending numeric order of the second evaluated values e i . 
     The local motion vector which is ranked first as a result of the above sorting is elected as a first representative motion vector V a  (refer to  FIG. 7 ). 
     Further, from the local motion vector which is ranked second or later local motion vectors, there are elected a second representative motion vector V b  and a third representative motion vector V c  (refer to  FIG. 7 ) which satisfy the following conditions: 
     The center coordinates of the blocks R a , R b , R c , which are sources of searches for the representative motion vectors V a , V b , V c , do not exist on the same straight line. 
     {Area of a triangle connecting center coordinates of the blocks R a , R b , R c }*{Sum of second evaluated values e a , e b , e c  of the representative motion vectors V a , V b , V c } is the largest. 
     The sum of the first evaluated values ΔE h , ΔE b , ΔE c  of the representative vectors V a , V b , V c  is the same as the sum of the first evaluated values of the local motion vectors at the top three ranks. 
     (Step S 5 ) 
       FIG. 8 ,  FIG. 9  are views explaining step S 5 . 
     In this step, as shown in  FIG. 8 , a distribution of an x-component Δx of a local motion vector V on an image is approximated based on x-components Δx a , Δx b , Δx c  of the representative motion vectors V a , V b , V c . 
       FIG. 8  shows three-dimensional coordinates includes two-dimensional coordinates (x, y) and a one-dimensional coordinate in the Δx direction on the image. At this time, the distribution of Δx is approximated by a plane Sx as shown by dotted lines in  FIG. 8 . 
     This plane Sx is a plane that passes three sets of coordinates (x a , y a , Δx a ), (x b , y b , Δx b ), (x c , y c , Δx c ) when the center coordinates of the block R a  are (x a , y a , 0), the center coordinates of the block R b  are (x b , y b , 0), and the center coordinates of the block R c  are (x c , y c , 0). That is, the plane Sx is specified by the following linear equation with three constants A 1 , A 2 , A 3 , which are determined from the aforementioned three sets of coordinates.
 
 Sx:Δx=A   1   x+A   2   y+A   3  
 
     Similarly, in this step, as shown in  FIG. 9 , a distribution of a y-component Δy of the local motion vector V on the image is approximated based on y-components Δy a , Δy b , Δy c  of the representative motion vectors V a , V b , V c . 
       FIG. 9  shows three-dimensional coordinates includes two-dimensional coordinates (x, y) and a one-dimensional coordinate in the Δy direction on the image. At this time, the distribution of Δy is approximated by a plane Sy as shown by dotted lines in  FIG. 9 . 
     This plane Sy is a plane that passes three sets of coordinates (x a , y a , Δy a ), (x b , y b , Δy b ), (x c , y c , Δy c ). The plane Sy is specified by the following linear equation with three constants B 1 , B 2 , B 3 , which are determined from the aforementioned three sets of coordinates.
 
 Sy:Δy=B   1   x+B   2   y+B   3  
 
     By the two planes Sx, Sy as above, the motion vector distribution on the image is represented. 
     Here, when the values of the aforementioned local motion vectors V 1 , V 2 , . . . , V 24  are replaced according to the motion vector distribution represented by the two planes Sx, Sy, results are as shown in  FIG. 10  for example. The distribution in the x-direction and the distribution in the y-direction of the local motion vectors V 1 , V 2 , . . . , V 24  after the replacement both become linear. 
     (Step S 6 ) 
     In this step, the local motion vectors V 1 , V 2 , . . . , V 24  after the replacement are used to search for the local motion vectors V 1 , V 2 , . . . , V 24  freshly from the blocks R 1 , R 2 , . . . , R 24  of the two detailed images before being subjected to the resolution-reducing conversion. 
     Upon the search, the same processings (edge-detecting filter processing, conversion into cumulative added images) as those performed on the two coarse images are performed on the two detailed images. 
     Further, since the general search is already finished by the coarse detection with the coarse images, a search range (variation range of the position shift amount (α, β)) of each block R i  is limited to the vicinity of values (Δx i , Δy i ) of the local motion vector V i  after the replacement. For example, when the sizes of the reference areas E 1 , E 2  are 200 pixels×200 pixels, the search range is set to α: Δx i ±6 pixels, β: Δy i ±6 pixels. Thus, when the search range is limited to a narrow range, the operation amount required for the search can be reduced. 
     However, since the local motion vector V i  after the replacement is obtained from the coarse images, it is counted by a small size. Therefore, the values (Δx i , Δy i ) need to be corrected before the search. In this correction, the values (Δx i , Δy i ) may be multiplied by a coefficient equal to the size ratio of the detailed image to the coarse image. 
     Under the above conditions, the local motion vectors V 1 , V 2 , . . . , V 24  are searched freshly from the respective blocks R 1 , R 2 , . . . , R 24 . The local motion vectors V 1 , V 2 , . . . , V 24  searched in this step are definitive local motion vectors. 
     (Step S 7 ) 
     In this step, the blocks R 1 , R 2 , . . . , R 24  of the two detailed images are position-adjusted and configured according to the definitive local motion vectors V 1 , V 2 , . . . , V 24 . Accordingly, one stabilized image is completed. Note that in this composition, necessary positions of the image are interpolated with pixels. 
     In this manner, when the position adjustment of an image is performed for every block, the definitive local motion vectors V 1 , V 2 , . . . , V 24  are used effectively, and thereby it becomes possible to handle cases such that a rotor component is generated in motion between two images, that zoom component is generated, and the like (end of step S 7 ). 
     As above, in the image stabilization processing of this embodiment, since the motion vector distribution is approximated based on the representative motion vectors having high reliability (steps S 3  to S 5 ), motion between images can be represented precisely without increasing the number of blocks. Moreover, since the approximate surface is a plane, the operation amount for approximating the motion vector distribution and the information amount for representing the motion vector distribution can be minimized. 
     Further, in the image stabilization processing of this embodiment, since the projected data are used to search for the local motion vector, the operation amount related to the search can be reduced more than by using the edge images as they are. Moreover, since the cumulative added images are created in advance, the operation amount related to creation of the projected data can be reduced as well. 
     Further, in the image stabilization processing of this embodiment, the general motion vector distribution is obtained based on the coarse images (step S 2  to S 5 ) in advance, and motion in detailed images is detected using the general motion vector distribution with a small operation amount (step S 6 ). Therefore, not only the motion can be detected with high precision, but also the operation amount required for the detection can be reduced. 
     (Other Matters) 
     Note that in the steps S 3 , S 6 , since the edge-detecting filter processing is adopted as the type of filter processing, the edge information are the feature quantity extracted from the blocks, but another type of filter processing may be adopted so as to replace the feature quantity extracted from the blocks with another feature quantity indicating a pattern in the blocks. 
     Further, in the step S 4 , the combination of the above-described first evaluated value and second evaluated value is used for electing the representative motion vectors V a , V b , V c , but the combination may be replaced with another combination as long as it indicates the reliability of a local motion vector. 
     Further, in the steps S 4 , S 5 , the three representative motion vectors V a , V b , V c  are used for approximating the motion vector distribution by a plane, but four or more representative motion vectors may be used. 
     Further, in the steps S 4 , S 5 , the approximated surface for the motion vector distribution is a plane, but it may be a curved surface. With a curved surface, motion between images can be represented in more detail. However, as the number of order of the approximated surface increases, the number of representative motion vectors to be elected in the step S 4  increases as well. 
     Further, in the image stabilization processing of this embodiment, the motion vector distributions of the coarse images are approximated (steps S 3  to S 5 ), and then the motion vectors for the detailed images are searched freshly (step S 6 ), but the similar approximation processing (steps S 3  to S 5 ) may be performed on the detailed images directly. However, dividing in two phases as in this embodiment is more efficient. 
     Further, in this embodiment, the image stabilization processing is executed by a circuit (image-processing circuit  17 ) in the electronic still camera, but a part or all of the image stabilization processing may be executed by another imaging apparatus such as a video camera, various types of image input apparatuses, a computer, or the like. In addition, when using a computer to execute the processing, a program for executing the processing may be installed in the computer. 
     Second Embodiment 
     Hereinafter, a second embodiment of the present invention will be explained. This embodiment is an embodiment of a motion vector detection processing. 
     Here, again for simplicity, the number of images are two, and the target of detection is a motion vector in a block in the two images. Further, the coordinates and size of the block are arbitrary. Therefore, when the size of the block is set large, motion vectors of the entire two images can be detected. In addition, when the block is divided into plural blocks and a motion vector (=local motion vector) of each block is detected, this detection processing can be applied to the step S 3  in the first embodiment. 
       FIG. 11  is a flowchart showing the flow of the detection processing of this embodiment. The respective steps will be explained in detail. 
     (Steps S 11 , S 21 ) 
     In these steps, two images f 1 (x, y), f 2 (x, y) obtained at an infinitesimal time interval are inputted. 
     (Steps S 12 , S 22 ) 
     In these steps, an edge-detecting filter processing is performed on each of the two images f 1 (x, y), f 2 (x, y). Filter outputs g 1 v(x, y), g 1 h(x, y) in the vertical/horizontal directions (x/y-directions) for the first image f 1 (x, y), and filter outputs g 2 v(x, y), g 2 h(x, y) in the vertical/horizontal directions (x/y-directions) for the second image f 2 (x, y) are represented by the following equations respectively:
 
 g   1   v ( x,y )=− f   1 ( x,y− 2)− f   1 ( x,y− 1)+ f   1 ( x,y+ 1)+ f   1 ( x,y+ 2)
 
 g   2   v ( x,y )=− f   2 ( x,y− 2)− f   2 ( x,y− 1)+ f   2 ( x,y+ 1)+ f   2 ( x,y+ 2)
 
 g   1   h ( x,y )=− f   1 ( x− 2 ,y )− f   1 ( x− 1 ,y )+ f   1 ( x+ 1 ,y )+ f   1 ( x+ 2 ,y )
 
 g   2   h ( x,y )=− f   2 ( x− 2 ,y )− f   2 ( x− 1 ,y )+ f   2 ( x+ 1 ,y )+ f   2 ( x+ 2 ,y )
 
     In these equations, the filter output g 1 v(x, y) corresponds to the edge image g 1 v(x, y) explained in the first embodiment ( FIG. 5 ), the filter output g 2 v(x, y) corresponds to the edge image g 2 v(x, y) explained in the first embodiment, the filter output g 1 h(x, y) corresponds to the edge image g 1 h(x, y) explained in the first embodiment, and the filter output g 2 h(x, y) corresponds to the edge image g 2 h(x, y) explained in the first embodiment. 
     However, the edge-detecting filter processing with the above equations supposes edge filters Fa, Fb as shown in  FIG. 12 , which have different sizes from the edge filters shown in  FIG. 5 . 
     Furthermore, in these steps, another processing is combined with such an edge-detecting filter processing. For example, a noise-reducing processing is combined. The noise-reducing processing compares respective pixel values of the aforementioned filter outputs g 1 v(x, y), g 1 h(x, y), g 2 v(x, y), g 2 h(x, y) with a predetermined threshold value, and replaces a pixel value that is equal to or lower than the threshold value with zero. Thus, noise in the filter outputs g 1 v(x, y), g 1 h(x, y), g 2 v(x, y), g 2 h(x, y) is reduced. 
     (Steps S 13 , S 23 ) 
     In these steps, images are created by the filter outputs g 1 v(x, y), g 2 v(x, y), g 1 h(x, y), g 2 h(x, y) respectively after the noise reduction. Hereinafter, the created images are referred to as feature-quantity images g 1 v(x, y), g 2 v(x, y), g 1 h(x, y), g 2 h(x, y). However, for reducing the amount of the memory to be used, it is also possible to omit these steps and use the filter outputs g 1 v(x, y), g 2 v(x, y), g 1 h(x, y), g 2 h(x, y) as they are in the next step. 
     (Steps S 14 , S 24 ) 
     In these steps, cumulative added images h 1 (x, y), h 2 (x, Y), v 1 (x, y), v 2 (x, y) are created based on the feature-quantity images g 1 v(x, y), g 2 v(x, y), g 1 h(x, y), g 2 h(x, y) respectively. Calculation equations for the cumulative added images h 1 (x, y), h 2 (x, y), v 1 (x, y), v 2 (x, y) are the same as those explained in the first embodiment and are as follows. 
     
       
         
           
             
               
                 
                   
                     
                       
                         
                           
                             h 
                             1 
                           
                           ⁡ 
                           
                             ( 
                             
                               x 
                               , 
                               y 
                             
                             ) 
                           
                         
                         = 
                         
                           
                             ∑ 
                             
                               
                                 x 
                                 ′ 
                               
                               = 
                               0 
                             
                             x 
                           
                           ⁢ 
                           
                             
                               g 
                               
                                 1 
                                 ⁢ 
                                 v 
                               
                             
                             ⁡ 
                             
                               ( 
                               
                                 
                                   x 
                                   ′ 
                                 
                                 , 
                                 y 
                               
                               ) 
                             
                           
                         
                       
                     
                     
                       
                         
                           
                             v 
                             1 
                           
                           ⁡ 
                           
                             ( 
                             
                               x 
                               , 
                               y 
                             
                             ) 
                           
                         
                         = 
                         
                           
                             ∑ 
                             
                               
                                 y 
                                 ′ 
                               
                               = 
                               0 
                             
                             y 
                           
                           ⁢ 
                           
                             
                               g 
                               
                                 1 
                                 ⁢ 
                                 h 
                               
                             
                             ⁡ 
                             
                               ( 
                               
                                 x 
                                 , 
                                 
                                   y 
                                   ′ 
                                 
                               
                               ) 
                             
                           
                         
                       
                     
                   
                   
                     
                       
                         
                           
                             h 
                             2 
                           
                           ⁡ 
                           
                             ( 
                             
                               x 
                               , 
                               y 
                             
                             ) 
                           
                         
                         = 
                         
                           
                             ∑ 
                             
                               
                                 x 
                                 ′ 
                               
                               = 
                               0 
                             
                             x 
                           
                           ⁢ 
                           
                             
                               g 
                               
                                 2 
                                 ⁢ 
                                 v 
                               
                             
                             ⁡ 
                             
                               ( 
                               
                                 
                                   x 
                                   ′ 
                                 
                                 , 
                                 y 
                               
                               ) 
                             
                           
                         
                       
                     
                     
                       
                         
                           
                             v 
                             2 
                           
                           ⁡ 
                           
                             ( 
                             
                               x 
                               , 
                               y 
                             
                             ) 
                           
                         
                         = 
                         
                           
                             ∑ 
                             
                               
                                 y 
                                 ′ 
                               
                               = 
                               0 
                             
                             y 
                           
                           ⁢ 
                           
                             
                               g 
                               
                                 2 
                                 ⁢ 
                                 h 
                               
                             
                             ⁡ 
                             
                               ( 
                               
                                 x 
                                 , 
                                 
                                   y 
                                   ′ 
                                 
                               
                               ) 
                             
                           
                         
                       
                     
                   
                 
               
               
                 
                   [ 
                   
                     Equation 
                     ⁢ 
                     
                         
                     
                     ⁢ 
                     4 
                   
                   ] 
                 
               
             
           
         
       
     
     (Step S 35 ) 
     In this step, center coordinates (cx, cy) of a block to be a source of search for a motion vector and sizes (Bx, By) thereof are determined. Bx is a size in the x-direction, and By is a size in the y-direction. In addition, when setting the source of search for a motion vector to the entire image, the center coordinates of the block may be set as the center coordinates of the image, and match the size of the block with the size of the entire image. 
     (Step S 36 ) 
     In this step, a reference area E 1  in a block in the first image f 1  and a reference area E 2  in a block in the second image f 2 (x, y) are set. However, sizes of the reference areas E 1 , E 2  are in common and the coordinates of the reference area E 1  in the block are fixed, and hence only a position shift amount (α, β) of the reference area E 2  with the reference area E 1  being the basis needs to be set here. 
     (Step S 37 ) 
     In this step, a correlation value d(α, β) between the reference areas E 1 , E 2  is calculated. The method of calculating the correlation value d(α, β) is basically the same as in the first embodiment. Specifically, in this step, as shown conceptually in  FIG. 12 , projected data L 1 h, L 1 v in the x/y directions of the reference area E 1  are created, similar projected data L 2 v, L 2 h are created also for the reference area E 2 , and these four types of projected data L 1h , L 1v , L 2h , L 2v  are added and/or subtracted to calculate the correlation value d(α, β). 
     Then, also in this step, the cumulative added images h 1 (x, y), h 2 (x, y), v 1 (x, y), v 2 (x, y) are used for calculating the projected data L 1h , L 1v , L 2h , L 2v . However, in this step, since the center coordinates (cx, cy) and the sizes (Bx, By) of the block are arbitrary, the following equation, which is made by generalizing the calculation equation explained in the first embodiment, is used for calculating the correlation value d(α, β): 
     
       
         
           
             
               
                 
                   
                     d 
                     ⁡ 
                     
                       ( 
                       
                         α 
                         , 
                         β 
                       
                       ) 
                     
                   
                   = 
                   
                     
                       
                         ∑ 
                         
                           i 
                           = 
                           
                             
                               c 
                               x 
                             
                             - 
                             
                               
                                 B 
                                 x 
                               
                               ⁢ 
                               
                                 / 
                               
                               ⁢ 
                               2 
                             
                           
                         
                         
                           
                             c 
                             x 
                           
                           + 
                           
                             
                               B 
                               x 
                             
                             ⁢ 
                             
                               / 
                             
                             ⁢ 
                             2 
                           
                         
                       
                       ⁢ 
                       
                          
                         
                           
                             { 
                             
                               
                                 
                                   v 
                                   1 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   
                                     i 
                                     , 
                                     
                                       
                                         c 
                                         y 
                                       
                                       + 
                                       
                                         
                                           B 
                                           y 
                                         
                                         ⁢ 
                                         
                                           / 
                                         
                                         ⁢ 
                                         2 
                                       
                                     
                                   
                                   ) 
                                 
                               
                               - 
                               
                                 
                                   v 
                                   1 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   
                                     i 
                                     , 
                                     
                                       
                                         c 
                                         y 
                                       
                                       - 
                                       
                                         
                                           B 
                                           y 
                                         
                                         ⁢ 
                                         
                                           / 
                                         
                                         ⁢ 
                                         2 
                                       
                                     
                                   
                                   ) 
                                 
                               
                             
                             } 
                           
                           - 
                           
                               
                           
                           ⁢ 
                           
                             { 
                             
                               
                                 
                                   v 
                                   2 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   
                                     
                                       i 
                                       + 
                                       α 
                                     
                                     , 
                                     
                                         
                                     
                                     ⁢ 
                                     
                                       
                                         c 
                                         y 
                                       
                                       + 
                                       
                                         
                                           B 
                                           y 
                                         
                                         ⁢ 
                                         
                                           / 
                                         
                                         ⁢ 
                                         2 
                                       
                                       + 
                                       β 
                                     
                                   
                                   ) 
                                 
                               
                               - 
                               
                                 
                                   v 
                                   2 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   
                                     
                                       i 
                                       + 
                                       α 
                                     
                                     , 
                                     
                                       
                                         c 
                                         y 
                                       
                                       - 
                                       
                                         
                                           B 
                                           y 
                                         
                                         ⁢ 
                                         
                                           / 
                                         
                                         ⁢ 
                                         2 
                                       
                                       + 
                                       β 
                                     
                                   
                                   ) 
                                 
                               
                             
                             } 
                           
                         
                          
                       
                     
                     + 
                     
                       
                         ∑ 
                         
                           j 
                           = 
                           
                             
                               c 
                               y 
                             
                             - 
                             
                               
                                 B 
                                 y 
                               
                               ⁢ 
                               
                                 / 
                               
                               ⁢ 
                               2 
                             
                           
                         
                         
                           
                             c 
                             y 
                           
                           + 
                           
                             
                               B 
                               y 
                             
                             ⁢ 
                             
                               / 
                             
                             ⁢ 
                             2 
                           
                         
                       
                       ⁢ 
                       
                          
                         
                           
                             { 
                             
                               
                                 
                                   h 
                                   1 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   
                                     
                                       
                                         c 
                                         x 
                                       
                                       + 
                                       
                                         
                                           B 
                                           x 
                                         
                                         ⁢ 
                                         
                                           / 
                                         
                                         ⁢ 
                                         2 
                                       
                                     
                                     , 
                                     j 
                                   
                                   ) 
                                 
                               
                               - 
                               
                                 
                                   h 
                                   1 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   
                                     
                                       
                                         c 
                                         x 
                                       
                                       - 
                                       
                                         
                                           B 
                                           x 
                                         
                                         ⁢ 
                                         
                                           / 
                                         
                                         ⁢ 
                                         2 
                                       
                                     
                                     , 
                                     j 
                                   
                                   ) 
                                 
                               
                             
                             } 
                           
                           - 
                           
                             { 
                             
                               
                                 
                                   h 
                                   2 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   
                                     
                                       
                                         c 
                                         x 
                                       
                                       + 
                                       
                                         
                                           B 
                                           x 
                                         
                                         ⁢ 
                                         
                                           / 
                                         
                                         ⁢ 
                                         2 
                                       
                                       + 
                                       α 
                                     
                                     , 
                                     
                                       j 
                                       + 
                                       β 
                                     
                                   
                                   ) 
                                 
                               
                               - 
                               
                                 
                                   h 
                                   2 
                                 
                                 ⁡ 
                                 
                                   ( 
                                   
                                     
                                       
                                         c 
                                         x 
                                       
                                       - 
                                       
                                         
                                           B 
                                           x 
                                         
                                         ⁢ 
                                         
                                           / 
                                         
                                         ⁢ 
                                         2 
                                       
                                       + 
                                       α 
                                     
                                     , 
                                     
                                       j 
                                       + 
                                       β 
                                     
                                   
                                   ) 
                                 
                               
                             
                             } 
                           
                         
                          
                       
                     
                   
                 
               
               
                 
                   [ 
                   
                     equation 
                     ⁢ 
                     
                         
                     
                     ⁢ 
                     5 
                   
                   ] 
                 
               
             
           
         
       
     
     (Steps S 38 , S 39 ) 
     The calculation of the correlation value d(α, β) as above is repeated while varying the position shift amount (α, β). 
     (Step S 40 ) 
     In this step, values (Δx, Δy) of the position shift amount (α, β) with which the correlation value d(α, β) becomes a minimum are found out, and these values (Δx, Δy) are outputted as the motion vector V. Thus, the motion vector detection processing is completed. 
     As above, also in this embodiment, since the projected data L 1h , L 1v , L 2h , L 2v  are used in the search for the motion vector V (steps S 36  to S 39 ), the operation amount related to the search can be reduced more than by using the feature-quantity images g 1 v(x, y), g 2 v(x, y), g 1 h(x, y), g 2 h(x, y) as they are. Moreover, since the cumulative added images h 1 (x, y), h 2 (x, y), v 1 (x, y), v 2 (x, y) are created in advance in the steps S 14 , S 24 , the operation amount related to creation of the projected data L 1h , L 1v , L 2h , L 2v  and further to calculation of the correlation value d(α, β) (step S 37 ) can be reduced as well. 
     Therefore, in this embodiment, the search for the motion vector V (steps S 36  to S 39 ) are performed quickly. 
     Further, in the steps S 12 , S 22  of this embodiment, since two or more processings including the edge-detecting filter processing are performed on the images f 1 (x, y), f 2 (x, y), feature-quantity images g 1 v(x, y), g 2 v(x, y), g 1 h(x, y), g 2 h(x, y) can be obtained in good state. Therefore, in this embodiment, the motion vector V can be detected with high precision. 
     (Other Matters) 
     Note that in the above-described steps S 12 , S 22 , the edge-detecting filter processing is adopted for extracting the edge information, but another type of filter processing may be adopted as long as it is capable of extracting the feature quantity. 
     Further, the detecting method of this embodiment processes plural inputted images, but images made by resolution-reducing conversion of plural inputted images may be processed similarly. 
     Further, the detecting method of this embodiment is applicable to an image stabilization method which position-adjusts and composes two images based on motion vectors of these images. 
     Further, the detecting method of this embodiment or an image stabilization method to which the detecting method is applied are executed by an imaging apparatus, an image input apparatus, a computer, or the like. When executing in a computer, a program for executing the processing thereof may be installed in the computer. 
     The many features and advantages of the embodiments are apparent from the detailed specification and, thus, it is intended by the appended claims to cover all such features and advantages of the embodiments that fall within the true spirit and scope thereof. Further, since numerous modifications and changes will readily occur to those skilled in the art, it is not desired to limit the inventive embodiments to exact construction and operation illustrated and described, and accordingly all suitable modifications and equivalents may be resorted to, falling within the scope thereof.