Abstract:
An automatic airview correction method comprises steps: moving a vehicle to an airview alignment pattern; capturing a plurality of airview alignment images of the surroundings of the vehicle; correcting distortion of the airview alignment images to obtain a plurality of corrected images; performing alignment compensation on the corrected images; searching for corner points of the corrected images and converting view point s to obtain a plurality of view angle-converted images; and searching for corner points of the view angle-converted images, and seaming the view angle-converted images to form a panoramic airview and obtain parameters corresponding to the panoramic airview. The present invention can automatically align images and can also automatically detect corner points to seam the images of the surroundings of a vehicle, whereby to form a panoramic airview.

Description:
BACKGROUND OF THE INVENTION 
       [0001]    1. Field of the Invention 
         [0002]    The present invention relates to an airview seaming technology, particularly to a method to automatically correct the airviews of the surroundings of a vehicle. 
         [0003]    2. Description of the Related Art 
         [0004]    Digital cameras are fast falling in price and have been widely used as auxiliary instruments of vehicles. As a driver is unlikely to watch several images simultaneously, some manufacturers have integrated several primitive images into a 2D surrounding airview, which is more suitable for human vision. The conventional technology uses three steps to achieve a panoramic airview, including distortion correction, view point conversion, and image seaming. However, the conventional technology needs human decisions and manual parameter setting in undertaking the three steps. Therefore, the conventional technology is inconvenient and time-consuming. Besides, the precision of the panoramic airview correction is likely to be degraded by errors of camera installation. 
         [0005]    A Taiwan patent No. I320756 disclosed a technology, which uses a fisheye lens to capture surrounding images of a vehicle and uses distortion correction, view point conversion, and image seaming to obtain a single piece of airview, wherein a planar homography matrix is used to convert the view points. The prior art finds out identical features of the overlapped regions of the images captured by two different cameras for seaming the images. However, the present invention is characterized in using a blocking matching technology to automatically search for the feature points, whereby to realize the automation of airview correction. Further, the present invention can automatically compensate the installation errors of cameras. 
         [0006]    A Taiwan patent No. 201103787 disclosed a technology, which uses a mapping table to correct image distortions, convert view points and seam images to obtain an airview. However, the prior art needs human decisions and manual parameter setting in image distortion correction, view point conversion and image seaming. Contrarily, the present invention provides an automatic corner point searching device and an automatic corner point matching device to replace human decision and manual parameter setting, whereby to achieve full automatic airview correction and higher-precision airviews. 
         [0007]    Thus, the present invention proposes an automatic airview correction method to solve the abovementioned problems. 
       SUMMARY OF THE INVENTION 
       [0008]    The primary objective of the present invention is to provide an automatic airview correction method, which can correct the errors of images automatically and can increase precision of image seaming, and which can automatically search for corner points to realize automation of image distortion correction, view point conversion, and multiple cycles of image seaming. 
         [0009]    Another objective of the present invention is to provide an automatic airview correction method, which can automatically perform multiple cycles of image seaming, whereby is obtained a more complete and clearer airview. 
         [0010]    A further objective of the present invention is to provide an automatic airview correction method applicable to various airview correction systems. 
         [0011]    To achieve the abovementioned objectives, the present invention proposes an automatic airview correction method, which comprises steps: moving a vehicle to an airview alignment pattern; capturing a plurality of airview alignment images from the surroundings of the vehicle; correcting distortion of the airview alignment images to obtain a plurality of corrected images; performing alignment compensation on the corrected images; searching for corner points of the corrected images and converting view angles to obtain a plurality of view angle-converted images; and searching for corner points of the view angle-converted images, and seaming the view angle-converted images to form a panoramic airview and obtain parameters corresponding to the panoramic airview. 
         [0012]    Below, embodiments are described in detail to make easily understood the objectives, technical contents, characteristics and accomplishments of the present invention. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0013]      FIG. 1  shows a flowchart of an automatic airview correction method according to one embodiment of the present invention; 
           [0014]      FIGS. 2   a - 2   h  schematically show steps of an automatic airview correction method according to one embodiment of the present invention; 
           [0015]      FIG. 3  schematically shows the image alignment of an automatic airview correction method according to one embodiment of the present invention; 
           [0016]      FIG. 4  shows a flowchart of an automatic corner point search method according to one embodiment of the present invention; 
           [0017]      FIGS. 5   a - 5   c  schematically show the feature of a corner point and the search of the corner point according to one embodiment of the present invention; 
           [0018]      FIGS. 6   a - 6   d  schematically show regions whose images are captured by image capture devices according to one embodiment of the present invention; 
           [0019]      FIG. 7  schematically shows the calculation of the correlation coefficient of overlapped sections according to one embodiment of the present invention; and 
           [0020]      FIG. 8  schematically shows a panoramic airview whose virtual view point is variable according to one embodiment of the present invention. 
       
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
       [0021]    Refer to  FIG. 1  and  FIGS. 2   a - 2   h . In Step S 10 , move a vehicle  10  to an airview alignment pattern  14 , as shown in  FIG. 2   a . The airview alignment pattern  14  is a template where a correction pattern is drawn. The airview alignment pattern  14  may be place in a correction field. Next, in Step S 12 , use a fisheye lens to capture a plurality of airview alignment images from the surroundings of the vehicle  10 . In this embodiment, only the image of the region before the vehicle  10  is used to exemplify the airview alignment images, as shown in  FIG. 2   b . The images captured by the fisheye lens are distorted images. Next, in Step S 14 , use a fisheye correction method to correct the distorted airview alignment images captured by the fisheye lens to obtain undistorted airview alignment images (corrected images), as shown in  FIG. 2   c . Next, in Step S 16 , perform alignment compensation on the corrected images and convert them into binary images (as shown in  FIG. 2   d ) to accelerate image processing. Next, in Step S  18 , automatically search for the corner points of the corrected images and use a space transformation method to convert the view point s to obtain a plurality of view angle-converted images, as shown in  FIG. 2   e . Next, in Step S 20 , search for the corner points of the view angle-converted images (as shown in  FIG. 2   f ), and use a bilinear method to seam the view angle-converted images into a panoramic airview (as shown in  FIG. 2   g ). As the distortions of the view angle-converted images are corrected simultaneously in seaming images, the seamed image is exempt from distortions. As the present invention adopts an automatic corner point search method, the present invention can undertake more cycles of image seaming to obtain a clearer panoramic airview, as shown in  FIG. 1   h . Then, obtain and record the parameters corresponding to the panoramic air view. When intending to watch the panoramic airview, the user utilizes the parameters to replay the panoramic airview. 
         [0022]    As the installation position of a camera has an error inevitably, the captured region is more or less deviated from the intended region. The precision of seaming images varies with the installation errors of cameras. Therefore, the present invention proposes an image alignment method to increase precision of panoramic airviews. Below is described the image alignment method. Refer to  FIG. 3 . The image alignment method compares alignment reference squares  16  of the airview alignment pattern  14  with the alignment reference squares  18  of a physical image to evaluate the deviation of the physical image. Then, the present invention compensates the deviation of the image. Thereby are compensated the installation errors of cameras and obtained clearer panoramic airviews. 
         [0023]    The present invention further uses an automatic corner point search method to realize full automation of airview synthesis. Below is described the automatic corner point search method. Refer to  FIG. 4 . In Step S 22 , define the feature of a corner point. Refer to  FIG. 5   a . A reticule  22  is formed on a corner point search square  20  to partition the corner point search square  20  into four blocks numbered in sequence. In  FIG. 5   a , the blocks  201 ,  202 ,  204  and  203  are respectively numbered with 1, 2, 3, and  4  clockwise. Then, the present invention establishes the gray levels of the blocks of the corner point to be searched for, and uses the gray levels of the blocks as the feature to recognize the corner point. Next, in Step S 24 , define the area to search for the corner point in the corrected image or the view angle-converted image according to the deviation of the alignment reference square  18 . Next, in Step S 26 , search for the corner point matching the feature of the intended corner point in the corrected image or the view angle-converted image. Refer to  FIG. 5   b . The center  24  of the reticule  22  is preset to be the center to search for the pixels matching the gray levels (feature) of the four blocks in the search area. The point whose surrounding pixels match the feature is regarded as the corner point. In  FIG. 5   b , the corner point search square  20  features that Block  3  ( 204 ) has a gray level different from the gray level of the other three blocks, and the upper left corner point matches the feature. In  FIG. 5   c , the corner point search square  20  features that Block  1  ( 201 ) has a gray level different from the gray level of the other three blocks, and the lower right corner point matches the feature. Thereby, the present invention can find out the corner points in the corrected image or the view angle-converted image. 
         [0024]    The present invention is characterized in replacing the conventional human decision and manual parameter setting with the automatic image alignment method and automatic corner point search method. Therefore, the present invention can realize a fully automatic airview correction system. The automatic corner point search method of the present invention can promote the precision of the panoramic airview. Further, the present invention can undertake multiple cycles of image seaming to form a clearer panoramic airview. 
         [0025]    After having obtained the parameters of a panoramic airview, the present invention can further apply to a panoramic airview-based obstacle detection method. The parameters of panoramic airviews can be used to convert a plurality of primitive images into a plurality of independent panoramic airviews. The correlation of the images of the overlapped sections of two independent panoramic airviews is used to detect obstacles. If the correlation is high, it means that none obstacle exists. If the correlation is low, it means that an obstacle appears in the overlapped sections. While detecting an obstacle appearing in the panoramic airview, the system emits a warning signal to remind the driver of the obstacle. 
         [0026]    Below is further described how to compare the overlapped sections of two independent airviews. The present invention uses a plurality of image capture devices to capture the images of the regions surrounding a vehicle and uses the abovementioned automatic airview correction method to convert the captured images into panoramic airviews. Refer to  FIGS. 6   a - 6   c . The present invention uses a plurality of image capture devices to capture the images of 6 regions surrounding a vehicle  10  and respectively designated by A, B, C, D, E, and F. Refer to  FIG. 6   d . Regions A, B, C, D, E, and F include overlapped sections  261 ,  262 ,  263 ,  264 ,  265 , and  266 . An image processor respectively converts the primitive images of the 6 regions into independent airviews. An obstacle calculation unit compares the images of the overlapped sections ( 261 ,  262 ,  263 ,  264 ,  265 , or  266 ) of each two neighboring panoramic airviews and determines the correlation of the images of the overlapped sections. Refer to  FIG. 7 . The overlapped section  262  of Region A in  FIG. 6   a  and the overlapped section  262  of Region E in  FIG. 6   c  are used to exemplify the comparison. The gray level of the image of the overlapped section  262  of Region A is expressed by f(s, t). The gray level of the image of the overlapped section  262  of Region E is expressed by w(s, t). The image processor divides the image of each overlapped section into 9 sub-sections and calculates the correlation coefficient γ of the gray levels of the corresponding sub-sections according to the following equation: 
         [0000]    
       
         
           
             γ 
             = 
             
               
                 
                   ∑ 
                   s 
                 
                  
                 
                   
                     ∑ 
                     t 
                   
                    
                   
                     
                       [ 
                       
                         
                           f 
                            
                           
                             ( 
                             
                               s 
                               , 
                               t 
                             
                             ) 
                           
                         
                         - 
                         
                           f 
                           _ 
                         
                       
                       ] 
                     
                      
                     
                       [ 
                       
                         
                           w 
                            
                           
                             ( 
                             
                               s 
                               , 
                               t 
                             
                             ) 
                           
                         
                         - 
                         
                           w 
                           _ 
                         
                       
                       ] 
                     
                   
                 
               
               
                 
                   { 
                   
                     
                       ∑ 
                       s 
                     
                      
                     
                       
                         ∑ 
                         t 
                       
                        
                       
                         
                           
                             [ 
                             
                               
                                 f 
                                  
                                 
                                   ( 
                                   
                                     s 
                                     , 
                                     t 
                                   
                                   ) 
                                 
                               
                               - 
                               
                                 f 
                                 _ 
                               
                             
                             ] 
                           
                           2 
                         
                          
                         
                           
                             ∑ 
                             s 
                           
                            
                           
                             
                               ∑ 
                               t 
                             
                              
                             
                               
                                 [ 
                                 
                                   
                                     w 
                                      
                                     
                                       ( 
                                       
                                         s 
                                         , 
                                         t 
                                       
                                       ) 
                                     
                                   
                                   - 
                                   
                                     w 
                                     _ 
                                   
                                 
                                 ] 
                               
                               2 
                             
                           
                         
                       
                     
                   
                   } 
                 
                 
                   1 
                   / 
                   2 
                 
               
             
           
         
       
     
         [0000]    wherein  f  and  w  are respectively the average gray levels of the images of the overlapped sections of Region A and Region E. The equation is to calculate the difference of each two corresponding points of the two overlapped sections and obtain the correlation coefficient γ of the two overlapped sections. If the two overlapped sections are completely identical, the correlation coefficient γ equals 1. If the two overlapped sections are completely different, the correlation coefficient γ equals 0. In the present invention, a correlation coefficient γ of 0.5 is used as a threshold for obstacle judgement. If the correlation coefficient γ is smaller than the threshold, it means that an obstacle exists. In  FIG. 7 , Areas  28  of the overlapped sections  262  has a correlation coefficient γ of smaller than 0.5 and thus has an obstacle therein. While the surfaces in two overlapped sections are planar and smooth, the panoramic airviews would not be distorted. In such a case, the correlation coefficient γ of the two overlapped sections is higher the threshold 0.5. Thus, none obstacle appears in the overlapped sections. Therefore, the present invention can determine whether an obstacle exists via comparing the overlapped sections of the panoramic airviews. 
         [0027]    After obtaining the parameters of the panoramic airviews, the present invention further obtains a relationship between the rotation angle of the steering wheel (by a graduation of such as 5 degrees) and the corresponding virtual view points to shift the panoramic airview from a view point to another view point and implement image transformation, using the spatial information of the cameras in the vehicle body, a virtual view point coordinate transformation matrix, a perspective transformation equation, and an inverse perspective transformation equation. While capturing images, the present invention selects the corresponding panoramic airview parameters according to the rotation angle of the steering wheel. Next, the present invention uses the panoramic airview parameters to fast process the captured images and obtain a panoramic airview. Refer to  FIG. 8  for an example. While the driver turns the vehicle  10  to right, the image processor receives a signal from the vehicle  10  and uses a preset parameter transformation to present to the driver a panoramic airview  32  of the region at the front right of the vehicle  10 , which is viewed from a virtual view point  30  at the rear left of the vehicle  10 . At the same time, the present invention also detects obstacles within the region at the front right of the vehicle  10 . 
         [0028]    In conclusion, the present invention can correct image errors and search for corner points automatically to realize the automation of image distortion correction, view point conversion, and image seaming. Further, the present invention can undertake multiple cycles of image seaming to obtain a clearer panoramic airview. The method of the present invention can apply to various airview correction systems. 
         [0029]    The embodiments described above are only to exemplify the present invention but not to limit the scope of the present invention. Any equivalent modification or variation according to the characteristic or spirit of the present invention is to be also included within the scope of the present invention.