Patent Abstract:
The invention discloses the asynchronous photography for dual camera apparatus and processing the method for real-time forward vehicle detection. Image is captured by a pair of monochrome camera and stored into a computer. After the video pre-process, the edge information is used to locate the forward vehicle position, and then obtained the disparity from a fast comparison search algorithm by the stereo vision methodology. Proposed algorithm calculation of the invention can conquer the asynchronous exposure problem from dual camera and lower the hardware cost.

Full Description:
BACKGROUND OF THE INVENTION 
       [0001]    1. Field of the Invention 
         [0002]    The invention relates to an automobile-detecting apparatus, particularly to an asynchronous photography automobile-detecting apparatus. 
         [0003]    2. Description of the Prior Art 
         [0004]    In the conventional technology, a binocular cameras system installed on a same horizontal plane is used to carry on the distance measurement of stereovision, in order to form the image at the same time. There is a fixed distance between the binocular cameras, thus there is a disparity between left image and right image. This disparity is the coordinate difference of left image and right image at X-axis. The distance between object and lens can be obtained from the disparity (such as ΔP) to obtain the goal of precise distance measurement. The images taken by the conventional synchronous binocular cameras system are formed on a same horizontal line, no matter the object and the synchronous binocular cameras system are under static state, or the object and the synchronous binocular cameras system are under dynamic state. It is to say that the pixels of the object will drop on the same Y-axis of left image and right image. When the pixels of the object drop on the same Y-axis of left and right images, it will be beneficial for searching and comparing step. It not only can raise the processing speed, but also can increase the success rate of searching. 
         [0005]    However, the synchronous binocular cameras system needs special extra hardware circuit and design, in order to trigger the exposure control of binocular cameras at the same time. It not only will increase the volume of camera, but also will raise the cost of cameras. Furthermore, the corresponding image retrieving device is required for the image retrieval at host computer end, which will increase the volume of equipment. 
         [0006]    In addition, when the asynchronous binocular cameras system is used for vehicle detection, a difference value on the Y-axis of left image and right image (such as ΔY) will be generated due to the difference of exposure time and the vibration of vehicle body. It will cause the difficulty of comparison. Moreover, the respective exposure of asynchronous binocular cameras will cause different illumination and gray level between the left image and right image, due to the difference of shutter speed and aperture size during the exposure time. 
         [0007]    As for the acquisition of disparity ΔP, the left image and right image are searched totally to find out the difference at X-axis conventionally. Assume Sw denotes the width of searching scope and the left image and right image of synchronous binocular cameras lie at the same level, then the searching scope will be Sw×1. As for the asynchronous binocular cameras system, ΔY has to be considered, thus the searching scope will be Sw×2×ΔY. 
         [0008]    Thus, in the conventional synchronous photography technique, there are drawbacks of too large volume and high cost, and the asynchronous photography technique is not completely suitable for current vehicle detection technique. In order to meet the demand of vehicle detection, the relevant photography techniques have to be developed to save the cost of manpower and time etc., and the image of vehicle can be detected correctly. 
       SUMMARY OF THE INVENTION 
       [0009]    The invention relates to a binocular and asynchronous photography automobile-detecting apparatus. Two low-cost and asynchronous cameras are installed at the same horizontal line on the rear-view mirror of vehicle. After the images are processed, the sound is used as the medium to transmit the distance information of the forward vehicles, so that the driver can make appropriate judgment. 
         [0010]    The invention relates to a binocular and asynchronous photography automobile-detecting apparatus. It can overcome the difference illumination of the left image and right image effectively, in order to raise the distinguishing and detecting rate. 
         [0011]    The invention relates to a binocular and asynchronous photography automobile-detecting apparatus. Under the condition without losing the precision, the problem of difference ΔY can be solved, and the searching scope Sw×2×ΔY can be reduced to obtain the disparityΔP quickly and precisely. 
         [0012]    The invention relates to a binocular and asynchronous photography automobile-detecting apparatus. The apparatus comprises a stereo image acquisition unit, an image pre-process unit, a searching object unit, a disparity matching unit, and an output unit. 
         [0013]    After the images acquired by the cameras of the invention are stored into a computer, the images are pre-processed, and the edge information is used to locate the forward vehicle position, and then the disparity is obtained from a fast comparison search algorithm by the stereovision methodology. According to the disparity, the relative distance between the forward vehicle and the camera can be calculated. 
         [0014]    The invention relates to a detection method for a binocular and asynchronous photography automobile-detecting apparatus. The above-mentioned vehicle coordinate information can be used for the edge accumulation detection, in order to obtain the vehicle coordinate information. And then the disparity of previous image can be used as the reference for the starting point of search. 
         [0015]    The invention can overcome the difference of asynchronous exposure between two cameras, which also can overcome the distance measurement of dynamic stereovision, wherein the object and camera are kept at moving state. 
         [0016]    The invention can effectively overcome the asynchronous exposure problem from dual cameras, reduce hardware construction cost of stereovision, accelerate the computing speed, and gain accurate real-time correct detection result in the practice. 
         [0017]    The advantage and spirit of the invention can be understood further by the following detail description of invention and attached figures. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0018]    The foregoing aspects and many of the attendant advantages of this invention will become more readily appreciated as the same becomes better understood by reference to the following detailed description, when taken in conjunction with the accompanying drawings, wherein: 
           [0019]      FIG. 1  schematically shows the apparatus for a preferred embodiment of the invention. 
           [0020]      FIG. 2  schematically shows the image pre-process unit of the invention. 
           [0021]      FIG. 3  schematically shows the searching vehicle unit of the invention. 
           [0022]      FIG. 4  schematically shows the disparity matching unit of the invention. 
           [0023]      FIG. 5  schematically shows the searching process of the invention. 
           [0024]      FIG. 6  schematically shows the side view for the apparatus of the invention. 
           [0025]      FIG. 7  schematically shows the setup for the apparatus of the invention in the vehicle. 
       
    
    
     DESCRIPTION OF THE PREFERRED EMBODIMENT 
       [0026]    The invention provides an asynchronous photography automobile-detecting apparatus and a detection method. As shown in  FIG. 1 , the asynchronous photography automobile-detecting apparatus of the invention comprises a stereo image acquisition unit  11 , an image pre-process unit  12 , a object searching (vehicle) unit  13 , a disparity matching unit  14 , and an output unit  15 . The details of each unit are described as follows. 
         [0027]    The function of stereo image acquisition unit  11  shown in  FIG. 1  is to capture the image into the memory of apparatus through dual cameras or multiple cameras. 
         [0028]    As shown in  FIG. 2 , the image pre-process unit  12  is used to pre-process the right image, in order to facilitate the acquisition of vehicle position. The function includes low resolution image process  121 , edge detection operation  122 , and object connection  123 . 
         [0029]    As shown in  FIG. 2 , the low resolution image process  121  will scan the horizontal and vertical directions of original frame and discard the extreme points to obtain the downsized image of original frame. Because the information of natural image owns good continuity, thus reducing the resolution of image not only can reduce the amount of information, but also can keep the most characteristics of the original image. Thus, the required operation amount of computation can be reduced greatly. 
         [0030]    As shown in  FIG. 2 , object connection  123  can detect the contour of vehicle roof, top and bottom edges of trunk, and vehicle chassis through the simple edge detection for a clearer vehicle image. As for the forward vehicle, because the outer shape of vehicle is difficult to be changed by the influence of shadow and color, the horizontal features of vehicle contour will be very distinct. Thus the Sobel operator can be used to obtain the horizontal edge features. 
         [0031]    As shown in  FIG. 2 , the object connection  123  can use the 8 connection way to connect the dual-valued image generated by the above-mentioned step, in order to output the coordinate values and image pixels of object at the left, right, up, and bottom sides of horizontal and vertical line segments. The 8 connection way is used to determine if two points belong to an object. Firstly judge whether two black points are adjacent, that is, whether two black points lie at the upper left, up, upper right, left, right, lower left, bottom, and lower right position. If two black points appear at any position of these 8 positions, it means any two dual-valued black points are connected each other. They can be regarded as the same object, which is so-called 8 connection way. 
         [0032]    As shown in  FIG. 3 , the function of searching object (vehicle) unit  13  is to search the possible region of vehicle through the vehicle features in a single right image. It includes the searching seed line segment  131  and the region of interest (ROI) construction  132 . 
         [0033]    As shown in  FIG. 3 , the searching seed line segment  131  can be used to facilitate the judgment of vehicle features, when the horizontal features of vehicle are continuous and distinct after the dual-valued image is generated by the pre-process. In this step, the horizontal line segments close to the vehicle features are found sequentially as the ROI of obstacle in the object connection matrices. The closest forward obstacle owns the highest threat level. Thus, the near-to-far method is used to find the obstacle from the bottom to the top row of input image. 
         [0034]    As shown in  FIG. 3 , the region of interest construction  132  uses the symmetry of the horizontal edge line segment to define the two-dimensional candidate region from the bottom to the top row of input image. The horizontal and vertical edge projection is carried out to obtain the highest and the second highest position, in order to cut the coordinate information correctly. 
         [0035]    As shown in  FIG. 4 , the disparity matching unit  14  carries out the full matching for the pixels of the right image and left image to obtain the disparity value. The invention can carry out the modification of full matching  141  to remove the illumination difference between dual images. It also has the function of multi-resolution of full matching  142 , adaptive matching area  143 , and object tracking  144  to overcome the ΔY problem generated by asynchronous cameras, and raise the matching accuracy and the computing speed. 
         [0036]    As shown in  FIG. 4 , the modification of full matching  141  owns the binocular independent exposure. Due to the difference of respect shutter and aperture, the illumination of binocular images will be different in the variant scene. In order to prevent the influence of variation on the accuracy of matching, the modification of full matching is used, and its operation equation is shown as follows: Standard full matching method: 
         [0000]    
       
         
           
             
               SAD 
                
               
                 ( 
                 
                   i 
                   , 
                   j 
                 
                 ) 
               
             
             = 
             
               
                 ∑ 
                 
                   k 
                   = 
                   
                     
                       - 
                       N 
                     
                     / 
                     2 
                   
                 
                 
                   N 
                   / 
                   2 
                 
               
                
               
                 
                   ∑ 
                   
                     l 
                     = 
                     
                       
                         - 
                         N 
                       
                       / 
                       2 
                     
                   
                   
                     N 
                     / 
                     2 
                   
                 
                  
                 
                   ABS 
                    
                   
                     ( 
                     
                       
                         ( 
                         
                           RP 
                            
                           
                             ( 
                             
                               
                                 x 
                                 + 
                                 k 
                               
                               , 
                               
                                 y 
                                 + 
                                 l 
                               
                             
                             ) 
                           
                         
                         ) 
                       
                       - 
                       
                         ( 
                         
                           LP 
                            
                           
                             ( 
                             
                               
                                 i 
                                 + 
                                 x 
                                 + 
                                 k 
                               
                               , 
                               
                                 j 
                                 + 
                                 y 
                                 + 
                                 l 
                               
                             
                             ) 
                           
                         
                         ) 
                       
                     
                     ) 
                   
                 
               
             
           
         
       
     
         [0037]    N: the width of the mask 
         [0038]    RP(x+k, y+l): the pixel of (x, y) in the right image 
         [0039]    LP(i+x+k, j+y+l): the pixel corresponding to (x, y) in the left image 
         [0000]    Modification of full matching method: 
         [0000]    
       
         
           
             
               SAD 
                
               
                 ( 
                 
                   i 
                   , 
                   j 
                 
                 ) 
               
             
             = 
             
               
                 ∑ 
                 
                   k 
                   = 
                   
                     
                       - 
                       N 
                     
                     / 
                     2 
                   
                 
                 
                   N 
                   / 
                   2 
                 
               
                
               
                 
                   ∑ 
                   
                     l 
                     = 
                     
                       
                         - 
                         N 
                       
                       / 
                       2 
                     
                   
                   
                     N 
                     / 
                     2 
                   
                 
                  
                 
                   ABS 
                    
                   
                     ( 
                     
                       
                         ( 
                         
                           
                             RP 
                              
                             
                               ( 
                               
                                 
                                   x 
                                   + 
                                   k 
                                 
                                 , 
                                 
                                   y 
                                   + 
                                   l 
                                 
                               
                               ) 
                             
                           
                           - 
                           RM 
                         
                         ) 
                       
                       - 
                       
                         ( 
                         
                           
                             LP 
                              
                             
                               ( 
                               
                                 
                                   
                                     
                                       
                                         i 
                                         + 
                                         x 
                                         + 
                                         k 
                                       
                                       , 
                                     
                                   
                                 
                                 
                                   
                                     
                                       j 
                                       + 
                                       y 
                                       + 
                                       l 
                                     
                                   
                                 
                               
                               ) 
                             
                           
                           - 
                           LM 
                         
                         ) 
                       
                     
                     ) 
                   
                 
               
             
           
         
       
       
         
           
             M 
             = 
             
               
                 1 
                 
                   N 
                   ⋆ 
                   N 
                 
               
                
               
                 
                   ∑ 
                   
                     i 
                     = 
                     
                       
                         - 
                         N 
                       
                       / 
                       2 
                     
                   
                   
                     N 
                     / 
                     2 
                   
                 
                  
                 
                   
                     ∑ 
                     
                       j 
                       = 
                       
                         
                           - 
                           N 
                         
                         / 
                         2 
                       
                     
                     
                       N 
                       / 
                       2 
                     
                   
                    
                   
                     P 
                      
                     
                       ( 
                       
                         i 
                         , 
                         j 
                       
                       ) 
                     
                   
                 
               
             
           
         
       
     
         [0040]    (The average gray value of pixel in the region) 
         [0041]    RM: the average gray value of the matching mask in the right image 
         [0042]    LM: the average gray value of the matching mask in the left image 
         [0043]    As for the multi-resolution of full matching  142  shown in  FIG. 4 , the modification of full matching is carried out first under ¼ low-resolution image. After the best pixel coordinate (x, y) is found, go back to the original image to carry out the modification of full matching from (2x−2, 2y−2) to (2x+2, 2y+2) again. Sh is the height of search area, and Sw is the width of search area. The searching loop is (N×N×Sh×Sw). When the multi-resolution of full matching is used, the searching is decreased to (N×N×Sh/2×Sw/2)+(N×N×4×4). 
         [0044]    As for the adaptive matching area  143  shown in  FIG. 4 , the full matching has to increase the searching height, due to ΔY problem of asynchronous cameras. However, if the searching height is increased, the computation of central processing unit (CPU) will also be increased, thus the adaptive matching method is proposed to conquer the problem. In the ROI generated by “ROI construction” of the invention, the modification of full matching is carried out for the sample points sequentially, in order to calculate the disparity value. Due to the disparity value of left image and right image is close for Y coordinate of every pixel in the same frame, the hightest one in top 20% will be used as the reference of the following sample point. Thus the searching height can be reduced and the computation efficiency can be increased effectively without influencing the accuracy rate. 
         [0045]    As for the object tracking  144  shown in  FIG. 4 , the frames own the continuity, which means the object will be appeared again in the next frame. When current frame possesses the object, the object tracking process is carried out to raise the computation efficiency and the accuracy rate. 
         [0046]    In addition, when the vehicle coordinate information of previous frame is used, the range can be increased to carry out the horizontal and vertical edge projection to obtain the highest and the second highest position, in order to cut the coordinate information quickly and correctly. 
         [0047]    The disparity value of previous frame can be used as the reference for the starting point of searching in the frame. It means the following computation is carried out in the modification of full matching: 
         [0000]    
       
         
           
             
               SAD 
                
               
                 ( 
                 
                   i 
                   , 
                   j 
                 
                 ) 
               
             
             = 
             
               
                 ∑ 
                 
                   k 
                   = 
                   
                     
                       - 
                       N 
                     
                     / 
                     2 
                   
                 
                 
                   N 
                   / 
                   2 
                 
               
                
               
                 
                   ∑ 
                   
                     l 
                     = 
                     
                       
                         - 
                         N 
                       
                       / 
                       2 
                     
                   
                   
                     N 
                     / 
                     2 
                   
                 
                  
                 
                   ABS 
                    
                   
                     ( 
                     
                       
                         ( 
                         
                           
                             RP 
                              
                             
                               ( 
                               
                                 
                                   x 
                                   + 
                                   k 
                                 
                                 , 
                                 
                                   y 
                                   + 
                                   l 
                                 
                               
                               ) 
                             
                           
                           - 
                           RM 
                         
                         ) 
                       
                       - 
                       
                         ( 
                         
                           
                             LP 
                              
                             
                               ( 
                               
                                 
                                   
                                     
                                       
                                         i 
                                         + 
                                         x 
                                         + 
                                         k 
                                       
                                       , 
                                     
                                   
                                 
                                 
                                   
                                     
                                       j 
                                       + 
                                       y 
                                       + 
                                       l 
                                     
                                   
                                 
                               
                               ) 
                             
                           
                           - 
                           LM 
                         
                         ) 
                       
                     
                     ) 
                   
                 
               
             
           
         
       
       
         
           
             
               i 
               = 
               0 
             
             , 
             1 
             , 
             2 
             , 
             … 
              
             
                 
             
             , 
             
               Sh 
               - 
               1 
             
           
         
       
       
         
           
             
               j 
               = 
               0 
             
             , 
             1 
             , 
             2 
             , 
             … 
              
             
                 
             
             , 
             
               Sw 
               - 
               1 
             
           
         
       
     
         [0048]    The original variable j is from 0 to Sw−1. It will be renewed as a new range from the 2/3 to 4/3 of the disparity of the current frame in the next frame. 
         [0049]    The original variable j is from 0 to Sw−1. It will be renewed as a new range from the 2/3 to 4/3 of the disparity of the current frame in the next frame. 
         [0050]    As for the output unit shown in  FIG. 1 , the audio-visual output equipment can be used to remind or warn the drivers actively, in order take the required countermeasures. 
         [0051]      FIG. 5  shows a detection method for an asynchronous photography automobile-detecting apparatus of the invention. The “searching object” and “matching” modules are used as two major and basic computation ways. The “searching object” module is to find out the vehicle coordinate information in the right image mainly. The “matching” module uses the vehicle position obtained from the searching object to carry out the modification of full matching in the left image, in order to obtain the distance information. As for the detection method for an asynchronous photography automobile-detecting apparatus of the invention shown in  FIG. 5 , the details are described as follows. Firstly, the stereo image acquisition unit  11  grasps the left image  511  and the right image  512 . The image pre-process unit  12  and the searching object (vehicle) unit  13  are employed to carry out the searching object  520 . It mainly carry out the searching for the symmetry of horizontal edge line segment  521  and the horizontal and vertical edge projection  522 . Among these, the searching of seed line segment is completed by employing the symmetry of horizontal edge line segment, and the construction of ROI is completed by employing the horizontal and vertical edge projection. As shown in  FIG. 5 , the disparity matching unit  14  is employed to match the left image  511  and the right image  512 . The matching process includes the matching of gray difference  531  and the matching of vertical coordinate variation  532 . Among these, the gray difference  531  can be employed to carry out the modification of full matching  141 , and the vertical coordinate variation  532  can be employed to carry out the optimization in frame  533  and the optimization between frame  534 . In the optimization in frame  533 , the multi-resolution of full matching  142  and the adaptive matching area  143  can be achieved. Finally, the object tracking  144  is carried out in the optimization between frame  534 . 
         [0052]    As the method for detecting the vehicle image of the invention shown in  FIG. 5 , after the left image  511  and the right image  512  are inputted, the right image  512  is employed to carry out the searching object  520 . It mainly carry out the searching for the symmetry of horizontal edge line segment  521  and the horizontal and vertical edge projection  522 . Among these, the searching of seed line segment is completed by employing the symmetry of horizontal edge line segment. The matching of the left image  511  is carried out in the matching  530 . The matching process includes the matching of gray difference  531  and the matching of vertical coordinate variation  532 . Among these, the gray difference  531  can be employed to carry out the modification of full matching  141 , and the vertical coordinate variation  532  can be employed to carry out the optimization in frame  533  and the optimization between frame  534 . In the optimization in frame  533 , the multi-resolution of full matching  142  and the adaptive matching area  143  can be achieved. Finally, the object tracking  144  is carried out the optimization between frame  534 . 
         [0053]    As shown in  FIG. 6 , the asynchronous binocular image detecting apparatus  60  of the invention includes a body  61  and a lens set  62 . 
         [0054]    As shown in  FIG. 7 , the asynchronous binocular image detecting apparatus  60  of the invention is installed in front of the rear-view mirror  70  in the vehicle. The lens set  62  should be faced ahead, in order to camera the forward road. Its installation should satisfy the adjustment and control of various angles, including the included angle, depression angle and pitch angle. In addition, the device must be small and the device just helps follow-up quantity to produce the good and cheap result. The asynchronous image of the twin-lens of the invention is detected. The asynchronous binocular image detecting apparatus of the invention should have small volume, good effect, and low cost for mass production. 
         [0055]    The invention can overcome the difference of asynchronous exposure between two cameras, which also can overcome the distance measurement of static and dynamic stereovision. The invention can effectively overcome the asynchronous exposure problem from dual cameras, reduce hardware construction cost of stereovision, accelerate the computing speed, and gain accurate real-time correct detection result in the practice. 
         [0056]    It is understood that various other modifications will be apparent to and can be readily made by those skilled in the art without departing from the scope and spirit of this invention. Accordingly, it is not intended that the scope of the claims appended hereto be limited to the description as set forth herein, but rather that the claims be construed as encompassing all the features of patentable novelty that reside in the present invention, including all features that would be treated as equivalents thereof by those skilled in the art to which this invention pertains.

Technology Classification (CPC): 6