Abstract:
A method and apparatus for detecting vehicle headlights and a region-of-interest (ROI) segmenting method and apparatus are disclosed. The ROI segmenting method includes: performing an edge extracting operation on a captured image to obtain edges of the captured image; selecting edges meeting predetermined criteria from the obtained edges, the predetermined criteria being the similarity between the region surrounded by the selected edges and the pattern formed by a vehicle headlight in physical reality at a position of the selected edges; determining the region surrounded by the selected edges within the captured image as a vehicle headlight pattern; and segmenting the ROI which potentially includes the vehicle pattern from the captured image based on the determined vehicle headlight pattern. With such a method and apparatus, the ROI of a vehicle may be acquired from the image without using the vehicle&#39;s bottom shadow.

Description:
RELATED APPLICATION 
     The present application claims priority to Chinese Patent Application Number 200710301433.0, filed Dec. 27, 2007, the entirety of which is hereby incorporated by reference. 
     FIELD OF THE INVENTION 
     This invention relates to a computer visualization-based method and apparatus for detecting vehicle headlights, and also relates to a region-of-interest (ROI) segmenting method and apparatus. 
     RELATED ART 
     In a computer-based, monocular, visualization-based motorcycle detecting system, it is common to segment a ROI potentially including a motorcycle pattern from an image captured by a camera device such as a camcorder based on feature information such as the vehicle&#39;s bottom shadow, vertical edge, and horizontal edge. 
     However, when the motorcycle is close to a target vehicle on which the camera is mounted or the motorcycle is travelling in a tunnel or a shadow caused by buildings, because the feature information of the vehicle&#39;s bottom shadow cannot be detected, it is difficult to segment the ROI potentially including the motorcycle pattern, and the detection of the motorcycle may be missed. 
     SUMMARY 
     In view of the above problems, an object of one embodiment of the invention is to provide a ROI segmenting method and apparatus which can segment the RIO potentially including the vehicle pattern from an image without using the feature information of vehicle&#39;s bottom shadow. 
     Another object is to provide a vehicle headlight detecting method and apparatus for detecting a vehicle headlight pattern in the image. 
     To accomplish the above objects, according to one aspect of embodiments of the invention, a method for segmenting a ROI from an image comprises: 
     performing an edge extracting operation on a captured image to obtain edges of the captured image; 
     selecting edges meeting predetermined criteria from the obtained edges, the predetermined criteria being the similarity between the region surrounded by the selected edges and the pattern formed by a vehicle headlight in physical reality at a position of the selected edges; 
     determining the region surrounded by the selected edges within the captured image as a vehicle headlight pattern; and 
     segmenting the ROI potentially including the vehicle pattern from the captured image based on the determined vehicle headlight pattern. 
     To accomplish the above objects, according to another aspect of embodiments of the invention, a method for detecting a vehicle headlight pattern from an image comprises: 
     performing an edge extracting operation on a captured image to obtain edges of the captured image; 
     selecting edges meeting predetermined criteria from the obtained edges, the predetermined criteria being the similarity between the region surrounded by the selected edges and the pattern formed by the vehicle headlight in physical reality at a position of the selected edges; and 
     determining the region surrounded by the selected edges within the captured image as a vehicle pattern. 
     To accomplish the above objects, according to still another aspect of embodiments of the invention, an apparatus for segmenting a ROI from an image comprises: 
     an edge extracting module for performing an edge extracting operation on a captured image to obtain edges of the captured image; 
     a selecting module for selecting edges meeting predetermined criteria from the obtained edges, the predetermined criteria being the similarity between the region surrounded by the selected edges and the pattern formed by a vehicle headlight in physical reality at a position of the selected edges; 
     a determining module for determining the region surrounded by the selected edges within the captured image as a vehicle headlight pattern; and 
     a segmenting module for segmenting the ROI potentially including the vehicle pattern from the captured image based on the determined vehicle headlight pattern. 
     To accomplish the above objects, according to another aspect of embodiments of the invention, an apparatus for detecting a vehicle headlight pattern from an image comprises: 
     an edge extracting module for performing an edge extracting operation on a captured image to obtain edges of the captured image; 
     a selecting module for selecting edges meeting predetermined criteria from the obtained edges, the predetermined criteria being the similarity between the region surrounded by the selected edges and the pattern formed by the vehicle headlight in physical reality at a position of the selected edges; and 
     a determining module for determining the region surrounded by the selected edges within the captured image as a vehicle headlight pattern. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  shows a schematic diagram of a perspective principle; 
         FIG. 2  is a flowchart of a method for segmenting a ROI from an image in accordance with an embodiment of the invention; 
         FIG. 3   a  shows an example of a captured image in accordance with an embodiment of the invention; 
         FIG. 3   b  shows a schematic diagram of a vehicle headlight candidate region in accordance with an embodiment of the invention; 
         FIG. 4  shows a schematic diagram of extracted edges in accordance with an embodiment of the invention; 
         FIG. 5   a  shows a schematic diagram of an image after a filling operation in accordance with an embodiment of the invention; 
         FIG. 5   b  shows a schematic diagram of an image after an eroding operation in accordance with an embodiment of the invention; 
         FIG. 5   c  shows a schematic diagram of an image after a dilating operation in accordance with an embodiment of the invention; 
         FIG. 6   a  shows a schematic diagram of a left and right sides candidate region of the ROI in accordance with an embodiment of the invention; 
         FIG. 6   b  shows a schematic diagram of edges extracted from a left side candidate region and a right side candidate region of the ROI in accordance with an embodiment of the invention; 
         FIG. 6   c  shows a histogram indicating the number of the edge points contained in each column within the edges of  FIG. 6   b;    
         FIG. 7   a  shows a schematic diagram of a bottom side candidate region of the ROI in accordance with an embodiment of the invention; 
         FIG. 7   b  shows a schematic diagram of edges extracted from the bottom side candidate region of the ROI in accordance with an embodiment of the invention; 
         FIG. 7   c  shows a histogram indicating the number of the edge points contained in each row within the edges of  FIG. 7   b ; and 
         FIG. 7   d  shows a schematic diagram of the bottom side of the ROI in accordance with an embodiment of the invention. 
     
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
     In accordance with one aspect of the invention, a headlight pattern of a vehicle is extracted from a captured image, then the ROI (Region of Interest) potentially including the vehicle pattern is segmented from the captured image based on the extracted headlight pattern. 
     First, a perspective principle to be adopted in some embodiments of the invention will be described. Referring to  FIG. 1 , the relationship between the actual height DE (h r ) in physical reality and the pixels&#39; height AB (h p ) formed within the captured image is as follows: 
     
       
         
           
             
               
                 
                   
                     
                       AB 
                       × 
                       
                         d 
                         y 
                       
                     
                     DE 
                   
                   = 
                   
                     OC 
                     d 
                   
                 
               
               
                 
                   ( 
                   1 
                   ) 
                 
               
             
           
         
       
     
     An equation (2) is obtained by transforming the equation (1): 
                     AB   =           OC   ×   DE     d     ×     1     d   y         =         f   ×     h   r       d     ×     1     d   y             ,     
     ⁢     i   .   e   .     ,       h   p     =         f   ×     h   r       d     ×     1     d   y                   (   2   )               
where DE represents the height from the bottom edge of the vehicle headlight to the ground; AB represents the pixels&#39; height from the bottom edge of the vehicle headlight pattern in the captured image to the bottom side of the captured image; OC represents the focal length f of the camera; d y  represents the tangential distortion of the camera; d represents the distance between a target vehicle and the camera in physical reality, which corresponds to a depth d p  of the bottom side of the vehicle in the image (i.e., the distance between the bottom side of the vehicle and the bottom side of the image). The relationship between d and d p  is as follows:
 
                   d   =       H   -     d   y           d   p     -   Vanishingline               (   3   )               
where H represents the height from the camera to the ground, and the Vanishingline represents the pixels&#39; distance from the road vanishing line to the bottom side of the image.
 
     Hereinafter, taking a motorcycle as an example of the vehicle, a ROI segmenting method according to an embodiment of the invention will be described in conjunction with  FIG. 2 . 
     Referring to  FIG. 2 , at step S 10 , a motorcycle headlight candidate region, i.e., an image region potentially including the motorcycle headlight pattern, is determined from the image captured by a camera such as the camcorder. As the motorcycle headlight normally does not appear beyond the road vanishing line in the image when the road is flat (referring to  FIG. 3   a ), an image portion under the road vanishing line within the captured image may be selected as the motorcycle headlight candidate region (referring to  FIG. 3   b ). In consideration of the influence of the road gradient, in the present embodiment, the position of ΔT=10 pixels extended upwards from the road vanishing line within the captured image is determined to be the top edge of the motorcycle headlight candidate region. 
     Then, at step S 20 , an edge extracting operation is performed on the determined motorcycle headlight candidate region by a known method such as Canny, Sobel or SUSAN. Referring to  FIG. 4 , in the present embodiment, the SUSAN method is preferably adopted to directly extract the edges from the determined motorcycle headlight candidate region (i.e., based on the RGB color space of the determined motorcycle headlight candidate region). 
     Next, at step S 30 , interference edges, for example, edges without a closed contour and/or edges featuring a closed contour with its perimeter less than the corresponding perimeter within the captured image of the smallest vehicle headlight in physical reality, are removed from the extracted edges. In the present embodiment, some operations derived from the mathematical morphology are adopted to remove the edges without a closed contour and/or the edges featuring a closed contour with its perimeter less than the corresponding perimeter within the captured image of the smallest vehicle headlight in physical reality. Specifically, the edges featuring the closed contour within the captured image are filled (referring to  FIG. 5   a ), then an eroding operation is performed on the determined motorcycle headlight candidate region by adopting a template with its size smaller than the pixels&#39; size of the smallest motorcycle to be identified in the motorcycle headlight candidate region, a round template of 5×5 being adopted herein (referring to  FIG. 5   b ) whereby the edges without a closed contour and/or the edges featuring a closed contour with its perimeter less than the corresponding perimeter within the captured image of the smallest headlight in physical reality are removed. Then a dilating operation is performed on the determined motorcycle headlight candidate region by adopting the template (referring to  FIG. 5   c ). 
     At step S 40 , a region, enclosed by the edges with its perimeter within a range between a perimeter of the largest motorcycle headlight and a perimeter of the smallest motorcycle headlight and an aspect ratio of its boundary rectangle within a predetermined range, is selected as the motorcycle headlight pattern from the captured edges, as shown by the region  1  in  FIG. 5   c.    
     At step S 50 , a left extension side and a right extension side are obtained by moving the left side and the right side of the determined motorcycle headlight pattern respectively to the left and to the right by a width of W d , and a bottom extension side is obtained by moving the bottom side of the determined motorcycle headlight pattern downwards by a depth of H p , thereby obtaining a left and right sides candidate region R m  potentially including the ROI of the motorcycle (hereinafter referred as the motorcycle ROI), formed by the bottom side, the left extension side, the right extension side, and the bottom extension side of the motorcycle headlight pattern, as shown in  FIG. 6   a , wherein:
 
 W   d   =k ×(2× R )  H   p   =t ×(2× R ).
 
     R represents the radius of the motorcycle headlight; k and t represent the empirical constants determined based on the radius of the motorcycle headlight. The coordinates of the candidate region R m  are as follows: 
             {               R   ml     =     Cx   -   R   -     W   d                     R   mr     =     Cx   +   R   +     W   d                     R   mt     =     Cy   +   R                   R   mb     =     Cy   +   R   +       H   p     .                           
where (C x , C y ) represents the coordinates of the center of the motorcycle headlight, and R represents the radius of the motorcycle headlight.
 
     At step S 60 , along the vertical line through the center of the motorcycle headlight pattern, the candidate region R m  is segmented into two regions, i.e., the left region and the right region, respectively, as the left side candidate region and the right side candidate region of the motorcycle ROI. 
     At step S 70 , an edge extracting operation is performed on the left side candidate region and the right side candidate region of the motorcycle ROI with an edge extraction operator such as Canny or Sobel, so as to extract edges from the two candidate regions (i.e., the candidate region R m ) as shown in  FIG. 6   b.    
     At step S 80 , the number of the edge points contained in each column within the left and right side candidate regions is counted.  FIG. 6   c  shows a histogram indicating the number of the edge points contained in each column within the edge pattern of  FIG. 6   b , wherein the abscissa of the histogram represents the positions of each column within the captured image, while the ordinate of the histogram represents the number of the edge points contained in each column. 
     At step S 90 , the column containing the most edge points in the left side candidate region is determined to be the left side of the motorcycle ROI, while the column containing the most edge points in the right side candidate region is determined to be the right side of the motorcycle ROI. 
     If there is no edge extractable in either the left side candidate region or the right side candidate region, the column in one candidate region, being symmetrical with the side of the determined motorcycle ROI in the other candidate region with respect to the center of the motorcycle headlight, is selected as the side of the motorcycle ROI in the one candidate region. 
     If there is no edge extractable from both of the left side candidate region and the right side candidate region, the left side and the right side of the candidate region R m  are determined to be the left side and the right side of the motorcycle ROI respectively. 
     At step S 100 , a bottom side candidate region of the motorcycle ROI is determined. Specifically, the distances d p  between each row within the captured image and the bottom side of the captured image are computed. Then, the actual height h r  in physical reality, corresponding to the pixels&#39; height h p  from each row within the captured image to the bottom side of the motorcycle headlight pattern, is computed in accordance with the above-mentioned equations (2) and (3). Next, a first particular row, whose corresponding actual height h r  equals the maximum height h r     —     max  from the motorcycle headlight to the ground in physical reality, and a second particular row, whose corresponding actual height h r  equals the minimum height h r     —     min  from the motorcycle headlight to the ground in physical reality, are found. Finally, the image region formed by the first and second particular rows, and the left and right sides of the motorcycle ROI determined within the captured image is selected as the bottom side candidate region of the motorcycle ROI, as shown in  FIG. 7   a.    
     At step S 110 , an edge extracting operation is performed on the bottom side candidate region of the motorcycle ROI by an edge extraction operator such as Canny or Sobel, so as to extract edges from the bottom side candidate region, as shown in  FIG. 7   b.    
     At step S 120 , the number of the edge points contained in each row within the bottom side candidate region is counted.  FIG. 7   c  shows a histogram indicating the number of the edge points contained in each row within the bottom side candidate region, wherein the abscissa of the histogram represents the positions of each row in the bottom side candidate region, while the ordinate of the histogram represents the number of the edge points contained in each row. 
     At step S 130 , the row containing the most edge points in the bottom side candidate region is determined to be the bottom side of the motorcycle ROI.  FIG. 7D  shows an example of the bottom side of the motorcycle ROI. 
     At step S 140 , the top side of the motorcycle ROI is determined. Specifically, after the bottom side of the motorcycle ROI is obtained, the distance d between the camera and the motorcycle is computed in accordance with equation (3) based on the distance d p  from the bottom side of the captured image to the row where the obtained bottom side is located; then the pixels&#39; height d pm  of the motorcycle within the captured image is computed, where the distance from the motorcycle to the camera in physical reality is d: 
               d   pm     =         f   ×     h   motor       d     ×       1   d     .             
The height of the motorcycle in physical reality is represented by h motor , which equals 1.5 m in this embodiment.
 
     Finally, the row which is spaced apart from the bottom side of the motorcycle ROI by the pixels&#39; height d pm  is determined to be the top side of the motorcycle ROI. 
     Although the discussed embodiment presumes that the edge has a closed contour with its perimeter within a range between a perimeter of the largest motorcycle headlight and a perimeter of the smallest motorcycle headlight and an aspect ratio of its boundary rectangle within a predetermined range, so as to determine the similarity between the region surrounded by the edges and the pattern formed by the vehicle headlight in physical reality at the position of the edges, a person skilled in the art will understand that the invention is not limited hereto. In other embodiments of the invention, the similarity between the region surrounded by the edges and the pattern formed by the vehicle headlight in physical reality at the position of the edges may also be determined by judging whether the edges feature a closed contour and whether the difference between the region surrounded by the edges and the invariant moment, the rectangular degree, the circular degree or the fourier descriptor of the vehicle headlight pattern formed at the position of the edges is within a predetermined range. Here, the above-mentioned method of judging the pattern&#39;s similarity by adopting the invariant moment, the rectangular degree, the circular degree, the fourier descriptor or the like is known to a person skilled in the image processing field, and therefore the description of such is omitted. 
     Although the discussed embodiments presume that the image portion below the road vanishing line within the captured image is the vehicle headlight candidate region, a person in the art will understand that the invention is not limited hereto. In other embodiments of the invention, the entire captured image may be selected as the vehicle headlight candidate region. 
     Although a motorcycle is adopted as an example in the discussed embodiment, a person in the art will understand that the invention is not limited to detecting a motorcycle. In other embodiments of the invention, other vehicles besides a motorcycle may be detected. 
     The vehicle headlight detecting method and the ROI segmenting method of this invention may be implemented either in software controlling the operation of a processor or a combination of software and hardware. 
     While there has been illustrated and described what is at present contemplated to be preferred embodiments of the present invention, it will be understood by those skilled in the art that various changes and modifications may be made, and equivalents may be substituted for elements thereof without departing from the true scope of the invention. In addition, many modifications may be made to adapt a particular situation to the teachings of the invention without departing from the central scope thereof. Therefore, it is intended that this invention not be limited to the particular embodiments disclosed, but that the invention will include all embodiments falling within the scope of the appended claims.