Abstract:
A method for adjusting image acquisition parameters to optimize object extraction is disclosed, which is applied to an object characterized by forming a specific cluster in a color coordinate space after performing a coordinate projection, and thus the specific cluster contributes to a specific color model, such as a human skin color model. This method first locates a target object within a search window in a selected image. Then applies the specific color model to obtain the image acquisition parameter(s) according to the color distribution and features of the target object. Therefore, the image is transformed according to the adjusted image acquisition parameter(s). Consequently, a complete and clear target object can be extracted from the transformed image by applying the specific color model, and the follow-up images having the same image acquisition conditions with the aforesaid image can also be transformed according to the same image acquisition parameter(s).

Description:
BACKGROUND OF THE INVENTION  
       [0001]     1. Field of the Invention  
         [0002]     The present invention relates to a method for adjusting image acquisition parameters to optimize object extraction, which is suitable for different color object extraction and, more particularly, to an object characterized by forming a specific cluster in a color coordinate space after performing a coordinate projection.  
         [0003]     2. Description of the Related Art  
         [0004]     In a prior art technique, there are two primary image detecting methods for the human face. One is the detection of important features on the face according to the geometric positions of the eyes, nose and lips; these individual features are combined to determine a position of the face. The other method collects a plurality of facial templates, and obtains a standard facial template by way of training steps; a template matching method is then utilized to find the position of the face in the image.  
         [0005]     However, in the first method, a predetermined facial model, or its geometric position equivalent, is used to detect the face; if the face in the image is rotated, it is very difficult to detect the correct facial position. In the second method, facial data for various different rotated angles can be provided for training a database, but the template matching method is performed by brute force on every pixel in the image, and this requires an enormous amount of time; in addition, when the image includes two differently sized human faces, the method needs to perform several upscaling or downscaling processes to find the correct positions. As shown in  FIG. 1 , image X includes two differently sizes human faces F 1 , F 2 . A fixed size search window Y is used for scanning every pixel in the image X and sending a selected image area to a human face verification means to determine whether the image area is a face. Since the search window Y has a fixed size, any human face that is larger or smaller than the search window Y may be ignored. Therefore, referring to  FIG. 2 , several upscaling or downscaling processes must be performed to obtain upscale or downscale images X 1 , X 2 , X 3 , and then the above-mentioned scanning and verification processes are performed to find all of the differently sized human faces F 1 , F 2 . Accordingly, the second method will consume enormous amounts of processing time.  
         [0006]     One other prior art technique utilizes skin color to reduce the range of the template matching method to accelerate the human face detecting process. Please refer to  FIG. 3A . In this prior art technique, large numbers of facial images are first manually collected; the pixels in the skin color regions, as shown in  FIG. 3B , are manually extracted, and the color information of these skin color pixels are projected onto a special coordinate system to establish a skin color model for further image processing, such as facial position or feature extraction. Since an image is composed of a plurality of pixels, the color values of the pixels are determined by the light impinging upon the skin and then reflecting into a light sensing device, such as a camera. A skin color model can be established by collecting these color values of the pixels and used as a reference for determining whether further pixels are of a skin color. This prior art technique usually uses the following formula to project the RGB color information of the skin color pixels into an equivalent YCbCr color coordinate: 
 
 Y =0.2989 ×R+ 0.5866 ×G+ 0.1145 ×B 
 
 Cb =−0.1687 ×R− 0.3312 ×G+ 0.5000 ×B, 
 
 Cr =0.5000 ×R− 0.4183 ×G− 0.0816 ×B  
 
         [0007]     A tight space collection (skin color model Z 1 ), as shown in  FIG. 3C , can be represented by the following math formula:  
         if   (     Y   &gt;   128     )     ⁢     {                 θ   1     =       -   2     +       256   -   Y     16         ;                   θ   2     =     20   -       256   -   Y     16         ;                   θ   3     =   6     ;       θ   4     =       -   8     :               ⁢     
     ⁢     if   ⁡     (     Y   ≤   128     )       ⁢     {                     θ   1     =   6     ;       θ   2     =   12     ;                   θ   2     =     2   +     Y   32         ;       θ   4     =       -   16     +     Y   16         ;           ⁢     
     ⁢     C   r       ≥       -   2     ⁢     (       C   b     +   24     )         ;           ⁢       C   r     ≥     -     (       C   b     +   17     )         ;     
     ⁢       C   r     ≥       -   4     ⁢     (       C   b     +   32     )         ;           ⁢       C   r     ≥     2.5   ⁢     (       C   b     +     θ   1       )         ;     
     ⁢       C   r     ≥     θ   3       ;           ⁢       C   r     ≥     0.5   ⁢     (       θ   4     -     C   b       )         ;     
     ⁢       C   r     ≤       220   -     C   b       6       ;           ⁢       C   r     ≤       4   3     ⁢     (       θ   2     -     C   b       )         ;               
 
         [0008]     Accordingly, any pixel projected into a YCbCr coordinate Z is checked to see if it matches the above mathematical formula to determined whether the pixel is in the skin color model and whether it is a skin color pixel; in this manner it is possible to outline a skin color area of this image.  
         [0009]     By using the skin color information, it is possible to reduce multiple exhaustive search times, and so the search window needs only to verify the skin color to find a face to accelerate processing efficiency. However, under different lighting conditions and different camera settings, the extracted images may have certain color differences, and the actual skin color value of these images may not match the skin color model based on other conditions or parameters. Therefore, a collection formed by skin color values in a color space will change according to the conditions and parameter settings. Consequently, when the conditions have errors, the determined skin color may be incorrect, which causes broken skin color areas, or added noise in background.  
         [0010]     In prior art skin color techniques, such as U.S. Pat. No. 4,987,482, No. 6,169,536 and No. 6,249,317, image signals are all assumed to mainly include people as objects, and hardware parameters are controlled, or compensation signals are added, by detecting skin color components in the image signal to obtain a better image quality. The skin color detecting techniques utilized by these three patents all separate the image signal into brightness (Y) and color components (R-Y and B-Y), and then use predetermined phase, area or special axis to distinguish skin color and non-skin color signals. However, even though the above-mentioned patents predefine the determination rules for skin color, different cameras, different lighting conditions, and different software and hardware parameters may still affect the image. When the skin color detecting process fails, subsequent processes, such as the adding of compensation signals or the controlling of hardware parameters, will fail as well.  
         [0011]     Therefore, it is desirable to provide a method for adjusting image acquisition parameters to optimize object extraction to mitigate and/or obviate the aforementioned problems.  
       SUMMARY OF THE INVENTION  
       [0012]     A main objective of the present invention is to provide a method for adjusting image acquisition parameters to optimize object extraction, which can automatically adjust image acquisition parameter(s) of an image device based on a characteristic of forming a specific cluster in a color coordinate space after performing a coordinate projection, so that the image device can extract a complete object for different cameras, different lighting conditions, and different software and hardware parameters without requiring predetermined rules.  
         [0013]     Another objective of the present invention is to provide a method for adjusting image acquisition parameters to optimize object extraction, which obtains a ratio for adjusting an image according to an ideal color value to obtain a complete object from the adjusted image by utilizing a skin color model.  
         [0014]     Another objective of the present invention is to provide a method for adjusting image acquisition parameters to optimize object extraction, which selects an image acquisition parameter set giving the object a maximum color difference, and then automatically adjusting the parameters of the image extraction device accordingly.  
         [0015]     In order to achieve the above-mentioned objective, the method for adjusting image acquisition parameters to optimize object extraction of the present invention includes: extracting an image, the image comprising at least one target object characterized by forming a specific cluster in a color coordinate space after performing a coordinate projection, and thus the specific cluster contributes to a specific color model; locating the target object in the image by a human face detection method or a manual selecting method; applying the specific color model to obtain an image acquisition parameter according to color distribution features of the target object; transforming the image according to the adjusted image acquisition parameter so the color of the transformed target object matches the specific color model; and extracting the target object from the transformed image by applying the specific color model.  
         [0016]     Further, the present invention can be applied in an offline status, image acquisition parameters are calculated based upon a von Kries color adaptation model, so the transformed object color corresponding to the target object matches the specific color model better. Furthermore, the present invention can be applied in an online status, optimized image acquisition parameters can be obtained for the image capturing device, so color distribution of the target object is in the specific color model better. The above-mentioned color can be human facial skin color or any other color capable of forming a specific cluster in a color coordinate space.  
         [0017]     Other objects, advantages, and novel features of the invention will become apparent from the following detailed description, when taken in conjunction with the accompanying drawings. 
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0018]      FIG. 1  is a state schematic drawing of a prior art technique utilizing a search window to search for a human face in an image;  
         [0019]      FIG. 2  is a state schematic drawing of a prior art technique providing an upscaling process to the search window to search for a face in an image;  
         [0020]      FIG. 3A  is a schematic drawing of an image of a face according to a prior art;  
         [0021]      FIG. 3B  is a skin color area schematic drawing of the facial image shown in  FIG. 3A  according to the prior art;  
         [0022]      FIG. 3C  is a schematic drawing of a skin color model based on  FIG. 3A  according to the prior art;  
         [0023]      FIG. 4  is a flow chart of an embodiment being performed in an offline status according to the present invention;  
         [0024]      FIG. 5A  shows an image having facial skin color according to the present invention;  
         [0025]      FIG. 5B  shows a skin color area corresponding to the image shown in  FIG. 5A ;  
         [0026]      FIG. 6  shows a search window of the embodiment according to the present invention;  
         [0027]      FIG. 7  is a detailed operational flow chart of step S 403  shown in  FIG. 4 ;  
         [0028]      FIG. 8A  is a schematic drawing of a transformed image according to the present invention;  
         [0029]      FIG. 8B  shows a transformed skin color area corresponding to the transformed image shown in  FIG. 8A ;  
         [0030]      FIG. 9  is a flow chart of another embodiment being performed in an online status according to the present invention; and  
         [0031]      FIG. 10  is a detailed operational flow chart of step S 903  shown in  FIG. 9 . 
     
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT  
       [0032]     Two embodiments are described in the following description.  
         [0033]     The method for adjusting image acquisition parameters to optimize object extraction in the present invention can filter effects caused by different acquisition environments, different lighting conditions, or different image extraction device parameters, and extracts a complete object area which can be used to perform further image processing procedures, such as detection, tracking, positioning, or recognition upon the image. The present invention can be preformed on both offline image data and online captured images. The offline image can be selected from well known JPEG, GIF or MPEG image files, and the image acquisition parameters are calculated according to this image and used to adjust other images obtained from the same acquisition environment (such as the same lighting conditions or background) to extract an optimized object image. The online capturing image device uses one image to calculate the image acquisition parameters, and then accordingly adjusts its parameter settings.  
         [0034]     The extracted object in the present invention has the characteristic of forming a specific cluster in a color coordinate space (such as an RGB coordinate space, an HSV coordinate space, a YCbCr coordinate space, etc. . . . ) after performing the appropriate coordinate projection. For example, when an RGB color component corresponding to the skin color of different facial types is projected into a YCbCr coordinate space, a specific cluster will be formed. In the following first and second embodiments, a skin color model (i.e. a specific color model) established by a specific cluster formed by facial skin colors (i.e. the object color) being projected into the YCbCr coordinate space is used as a reference for extracting a target color (i.e. target object) in the image. Therefore, after a skin color area is extracted, any related subsequent facial image processing procedures will have higher recognition efficiencies. Any object that has this characteristic is suitable for the present invention.  
         [0000]     First Embodiment (Calculating the Image Acquisition Parameters in an Offline Status):  
         [0035]     Please refer to  FIG. 4  and  FIG. 5A .  FIG. 4  is a flow chart of an embodiment being performed in an offline status according to the present invention.  FIG. 5A  shows an image having facial skin colors according to the present invention. In this embodiment, an image  1  including human facial skin colors, as shown in  FIG. 5A , is extracted (step  401 ), which is an existing image from an offline source, such as film or monitor video; this offline data includes hardware settings of the image capturing device that cannot be changed. According to a skin color model established from large amounts of skin color information, a skin color area  11  as shown in  FIG. 5B  is extracted from the image  1 , which is broken and has some background noise. Therefore, the skin color area  11  in the image  1  is not represented completely, which is not ideal for subsequent facial detection, positioning or tracking image processing procedures.  
         [0036]     In order to increase the extraction efficiency for objects, a position of the face in the image  1  is located (step  402 ), as shown in  FIG. 6 . In this embodiment, a search window  2  is used for locating a facial area in the image  1 , which utilizes an automatic facial detection method, or a manual selection method. Furthermore, image acquisition parameters are calculated based upon a von Kries color adaptation model (step  403 ); the formula for the von Kries color adaptation model is: 
 
 L (λ)= E (λ) R (λ), 
 
 wherein R(λ) is a reflecting panel, E(λ) is an incident light projected onto the reflecting panel, L(λ) is a reflected light from the reflecting panel, and λ is a wave length of the light. Accordingly, a excitation value of the reflected light is determined by the characteristics of the incident light and the reflected panel, and the three original R, G, B color components of L(λ), E(λ), and R(λ) are L R (λ){grave over ()} L G (λ){grave over ()} L B (λ), E R (λ){grave over ()} E G (λ){grave over ()} E B (λ), and R R (λ){grave over ()} R G (λ){grave over ()} R B (λ). Since in this embodiment all reflecting characteristics of the face are all the same, which also indicates R(λ) has a fixed value, the following relationship rule can be obtained according to the von Kries color adaptation model:  
       {                 L   1     ⁡     (   λ   )       =         E   1     ⁡     (   λ   )       ⁢     R   ⁡     (   λ   )                         L   2     ⁡     (   λ   )       =         E   2     ⁡     (   λ   )       ⁢     R   ⁡     (   λ   )                 ,         
 
         [0037]     and furthermore:  
               L   1     ⁡     (   λ   )           L   2     ⁡     (   λ   )         =         E   1     ⁡     (     ⁣   λ     )           E   2     ⁡     (   λ   )           ,       
 
 wherein L 1 (λ) is an ideal color value, L 2 (λ) is an object color value corresponding to the target object, E 1 (λ) is a background light source when the ideal color value L 1 (λ) is detected, and E 2 (λ) is background light source when the object color value L 2 (λ) is detected. 
 
         [0038]     According to the above relationships, please refer to  FIG. 7  with  FIG. 4 , which shows a detailed operational flow chart of step  403  shown in  FIG. 4 . Firstly, the ideal color value L 1 (λ) is extracted (step  701 ); in this embodiment, a center point of the skin color model distributed in the color coordinate is directly used as the ideal color value L 1 (λ). Next, based on a color histogram corresponding to the facial area located by the search window  2 , an actual object color value L 2 (λ) corresponding to the skin color components in the search window  2  is extracted (step  702 ); since it is in a facial area, the smooth skin color utilizes more brighter color components, providing color components with higher grey values, or the brightest first 80% (depending on actual conditions, or being selected manually) of the color components in the color histogram corresponding to this facial area, which are extracted and the three average values R m , G m , B m  of the three original colors are obtained to indicate an actual skin color value L 2 (λ) before adjusting the image acquisition parameters. By using the ratio  
               L   1     ⁡     (   λ   )           L   2     ⁡     (   λ   )         =         E   1     ⁡     (     ⁣   λ     )           E   2     ⁡     (   λ   )           ,       
 
 an image acquisition parameter is obtained as  
           L   1     ⁡     (   λ   )           L   2     ⁡     (   λ   )           
 
 (step  703 ). 
 
         [0039]     Please refer again to  FIG. 4 ; after the skin color values L 2 (λ) of all pixels in the image  1  are multiplied by the image acquisition parameter  
             L   1     ⁡     (   λ   )           L   2     ⁡     (   λ   )         ,       
 
 a transformed image  3  as shown in  FIG. 8A  is obtained (step  404 ) to emphasize skin color characteristics and to remove background noise. Moreover, a transformed skin color area  31  in the transformed image  3  as shown in  FIG. 8B  can be extracted by a calculation of the skin color model (step  405 ), which is much more clear and complete than the skin color area  11  shown in  FIG. 5B . Therefore, a range covered by this optimized transformed skin color area  31  can accelerate subsequent image processing procedures, and clearly reduces a probability of detecting incorrect facial areas. 
 
         [0040]     The image  1  in  FIG. 5A  is one image from a series of continuous video images, and so after multiplying the acquisition parameters  
           L   1     ⁡     (   λ   )           L   2     ⁡     (   λ   )           
 
 all other images having the same acquisition environment as the image  1  will have the same skin color effect, and all can provide a clear and complete skin color area after being determined by the skin color model. 
 
         [0041]     In addition, although this embodiment is used for obtaining the image acquisition parameters in an offline status, the image acquisition parameters  
           L   1     ⁡     (   λ   )           L   2     ⁡     (   λ   )           
 
 can still be used for the image captured by an online camera. 
 
 Second Embodiment (Calculating the Image Acquisition Parameters in an Online Status): 
 
         [0042]     Please refer to  FIG. 9 .  FIG. 9  is a flow chart of another embodiment being performed in online status according to the present invention. In this embodiment, the image  1  shown in  FIG. 5A  is used as an online image. The image  1  is sent from an operating image capturing device (step  901 ). The image capturing device can be a video camera or a still camera. Next, the search window shown in  FIG. 6  is also used for locating the facial area in the image  1  (step  902 ).  
         [0043]     Please refer to  FIG. 10 .  FIG. 10  is a detailed operational flow chart of step  903  shown in  FIG. 9 . In step  903  , a skin color distribution of the image  1  is extracted, and then the skin color model is used for calculating different parameter sets to find a flowchart for optimized image acquisition parameters. In this embodiment, the skin color model is used for extracting the skin color distribution corresponding to different parameter sets κ i  in the image  1  (step  101 ), wherein the parameter sets can be RGB gain values κ Gr   i , κ Gg   i , κ Gb   i , or even combinations of exposure values or brightness parameters. Furthermore, all possible parameter sets κ i  are input into a formula: C i =S i   inside −S i   outside  to obtain their color differences (step  102 ); wherein, S i   inside  is a total number of pixel in the search window  2  and matching the skin color model, and S i   outside  is a total number of pixels outside the search window  2  and matching the skin color model S i   inside . Accordingly, the parameter set having the maximum color difference C i   max  is the image acquisition parameters for this embodiment (step  103 ). Therefore, the image capturing device can be adjusted according to the image acquisition parameters to obtain the transformed image  3  matching the skin color model as shown in  FIG. 8A  (step  104 ), and then a transformed skin color area  31  in the transformed image  3  as shown in  FIG. 8B  can be extracted by calculation with the skin color model (step  105 ).  
         [0044]     Certainly, the image capturing device will be adjusted according to the image acquisition parameters, and so a more clear and complete skin color area can be extracted from all further dynamic images by utilizing the skin color model.  
         [0045]     According to the above-mentioned description, the present invention can also be applied to other objects with different colors, as long as the object has the characteristic of forming a specific cluster in a color coordinate space after performing a coordinate projection; the image device can extract a complete object under different camera settings, different lighting conditions and different software and hardware parameters. The present invention can automatically adjust the image capturing device according to the image acquisition parameters without requiring manual adjustment. After undergoing processing by the method of the present invention, the image can provide better objects for subsequent image processing procedures.  
         [0046]     Although the present invention has been explained in relation to its preferred embodiment, it is to be understood that many other possible modifications and variations can be made without departing from the spirit and scope of the invention as hereinafter claimed.