Abstract:
Left and right half-face images are processed as independent components in a face-recognition algorithm. To provide compatibility with full-face image recognition systems, mirror-images of the half-face images are used to create full-face images corresponding to each of the left and right half-face images. Each of the created full-face images is compared to a reference full-face image, using conventional face-recognition algorithms. By comparing each of the left-based image and right-based image, the system overcomes the recognition problems that are caused by directional or non-uniform illumination. Alternatively, a composite full-face image can be created based on a blending of the characteristics of each of the left and right half-face images, thereby filtering the illumination variations.

Description:
BACKGROUND OF THE INVENTION  
         [0001]    1. Field of the Invention  
           [0002]    This invention relates to the field of computer vision, and in particular to recognition systems based on facial characteristics.  
           [0003]    2. Description of Related Art  
           [0004]    Face recognition is commonly used for security purposes. In a manual security system, security badges containing facial photographs are used to control access to secured areas or secured material. In automated and semi-automated systems, face recognition software is used to similarly match a current image of a person, from, for example, a video camera, with a stored image. In conventional systems, the user identifies himself or herself, and the face recognition software compares the video image with one or more stored images of the identified person.  
           [0005]    Face recognition is also used in a variety of other applications as well. Copending U.S. patent application, “DEVICE CONTROL VIA IMAGE-BASED RECOGNITION”, Ser. No. 09/685,683, filed Oct. 10, 2000 for Miroslav Trajkovic, Yong Yan, Antonio Colmenarez, and Srinivas Gutta, Attorney Docket US000269, incorporated by reference herein, discloses the automated control of consumer appliances, based on a facial recognition of a user, and preferences associated with the recognized user.  
           [0006]    U.S. Pat. No. 5,956,482, “MULTIMEDIA INFORMATION SERVICE ACCESS” issued Sep. 21, 1999 to Agraharam et al, and incorporated by reference herein, presents a security technique wherein a user requests access to an information service, the system takes a video snapshot of the user, and grants access to the information service only if the snapshot corresponds to an authorized user. U.S. Pat. No. 5,835,616, “FACE DETECTION USING TEMPLATES”, issued Nov. 10, 1998 to Lobo et al, and incorporated by reference herein, presents a two step process for automatically finding a human face in a digitized image, and for confirming the existence of the face by examining facial features. The system of Lobo et al is particularly well suited for finding one or more faces within a camera&#39;s field of view, even though the view may not correspond to a typical facial snapshot.  
           [0007]    A common problem with face recognition algorithms is varying illumination levels. As a person travels from one area to another, the person&#39;s face is typically illuminated from different directions. As the illumination level and direction of a current facial image differs from the illumination level and direction of the reference facial image that is used to identify the person, the ability of the system to recognize the person degrades. A shadowed cheek, for example, can be misinterpreted as a beard, because the ability to distinguish color is substantially reduced in dark images. In like manner, strong lighting can diminish features and details that would normally be apparent due to shading.  
         BRIEF SUMMARY OF THE INVENTION  
         [0008]    It is an object of this invention to improve the effectiveness of facial recognition algorithms. It is a further object of this invention to reduce the variations in an image caused by variations in illumination level and direction.  
           [0009]    These objects and others are achieved by processing the left and right half-face images as independent components in a face-recognition algorithm. To provide compatibility with full-face image recognition systems, mirror-images of the half-face images are used to create full-face images corresponding to each of the left and right half-face images. Each of the created fall-face images is compared to the reference full-face image, using conventional face-recognition algorithms. By comparing each of the left-based image and right-based image, the system overcomes the recognition problems that are caused by directional or non-uniform illumination. Alternatively, a composite full-face image can be created based on a blending of the characteristics of each of the left and right half-face images, thereby filtering the illumination variations.  
       
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0010]    The invention is explained in further detail, and by way of example, with reference to the accompanying drawings wherein:  
         [0011]    [0011]FIG. 1 illustrates an example block diagram of a face-recognition system in accordance with this invention.  
         [0012]    [0012]FIG. 2 illustrates an example flow diagram of a face-recognition system in accordance with this invention.  
         [0013]    [0013]FIG. 3 illustrates an example flow diagram for composing faces in a face-recognition system in accordance with this invention. 
     
    
       [0014]    Throughout the drawings, the same reference numerals indicate similar or corresponding features or functions.  
       DETAILED DESCRIPTION OF THE INVENTION  
       [0015]    This invention is premised on the observation that, except in abnormal situations, a person&#39;s face is left-right symmetric. As such, a full-face image contains redundant information. Alternatively stated, a half-face image can be used to create a full-face image, or, the two halves of a full-face image can be used to form a composite full-face image based on a blending of the symmetrically redundant information. Copending U.S. patent application “System and Method of Face Recognition through ½ Faces”, Ser. No. 09/966436 filed Sep. 28, 2001 for Srinivas Gutta, Miroslav Trajkovic, and Vasanth Philomin, Attorney docket US010471, discloses an image classifier that can be trained to learn on half-face or full-face images, and is incorporated by reference herein.  
         [0016]    [0016]FIG. 1 illustrates an example block diagram of a face-recognition system  100  in accordance with this invention. A face-finder  110  is configured to recognize faces within an image, using techniques common in the art. Typically, for example, faces are recognized by finding local areas of flesh tones, with darker areas corresponding to eyes. At  120 , each located face is processed to provide two half-faces.  
         [0017]    In a preferred embodiment, the face in the image is “warped” (translated, rotated, and projected) to form a facial image that is substantially “full-faced”, and this full-faced image is split in half to form a left and right half-face image. Assuming that both eyes are visible in the image, the full-faced image is produced by projecting a line between the eye-corners in the image, and translating and rotating the image such that the line is horizontal, and lies on a plane that is parallel to the image plane. Thereafter, left and right half-face images are produced by bisecting this plane at the midpoint of the line between the eye-corners. Other techniques for partitioning a face image into two half-face images will be evident to one of ordinary skill in the art. Similarly, techniques for extracting a single half-face image, when, for example, the face image is in profile, will also be evident to one of ordinary skill in the art.  
         [0018]    A face-composer  130  is configured to create one or more full-face images based on the half-face images provided by the face-splitter  120 . In a preferred embodiment, as discussed further below, each half-face image is used to create a full-face image, by combining the half-face image with its mirror image. Except in abnormal circumstances, differences between two opposing half-face images are generally indicative of different illumination on each side of the face image. Because the illumination in most environments is directional, if the half-face images differ, it is usually because one side of the face is properly illuminated, and the other half is not. Thus, the two created full-face images are likely to include one properly illuminated full-face image that can be compared to a reference image, via a conventional face-comparator  140 . Even if neither half-face image is properly illuminated, the created full-face images will be, by creation, symmetrically illuminated, and therefore more likely to match a symmetrically illuminated reference image.  
         [0019]    Techniques may be employed to select which of the two created full-face images is more properly illuminated, and compare the more properly illuminated image to the reference image. In a preferred embodiment, however, the selection process is eliminated in preference to comparing both created full-face images to the reference image, because the processing time required to compare the two created images with each other is likely to be comparable to the processing time required to compare each of the created images with the reference image.  
         [0020]    Other techniques may be employed to create full-face images from the extracted half-face images. For example, in another preferred embodiment, the aforementioned two created full-face images are merged to form another full-face image. The merging may be based on a simple averaging of pixel values within each image, or it may be based on more sophisticated techniques, such as those used for ‘morphing’ images in conventional image processing systems.  
         [0021]    The face-comparator  140  uses conventional face comparison techniques, such as those presented in the patents referenced in the background of the invention. Note that this invention is particularly well suited as an independent “add-on” process to a conventional face comparison system. The blocks  110 - 130  merely present the original and the created images to the face comparator  140  as separate images for comparison with the reference face image.  
         [0022]    [0022]FIG. 2 illustrates an example flow diagram of a face-recognition system in accordance with this invention. At  210 , a scene image is received, from which one or more faces are extracted, at  220 . Not illustrated, the extracted face images may be processed or composed based on a plurality of image scenes, using techniques common in the art to highlight features, reduce noise, and so on. Each face image is processed via the loop  230 - 280  to provide alternative faces that are each compared to one or more reference faces, at  270 .  
         [0023]    At  240 , each full-face image is processed to extract a left-face and a right-face image. If the face extraction process of  220  does not provide a full-face image, the process  240  performs the necessary translation and rotation processes to provide a full-face image, as discussed above. If both the left and right face are substantially equivalent, then the created new faces based on these equivalent halves will generally be substantially equivalent to the original full-face image. To avoid the needless creation of equivalent new faces, the face composition block  260  is bypassed when, at  250 , the two half-face images are determined to be substantially equivalent. Any of a variety of techniques may be used to determine equivalence between the half-face images. In a preferred embodiment, a sum-of-squares difference measure is used to determine the magnitude of the differences between each half-image.  
         [0024]    An example face composition process  260  is detailed in FIG. 3. Each half-face image is processed via the loop  310 - 340 . At  320 , a mirror image of the half-face image is created, and this mirror image is combined with the half-face image to produce a full-face image, at  330 . Note that if the extraction process  240  of FIG. 2 only produces one half-face image, such as when the face image is in profile, the process  260  provides at least one full-face image for comparison with the reference image, via this mirror-and-combine process  320 - 330 . If the extraction process  240  of FIG. 2 provides both half-face images, two full-face images are produced. Optionally, as discussed above, other full-face images may be produced based on a merging of select characteristics of each of the half-face images, at  350 .  
         [0025]    Returning to FIG. 2, each of the created images, and optionally the original image, is compared to one or more reference images, at  270 , to identify a potential match. Because each of the created images represent, effectively, the same face at different illuminations, the process of this invention increases the likelihood of properly identifying a face even when the illumination level and/or direction is not uniform or consistent.  
         [0026]    The foregoing merely illustrates the principles of the invention. It will thus be appreciated that those skilled in the art will be able to devise various arrangements which, although not explicitly described or shown herein, embody the principles of the invention and are thus within its spirit and scope. For example, the invention is presented in the context of processing half-faces to form a variety of fall-faces for comparison with a reference full-face image. Alternatively, the reference face image may be stored as a half-face image, and the aforementioned processing and comparisons may be relative to the half-face reference image, consistent with the techniques disclosed in copending U.S. patent application Ser. No. 09/966436, referenced above. That is, in this alternative embodiment, each half-face image or its mirror is compared directly with the half-face reference image. Additionally, a composite half-face that is based on characteristics of both of the half-face images can be compared to the half-face reference image. These and other system configuration and optimization features will be evident to one of ordinary skill in the art in view of this disclosure, and are included within the scope of the following claims.