Abstract:
A method and implementation of enhancing a facial recognition process to increase the judgment confidence on identifying a person from a large image database of multiple persons. The method may include reconstructing a database of 2D images having only a small number of images with respect to each person into a database having multiple images, perhaps hundreds or thousands, of each person. The multiple images represent different camera angles or different lighting conditions. The method further includes adding an extra confidence percentage to matching images in the database when multiple images of a person are identified as matching.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS  
       [0001]    Memory-Resident Database Management System and Implementation Thereof; Ser. No. 10/347,678; Filed on Jan. 22, 2003; Attorney Docket Number 0299-0005; Inventors: Tianlong Chen, Jonathan Vu.  
         [0002]    Distributed Memory Computing Environment and Implementation Thereof; application Ser. No. 10/347,677, Filed on Jan. 22, 2003; Attorney Docket Number 0299-0006; Inventors: Tianlong Chen, Jonathan Vu, Yingbin Wang.  
         [0003]    Invariant Memory Page Pool and Implementation Thereof; Filed on Apr. 30, 2003; Attorney Docket Number 0299-0014; Inventors: Tianlong Chen, Yingbin Wang, Yinong Wei.  
         [0004]    Central Linked List Data Structure and Methods of Use; Filed Jul. 9, 2002, Provisional Application S No. 60/394,257; Attorney Docket No. 0299-0001; Inventor: Jonathan Vu.  
         [0005]    A Method and or System to Perform Automated Facial Recognition and Comparison Using Multiple 2D Facial Images Parsed from a Captured 3D Facial Image;  
         [0006]    Image Indexing Search and Implementation Thereof; U.S. Provisional Application No. 60/454,315 filed on Mar. 14, 2003; Inventors: Tianlong Chen, Yi Rui, Yingbin Wang, and Yinong Wei.  
         [0007]    The entirety of each of the aforementioned patent applications is incorporated by reference herein. 
     
    
     
       STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT  
         [0008]    Not applicable.  
         FIELD OF THE INVENTION  
         [0009]    The present invention is related to a method and implementation of enhancing a facial recognition process to increase the judgment confidence on identifying a person from a large image database of multiple persons.  
         BACKGROUND OF THE INVENTION  
         [0010]    In traditional facial identification processes of identifying one person from an image database of multiple persons in 2D facial recognition, a face photo of the person is compared (or called “matching”) with all face photos in the database using a recognition algorithm or recognition software. The 2D facial recognition algorithm or software will generate a confidence percentage for each matching photo in the database on how much similarity the 2D facial recognition algorithm finds between the face photo of the person and the matching photo in the database. The match of the highest confidence percentage or several matches with top confidence percentages are returned for the operator to make a judgment as to whether there are matches.  
           [0011]    Traditional 2D facial recognition algorithms, however, normally have problems identifying face photos because of a large number of variables, such as the lighting present at the time a particular photo is taken or the angle at which the photo is taken, with respect to the face photo being taken of the person and the face photos in the database. Other such variables would be apparent to those skilled in the art. These variables can cause the confidence percentages generated by the  2   d  facial recognition algorithm or software to be artificially low.  
         SUMMARY OF THE INVENTION  
         [0012]    The present invention enhances the matching process by reconstructing an image database of multiple persons into one in which each person has multiple pictures representing different angles, lighting conditions and etc. Preferable, each person will have a similar or identical number of photos in the reconstructed database. The invention enhances the confidence percentage for a particular match based upon the number of matching photos found for a particular person. The present invention provides a method and implementation on how to construct and define such increased confidence over existing single-image-per-person facial recognition systems.  
           [0013]    Still other aspects, features, and advantages of the present invention are readily apparent from the following detailed description, simply by illustrating preferable embodiments and implementations. The present invention is also capable of other and different embodiments, and its several details can be modified in various respects, all without departing from the spirit and scope of the present invention. Accordingly, the drawings and descriptions are to be regarded as illustration in nature, and not as restrictive. 
       
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0014]    The accompanying drawings, which are incorporated in and constitute a part of this specification illustrate some embodiments of the invention and, together with the description, serve to explain the objects, advantages, and principles of the invention. In the drawings,  
         [0015]    [0015]FIG. 1 illustrates a conceptual dataflow diagram of the enhanced facial recognition architecture in accordance with the present invention. 
     
    
     DESCRIPTION OF THE PREFERRED EMBODIMENT  
       [0016]    The enhanced facial recognition system and method of the present invention is built upon existing 2D facial recognition algorithms. The present invention relates to enhancement of a confidence level of an identity of a person from an image database of multiple persons, sometimes called a one-to-many identification process.  
         [0017]    Referring to FIG. 1, normally each person in a 2D image database  101  has only one (frontal) or two (frontal and profile, i.e. 90 degree) 2D pictures in the database. Given such a limited database, a first step of the present invention is to reconstruct  109  the 2D image database  102  in such a way that each person will have same number or similar number (preferably the difference in number being less than ±5% of the average number  
         [0018]    There exists a technology that can generate a 3D module from a 2D frontal picture or from both a 2D frontal picture and a 2D profile picture of a person (Such as the software from CyberExtruder, http://www.cyberextruder.com/). This 2D to 3D processing is sometimes referred to as “2D-3D processing.” The 3D module can be rotated and be added with different lighting and other parameters as one wishes. Such lighting parameters may include brightness, hue, contrast, the angle of lighting, and others. One also can save a 2D picture (sometimes called “parsed image” or “morphed image”) from the generated 3D module at each orientation and lighting condition from the 3D module. Theoretically, one could generate a virtually infinite number of 2D pictures of a person with different orientation, lighting and other parameters. This 3D to 2D processing may be referred to as “3D-2D processing,” “3D parsed to 2D processing,” or “parsing a 3D image into multiple 2D images.” The whole process of converting one or two 2D picture(s) of a person into a 3D module and then further generating multiple parsed 2D pictures of a person is sometimes referred as “2D-3D-2D processing.” While the above represents one method of generating multiple pictures of a person for the reconstruction of a 2D image database, other methods may be used in connection with the present invention.  
         [0019]    Referring to FIG. 1, using the above 2D-3D-2D processing technology, one can generate multiple 2D pictures with different orientations, different lighting conditions and other parameters from one or two 2D pictures of a person. Typically based on intended applications, one original image is to be parsed into 50 to 1000 parsed images. This provides a method  109  to reconstruct an original 2D image database  101  into an enhanced 2D image database  102 . Further, rules may be set for the reconstruction. First, the number of pictures generated for each person preferably is set to be substantially same (or the difference in number is less than ±5% of the average number of pictures per person). Second, the orientation, lighting conditions and other parameter settings preferably are set to be substantially the same for all persons in the image database. Third, one original picture per person, i.e., the picture from which the multiple parsed images was generated and normally is a frontal picture, is included in the reconstructed database. These three rules provide for each person in the enhanced 2D image database  102  to be relatively equivalent. Other reconstruction rules, or a subset of these three rules, are, of course, possible and may be used with the present invention.  
         [0020]    Still referring to FIG. 1, in a 2D image database having multiple pictures of each of multiple people (such as the reconstructed database), each image in the 2D image database  102  may be digitized into a template to be used by a 2D facial recognition matching algorithm for one-to-one comparison with a photo taken of a person. At this point, each person in the enhanced image database  102  has multiple parsed images (including original image) and multiple templates (one template per each parsed image and the original image). Since only templates are used in 2D facial matching algorithm, and there is no need to display a large number of images of the same person to an operator when matches are found, all but one or a few of the multiple parsed images can optionally be removed after their templates are generated in order to save storage space. Typically, the original frontal image would be kept for display to an operator in the event of a match for a particular person in the database. Each person in the enhanced 2D image database  102  has one group of templates, and every template in a person&#39;s group is assigned with same group ID.  
         [0021]    Referring to FIG.1, the present invention shows an enhanced judgment process ( 105 ,  104 ,  106 ,  107 , and  108 ) based on newly reconstructed 2D image database  102 .  
         [0022]    Still referring to FIG. 1, the process starts from one incoming target image  103 , and the goal is to find matches which have above a pre-selected confidence threshold, say PS 1  in percentage (%), from the enhanced 2D image database  102 .  
         [0023]    Still referring to FIG. 1, the present process starts by finding  105  matches of the incoming target image  103  from the enhanced 2D image database  102 , and the matches should have over a second pre-selected confidence threshold, say PS 2  in percentage (%) (PS 2  is also called “qualification percentage”), and choose a number N 2  (called “qualification candidate number”) of top matches from the matches to form a group, called voting group  104 . And typically choose PS 2 &lt;=PS 1 , and N 2  ideally be no limit. If N 2  being unlimited results in too many matches and therefore a slow speed, N 2  is typically chosen to be the number of morphed images each person has, which is in 100s or 1000s. In the well-constrained environments in which the incoming target image is known to be within certain angle and lighting conditions, then the number of morphed images can be configured to be much less.  
         [0024]    Still referring to FIG. 1, it is possible that one person has multiple matches in the voting group  104 , and it is often the case in real world. And normally each match can have different matching confidence percentage.  
         [0025]    Another possibility is no match above PS 2 . Therefore, there will be no match above PS 1  (&gt;=PS 2 ) either, and the final judgment or result is no match in such a case. procedure is that if multiple judges say he is the target, it will be more confident than one judge says he is the target.  
         [0026]    Still referring to FIG. 1, based on the above design philosophy, there are many ways to implement the procedure  106  (called voting procedure). Here shows one possible way. Let us assume that there are N number of matches to the incoming target in the voting group  104 , and those matches correspond to L number of persons, each person has, respectively, p 1 , p 2 , p 3  . . . p L  number of matches, and p 1 +p 2 +p 3 + . . . +p L =N, and L&lt;=N, and each person has its maximum match value, respectively, m 1 , m 2 , m 3 , . . . mL in the voting group  104 . And choose a percentage number X that is the maximum extra confidence percentage added above the regular confidence percentage number from the 2D facial matching algorithm. X is typically chosen to be 10 to 20 (%). Then each person in the voting group  104  will have the final match confidence percentage, respectively, (m 1 +p 1 /N*X), (m 2 +p 2 /N*X), . . . (m L +p L /N*X). Another way to determine the final match confidence percentage is to choose the maximum value of each person in the voting group m 1 , m 2 , m 3 , . . . mL, respectively, as the final match confidence percentage for each person in the voting group. If the final match percentage value is over 100, set to be 100. By the way, the match percentage value is between 0 and 100 including 0 and 100.  
         [0027]    Still referring to FIG. 1, then each person in the voting group  104  will have one single final match percentage value. Choose those persons whose final match percentage value is over PS 2  for final output  107 ,  108  if any, and display the result in sorted order.  
         [0028]    The foregoing description of the preferred embodiments of the invention has been presented for purposes of illustration and description. It is not intended to be exhaustive are possible in light of the above teachings or may be acquired from practice of the invention. The embodiments were chosen and described in order to explain the principles of the invention and its practical application to enable one skilled in the art to utilize the invention in various embodiments as are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the claims appended hereto, and their equivalents. The entirety of each of the aforementioned documents is incorporated by reference herein.