Patent Publication Number: US-9430694-B2

Title: Face recognition system and method

Description:
FIELD OF THE INVENTION 
     The present invention generally relates to the field of computer technologies and, more particularly, to techniques for a face recognition system and method. 
     BACKGROUND 
     Face recognition systems have been quite popular in today&#39;s commercial and entertainment businesses. Face recognition in videos is a technical problem in computer vision that targets at locating and identifying faces in a video sequence by a given set of images that contain the faces with known identities. For example, video face recognition has been driven by its huge potential in developing applications in many domains including video surveillance security, augmented reality, automatic video tagging, medical analysis, quality control, and video-lecture assessment. Even though the face recognition is a relatively easy task for human brains, it is challenging for machines due to large variations in appearance of identified objects in terms of orientation, illumination, expression and occlusion. 
     Many challenges exist for the face recognition using currently-available techniques. Recently, face recognition (FR) via sparse representation-based classification (SRC) and its extensions have proven to provide state-of-the-art performance. The main idea is that a subject&#39;s face sample can be represented as a sparse linear combination of available images of the same subject captured under different conditions (e.g., poses, lighting conditions, occlusions etc.). The same principle can also be applied when a face image is represented in a lower dimensional space describing important and easily identifiable features. In order to enforce sparsity, l 1  optimization algorithms can be employed. Then, the face class that yields a minimum reconstruction error is selected in order to classify or identify the subject, whose test image or sample is available. Sparse coding has also been proposed to jointly address the problems of blurred face recognition and blind image recovery. 
     However, l 1  optimization methods for improved face recognition rates can only be successful under certain conditions. Specifically, the sparse representation based face recognition assumes that training images have been carefully controlled and that the number of samples per class is sufficiently large. 
     From a different point of view, in order to remove outlier pixels from corrupted training data, the low-rank structure of face images has been recently investigated. The low-rank structure of similar faces is explored under the assumption that the images are of some convex Lambertian object under varying illumination. To recover subspace structures from data containing errors, methods such as Robust Principal Component Analysis (RPCA) and Low-Rank Representation (LRR) have been proposed. However, the above methods are transductive and cannot remove corruptions from new data efficiently. A desired property in face recognition is not only to recover clean images from corrupted training data, but also to recover a clean image from complex occlusions and corruptions for any given test sample. 
     The disclosed methods and systems are directed to solve one or more problems set forth above and other problems. 
     BRIEF SUMMARY OF THE DISCLOSURE 
     One aspect of the present disclosure includes a face recognition method. The method includes dividing an input video into different sets of frames and detecting faces of each frame in the input video. The method also includes generating face tracks for the whole video. Further, the method includes applying a robust collaborative representation-based classifier to recover a clean image from complex occlusions and corruptions for a face test sample and perform classification. In addition, the method also includes outputting the video containing the recognized face images. 
     Another aspect of the present disclosure includes a face recognition system. The system includes a face detection module configured to find automatically location of faces in a sequence of video frames and an algorithm module configured to recover a clean image from complex occlusions and corruptions through an inductive robust principal component analysis (IRPCA) algorithm to initialize a low-rank representation with an l 1  half quadratic (LRR-HQ-L1) algorithm and estimate a weight matrix through the LRR-HQ-L1 algorithm. The system also includes a face classifier configured to perform classification through a robust collaborative representation (RCR) algorithm and a dictionary configured to store face images in a database. Further, the system includes an output module configured to output the video containing the recognized face images. 
     Other aspects of the present disclosure can be understood by those skilled in the art in light of the description, the claims, and the drawings of the present disclosure. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  illustrates an exemplary environment incorporating certain embodiments of the present invention; 
         FIG. 2  illustrates an exemplary computing system consistent with the disclosed embodiments; 
         FIG. 3  illustrates a structure schematic diagram of an exemplary face recognition system consistent with the disclosed embodiments; 
         FIG. 4  illustrates a flow chart of an exemplary face recognition process consistent with the disclosed embodiments; 
         FIG. 5  illustrates a video stream being divided into different video components consistent with the disclosed embodiments; 
         FIG. 6  illustrates an exemplary robust collaborative representation process consistent with the disclosed embodiments; and 
         FIG. 7  illustrates another exemplary robust collaborative representation process consistent with the disclosed embodiments. 
     
    
    
     DETAILED DESCRIPTION 
     Reference will now be made in detail to exemplary embodiments of the invention, which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts. 
       FIG. 1  illustrates an exemplary environment  100  incorporating certain embodiments of the present invention. As shown in  FIG. 1 , environment  100  may include a television set (TV)  102 , a remote control  104 , a server  106 , a user  108 , and a network  110 . Other devices may also be included. 
     TV  102  may include any appropriate type of TV, such as plasma TV, liquid crystal display (LCD) TV, projection TV, non-smart TV, or smart TV. TV  102  may also include other computing systems, such as a personal computer (PC), a tablet or mobile computer, or a smart phone, etc. Further, TV  102  may be any appropriate content-presentation device capable of presenting multiple programs in one or more channels, which may be controlled through the remote control  104 . 
     The remote control  104  may include any appropriate type of remote control that communicates with and controls the TV  102 , such as a customized TV remote control, a universal remote control, a tablet computer, a smart phone, or any other computing device capable of performing remote control functions. The remote control  104  may also include other types of devices, such as a motion-sensor based remote control, or a depth-camera enhanced remote control, as well as simple input/output devices such as a keyboard, a mouse, and a voice-activated input device, etc. 
     Further, the server  106  may include any appropriate type of server computer or a plurality of server computers for providing video contents to the user  108 . The server  106  may also facilitate communication, data storage, and data processing between the remote control  104  and TV  102 . TV  102 , remote control  104 , and server  106  may communicate with each other through one or more communication networks  110 , such as a cable network, a phone network, and/or a satellite network, etc. 
     The user  108  may interact with TV  102  using remote control  104  to watch various programs and perform other activities of interest, or the user may simply use hand or body gestures to control TV  102  if motion sensor or depth-camera is used by TV  102 . The user  108  may be a single user or a plurality of users, such as family members watching TV programs together. 
     TV  102 , remote control  104 , and/or server  106  may be implemented on any appropriate computing circuitry platform.  FIG. 2  shows a block diagram of an exemplary computing system  200  capable of implementing TV  102 , remote control  104 , and/or server  106 . 
     As shown in  FIG. 2 , computing system  200  may include a processor  202 , a storage medium  204 , a display  206 , a communication module  208 , a database  210 , and peripherals  212 . Certain devices may be omitted and other devices may be included. 
     Processor  202  may include any appropriate processor or processors. Further, processor  202  can include multiple cores for multi-thread or parallel processing. Storage medium  204  may include memory modules, such as ROM, RAM, flash memory modules, and mass storages, such as CD-ROM and hard disk, etc. Storage medium  204  may store computer programs for implementing various processes when the computer programs are executed by processor  202 . 
     Further, peripherals  212  may include various sensors and other I/O devices, such as keyboard and mouse, and communication module  208  may include certain network interface devices for establishing connections through communication networks. Database  210  may include one or more databases for storing certain data and for performing certain operations on the stored data, such as database searching. 
     In operation, the server  106 , the TV  102 , and/or the remote control  104  may perform a face recognition process for the user  108 . 
       FIG. 3  illustrates a structure schematic diagram of an exemplary face recognition system  300  consistent with the disclosed embodiments. As shown in  FIG. 3 , the face recognition system  300  includes video content  302 , a face detection module  304 , an algorithm module  306 , a face classifier  308 , a dictionary  310 , and an output module  312 . Certain components may be omitted and other components may be added. The system  300  (e.g., the various components) may be implemented in hardware, software, or a combination of hardware and software. 
     The input video  302  may include any appropriate type of source for video contents and may contain various video sources. The contents from the input video  302  may include both video data and metadata. Plurality of frames may be associated with the video contents and may be provided to other modules for processing. A single picture may also be included. The input video is divided into different sets of frames. 
     The face detection module  304  may be configured to find automatically the location of the faces in a sequence of video frames. The face detection module  304  may reside within TV  102  or outside the TV  102  depending on particular applications. 
     The algorithm module  306  may be configured to recover a clean image from complex occlusions and corruptions through an inductive robust principal component analysis (IRPCA) algorithm to initialize a low-rank representation with an l 1  half quadratic (LRR-HQ-L1) algorithm and estimate the weight matrix through an LRR-HQ-L1 algorithm. 
     The face classifier  308  may be configured to perform classification through a robust collaborative representation (RCR) algorithm and give a final decision of the class identity. 
     Further, the dictionary  310  may include any appropriate database to store face images, etc. For example, an in-house or online database has collection of face-relevant photos. Specifically, for face recognition in a given movie and a set of actors, each actor&#39;s (i.e., targeting actor) name and movie&#39;s name are used as key words for querying in the dictionary to retrieve a set of images. This image set may contain mostly the targeting actor&#39;s images and a few other actors&#39; images, with each image containing one or more actor&#39;s faces. 
     The output module  312  may be configured to output identified faces. Certain modules may be merged or omitted and other modules may be included. 
       FIG. 4  illustrates a flow chart of an exemplary face recognition process  400  performed by the various modules in the face recognition system consistent with the disclosed embodiments. 
     As shown in  FIG. 4 , at the beginning, an input video is divided into different sets of frames (S 402 ). Because a video is a sequence of frames and the changes between consecutive frames are relatively small due to typical frame rate for a video (e.g. 25 frames/second), instead of dealing with each frame individually, some grouping or clustering techniques may be applied to separate the whole video into different sets of frames with each set has similar frames for further processing. 
     For example,  FIG. 5  illustrates a video stream being divided into different video components. As show in  FIG. 5 , a video stream may be divided into scenes, a scene may be divided into shots, and a shot may be divided into frames, etc. The frame can be further divided into objects and features of the video frame may be extracted for further processing. 
     Returning to  FIG. 4 , after obtaining the input video sequences, the video sequence is simplified into a frame sequence corresponding to registered camera that is aligned to the original camera setup by registering each camera from incoming video frames (S 404 ). 
     A typical movie sequence is an interleaved format of a number of camera shots, and a camera take is a continuous recorded performance with a given camera setup. Camera registration, as used herein, may refer to registration of different cameras capturing video frames in a video sequence/stream. The concept of camera registration is based on the camera takes in reconstruction of video edits. A typical video sequence is an interleaved format of a number of camera shots, and a camera take is a continuous recorded performance with a given camera setup. By registering each camera from the incoming video frames, the original interleaved format can be separated into a number of sequences with each corresponding to a registered camera that is aligned to the original camera setup. 
     After the camera take is identified, faces of each frame in the video are detected (S 406 ). Detecting the faces of each frame in the video can facilitate to produce face tracks before applying face recognition. That is, the face detection can be applied for each frame within each camera take. The face detection may find automatically the location of the faces in a sequence of frames. 
     The group of detected faces within a same camera take may be used as a face track. In this way, face tracks are generated for the whole video (S 408 ). Specifically, a new face track is initialized by the first detected face for each camera take. For the rest frames of each camera take, if the distance between two detected faces from consecutive frames passes a pre-defined threshold, a new face track is initialized; otherwise this face is added to the current face track. 
     Further, a robust collaborative representation-based classifier is applied to recover a clean image from complex occlusions and corruptions for a face test sample and perform classification (S 410 ). Existing state-of-the-art outlier detection algorithms and classifiers can be conveniently incorporated with a robust framework for collaborative representation. It is assumed that yε   d  denotes a face test sample and T=[T i , . . . , T c ]ε   d×n  denotes a matrix (i.e., dictionary) with a set of samples of c subjects stacked in columns, where T i ε   d×n     i    denotes the n i  set of samples of the i th  subject, such that Σ i  n i =n. 
     Provided that the outlier detection inputs a test sample y and a face dictionary T and produces a weight diagonal matrix W, such that Wy is the the weighted test sample, where small values (close to zero) are given to outlier components (e.g., sunglass pixels). The outlier detection (OLD) process is denoted by:
 
 W=OLD ( y,T )  (1)
 
     Provided that a classification (CLF) process inputs a test sample, a face dictionary and a weight diagonal matrix W and outputs an estimated class index, the estimated class index is denoted by:
 
Identity( y )= CLF ( y,T,W )  (2)
 
     Formula (1) is solved with the l 1  Half Quadratic Minimization which has been proven robust to occlusions and corruptions. 
       FIG. 6  illustrates an exemplary robust collaborative representation process consistent with the disclosed embodiments. As shown in  FIG. 6 , in order to initialize W for the Half Quadratic (HQ) Minimization, a clean test image through Inductive Robust Principal Component Analysis (IRPCA) is estimated (S 4102 ). 
     A function K is half quadratic if the function K depends on two sets of variables a and w, so that K is a quadratic function of a. The augmented Half Quadratic function K is constructed from some nonquadratic criteria J and is defined by: 
     
       
         
           
             
               
                 
                   
                     
                       
                         min 
                         w 
                       
                       ⁢ 
                       
                         K 
                         ⁡ 
                         
                           ( 
                           
                             a 
                             , 
                             w 
                           
                           ) 
                         
                       
                     
                     = 
                     
                       J 
                       ⁡ 
                       
                         ( 
                         a 
                         ) 
                       
                     
                   
                   , 
                 
               
               
                 
                   ( 
                   3 
                   ) 
                 
               
             
           
         
       
     
     J(a) is the reconstruction function defined by:
 
 J ( a )=Φ( a )+ Q ( a ).  (4)
 
     where Φ( a ) and Q(a) are defined as,
 
Φ( a )=Σ i=1   d φ(δ i )
 
 Q ( a )=∥ a∥   1 .  (5)
 
     φ(δ i ) is defined as, 
     
       
         
           
             
               
                 
                   
                     ϕ 
                     ⁡ 
                     
                       ( 
                       
                         δ 
                         i 
                       
                       ) 
                     
                   
                   = 
                   
                     
                       
                         min 
                         
                           w 
                           i 
                         
                       
                       ⁢ 
                       
                         
                           1 
                           2 
                         
                         ⁢ 
                         
                           
                             
                               w 
                               i 
                             
                             ⁡ 
                             
                               ( 
                               
                                 δ 
                                 i 
                               
                               ) 
                             
                           
                           2 
                         
                       
                     
                     + 
                     
                       
                         ψ 
                         ⁡ 
                         
                           ( 
                           
                             w 
                             i 
                           
                           ) 
                         
                       
                       . 
                     
                   
                 
               
               
                 
                   ( 
                   6 
                   ) 
                 
               
             
           
         
       
     
     From (4), (5) and (6), K(a,w) is defined by:
 
 K ( a,w )=∥ W ( y−Ta )∥ 2   2 +ψ( w )+λ∥ a∥   1   (7)
 
     where ψ can be defined from 40 through convex duality relations. The overall l 1  HQ problem is minimized along a and w in an alternate fashion, 
                         w   ⋒     i     (     t   +   1     )       =         ϕ   ′     ⁡     (     δ   i   t     )         δ   i   t         ,     
     ⁢     i   =   1     ,   …   ⁢           ,   d           (   8   )                 a   (t+1) =argmin a   ∥W   t+1 ( y−Ta )∥ 2   2   +λ∥a∥   1 .  (9)
 
     The choice of the potential function φ and its corresponding weight function w can be given from M-estimators or any function that fullfils the conditions of the HQ potential functions as defined. 
     The above iterative algorithm is an iteratively reweighted least squares (IRLS) algorithm. Thus, there is a clear connection between Half Quadratic (HQ) and reweighted least squares. Even though under certain conditions a global convergence of K(a,b) is proven for any initial δ i   0 , it is not clear whether the HQ function globally converges in the l 1  framework. For different initializations δ i   0 , different Ws are obtained. A careful selection of the initialization can be to somewhat important for the FR performance. First, two potential initializations of δ i   0 , are listed. Then, the initialization solution is provided as follows. 
     1. The initialization can be the test sample minus the mean dictionary image. That is, δ i   0 =[Ta 0 −y], where 
     
       
         
           
             
               a 
               0 
             
             = 
             
               
                 [ 
                 
                   
                     1 
                     n 
                   
                   , 
                   … 
                   ⁢ 
                   
                       
                   
                   , 
                   
                     1 
                     n 
                   
                 
                 ] 
               
               . 
             
           
         
       
     
     2. Another potential solution for δ i   0  that depends on a 0  is naive reconstruction of a 0 . That is, a 0 =T/y. 
     The representation of the first suggestion may diverge a lot from the test sample face since a mean face does not necessarily look like the test sample and can have a lot of errors. Thus, when the error with the mean face is initialized, it can lead to an error W. In the cases that small corruptions appear between the test sample and the faces in the dictionary, the second suggestion could lead to an accurate W. In this case, all elements in W can be close to zero. However, when a lot of corruption appears in the test sample, the naive reconstruction can initialize the IRLS with totally wrong weights. 
     A test sample without free of corruptions is then desirable for the initialization of the IRLS. The initialization of the IRLS plays a crucial role for the FR performance. Since this is difficult to happen, the algorithm is initialized with a clean from corruption test image that is estimated through the low-rank method IRPCA. That is, IRPCA can be used to reconstruct an approximate clean face to initialize a residual image δ i   0 , which can be sufficient to provide an accurate W after few IRLS iterations. 
     In order to recover an IRPCA image, at the beginning, a low-rank representation of the face dictionary needs to be recovered. To handle multiple low-rank subspaces, a general rank minimization problem called low-rank representation (LRR) is defined by: 
     
       
         
           
             
               
                 
                   
                     
                       
                         minimize 
                         
                           Z 
                           , 
                           E 
                         
                       
                       ⁢ 
                       
                           
                       
                       ⁢ 
                       
                         
                            
                           Z 
                            
                         
                         * 
                       
                     
                     + 
                     
                       λ 
                       ⁢ 
                       
                         
                            
                           E 
                            
                         
                         
                           2 
                           , 
                           1 
                         
                       
                     
                   
                   ⁢ 
                   
                     
 
                   
                   ⁢ 
                   
                     
                       
                         subjectto 
                         ⁢ 
                         
                             
                         
                         ⁢ 
                         T 
                       
                       = 
                       
                         AZ 
                         + 
                         E 
                       
                     
                     , 
                   
                 
               
               
                 
                   ( 
                   10 
                   ) 
                 
               
             
           
         
       
     
     where λ&gt;0 is a parameter, and Z is the lowest-rank representation of data T with respect to a dictionary A. In order to recover a discriminative dictionary, the incoherence between class matrices is promoted in the Low-Rank Representation (LRR) problem defined by: 
     
       
         
           
             
               
                 
                   
                     
                       
                         minimize 
                         
                           Z 
                           , 
                           E 
                         
                       
                       ⁢ 
                       
                           
                       
                       ⁢ 
                       
                         
                            
                           
                             Z 
                             i 
                           
                            
                         
                         * 
                       
                     
                     + 
                     
                       λ 
                       ⁢ 
                       
                         
                            
                           
                             E 
                             i 
                           
                            
                         
                         
                           2 
                           , 
                           1 
                         
                       
                     
                     + 
                     
                       
                         ζ 
                         2 
                       
                       ⁢ 
                       
                         
                           ∑ 
                           
                             i 
                             = 
                             1 
                           
                           n 
                         
                         ⁢ 
                         
                           
                              
                             
                               
                                 
                                   ( 
                                   
                                     
                                       A 
                                       i 
                                     
                                     ⁢ 
                                     
                                       Z 
                                       i 
                                     
                                   
                                   ) 
                                 
                                 T 
                               
                               ⁢ 
                               
                                 A 
                                 i 
                               
                               ⁢ 
                               
                                 Z 
                                 i 
                               
                             
                              
                           
                           F 
                           2 
                         
                       
                     
                   
                   ⁢ 
                   
                     
 
                   
                   ⁢ 
                   
                     
                       
                         subjectto 
                         ⁢ 
                         
                             
                         
                         ⁢ 
                         
                           T 
                           i 
                         
                       
                       = 
                       
                         
                           
                             A 
                             i 
                           
                           ⁢ 
                           
                             Z 
                             i 
                           
                         
                         + 
                         
                           E 
                           i 
                         
                       
                     
                     , 
                   
                 
               
               
                 
                   ( 
                   11 
                   ) 
                 
               
             
           
         
       
     
     where Z i ε   n     i     ×n     i    is lowest-rank representation of class data T i  with respect to a dictionary A i ; i is the i th  face class in the dictionary, and ζ is a scalar parameter. 
     The goal of IRPCA is to integrate the advantages of both Principal Component Analysis (PCA) and Robust Principal Component Analysis (RPCA). That is, the IRPCA can not only handle data with gross corruptions but also own a good method for processing new data. After obtaining the clear from corruption training faces AZ solving the problem in Formula (11), a low-rank projection matrix Pε   d×d  multiplies with any face in T. Thus, the process gives the corresponding clear face in AZ. Having learnt how to clean faces from corruptions using the training images, any new data y can be processed and the corruptions can be removed by performing an operation Py. Matrix P can be defined by: 
     
       
         
           
             
               
                 
                   
                     
                       minimize 
                       P 
                     
                     ⁢ 
                     
                         
                     
                     ⁢ 
                     
                       
                          
                         P 
                          
                       
                       * 
                     
                   
                   ⁢ 
                   
                     
 
                   
                   ⁢ 
                   
                     
                       subjectto 
                       ⁢ 
                       
                           
                       
                       ⁢ 
                       AZ 
                     
                     + 
                     
                       PT 
                       . 
                     
                   
                 
               
               
                 
                   ( 
                   12 
                   ) 
                 
               
             
           
         
       
     
     Formula (12) is convex but not strongly convex. So it is possible that multiple optimal solutions exist. It is proven that the minimizer to Formula (12) is always uniquely defined by:
 
 P*=AZT   +   (13)
 
     After P is obtained, a clean image Py can be obtained. Since P is a projection matrix that learns how to remove corruptions, based on the training data, two arguments can be made: 
     1. Whether the given new sample has a corruption known in the face dictionary cannot be guaranteed. For example, a test sample might have an occluded object that does not appear to any face in training samples. 
     2. Even if the occluded object appears (e.g., sunglasses) in training samples, whether the occluded object looks like the one in the test sample cannot be guaranteed. 
     Thus, the learnt corruptions in P may be different from one in test samples. In order to prevent such cases, the process needs to balance between the Py image and the mean  AZ  image. Thus, the clean image is introduced to represent as,
 
 y   c   =θPy +(1−θ)   AZ ,   (14)
 
     where 0≦θ≦1. θ can be estimated empirically, and small values should be given to datasets where training samples are clean and no corruptions can be learnt. 
     Table 1 shows an IRPCA initialization algorithm. The entire process for estimating P is offline, so significant overhead to the classification process cannot be added. The algorithm for calculating y c  is summarized in Algorithm 1 in Table 1. 
     
       
         
           
               
             
               
                 TABLE 1 
               
               
                   
               
               
                 Algorithm 1: IRPCA initialization algorithm 
               
               
                 Algorithm The IRPCA initialization Algorithm 
               
               
                   
               
             
            
               
                   
               
            
           
           
               
               
            
               
                   
                 Inputs: LRR(y, T). 
               
            
           
           
               
               
               
            
               
                   
                 1) 
                 Estimate offline the low-rank matrices Z and P solving 
               
               
                   
                   
                 the problems in (10) and in (12) respectively. 
               
               
                   
                 2) 
                 Estimate y c  = θPy + (1 − θ) AZ   
               
            
           
           
               
               
            
               
                   
                 Output: vector y c   
               
               
                   
                   
               
            
           
         
       
     
     After the clean image is obtained, δ i   0 =[y c −y] i  may be used to initialize a low-rank representation with an l 1  half quadratic (LRR-HQ-L1) algorithm (S 4104 ). That is, the OLD approach, namely an LRR-HQ-L1 algorithm, is presented in Algorithm 2 in Table 2. 
     
       
         
           
               
             
               
                 TABLE 2 
               
               
                   
               
               
                 Algorithm 2: LRR-HQ-L1 algorithm 
               
               
                 Algorithm The Outlier Detection LRR-HQL1 Algorithm 
               
               
                   
               
             
            
               
                   
               
            
           
           
               
               
               
            
               
                   
                  1: 
                 function LRRHQL1(y,T) 
               
               
                   
                  2: 
                  Calculate y c  = LRR(y, T) 
               
               
                   
                  3: 
                  Initialize δ i   1  = [y c  − y] i  and t = 1 
               
               
                   
                  4: 
                  repeat 
               
               
                   
                  5: 
                   
             w   ^     i     (   t   )       =         ϕ   ′     ⁡     (     δ   i   t     )         δ   i   t         ,     i   =   1     ,   …   ⁢           ,   d       
 
               
               
                   
                  6: 
                   â (t)  = argmin a ||W t (y − Ta)|| 2   2  + λ||a|| 1   
               
               
                   
                  7: 
                   t = t + 1 
               
               
                   
                  8: 
                   δ i   t  = [Ta t − 1  − y] i   
               
               
                   
                  9: 
                  until 
                    w   t     -     w     t   -   1              2              w   t          2       &lt;       ε   w     ⁢           ⁢   or   ⁢           ⁢   t     &gt;     ε   i         
 
               
               
                   
                 10: 
                  Return W 
               
               
                   
                 11: 
                 end function 
               
               
                   
               
            
           
         
       
     
       FIG. 7  illustrates another exemplary robust collaborative representation process consistent with the disclosed embodiments. As shown in  FIG. 7 , the first row shows an example test face from an AR face database with 50% block occlusion and its corresponding ground truth image. The second row shows a reconstructed face y c  using Algorithm 1. The reconstructed face looks like a target identity (more noise needs to be removed), and it is served as a good candidate for the initialization of Algorithm 1. The next image shows a weight image learnt by LRR-HQ-L1 Algorithm 2. The last two images show a reconstructed face Tâ after classification and an error image y−Tâ between a test sample and dictionary samples. In the third row, the first image is a mean image Ta, where 
               a   0     =     [       1   n     ,   …   ⁢           ,     1   n       ]           
estimated by training samples. It is obvious that this image does not look like the target identity. The second image is the weight image. The third image shows the reconstructed image after classification and the last image is an estimated error between the test sample and dictionary samples. It is clear that better estimation of the error can be obtained by initializing the Algorithm 1 with y c .
 
     After obtaining the weight W, a classification task is processed (S 4106 ). CLF is solved by using the l 2  collaborative representation since the l 2  collaborative representation can provide better recognition rates than the l 1  optimization problems in environments without occlusions or corruptions. Since an outlier detection mask W is provided, the classification task can be treated as an occlusion or corruption free task. As used herein, the l 1  half quadratic minimization is integrated with the l 2  collaborative representation for classification. 
     An estimation vector f for classification is used by solving the regularized least squares problem.
 
 f =argmin f   ∥W ( y−Tf )∥ 2   2   +λ∥f∥   2   2   (15)
 
     where fε   n  is the estimated vector. 
     After the vector f is obtained, the identity of the face class is given by the same way as in the SRC. A Robust Collaborative Representation (RCR) algorithm is summarized in Algorithm 3 in Table 3. 
     
       
         
           
               
             
               
                 TABLE 3 
               
               
                   
               
               
                 Algorithm 3: RCR algorithm 
               
               
                 Algorithm The Classification RCR Algorithm 
               
               
                   
               
             
            
               
                   
               
            
           
           
               
               
            
               
                   
                 Inputs: Vector y and matrices T, W = LRRHQL1(y, T). 
               
            
           
           
               
               
               
            
               
                   
                  1) 
                 Estimate: {circumflex over (f)} solving the problem, 
               
               
                   
                   
                 
                   
                     
                       
                         
                           
                             f 
                             ^ 
                           
                           = 
                           
                             
                               
                                 
                                   arg 
                                   ⁢ 
                                   
                                       
                                   
                                   ⁢ 
                                   min 
                                 
                                 f 
                               
                               ⁢ 
                               
                                 
                                    
                                   
                                     W 
                                     ⁡ 
                                     
                                       ( 
                                       
                                         y 
                                         - 
                                         Tf 
                                       
                                       ) 
                                     
                                   
                                    
                                 
                                 2 
                                 2 
                               
                             
                             + 
                             
                               λ 
                               ⁢ 
                               
                                 
                                    
                                   f 
                                    
                                 
                                 2 
                                 2 
                               
                             
                           
                         
                         , 
                       
                     
                   
                 
               
               
                   
                  2) 
                 Compute the residuals for each class i as, 
               
               
                   
                   
                 e i (y) = ||W(y − T i {circumflex over (f)} i || 2 , 
               
               
                   
                   
                 where {circumflex over (f)} i  is the coding coefficient vector associated with class i. 
               
            
           
           
               
               
            
               
                   
                 Output: Identity of y as, Identity(y) = argmin i {e i }. 
               
               
                   
               
            
           
         
       
     
     The l 1  minimization is used to solve the classification task. The method identifies Robust sparse coding (RSC) and l 1  HQ. By choosing the l 2 -norm to solve the outlier detection problem, the process identifies Regularized Robust Coding (RRC) with the l 2 . In RRC, the l 1  or l 2  minimization problems are transformed into an iteratively reweighted regularized robust coding problems with a designed weight function for robust face recognition. 
     Solving the classification task with l 1  is not optimal in corruption free environments. Similarly, for the outlier detection algorithm, since the linear system in face recognition (FR) is always underdetermined, choosing to solve the outlier detection problem with l 2  could overfit the data and might lead to an inaccurate W and poor FR recognition. 
     Since a face recognition task can be split into two parts, OLD and CLF, the recognition rate can be improved by combining multiple outlier detection and classification algorithms. 
     Further, an Additive Robust Collaborative Representation Classifier (ARCR-C) is provided, where residual of each class is defined by:
 
 e   i ( y )=Σ j=1   k ξ j   e   i   j ( y )  (16)
 
     where e i   j (y) denotes the residual of the i th  class of the j th  classifier and is defined by:
 
 e   i   j ( y )=∥ W   j ( y−T   i   f   i   j )∥ 2 ,  (17)
 
     where W j , f i   j  and ξ j  denote the weight outlier detection matrix, the estimated vector and the weight of the j th  classifier, respectively. 
     Different outlier detection algorithms that can provide different weight matrices W are chosen. In Formula (16), when k=2, the followings are used to solve the outlier detection algorithm described in Algorithm 1 but with two different initializations, that is, δ i   0 =[y c −y] i , and δ i   0 =[Ta 0 −y] i  with 
     
       
         
           
             
               a 
               0 
             
             = 
             
               
                 [ 
                 
                   
                     1 
                     n 
                   
                   , 
                   … 
                   ⁢ 
                   
                       
                   
                   , 
                   
                     1 
                     n 
                   
                 
                 ] 
               
               . 
             
           
         
       
     
     Assuming that the weight matrix is from the first initialiation W lrr  and from the second initialization W mean , the subject&#39;s identity is then defined by the two classifiers,
 
Identity( y )=argmin i {ξ 1   e   i   lrr +ξ 2   e   i   mean }  (18)
 
     The description of the ARCR-C algorithm is summarized in Algorithm 4 in Table 4. 
     
       
         
           
               
             
               
                 TABLE 4 
               
               
                   
               
               
                 Algorithm 4: ARCR-C algorithm 
               
               
                 Algorithm The Additive Robust Collaborative Representa- 
               
               
                 tion Classifier (ARCR-C) 
               
               
                   
               
             
            
               
                   
               
            
           
           
               
               
            
               
                   
                 Inputs: Vector y and matrix T. 
               
            
           
           
               
               
               
            
               
                   
                  1) 
                 Estimate W lrr  solving the problem in (8) with δ i   0  = 
               
               
                   
                   
                 [y c  − y] i , then estimate {circumflex over (f)} 1 , 
               
               
                   
                   
                 
                   
                     
                       
                         
                           
                             
                               f 
                               ^ 
                             
                             1 
                           
                           = 
                           
                             
                               
                                 
                                   arg 
                                   ⁢ 
                                   
                                       
                                   
                                   ⁢ 
                                   min 
                                 
                                 f 
                               
                               ⁢ 
                               
                                 
                                    
                                   
                                     
                                       W 
                                       lrr 
                                     
                                     ⁡ 
                                     
                                       ( 
                                       
                                         y 
                                         - 
                                         Tf 
                                       
                                       ) 
                                     
                                   
                                    
                                 
                                 2 
                                 2 
                               
                             
                             + 
                             
                               λ 
                               ⁢ 
                               
                                 
                                    
                                   f 
                                    
                                 
                                 2 
                                 2 
                               
                             
                           
                         
                         , 
                       
                     
                   
                 
               
               
                   
                  2) 
                 Estimate W mean  solving the problem in (8) with 
               
               
                   
                   
                 δ i   0  = [Ta 0  − y] i  and 
           a   0     =     [       1   n     ,   …   ⁢           ,     1   n       ]       ,       
 
               
               
                   
                   
                 then estimate {circumflex over (f)} 2 , 
               
               
                   
                   
                 
                   
                     
                       
                         
                           
                             
                               f 
                               ^ 
                             
                             2 
                           
                           = 
                           
                             
                               
                                 
                                   arg 
                                   ⁢ 
                                   
                                       
                                   
                                   ⁢ 
                                   min 
                                 
                                 f 
                               
                               ⁢ 
                               
                                 
                                    
                                   
                                     
                                       W 
                                       mean 
                                     
                                     ⁡ 
                                     
                                       ( 
                                       
                                         y 
                                         - 
                                         Tf 
                                       
                                       ) 
                                     
                                   
                                    
                                 
                                 2 
                                 2 
                               
                             
                             + 
                             
                               λ 
                               ⁢ 
                               
                                 
                                    
                                   f 
                                    
                                 
                                 2 
                                 2 
                               
                             
                           
                         
                         , 
                       
                     
                   
                 
               
               
                   
                  3) 
                 Compute the residuals for each class i as, 
               
               
                   
                   
                 e i   lrr (y) = ||W lrr (y −T i {circumflex over (f)} i   1 )|| 2   
               
               
                   
                   
                 e i   mean (y) = ||W mean (y − T i {circumflex over (f)} i   2 )|| 2   
               
               
                   
                   
                 where {circumflex over (f)} i  is the coding coefficient vector associated 
               
               
                   
                   
                 with class i. 
               
            
           
           
               
               
            
               
                   
                 Output: Identity of y as, 
               
               
                   
               
            
           
         
       
     
     The performance of a face recognition system depends on two metrics, the recognition rate and computational complexity. The classification algorithm 3 is fast since there is a closed form solution to a regularized least square problem. However, the outlier detection algorithm is an iterative algorithm and an l 1  minimization problem is solved in each iteration. 
     It can be shown that the estimated support of the l 1  problem through the iterations does not significantly change. In other words, the same faces are chosen to reconstruct the test sample through the iterations. After the t th  iteration, the support does not change or converge to a very small number. If this t th  iteration is known, the process can just get the support of the previous one and solve a very fast least squares problem for the next iteration. 
     The t th  iteration denoted by t c  is empirically chosen. The iterations after having a fixed support are defined by: 
     
       
         
           
             
               
                 
                   
                     
                       
                         w 
                         ⋒ 
                       
                       i 
                       
                         ( 
                         
                           t 
                           + 
                           1 
                         
                         ) 
                       
                     
                     = 
                     
                       
                         
                           ϕ 
                           ′ 
                         
                         ⁡ 
                         
                           ( 
                           
                             δ 
                             i 
                             t 
                           
                           ) 
                         
                       
                       
                         δ 
                         i 
                         t 
                       
                     
                   
                   , 
                   
                     
 
                   
                   ⁢ 
                   
                     i 
                     = 
                     1 
                   
                   , 
                   … 
                   ⁢ 
                   
                       
                   
                   , 
                   d 
                 
               
               
                 
                   ( 
                   19 
                   ) 
                 
               
             
             
               
                 
                   
                     a 
                     
                       ( 
                       
                         t 
                         + 
                         1 
                       
                       ) 
                     
                   
                   = 
                   
                     ( 
                     
                       
                         
                           
                             
                               
                                 
                                   min 
                                   a 
                                 
                                 ⁢ 
                                 
                                   
                                      
                                     
                                       
                                         W 
                                         
                                           t 
                                           + 
                                           1 
                                         
                                       
                                       ⁡ 
                                       
                                         ( 
                                         
                                           y 
                                           - 
                                           Ta 
                                         
                                         ) 
                                       
                                     
                                      
                                   
                                   2 
                                   2 
                                 
                               
                               + 
                               
                                 λ 
                                 ⁢ 
                                 
                                   
                                      
                                     a 
                                      
                                   
                                   1 
                                 
                               
                             
                             , 
                           
                         
                         
                           
                             
                               if 
                               ⁢ 
                               
                                   
                               
                               ⁢ 
                               t 
                             
                             &lt; 
                             
                               t 
                               c 
                             
                           
                         
                       
                       
                         
                           
                             
                               
                                 min 
                                 a 
                               
                               ⁢ 
                               
                                 
                                    
                                   
                                     
                                       W 
                                       
                                         t 
                                         + 
                                         1 
                                       
                                     
                                     ⁡ 
                                     
                                       ( 
                                       
                                         y 
                                         - 
                                         
                                           
                                             T 
                                             ~ 
                                           
                                           ⁢ 
                                           a 
                                         
                                       
                                       ) 
                                     
                                   
                                    
                                 
                                 2 
                                 2 
                               
                             
                             , 
                           
                         
                         
                           otherwise 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   20 
                   ) 
                 
               
             
           
         
       
     
     where the dictionary Tε   d×n  is given by,
 
 T=TM   (21)
 
     with Mε   n×n  being a diagonal matrix where its elements i, i are given by, 
     
       
         
           
             
               
                 
                   
                     M 
                     
                       i 
                       , 
                       i 
                     
                   
                   = 
                   
                     ( 
                     
                       
                         
                           
                             
                               0 
                               , 
                             
                           
                           
                             
                               
                                 if 
                                 ⁢ 
                                 
                                     
                                 
                                 ⁢ 
                                 
                                   
                                     a 
                                     ^ 
                                   
                                   i 
                                   
                                     ( 
                                     
                                       
                                         t 
                                         c 
                                       
                                       - 
                                       1 
                                     
                                     ) 
                                   
                                 
                               
                               = 
                               0 
                             
                           
                         
                         
                           
                             
                               1 
                               , 
                             
                           
                           
                             otherwise 
                           
                         
                       
                       , 
                     
                   
                 
               
               
                 
                   ( 
                   22 
                   ) 
                 
               
             
           
         
       
     
     where â i   (t     c     −1)  is the i th  component of the estimated vector a (t     c     −1)  at the t c −1 iteration. 
     The description of the FAST-LRR-HQ_L1 algorithm is summarized in Algorithm 5 in Table 5. 
     
       
         
           
               
             
               
                 TABLE 5 
               
               
                   
               
               
                 Algorithm 5: FAST-LRR-HQ_L1 algorithm 
               
               
                 Algorithm The Fast LRR-HQL1 Algorithm 
               
               
                   
               
             
            
               
                   
               
            
           
           
               
               
               
            
               
                   
                  1: 
                 function FAST-LLRHQL1(y, T, t c ) 
               
               
                   
                  2: 
                  Calculate y c  = LRR(y, T) 
               
               
                   
                  3: 
                  Initialize δ i   1  = [y c  − y] i  and t = 1 
               
               
                   
                  4: 
                  repeat 
               
               
                   
                  5: 
                   
             w   ^     i     (   t   )       =         ϕ   ′     ⁡     (     δ   i   t     )         δ   i   t         ,     i   =   1     ,   …   ⁢           ,   n       
 
               
               
                   
                  6: 
                   if &gt;= t c  then 
               
               
                   
                  7: 
                    if t = t c  then 
               
               
                   
                  8: 
                     Construct and Store {tilde over (T)} according to (19) 
               
               
                   
                  9: 
                    end if 
               
               
                   
                 10: 
                    â (t)  = argmin a ||W t (y − {tilde over (T)}a)|| 2   2   
               
               
                   
                 11: 
                   else 
               
               
                   
                 12: 
                    â (t)  = argmin a ||W t (y − Ta)|| 2   2  + λ||a|| 1   
               
               
                   
                 13: 
                   end if 
               
               
                   
                 14: 
                   t = t + 1 
               
               
                   
                 15: 
                   δ i   t  = [Ta t − 1  − y] i   
               
               
                   
                 16: 
                  until 
                    w   t     -     w     t   -   1              2              w   t          2       &lt;       ε   w     ⁢           ⁢   or   ⁢           ⁢   t     &gt;     ε   i         
 
               
               
                   
                 17: 
                  Return W 
               
               
                   
                 18: 
                 end function 
               
               
                   
               
            
           
         
       
     
     Several seconds per image might be required for an l 1  estimator, even with small dictionaries. Thus, a large drawback of the l 1  minimization is the computational complexity. 
     On the other hand, l 2  minimization is very fast. Furthermore, although the l 2  solution is dense, the highest peaks are similar to the l 1  solution and correspond to the training images that match the identity of the test image. The l 2  has a similar shape to l 1  and is likely to serve as a good approximation. This correlation is largely related to the fact that both obtain global solutions on similar error functions with different norm constraints. 
     The Algorithm 5 can be speeded up by substituting the l 1  problem in line-12 with a regularized least squares problem. Then, the smaller values of the estimated dense vector can be supressed to zero according to a criteria. This approximated l 1  process can be written in two steps. 
     1. the regularized least square problem is solved by:
 
ν=argmin ν   ∥y−Tν∥   2   2   +λ∥a∥   2   2 ,  (23)
 
     where νε   d . 
     2. few values of the estimated vector ν is supressed to zero according to a criteria to obtain the final estimated vector a as needed in line-12 of Algorithm 5, 
     
       
         
           
             
               
                 
                   
                     
                       a 
                       ^ 
                     
                     i 
                   
                   = 
                   
                     ( 
                     
                       
                         
                           
                             0 
                             , 
                           
                         
                         
                           
                             
                               if 
                               ⁢ 
                               
                                   
                               
                               ⁢ 
                               
                                 
                                   v 
                                   ^ 
                                 
                                 i 
                               
                             
                             &lt; 
                             
                               v 
                               _ 
                             
                           
                         
                       
                       
                         
                           
                             
                               v 
                               i 
                             
                             , 
                           
                         
                         
                           otherwise 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   24 
                   ) 
                 
               
             
           
         
       
     
     where  ν  denotes the mean of the estimated vector ν. 
     Returning to  FIG. 4 , after face recognition of the video is completed, the video with the recognized face images is outputted (S 412 ). For example, the video with recognized face may be displayed to the user, or the video with recognized face may be provided to another program or device for further processing. 
     By using the disclosed systems and methods, many advantageous applications can be implemented. The face recognition techniques may be applied to the news video domain where faces of key persons in a video can be automatically detected and recognized to facilitate various multimedia management tasks, such as news video summarization, retrieval and browsing. 
     It is understood that the disclosed face recognition model is not limited to TV usage scenario. The disclosed systems and methods can also be applied to any devices with displays, such as smart phones, tablets, PCs, smart watches, and so on. The disclosed methods can also be applied as the core function for other systems, such as content-based video recommendation systems, content-based video retrieval systems, and so on. 
     Further, although the face recognition method is disclosed for illustrative purposes, similar concept and approach can be applied to other recognition systems, for example, image recognition, etc. Other applications, advantages, alternations, modifications, or equivalents to the disclosed embodiments are obvious to those skilled in the art.