Abstract:
A video image tracking apparatus and a video image tracking method are provided. The method makes it possible to detect and track a target image having a variety of angles without a multi-view detector, easily adapt to addition of a new target image and removal of an existing target image, reduce calculation time and memory consumption for detecting and tracking the target image having a variety of angles, so that embedded software or chip can be realized, and tracking the target image at high speed.

Description:
CROSS-REFERENCE TO RELATED PATENT APPLICATIONS 
       [0001]    This application claims the benefit of Korean Patent Application No. 10-2007-0011122, filed on 2 Feb. 2007, in the Korean Intellectual Property Office, the disclosure of which is incorporated herein in its entirety by reference. 
       BACKGROUND OF THE INVENTION 
       [0002]    1. Field of the Invention 
         [0003]    The present invention relates to a method and apparatus for tracking a video image, and more particularly, to a method and apparatus for tracking a video image which perform the 3As (auto focusing, auto white balance, and auto exposure) using a face image captured by a digital camera, a camcorder, and a cellular phone. 
         [0004]    2. Description of the Related Art 
         [0005]    As image processing technology has developed, a variety of technologies for detecting and tracking faces are being developed. Since portable image taking devices have limited size, power, and computing resources, but need to perform real-time processing, systems for detecting and tracking faces adapted to portable image taking devices are required. 
         [0006]    In “Robust Real-time Object Detection (2001)” by Viola and Jones, a method of detecting a person&#39;s full face in real time using a discriminative boosting technique is disclosed. However, a full face detector is too limited to find out various angles of a face due to a difference between a full face and a facial profile. 
         [0007]    In “Vector Boosting for Rotation Invariant Multi-view Face Detection (2005)” by Chang Huang, a multi-view detection system for detecting a multi-view face using a vector boosting technique is disclosed. However, calculation time and memory consumption increase in the multi-view detection system, which is limited to detect and track a moving target. 
         [0008]    In “Kernel-Based Object Tracking (2003)” by Dorin Comaniciu, a mean shift-based tracking method is disclosed. However, since the method uses kernel calculation and increases the complexity of calculation of similarity and tracking location, the method is not used to detect a target in real time and at high speed, and fails to track a new target. 
       SUMMARY OF THE INVENTION 
       [0009]    The present invention provides a method and apparatus for tracking a video image, which makes it possible to detect and track a target image having a variety of angles without a multi-view detector, easily add a new target image and remove an existing target image, reduce calculation time and memory consumption for detecting and tracking the target image having a variety of angles, thereby being realized as embedded software or chip, and tracking the target image at high speed. 
         [0010]    According to an aspect of the present invention, there is provided a video image tracking method comprising: tracking a target model and determining a target candidate of a tracked frame; detecting a target image from the tracked frame or a frame subsequent to the tracked frame; and renewing the target model using the target candidate or the target image and initializing tracking. 
         [0011]    The renewing of the target model may comprise: if an overlapping region between the target candidate and the target image is greater than a predetermined reference value, removing the target candidate and renewing the target model using the target image. 
         [0012]    The tracking of the target model may comprise: calculating a similarity or distance between the statistical distribution characteristic of the target model and a statistical distribution characteristic of a target candidate identified as a result of tracking a frame previous to the tracked frame, modifying the location of the target candidate based on the target model and the statistical distribution characteristic of the target candidate, calculating a similarity or distance between the statistical distribution characteristic of the target model and a statistical distribution characteristic of the target candidate according to the modified location of the target candidate, and performing tracking using the similarity or the distance. 
         [0013]    According to another aspect of the present invention, there is provided a video image tracking apparatus comprising: a tracking unit tracking a target model and determining a target candidate of each frame; a detector detecting a target image at predetermined frame intervals; and a controller renewing the target model using the target candidate determined by the tracking unit and the target image detected by the detector, and initializing tracking. 
         [0014]    The tracking unit may comprise: a tracking location determiner determining the target candidate in a frame to be tracked based on a statistical distribution characteristic of the target model; and a histogram extractor extracting a histogram reflecting a statistical distribution characteristic of the target candidate determined by the tracking location determiner. 
         [0015]    The controller may comprise: a scheduler managing a tracking process performed by the tracking unit and a detecting process performed by the detector; and a combiner combining the target candidate and the target image and renewing the target model. 
         [0016]    According to another aspect of the present invention, there is provided a computer readable recording medium having embodied thereon a computer program for executing the video image tracking method. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0017]    The above and other features and advantages of the present invention will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings in which: 
           [0018]      FIG. 1  is a block diagram of a video image tracking apparatus according to an embodiment of the present invention; 
           [0019]      FIG. 2  is a diagram illustrating tracking video images obtained by the video image tracking apparatus as illustrated in  FIG. 1 , according to an embodiment of the present invention; 
           [0020]      FIG. 3  is a diagram illustrating a combination video image obtained by the apparatus for tracking the video image as illustrated in  FIG. 1 , according to an embodiment of the present invention; 
           [0021]      FIG. 4  is a flowchart of a video image tracking method according to an embodiment of the present invention; and 
           [0022]      FIG. 5  is a detailed flowchart of Operation  500  as illustrated in  FIG. 4 , according to an embodiment of the present invention. 
       
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
       [0023]    The present invention will now be described more fully with reference to the accompanying drawings, in which exemplary embodiments of the invention are shown. 
         [0024]      FIG. 1  is a block diagram of a video image tracking apparatus according to an embodiment of the present invention. Referring to  FIG. 1 , the video image tracking apparatus includes a tracking unit  10 , a detector  20 , and a controller  30 . 
         [0025]    The tracking unit  10  tracks a predetermined target model to determine a target candidate based on a current frame, i.e., an n th  frame. The tracking is repeated by a specific number of times until the tracking unit  10  determines a final target candidate according to the current frame. 
         [0026]    In the present embodiment, the predetermined target model tracked by the tracking unit  10  is a sub-image or its histogram determined by tracking initialization at a frame previous to the current frame. The tracking initialization is carried out at regular intervals of frames starting from a frame from which an initial target image is detected. If an initial target image is detected, the detection result leads to the tracking initialization. However, if a subsequent target image is detected, a combination of tracking and detection results leads to the tracking initialization. For example, a target model may be a detected face image, i.e., an image having a region including a face. Further, the target candidate results from the repetitive tracking within the current frame, and is an image identified by a specific location and size. 
         [0027]    The tracking unit  10  includes a tracking location determiner  11 , a histogram extractor  12 , a comparator  13 , a weight regulator  14 , and a scale regulator  15 . 
         [0028]    The tracking location determiner  11  determines the location of a sub window identifying the target candidate in frame-unit image information. The frame-unit image information is received from an image information receiver  31 . In the present embodiment, since the sub window is identified by its center location y and half width h, the identified sub window leads to identification of the target model that is part of an entire frame image. 
         [0029]    If a target or a video image taking device moves, the size and location of the sub window identifying the target candidate vary in each frame. The tracking location determiner  11  identifies the sub window in each frame using inputs received from the histogram extractor  12 , the comparator  13 , the weight regulator  14 , the scale regulator  15 , and a scheduler  32  whenever the tracking is carried out. For example, after a photography mode of the video image starts, the tracking location determiner  11  tracks an initial face model in a frame subsequent to a frame at which tracking is initialized based on the initial face model to determine a face candidate. 
         [0030]    The initial face model is an initially detected face image or a color histogram of the initially detected face image as a first frame or a frame subsequent to the first frame. The detector  20  detects the initial face model. The scheduler  32  stores a result of the detection by the tracking initialization. The tracking location determiner  11  tracks a target of a current frame, i.e., the location of a face, based on a location and a histogram of the detected face model. 
         [0031]    If the tracking is carried out at least once, the tracking location determiner  11  calculates the center location y and half width h for identifying the target candidate of the current frame using a result of the calculation of the comparator  13  or the weight regulator  14 , and determines an image identified by the center location y and half width h as the target candidate of the current frame. 
         [0032]    The histogram extractor  12  extracts a histogram reflecting statistical distribution characteristics of the target candidate identified by the tracking location determiner  11 . The histogram extractor  12  extracts a histogram reflecting statistical distribution characteristics of an initialized target candidate stored by the scheduler  32 . An example of the histogram in the present embodiment is the color histogram or an edge histogram. The histogram extractor  12  calculates the color histogram of the target model according to equation 1 below, 
         [0000]    
       
         
           
             
               
                 
                   
                     q 
                     u 
                   
                   = 
                   
                     
                       ∑ 
                       
                         i 
                         = 
                         1 
                       
                       n 
                     
                      
                     
                       δ 
                        
                       
                         [ 
                         
                           
                             b 
                              
                             
                               ( 
                               
                                 x 
                                 i 
                               
                               ) 
                             
                           
                           - 
                           u 
                         
                         ] 
                       
                     
                   
                 
               
               
                 
                   1 
                   ) 
                 
               
             
           
         
       
     
         [0033]    wherein x i  denotes (“the number of”?) a plurality of pixels forming the target model, b(x i ) denotes a bin value of each pixel, u denotes colors of the pixels, and q u  denotes a color histogram according to each of the colors u of the pixels. {q u } denotes a set of pixels having the colors u among the plurality of pixels forming the target model. {q u } reflects critical statistical distribution characteristics reflecting the features of the target model, and can be briefly calculated according to equation 2 below, 
         [0000]      { q   u } u=1 . . . m =histogram( r&gt;&gt; 4, g&gt;&gt; 4, b&gt;&gt; 4)  2) 
         [0034]    wherein, q u  denotes a histogram of the target model, r&gt;&gt;4, g&gt;&gt;4, and b&gt;&gt;4 denote left-shifting of r, g, and b, respectively, and m denotes 16×16×16. In more detail, q u  denotes the histogram obtained by dividing the r, g, and b by 2 4 . 
         [0035]    Pixel colors are generally expressed as RGB values in the range 0˜255, which increases the complexity of calculation and processing time. However, to address this problem, the present invention lowers the degree of dispersion of the RGB values and expresses pixel colors using a new color variable u. For example, a three-dimensional RGB color obtained by dividing the r, g, and b values by 2 4  and summing the divided r, g, and b values according to a predetermined weight is a color u having a primary value, and this procedure lowers the complexity of a calculation. Further, a probability density function (PDF) according to the target model can be used as q u . When the PDF is used as q u , q u  satisfies the equation 
         [0000]    
       
         
           
             
               
                 ∑ 
                 
                   u 
                   = 
                   1 
                 
                 m 
               
                
               
                 q 
                 u 
               
             
             = 
             1. 
           
         
       
     
         [0000]    As with the target model, a histogram of the target candidate can be calculated according to equation 3 below, 
         [0000]      { p   u ( y   0   ,h   0 )} u=1 . . . m =histogram( r&gt;&gt; 4, g&gt;&gt; 4, b&gt;&gt; 4)  3) 
         [0036]    wherein, {p u (y 0 , h 0 )} denotes the histogram of the target candidate where a color value is u, a center coordinate is y 0 , and a half width is h 0 . 
         [0037]    The comparator  13  calculates histogram similarities and compares the calculated similarities. In particular, the comparator  13  performs a comparison to determine if a predetermined target model is similar to a first target candidate or a second target candidate of the current frame. The first target candidate is obtained as a result of first tracking of the current frame (n th  frame). The second target candidate is obtained as a result of second tracking of the current frame (n th  frame). 
         [0038]    The comparator  13  calculates a first similarity between color histograms of the first target candidate and the target model, calculates a second similarity between color histograms of the second target candidate and the target model, compares the calculated first and second similarities, and selects one of the first target candidate and the second target candidate that maximizes a tracking hit rate as the target candidate of the current frame. 
         [0039]    For example, if the first similarity between color histograms of the first target candidate and the target model is smaller than the second similarity, the first target candidate is deleted and then the second target candidate is determined to be the target candidate of the current frame. If the current frame tracking is carried out, the comparator  13  compares the first and second target candidates and a third target candidate, and selects one of the first, second, and third target candidates that has the greatest similarity to the target model as the final target candidate of the current frame. If the first similarity between color histograms of the first target candidate and the target model is greater than the second similarity, the second target candidate is deleted and then the first target candidate is selected as the target candidate of the current frame. In this regard, since it is inefficient and unnecessary to track an additional target candidate, the current frame tracking is no longer carried out. 
         [0040]    If similarity between the target candidate determined as a result of the final current frame tracking and the target model is smaller than a predetermined value, a current target model is deleted, and the current target model tracking is no longer carried out in a subsequent frame. For example, if one person among a plurality of people existing in a previous frame disappears, tracking of the face of the person that disappeared is no longer carried out. 
         [0041]    The target candidate is determined based on similarities between histograms as described above. However, the target candidate can be determined using distances between histograms. Distances between histograms can be calculated using an L1 distance function according to equation 4 below, 
         [0000]    
       
         
           
             
               
                 
                   
                     d 
                      
                     
                       ( 
                       y 
                       ) 
                     
                   
                   = 
                   
                     
                       
                         ∑ 
                         
                           u 
                           = 
                           1 
                         
                         m 
                       
                        
                       
                          
                         
                           
                             
                               
                                 p 
                                 u 
                               
                                
                               
                                 ( 
                                 y 
                                 ) 
                               
                             
                             
                               N 
                               
                                 p 
                                  
                                 
                                   ( 
                                   y 
                                   ) 
                                 
                               
                             
                           
                           - 
                           
                             
                               q 
                               u 
                             
                             
                               N 
                               q 
                             
                           
                         
                          
                       
                     
                     = 
                     
                       
                         1 
                         
                           
                             N 
                             
                               p 
                                
                               
                                 ( 
                                 y 
                                 ) 
                               
                             
                           
                            
                           
                             N 
                             q 
                           
                         
                       
                        
                       
                         
                           ∑ 
                           
                             u 
                             = 
                             1 
                           
                           m 
                         
                          
                         
                            
                           
                             
                               
                                 N 
                                 q 
                               
                                
                               
                                 
                                   p 
                                   u 
                                 
                                  
                                 
                                   ( 
                                   y 
                                   ) 
                                 
                               
                             
                             - 
                             
                               
                                 N 
                                 
                                   p 
                                    
                                   
                                     ( 
                                     y 
                                     ) 
                                   
                                 
                               
                                
                               
                                 q 
                                 u 
                               
                             
                           
                            
                         
                       
                     
                   
                 
               
               
                 
                   4 
                   ) 
                 
               
             
           
         
       
     
         [0042]    wherein, d(y) denotes a distance between the target model and the target candidate, N q  denotes the number of pixels of the target model, N p (y) denotes the number of pixels of the target candidate, p u (y) denotes a color histogram of the target candidate, and q u  denotes a color histogram of the target model. 
         [0043]    The weight calculator  14  calculates weights of all pixels belonging to the target candidate using the comparison result of the comparator  13 . The tracking location determiner  11  calculates a new center location y 1  from the center location y 0  using the calculated weights according to equation 5 below, 
         [0000]    
       
         
           
             
               
                 
                   
                     y 
                     1 
                   
                   = 
                   
                     
                       
                         ∑ 
                         
                           i 
                           = 
                           1 
                         
                         
                           n 
                           
                             h 
                              
                             
                                 
                             
                              
                             0 
                           
                         
                       
                        
                       
                         
                           w 
                           i 
                         
                          
                         
                           x 
                           i 
                         
                       
                     
                     
                       
                         ∑ 
                         
                           i 
                           = 
                           1 
                         
                         
                           n 
                           
                             h 
                              
                             
                                 
                             
                              
                             0 
                           
                         
                       
                        
                       
                         w 
                         i 
                       
                     
                   
                 
               
               
                 
                   5 
                   ) 
                 
               
             
           
         
       
     
         [0044]    wherein, N h0  denotes the total number of pixels of a tracking candidate model, and y 1  denotes a center coordinate of a tracking candidate modified according to a weight w i . The center coordinate of the tracking candidate is modified according to the definition of the weight w i . There is no particular restriction to a weight determining method. For example, when a face is tracked, a high weight is provided to a region of high frequency of a value u corresponding to complexion of the face on a histogram in order to move the center location y 0  to the center location y 1  of the high frequency region corresponding to the complexion. In more detail, the weight calculator  14  calculates the weight according to equation 6 below, 
         [0000]        v   i =(Log( q   u )−Log( p   u ( y )))&gt;&gt;1 
         [0000]        s   i =min(max( v   i ,−5),5) 
         [0000]        w   i =1&lt;&lt; s   i   6) 
         [0045]    wherein, w i  denotes a weight of each pixel, a Log( ) function denotes a function rounding off a log 2 ( ) value, i denotes a coordinate of a pixel, which is identified by a half width h 0 , and 1&lt;&lt;s i  denotes 2 si . (Drafter: Is this correct?) Equation 6 is used to calculate the weight w i  using p u  and q u (y) having the center location y and the color value u of the pixel coordinate i. In particular, since the weight w i  is an integral and can be calculated using a relatively easy operation in equation 6, equation 6 is suitable for calculating a weight in an embedded system. 
         [0046]    The scale regulator  15  regulates a scale of the target candidate. When a distance between a video image tracking device and a person changes, it is necessary to regulate the scale in order to increase a hit rate in face tracking. The scale regulator  15  regulates the scale through regulation of a half width h. As an example of regulating the scale, if an original half width is denoted h 0 , the scale regulator  15  regulates the scale of the target candidate using different half widths h 1 , and h 2  like h 1 =1.1h 0 , h 2 )=0.90h 0 . 
         [0047]      FIG. 2  is a diagram illustrating tracking video images obtained by the video image tracking apparatus as illustrated in  FIG. 1 , according to an embodiment of the present invention. Referring to  FIG. 2 , a video image “a” (of a previous frame) and a video image “b” (of a current frame) of two adjacent frames are obtained by an image obtaining apparatus such as a digital camera or a camcorder, in particular, an image obtaining apparatus having a tracking function. 
         [0048]    In image “a”, y 0  denotes a center location of a target candidate determined as a result of final tracking of the previous frame, and h 0  denotes a half width of the target candidate. The target candidate of the video image “a” is an image in a region identified by a sub window. However, the video image “b” is obtained as a result of incomplete tracking of a target model. The tracking for determining the target candidate is repeated in the video image “b” of the current frame several times within a limited number of times. 
         [0049]    Initial tracking of the video image “a” is carried out based on the same sub window condition, i.e. y 0  and h 0 , of the target candidate determined in the video image “a” of the previous frame. A color histogram that is extracted from the target candidate determined through the sub window and a color histogram that is extracted from a predetermined target model can be used to calculate the weight w i  and the new center location y 1  according to equations 5 and 6. 
         [0050]    The comparator  13  calculates a first similarity between color histograms of the first target candidate and the target model based on the sub window condition y 0 , h 0 , calculates a second similarity between color histograms of the second target candidate and the target model based on a new window condition y 1 , h 0 , compares the calculated first and second similarities, and selects one of the first target candidate and the second target candidate that has the greatest similarity to the target model as the target candidate of the video image b. 
         [0051]    In  FIG. 2 , the target candidate is selected based on the new window condition y 1 , h 0  instead of the sub window condition y 0 , h 0 . The weight calculator  14  calculates a new weight using values of the color histogram extracted from the selected target candidate and the color histogram extracted from the target model. The tracking location determiner  11  calculates a center location y 2  of a new sub window, h 0  using the new weight and the center location y 1  of the current sub window. The tracking location determiner  11  selects one of a third target candidate identified by the new sub window having coordinates y 2 , h 0  and the second target candidate identified by the new window condition y 1 , h 0  which has the greatest similarity to the predetermined target model. If the current frame tracking is complete, similarity between the finally selected target candidate and the target model is greater than a predetermined reference value, and the target model tracking continues. However, if the similarity therebetween is smaller than the predetermined reference value, the target model tracking no longer continues. 
         [0052]    The detector  20  detects a target image from the video image. Taking into account the time required to detect the target image, the target image may be detected at intervals of a predetermined number of frames, e.g., 15 frames. 
         [0053]    The controller  30  combines the target candidate identified by the tracking location determiner  10  and the target image detected by the detector  20  and renews the target model. Further, the controller  30  controls performance of the current frame tracking or detection of the target image, finishes the current frame tracking, and controls performance of next frame tracking. 
         [0054]    The controller  30  comprises the image information receiver  31 , the scheduler  32 , and a combiner  33 . The image information receiver  31  receives image information from an image obtaining means. The scheduler  32  schedules whether to perform the current frame tracking or detect the target image. The scheduler  32  also initializes tracking according to a combination image obtained by the combiner  33 . The target model is renewed by the tracking initialization. The combiner  33  combines the target candidate determined by the tracking unit  10  and the target image detected by the detector  20 . 
         [0055]      FIG. 3  is a diagram illustrating a combination video image obtained by the video image tracking apparatus as illustrated in  FIG. 1 , according to an embodiment of the present invention. Referring to  FIG. 3 , a tracking video image obtained by the tracking unit  10  includes four square sub windows that identify locations of target candidates. Video image tracking is performed per frame by a predetermined target model. Thus, when a new target that is not included in a previous frame appears on a current frame, it is impossible to track the new target in the current frame. Further, though a full face can be detected relatively accurately, it is difficult and takes much time to detect a facial profile. Thus, it is impossible to perform video image tracking per frame. In the present embodiment, tracking disadvantages are overcome by a combination of detection and tracking images. 
         [0056]    In  FIG. 3 , a detection video image includes a target image detected using a full face detector for detecting a full face of the current frame. Although four face images are tracked in the tracking video image, the center two face images are not detected in the detection video image. A multi-view face detector capable of detecting the full face and the facial profile can detect the center two face images. However, since the multi-view detector needs a long detection time and consumes a lot of memory, it is difficult to operate the multi-view detector in real time. The tracking disadvantages can be overcome if targets in a video image are tracked and simultaneously detected using the full face detector and tracking and detection video images are combined. 
         [0057]    Four face images in boxes in the tracking video image are target candidates of the current frame. Two face images in boxes of the detection video image are target images of the current frame. A right face image includes a target candidate consisting of a region partially overlapping the target image. If the partially overlapped region is greater than a predetermined reference value, the target candidate is removed. The combination video image includes the center two face images that are not detected in the tracking of targets in the video image and excludes both edge face images which are detected in the tracking video image. Tracking is initialized according to the combination video image and then the frame tracking is carried out according to the target model identified by the tracking initialization and the sub windows. In detail, the existing target model is applied to the center two face images and tracking for a subsequent frame is carried out based on the center location and the half width y, h. In the combination video image, the previously tracked images are removed from both edge face images and new target models are determined based on currently detected images. Center location and scale information of each of the target models are transferred to the tracking location determiner  11  through the scheduler  32 . The tracking location determiner  11  performs tracking for the target models using sub windows of a previous frame. The tracking, detection, and combination process is repeated until a photography mode ends. If an overlapping region between a target candidate of a specific person and target models is smaller than the predetermined reference value, the target candidate and the target model are maintained and tracking for each target model is carried out in the subsequent frame. In detail, the tracking is carried out for two different target models extracted from one person&#39;s face image. However, repetitive tracking unites the two different target models, resulting in one target model for one person. 
         [0058]    A video image tracking method of the present invention will now be described in detail with reference to  FIGS. 4 and 5  and their embodiments. 
         [0059]      FIG. 4  is a flowchart of a video image tracking method according to an embodiment of the present invention. Referring to  FIG. 4 , the video image tracking method of the present embodiment is performed in time series by a video image tracking apparatus. 
         [0060]    If a photography mode starts, the detector  20  detects a target image in a video image of a first frame received from the image information receiver  31  (Operation  100 ). An example of the target image is a face image that is described in the present embodiment. 
         [0061]    The scheduler  32  determines whether the target image is detected (Operation  200 ). If the scheduler  32  determines that the target image is not detected, the detector  20  detects the target image from a video image of a next frame. 
         [0062]    If the scheduler  32  determines that the target image is detected, the scheduler  32  determines the detected target image as a target model, and initializes tracking (Operation  300 ). The tracking initialization means identification of a center coordinate y 0  and a half width h 0  of a sub window. If a new target appears, the tracking initialization includes (calculation of a histogram from the new target. The histogram extractor  12  extracts a color histogram or an edge histogram from the target model and stores the color histogram or the edge histogram. 
         [0063]    The image information receiver  31  retrieves video image information of each frame (Operation  400 ). count ++  denotes an increase of a frame number by 1. 
         [0064]    The tracking location determiner  11  determines a target candidate of each frame (Operation  500 ). The determination of the target candidate means determination of the location of the target candidate, i.e. information (y, h) of the sub window. 
         [0065]      FIG. 5  is a detailed flowchart of Operation  500  as illustrated in  FIG. 4 , according to an embodiment of the present invention. Referring to  FIG. 1 , the histogram extractor  12  extracts a histogram of the target candidate (a first target candidate) according to the information (y 0 , h 0 ) of the sub window from a video image of a second frame (Operation  502 ). In detail, the histogram extractor  12  extracts a histogram of the target candidate from the same location as the target model in a first frame. When tracking is performed without tracking a previous frame, the histogram extractor  12  extracts a histogram of the target candidate of a current frame from a specification location as a result of tracking the previous frame. 
         [0066]    The comparator  13  calculates first similarity between histograms of the target model and the first target candidate (Operation  504 ). The target model and the first target candidate are within the same sub window. However, the target model and the first target candidate are different from each other in that the target model is an image identified in the first frame, whereas the first target candidate is an image identified in a second frame. 
         [0067]    The weight regulator  14  calculates a first weight according to equation 6 using the histograms of the target model and the first target candidate (Operation  506 ). 
         [0068]    The tracking location determiner  11  calculates a new center coordinate y 1  according to equation 5 using the first weight and the center coordinate y 0  of the sub window (Operation  508 ). 
         [0069]    The histogram extractor  12  extracts a histogram of a second target candidate identified by coordinates (y 1 , h 0 ) from a video image of the second frame (Operation  510 ). 
         [0070]    The comparator  13  calculates second similarity between histograms of the target model and the second target candidate (Operation  512 ). 
         [0071]    The comparator  13  compares the first and second similarities (Operation  514 ). If the second similarity is greater than the first similarity, the first target candidate is removed, and a subsequent tracking process follows location and scale of the second target candidate. Similarity and distance between the histograms have an inverse relationship. The comparator  13  calculates the distance between the histograms according to equation 4. If it is satisfied that d(y 0 , h 0 )&gt;d(y 1 , h 0 ), the tracking location determiner  11  performs tracking based on the coordinates (y 1 , h 0 ). However, if it is satisfied that d(y 0 , h 0 )&gt;d(y 1 , h 0 ), since the distance between the first target candidate and the target model is shorter than that between the second target candidate and the target model, the second target candidate is removed, tracking for the current frame ends, and tracking for subsequent frames is performed based on the location of the first target candidate. 
         [0072]    The scale regulator  14  regulates the scale of the target candidates, and the tracking location determiner  11  determines a new target candidate according to a newly regulated scale (Operation  516 ). The histogram extractor  12  extracts a color histogram from the new target candidate having a regulated scale. 
         [0073]    The tracking location determiner  11  selects a pair of coordinates (y, h) having the maximum similarity value, and calculates a new pair of coordinates (y 0 , h 0 ) using the selected coordinates (y, h) (Operation  518 ). For example, if h 1 =1.1 h 0  (10% scale up), and h 2 =0.9 h 0  (10% scale down), the tracking location determiner  11  calculates d(y 1 , h 1 ) and d(y 1 , h 2 ) and then calculates d min  of one of d(y 1 , h 0 ), d(y 1 , h 1 ), and d(y 1 , h 2 ) which has the minimum center coordinate and half width. If d min =d(y 1 , h 0 ), then h 0 =h 0 . If d min =d(y 1 , h 1 ), then h 0 =r 1 h 1 +(1−r 1 )h 0 . If d min =d(y 1 , h 2 ), then h 0 =r 2 h 2 +(1−r 2 )h 0 . r1 and r2 are weights of center coordinates corresponding to the center coordinate h 0  and d min  according to previous tracking. For example, r 1  and r 2  can be set such that r 1 =0.8, and r 2 =0.2. 
         [0074]    The scheduler  32  compares a tracking repetition number t of the current frame and a predetermined iteration value and determines whether the tracking unit  10  resumes tracking for the current frame, or the tracking unit  10  ends the tracking for the current frame and performs tracking for a next frame (Operation  520 ). 
         [0075]    The scheduler  32  divides a number of the current frame by a definite number and determines whether a remainder is 0 (Operation  600 ). For example, when frames are detected at 15 frame intervals, the scheduler  32  divides the number of the current frame by 15 and determines whether the remainder is 0. If the remainder is 0, Operation  700  is performed. If the remainder is not 0, Operation  400  is performed. In other words, the detector  20  detects the target model every 15n frames (n is a positive number). 
         [0076]    The detector  20  detects the target image from a tracked frame or a frame subsequent to the tracked frame (Operation  700 ). When a full face detector is used as the detector  20  in the present embodiment, the detector  20  detects a full face every 15 frames. Although the detector  20  does not detect a facial profile, the tracking unit  10  can capture it. 
         [0077]    The combiner  33  combines a tracked image and a detected image (Operation  800 ). The combination is described with reference to  FIG. 3  and thus its description is not repeated. 
         [0078]    The scheduler  32  determines whether the photography mode ends (Operation  900 ). If the photography mode ends, the tracking process is complete. If the photography mode does not end, Operations  300  through  800  are repeated. 
         [0079]    The present invention can also be embodied as computer readable code on a computer readable recording medium. The computer readable recording medium is any data storage device that can store data which can be thereafter read by a computer system. Examples of the computer readable recording medium include read-only memory (ROM), random-access memory (RAM), CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, and carrier waves. The computer readable recording medium can also be distributed network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. Also, functional programs, code and code segments for accomplishing the present invention can be easily construed by programmers skilled in the art to which the present invention pertains. 
         [0080]    The present invention combines a tracking image and a detection image, initializes tracking according to a combination image, and performs further tracking based on the initialized tracking, thereby tracking a face having various angles without a multi-view target detector at high speed, and realizing 3As (auto focusing, auto white balance, and auto exposure) for a face image on a display screen of a next-generation digital still camera (DSC). 
         [0081]    While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention as defined by the following claims. The preferred embodiments should be considered in a descriptive sense only and not for purposes of limitation. Therefore, the scope of the invention is defined not by the detailed description of the invention but by the appended claims, and all differences within the scope will be construed as being included in the present invention.