Abstract:
The present invention relates to a method for object tracking where the tracking is realized based on object classes, where the classifiers of the objects are trainable without a need for supervision and where the tracking errors are reduced and robustness is increased.

Description:
FIELD OF THE INVENTION 
       [0001]    The present invention relates to a method for object tracking where the tracking is realized based on classification of objects. 
       BACKGROUND OF THE INVENTION 
       [0002]    Primitive surveillance systems used to provide users with periodically updated images or motion pictures. As the expectations on a surveillance system increase, the surveillance systems features must improve. For example, higher frame rates and better picture quality are constant goals. In addition to better sensory input, they are enriched with new algorithmic features. For example, motion detection and tracking features have been implemented in these systems. 
         [0003]    There are several ways for achieving object tracking in the state-of-the-art. One of those methods is feature tracking. This method is based on the idea of tacking especially the distinguishing features of the objects to be tacked. However, this method fails to track the target when the target is small (or too far away), or when the image is too noisy. Another method is template matching in which a representative template is saved and used for localizing (using correlation etc.) the object of interest in the following frames. The template is updated from frame to frame in order to adjust to appearance changes. The problem with this approach is its inability to store a wide range of object appearances in a single template, hence its weak representative power of the object. 
         [0004]    Another one of tracking, methods is tracking by classification in which the object of interest and the background constitute two separate classes. 
         [0005]    The abstract titled “An Analysis of Single-Layer Networks in Unsupervised Feature Learning” (Adam Coates et al.) discloses a method for unsupervised dictionary learning and classification based on the learned dictionary. 
         [0006]    The abstract titled “Sparse coding with an overcomplete basis set: A strategy employed by V1?” (Olshausen, B. A., Field, D. J.) discloses usage of sparse representation. 
         [0007]    The articles titled “Support Vector Tracking” (Avidan), “P-N Learning: Bootstrapping Binary Classifiers by Structural constraints” (Kalal et al.). “Robust Object Tracking with Online Multiple Instance Learning” (Babenko et al.), “Robust tracking via weakly supervised ranking SVM” (Bai et al.) disclose methods for classification based tracking of objects. 
         [0008]    The article titled “Visual tracking via adaptive structural local sparse appearance models” (Jia et al.) discloses a method for using sparse representation for target tracking. 
         [0009]    The United States patent application numbered US2006165258 discloses a method for tracking objects in videos with adaptive classifiers. 
         [0010]    Classification based methods, although shown to be more powerful than other approaches, still suffer from drifting caused by image clutter, inability to adjust to appearance changes due to limited appearance representation capacity and sensitivity to occlusion due to lack of false training rejection mechanisms. 
       OBJECTS OF THE INVENTION 
       [0011]    The object of the invention is to provide a method for object tracking where the tracking is realized based on classification of objects. 
         [0012]    Another object of the invention is to provide a method for object tracking where the classifiers of the objects are trainable without a need for supervision. 
         [0013]    Another object of the invention is to provide as method fix object tracking where the tracking errors are reduced and robustness is increased 
         [0014]    Another object of the invention is to provide a method for object tracking where the trained classifiers are stored in a database in order to be reusable. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0015]    A method for object tracking in order to fulfill the objects of the present invention is illustrated in the attached figures, where: 
           [0016]      FIG. 1  is the flowchart of the method object tracking. 
           [0017]      FIG. 2  is the flowchart of the sub-steps of step  103 . 
           [0018]      FIG. 3  is the flowchart of the sub-steps of step  104 . 
       
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
       [0019]    A method for object tracking ( 100 ) comprises the steps of:
       receiving the coordinates (bounding box) of the target in an input, image from the user ( 101 ),   determining if the acquired image is the first image acquired or not ( 102 ),   if the acquired image is the first image acquired then training of a classifier discriminates target from the background ( 103 ),   if the acquired image is not the first image acquired then detecting the target using the classifier that is trained in the step  103 . ( 104 ),   determining if the detection is successful or not ( 105 ),   if the detection is successful then updating the classifier ( 106 ),   if the detection is unsuccessful for a predefined number of consecutive frames then termination of tracking ( 107 ).       
 
         [0027]    In the preferred embodiment of the invention, the step  103  comprises the sub-steps of:
       extracting the feature representation of image patches from an input image ( 201 ),   training a classifier ( 202 ),   determining if the change in the classifier is greater than a predefined value ( 203 ),   if the change in the classifier is greater than a predefined value then rejecting the training output ( 204 ),   if the change in the classifier is not greater than a predefined value then updating the classifier ( 205 ),   comparing the change in the classifier with another predefined value ( 206 ),   if the change in the classifier is greater the said another predefined value, then saving the original classifier in a database ( 207 )       
 
         [0035]    In the preferred embodiment of the invention, the step  104  comprises the sub-steps of;
       using the current classifier for labeling the target patches ( 301 ),   using the classifier that is in the database for labeling the target patches ( 302 ),   comparing the number of patches acquired in the steps  301  and  302  ( 303 ),   if using the current classifier for labeling the target patches produces a bigger number of target patches then using the current classifier as classifier ( 304 ),   if using the classifier that is in the database for labeling the target patches produces a bigger number of target patches by a predetermined ratio then using the classifier that is in the database as classifier ( 305 ),   determining the putative target pixels, which are the centers of each classified target patch ( 306 ),   determining clusters of pixels which are classified to be the target ( 307 ),   assigning the cluster with the closest center to the previously blown target center as the correct cluster ( 308 ).       
 
         [0044]    In the method for object tracking ( 100 ), the coordinates (bounding box) of the target in an input image that is supplied by an imaging unit or a video feed, is acquired from the user ( 101 ). After acquiring the bounding box, the processed image frame is evaluated in order to determine if it is the first image frame or not ( 102 ). If the image is the first image acquired, then there cannot be any classifiers trained for the target that is wanted to be tracked. Hence, a classifier is trained ( 103 ). If the image is not the first image acquired then the target is detected using the classifier that is trained in the step  103  ( 104 ). After detecting the target positions, success of the detection is evaluated ( 106 ). If the detection is successful then the classifier is updated in order to better separate the target from the background ( 107 ). If the detection is unsuccessful for a predefined number of consecutive frames then the tracking is terminated ( 108 ). 
         [0045]    In the preferred embodiment of the invention, the classifier is trained as follows. The feature representation of image patches is extracted from the input image ( 201 ). Afterwards a linear classifier is trained ( 202 ). As the classifier is trained, it is compared with a previously trained classifier ( 203 ). If the change in the trained classifier is greater than a predefined value then the training is ignored and the process is stopped ( 204 ). If the change in the trained classifier is not greater than a predefined value then the classifier is updated ( 205 ). Afterwards, the change in the classifier is compared with another predefined value ( 206 ). If the change in the classifier is greater than the said another predefined value, then the original classifier is saved in a database ( 206 ). As a result, new target appearances are learned and stored, and the appearance database is updated without the need of supervision. 
         [0046]    In the preferred embodiment of the invention, detection is realized as follows: 
         [0047]    Image patches are extracted around the last known location of the target, that are the same size as the target. The sampling scheme of image patch extraction can be adjusted according to the size and speed characteristics of the tracked object. The image patches are labeled using the current classifier that has been trained ( 301 ). The image patches are also labeled using the classifiers that are in the database ( 302 ). Numbers of label of target patches generated in the steps  301  and  302  are then compared ( 303 ). If using the current classifier for labeling the target patches produces a bigger number of target patches, then the current classifier is used as classifier ( 304 ). If one of the classifiers that is in the database produces as bigger number of target patches by a predetermined ratio, then the classifier that is in the database is used as classifier ( 305 ). This ensures that the tracking system remembers a previously stored appearance of the target. Afterwards, the putative target pixels, which are the centers of each classified target patches, are determined ( 306 ). These target pixels are clustered according to their pixel coordinates and the clusters of pixels are determined ( 307 ). The cluster center closest to the previously known target center is then assigned as the correct cluster ( 308 ). Clustering of target pixels and selection of closest cluster avoids drill of target location due to clutter or multiple target instances. In a preferred embodiment of the invention, the number of clusters can be determined by methods such as Akaike Information Criterion (Akaike, 1974). 
         [0048]    In the preferred embodiment of the invention, the deter position of the target is compared with the position of the target in the previous image frame. If the difference between the positions of the target is unexpectedly high or more than one target appears in the latter frame, then the tracking can be evaluated as inconsistent. 
         [0049]    In the preferred embodiment of the invention, once the classifier is trained, it is used for detecting the target by means of distinguishing it from the background. Once the target is detected, its position is updated on the image. In this embodiment, the classifier is further trained in every frame. This periodic training enables plasticity to appearance changes. 
         [0050]    In the preferred embodiment of the invention, multiple instances of the classifier is saved and utilized. This provides the tracker an appearance memory; in which representation of the target is very efficient. 
         [0051]    The step extracting a sparse feature representation of image patches from an input image ( 201 ) provides a representation of the target in a high dimensional feature space, hence the discrimination of target from the background is accurate and robust. 
         [0052]    In the preferred embodiment of the invention, the trained classifiers are stored in a database so that they can be used later when they are needed again. Thus, when the tracked object makes a sudden motion and to previously observed target is observed again, it is recognized instead of being declared lost. 
         [0053]    In the preferred embodiment of the invention, the classifiers that differ from the previous classifier by more than a predefined value are neglected. This provides rejecting false trainings due to tracking errors or occlusions.