Patent Publication Number: US-10331968-B2

Title: One shot color calibrated metric learning for object re-identification

Description:
BACKGROUND 
     Field of the Invention 
     The present disclosure relates to digital video processing, and more specifically, to techniques for unsupervised learning for recognizing objects appearing across multiple, non-overlapping video streams. 
     Description of the Related Art 
     Object identification is useful in many different fields. However, many conventional techniques require the individual objects be affirmatively registered with the recognition system, and thus are ill-suited for situations in which many objects (such as people, animals, devices. vehicles and other things) are moving quickly through a location (e.g., an airport). Image processing techniques can be used to recognize objects within frames captured by a camera device. One challenge when detecting objects across multiple camera devices is handling variations in lighting, camera position and object pose across multiple camera devices. For instance, an object may have a certain set of color characteristics in an image captured by a first camera device as a function of the ambient light temperature and direction as well as the first camera device&#39;s image sensor and processing electronics, and may be captured with different color characteristics in an image captured by a different camera device. As a result, conventional techniques for determining that objects captured across images from multiple camera devices are indeed the same object may be inaccurate. 
     SUMMARY 
     Embodiments provide a method, system, and computer-readable medium that include capturing, using a first camera and a second camera, a first plurality of images and a second plurality of images, respectively. A first and a second reference image are captured using the first and second cameras. Color descriptors and texture descriptors are extracted from the first plurality of images and the second plurality of images. A first color subspace and a second color subspace are modelled for the first camera and the second camera, respectively, based on the first and second pluralities of images and the first and second reference images. A data model for identifying objects appearing in images captured using the first and second cameras is generated, based on the extracted color descriptors, texture descriptors and the first and second color subspaces. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       So that the manner in which the above recited aspects are attained and can be understood in detail, a more particular description of embodiments of the invention, briefly summarized above, may be had by reference to the appended drawings. 
       It is to be noted, however, that the appended drawings illustrate only typical embodiments of this invention and are therefore not to be considered limiting of its scope, for the invention may admit to other equally effective embodiments. 
         FIG. 1  is a system configured with a color calibration component, according to one embodiment described herein. 
         FIG. 2  illustrates a workflow for calibrating camera devices for use in modelling appearances of objects in images across multiple video feeds, according to one embodiment described herein. 
         FIG. 3  is a block diagram illustrating a method of modelling appearances of objects in images across multiple video feeds, according to one embodiment described herein. 
         FIG. 4  is a block diagram illustrating a method of generating an aggregate distance function for identifying objects in images across multiple video feeds, according to one embodiment described herein. 
         FIGS. 5A-B  illustrate a mapping of a color space for a calibrated camera device, according to one embodiment described herein. 
         FIG. 6  is a block diagram illustrating a system configured to execute a color calibration component, according to one embodiment described herein. 
     
    
    
     DETAILED DESCRIPTION 
     Generally, object re-identification refers to the technical challenge of identifying when an object, such as a person, item, animal, vehicle or other thing, appearing in an image(s) captured using a first video camera subsequently appears in an image(s) captured using a second, non-overlapping video camera. Object re-identification approaches can generally be divided into unsupervised and supervised approaches. Unsupervised approaches focus on designing features (often handcrafted) that should be robust to changes in imaging conditions. On the other hand, supervised methods usually employ metric learning that uses training data to search for effective distance functions for comparing images of objects captured from different cameras. Supervised methods typically learn a trade-off between feature invariance and their discriminative power directly from data, thus achieving better performance than conventional unsupervised approaches. By maximizing inter-class variations and minimizing intra-class variations, such supervised learning techniques look for optimal feature embedding to then perform object re-identification. 
     Many different machine learning algorithms can be used for learning a robust similarity function. Such algorithms may include, for example, feature selection and weighting, as well as defining object re-identification as a ranking problem and using an ensemble of Ranking Support Vector Machines (SVMs). A Ranking SVM refers to a variant of a Support Vector Machine configured for ranking elements of a data set based on attributes of the elements. Additionally, or alternatively, the family of Mahalanobis distance functions can be used for learning a metric. A Mahalanobis distance can directly model the transition in feature space between two camera views, and thus may achieve better performance than the similarity functions directly learnt in the original feature space. Mahalanobis metric learning usually aims at improving k nearest neighbor (k-nn) classification by iteratively adapting the metric (e.g., Large Margin Nearest Neighbor Learning (LMNN), Information Theoretic Metric Learning (ITML), Logistic Discriminant Metric Learning (LDML)). Other algorithms include the Keep It Simple and Straightforward (KISS) metric which uses a statistical inference based on a likelihood-ratio test of two Gaussian distributions modeling positive and negative pairwise differences between features. The KISS metric has further been extended to linear and non-linear subspace embeddings. Mahalanobis metric learning approaches usually require hundreds of labeled image pairs to produce effective similarity measures. 
     Embodiments described herein provide techniques for object re-identification across multiple, non-overlapping camera devices. Generally, solutions for re-identifying objects in surveillance scenarios must deal with significant changes in appearance caused by variations in illumination, viewing angle and an object&#39;s pose. Unlike conventional metric learning approaches that learn a distance measure using hundreds or thousands of annotated subjects, embodiments described herein provide techniques for learning a metric using a one shot learning approach, in which a data model is trained using one (or only a few) images for each camera device. Embodiments described herein learn a texture feature representation that is invariant across datasets, and account for differences in camera color distributions by sampling patches on registered images of a color calibration target (e.g., a ColorChecker® board). 
     One embodiment provides a method for object re-identification that includes capturing, using each of a plurality of camera devices, a respective plurality of images. Additionally, the method includes capturing, from each of the plurality of camera devices, a respective reference image. The method also includes modelling a color subspace for each of the plurality of camera devices, based on the plurality of images and the respective reference image captured using the respective camera device. The method further includes modelling color descriptors extracted from the respective plurality of images for each of the plurality of camera devices, using the modelled color subspace, and modelling texture descriptors extracted from the respective plurality of images for each of the plurality of camera devices. A data model is then generated for re-identifying objects appearing in images captured using two or more of the plurality of camera devices, based on the modelled color descriptors and the modelled texture descriptors. 
       FIG. 1  is a system configured with a color calibration component, according to one embodiment. System  100  includes a color calibration system  110 , connected to a plurality of camera devices  140 ( 1 )-(N) via a network  130 . The color calibration system  110  includes a color calibration component  120  and stored images  125 . The images  125  comprise data representing images captured using the camera devices  140 ( 1 )-(N). In the example, the camera devices  140 ( 1 )-(N) represent non-overlapping (i.e., images captured by different camera devices of the camera devices  140 ( 1 )-(N), where each camera device captures images of a unique physical location) surveillance cameras positioned throughout a location (e.g., an airport). The network  130  generally represents a communications network (e.g., a wired communications network, a wireless communications network, etc.) suitable for transferring data between the camera devices  140 ( 1 )-(N) and the color calibration system  110 . 
     The color calibration component  120  is generally configured to analyze the images  125  and to perform an object re-identification operation to determine when an object who appeared within one or more images captured by one of the camera devices  140 ( 1 )-(N) appears within one or more images captured by a different one of the camera devices  140 ( 1 )-(N). Generally, object re-identification across the camera devices  140 ( 1 )-(N) can be used for a number of purposes. For instance, the location of a particular object of interest could be tracked throughout the camera devices  140 ( 1 )-(N). As another example, the color calibration component  120  could access data specifying the geographical locations of the camera devices  140 ( 1 )-(N), and the camera devices  140 ( 1 )-(N) could perform object re-identification techniques to track the flow of objects throughout a venue (e.g., an airport, a train station, etc.). Moreover, the color calibration component  120  could use the positions of objects, combined with the geographical locations of the camera devices  140 ( 1 )-(N), to estimate the time it takes the objects to traverse parts of the venue. For instance, the color calibration component  120  could estimate the travel time between particular locations at a site (e.g., an airport), based on the time it takes objects (e.g., individuals) to appear in a particular camera (e.g., a gate for a flight), after appearing in another camera (e.g., a security checkpoint). 
     The color calibration component  120  can perform a learning process through which the color calibration component  120  learns positive covariance of color feature differences and covariance of texture feature differences across camera devices. As the covariance of color feature differences can vary greatly between camera devices (e.g., due to differences in the cameras, due to differences in the lighting conditions where the camera devices are located, etc.), the color calibration component  120  can learn the covariance of color feature differences specifically for the cameras  140 ( 1 )-(N). On the other hand, as the covariance of texture feature differences can be assumed to be relatively constant between camera devices, the color calibration component  120  can learn the covariance of color feature differences from an annotated set of images in an auxiliary image library (e.g., images captured from cameras other than the cameras  140 ( 1 )-(N)). 
     In learning the covariance of color and texture feature differences, the color calibration component  120  could divide each of the images  125  into a respective plurality of patches. Each patch is a distinct, fixed size portion of the respective image. In a specific embodiment, each patch comprises a 24×12 pixel region of the image. More generally, any size patch can be used, consistent with the functionality described herein. Each patch can then be extracted from the image by retrieving pixel values for the pixels within the pixel region for the respective patch. 
     The color calibration component  120  then extracts color and texture descriptors from each patch. In one embodiment, the color calibration component  120  could apply a dimensionality reduction on the texture descriptors. Doing so can help to improve the performance and the accuracy of machine learning models processing the texture descriptors by avoiding over-fitting of covariance matrices learned from the color and texture descriptors. Additionally, such dimensionality reduction can help to reduce the amount of computing resources needed for the object re-identification. For example, the color calibration component  120  could perform a principal component analysis operation to reduce the dimensionality of the texture descriptors. More generally, however, any dimensionality technique can be used, consistent with the functionality described herein. The color calibration component  120  then applies a metric learning algorithm on the reduced dimensionality texture descriptors using an auxiliary labeled data set. 
     In learning the covariance of color feature differences, the color calibration component  120  could then acquire a single image of a color calibration chart from each of the camera devices  140 ( 1 )-(N). In one embodiment, the color calibration chart comprises a grid of various different colors, and the color calibration component  120  can use the appearance of the chart within images captured by the various camera devices to normalize the color space between the camera devices. The color calibration component  120  divides the portion of a captured image that depicts the chart into a plurality of patches, and for each patch, the color calibration component  120  extracts color descriptors. The color calibration component  120  can apply a dimensionality reduction operation on the extracted color descriptors (e.g., using principal component analysis or any suitable dimensionality reduction technique). As discussed above, such a dimensionality reduction can improve the performance and accuracy of data models that analyze the extracted color descriptors. In such an embodiment, the color calibration component  120  then projects the ColorChecker color descriptors onto the previously reduced color subspace. Additionally, the color calibration component  120  could apply a metric learning operation on the color descriptors extracted from the object image using the color descriptors extracted from the ColorChecker image. 
     The color calibration component  120  then combines the color and texture metrics into a unified patch metric, and combines the individual patch metrics for the object image into an aggregate distance measure. The color calibration component  120  could then use the aggregate distance measure to re-identify objects appearing within images captured using various different camera devices of the camera devices  140 ( 1 )-(N). Doing so provides an improved unsupervised learning technique for object re-identification which is capable of accounting for variances in the color subspace across the various camera devices  140 ( 1 )-(N). 
       FIG. 2  illustrates a workflow for calibrating camera devices for use in modelling appearances of objects in images across multiple video feeds, according to one embodiment described herein. As shown, the workflow  200  includes a source domain of labelled texture data  210  and a target domain of unlabeled color data  220 . The source domain of labelled texture data  210  refers to texture data determined from one or more auxiliary data sets (e.g., annotated image libraries, depicting objects across non-overlapping camera devices, which may be captured using camera devices  140 ( 1 )-(N) or other camera devices). As texture data is generally constant across different camera devices, the color calibration component  120  can learn the covariance of texture feature differences Σ +   t  from the one or more auxiliary data sets. In the depicted example, the color calibration component  120  processes texture data from the source domain  210  using a Mahalanobis metric learning algorithm at block  230 . 
     The target domain of unlabeled color data  220  represents unlabeled data captured from the target domain in which the cameras  140 ( 1 )-(N) are deployed. The color calibration component  120  learns a color embedding W c  from the unlabeled data  220  and then applies the learned color embedding to patches extracted from registered images of a color calibration chart captured using the non-overlapping cameras  140 ( 1 )-(N). The color calibration component  120  estimates the positive covariance of color feature differences Σ +   C  from these corresponding patches. 
     For the following discussion x i  and x j  are defined as feature representations of bounding boxes i and j from cameras p and q, respectively. For example, such feature representations can include a concatenation of color histograms, dense Scale Invariant Feature Transform (SIFT), Histogram of Oriented Gradients (HOG), Local Binary Pattern (LBP), etc. 
     Moreover,    +  represents the hypothesis that x i  and x j  are two distinct views of the same object, and   represents the hypothesis that the two views are of different objects. From a Bayesian point of view, the optimal decision as to whether x i  and x j  represent the same object or not, can be made through a likelihood ratio test involving the difference x ij =x i −x j  of features, where δ(i,j)→−∞ implies that x i  and x j  are the same object, as shown below in Equation 1. 
     
       
         
           
             
               
                 
                   
                     δ 
                     ⁡ 
                     
                       ( 
                       
                         i 
                         , 
                         j 
                       
                       ) 
                     
                   
                   = 
                   
                     log 
                     ⁢ 
                     
                         
                     
                     ⁢ 
                     
                       
                         P 
                         ⁡ 
                         
                           ( 
                           
                             
                               x 
                               ij 
                             
                             ❘ 
                             
                               ℋ 
                               - 
                             
                           
                           ) 
                         
                       
                       
                         P 
                         ⁡ 
                         
                           ( 
                           
                             
                               x 
                               ij 
                             
                             ❘ 
                             
                               ℋ 
                               + 
                             
                           
                           ) 
                         
                       
                     
                   
                 
               
               
                 
                   Equation 
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   1 
                   ⁢ 
                   
                     - 
                   
                   ⁢ 
                   Likelihood 
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   Ratio 
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   Test 
                 
               
             
           
         
       
     
     The numerator represents the conditional probability that the features x i  and x j  are two views of different objects, and the denominator represents the conditional probability that the features x i  and x j  are two views of the same object. To implement the Likelihood Ratio Test shown in Equation 1, the two conditional probabilities from the numerator and the denominator can be modeled as multivariate Gaussian distributions  (0,Σ − ) and  (0,Σ + ). As the densities are zero-mean because of the symmetry of differences (i.e., for each x ij =x i −x j  exists x ij =x j −x i ), the likelihood ratio resembles the difference of Mahalanobis distances. As the difference between the Mahalanobis distances is relatively easier to compute computationally, the formulation, shown in Equation 2, is generally suitable for re-identification, due to its relative simplicity and effectiveness.
 
δ( i,j )= x   ij   T (Σ +   −1 −Σ −   −1 ) x   ij    Equation 2—Difference of Mahalanobis Distances
 
     Color calibration component  120  may implement x i  and x j  as low-dimensional projections of high-dimensional feature vectors b i  and b j  extracted from image data (e.g., a concatenation of color histograms, dense SIFT, HOG, LBP, etc.). Such low-dimensional representations can make the inverse covariance matrices of Equation 2 (i.e., Σ −   −1  and Σ +   −1 ) more practical to compute. Moreover, by using such low-dimensional representations, the color calibration component  120  can avoid over-fitting the model while estimating the covariance matrices (e.g., the positive covariance of color feature differences and texture feature differences). In one embodiment, the color calibration component  120  is configured to use Principal Component Analysis (PCA) to reduce the dimensionality of the feature representations. 
     The color calibration component  120  is configured to operate on patches, rather than the entire bounding box within an image. A bounding box may be represented as a concatenation of K per-patch features. As a result, the dissimilarity between two bounding boxes (quantified by Equation 2) when operating on patches is shown in Equation 3. 
     
       
         
           
             
               
                 
                     
                 
               
               
                 
                   Equation 
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   3 
                   ⁢ 
                   
                     - 
                   
                   ⁢ 
                   Bounding 
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   Box 
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   Dissimilarity 
                 
               
             
             
               
                 
                   
                     
                       δ 
                       ⁡ 
                       
                         ( 
                         
                           i 
                           , 
                           j 
                         
                         ) 
                       
                     
                     = 
                     
                       
                         ∑ 
                         
                           k 
                           = 
                           1 
                         
                         K 
                       
                       ⁢ 
                       
                         
                           
                             ( 
                             
                               p 
                               ij 
                               k 
                             
                             ) 
                           
                           T 
                         
                         ⁢ 
                         
                           M 
                           k 
                         
                         ⁢ 
                         
                           p 
                           ij 
                           k 
                         
                       
                     
                   
                   , 
                 
               
               
                 
                     
                 
               
             
             
               
                 where 
               
               
                 
                     
                 
               
             
             
               
                 
                   
                     M 
                     k 
                   
                   = 
                   
                     
                       
                         ( 
                         
                           ∑ 
                           + 
                           k 
                         
                         ) 
                       
                       
                         - 
                         1 
                       
                     
                     - 
                     
                       
                         ( 
                         
                           ∑ 
                           - 
                           k 
                         
                         ) 
                       
                       
                         - 
                         1 
                       
                     
                   
                 
               
               
                 
                     
                 
               
             
           
         
       
     
     In practice, p i   k  is usually a low-dimensional representation of high-dimensional patch features r i   k , and typically the dimensionality reduction technique is invariant to the patch location k. In one embodiment, at block  230 , the color calibration component  120  is configured to learn a Mahalanobis distance metric M k  for each patch. Generally, such Mahalanobis distance metric can directly model the transition in the feature space between two camera views, thus achieving better performance than the similarity functions directly learned in the original feature space. In a particular embodiment, the color calibration component  120  is configured to learn a single metric M (at block  230 ) that is common to all patches, which in turn drastically increases the amount of training data and, in turn, improves generalization performance. Such a metric M is shown in Equation 4.
 
 M=Σ   +   −1 −Σ −   −1 .   Equation 4—Bounding Box Dissimilarity M Metric
 
     In one embodiment, the color calibration component  120  treats the B p  and B q  of bounding box locations from cameras p and q as available, but the labellings indicating which bounding boxes correspond to images of the same object as unavailable. Additionally, techniques disclosed herein may define each high-dimensional feature vector r i   k  extracted for each patch (across all bounding boxes) as a combination of color u i   k  and texture v i   k  descriptors (color histograms, dense SIFT, etc.). 
     The color calibration component  120  may generate the low-dimensional patch feature p i   k  using a suitable dimensionality reduction technique (e.g., PCA) and may process the color and texture descriptors independently, as shown in Equation 5, where W c  represents the color calibration chart-based positive covariance matrix, u i   k  represents a color descriptor, W t  represents the learned positive covariance matrix, v i   k  represents a texture descriptor, c i   k  represents the extracted color feature vector, and t i   k  represents the extracted texture feature vector. 
     
       
         
           
             
               
                 
                     
                 
               
               
                 
                   Equation 
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   5 
                   ⁢ 
                   
                     - 
                   
                   ⁢ 
                   Low 
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   dimensionality 
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   patch 
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   feature 
                 
               
             
             
               
                 
                   
                     
                       
                         
                           
                             p 
                             i 
                             k 
                           
                           = 
                             
                           ⁢ 
                           
                             [ 
                             
                               
                                 
                                   
                                     
                                       W 
                                       c 
                                     
                                     ⁢ 
                                     
                                       u 
                                       i 
                                       k 
                                     
                                   
                                 
                                 
                                   
                                     
                                       W 
                                       t 
                                     
                                     ⁢ 
                                     
                                       v 
                                       i 
                                       k 
                                     
                                   
                                 
                               
                             
                             ] 
                           
                         
                         , 
                       
                     
                   
                   
                     
                       
                         
                           = 
                             
                           ⁢ 
                           
                             [ 
                             
                               
                                 
                                   
                                     c 
                                     i 
                                     k 
                                   
                                 
                                 
                                   
                                     t 
                                     i 
                                     k 
                                   
                                 
                               
                             
                             ] 
                           
                         
                         , 
                       
                     
                   
                   
                     
                       
                         
                           ≈ 
                             
                           ⁢ 
                           
                             Wr 
                             i 
                             k 
                           
                         
                         , 
                       
                     
                   
                 
               
               
                 
                     
                 
               
             
           
         
       
     
     While the color calibration component  120  learns positive examples of objects detected across multiple images from multiple camera devices from training data sets, the color calibration component  120  can generate a set of negative examples by randomly selecting bounding boxes from B p  and B q . By doing so, even when the randomly generated bounding boxes correspond to the same object, the odds of this happening frequently are low. Therefore, the covariance of the negative examples can be computed from the random sampling shown in Equation 6, where p i,j   −  represents a negative example determined via random (or pseudo-random) sampling.
 
Σ − =Σ( p   ij   − )( p   ij   − ) T .   Equation 6—Negative Example Covariance
 
     With regard to positive covariance, similar to the dimensionality compression using PCA, the color calibration component  120  can be configured to assume the low-dimensional per-patch color and texture descriptors of positive examples are independent, which means the positive covariance matrix is in the form shown in Equation 7. 
     
       
         
           
             
               
                 
                   
                     
                       
                         
                           ∑ 
                           + 
                         
                         ⁢ 
                         
                           = 
                           
                               
                             
                               
                                 [ 
                                 
                                   
                                     
                                       
                                         α 
                                         ⁢ 
                                         
                                           ∑ 
                                           + 
                                           c 
                                         
                                       
                                     
                                     
                                       0 
                                     
                                   
                                   
                                     
                                       0 
                                     
                                     
                                       
                                         
                                           ( 
                                           
                                             1 
                                             - 
                                             α 
                                           
                                           ) 
                                         
                                         ⁢ 
                                         
                                           ∑ 
                                           + 
                                           t 
                                         
                                       
                                     
                                   
                                 
                                 ] 
                               
                               , 
                             
                           
                         
                       
                     
                   
                   
                     
                       where 
                     
                   
                   
                     
                       
                         
                           
                             ∑ 
                             + 
                             c 
                           
                           ⁢ 
                           
                             = 
                             
                               ∑ 
                               
                                 
                                   ( 
                                   
                                     c 
                                     ij 
                                     + 
                                   
                                   ) 
                                 
                                 ⁢ 
                                 
                                   
                                     ( 
                                     
                                       c 
                                       ij 
                                       + 
                                     
                                     ) 
                                   
                                   T 
                                 
                               
                             
                           
                         
                         , 
                       
                     
                   
                   
                     
                       
                         
                           ∑ 
                           + 
                           t 
                         
                         ⁢ 
                         
                           = 
                           
                             ∑ 
                             
                               
                                 ( 
                                 
                                   t 
                                   ij 
                                   + 
                                 
                                 ) 
                               
                               ⁢ 
                               
                                 
                                   
                                     ( 
                                     
                                       t 
                                       ij 
                                       + 
                                     
                                     ) 
                                   
                                   T 
                                 
                                 . 
                               
                             
                           
                         
                       
                     
                   
                 
               
               
                 
                   Equation 
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   7 
                   ⁢ 
                   
                     - 
                   
                   ⁢ 
                   Positive 
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   Example 
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   Covariance 
                 
               
             
           
         
       
     
     Σ +   C  represents the positive covariance of the low-dimensional color descriptors, E +   t  represents the positive covariance of low-dimensional texture descriptors, c i,j  represents the extracted color feature vector at (i,j), t i,j  represents the extracted texture feature vector at (i,j), and a controls the importance of color on the current dataset relative to its importance in the reference dataset from which W t  and E +   t  were learned. 
     For purposes of the present disclosure, the covariance E +   t  of the low-dimensional texture descriptors is invariant between camera pairs, and thus the color calibration component  120  can learn the covariance E +   t  using an annotated auxiliary dataset (i.e., annotated image data of common objects across different cameras, captured using any cameras and not restricted to images captured using the cameras  140 ( 1 )-(N)). On the other hand, the color calibration component  120  can learn the projection W c  of color features from the sets of input bounding boxes for the specific pair of cameras (since PCA does not need labelled examples). The color calibration component  120  can learn the positive covariance Σ +   c  of the low-dimensional color descriptors from a single pair of registered images of a color calibration target (shown in  FIG. 5A ). 
       FIG. 3  is a block diagram illustrating a method  300  of modelling appearances of objects in images across multiple video feeds. At block  310 , where the color calibration component  120  captures, from each of the plurality of cameras  140 ( 1 )-(N), a respective plurality of images  125 . The cameras  140 ( 1 )-(N) can be configured to periodically capture images (e.g., one image per second) and the images can be transmitted (e.g., across a data communications network) to a server configured with the color calibration component  120  for further analysis. A respective reference image of a color calibration chart is captured from each of the plurality of cameras  140 ( 1 )-(N) (block  315 ). Generally, such captured images can be in a raw image format or a compressed image format (such as the Joint Photographic Experts Group (JPEG) format, the Portable Network Graphics (PNG) format, etc.). 
     A color subspace is modelled for each of the plurality of camera devices at operation  320 , by projecting color descriptors extracted from the respective reference image of the color calibration target onto a reduced dimensionality color subspace. A first metric learning algorithm is applied to color descriptors extracted from the respective plurality of images for each camera device, using the modeled color subspace for the respective camera device at operation  325 . 
     A second metric learning algorithm is applied to texture descriptors extracted from the respective plurality of images for each camera devices at block  330 . By applying the first metric learning algorithm to the color descriptors and the second metric learning algorithm to the texture descriptors, the color calibration component  120  models the texture descriptors independently from the color descriptors extracted from the images. A data model for identifying objects appearing in images captured using two or more of the plurality of camera devices is generated at block  335 . 
       FIG. 4  is a block diagram illustrating a method  400  of generating an aggregate distance function for identifying objects in images across multiple cameras  140 ( 1 )-(N). The color calibration component  120  acquires object images from each of a plurality of camera devices at operation  410 . Each object image is divided into a plurality of patches at operation  425 . For example, each patch could be a portion of an image of a predefined size, e.g., 24 by 12 pixels. 
     Color and texture descriptors are extracted for each patch for each image at operation  420 . A dimensionality reduction algorithm is applied on the extracted texture descriptors (e.g., using principal component analysis) at operation  425 , and, at operation  430 , a metric learning algorithm is applied to the texture descriptors using an auxiliary labeled dataset. Such an auxiliary dataset can include annotated images captured using camera devices (e.g., camera devices other than the cameras  140 ( 1 )-(N)), as the learned texture descriptors are relatively constant across cameras. 
     Additionally, a dimensional reduction is applied on the extracted color descriptors, e.g., principal component analysis, at block  435 . The color calibration component  120  further acquires a respective reference image of a color calibration target for each of the camera devices (block  440 ). Such an image can be stored in any suitable format (e.g., a raw format, a compressed format, etc.) and can be transmitted over a data communications network for analysis by the color calibration component  120 . The reference image of the color calibration chart is divided into a plurality of patches at operation  445 , and for each patch, a respective color descriptor is extracted at block  450 . 
     The color descriptors extracted from the patches of the reference image of the color checker chart are projected onto the previously reduced color subspace (block  455 ). At operation  460 , a metric learning algorithm is applied on the color descriptors extracted from the object images using the projected color descriptors for the respective camera device. At block  465 , the color calibration component  120  combines the color and texture metrics into a unified patch metric. The color calibration component  120  combines the individual patch metrics into an aggregate distance measure (block  470 ). 
       FIG. 5A  illustrates a color calibration chart, according to one embodiment. As shown, the chart  500  comprises color patches similar to natural objects such as human skin, foliage, and flowers. Such a chart is generally designed to evaluate color reproduction processes by comparing images captured of the chart to the original chart.  FIG. 5B  illustrates a mapping  510  of the colors from the color calibration chart  500  onto a color space. 
     To generate a set of {c + } of per-patch color features from positive examples, the color calibration component  120  can register images of the color calibration chart  500  acquired from two different cameras using manually defined point correspondences. In doing so, the color calibration component  120  can extract a low-dimensional patch feature from each color square to produce a set of feature differences. The color calibration component  120  can further randomly sample patches in off-grid locations and the color calibration component  120  can compute color differences therebetween. Generally, sampling from off-grid locations generates more training samples that linearly interpolate the space of feature differences. 
     According to one embodiment, the colored regions of the color calibration chart  500  are almost completely homogeneous. In a particular embodiment, to make the extracted features more representative of features that would be extracted from clothing in hand-labelled example image pairs, the color calibration component  120  can apply kernel density estimation. For purposes of the following discussion, assume that each extracted color feature vector c i   k  can be written in the form shown in Equation 8. 
     
       
         
           
             
               
                 
                   c 
                   i 
                   ′ 
                 
                 ⁡ 
                 
                   ( 
                   z 
                   ) 
                 
               
               = 
               
                 
                   ∑ 
                   
                     j 
                     = 
                     1 
                   
                   N 
                 
                 ⁢ 
                 
                   G 
                   ⁡ 
                   
                     ( 
                     
                       
                         z 
                         - 
                         
                           z 
                           j 
                         
                       
                       , 
                       θ 
                     
                     ) 
                   
                 
               
             
             , 
           
         
       
     
     where θ is the bandwidth parameter of the Gaussian kernel G 
     
       
         
           
             
               
                 
                   
                     G 
                     ⁡ 
                     
                       ( 
                       
                         x 
                         , 
                         θ 
                       
                       ) 
                     
                   
                   = 
                   
                     exp 
                     ⁢ 
                     
                         
                     
                     ⁢ 
                     
                       
                         ( 
                         
                           - 
                           
                             
                               x 
                               2 
                             
                             
                               2 
                               ⁢ 
                               
                                 θ 
                                 2 
                               
                             
                           
                         
                         ) 
                       
                       . 
                     
                   
                 
               
               
                 
                   Equation 
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   8 
                   ⁢ 
                   
                     - 
                   
                   ⁢ 
                   Extracted 
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   Color 
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   Feature 
                   ⁢ 
                   
                       
                   
                   ⁢ 
                   Vector 
                 
               
             
           
         
       
     
     The estimate of feature vector differences can be expressed, as shown in Equation 9.
 
 c   ij   +   =c   ij   (p)+   ∪c   ij   (r)+   ∪c′   ij   (p)+   ∪c′   ij   (r)+   Equation 9—Estimated Color Feature Vector Differences
 
     The color calibration component  120  can use such an estimate to estimate the positive example covariance value (e.g., using Equation 7 shown above). The color calibration component  120  can then compute the M metric, e.g., using Equation 4 shown above. Additionally, the color calibration component  120  can compute the bounding box dissimilarity likelihood ratio (e.g., using Equation 3) using the computed M metric. Such a process is summarized in Algorithm 1, shown below. 
     
       
         
           
               
             
               
                   
               
               
                 Algorithm 1: Color Calibrated Metric Learning (CML) algorithm 
               
               
                   
               
             
            
               
                   
               
            
           
           
               
            
               
                 Data: People images from cameras p and q: 
               
            
           
           
               
               
            
               
                   
                 B = {B 1   p , B 2   p , . . . , B N   p , B 1   q , B 2   q , . . . , B M   q }, 
               
            
           
           
               
               
            
               
                   
                 ColorChecker images from camera p and camera q 
               
            
           
           
               
               
            
               
                   
                 A = {A p , B q } 
               
            
           
           
               
               
            
               
                   
                 Partial PCA subsapce W t   
               
               
                   
                 Low-dimensional Texture Covariance Σ +   t   
               
            
           
           
               
            
               
                 Result: Dissimilarity metric for patches M 
               
               
                 begin 
               
            
           
           
               
               
            
               
                  | 
                 Compute partial PCA subspace W c  using u i   k  from B 
               
               
                  | 
                 Compute Σ −  using random sampling from B and embedding [W c  W t ] 
               
               
                  | 
                 Compute c i   k  for all patches in B using W c   
               
               
                  | 
                 Estimate c ij   +  = c ij   (p)+  ∪ c ij   (r)+  ∪ c′ ij   (p)+  ∪ c′ ij   (r)+   
               
               
                  | 
                 Compute Σ +   
               
               
                  | —   
                 M = Σ +   −1  − Σ −   −1   
               
               
                   
               
            
           
         
       
     
     Algorithm 1—Color Calibrated Metric Learning Algorithm 
     The color calibration component  120  can then use aggregate distance measure for re-identifying objects appearing in frames across two or more camera devices within a network of camera devices. For example, the color calibration component  120  could determine that two images, captured from two different camera devices, both depict the same object, when the aggregate distance measure between two images is less than a predefined threshold amount of distance. Doing so enables the color calibration component  120  to accurately re-identify objects across multiple, non-overlapping camera devices, with minimal supervision and labelling of color descriptors within a training set. 
       FIG. 6  depicts one architecture of a system  600  within which embodiments of the present disclosure may be implemented. This figure in no way limits or is intended to limit the scope of the present disclosure. System  600  may be a desktop computer, video game console, digital assistant, rendering engine, or any other device suitable for practicing one or more embodiments of the present disclosure. 
     As shown, system  600  includes a central processing unit (CPU)  610 , a memory  620 , storage  640  and a network adapter  650 . CPU  610  includes one or more processing cores, and, in operation, CPU  610  is the master processor of system  600 , controlling and coordinating operations of other system components. The system memory  620  stores software applications and data for use by CPU  610 , and as shown, the memory  620  includes the color calibration component  110  and an operating system  630 . 
     Storage  640  represents non-volatile storage for applications and data and may include fixed or removable hard disk drives, flash memory devices, and CD-ROM, DVD-ROM, Blu-ray, HD-DVD, or other magnetic, optical, or solid state storage devices. The storage  640  (which can be connected to the CPU  610  via an I/O bridge, not shown) may be configured to store content and applications and data for use by CPU  610 . As shown, storage  640  contains images  125 , which represent images captured from cameras  140 ( 1 )-(N). 
     Network adapter  650  allows the system  600  to communicate with other systems via an electronic communications network, and may include wired or wireless communication over local area networks and wide area networks such as the Internet. For example, the images  125  can be received over a data communications network by the network adapter  650  from the cameras  140 ( 1 )-(N) and stored in the storage  640 , e.g., until analyzed by the color calibration component  110 . 
     In the preceding, reference is made to embodiments of the invention. However, it should be understood that the invention is not limited to specific described embodiments. Instead, any combination of the described features and elements, whether related to different embodiments or not, is contemplated to implement and practice the invention. Furthermore, although embodiments of the invention may achieve advantages over other possible solutions and/or over the prior art, whether or not a particular advantage is achieved by a given embodiment is not limiting of the invention. Thus, the aspects, features, embodiments and advantages described herein are merely illustrative and are not considered elements or limitations of the appended claims except where explicitly recited in a claim(s). Likewise, reference to “the invention” shall not be construed as a generalization of any inventive subject matter disclosed herein and shall not be considered to be an element or limitation of the appended claims except where explicitly recited in a claim(s). 
     As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or computer program product. Accordingly, aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module” or “system.” Furthermore, aspects of the present invention may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon. 
     Aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. 
     These computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks. The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. 
     The preceding flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order or out of order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. 
     While the foregoing is directed to embodiments of the present invention, other and further embodiments of the invention may be devised without departing from the basic scope thereof, and the scope thereof is determined by the claims that follow.