Abstract:
A method and apparatus for improving stability of histogram correlation based image and video processing algorithms. The method includes computing a histogram for target signal and reference signal for generating a target histogram and a reference histogram, performing low pass filtering of the input signal and the reference signal and producing smoothed histograms, and performing correlation on the smoothed histograms for improving stability of histogram correlation.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
       [0001]    This application claims benefit of U.S. provisional patent application Ser. No. 61/262,953, filed Oct. 20, 2009, which is herein incorporated by reference. 
     
    
     BACKGROUND OF THE INVENTION 
       [0002]    1. Field of the Invention 
         [0003]    Embodiments of the present invention generally relate to a method and apparatus for improving stability of histogram correlation. 
         [0004]    2. Description of the Related Art 
         [0005]    A correlation based algorithm makes a decision by computing the correlation score between two signals/images. One of the challenges of a correlation algorithm is the stability of the correlation score. Small disturbance in the input signal/image can lead to fluctuation in the correlation score. 
         [0006]    The histogram of the chromaticity of an image may contain peaks in close proximity, such peaks coupled with the discreteness nature of the histogram lead to unsmooth correlation score among adjacent video frames. Therefore, there exists a need for improving the stability of histogram correlation based image and video processing algorithms. 
       SUMMARY OF THE INVENTION 
       [0007]    Embodiments of the present invention relate to a method and apparatus for improving stability of histogram correlation. The method includes computing a histogram for target signal and reference signal for generating a target histogram and a reference histogram, performing low pass filtering of the input signal and the reference signal and producing smoothed histograms, and performing correlation on the smoothed histograms for improving stability of histogram correlation. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0008]    So that the manner in which the above recited features of the present invention can be understood in detail, a more particular description of the invention, briefly summarized above, may be had by reference to embodiments, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only typical embodiments of this invention and are therefore not to be considered limiting of its scope, for the invention may admit to other equally effective embodiments. 
           [0009]      FIGS. 1  ( a ) and ( b ) are embodiments depicting two consecutive shots of a brick wall,  FIG. 1(   c ) is the chromaticity histogram of (a), and  FIG. 1(   d ) is the histogram of (b); 
           [0010]      FIG. 2  is an embodiment of an Illustration of moving an input histogram towards upper left, upper right, lower left, and lower right direction; 
           [0011]      FIG. 3  is an embodiment of Gaussian low pass filter; 
           [0012]      FIGS. 4  ( a ) and ( b ) are embodiments of Gaussian low pass filtered histogram of  FIG. 1(   c ) and  FIG. 1(   d ), respectively,  FIGS. 4(   c ) and ( d ) are the new results of computing correlation score after low filtering; 
           [0013]      FIG. 5  is a flow diagram depicting an embodiment of a method for improving stability of histogram correlation; and 
           [0014]      FIG. 6  is a flow diagram depicting another embodiment of a method for improving stability of histogram correlation 
       
    
    
     DETAILED DESCRIPTION 
       [0015]      FIGS. 1  ( a ) and ( b ) are embodiments depicting two consecutive shots of a brick wall,  FIG. 1(   c ) is the chromaticity histogram of (a), and  FIG. 1(   d ) is the histogram of (b). In the histogram correlation based white balance algorithm, two consecutive shots of the same scene are taken. The histogram correlation score turned out to be very different for  FIGS. 1  ( a ) and ( b ), and consequently the color temperature estimation and the white balance gains are very different. 
         [0016]    The fluctuation in correlation score is due to the these factors: (1) the chromaticity histograms of these two images are narrowly concentrated in a small region, (2) the location of the peaks of the histograms are slightly different, and (3) both the histogram of the references and the histogram of the input image are discrete. 
         [0017]    As a result of these properties, small disturbance of the histogram distribution can lead to a significant change of the correlation score, when the peak of the histogram of the input image overlaps with one of the peaks of the reference histogram, the correlation score will be very high, otherwise the score will be very low. The chromaticity histograms of the two images are shown in  FIGS. 1  ( c ) and ( d ). Such instability has to be dealt with. 
         [0018]    Since the fluctuation of the correlation score is caused mainly by the discreteness of the histograms (or any input signals to a correlation algorithm), and it is usually worsened by the very narrowly concentrated histograms. Thus, it may be beneficial to “move around” the input signal in a small neighborhood and correlate it with the references multiple times.  FIG. 2  is an embodiment of an Illustration of moving an input histogram towards upper left, upper right, lower left, and lower right direction. 
         [0019]    Then, one may combine the correlation scores to get the final score, as shown in Eqn (1). 
         [0000]    
       
         
           
             
               
                 
                   
                     Corr 
                     Final 
                   
                   = 
                   
                     
                       ∑ 
                       k 
                       
                           
                       
                     
                      
                     
                         
                     
                      
                     
                       
                         w 
                         k 
                       
                       · 
                       
                         CORR 
                          
                         
                           ( 
                           
                             
                               H 
                                
                               
                                 ( 
                                 
                                   
                                     i 
                                     - 
                                     
                                       n 
                                       k 
                                     
                                   
                                   , 
                                   
                                     j 
                                     - 
                                     
                                       m 
                                       k 
                                     
                                   
                                 
                                 ) 
                               
                             
                             , 
                             
                               G 
                                
                               
                                 ( 
                                 
                                   i 
                                   , 
                                   j 
                                 
                                 ) 
                               
                             
                           
                           ) 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   1 
                   ) 
                 
               
             
           
         
       
     
         [0020]    This is equivalent to detecting a peak correlation in a neighborhood of the input signal, instead of just at one spot. This way the correlation score is more robust to small disturbance in the input histograms. In Eqn (1), CORR(H, G) is the operation of computing correlation between H and G. H is the target histogram/signal, G is the reference histogram/signal. w k  is the weight applied to the k-th correlation. n k  and m k  are the amount of offset in shifting H. 
         [0021]    Such an algorithm may be implemented by applying low pass filtering to the input histogram. The Gaussian low pass filter is given in Eqn (2) 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       Gauss 
                        
                       
                           
                       
                        
                       
                         ( 
                         
                           m 
                           , 
                           n 
                         
                         ) 
                       
                     
                     = 
                     
                       C 
                       · 
                       
                          
                         
                           - 
                           
                             
                               
                                 m 
                                 2 
                               
                               + 
                               
                                 n 
                                 2 
                               
                             
                             
                               2 
                                
                               
                                   
                               
                                
                               
                                 σ 
                                 2 
                               
                             
                           
                         
                       
                     
                   
                    
                   
                       
                   
                    
                   Where 
                 
               
               
                 
                   ( 
                   2 
                   ) 
                 
               
             
             
               
                 
                   C 
                   = 
                   
                     1 
                     
                       
                         ∑ 
                         m 
                         
                             
                         
                       
                        
                       
                           
                       
                        
                       
                         
                           ∑ 
                           n 
                           
                               
                           
                         
                          
                         
                             
                         
                          
                         
                            
                           
                             - 
                             
                               
                                 
                                   m 
                                   2 
                                 
                                 + 
                                 
                                   n 
                                   2 
                                 
                               
                               
                                 2 
                                  
                                 
                                     
                                 
                                  
                                 
                                   σ 
                                   2 
                                 
                               
                             
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   3 
                   ) 
                 
               
             
           
         
       
     
         [0000]    We chose σ=1.0 and the kernel size of the Gaussian filter to be 5×5. 
         [0022]      FIG. 3  is an embodiment of Gaussian low pass filter. To improve the accuracy of the correlation, one may apply the Gaussian low pass filter to the reference histograms before computing the correlation. Now the correlation score is computed as follows: 
         [0000]      Step 1:    H   =Gauss* H    (4),
 
         [0000]    where H is the input histogram/signal 
         [0000]      Step 2:    G   =Gauss* G    (5),
 
         [0000]    where G is the reference histogram/signal 
         [0000]      Step 3: Corr Final =CORR(  H ,  G ),   (6)
 
         [0000]    where the correlation operation CORR(H,G) is defined below: 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       CORR 
                        
                       
                         ( 
                         
                           H 
                           , 
                           G 
                         
                         ) 
                       
                     
                     = 
                     
                       
                         
                           ∑ 
                           m 
                           
                               
                           
                         
                          
                         
                             
                         
                          
                         
                           
                             ∑ 
                             n 
                             
                                 
                             
                           
                            
                           
                             
                               H 
                                
                               
                                 ( 
                                 
                                   m 
                                   , 
                                   n 
                                 
                                 ) 
                               
                             
                             · 
                             
                               G 
                                
                               
                                 ( 
                                 
                                   m 
                                   , 
                                   n 
                                 
                                 ) 
                               
                             
                           
                         
                       
                       
                         StdDev 
                          
                         
                             
                         
                          
                         
                           
                             ( 
                             H 
                             ) 
                           
                           · 
                           
                             StdDev 
                              
                             
                               ( 
                               G 
                               ) 
                             
                           
                         
                       
                     
                   
                   , 
                   
                     
 
                   
                    
                   and 
                 
               
               
                 
                   ( 
                   7 
                   ) 
                 
               
             
             
               
                 
                   
                     StdDev 
                      
                     
                         
                     
                      
                     
                       ( 
                       X 
                       ) 
                     
                   
                   = 
                   
                     
                       
                         1 
                         
                           M 
                           · 
                           N 
                         
                       
                        
                       
                         
                           ∑ 
                           
                             n 
                             = 
                             1 
                           
                           N 
                         
                          
                         
                             
                         
                          
                         
                           
                             ∑ 
                             
                               m 
                               = 
                               1 
                             
                             M 
                           
                            
                           
                             
                               X 
                                
                               
                                 ( 
                                 
                                   m 
                                   , 
                                   n 
                                 
                                 ) 
                               
                             
                             2 
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   8 
                   ) 
                 
               
             
           
         
       
     
         [0023]    The low pass filtering of the histograms make the histograms much more robust to small disturbance, and consequently leading to much more stable correlation scores. The filtered histograms and the resulting images are shown in  FIG. 4 .  FIGS. 4  ( a ) and ( b ) are embodiments of Gaussian low pass filtered histogram of  FIG. 1(   c ) and  FIG. 1(   d ), respectively,  FIGS. 4(   c ) and ( d ) are the new results of computing correlation score after low passing. 
         [0024]    The technique described in this disclosure may be applied to improve the robustness and stability of any correlation algorithms in general, where the signals are low pass filtered to reduce the influence of noise. The selection of the parameter of the low pass filter is very important. In terms of a Gaussian filter, the kernel size should be at least 5 times of the standard deviation of the Gaussian filter, and the standard deviation should be small to avoid excessive expansion of the signal, as well as restraining computation to minimal. 
         [0025]      FIG. 5  is a flow diagram depicting an embodiment of a method  500  for improving stability of histogram correlation. The method  500  starts at step  502  and proceeds to step  504 . At step  504 , the method  500  computes a histogram for target signal and reference signal. At step  506 , the method  500  performs low pass filtering of input and reference signal and produces smoothed histograms. At step  508 , the method  500  performs correlation on the smoothed histograms. The method  500  ends at step  510 . 
         [0026]      FIG. 6  is a flow diagram depicting another embodiment of a method  600  for improving stability of histogram correlation. The method  600  starts at step  602  and proceeds to step  604 . At step  604 , the method  600  computes a histogram for target signal and reference signal. At step  606 , the method  600  shifts the target histogram towards upper left, upper right, lower left and lower right in a neighborhood. At step  608 , the method  600  performs correlation between reference histogram and target histogram and between shifted target histogram and the referenced histogram. At step  610 , the method  600  combines correlation scores. The method  600  ends at step  612 . 
         [0027]    While the foregoing is directed to embodiments of the present invention, other and further embodiments of the invention may be devised without departing from the basic scope thereof, and the scope thereof is determined by the claims that follow.