page_title: Multivariate kernel density estimation
section_title: Objective and data-driven kernel selection
hierarchical_section_title: Multivariate kernel density estimation / Objective and data-driven kernel selection
caption_attribution_description: English: C
context_page_description: Kernel density estimation is a nonparametric technique for density estimation i.e., estimation of probability density functions, which is one of the fundamental questions in statistics. It can be viewed as a generalisation of histogram density estimation with improved statistical properties. Apart from histograms, other types of density estimators include parametric, spline, wavelet and Fourier series. Kernel density estimators were first introduced in the scientific literature for univariate data in the 1950s and 1960s and subsequently have been widely adopted. It was soon recognised that analogous estimators for multivariate data would be an important addition to multivariate statistics. Based on research carried out in the 1990s and 2000s, multivariate kernel density estimation has reached a level of maturity comparable to its univariate counterparts.
context_section_description: Recent research has shown that the kernel and its bandwidth can both be optimally and objectively chosen from the input data itself without making any assumptions about the form of the distribution.  The resulting kernel density estimate converges rapidly to the true probability distribution as samples are added: at a rate close to the  expected for parametric estimators.  This kernel estimator works for univariate and multivariate samples alike.  The optimal kernel is defined in Fourier space—as the optimal damping function  (the Fourier transform of the kernel  )-- in terms of the Fourier transform of the data , the empirical characteristic function (see Kernel density estimation):
   
  where, N is the number of data points, d is the number of dimensions (variables), and  is a filter that is equal to 1 for 'accepted frequencies' and 0 otherwise.  There are various ways to define this filter function, and a simple one that works for univariate or multivariate samples is called the 'lowest contiguous hypervolume filter';  is chosen such that the only accepted frequencies are a contiguous subset of frequencies surrounding the origin for which  (see  for a discussion of this and other filter functions).
  Note that direct calculation of the empirical characteristic function (ECF) is slow, since it essentially involves a direct Fourier transform of the data samples.  However, it has been found that the ECF can be approximated accurately using a non-uniform fast Fourier transform (nuFFT) method, which increases the calculation speed by several orders of magnitude (depending on the dimensionality of the problem).  The combination of this objective KDE method and the nuFFT-based ECF approximation has been referred to as fastKDE in the literature.
caption: Demonstration of the filter function 
    
      
        
          
            I
            
              
                
                  A
                  →
                
              
            
          
          (
          
            
              
                t
                →
              
            
          
          )
        
      
      {\displaystyle I_{\vec {A}}({\vec {t}})}
    
  . The square of the empirical distribution function 
    
      
        
          
            |
          
          
            
              
                φ
                ^
              
            
          
          
            
              |
            
            
              2
            
          
        
      
      {\displaystyle |{\hat {\varphi }}|^{2}}
    
   from N=10,000 samples of the ‘transition distribution’ discussed in Section  3.2 (and shown in Fig. 4), for 
    
      
        
          
            |
          
          
            
              
                φ
                ^
              
            
          
          
            
              |
            
            
              2
            
          
          ≥
          4
          (
          N
          −
          1
          )
          
            N
            
              −
              2
            
          
        
      
      {\displaystyle |{\hat {\varphi }}|^{2}\geq 4(N-1)N^{-2}}
    
  . There are two color schemes present in this figure. The predominantly dark, multicolored colored ‘X-shaped’ region in the center corresponds to values of 
    
      
        
          
            |
          
          
            
              
                φ
                ^
              
            
          
          
            
              |
            
            
              2
            
          
        
      
      {\displaystyle |{\hat {\varphi }}|^{2}}
    
   for the lowest contiguous hypervolume (the area containing the origin); the colorbar at right applies to colors in this region. The lightly-colored, monotone areas away from the first contiguous hypervolume correspond to additional contiguous hypervolumes (areas) with 
    
      
        
          
            |
          
          
            
              
                φ
                ^
              
            
          
          
            
              |
            
            
              2
            
          
          ≥
          4
          (
          N
          −
          1
          )
          
            N
            
              −
              2
            
          
        
      
      {\displaystyle |{\hat {\varphi }}|^{2}\geq 4(N-1)N^{-2}}
    
  . The colors of these areas are arbitrary and only serve to visually differentiate nearby contiguous areas from one another.
exif: {"Image Orientation": "Horizontal (normal)", "Image XResolution": "113", "Image YResolution": "113", "Image ResolutionUnit": "Pixels/Inch", "Image Software": "Adobe Photoshop CS2 Macintosh", "Image DateTime": "2016:03:03 17:18:45", "Image ExifOffset": "164", "Thumbnail Compression": "JPEG (old-style)", "Thumbnail XResolution": "113", "Thumbnail YResolution": "113", "Thumbnail ResolutionUnit": "Pixels/Inch", "Thumbnail JPEGInterchangeFormat": "302", "Thumbnail JPEGInterchangeFormatLength": "6220", "EXIF ColorSpace": "sRGB", "EXIF ExifImageWidth": "1653", "EXIF ExifImageLength": "1372"}