Pseudo-Spherical Contrastive Divergence

1 Introduction . Energy-based models ( EBMs ) provide a unified framework for generative and discriminative learning by capturing dependencies between random variables with an energy function . Due to the absence of the normalization constraint , EBMs offer much more flexibility in distribution parametrization and architecture design compared to properly normalized probabilistic models such as autoregressive models [ 53 , 23 ] , flow-based models [ 14 , 15 , 46 ] and sum-product networks [ 72 ] . Recently , deep EBMs have achieved considerable success in realistic image generation [ 17 , 66 , 12 , 30 ] , molecular modeling [ 90 ] and model-based planning [ 16 ] , thanks to modern deep neural networks [ 54 , 49 , 35 ] for parametrizing expressive energy functions and improved Markov Chain Monte Carlo ( MCMC ) techniques [ 62 , 75 , 40 , 17 , 66 ] for efficiently sampling from EBMs . Training EBMs consists of finding an energy function that assigns low energies to correct configurations of variables and high energies to incorrect ones [ 55 ] , where a central concept is the loss functional that is used to measure the quality of the energy function and is minimized during training . The flexibility of EBMs does not come for free : it makes the design of loss functionals particularly challenging , as it usually involves the partition function that is generally intractable to compute . As a result , EBMs are typically trained via CD [ 37 ] , which belongs to the “ analysis by synthesis ” scheme [ 31 ] and performs a sampling-based estimation of the gradient of KL between data distribution and energy-based distribution . Since different loss functionals will induce different solutions in practice ( when the model is mis-specified and data is finite ) and KL may not provide the right inductive 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . bias [ 25 , 91 ] , inspired by the great success of implicit generative models [ 28 , 67 , 3 ] , [ 89 ] proposed a variational framework to train EBMs by minimizing general f -divergences [ 10 ] . Although this framework enables us to specify various modeling preferences such as diversity/quality tradeoff , they rely on learning additional components ( variational functions ) within a minimax framework , where the optimization is complicated by the notion of Nash equilibrium and local optimality [ 42 ] and suffers from instability and non-convergence issues [ 59 ] . Along this line , noise contrastive estimation ( NCE ) [ 34 ] can train EBMs with a family of loss functionals induced by different Bregman divergences . However , in practice , NCE usually relies on carefully-designed noise distribution such as context-dependent noise distribution [ 41 ] or joint learning of a flow-based noise distribution [ 21 ] . In this paper , we draw inspiration from statistical decision theory [ 13 ] and propose a novel perspective for designing loss functionals for training EBMs without involving auxiliary models or variational optimization . Specifically , to generalize maximum likelihood training of EBMs , we focus on maximizing pseudo-spherical scoring rules [ 76 , 27 ] , which are strictly proper such that the data distribution is the unique optimum and homogeneous such that they can be evaluated without the knowledge of the normalization constant . Under the “ analysis by synthesis ” scheme used in CD and f -EBM [ 89 ] , we then derive a practical algorithm termed Pseudo-Spherical Contrastive Divergence ( PS-CD ) . Different from f -EBM , PS-CD enables us to specify flexible modeling preferences without requiring additional computational cost or unstable minimax optimization . We provide a theoretical analysis on the sample complexity and convergence property of PS-CD , as well as its connections to maximum likelihood training . With experiments on both synthetic data and commonly used image datasets , we show that PS-CD achieves significant sample quality improvement over conventional maximum likelihood training and competitive performance to f -EBM without expensive variational optimization . Based on a set of recently proposed generative model evaluation metrics [ 61 ] , we further demonstrate the various modeling tradeoffs enabled by PS-CD , justifying its modeling flexibility . Moreover , PS-CD is also much more robust than CD in face of data contamination . 2 Preliminaries . 2.1 Energy-Based Distribution Representation and Sampling . Given a set of i.i.d . samples { xi } Ni=1 from some unknown data distribution p ( x ) defined over the sample space X ⊂ Rm , the goal of generative modeling is to learn a θ-parametrized probability distribution qθ ( x ) to approximate the data distribution p ( x ) . In the context of energy-based modeling , instead of directly parametrizing a properly normalized distribution , we first parametrize an unnormalized energy function Eθ : X → R , which further defines a normalized probability density via the Boltzmann distribution : qθ ( x ) = qθ ( x ) Zθ = exp ( −Eθ ( x ) ) Zθ , ( 1 ) where Zθ : = ∫ X exp ( −Eθ ( x ) ) dx is the partition function ( normalization constant ) . In this paper , unless otherwise stated , we will use q to denote an unnormalized density and q to denote the corresponding normalized distribution . We also assume that the exponential of the negative energy belongs to the L1 space , E : = { Eθ : X → R : ∫ X exp ( −Eθ ( x ) ) dx < ∞ } , i.e. , Zθ is finite . Since energy-based models ( EBMs ) represent a probability distribution by assigning unnormalized scalar values ( energies ) to the data points , we can use any model architecture that outputs a bounded real number given an input to implement the energy function , which allows extreme flexibility in distribution parametrization . However , it is non-trivial to sample from an EBM , usually requiring MCMC [ 75 ] techniques . Specifically , in this work we consider using Langevin dynamics [ 62 , 88 ] , a gradient-based MCMC method that performs noisy gradient descent to traverse the energy landscape and arrive at the low-energy configurations : x̃t = x̃t−1 − 2 ∇xEθ ( x̃t−1 ) + √ zt , ( 2 ) where zt ∼ N ( 0 , I ) . The distribution of x̃T converges to the model distribution qθ ( x ) ∝ exp ( −Eθ ( x ) ) when → 0 and T → ∞ under some regularity conditions [ 88 ] . In order to sample from an energy-based distribution efficiently , many scalable techniques have been proposed such as learning non-convergent , non-persistent , short-run MCMC [ 66 ] and using a sample replay buffer to improve mixing time and sample diversity [ 17 ] . In this work , we leverage these recent advances when we need to obtain samples from an EBM . 2.2 Maximum Likelihood Training of EBMs via Contrastive Divergence . The predominant approach to training explicit density generative models is to approximately minimize the KL divergence between the ( empirical ) data distribution and model distribution . Minimizing KL divergence is equivalent to the following maximum likelihood estimation ( MLE ) objective : min θ LMLE ( θ ; p ) = min θ −Ep ( x ) [ log qθ ( x ) ] = min θ Ep ( x ) [ Eθ ( x ) ] + logZθ . ( 3 ) Because of the intractable partition function ( an integral over the sample space ) , we can not directly optimize the above MLE objective . To tackle this issue , [ 37 ] proposed contrastive divergence ( CD ) algorithm as a convenient way to estimate the gradient of LMLE ( θ ; p ) using samples from qθ : ∇θLMLE ( θ ; p ) = Ep ( x ) [ ∇θEθ ( x ) ] +∇θ logZθ = Ep ( x ) [ ∇θEθ ( x ) ] − Eqθ ( x ) [ ∇θEθ ( x ) ] , ( 4 ) which can be interpreted as decreasing the energies of real data from p and increasing the energies of fake data generated by qθ . As discussed above , evaluating Equation ( 4 ) typically relies on MCMC methods such as the Langevin dynamics sampling procedure defined in Equation ( 2 ) to produce samples from the model distribution qθ , which induces a surrogate gradient estimation : ∇θLCD−K ( θ ; p ) = Ep ( x ) [ ∇θEθ ( x ) ] − EqKθ ( x ) [ ∇θEθ ( x ) ] , ( 5 ) where qKθ denotes the distribution after K steps of MCMC transitions from an initial distribution ( typically data distribution or uniform distribution ) , and Equation ( 4 ) corresponds to LCD−∞ . 2.3 Strictly Proper Scoring Rules . Stemming from statistical decision theory [ 13 ] , scoring rules evaluate the quality of probabilistic forecasts by assigning numerical scores based on the predictive distributions and the events that materialize . Formally , consider a compact sample space X . Let M be a space of all locally 1- integrable non-negative finite measures and P be a subspace consisting of all probability measures on the sample space X . A scoring rule S ( x , q ) specifies the utility of forecasting using a probability forecast q ∈ P for a given sample x ∈ X . With slightly abused notation , we write the expected score of S ( x , q ) under a reference distribution p as : S ( p , q ) : = Ep ( x ) [ S ( x , q ) ] . ( 6 ) Definition 1 ( Proper Scoring Rules [ 26 ] ) . A scoring rule S : X × P → R is called proper relative to P if the corresponding expected score satisfies : ∀p , q ∈ P , S ( p , q ) ≤ S ( p , p ) . ( 7 ) It is strictly proper if the equality holds if and only if q = p. In prediction and elicitation problems , strictly proper scoring rules encourage the forecaster to make honest predictions based on their true beliefs [ 22 ] . In estimation problems , where we want to approximate a distribution p with another parametric distribution qθ , strictly proper scoring rules provide attractive learning objectives : arg max qθ∈Pθ S ( p , qθ ) = arg max qθ∈Pθ Ep ( x ) [ S ( x , qθ ) ] = p ( when p ∈ Pθ ) . ( 8 ) When a scoring rule S is strictly proper relative to P , the associated generalized entropy function and divergence function are defined as : G ( p ) : = sup q∈P S ( p , q ) = S ( p , p ) , D ( p , q ) : = S ( p , p ) − S ( p , q ) ≥ 0 . ( 9 ) G ( p ) is convex and represents the maximally achievable utility , while D ( p , q ) is the Bregman divergence [ 6 ] associated with the convex function G and the equality holds only when p = q . Next , we introduce a specific kind of scoring rules that are particularly suitable for learning unnormalized statistical models . Definition 2 ( Homogeneous Scoring Rules [ 69 ] ) . A scoring rule is homogeneous if it satisfies ( here the domain of the score function is extended to X ×M ) : ∀λ > 0 , x ∈ X , S ( x , q ) = S ( x , λ · q ) . ( 10 ) Since scaling the model distribution q by a positive constant λ does not change the value of a homogeneous scoring rule , such homogeneity allows us to evaluate it without computing the intractable partition function of an energy-based distribution . Thus , strictly proper and homogeneous scoring rules are natural candidates for new training objectives of EBMs . Example 1 . A notable example of scoring rules is the widely used logarithm score : S ( x , q ) = log q ( x ) . The associated generalized entropy is the negative Shannon entropy : G ( p ) = Ep ( x ) [ log p ( x ) ] , and the associated Bregman divergence is the KL divergence : D ( p , q ) = Ep ( x ) [ log ( p ( x ) /q ( x ) ) ] . From Definitions 1 and 2 , we know that the logarithm score is strictly proper but not homogeneous . Specifically , for a θ-parametrized energy-based distribution qθ = qθ/Zθ , since S ( x , qθ ) = S ( x , Zθ · qθ ) = S ( x , qθ ) + logZθ 6= S ( x , qθ ) and logZθ can not be ignored during the optimization of θ , we need to use tailored methods such as contrastive divergence [ 37 ] or doubly dual embedding [ 11 ] to tackle the intractable partition function .

Summary: The paper introduces a class of divergence to learn EBMs. The class of divergence includes KL as a special case. The paper approximates the gradient of the class of divergence by self-normalized importance sampling and analyzes the convergence when optimizing with the approximate gradient. Experiments are conducted to evaluate the influence of $\gamma$.