Learning Neural Random Fields with Inclusive Auxiliary Generators

1 INTRODUCTION . One of the core research problems in machine learning is learning with probabilistic models , which can be broadly classified into two classes - directed and undirected1 ( Koller & Friedman , 2009 ) . Significant progress has been made recently on learning with deep generative models ( DGMs ) , which generally refer to models with multiple layers of stochastic or deterministic variables . There have emerged a bundle of deep directed generative models , such as variational AutoEncoders ( VAEs ) ( Kingma & Welling , 2014 ) , generative adversarial networks ( GANs ) ( Goodfellow et al. , 2014 ) and so on . In contrast , undirected generative models ( also known as random fields ( Koller & Friedman , 2009 ) , energy-based models ( LeCun et al. , 2006 ) ) received less attention with slow progress . This is presumably because fitting undirected models is more challenging than fitting directed models . In general , calculating the log-likelihood and its gradient is analytically intractable , because it involves the expectation with respect to ( w.r.t . ) the model distribution . In this paper , we aims to advance the learning of neural random fields ( RFs ) which use neural networks with multiple ( deterministic ) layers to define the potential function2 uθ ( x ) over observation x with parameter θ . The probability distribution pθ ( x ) ∝ exp ( uθ ( x ) ) is then defined by normalizing the exponentiated potential function . This type of RFs has been studied several times in different contexts , once called deep energy models ( DEMs ) ( Ngiam et al. , 2012 ; Kim & Bengio , 2016 ) , descriptive models ( Xie et al. , 2016 ) , generative ConvNet ( Dai et al. , 2014 ) , neural random field language models ( Wang & Ou , 2017 ) . For convenience , we refer to such models as neural random fields ( NRFs ) in general . An important method of maximum likelihood ( ML ) learning of random fields is called stochastic maximum likelihood ( SML ) ( Younes , 1989 ) , which approximates the model expectations by Monte Carlo sampling for calculating the gradient . A recent progress in learning NRFs as studied in Kim & Bengio ( 2016 ) ; Xie et al . ( 2016 ) ; Wang & Ou ( 2017 ) ; Kuleshov & Ermon ( 2017 ) is to 1An easy way to tell an undirected model from a directed model is that an undirected model involves the normalizing constant ( also called the partition function in physics ) , while the directed model is self-normalized . 2Note that compared to modeling with multiple deterministic layers , modeling with multiple stochastic layers , e.g . deep Boltzmann machines ( DBMs ) ( Salakhutdinov & Hinton , 2009 ) , presents much greater challenge for model learning and thus yields inferior performance . This is observed in both directed and undirected models . pair the target random field pθ with an auxiliary directed generative model ( often called generator ) qφ ( x ) parameterized by φ , which approximates sampling from the target random field . Learning is performed by maximizing the log-likelihood of training data under pθ or some bound of the log-likelihood , and simultaneously minimizing some divergence between the target random field pθ and the auxiliary generator qφ . Different learning algorithms differ in the objective functions used in the joint training of pθ and qφ , and thus have different computational and statistical properties ( partly illustrated in Figure 1 ) . For example , minimizing the exclusive-divergence KL [ qφ||pθ ] , ∫ qφ log ( qφ/pθ ) w.r.t . φ , as employed in Kim & Bengio ( 2016 ) , involves the intractable entropy term and tends to enforce the generator to seek modes , yielding missing modes . There are also other factors , e.g . modeling discrete or continuous data , different choices of the target RF and the generator , which lead to different algorithms . We leave detailed comparison and connection of our approach with existing studies to section 3 ( related work ) . In this paper , we propose to use inclusive-divergence minimized auxiliary generators ( section 2.1 ) . And particularly for continuous data ( e.g . images ) , we propose to use SGLD ( stochastic gradient Langevin dynamics ( Welling & Teh , 2011 ) ) and SGHMC ( stochastic gradient Hamiltonian Monte Carlo ( Chen et al. , 2014 ) ) to exploit gradient information in model sampling with solid theoretical examination ( section 2.2 ) . The new approach , abbreviated as the inclusive-NRF approach , offers some advantages over previous methods . First , minimizing the inclusive-divergence KL [ pθ||qφ ] , ∫ pθ log ( pθ/qφ ) w.r.t . φ avoids the annoying entropy term and tends to drive the generator to cover modes of the target density pθ . The SGLD/SGHMC sampling further pushes the samples towards the modes of pθ . Presumably , this helps to produce Markov chains that mix fast between modes and facilitate model learning . Second , the new approach enables us to flexibly use NRFs in unsupervised/supervised image generation and semi-supervised classification ( section 2.3 ) , and empirically to the best of our knowledge , represents the best-performed random fields in these tasks . The main contributions of this paper can be summarized as follows : • We develop the inclusive-NRF approach , which learns NRFs with inclusive auxiliary generators and particularly for continuous data , exploits gradient information in model sampling with solid theoretical examination . • Inclusive-NRFs achieve state-of-the-art sample generation quality , measured by both Incep- tion Score ( IS ) and Frechet Inception Distance ( FID ) . On CIFAR-10 , we obtain unsupervised IS 8.28 ( FID 20.9 ) and supervised IS 9.06 ( FID 18.1 ) , both using unconditional generation . • Semi-supervised inclusive-NRFs show strong classification results on par with state-of- the-art DGM-based semi-supervised learning ( SSL ) methods , and simultaneously achieve superior generation , on the widely benchmarked datasets - MNIST , SVHN and CIFAR-10 . 2 THE INCLUSIVE-NRF APPROACH . Consider a random field for modeling observation x with parameter θ : pθ ( x ) = 1 Z ( θ ) exp [ uθ ( x ) ] ( 1 ) where Z ( θ ) = ∫ exp ( uθ ( x ) ) dx is the normalizing constant , uθ ( x ) is the potential function3 which assigns a scalar value to each configuration of random variable x . The general idea of neural random fields ( NRFs ) is to implement uθ ( x ) : Rdx → R , by a neural network , taking the multi-dimensional x ∈ Rdx as input and outputting the scalar uθ ( x ) ∈ R. In this manner , we can take advantage of the representation power of neural networks for RF modeling . It is usually intractable to maximize the data log-likelihood logpθ ( x̃ ) for observed x̃ , since the gradient involves expectation w.r.t . the model distribution , as shown below : ∇θ log pθ ( x̃ ) = ∇θuθ ( x̃ ) − Epθ ( x ) [ ∇θuθ ( x ) ] ( 2 ) 2.1 INTRODUCING INCLUSIVE-DIVERGENCE MINIMIZED AUXILIARY GENERATORS . In this paper , we further develop NRF learning with auxiliary generators . We are mainly concerned with modeling fixed-dimensional continuous observations x ∈ Rdx ( e.g . images ) , and choose a 3Negating the potential function defines the energy function . Algorithm 1 Learning NRFs with inclusive auxiliary generators repeat Sampling : Draw a minibatchM = { ( x̃i , xi , hi ) , i = 1 , · · · |M| } from p̃ ( x̃ ) pθ ( x ) qφ ( h|x ) ( see Algorithm 2 ) ; Updating : Update θ by ascending : 1|M| ∑ ( x̃ , x , h ) ∼M [ ∇θuθ ( x̃ ) −∇θuθ ( x ) ] ; Update φ by ascending : 1|M| ∑ ( x̃ , x , h ) ∼M∇φ log qφ ( x , h ) ; until convergence directed generative model , qφ ( x , h ) , q ( h ) qφ ( x|h ) , for the auxiliary generator , which specifically is defined as follows4 : h ∼ N ( 0 , Ih ) , x = gφ ( h ) + , ∼ N ( 0 , σ2I ) , ( 3 ) where gφ ( h ) : Rdh → Rdx is implemented as a neural network with parameter φ , which maps the latent code h to the observation space . Ih and I denote the identity matrices , with dimensionality implied by h and respectively . Drawing samples from the generator qφ ( x , h ) is simple as it is just ancestral sampling from a 2-variable directed graphical model . Suppose that data D = { x̃1 , · · · , x̃n } , consisting of n observations , are drawn from the true but unknown data distribution p0 ( · ) . p̃ ( x̃ ) , 1n ∑n k=1 δ ( x̃ − x̃k ) denotes the empirical data distribution . Then we formulate the maximum likelihood learning of pθ ( x ) with the inclusive-divergence minimized generator qφ ( x ) as optimizing5 min θ KL [ p̃ ( x̃ ) ||pθ ( x̃ ) ] min φ KL [ pθ ( x ) ||qφ ( x ) ] ( 4 ) The first line of Eq . ( 4 ) is equivalent to maximum likelihood training of the target RF pθ under the empirical data p̃ , which requires sampling from pθ . Simultaneously , the second line optimizes the generator qφ to be close to pθ so that qφ becomes a good proposal for sampling from pθ . It can be easily seen that the gradients w.r.t . θ and φ ( to be ascended ) are defined as follows : { ∇θ = Ep̃ ( x̃ ) [ ∇θ log pθ ( x̃ ) ] = Ep̃ ( x̃ ) [ ∇θuθ ( x̃ ) ] − Epθ ( x ) [ ∇θuθ ( x ) ] , ∇φ = Epθ ( x ) [ ∇φ log qφ ( x ) ] = Epθ ( x ) qφ ( h|x ) [ ∇φ log qφ ( x , h ) ] . ( 5 ) Both lines of Eq . ( 5 ) hold , as proved in Proposition 1 in the Supplement . In practice , we calculate noisy gradient estimators , and apply minibatch based stochastic gradient descent ( SGD ) to solve the optimization problem Eq . ( 4 ) , as shown in Algorithm 1 . 2.2 APPLYING SGLD/SGHMC FOR MODEL SAMPLING . In Algorithm 1 , we need to draw samples from pθ ( x ) qφ ( h|x ) given current θ and φ . For continuous observations , SGLD ( stochastic gradient Langevin dynamics ) ( Welling & Teh , 2011 ) and SGHMC ( Stochastic Gradient Hamiltonian Monte Carlo ) ( Chen et al. , 2014 ) sampling provide mechanisms for exploiting ( stochastic ) gradients of the target density pθ ( x ) qφ ( h|x ) , enabling efficient exploration of the state space . We take the theoretical results about SGLD from Teh et al . ( 2016 ) and SGHMC from Chen et al . ( 2014 ) , which are briefly summarized in Theorem 1 in the Supplement , and apply them in the sampling step in Algorithm 1 . Denoting the target density as p ( z ; λ ) with given λ , Theorem 1 shows that SGLD/SGHMC , by utilizing ∂∂z log p ( z ; λ ) , yields a non-homogeneous Markov chain { z ( l ) , l ≥ 1 } , which converges to the equilibrium distribution p ( z ; λ ) . By letting z , ( x , h ) , p ( z ; λ ) , pθ ( x ) qφ ( h|x ) , λ , ( θ , φ ) T in Theorem 1 , we can perform the sampling step in Algorithm 1 by running |M| parallel chains , each chain being executed as shown 4Note that during training , σ2 is absorbed into the learning rates and does not need to be estimated . 5Such optimization using two objectives is employed in a number of familiar learning methods , such as GAN with logD trick ( Goodfellow et al. , 2014 ) , wake-sleep algorithm ( Hinton et al. , 1995 ) . Algorithm 2 Sampling from pθ ( x ) qφ ( h|x ) 1 . Do ancestral sampling by the generator , namely first drawing h′ ∼ p ( h′ ) , and then drawing x′ ∼ qφ ( x′|h′ ) ; 2 . Starting from ( x′ , h′ ) = z ( 0 ) , run finite steps of SGLD/SGHMC ( l = 1 , · · · , L ) to obtain ( x , h ) = z ( L ) , which we call sample revision , according to Eq . ( 11 ) / ( 12 ) . Return ( x , h ) . in Algorithm 2 . In sample revision , the calculation of the gradient w.r.t . h , ∂∂h log p ( z ; λ ) = ∂ ∂h log qφ ( h|x ) = ∂ ∂h log qφ ( h , x ) , is straightforward . For the gradient w.r.t . x , we have ∂ ∂x log p ( z ; λ ) = ∂ ∂x log pθ ( x ) + ∂ ∂x log qφ ( h , x ) − ∂ ∂x log qφ ( x ) ≈ ∂ ∂x log pθ ( x ) . ( 6 ) The reason is that ∂∂x log qφ ( x ) can be approximated by an unbiased estimate , as proved in Proposition 2 in the Supplement : ∂ ∂x log qφ ( x ) ≈ ∂ ∂x log qφ ( h , x ) . Therefore , we can use ∂∂x log pθ ( x ) 6 as an unbiased estimate of the gradient ∂∂x log p ( z ; λ ) , and we can apply Theorem 1 in the Supplement with tractable gradients w.r.t . both x and h. Remarks . Intuitively , the generator gives a proposal ( x′ , h′ ) , and then the system follows the gradients of pθ ( x ) and qφ ( h , x ) ( w.r.t . x and h respectively ) to revise ( x′ , h′ ) to ( x , h ) . The gradient terms pull samples moving to low energy region of the random field and adjust the latent code of the generator , while the noise term brings randomness . In this manner , we obtain Markov chain samples from pθ ( x ) qφ ( h|x ) . Note that finite steps in sample revision will produce biased estimates of the gradients ∇θ and∇φ in Eq . ( 5 ) . We did not find this to pose problems to the SGD optimization in practice , as similarly found in Bornschein & Bengio ( 2015 ) and Kuleshov & Ermon ( 2017 ) , which work with biased gradient estimators .

Summary: The paper proposes the inclusive neural random field model. Compared the existing work, the model is different because of the use of the inclusive-divergence minimization for the generative model and the use of stochastic gradient Langevin dynamics (SGLD) and stochastic gradient Hamiltonian Monte Carlo  (SGHMC) for sampling. Experimental results are reported for unsupervised, semi-supervised, and supervised learning problems on both synthetic and real-world datasets. Specific comments follow: