Learning Neural Random Fields with Inclusive Auxiliary Generators

1 INTRODUCTION . One of the core research problems in machine learning is learning with probabilistic models , which can be broadly classified into two classes - directed and undirected1 ( Koller & Friedman , 2009 ) . Significant progress has been made recently on learning with deep generative models ( DGMs ) , which generally refer to models with multiple layers of stochastic or deterministic variables . There have emerged a bundle of deep directed generative models , such as variational AutoEncoders ( VAEs ) ( Kingma & Welling , 2014 ) , generative adversarial networks ( GANs ) ( Goodfellow et al. , 2014 ) and so on . In contrast , undirected generative models ( also known as random fields ( Koller & Friedman , 2009 ) , energy-based models ( LeCun et al. , 2006 ) ) received less attention with slow progress . This is presumably because fitting undirected models is more challenging than fitting directed models . In general , calculating the log-likelihood and its gradient is analytically intractable , because it involves the expectation with respect to ( w.r.t . ) the model distribution . In this paper , we aims to advance the learning of neural random fields ( RFs ) which use neural networks with multiple ( deterministic ) layers to define the potential function2 uθ ( x ) over observation x with parameter θ . The probability distribution pθ ( x ) ∝ exp ( uθ ( x ) ) is then defined by normalizing the exponentiated potential function . This type of RFs has been studied several times in different contexts , once called deep energy models ( DEMs ) ( Ngiam et al. , 2012 ; Kim & Bengio , 2016 ) , descriptive models ( Xie et al. , 2016 ) , generative ConvNet ( Dai et al. , 2014 ) , neural random field language models ( Wang & Ou , 2017 ) . For convenience , we refer to such models as neural random fields ( NRFs ) in general . An important method of maximum likelihood ( ML ) learning of random fields is called stochastic maximum likelihood ( SML ) ( Younes , 1989 ) , which approximates the model expectations by Monte Carlo sampling for calculating the gradient . A recent progress in learning NRFs as studied in Kim & Bengio ( 2016 ) ; Xie et al . ( 2016 ) ; Wang & Ou ( 2017 ) ; Kuleshov & Ermon ( 2017 ) is to 1An easy way to tell an undirected model from a directed model is that an undirected model involves the normalizing constant ( also called the partition function in physics ) , while the directed model is self-normalized . 2Note that compared to modeling with multiple deterministic layers , modeling with multiple stochastic layers , e.g . deep Boltzmann machines ( DBMs ) ( Salakhutdinov & Hinton , 2009 ) , presents much greater challenge for model learning and thus yields inferior performance . This is observed in both directed and undirected models . pair the target random field pθ with an auxiliary directed generative model ( often called generator ) qφ ( x ) parameterized by φ , which approximates sampling from the target random field . Learning is performed by maximizing the log-likelihood of training data under pθ or some bound of the log-likelihood , and simultaneously minimizing some divergence between the target random field pθ and the auxiliary generator qφ . Different learning algorithms differ in the objective functions used in the joint training of pθ and qφ , and thus have different computational and statistical properties ( partly illustrated in Figure 1 ) . For example , minimizing the exclusive-divergence KL [ qφ||pθ ] , ∫ qφ log ( qφ/pθ ) w.r.t . φ , as employed in Kim & Bengio ( 2016 ) , involves the intractable entropy term and tends to enforce the generator to seek modes , yielding missing modes . There are also other factors , e.g . modeling discrete or continuous data , different choices of the target RF and the generator , which lead to different algorithms . We leave detailed comparison and connection of our approach with existing studies to section 3 ( related work ) . In this paper , we propose to use inclusive-divergence minimized auxiliary generators ( section 2.1 ) . And particularly for continuous data ( e.g . images ) , we propose to use SGLD ( stochastic gradient Langevin dynamics ( Welling & Teh , 2011 ) ) and SGHMC ( stochastic gradient Hamiltonian Monte Carlo ( Chen et al. , 2014 ) ) to exploit gradient information in model sampling with solid theoretical examination ( section 2.2 ) . The new approach , abbreviated as the inclusive-NRF approach , offers some advantages over previous methods . First , minimizing the inclusive-divergence KL [ pθ||qφ ] , ∫ pθ log ( pθ/qφ ) w.r.t . φ avoids the annoying entropy term and tends to drive the generator to cover modes of the target density pθ . The SGLD/SGHMC sampling further pushes the samples towards the modes of pθ . Presumably , this helps to produce Markov chains that mix fast between modes and facilitate model learning . Second , the new approach enables us to flexibly use NRFs in unsupervised/supervised image generation and semi-supervised classification ( section 2.3 ) , and empirically to the best of our knowledge , represents the best-performed random fields in these tasks . The main contributions of this paper can be summarized as follows : • We develop the inclusive-NRF approach , which learns NRFs with inclusive auxiliary generators and particularly for continuous data , exploits gradient information in model sampling with solid theoretical examination . • Inclusive-NRFs achieve state-of-the-art sample generation quality , measured by both Incep- tion Score ( IS ) and Frechet Inception Distance ( FID ) . On CIFAR-10 , we obtain unsupervised IS 8.28 ( FID 20.9 ) and supervised IS 9.06 ( FID 18.1 ) , both using unconditional generation . • Semi-supervised inclusive-NRFs show strong classification results on par with state-of- the-art DGM-based semi-supervised learning ( SSL ) methods , and simultaneously achieve superior generation , on the widely benchmarked datasets - MNIST , SVHN and CIFAR-10 . 2 THE INCLUSIVE-NRF APPROACH . Consider a random field for modeling observation x with parameter θ : pθ ( x ) = 1 Z ( θ ) exp [ uθ ( x ) ] ( 1 ) where Z ( θ ) = ∫ exp ( uθ ( x ) ) dx is the normalizing constant , uθ ( x ) is the potential function3 which assigns a scalar value to each configuration of random variable x . The general idea of neural random fields ( NRFs ) is to implement uθ ( x ) : Rdx → R , by a neural network , taking the multi-dimensional x ∈ Rdx as input and outputting the scalar uθ ( x ) ∈ R. In this manner , we can take advantage of the representation power of neural networks for RF modeling . It is usually intractable to maximize the data log-likelihood logpθ ( x̃ ) for observed x̃ , since the gradient involves expectation w.r.t . the model distribution , as shown below : ∇θ log pθ ( x̃ ) = ∇θuθ ( x̃ ) − Epθ ( x ) [ ∇θuθ ( x ) ] ( 2 ) 2.1 INTRODUCING INCLUSIVE-DIVERGENCE MINIMIZED AUXILIARY GENERATORS . In this paper , we further develop NRF learning with auxiliary generators . We are mainly concerned with modeling fixed-dimensional continuous observations x ∈ Rdx ( e.g . images ) , and choose a 3Negating the potential function defines the energy function . Algorithm 1 Learning NRFs with inclusive auxiliary generators repeat Sampling : Draw a minibatchM = { ( x̃i , xi , hi ) , i = 1 , · · · |M| } from p̃ ( x̃ ) pθ ( x ) qφ ( h|x ) ( see Algorithm 2 ) ; Updating : Update θ by ascending : 1|M| ∑ ( x̃ , x , h ) ∼M [ ∇θuθ ( x̃ ) −∇θuθ ( x ) ] ; Update φ by ascending : 1|M| ∑ ( x̃ , x , h ) ∼M∇φ log qφ ( x , h ) ; until convergence directed generative model , qφ ( x , h ) , q ( h ) qφ ( x|h ) , for the auxiliary generator , which specifically is defined as follows4 : h ∼ N ( 0 , Ih ) , x = gφ ( h ) + , ∼ N ( 0 , σ2I ) , ( 3 ) where gφ ( h ) : Rdh → Rdx is implemented as a neural network with parameter φ , which maps the latent code h to the observation space . Ih and I denote the identity matrices , with dimensionality implied by h and respectively . Drawing samples from the generator qφ ( x , h ) is simple as it is just ancestral sampling from a 2-variable directed graphical model . Suppose that data D = { x̃1 , · · · , x̃n } , consisting of n observations , are drawn from the true but unknown data distribution p0 ( · ) . p̃ ( x̃ ) , 1n ∑n k=1 δ ( x̃ − x̃k ) denotes the empirical data distribution . Then we formulate the maximum likelihood learning of pθ ( x ) with the inclusive-divergence minimized generator qφ ( x ) as optimizing5 min θ KL [ p̃ ( x̃ ) ||pθ ( x̃ ) ] min φ KL [ pθ ( x ) ||qφ ( x ) ] ( 4 ) The first line of Eq . ( 4 ) is equivalent to maximum likelihood training of the target RF pθ under the empirical data p̃ , which requires sampling from pθ . Simultaneously , the second line optimizes the generator qφ to be close to pθ so that qφ becomes a good proposal for sampling from pθ . It can be easily seen that the gradients w.r.t . θ and φ ( to be ascended ) are defined as follows : { ∇θ = Ep̃ ( x̃ ) [ ∇θ log pθ ( x̃ ) ] = Ep̃ ( x̃ ) [ ∇θuθ ( x̃ ) ] − Epθ ( x ) [ ∇θuθ ( x ) ] , ∇φ = Epθ ( x ) [ ∇φ log qφ ( x ) ] = Epθ ( x ) qφ ( h|x ) [ ∇φ log qφ ( x , h ) ] . ( 5 ) Both lines of Eq . ( 5 ) hold , as proved in Proposition 1 in the Supplement . In practice , we calculate noisy gradient estimators , and apply minibatch based stochastic gradient descent ( SGD ) to solve the optimization problem Eq . ( 4 ) , as shown in Algorithm 1 . 2.2 APPLYING SGLD/SGHMC FOR MODEL SAMPLING . In Algorithm 1 , we need to draw samples from pθ ( x ) qφ ( h|x ) given current θ and φ . For continuous observations , SGLD ( stochastic gradient Langevin dynamics ) ( Welling & Teh , 2011 ) and SGHMC ( Stochastic Gradient Hamiltonian Monte Carlo ) ( Chen et al. , 2014 ) sampling provide mechanisms for exploiting ( stochastic ) gradients of the target density pθ ( x ) qφ ( h|x ) , enabling efficient exploration of the state space . We take the theoretical results about SGLD from Teh et al . ( 2016 ) and SGHMC from Chen et al . ( 2014 ) , which are briefly summarized in Theorem 1 in the Supplement , and apply them in the sampling step in Algorithm 1 . Denoting the target density as p ( z ; λ ) with given λ , Theorem 1 shows that SGLD/SGHMC , by utilizing ∂∂z log p ( z ; λ ) , yields a non-homogeneous Markov chain { z ( l ) , l ≥ 1 } , which converges to the equilibrium distribution p ( z ; λ ) . By letting z , ( x , h ) , p ( z ; λ ) , pθ ( x ) qφ ( h|x ) , λ , ( θ , φ ) T in Theorem 1 , we can perform the sampling step in Algorithm 1 by running |M| parallel chains , each chain being executed as shown 4Note that during training , σ2 is absorbed into the learning rates and does not need to be estimated . 5Such optimization using two objectives is employed in a number of familiar learning methods , such as GAN with logD trick ( Goodfellow et al. , 2014 ) , wake-sleep algorithm ( Hinton et al. , 1995 ) . Algorithm 2 Sampling from pθ ( x ) qφ ( h|x ) 1 . Do ancestral sampling by the generator , namely first drawing h′ ∼ p ( h′ ) , and then drawing x′ ∼ qφ ( x′|h′ ) ; 2 . Starting from ( x′ , h′ ) = z ( 0 ) , run finite steps of SGLD/SGHMC ( l = 1 , · · · , L ) to obtain ( x , h ) = z ( L ) , which we call sample revision , according to Eq . ( 11 ) / ( 12 ) . Return ( x , h ) . in Algorithm 2 . In sample revision , the calculation of the gradient w.r.t . h , ∂∂h log p ( z ; λ ) = ∂ ∂h log qφ ( h|x ) = ∂ ∂h log qφ ( h , x ) , is straightforward . For the gradient w.r.t . x , we have ∂ ∂x log p ( z ; λ ) = ∂ ∂x log pθ ( x ) + ∂ ∂x log qφ ( h , x ) − ∂ ∂x log qφ ( x ) ≈ ∂ ∂x log pθ ( x ) . ( 6 ) The reason is that ∂∂x log qφ ( x ) can be approximated by an unbiased estimate , as proved in Proposition 2 in the Supplement : ∂ ∂x log qφ ( x ) ≈ ∂ ∂x log qφ ( h , x ) . Therefore , we can use ∂∂x log pθ ( x ) 6 as an unbiased estimate of the gradient ∂∂x log p ( z ; λ ) , and we can apply Theorem 1 in the Supplement with tractable gradients w.r.t . both x and h. Remarks . Intuitively , the generator gives a proposal ( x′ , h′ ) , and then the system follows the gradients of pθ ( x ) and qφ ( h , x ) ( w.r.t . x and h respectively ) to revise ( x′ , h′ ) to ( x , h ) . The gradient terms pull samples moving to low energy region of the random field and adjust the latent code of the generator , while the noise term brings randomness . In this manner , we obtain Markov chain samples from pθ ( x ) qφ ( h|x ) . Note that finite steps in sample revision will produce biased estimates of the gradients ∇θ and∇φ in Eq . ( 5 ) . We did not find this to pose problems to the SGD optimization in practice , as similarly found in Bornschein & Bengio ( 2015 ) and Kuleshov & Ermon ( 2017 ) , which work with biased gradient estimators .

Summary: This paper addresses an important problem of learning the random field using neural networks by using a inclusive auxiliary generator. Comparing to existing state-of-the-art methods for learning neural random fields, this paper used a the inclusive-divergence (KL divergence of the density approximate and the auxiliary generator) which avoids the intractable entropy term. SGLD/SGHMC are used to revise samples drawn from the auxiliary generator and these two sampling methods are examined theoretically.