To Relieve Your Headache of Training an MRF, Take AdVIL

1 INTRODUCTION . Markov random fields ( MRFs ) find applications in a variety of machine learning areas ( Krähenbühl & Koltun , 2011 ; Salakhutdinov & Larochelle , 2010 ; Lafferty et al. , 2001 ) . In particular , one famous example is conditional random fields ( Lafferty et al. , 2001 ) , a conditional version of MRFs that was developed to address the limitations ( e.g. , local dependency and label bias ) of directed models for sequential data ( e.g. , hidden Markov models and other discriminative Markov models based on directed graphical models ) . However , the inference and learning of general MRFs are challenging due to the presence of a global normalizing factor , i.e . partition function , especially when latent variables are present . Extensive efforts have been devoted to developing approximate methods . On one hand , sample-based methods ( Neal , 1993 ) and variational approaches ( Jordan et al. , 1999 ; Welling & Sutton , 2005 ; Salakhutdinov & Larochelle , 2010 ) are proposed to infer the latent variables . On the other hand , extensive work ( Meng & Wong , 1996 ; Neal , 2001 ; Hinton , 2002 ; Tieleman , 2008 ; Wainwright et al. , 2005 ; Wainwright & Jordan , 2006 ) has been done to estimate the partition function . Among these methods , contrastive divergence ( Hinton , 2002 ) is proven effective in certain types of models . Most of the existing methods highly depend on the model structure and require model-specific analysis in new applications , which makes it important to develop black-box inference and learning methods . Previous work ( Ranganath et al. , 2014 ; Schulman et al. , 2015 ) shows the ability to automatically infer the latent variables and obtain gradient estimate in directed models . However , there is no black-box learning method for undirected models except the recent work of NVIL ( Kuleshov & Ermon , 2017 ) . NVIL introduces a variational distribution and derives an upper bound of the partition function in a general MRF , in the same spirit as amortized inference ( Kingma & Welling , 2013 ; Rezende et al. , 2014 ; Mnih & Gregor , 2014 ) for directed models . NVIL has several advantages over existing methods , including the ability of black-box learning , tracking the partition function during training and getting approximate samples efficiently during testing . However , NVIL also comes with two disadvantages : ( 1 ) it leaves the inference problem of MRFs unsolved1 and only trains simple MRFs with tractable ∗Dept . of Comp . Sci . & Tech. , BNRist Center , Institute for AI , THBI Lab , Tsinghua University , Beijing , 100084 , China †University of Amsterdam , and the Canadian Institute for Advanced Research ( CIFAR ) . 1NVIL ( Kuleshov & Ermon , 2017 ) presents a hybrid model . The “ inference ” in the title refers to directed part but not for an MRF . posteriors , and ( 2 ) the upper bound of the partition function can be underestimated ( Kuleshov & Ermon , 2017 ) , resulting in sub-optimal solutions on high-dimensional data . We propose Adversarial Variational Inference and Learning ( AdVIL ) to relieve some headache of learning an MRF model . AdVIL is a black-box inference and learning method that partly solves the two problems of NVIL and retains the advantages of NVIL at the same time . First , AdVIL introduces a variational encoder to infer the latent variables , which provides an upper bound of the free energy . Second , AdVIL introduces a variational decoder for the MRF , which provides a lower bound of the log partition function . The two variational distributions provide an estimate of the negative log-likelihood of the MRF . On one hand , the estimate is in an intuitive form of an approximate contrastive free energy , which is expressed in terms of the expected energy and the ( conditional ) entropy of the corresponding variational distribution . On the other hand , similar to GAN ( Goodfellow et al. , 2014 ) , the estimate is a minimax optimization problem , which is solved by stochastic gradient descent ( SGD ) in an alternating manner . Theoretically , our algorithm is convergent if the variational decoder approximates the model well . This motivates us to introduce an auxiliary variable to enhance the flexibility of the variational decoder , whose entropy is approximated by the third variational trick . We evaluate AdVIL in various undirected generative models , including restricted Boltzmann machines ( RBM ) ( Ackley et al. , 1985 ) , deep Boltzmann machines ( DBM ) ( Salakhutdinov & Hinton , 2009 ) , and Gaussian restricted Boltzmann machines ( GRBM ) ( Hinton & Salakhutdinov , 2006 ) , on several real datasets . We empirically demonstrate that ( 1 ) compared to the black-box NVIL ( Kuleshov & Ermon , 2017 ) method , AdVIL provides a tighter estimate of the log partition function and achieves much better log-likelihood results ; and ( 2 ) compared to contrastive divergence based methods ( Hinton , 2002 ; Welling & Sutton , 2005 ) , AdVIL can deal with a broader family of MRFs without model-specific analysis and obtain better results when the model structure gets complex as in DBM . 2 BACKGROUND . We consider a general case where the model consists of both visible variables v and latent variables h. An MRF defines the joint distribution over v and h as P ( v , h ) = e −E ( v , h ) Z , where E denotes the associated energy function that assigns a scalar value for a given configuration of ( v , h ) and Z is the partition function such that Z = ∫ v , h e−E ( v , h ) dvdh . Let PD ( v ) denote the empirical distribution of the training data . Minimizing the negative loglikelihood ( NLL ) of an MRF is a commonly chosen learning criterion and it is given by : L ( θ ) : =− EPD ( v ) [ log ∫ h e−E ( v , h ) Z dh ] , ( 1 ) where θ denotes the trainable parameters in E . Further , the gradient of θ is : ∇θL ( θ ) = EPD ( v ) [ ∇θF ( v ) ] − EP ( v ) [ ∇θF ( v ) ] , ( 2 ) where F ( v ) = − log ∫ h e−E ( v , h ) dh denotes the free energy and the gradient in Eqn . ( 2 ) is the difference of the free energy in two phases . In the first positive phase , the expectation of the free energy under the data distribution is decreased . In the second negative phase , the expectation of the free energy under the model distribution is increased . Unfortunately , both the NLL in Eqn . ( 1 ) and its gradient in Eqn . ( 2 ) are intractable in general for two reasons . First , the integral of the latent variables in Eqn . ( 1 ) or equivalently the computation of the free energy in Eqn . ( 2 ) is intractable . Second , the computation of the partition function in Eqn . ( 1 ) or equivalently the negative phase in Eqn . ( 2 ) is intractable . Variational inference . Extensive work introduces deterministic approximations for the intractability of inference , including the mean-field approximation ( Welling & Hinton , 2002 ; Salakhutdinov & Hinton , 2009 ) , the Kikuchi and Bethe approximations ( Welling & Sutton , 2005 ) and the recognition model approach ( Salakhutdinov & Larochelle , 2010 ) . In this line of work , the intractability of the partition function is addressed using Monte Carlo based methods . Contrastive free energy . Contrastive divergence ( CD ) ( Hinton , 2002 ) addresses the intractability of the partition function by approximating the negative phase in Eqn . ( 2 ) as follows : ∇θL ( θ ) = EPD ( v ) [ ∇θF ( v ) ] − EPCD ( v ) [ ∇θF ( v ) ] , ( 3 ) where PCD ( v ) denotes the empirical distribution obtained by starting from a data point and running several steps of Gibbs sampling according to the model distribution and the free energy F ( v ) is assumed to be tractable . Existing methods ( Welling & Hinton , 2002 ; Welling & Sutton , 2005 ) approximate F ( v ) using certain function G ( v ) and the gradient of θ is : ∇θL ( θ ) ≈ EPD ( v ) [ ∇θG ( v ) ] − EPCD ( v ) [ ∇θG ( v ) ] . ( 4 ) Although these generalized methods exist , it is nontrivial to extend CD-based methods to general MRFs because the Gibbs sampling procedure is highly dependent on the model structure . Black-box learning . The recent work of NVIL ( Kuleshov & Ermon , 2017 ) addresses the intractability of the partition function in a black-box manner via a variational upper bound of the partition function : Eq ( v ) [ P̃ ( v ) 2 q ( v ) 2 ] ≥ Z2 , ( 5 ) where P̃ ( v ) = e−F ( v ) is the unnormalized marginal distribution on v and q ( v ) is a neural variational distribution . As a black-box learning method , NVIL potentially allows application to broader model families and improves the capabilities of probabilistic programming systems ( Carpenter et al. , 2017 ) . Though promising , NVIL leaves the intractability of inference in an MRF unsolved , and the bound in Eqn . ( 5 ) is of high variance and is easily underestimated ( Kuleshov & Ermon , 2017 ) . 3 METHOD . As stated above , the black-box inference and learning of MRFs are still largely open . In this paper , we make a step towards solving the problems by a new variational approach . For simplicity , we focus on the resulting objective function in this section . See Appendix A for detailed derivation . 3.1 ADVERSARIAL VARIATIONAL INFERENCE AND LEARNING . First , we rewrite the NLL of the MRF ( See an illustration in Fig . 1 ) as follows : L ( θ ) = −EPD ( v ) [ −F ( v ) ] + logZ , ( 6 ) where the negative free energy and the log partition function are in the form of a logarithm of an integral . Naturally , we can apply the variational trick ( Jordan et al. , 1999 ) twice and approximate the two terms individually . Due to the presence of the minus before the first term in Eqn . ( 6 ) , the two variational tricks bound the two parts of the NLL in the opposite directions , detailed as below . Formally , on one hand , we introduce an approximate posterior for the latent variables Q ( h|v ) , which is parameterized as a neural variational encoder ( See an illustration in Fig . 1 ) , to address the intractability of inference as follows : L ( θ ) ≤ EPD ( v ) Q ( h|v ) [ E ( v , h ) + logQ ( h|v ) ] + logZ : = L1 ( θ , φ ) , ( 7 ) where φ denotes the trainable parameters in Q ( h|v ) . The upper bound is derived via applying the Jensen inequality and the equality holds if and only if Q ( h|v ) = P ( h|v ) for all v. In the bound , the first term is the expected energy , which encourages Q ( h|v ) to infer latent variables that have low values of the energy function E ( v , h ) , or equivalently high probabilities of P ( v , h ) . The second term corresponds to the negative conditional entropy of Q ( h|v ) , which increases the uncertainty of Q ( h|v ) . In the paper , we denote the conditional entropy of Q ( h|v ) asH ( Q ) : = −EPD ( v ) Q ( h|v ) [ logQ ( h|v ) ] . On the other hand , we introduce an approximate sampler q ( v , h ) , which is parameterized by a neural variational decoder ( See Fig . 1 ) , to address the intractability of the partition function as follows : L1 ( θ , φ ) ≥EPD ( v ) Q ( h|v ) energy term︷ ︸︸ ︷E ( v , h ) + entropy term︷ ︸︸ ︷logQ ( h|v ) ︸ ︷︷ ︸ Positive Phase −Eq ( v , h ) energy term︷ ︸︸ ︷E ( v , h ) + entropy term︷ ︸︸ ︷log q ( v , h ) ︸ ︷︷ ︸ Negative Phase : = L2 ( θ , φ , ψ ) , ( 8 ) where ψ denotes the trainable parameters in q ( v , h ) . The lower bound is derived via applying the Jensen inequality as well , and the equality holds if and only if q ( v , h ) = P ( v , h ) . It can be seen that the lower bound given by q ( v , h ) consists of the entropy ( denoted asH ( q ) ) and energy terms , which is similar to the upper bound in Eqn . ( 7 ) , and the overall objective is in the form of approximate contrastive free energy ( Hinton , 2002 ; Welling & Sutton , 2005 ) . Because the double variational trick bounds the NLL in opposite directions as above , we have a minimax optimization problem : min θ min φ max ψ L2 ( θ , φ , ψ ) . ( 9 ) The minimax formulation has been investigated in GAN ( Goodfellow et al. , 2014 ) and it is interpreted as an adversarial game between two networks . We name our framework adversarial variational inference and learning ( AdVIL ) following the well-established literature . Note that L2 ( θ , φ , ψ ) is neither an upper bound , nor a lower bound of L ( θ ) due to the double variational trick . However , we argue that solving the optimization problem in Eqn . ( 9 ) is reasonable because ( 1 ) it is equivalent to optimizing L ( θ ) under the nonparametric assumption , which is similar to GAN ( Goodfellow et al. , 2014 ) ; and ( 2 ) it converges to a stationary point of L1 ( θ , φ ) , which is an upper bound of L ( θ ) , under a weaker assumption , as stated in the following theoretical analysis .

Summary: This manuscript proposes a new approach to fitting Markov Random Fields (MRFs).  The general structure of the algorithm is amenable to many MRF structures and can be fairly straightforwardly applied to learning on a wide variety of problems.  The theoretical analysis supports that the algorithm is reasonable.  Experimental results show strong results on several different MRF models, albeit on relatively small problems.