Structured Energy Network as a dynamic loss function. Case study. A case study with multi-label Classification

1 INTRODUCTION . Structured prediction is a popular machine learning task wherein the model learns a mapping function from an input x to a multivariate-structured output y . Popular examples of this include image segmentation ( Müller , 2014 ) , extracting parse trees or semantic role labels from a text ( Palmer et al. , 2010 ) , and multi-label classification ( Belanger & McCallum , 2016 ; Gygli et al. , 2017 ) . In structured prediction , the output space Y is often extremely large . For example , in multi-label classification , the size of Y is 2L where model needs to predict output y ∈ { 0 , 1 } L. There are two key aspects that a model for structured prediction needs to balance : statistical efficiency and computational efficiency . Most models for structured prediction can be categorized into two categories : the feed-forward approach , wherein one learns a neural network that models the direct mapping between the input and the structured output , and the structured approach , in which the model explicitly models the interactions ( structure ) in the output space . The feed-forward approach learns all dimensions of y jointly in a conditionally independent manner given input x , relying on the representational power of the network to capture dependencies in output structure implicitly . While the feed-forward approach is computationally efficient , since it does not capture relationships in the label space , it lacks statistical efficiency . The traditional structured approaches , on the other hand , model a joint probability distribution P ( x , y ) that can capture the label relationships . However , due to the intractability of modeling the full joint distribution which captures every possible interaction of output space , these approaches resort to limiting the interaction terms to local subsets of the output space ( Lafferty et al. , 2001 ; Ghamrawi & McCallum , 2005 ) . In order to make inference for structured models more efficient , a recent line of work replaces the joint probability with a structured energy function E ( x , y ) ( LeCun et al. , 2006 ) , which can be thought of as an unnormalized probability distribution , allowing the model to learn arbitrary global dependencies in the output space . There have been several works that propose efficient approximate inference procedures for structured energy networks using gradient based inference ( Belanger et al. , 2017 ; Gygli et al. , 2017 ; Rooshenas et al. , 2019 ) , hence also called ’ prediction networks ’ . These models showed noticeable gains in predictive performance over feed-forward models and graphical models that assume partial structure such as full pairwise potential ( Chen et al. , 2015 ; Schwing & Urtasun , 2015 ) . Despite these efforts , the inference for energy-based models still remains relatively inefficient when compared to the feed-forward approach ( Tu & Gimpel , 2019 ) . Moreover , in our experience , the models using gradient based inference ( GBI ) are finicky to train as the training also utilizes GBI , requiring numerous hyperparameters : step size , number of iterations for GBI , and initial point to begin GBI . This raises a question : Can the energy network be used in a way that is as expressive as a full joint probability , as efficient at inference as a feed-forward approach , and also stable and easy to train ? We believe that using structured energy networks as a parameterized dynamic loss function for feed-forward networks , instead of a prediction network , can fulfill all these requirements . In this paper , we propose the Structured Energy As Loss ( SEAL ) framework that uses a trainable structured energy network ( SEN ) 1 as a loss function guiding the training of a feed-forward network . The key idea is to provide the feed-forward network access to rich relationships in the output space through a learned loss function . We also propose to learn SEN in a dynamic fashion by adjusting the energy function to be confident with the most up-to-date outputs of the feed-forward network . We show that learning the loss function dynamically leads to more efficient , more stable , and better performance . SEAL can be viewed as general-purpose framework where one can plugin various loss functions and architectures to train SEN as well as the feed-forward network . Through experiments ( §4 ) , we analyze the effect of applying different energy losses ( e.g . margin-based , regression-based ) within SEAL framework . We also propose the noise-contrastive ranking loss ( NCEranking ) for SEN which performs the best within the SEAL framework . To summarize , we introduce a general framework SEAL that interprets structured energy networks ( SEN ) as a dynamic loss functions . Through empirical evaluation on the task of multi-label classification , we analyse the impact of various loss functions for updating SEN . Finally , we propose an NCE ranking loss that is uniquely suited for the SEAL framework , and demonstrate its superior performance on 7 feature-based as well as 3 text-based multi-label classification datasets , when compared to simple feed-forward approach and various energy based models . Algorithm 1 : SEAL Algorithm Require : ( x , y ) : Training Instance Require : FΦ : Feedforward Network Require : sampling : True/False flag Require : optimizerΘ , optimizerΦ Require : T : No . of steps t← 0 Θ0 , Φ0 ← Random initialization while t < T do if sampling then S ← { y ( i ) , i = 1 , . . . , K|y ( i ) ∼ FΦt ( x ) } else S ← { FΦt ( x ) } . singleton set t← t + 1 Update Θ as Eqn . 3 using ỹ ∈ S and optimizerΘ Update Φ as Eqn . 4 using optimizerΦ 1The term comes from structured prediction energy network , minus ’ prediction ’ , as we do not use it for prediction anymore . For brevity , we use terms ’ energy network ’ and ’ strcutured energy newtork ’ interchangeably . 2 STRUCTURED ENERGY NETWORK AS LOSS ( SEAL ) This section first describes the proposed SEAL framework . LetX denote the input space , Y = { 0 , 1 } L the output space , and Ỹ = [ 0 , 1 ] L the continuous relaxation of Y . Then the structured energy EΘ : X ×Ỹ → R is defined to be a parameterized function of the input and the continuous relaxation of the output . The feedforward network FΦ : X → Ỹ is defined to be a neural network that maps an input to the continuous relaxation of the output space . We denote j-th training instance pair as ( x ( j ) , y ( j ) ) ∈ D and yi ∈ { 0 , 1 } , ỹi ∈ [ 0 , 1 ] to denote i-th label dimension where i ∈ { 1 , . . . , L } . As shown in Figure 1 , SEAL consists two loss functions : the energy loss LE that trains structured energy network ( Θ ) , and feedforward loss LF that guides the training of the feedforward network ( Φ ) . We first discuss how structured energy network implicitly affects training of the feedforward network by defining LF ( Φ ) . Given a training instance ( x , y ) , the feedforward loss is defined as : LF ( Φ ) = λ1EΘ ( x , FΦ ( x ) ) − λ2 L∑ j=1 [ yi logFΦ ( x ) i + ( 1− yi ) log ( 1− FΦ ( x ) i ) ] . ( 1 ) Here , the first term involving the energy captures interaction across label space whereas binary cross-entropy considers the predictions of each label independently ( More details in Appendix B ) . Since the training of feedforward network depends on the quality of the parameterized energy network , it is critical to find the parameters Θ that produce the best loss surface for training the feedforward network . This can be done in two ways : by first training the energy separately , or by training the energy and the feedforward network simultaneously . SEAL uses the latter , which is shown to perform better ( see §5 ) than the former . We denote the former method as SEAL-static for distinction . In SEAL-static , we first estimate the energy network parameter Θ̂ over training data and optimize Φ by plugging fixed Θ̂ into LF . Given ( x , y ) ∈ D , Φ is trained as equation 2. min Φ 1 |D| ∑ D LF ( Φ ) s.t . Θ̂ = arg min Θ 1 |D| ∑ D LE ( x , y , ỹ ; Θ ) ( 2 ) We finally present SEALwhich learns Θ̂t dynamically at step t. To do so , we alternate the optimization steps of Θ and Φ . Given a training instance ( x , y ) ∈ Bt , a complete training step is given as : Θt ← Θt−1 −∇Θ 1 |Bt| ∑ Bt LE ( x , y , FΦt−1 ( x ) ; Θ ) ( 3 ) Φt ← Φt−1 −∇Φ 1 |Bt| ∑ Bt LF ( Φ ) ( 4 ) Note that , from ( 2 ) to ( 3 ) , arbitrary ỹ got replaced with FΦt−1 ( x ) in estimating Θt so that estimation of energy surface depends on model output FΦt−1 ( x ) . Further , the update step for feedforward network Φ relies on Θt rather than static Θ̂ . In both SEAL and SEAL-static , test-time predictions are performed solely using the feedfoward network as ŷi = 1 ( FΦ ( x ) i ≥ 0.5 ) . The motivation for proposing dynamic loss function in SEAL is as following . It would be useful to learn perfect energy surface , if possible , by estimating Θ̂ that is static . However , doing so is challenging as we are estimating energy surface on the joint space of x , ỹ , a high-dimensional continuous space with limited data and resource . Instead , we hypothesize that it is more important to concentrate resource in depicting accurate energy surface around the current input , output pair ( x , FΦ ( x ) ) . In SEAL , utilizing the fact that we have access to trainable loss function , we dynamically adapt energy surface so that the region of interest for training feedforward network is well represented . We now discuss various energy loss ( margin-based , regression-based , and contrastive-sampling-based ) that can be plugged into LE in SEAL . For brevity , we use LE ( Θ ) and LE ( x , y , ỹ ; Θ ) interchangeably where again ( x , y ) denotes labeled data and ỹ denotes probability vector . We defer the description of specific energy network structure to the experiment section ( §4 ) as SEAL framework can work with arbitrary network structures . Margin-based ( LE−margin ) SPEN ( Belanger & McCallum , 2016 ) learned structured energy network with SSVM loss ( Taskar et al. , 2004 ; Tsochantaridis et al. , 2004 ) so that energy functions learn to have sufficient energy difference , larger than the margin ∆ ( ỹ , y ) , between arbitrary output ỹ and true output y . To examine the effect of the margin-based loss in SEAL , we follow SPEN and utilize the SSVM loss as LE−margin = ∑ x , y max ỹ [ ∆ ( ỹ , y ) − EΘ ( x , ỹ ) + EΘ ( x , y ) ] + ( 5 ) Regression-based ( LE−regression ) Deep Value Network ( DVN ) ( Gygli et al. , 2017 ) attempts to learn an energy network which directly outputs a score that is similar to the metric s ( ỹ , y ) of interest , such as F1 score , that compares arbitrary ỹ with true output y . Following DVN , by making score s ( · ) 2 and −E ( · ) to be between [ 0 , 1 ] , we express regression loss as cross entropy LE−regression = −s ( ỹ , y ) log−EΘ ( x , ỹ ) − ( 1− s ( ỹ , y ) ) log ( 1 + EΘ ( x , ỹ ) ) . ( 6 ) Noise-contrastive ranking ( LE−NCEranking and LE−ranking ) In SEAL , we are interested in capturing the output region that feedforward network has a high probability . We ask , rather than just taking a single point ỹ = FΦ ( x ) from a feedforward network , whether sampling many discrete binary vectors from ỹ and taking all those samples into consideration for learning EΘ could lead in better estimation of energy surface . Motivated from noise contrastive estimation ( NCE ) ( Ma & Collins , 2018 ) , we ask whether energy network EΘ trained to contrast the true output y from groups of K negative samples drawn from feedforward output FΦ ( x ) can induce a good loss for teaching Φ . The intuition is as the ỹ = FΦ ( x ) becomes better and better , the loss surface teaching feedforward network should become more fine grained as ỹ and samples from ỹ will be already close to the true y . Before we discuss LE−NCEranking of our choice , we first review original form of NCE ranking loss from Ma & Collins ( 2018 ) . For K samples y ( k ) ∼ PN , k = 1 , . . . , K drawn from noise distribution PN and rewriting y as y ( 0 ) without loss of generality , NCE ranking loss is defined as log exp ( s ( x , y ( 0 ) ; Θ ) ) ∑K k=1 exp ( s ( x , y ( k ) ; Θ ) ) , where s ( x , y ; Θ ) = −EΘ ( x , y ) − logPN ( y ) . ( 7 ) Minimization of this loss makes P ( y|x ; Θ ) = exp ( −EΘ ( x , y ) ) ∑ y∈Y exp ( −EΘ ( x , y ) ) as an unbiased estimator of true distribution P ( Y |X ) . Thus , given perfect energy E ( · ) , minimizing −EΘ ( x , y ) with respect to y in LF would be equivalent to maximizing the estimate of − logP ( y|x ) . From the presented NCE ranking method , we propose to use P ( y|x ; Φt ) = ∏ i P ( yi|x ; Φt ) in place of PN ( y ) . The novelty of this proposal is that we view the output of feedforward network that we train as a noise distribution , and that the energy model that contrasts noise distribution teaches the noise distribution as well through our SEAL framework ( equation 3 ) . In short , we get LE−NCEranking when we plug in s ( x , y ; Θ ) = −EΘ ( x , y ) − logPΦ ( y|x ; Φt ) into equation 7 , i.e . LE−NCEranking = log exp ( s ( x , y ( 0 ) ; Θ ) ) ∑K k=1 exp ( s ( x , y ( k ) ; Θ ) ) , s ( x , y ; Θ ) = −EΘ ( x , y ) − logPΦ ( y|x ; Φt ) . ( 8 ) This achieves two benefits that we have in mind in using NCE . First benefit is that it captures the region of interest by sampling from the output distribution of Φt . Second benefit is that it brings more efficient NCE method . NCE method is known to work best when the noise distribution is close to the data distribution ( Gutmann & Hirayama , 2012 ) but not exactly the same . In our SEAL learning procedure , we hypothesize that P ( y|x ; Φt ) becomes closer to the data distribution as training steps proceed . Lastly , motivated that NCE can estimate the true distribution , we propose yet another loss function that can estimate the difference between the true probability P ( y|x ) and feedforward network probability . We show in appendix A , if we use plain ranking loss that sets s ( x , y ; Θ ) = −EΘ ( x , y ) in equation 7 , unbiased estimator of P ( y|x ) now becomes PN ( y ) exp ( −EΘ ( x , y ) ) ∑ y∈Y PN ( y ) exp ( −EΘ ( x , y ) ) ( 9 ) . This 2In this paper , we adopt soft F1 score s ( ỹ , y ) from Gygli et al . ( 2017 ) that is defined on the continuous ỹ ∈ [ 0 , 1 ] L. means if we take a derivative of EΘwith respect to y , then we are taking derivative with respect to − logP ( y|x ) + logPN ( y ) . With the trick of replacing the PN ( y ) with P ( y|x ; Φt ) in SEAL , we hypothesize that loss captured by energy network can focus only on the difference between true and feedforward probability which may end up being more informative surface . In short , we get plain ranking loss LE−ranking by plugging s ( x , y ; Θ ) = −EΘ ( x , y ) into equation 7 , resulting in LE−ranking = log exp ( −E ( x , y ( 0 ) ; Θ ) ) ∑K k=1 exp ( −E ( x , y ( k ) ; Θ ) ) . ( 10 )

Summary: This paper proposes a framework to use Structured Prediction Energy Network (SPEN, a prior work) as a loss function. Previously SPEN is a deep architecture to learn an energy function of candidate labels, which captures structural dependencies between labels that would lead to intractable graphical models. The author of this paper finds out that it is effective to jointly learn the parameters in SPEN and the parameters in the feed-forward network (MLP). In the training time, the model alternatingly minimizes the parameters in SPEN by fixing the parameters in MLP; and then minimizes the parameters in MLP by fixing the parameters in SPEN. Empirical results in 10 multi-label datasets are given, which shows good performance boost from using the proposed methods.