Structured Energy Network as a dynamic loss function. Case study. A case study with multi-label Classification

1 INTRODUCTION . Structured prediction is a popular machine learning task wherein the model learns a mapping function from an input x to a multivariate-structured output y . Popular examples of this include image segmentation ( Müller , 2014 ) , extracting parse trees or semantic role labels from a text ( Palmer et al. , 2010 ) , and multi-label classification ( Belanger & McCallum , 2016 ; Gygli et al. , 2017 ) . In structured prediction , the output space Y is often extremely large . For example , in multi-label classification , the size of Y is 2L where model needs to predict output y ∈ { 0 , 1 } L. There are two key aspects that a model for structured prediction needs to balance : statistical efficiency and computational efficiency . Most models for structured prediction can be categorized into two categories : the feed-forward approach , wherein one learns a neural network that models the direct mapping between the input and the structured output , and the structured approach , in which the model explicitly models the interactions ( structure ) in the output space . The feed-forward approach learns all dimensions of y jointly in a conditionally independent manner given input x , relying on the representational power of the network to capture dependencies in output structure implicitly . While the feed-forward approach is computationally efficient , since it does not capture relationships in the label space , it lacks statistical efficiency . The traditional structured approaches , on the other hand , model a joint probability distribution P ( x , y ) that can capture the label relationships . However , due to the intractability of modeling the full joint distribution which captures every possible interaction of output space , these approaches resort to limiting the interaction terms to local subsets of the output space ( Lafferty et al. , 2001 ; Ghamrawi & McCallum , 2005 ) . In order to make inference for structured models more efficient , a recent line of work replaces the joint probability with a structured energy function E ( x , y ) ( LeCun et al. , 2006 ) , which can be thought of as an unnormalized probability distribution , allowing the model to learn arbitrary global dependencies in the output space . There have been several works that propose efficient approximate inference procedures for structured energy networks using gradient based inference ( Belanger et al. , 2017 ; Gygli et al. , 2017 ; Rooshenas et al. , 2019 ) , hence also called ’ prediction networks ’ . These models showed noticeable gains in predictive performance over feed-forward models and graphical models that assume partial structure such as full pairwise potential ( Chen et al. , 2015 ; Schwing & Urtasun , 2015 ) . Despite these efforts , the inference for energy-based models still remains relatively inefficient when compared to the feed-forward approach ( Tu & Gimpel , 2019 ) . Moreover , in our experience , the models using gradient based inference ( GBI ) are finicky to train as the training also utilizes GBI , requiring numerous hyperparameters : step size , number of iterations for GBI , and initial point to begin GBI . This raises a question : Can the energy network be used in a way that is as expressive as a full joint probability , as efficient at inference as a feed-forward approach , and also stable and easy to train ? We believe that using structured energy networks as a parameterized dynamic loss function for feed-forward networks , instead of a prediction network , can fulfill all these requirements . In this paper , we propose the Structured Energy As Loss ( SEAL ) framework that uses a trainable structured energy network ( SEN ) 1 as a loss function guiding the training of a feed-forward network . The key idea is to provide the feed-forward network access to rich relationships in the output space through a learned loss function . We also propose to learn SEN in a dynamic fashion by adjusting the energy function to be confident with the most up-to-date outputs of the feed-forward network . We show that learning the loss function dynamically leads to more efficient , more stable , and better performance . SEAL can be viewed as general-purpose framework where one can plugin various loss functions and architectures to train SEN as well as the feed-forward network . Through experiments ( §4 ) , we analyze the effect of applying different energy losses ( e.g . margin-based , regression-based ) within SEAL framework . We also propose the noise-contrastive ranking loss ( NCEranking ) for SEN which performs the best within the SEAL framework . To summarize , we introduce a general framework SEAL that interprets structured energy networks ( SEN ) as a dynamic loss functions . Through empirical evaluation on the task of multi-label classification , we analyse the impact of various loss functions for updating SEN . Finally , we propose an NCE ranking loss that is uniquely suited for the SEAL framework , and demonstrate its superior performance on 7 feature-based as well as 3 text-based multi-label classification datasets , when compared to simple feed-forward approach and various energy based models . Algorithm 1 : SEAL Algorithm Require : ( x , y ) : Training Instance Require : FΦ : Feedforward Network Require : sampling : True/False flag Require : optimizerΘ , optimizerΦ Require : T : No . of steps t← 0 Θ0 , Φ0 ← Random initialization while t < T do if sampling then S ← { y ( i ) , i = 1 , . . . , K|y ( i ) ∼ FΦt ( x ) } else S ← { FΦt ( x ) } . singleton set t← t + 1 Update Θ as Eqn . 3 using ỹ ∈ S and optimizerΘ Update Φ as Eqn . 4 using optimizerΦ 1The term comes from structured prediction energy network , minus ’ prediction ’ , as we do not use it for prediction anymore . For brevity , we use terms ’ energy network ’ and ’ strcutured energy newtork ’ interchangeably . 2 STRUCTURED ENERGY NETWORK AS LOSS ( SEAL ) This section first describes the proposed SEAL framework . LetX denote the input space , Y = { 0 , 1 } L the output space , and Ỹ = [ 0 , 1 ] L the continuous relaxation of Y . Then the structured energy EΘ : X ×Ỹ → R is defined to be a parameterized function of the input and the continuous relaxation of the output . The feedforward network FΦ : X → Ỹ is defined to be a neural network that maps an input to the continuous relaxation of the output space . We denote j-th training instance pair as ( x ( j ) , y ( j ) ) ∈ D and yi ∈ { 0 , 1 } , ỹi ∈ [ 0 , 1 ] to denote i-th label dimension where i ∈ { 1 , . . . , L } . As shown in Figure 1 , SEAL consists two loss functions : the energy loss LE that trains structured energy network ( Θ ) , and feedforward loss LF that guides the training of the feedforward network ( Φ ) . We first discuss how structured energy network implicitly affects training of the feedforward network by defining LF ( Φ ) . Given a training instance ( x , y ) , the feedforward loss is defined as : LF ( Φ ) = λ1EΘ ( x , FΦ ( x ) ) − λ2 L∑ j=1 [ yi logFΦ ( x ) i + ( 1− yi ) log ( 1− FΦ ( x ) i ) ] . ( 1 ) Here , the first term involving the energy captures interaction across label space whereas binary cross-entropy considers the predictions of each label independently ( More details in Appendix B ) . Since the training of feedforward network depends on the quality of the parameterized energy network , it is critical to find the parameters Θ that produce the best loss surface for training the feedforward network . This can be done in two ways : by first training the energy separately , or by training the energy and the feedforward network simultaneously . SEAL uses the latter , which is shown to perform better ( see §5 ) than the former . We denote the former method as SEAL-static for distinction . In SEAL-static , we first estimate the energy network parameter Θ̂ over training data and optimize Φ by plugging fixed Θ̂ into LF . Given ( x , y ) ∈ D , Φ is trained as equation 2. min Φ 1 |D| ∑ D LF ( Φ ) s.t . Θ̂ = arg min Θ 1 |D| ∑ D LE ( x , y , ỹ ; Θ ) ( 2 ) We finally present SEALwhich learns Θ̂t dynamically at step t. To do so , we alternate the optimization steps of Θ and Φ . Given a training instance ( x , y ) ∈ Bt , a complete training step is given as : Θt ← Θt−1 −∇Θ 1 |Bt| ∑ Bt LE ( x , y , FΦt−1 ( x ) ; Θ ) ( 3 ) Φt ← Φt−1 −∇Φ 1 |Bt| ∑ Bt LF ( Φ ) ( 4 ) Note that , from ( 2 ) to ( 3 ) , arbitrary ỹ got replaced with FΦt−1 ( x ) in estimating Θt so that estimation of energy surface depends on model output FΦt−1 ( x ) . Further , the update step for feedforward network Φ relies on Θt rather than static Θ̂ . In both SEAL and SEAL-static , test-time predictions are performed solely using the feedfoward network as ŷi = 1 ( FΦ ( x ) i ≥ 0.5 ) . The motivation for proposing dynamic loss function in SEAL is as following . It would be useful to learn perfect energy surface , if possible , by estimating Θ̂ that is static . However , doing so is challenging as we are estimating energy surface on the joint space of x , ỹ , a high-dimensional continuous space with limited data and resource . Instead , we hypothesize that it is more important to concentrate resource in depicting accurate energy surface around the current input , output pair ( x , FΦ ( x ) ) . In SEAL , utilizing the fact that we have access to trainable loss function , we dynamically adapt energy surface so that the region of interest for training feedforward network is well represented . We now discuss various energy loss ( margin-based , regression-based , and contrastive-sampling-based ) that can be plugged into LE in SEAL . For brevity , we use LE ( Θ ) and LE ( x , y , ỹ ; Θ ) interchangeably where again ( x , y ) denotes labeled data and ỹ denotes probability vector . We defer the description of specific energy network structure to the experiment section ( §4 ) as SEAL framework can work with arbitrary network structures . Margin-based ( LE−margin ) SPEN ( Belanger & McCallum , 2016 ) learned structured energy network with SSVM loss ( Taskar et al. , 2004 ; Tsochantaridis et al. , 2004 ) so that energy functions learn to have sufficient energy difference , larger than the margin ∆ ( ỹ , y ) , between arbitrary output ỹ and true output y . To examine the effect of the margin-based loss in SEAL , we follow SPEN and utilize the SSVM loss as LE−margin = ∑ x , y max ỹ [ ∆ ( ỹ , y ) − EΘ ( x , ỹ ) + EΘ ( x , y ) ] + ( 5 ) Regression-based ( LE−regression ) Deep Value Network ( DVN ) ( Gygli et al. , 2017 ) attempts to learn an energy network which directly outputs a score that is similar to the metric s ( ỹ , y ) of interest , such as F1 score , that compares arbitrary ỹ with true output y . Following DVN , by making score s ( · ) 2 and −E ( · ) to be between [ 0 , 1 ] , we express regression loss as cross entropy LE−regression = −s ( ỹ , y ) log−EΘ ( x , ỹ ) − ( 1− s ( ỹ , y ) ) log ( 1 + EΘ ( x , ỹ ) ) . ( 6 ) Noise-contrastive ranking ( LE−NCEranking and LE−ranking ) In SEAL , we are interested in capturing the output region that feedforward network has a high probability . We ask , rather than just taking a single point ỹ = FΦ ( x ) from a feedforward network , whether sampling many discrete binary vectors from ỹ and taking all those samples into consideration for learning EΘ could lead in better estimation of energy surface . Motivated from noise contrastive estimation ( NCE ) ( Ma & Collins , 2018 ) , we ask whether energy network EΘ trained to contrast the true output y from groups of K negative samples drawn from feedforward output FΦ ( x ) can induce a good loss for teaching Φ . The intuition is as the ỹ = FΦ ( x ) becomes better and better , the loss surface teaching feedforward network should become more fine grained as ỹ and samples from ỹ will be already close to the true y . Before we discuss LE−NCEranking of our choice , we first review original form of NCE ranking loss from Ma & Collins ( 2018 ) . For K samples y ( k ) ∼ PN , k = 1 , . . . , K drawn from noise distribution PN and rewriting y as y ( 0 ) without loss of generality , NCE ranking loss is defined as log exp ( s ( x , y ( 0 ) ; Θ ) ) ∑K k=1 exp ( s ( x , y ( k ) ; Θ ) ) , where s ( x , y ; Θ ) = −EΘ ( x , y ) − logPN ( y ) . ( 7 ) Minimization of this loss makes P ( y|x ; Θ ) = exp ( −EΘ ( x , y ) ) ∑ y∈Y exp ( −EΘ ( x , y ) ) as an unbiased estimator of true distribution P ( Y |X ) . Thus , given perfect energy E ( · ) , minimizing −EΘ ( x , y ) with respect to y in LF would be equivalent to maximizing the estimate of − logP ( y|x ) . From the presented NCE ranking method , we propose to use P ( y|x ; Φt ) = ∏ i P ( yi|x ; Φt ) in place of PN ( y ) . The novelty of this proposal is that we view the output of feedforward network that we train as a noise distribution , and that the energy model that contrasts noise distribution teaches the noise distribution as well through our SEAL framework ( equation 3 ) . In short , we get LE−NCEranking when we plug in s ( x , y ; Θ ) = −EΘ ( x , y ) − logPΦ ( y|x ; Φt ) into equation 7 , i.e . LE−NCEranking = log exp ( s ( x , y ( 0 ) ; Θ ) ) ∑K k=1 exp ( s ( x , y ( k ) ; Θ ) ) , s ( x , y ; Θ ) = −EΘ ( x , y ) − logPΦ ( y|x ; Φt ) . ( 8 ) This achieves two benefits that we have in mind in using NCE . First benefit is that it captures the region of interest by sampling from the output distribution of Φt . Second benefit is that it brings more efficient NCE method . NCE method is known to work best when the noise distribution is close to the data distribution ( Gutmann & Hirayama , 2012 ) but not exactly the same . In our SEAL learning procedure , we hypothesize that P ( y|x ; Φt ) becomes closer to the data distribution as training steps proceed . Lastly , motivated that NCE can estimate the true distribution , we propose yet another loss function that can estimate the difference between the true probability P ( y|x ) and feedforward network probability . We show in appendix A , if we use plain ranking loss that sets s ( x , y ; Θ ) = −EΘ ( x , y ) in equation 7 , unbiased estimator of P ( y|x ) now becomes PN ( y ) exp ( −EΘ ( x , y ) ) ∑ y∈Y PN ( y ) exp ( −EΘ ( x , y ) ) ( 9 ) . This 2In this paper , we adopt soft F1 score s ( ỹ , y ) from Gygli et al . ( 2017 ) that is defined on the continuous ỹ ∈ [ 0 , 1 ] L. means if we take a derivative of EΘwith respect to y , then we are taking derivative with respect to − logP ( y|x ) + logPN ( y ) . With the trick of replacing the PN ( y ) with P ( y|x ; Φt ) in SEAL , we hypothesize that loss captured by energy network can focus only on the difference between true and feedforward probability which may end up being more informative surface . In short , we get plain ranking loss LE−ranking by plugging s ( x , y ; Θ ) = −EΘ ( x , y ) into equation 7 , resulting in LE−ranking = log exp ( −E ( x , y ( 0 ) ; Θ ) ) ∑K k=1 exp ( −E ( x , y ( k ) ; Θ ) ) . ( 10 )

Summary: The paper presents Structured Energy As Loss (SEAL) that uses structured energy networks as a trainable loss for a feedforward network. SEAL is a general framework that supports multiple loss functions, including margin-based loss, regression-based loss, and NCE-based loss, and generalizes the recent approach in [Tu et al., 2020]. The paper presents extensive experiments on multi-label classification on a range of feature-based and text-based datasets. SEAL shows significant improvement over standard cross entropy trained networks, as well as approaches based on Structured Prediction Energy Networks (SPEN).