Structured Energy Network as a Loss Function
Abstract
Belanger & McCallum (2016) and Gygli et al.
(2017) have shown that energy networks can capture arbitrary dependencies amongst the output variables in structured prediction; however, their reliance on gradient based inference (GBI) makes the inference slow and unstable.
In this work, we propose Structured Energy As
Loss (SEAL) to take advantage of the expressivity of energy networks without incurring the high inference cost.
This is a novel learning framework that uses an energy network as a trainable loss function (loss-net) to train a separate neural network (task-net), which is then used to perform inference through a forward pass.
We establish SEAL as a general framework wherein various learning strategies like margin-based, regression, and noise-contrastive could be employed to learn the parameters of loss-net.
Through extensive evaluation on multi-label classiﬁcation, semantic role labeling, and image segmentation, we demonstrate that SEAL provides various useful design choices, is faster at inference than GBI, and leads to signiﬁcant performance gains over the baselines.
Introduction
Structured prediction is an important machine learning problem in which, given an input x , the model needs to predict the output y 2 Y [-
 ×-]1 ⇥ Y 2 ⇥ · · · ⇥ Y L where there are dependencies across y 1 , . . .
, y L .
Common structured prediction tasks include multi-label classiﬁcation (Belanger & McCallum, 2016;
Gygli et al., 2017), extracting parse trees or semantic role labels from texts (Palmer et al., 2010), and image segmentation (Müller, 2014).
Traditionally, these tasks are performed using models that explicitly represent the dependencies among output variables, such as directed and undirected graphical models, including Hidden Markov Models and Conditional Random Fields.
However, except in cases with very limited dependency patterns (Lafferty et al., 2001; Ghamrawi & McCallum, 2005), inference algorithms in these methods are difﬁcult to engineer and expensive to run because structured output spaces are exponentially large.
In a multi-label classiﬁcation task, for example, the size of the label space Y is 2 L given Y i
= { 0 , 1 } .
With their growing expressivity and capacity, it is common to use large feedforward neural networks to implicitly capture dependencies among output variables.
Given an input x , the elements of an output y are predicted in a conditionally independent manner in parallel, solely relying on the representation of input x to capture dependencies among output variables.
While the feedforward approach is computationally efﬁcient, the widely used cross-entropy loss does not directly capture 36th Conference on Neural Information Processing Systems (NeurIPS 2022).
dependencies in the output space and thus often lacks statistical efﬁciency in capturing multivariate correlations.
A recent line of methods (Belanger & McCallum, 2016; Gygli et al., 2017; Rooshenas et al., 2019) model the joint space of x and multivariate y explicitly with an energy function
E ⇥ ( x , y ) (LeCun et al., 2006), which aims to learn arbitrary global dependencies in the output space.
They relax the combinatorial output space Y to ˜ Y =
[0 , 1] L and apply gradient-based inference (GBI) directly on ˜ y to search for optima on the relaxed energy surface as arg min ˜ y E
( x , ˜ y ) .
These energy models are therefore often called structured prediction energy networks (SPENs).
SPENs show noticeable gains in predictive performance over feedforward models and traditional graphical models such as (Chen et al., 2015; Schwing & Urtasun, 2015).
Due to the use of GBI, however, inference for SPENs remains relatively inefﬁcient when compared to feedforward approaches (Tu & Gimpel, 2019).
Moreover, models using GBI are often difﬁcult to train and sensitive to sampling methods and numerous hyperparameters, including step size, number of iterations, and initialization for GBI.
This raises a question: Can energy networks be used in a way that is as expressive as SPENs, as efﬁcient at inference as feedforward approaches, and also easy to train?
To this end, we propose using structured energy networks as parameterized loss functions for feedforward networks, instead of as prediction networks.
In this paper, we introduce the S tructured E nergy
A s L oss ( SEAL ) framework which uses trainable
SPENs (loss-net 1 ) as loss functions to guide the training of other feedforward networks (task-net).
The key idea is to provide the feedforward network with access to rich dependencies in the output space through a learned loss-net (§2.1 SEAL -static).
We further propose to learn the loss-net in a dynamic fashion (§2.2 SEAL -dynamic) by adjusting the loss-net to be tailored to the most up-to-date outputs of the feedforward network; and then propose an NCE ranking loss uniquely suited for the SEAL -dynamic framework.
Using feature-based multi-label classiﬁcation as a case study, we extensively analyze the impact of various loss functions for updating the loss-net (§2.3).
We further demonstrate the faster inference and higher performance of SEAL , compared to other baseline methods, over feature-, text-based multi-label classiﬁcation, semantic role labeling, and image segmentation (§4).
Additionally, we show that energy models capture the dependencies on the output space through analyzing gradient signals in an ablation study (§5 and Appendix G).
[Figure]
Require: ( x , y ) :
Training Instance
Require: F  : Feedforward Network Require: sampling : True/False ﬂag
Require: optimizer ⇥ ,
optimizer  Require: T : No. of steps ⇥ 0 ,  Random initialization, t while t < T do if sampling then
{ y ( i ) , i = 1 , . .
, K | y
( i ) ⇠ F  t ( x ) } else
Structured Energy Network as Loss ( SEAL )
Notation.
X denotes the input space, Y = { 0 , 1 } L the label space, and ˜ Y =
[0 , 1] L the continuous relaxation of Y .
Let D =
( x ( j ) , y ( j ) )
2 X ⇥ Y} Nj =1 denote the dataset, and y i 2 { 0 , 1 } the i -th dimension in the label vector y for i 2 { 1 , . . .
, L } , and ˜ y i 2 [0 , 1] its continuous relaxation.
The feedforward network F  : X ! ˜ Y is a parametrized function that maps an input x 2 X to an output
F  ( x ) 2 ˜ Y , where F  ( x ) i is the predicted probability of label i occurring, given the input x .
The structured energy network E ⇥ : X ⇥ ˜ Y !
R is a parameterized function that maps an input x and a continuously relaxed output ˜ y to real-valued scalar energy.
For clarity, we refer to the feedforward network as task-net , and the structured energy network as loss-net .
As depicted in Figure 1, the proposed SEAL framework consists of two parametric components.
First, the loss-net E ⇥ is trained by an energy loss L E to capture an energy surface that reﬂects the latent dependencies among output variables.
Low energy indicates a high quality of a label prediction given an input.
Second, a task-net F  is trained using a loss L F that involves minimizing the energy E ⇥ estimated by the loss-net.
After training ﬁnishes, the loss-net is no longer utilized; inference time predictions are obtained solely from the task-net as ˆ y i = 1 ( F  ( x ) i  0 . 5) .
Unlike SPEN which requires iterative gradient-based inference, a task-net trained with the SEAL framework has the same computational efﬁciency at inference-time as standard feedforward networks (Appendix D), while enjoying a loss that captures the label dependencies through the loss-net.
To guide the task-net training with the loss-net E ⇥ , we propose the task-net training loss L F (  ; ⇥ ) to be a weighted sum of estimated energy and a binary-cross entropy (BCE) loss as
[Equation]
where ( x , y ) is a training instance and BCE ( y j , F  ( x ) j )
=
[ y j log F  ( x ) j + (1  y j ) log (1  F  ( x ) j )] .
By incorporating the energy term in L F , we enable the loss function to reﬂect the latent variable dependencies in the output space, whereas BCE treats L labels in a conditionally independent fashion.
We present the formulation of E ⇥ used for multi-label classiﬁcation to analyze the effect of introducing E ⇥ into L F without loss of generality.
We use energy network E ⇥ ( x, y ) that has the same structure with Belanger & McCallum (2016) and Gygli et al.
(2017) for a fair comparison with them.
The energy E ⇥ is the sum of local energy E local ⇥ and global energy E global ⇥ which are deﬁned as:
[Equation]
where b i , v , M , T E contain learnable parameters, T E is a feature network to represent an input x in the loss-net, and  ( z )
= log(1 + e z ) is the softplus activation function.
We observe from the derivative of the global energy @ E global ⇥ @
y i
[Equation]
i that all dimensions of y decides the gradient signal for each label i .
The global energy term E global ⇥ in L F , speciﬁcally, allows the gradient of the task-net loss to capture the dependency of y i on all L dimensions of y .
In contrast, the gradient of BCE for each label i only depends on the i -th dimension of y .
More details are in
Appendix A.
2.1 SEAL -static
Since the task-net loss L F depends on the quality of the loss-net, it is critical to ﬁnd the loss-net parameters ⇥ that produce the best loss surface for training the task-net  .
In order to achieve this, we ﬁrst estimate the loss-net parameters b ⇥ over training data and then optimize the task-net parameters  by plugging in the ﬁxed loss-net parameters b ⇥ into L F .
Formally, given training instances ( x , y ) 2 D , the task-net parameters  are trained in the following manner:
[Equation]
Since the loss function L F is static after pretraining b ⇥ , we call this approach SEAL -static.
We discuss various types of L E losses in §2.3.
2.2 SEAL -dynamic
Ideally, we would like to learn a perfect energy surface by estimating b ⇥ that is static.
However, estimating an accurate energy surface over a high-dimensional, continuous joint space X ⇥ ˜ Y is a challenging task.
Instead, we hypothesize that using loss-net to dynamically estimate the energy surface that is focused around the current input and task-net output pair ( x , F  ( x )) could be more beneﬁcial in teaching task-net.
To this end, we propose SEAL -dynamic which dynamically adapts loss-net so that the region of interest for training task-net is well represented.
Given a batch of training data B t , a complete training step of SEAL -dynamic is provided as
[Equation]
where the loss-net parameters ⇥ t and the task-net parameters  t are optimized in alternating fashion.
Note that, from Eq.(3) to Eq.(4), arbitrary label samples ˜ y were replaced with F  t  1 ( x ) from the tasknet during the training of ⇥ t parameters.
Thus the energy surface learned by the loss-net is adaptive to the task-net.
On the contrary, in SEAL -static, the training of the static loss-net is independent of the task-net.
Further, the update step for the task-net parameters  t relies on dynamic loss-net parameters ⇥ t rather than static b ⇥ .
In §4, we show the effectiveness of the proposed SEAL framework compared to standard feedforward networks and SPENs, and further that SEAL -dynamic often improves over
SEAL -static.
2.3 Energy losses L E
We now discuss various energy loss functions L E with different characteristics such as margin-based, regression-based, and noise-contrastive-sampling-based, that can be plugged into the SEAL framework to train the loss-net.
For brevity, we use L E ( ⇥ ) and L E ( x , y , ˜ y ; ⇥ ) interchangeably.
Margin-based (L E  margin ).
SPEN (Belanger & McCallum, 2016) learns structured energy network with the SSVM loss (Taskar et al., 2004;
Tsochantaridis et al., 2004)
so that energy function learns to predict sufﬁcient energy difference, larger than the margin  ( ˜ y , y ) , between an arbitrary output ˜ y and a true output y .
To examine the effect of the margin-based loss in SEAL , we follow SPEN and utilize the SSVM loss as
[Equation]
SPEN resorts to gradient ascent on ˜ y to solve the optimization problem arg max ˜ y  ( ˜ y , y )
E ⇥ ( x , ˜ y ) .
InfNet (Tu & Gimpel, 2018) proposes an adversarial training of the task-net  together with the energy network ⇥ .
We identify InfNet as a special case of SEAL -dynamic that uses L E  margin .
We defer a detailed description of InfNet to the related work section §3.
Regression-based (L E  regression ).
Deep Value Network (DVN) (Gygli et al., 2017) attempts to learn an energy network which directly outputs a score that is similar to the metric s (˜ y , y ) of interest, such as F1 score, that compares arbitrary ˜
y with true output y .
Following DVN, we let the true score s ( · ) 2 and  (  E ( · )) be between [0 , 1] and express the regression-based loss as cross entropy between s and  (  E ) :
[Equation]
where  ( · ) is a sigmoid function.
DVN utilizes two samples to train the model: the model prediction that minimizes the energy and an adversarial sample ˜ y that maximizes the loss L E  regression .
We follow this approach for training SEAL -static with L E  regression .
To train SEAL -dynamic with L E  regression , we simply use the output of task-net, F  t  1 ( x ) , as described in Eq.(4).
To better capture the energy surface in SEAL -dynamic, we further introduce regression-s loss
L E  regression-s =
P ˜ y 2 S L E  regression ( x , y ⇤ , ˜ y ; ⇥ ) where the sample set S is taken by continuously perturbing task-net output F  t  1 ( x ) with Gaussian noise (experimental analysis on the ‘effect of samples’ can be found at Appendix E.1).
Noise-contrastive ranking (L E  NCE ).
Lastly, we propose to use noise contrastive estimation (NCE) ranking loss (Ma & Collins, 2018) uniquely for SEAL -dynamic.
We are interested in capturing appropriate energy surface in high-probability output spaces of task-net.
To reﬂect such space accurately, we propose to train the loss-net E ⇥ to contrast the true output y with a group of K negative samples drawn from the task-net prediction F  ( x ) motivated by NCE.
Before we discuss L E  NCE for SEAL -dynamic in more detail, we ﬁrst review the original form of
NCE ranking loss from Ma & Collins (2018).
For K samples y ( k ) ⇠ P N , k = 1 , . . .
, K, drawn from a noise distribution P N , rewriting ground truth y as y (0) without loss of generality, the NCE ranking loss is deﬁned as
[Equation]
where s ( x , y ; ⇥ ) =  E ⇥ ( x , y )  log P N ( y ) . Given the noise samples y ( k ) , minimizing Eq.(8) minimizes the energy of the ground truth sample y (0) while increasing the energy of the noise samples.
In SEAL -dynamic, we propose to use Q i P ( y i | x ;  t ) , which is dynamically changing, in place of the static P N ( y ) in Eq.(8).
The NCE ranking loss is thus
[Equation]
but with
[Equation]
The novelty of this proposal is that task-net, viewed as producing a dynamic noise distribution, is in turn trained using loss-net.
The NCE ranking loss L E  NCE brings two beneﬁts in SEAL -dynamic.
First, L E  NCE captures the output distribution of the task-net rather than just a single point.
Second, L E  NCE allows estimation of a more ﬁne-grained loss surface as the training steps proceed.
In our SEAL -dynamic learning procedure, we hypothesize that
Q i P ( y i | x ;  t ) becomes closer to the true data distribution as training steps proceed.
Contrasting these noise samples to the ground truth could lead the loss surface estimated by loss-net to be more ﬁne-grained.
This is in line with the known fact that NCE methods work the best when the noise distribution is close to the data distribution but not the same (Gutmann & Hirayama, 2012).
To thoroughly experiment with the proposed NCE ranking loss L E  NCE , we also train a static energy network with L E  NCE in order to utilize it as a prediction network with GBI and as a loss-net in
SEAL -static.
We set P N ( y ) =
Q i P ( y i | x ; ) in Eq.(8), where is a ﬁxed pretrained feedforward network.
When we train an energy network E ⇥ with L E  NCE , [Equation] ) becomes an unbiased estimator of the true distribution P ( y | x ) (Ma & Collins, 2018).
Given a perfect energy
E ⇥ ( x , y ) , minimizing it with respect to y in SEAL -static and for prediction with GBI is equivalent to minimizing the estimate of  log P ( y | x ) .
Related Work
Learning with dynamic loss.
Wu et al.
and
Huang et al.
(2019) attempt to learn a dynamic loss for multi-class classiﬁcation tasks.
Both approaches try to learn a parametrized loss function that can directly increase the model’s metric score m , such as the BLEU or 0-1 accuracy, on the validation set.
Since m is usually non-differentiable, Huang et al.
(2019) view this problem as a reinforcement learning problem and attempt to learn a loss function that maximizes the expected reward.
On the other hand, Wu et al.
(2018) relax the m to be differentiable and try to learn a loss parameter ( ⇥ ) that can improve the relaxed score on feed-forward (  ) output.
There are three major differences between SEAL and the previous work.
While both Wu et al.
Huang et al.
(2019) use the parametric form  ( y T ⇥ log F  ( x )) to deﬁne the loss function, SEAL utilizes a bigger class of functions expressed using general energy network E ⇥ .
Moreover, SEAL does not require manually designed state vectors and works directly with the task-net outputs.
Lastly, while the SEAL framework is designed for structured prediction, Wu et al.
and Huang et al.
(2019) are designed for multi-class classiﬁcation and it is non-trivial to extend these methods for structured prediction.
Structured Prediction Energy Network (SPEN).
Structured prediction energy networks (Belanger & McCallum, 2016) and its variants (Gygli et al., 2017; Rooshenas et al., 2019) (together referred to as SPENs) learn an energy network E ⇥ : X ⇥ ˜ Y ! R and predict the output using gradient-based inference (GBI) with the objective
y = arg
min y E ( x , y ) .
The optimization objective for training the
SPENs relies on ﬁnding adversarial or margin-violating points in the label space using gradient based inference (GBI).
This learning process in SPENs is highly sensitive with respect to hyper-parameters such as the learning rate and initial points for the GBI, resulting in high variance in the model performance.
Nonetheless, SPENs are strong baselines for structured prediction tasks and are used as motivations for the design of SEAL .
They are further discussed in §2.3.
Jointly learning ⇥ and  .
Inference Network (InfNet) (Tu & Gimpel, 2018) is the ﬁrst work that interactively learns a structured energy network and a feedforward network.
It aims to replace the expensive GBI in SPEN with a feedforward network’s forward pass.
During its training with the SSVM loss (Eq.(6)), SPEN maximizes  ( ˜ y , y )  E ⇥ ( x , ˜ y ) with respect to ˜ y by GBI.
InfNet, on the contrary, trains a feedforward network F  to generate ˜ y
= F  ( x ) that maximizes the term.
This results in an adversarial framework (Goodfellow et al., 2014), where a feedforward network maximizes the margin-based loss while an energy network minimizes it.
The approach, however, suffers from train-test-time objective mismatch (Tu et al., 2020).
While the feedforward network serves as an adversarial sampler during training, the end goal is to learn a feedforward network that predicts outputs with low energy at test time.
Tu et al.
(2020) resolves this by learning two separate feed-forward networks: one adversarial sampler for training and the other for performing inference.
Our proposed SEAL generalizes, and differs from, InfNet by providing a framework that can employ completely independent loss functions for training loss-net and task-net.
SEAL updates task-net in the direction of reducing energy deﬁned by loss-net; loss-net is only used as a tool for evaluation of loss, and how loss-net is trained does not matter as long as it estimates a good loss surface for task-net.
The SEAL framework is not interested in producing the similar test-time behaviour of SPEN as InfNet did.
Despite the difference in the end goal, mechanically speaking, InfNet is a special case of SEAL -dynamic (i.e. SEAL -dynamic with L E  margin ) with the margin removed from the SSVM loss for the feedforward model training.
In fact, Tu et al.
(2020) reports that this special case leads to more efﬁcient and stable learning of the feedforward network .
We use this special case of InfNet to represent SEAL with L E  margin in the main experiment, and by doing so, we reinterpret the success of
InfNet as a result of utilizing energy network as a loss function, as our formulation shows, rather than the result of task-net mimicking the test-time behaviour of energy network.
Experiments
Tasks and Datasets.
We demonstrate the effectiveness of SEAL on widely deployed structured prediction tasks: feature and text-based multi-label classiﬁcation (MLC), semantic role labeling (SRL), and image segmentation.
Feature-based MLC is a standard evaluation task in previous works on SPENs (Belanger & McCallum, 2016; Gygli et al., 2017; Tu & Gimpel, 2018).
We utilizefeature-based MLC datasets which cover various label sizes (ranging from 27 to 4k), training sizes, and input characteristics (binary, real) as shown in Table 5 to extensively compare baselines and proposed SEAL approaches: cross-entropy (CE), energy network, SEAL -static, and SEAL -dynamic.
Then we verify that SEAL -dynamic, which outperforms SEAL -static on feature-based MLC, can also effectively train large pre-trained language models like BERT (Devlin et al., 2019) on text-based MLC and SRL tasks and convolutional networks on image segmentation.
Dataset statistics are described in
Appendix B.
Feature network, loss-net and task-net.
In MLC and SRL experiments, to represent input x , we utilize the same feature network structures T E and T F for the energy and feedforward networks, which correspond to the loss-net and the task-net for SEAL framework.
The energy network utilizes
T E in the manner shown in equation 2.
Given a feature network T F that generates features in R h , the task-net F  ( x ) =
G T F ( x ) , where G 2 R L ⇥ h is a matrix consisting of learnable embeddings, one row of embedding for each label.
Speciﬁc feature network structures vary by tasks: multi-layer perceptron for feature-based MLC and BERT for text-based MLC and SRL.
In image segmentation experiments, we follow Gygli et al.
and use a Fully Convolutional Network (FCN) as the task-net; the loss-net uses the same convolutional layers in the FCN, followed by fully connected
[Table]
layers which replaces the tranposed convolutional layers in FCN to output scalar scores, instead of the energy formulation in Eq.(2)
[Table]
Training.
We use separate ADAM optimizers (Kingma & Ba, 2014) for the loss-net ( ⇥ ) and the task-net (  ) which optimize parameters in alternating fashion 3 .
We defer speciﬁc training details such as hyperparameters, gpu environment, and number of random seed runs to the Appendix C.1.
4.1 Multi-label classiﬁcation for feature-based datasets
Multi-label classiﬁcation (MLC) is a structured prediction task that does not hold an obvious latent structure in the multivariate space.
In addition to including Bibtex and Delicious that were used in previous works (Belanger & McCallum, 2016; Gygli et al., 2017; Tu & Gimpel, 2018), we carefully picked smaller (Genbase, Cal500) and larger datasets (Eurlex-ev) as well as datasets with known taxonomy (Spo_FUN, Expr_FUN) to examine SEAL in various scenarios.
The experiment results (test F1) across 7 feature-based dataset are presented in Table 1: feedforward network trained with cross-entropy (row 1), energy networks (row 2-4) described in §3 that is evaluated with GBI, task-nets trained with SEAL -static (row 5-7), and task-nets trained with SEAL -dynamic (row 8-11).
In general, SEAL -static and SEAL -dynamic outperform both cross-entropy (CE) and prediction energy networks with GBI on almost all datasets with average gains of SEAL -dynamic over CE ranging from +0.67 to +2.93 F1 points, excluding results on genbase dataset.
While the near-perfect performance of SEAL on genbase is interesting (see Appendix E.2 for detailed analysis), we exclude this excessive gain for computing the average performance when comparing different models throughout this paper.
Since SEAL uses single forward pass of the task-net for inference, it is as fast as the feedforward model and 3.63x faster (on average) at inference than the corresponding energy network trained using
GBI.
However, in training time, SEAL utilizes about twice the parameter to feedforward model as it loads both task-net and loss-net and takes about 5x more time to train compared to feedforward model.
In summary, SEAL is better performing than both energy network and feedforward model and as fast as feedforward model at inference time by utilizing more computations in training time.
The training time, inference time, and parameter size of different methods are reported in Appendix D.
Cross entropy vs SEAL .
While cross entropy cannot capture the dependencies between the labels, we ﬁnd that global energy of the loss-net visibly captures some of these dependencies.
We plot the gradients of global energy with respect to a pair of labels to analyse pairwise dependencies.
These plots can be found in §5 and the Appendix G.
Energy network vs. SEAL -static.
There is a high variability in the performance of the energy network with GBI.
For instance, while SPEN and DVN surpass CE models with large margins for half of the datasets, they underperform on the rest of the datasets with similar margins.
Further, even though DVN has the best performance on three datasets, its average performance (-genbase) is lower than CE.
Nonetheless, in SEAL -static, when the identical energy network is utilized as a loss-net, the performances increase in a stable manner (0.42-2.26 F1 on average) with faster (1.8x-7.4x) inference time.
This trend is most evident with the NCE ranking loss, where the energy network trained with it drastically fails to provide a good energy surface for predicting output with GBI, whereas it provides a useful loss surface for training task-net.
We believe there are two factors for this stable improvement of SEAL -static.
One is that injection of an energy surface as a loss is a softer injection of gradients compared to it being used as a surface for inference.
The second factor is a synergy between cross-entropy and the energy surface.
Even when certain regions of the energy surface are less accurate, the cross-entropy signal could compensate for the erroneous surface.
Thus, we claim that an energy network as a loss is a more stable way of utilizing it compared to GBI.
SEAL -static vs. SEAL -dynamic.
As seen in Table 1, with an average gain of 0.47 to 1.97 F1 points,
SEAL -dynamic is always on par or better than the corresponding SEAL -static.
This conﬁrms our hypothesis that it is helpful for the loss surface to consider the current state of the task-net as opposed to estimating the global energy landscape.
Lastly, SEAL -dynamic not only brings better performance but also simpliﬁes the engineering as both loss-net and task-net can be trained from scratch.
We observed that binary cross entropy in L F stabilizes the learning signal even in the early stage when loss-net might not be well-trained.
In summary, the SEAL framework outperforms CE loss and energy networks with GBI.
We observe the generality of the SEAL framework as all of the energy loss types beneﬁt working with SEAL , although more effective types of energy loss exist.
SEAL -dynamic brings extra gain in performance to
SEAL -static and among them, the NCE ranking loss shows the most notable improvement.
Lastly, as can be seen with the example of the NCE ranking loss, an energy network that serves as a poor prediction network does not necessarily correlate to a poor energy loss for SEAL .
4.2 Multi-label classiﬁcation for large text datasets
[Table]
To test the effectiveness of SEAL on large language models, we ﬁnetune pre-trained BERT as the feature network T E and T F on text MLC datasets.
Considering the computational expense of running hyper-parameter searches on BERT, we compare the best-performing SEAL from Table 1, SEAL -dynamic with NCE ranking, with baselines: BERT ﬁnetuned with cross-entropy and the state-of-the-art text-MLC model (LACO) (Zhang et al., 2021).
Following LACO, we experiment with the Arxiv Academic Paper Dataset (AAPD)
(Yang et al., 2018) which consists of computer science papers with annotations of 54 topics.
Table 2 shows that SEAL -dynamic with the NCE ranking loss is effective on pre-trained models that utilizes large text datasets as well.
SEAL -dynamic again improves task-net trained with cross entropy and performs on par with MLCspeciﬁc model architecture.
To efﬁciently work with large pre-trained models, we further studied utilizing BERT-adapter models on three additional text-based datasets (Appendix F) which also showed SEAL -dynamic-NCE model outperforming cross-entropy and InfNet.
4.3 Semantic role labeling
To examine SEAL on capturing label dependencies at a sequence level, we perform experiments on semantic role labeling (SRL)
(Palmer et al., 2010) using standard benchmark (CoNLL-12) (Pradhan et al., 2013).
SRL aims to annotate the predicate-argument structure on plain text extracting who did what to whom .
Because the argument roles in a sentence are dependent to each other, injection of structural priors has shown improvements in performance (Lee et al., 2019; Mehta et al., 2020; Li et al., 2020).
To apply SEAL , we utilize pre-trained BERT as the feature network T E and T F .
To capture the energy on variable-length output, we modify E global ⇥ in equation 2 to use self-attention layers.
We defer more detailed task description, data statistics, and energy architecture to appendix.
We compare SEAL -dynamic (NCE) on BERT-base against two models: one model trained with cross-entropy loss and another model trained with structured tuning framework (Li et al., 2020) where both models utilize BERT-base with CRF layers (Lafferty et al., 2001).
Structured tuning enforces the model to learn the SRL-speciﬁc constraint (unique core role) through differentiable constraint loss.
Unique core role violation refers to model predicting a type of core argument more than once when it should be unique.
To examine effectiveness of SEAL in capturing output dependencies, we report unique-core-role violation rate (UCR) in percentage along with F1 score.
While SEAL -dynamic does not enforce any speciﬁc constraints nor utilize CRF layers, it shows notable reduction in constraint violation and higher improvement in performance.
[Table] [Table]
4.4 Image segmentation
We evaluate SEAL on binary image segmentation using the Weizmann Horse Image dataset (Borenstein & Ullman, 2004).
This task requires a model to learn to predict detailed structures of horse shapes from a scarce 160 training images of a low 32 ⇥ 32 resolution.
Following Gygli et al.
, we implement Fully Convolutional Network (FCN)
(Long et al., 2015) as our feedforward baseline and
DVN (Gygli et al., 2017) as our energy network baseline.
Given a 3-channel L -pixel RGB image input x , FCN outputs the probability ˜y 2 [0 , 1] L that each pixel belongs to a horse object; a binary segmentation mask can be obtained through 1 ( ˜y  0 . 5) .
In contrast, DVN takes a pair of image x and soft segmentation mask ˜y to output their compatibility score and resort to gradient-based inference for prediction.
Following Gygli et al.
(2017), we train DVN by plugging the soft IoU score
[Equation]
, while
[Equation]
into the Eq.(7).
For training with the SEAL framework, we adopt the FCN and DVN architectures for the task-net and the loss-net respectively.
In our experiment, we examine SEAL -dynamic with regression which uses the same energy loss as DVN and SEAL -dynamic with the NCE loss because it performs the best in multiple other experiments.
More dataset and model details are in Appendix B; experiment details are in Appendix C.2.
Table 4 shows test performances of FCN trained by cross-entropy, DVN, and FCN trained with
SEAL -dynamic.
Similar to §4.1, we observe that the GBI procedure in DVN training and inference is slow and unstable (Appendix B.3 for more details).
While DVN does not outperform feedforward model learned with cross-entropy, the same energy network structure used as a loss-net in SEALdynamic (regression) improves the mean IoU of FCN by 1.43 IoU point and has less variance across random-seed runs.
Additionally, we ﬁnd that SEAL -dynamic with the NCE loss proposed for MLC does not improve FCN.
This is likely because many negative samples in NCE are non-realistic as we sample each pixel independently and do not take into account the prior that nearby pixels on image are likely to be in the same class.
We leave the exploration of other energy network architectures and sampling strategies in the image domain as future work.
Further analysis
In this section, we analyze (1) whether gradient of loss-net rightly captures multivariate dependencies, (2) the effect of ranking loss, and (3) we provide pointers to the data-speciﬁc analysis and discussion on the effect of samples in Appendix E.
Visualization of gradients In order to inspect the learning signal provided to the task-net by the loss-net, we plot a single component of the gradient from the global energy of the loss-net, i.e. E global ⇥ @
y k ( y i , y k ) .
In Figure 2, we inspect the global energy function of DVN trained on the expr_fun, a dataset that provides a taxonomy on the output space.
We ﬁrst plot the discussed gradient component when l k have positive association with l i , i.e., l i ! l k 4 (2b).
Contrasting this with the case when l k does not have a signiﬁcant association with l i (2c), we see that the gradient of the global energy rightly captures the presence and absence of association between the labels.
In contrast, binary cross-entropy is incapable of capturing label-to-label association (2a) which is obvious when we inspect its gradient as shown in Appendix A. See Appendix G for more examples and for detailed description of plotting procedure. [Figure]
[Figure]
Effect of applying a ranking loss directly on F
With the large improvement that SEAL -NCE achieves, we examine whether the gain is coming from our SEAL framework or simply from the power of a ranking loss itself that can also be applied to the task-net F  directly (i.e. s ( x , y ; ⇥ ) =
P i log P ( y i | x ;  t ) in equation 8).
We conduct this ablation study over relatively small datasets: genbase, cal500, and delicious.
While we observe a near +10 F1 point in genbase, we see a decrease in performance on cal500 (-1.0 F1) and delicious (-2.3 F1) compared to the plain cross-entropy model.
With these experiments, we ﬁrst reafﬁrm that genbase is an outlier dataset where capturing structure can signiﬁcantly increase the performance.
Second, we conclude that the ranking loss without an loss-net is not very effective and that the loss-net indeed plays a major role in the SEAL framework.
Conclusion
We propose SEAL , a framework that can adopt a structured energy network as a trainable loss function for training a feedforward network.
We show that SEAL is a general framework that can work with various loss functions, such as margin-based, regression-based, and NCE ranking losses, and with different architectures, such as MLPs, CNNs, and pretrained models (BERT).
Through extensive experiments on feature-based, text-based MLC, SRL, and image segmentation, we show that SEAL provides an effective way to utilize an energy network and a cross-entropy loss together: performing better, faster, and in a more stable manner when combined.
Lastly, through the ablation study, we examine how the loss-net captures the dependencies among output variables.
This research opens up doors for various future work, including but not limited to, exploration of different architectures of energy network; application of learned loss-nets in semi-supervised setups.
In the long horizon, we are interested in whether general pretrained scoring neural networks, such as BLEURT (Sellam et al., 2020) and BERT-score (Zhang et al., 2019), could serve as a loss function when it can provide backpropagatable gradients to the output space of a feedforward network.
Acknowledgments and Disclosure of Funding
This work was partially supported by New Faculty Startup Fund from Seoul National University, IBM
Research AI through the AI Horizons Network and the Chan Zuckerberg Initiative under the project
Scientiﬁc Knowledge Base Construction.
Additional support was provided by the NSF under Grant
Number IIS-2106391, and the Ofﬁce of Naval Research (ONR) via Contract No. N under Subaward No. 123875727 from the University of Southern California.
Checklist
For all authors.
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[No] Althgouh we do to some degree, for example, limitation of SEAL -dynamic-NCE in image segmentaiton.
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
We propose a learning framework which does not have direct negative soceital impacts.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[N/A] (b) Did you include complete proofs of all theoretical results?
[N/A] 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] The details are in Appendix.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] The details are in Appendix.
(c) Did you report error bars (e g., with respect to the random seed after running experiments multiple times)?
[No] (d) Did you include the total amount of compute and the type of resources used (e g., type of GPUs, internal cluster, or cloud provider)?
[Yes] In the appendix. 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[Yes] (c) Did you include any new assets either in the supplemental material or as a URL?
[No] (d) Did you discuss whether
and how consent was obtained from people whose data you’re using/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]
[-y k 
-]
[-y k 
-]
[-y k 
-]
[-y k 
-]
[-y k 
-]
[-y k 
-]
[-y k 
-]