Structured Energy Network as a Loss Function
Abstract
Belanger & McCallum (2016) and Gygli et al.
(2017) have shown that energy networks can capture arbitrary dependencies amongst the output variables in structured prediction; however, their reliance on gradient based inference (GBI) makes the inference slow and unstable.
In this work, we propose Structured Energy As
Loss (SEAL) to take advantage of the expressivity of energy networks without incurring the high inference cost.
This is a novel learning framework that uses an energy network as a trainable loss function (loss-net) to train a separate neural network (task-net), which is then used to perform inference through a forward pass.
We establish SEAL as a general framework wherein various learning strategies like margin-based, regression, and noise-contrastive could be employed to learn the parameters of loss-net.
Through extensive evaluation on multi-label classification, semantic role labeling, and image segmentation, we demonstrate that SEAL provides various useful design choices, is faster at inference than GBI, and leads to significant performance gains over the baselines.
Introduction
Structured prediction is an important machine learning problem in which, given an input x , the model needs to predict the output y ∈ Y {+
 ×+} Y 2 × · · · × Y L where there are dependencies across y 1 , . . .
, y L .
Common structured prediction tasks include multi-label classification (Belanger & McCallum, 2016;
Gygli et al., 2017), extracting parse trees or semantic role labels from texts (Palmer et al., 2010), and image segmentation (Müller, 2014).
Traditionally, these tasks are performed using models that explicitly represent the dependencies among output variables, such as directed and undirected graphical models, including Hidden Markov Models and Conditional Random Fields.
However, except in cases with very limited dependency patterns (Lafferty et al., 2001; Ghamrawi & McCallum, 2005), inference algorithms in these methods are difficult to engineer and expensive to run because structured output spaces are exponentially large.
In a multi-label classification task, for example, the size of the label space Y is 2 L given Y i
= { 0 , 1 } .
With their growing expressivity and capacity, it is common to use large feedforward neural networks to implicitly capture dependencies among output variables.
Given an input x , the elements of an output y are predicted in a conditionally independent manner in parallel, solely relying on the representation of input x to capture dependencies among output variables.
While the feedforward approach is computationally efficient, the widely used cross-entropy loss does not directly capture 36th Conference on Neural Information Processing Systems (NeurIPS 2022). dependencies in the output space and thus often lacks statistical efficiency in capturing multivariate correlations.
A recent line of methods (Belanger & McCallum, 2016; Gygli et al., 2017; Rooshenas et al., 2019) model the joint space of x and multivariate y explicitly with an energy function E Θ ( x , y )
(LeCun et al., 2006), which aims to learn arbitrary global dependencies in the output space.
They relax the combinatorial output space Y to ˜ Y =
[0 , 1] L and apply gradient-based inference (GBI) directly on ˜ y to search for optima on the relaxed energy surface as arg min ˜ y E
( x , ˜ y ) .
These energy models are therefore often called structured prediction energy networks (SPENs).
SPENs show noticeable gains in predictive performance over feedforward models and traditional graphical models such as (Chen et al., 2015; Schwing & Urtasun, 2015).
Due to the use of GBI, however, inference for SPENs remains relatively inefficient when compared to feedforward approaches (Tu & Gimpel, 2019).
Moreover, models using GBI are often difficult to train and sensitive to sampling methods and numerous hyperparameters, including step size, number of iterations, and initialization for GBI.
This raises a question: Can energy networks be used in a way that is as expressive as SPENs, as efficient at inference as feedforward approaches, and also easy to train?
To this end, we propose using structured energy networks as parameterized loss functions for feedforward networks, instead of as prediction networks.
In this paper, we introduce the S tructured E nergy
A s L oss ( SEAL ) framework which uses trainable
SPENs (loss-net 1 ) as loss functions to guide the training of other feedforward networks (task-net).
The key idea is to provide the feedforward network with access to rich dependencies in the output space through a learned loss-net (§2.1 SEAL -static).
We further propose to learn the loss-net in a dynamic fashion (§2.2 SEAL -dynamic) by adjusting the loss-net to be tailored to the most up-to-date outputs of the feedforward network; and then propose an NCE ranking loss uniquely suited for the SEAL -dynamic framework.
Using feature-based multi-label classification as a case study, we extensively analyze the impact of various loss functions for updating the loss-net (§2.3).
We further demonstrate the faster inference and higher performance of SEAL , compared to other baseline methods, over feature-, text-based multi-label classification, semantic role labeling, and image segmentation (§4).
Additionally, we show that energy models capture the dependencies on the output space through analyzing gradient signals in an ablation study (Appendix G).
Structured Energy Network as Loss ( SEAL )
Notation.
X denotes the input space, Y = { 0 , 1 } L the label space, and ˜ Y =
[0 , 1] L the continuous relaxation of Y .
Let D =
( x ( j ) , y ( j ) ) ∈
X × Y} Nj =1 denote the dataset, and y i ∈ { 0 , 1 } the i -th dimension in the label vector y for i ∈ { 1 , . . .
, L } , and ˜ y i ∈ [0 , 1] its continuous relaxation.
The feedforward network F Φ : X → ˜ Y is a parametrized function that maps an input x ∈ X to an output
F Φ ( x ) ∈ ˜ Y , where F Φ ( x ) i is the predicted probability of label i occurring, given the input x .
The structured energy network E Θ : X × ˜ Y
→ R is a parameterized function that maps an input x and a continuously relaxed output ˜ y to real-valued scalar energy.
For clarity, we refer to the feedforward network as task-net , and the structured energy network as loss-net .
As depicted in Figure 1, the proposed SEAL framework consists of two parametric components.
First, the loss-net E Θ is trained by an energy loss L E to capture an energy surface that reflects the latent dependencies among output variables.
Low energy indicates a high quality of a label prediction given an input.
Second, a task-net F Φ is trained using a loss L F that involves minimizing the energy E Θ estimated by the loss-net.
After training finishes, the loss-net is no longer utilized; inference time predictions are obtained solely from the task-net as ˆ y i =
1 ( F Φ ( x ) i ≥ 0 . 5) .
Unlike SPEN which requires iterative gradient-based inference, a task-net trained with the SEAL framework has the same computational efficiency at inference-time as standard feedforward networks, while enjoying a loss that captures the label dependencies through the loss-net (Appendix D).
To guide the task-net training with the loss-net E Θ , we propose the task-net training loss L F (Φ; Θ) to be a weighted sum of estimated energy and a binary-cross entropy (BCE) loss as
[Equation]
where ( x , y ) is a training instance and BCE ( y j , F Φ ( x ) j )
= −
[ y j log F Φ ( x ) j + (1 − y j ) log (1 − F Φ ( x ) j )] .
By incorporating the energy term in L F , we enable the loss function to reflect the latent variable dependencies in the output space, whereas BCE treats L labels in a conditionally independent fashion.
We present the formulation of E Θ used for multi-label classification to analyze the effect of introducing E Θ into L F without loss of generality.
We use energy network E Θ ( x, y ) that has the same structure with Belanger & McCallum (2016) and Gygli et al.
(2017) for a fair comparison with them.
The energy E Θ is the sum of local energy E local Θ and global energy E global Θ which are defined as:
[Equation]
where b i , v , M , T E contain learnable parameters, T E is a feature network to represent an input x in the loss-net, and γ ( z )
= log(1 + e z ) is the softplus activation function.
We observe from the derivative of the global energy
[Equation]
(3) that all dimensions of y decides the gradient signal for each label i .
The global energy term E global Θ in L F , specifically, allows the gradient of the task-net loss to capture the dependency of y i on all L dimensions of y .
In contrast, the gradient of BCE for each label i only depends on the i -th dimension of y .
More details are in Appendix A.
2.1 SEAL -static
Since the task-net loss L F depends on the quality of the loss-net, it is critical to find the loss-net parameters Θ that produce the best loss surface for training the task-net Φ .
In order to achieve this, we first estimate the loss-net parameters (cid:98) Θ over training data and then optimize the task-net parameters Φ by plugging in the fixed loss-net parameters (cid:98) Θ into L F .
Formally, given training instances ( x , y ) ∈ D , the task-net parameters Φ are trained in the following manner:
[Equation]
Since the loss function L F is static after pretraining (cid:98) Θ , we call this approach SEAL -static.
We discuss various types of L E losses in §2.3.
2.2 SEAL -dynamic
Ideally, we would like to learn a perfect energy surface by estimating (cid:98) Θ that is static.
However, estimating an accurate energy surface over a high-dimensional, continuous joint space X × ˜ Y is a challenging task.
Instead, we hypothesize that using loss-net to dynamically estimate the energy surface that is focused around the current input and task-net output pair ( x , F Φ ( x )) could be more beneficial in teaching task-net.
To this end, we propose SEAL -dynamic which dynamically adapts loss-net so that the region of interest for training task-net is well represented.
Given a batch of training data B t , a complete training step of SEAL -dynamic is provided as
[Equation]
where the loss-net parameters Θ t and the task-net parameters Φ t are optimized in alternating fashion.
Note that, from Eq.(4) to Eq.(5), arbitrary label samples ˜ y were replaced with F Φ t − 1 ( x ) from the tasknet during the training of Θ t parameters.
Thus the energy surface learned by the loss-net is adaptive to the task-net.
On the contrary, in SEAL -static, the training of the static loss-net is independent of the task-net.
Further, the update step for the task-net parameters Φ t relies on dynamic loss-net parameters Θ t rather than static (cid:98) Θ .
In §4, we show the effectiveness of the proposed SEAL framework compared to standard feedforward networks and SPENs, and further that SEAL -dynamic often improves over
SEAL -static.
[Figure]
Require: ( x , y ) :
Training Instance
Require: F Φ : Feedforward Network Require: sampling : True/False flag
Require: optimizer Θ , optimizer Φ Require:
T : No. of steps Θ 0 , Φ 0 ← Random initialization, t ← 0 while t < T do if sampling then
S ← { y ( i ) , i = 1 , . . .
, K | y
( i ) ∼ F Φ t ( x ) } else S ← { F Φ t ( x ) } ▷ singleton set t ← t +
Update Θ as Eqn.
5 using ˜ y ∈ S and optimizer Θ Update Φ as Eqn. 6 using optimizer Φ
2.3 Energy losses L E
We now discuss various energy loss functions L E with different characteristics such as margin-based, regression-based, and contrastive-sampling-based, that can be plugged into the SEAL framework to train the loss-net.
For brevity, we use L E (Θ) and L E ( x , y , ˜ y ; Θ) interchangeably.
Margin-based (L E − margin ).
SPEN (Belanger & McCallum, 2016) learns structured energy network with the SSVM loss (Taskar et al., 2004;
Tsochantaridis et al., 2004)
so that energy function learns to predict sufficient energy difference, larger than the margin ∆ (˜ y , y ) , between an arbitrary output ˜ y and a true output y .
To examine the effect of the margin-based loss in SEAL , we follow SPEN and utilize the SSVM loss as
[Equation]
SPEN resorts to gradient ascent on ˜ y to solve the optimization problem arg max ˜ y ∆ (˜ y , y ) −
E Θ ( x , ˜ y ) .
InfNet (Tu & Gimpel, 2018) proposes an adversarial training of the task-net Φ together with the energy network Θ .
We identify InfNet as a special case of SEAL -dynamic that uses L E − margin .
We defer a detailed description of it to the related work (§3).
Regression-based (L E − regression ).
Deep Value Network (DVN) (Gygli et al., 2017) attempts to learn an energy network which directly outputs a score that is similar to the metric s (˜ y , y ) of interest, such as F1 score, that compares arbitrary ˜
y with true output y .
Following DVN, we let the true score s ( · ) 2 and σ ( − E ( · )) be between [0 , 1] and express the regression-based loss as cross entropy between s and σ ( − E ) :
[Equation]
where σ ( · ) is a sigmoid function.
DVN utilizes two samples to train the model: the model prediction that minimizes the energy and an adversarial sample ˜ y that maximizes the loss L E − regression .
We this approach for training SEAL -static with L E − regression .
To train SEAL -dynamic with L E − regression , we simply use the output of task-net, F Φ t − 1 ( x ) , as described in Eq.(5).
To better capture the energy surface in SEAL -dynamic, we define the regression-s loss L E − regression-s = (cid:80) ˜
y ∈ S L E − regression ( x , y ∗ , ˜ y ; Θ) where the sample set S is taken by continuously perturbing task-net output F Φ t − 1 ( x ) with Gaussian noise (experimental analysis on the ‘effect of samples’ is in §5).
Noise-contrastive ranking (L E − NCE ).
Lastly, we propose to use noise contrastive estimation (NCE) ranking loss (Ma & Collins, 2018) uniquely for SEAL -dynamic.
We are interested in capturing appropriate energy surface in high-probability output spaces of task-net.
To reflect such space accurately, we propose to train the loss-net E Θ to contrast the true output y with a group of K negative samples drawn from the task-net prediction F Φ ( x ) motivated by NCE.
Before we discuss L E − NCE for SEAL -dynamic in more detail, we first review the original form of
NCE ranking loss from Ma & Collins (2018).
For K samples y ( k ) ∼ P N , k = 1 , . . .
, K, drawn from a noise distribution P N , rewriting ground truth y as y (0) without loss of generality, the NCE ranking loss is defined as
[Equation]
(8) where s ( x , y ; Θ) = − E Θ ( x , y ) − log P N ( y ) .
Given the noise samples y ( k ) , minimizing Eq.(8) minimizes the energy of the ground truth sample y (0) while increasing the energy of the noise samples.
In SEAL -dynamic, we propose to use (cid:81) i P ( y i | x ; Φ t ) which is dynamically changing in place of the static P N ( y ) in Eq.(8).
The NCE ranking loss is thus
[Equation]
The novelty of this proposal is that we view the output of a task-net as a dynamic noise distribution, and that the loss-net in turn teaches the task-net that generates the contrastive noise samples (Eq.(5)).
The NCE ranking loss L E − NCE brings two benefits in SEAL -dynamic.
First, L E − NCE captures the output distribution of the task-net rather than just a single point.
Second, L E − NCE allows estimation of a more fine-grained loss surface as the training steps proceed.
In our SEAL -dynamic learning procedure, we hypothesize that (cid:81) i P
( y i | x ; Φ t ) becomes closer to the true data distribution as training steps proceed.
Contrasting these noise samples to the ground truth could lead the loss surface estimated by loss-net to be more fine-grained.
This is in line with the known fact that NCE methods work the best when the noise distribution is close to the data distribution but not the same (Gutmann & Hirayama, 2012).
To thoroughly experiment with the proposed NCE ranking loss L E − NCE , we also train a static energy network with L E − NCE in order to utilize it as a prediction network with GBI and as a loss-net in
SEAL -static.
We set P N ( y ) =
(cid:81) i P ( y i | x ; Ψ) in Eq.(8), where Ψ is a fixed pretrained feedforward network.
When we train an energy network E Θ with L E − NCE , [Equation] − [Equation]
x , y [Equation] [Equation] y
[Equation]
− E
[Equation]
x
[Equation]
y )) becomes an unbiased estimator of the true distribution P ( y | x ) (Ma & Collins, 2018).
Given a perfect energy
E Θ ( x , y ) , minimizing it with respect to y in SEAL -static and for prediction with GBI is equivalent to minimizing the estimate of − log P ( y | x ) .
Related Work
Learning with dynamic loss.
Wu et al.
and
Huang et al.
(2019) attempt to learn a dynamic loss for multi-class classification tasks.
Both approaches try to learn a parametrized loss function that can directly increase the model’s metric score m , such as the BLEU or 0-1 accuracy, on the validation set.
Since m is usually non-differentiable, Huang et al.
(2019) view this problem as a reinforcement learning problem and attempt to learn a loss function that maximizes the expected reward.
On the other hand, Wu et al.
(2018) relax the m to be differentiable and try to learn a loss parameter ( Θ ) that can improve the relaxed score on feed-forward ( Φ ) output.
There are three major differences between SEAL and the previous work.
While both Wu et al.
Huang et al.
(2019) use the parametric form σ ( y T Θ log F Φ ( x )) to define the loss function, SEAL utilizes a bigger class of functions expressed using general energy network E Θ .
Moreover, SEAL does not require manually designed state vectors and works directly with the task-net outputs.
Lastly, while the SEAL framework is designed for structured prediction, Wu et al.
and Huang et al. (2019) are designed for multi-class classification and it is non-trivial to extend these methods for structured prediction.
Structured Prediction Energy Network (SPEN).
Structured prediction energy networks (Belanger & McCallum, 2016) and its variants (Gygli et al., 2017; Rooshenas et al., 2019) (together referred to as SPENs) learn an energy network E Θ : X × ˜ Y → R and predict the output using gradient-based inference (GBI) with the objective
y = arg
min y E ( x , y ) .
The optimization objective for training the
SPENs relies on finding adversarial or margin-violating points in the label space using gradient based inference (GBI).
This learning process in SPENs is highly sensitive with respect to hyper-parameters such as the learning rate and initial points for the GBI, resulting in high variance in the model performance.
Nonetheless, SPENs are strong baselines for structured prediction tasks and are used as motivations for the design of SEAL .
They are further discussed in §2.3.
Jointly learning Θ and Φ .
Inference Network (InfNet) (Tu & Gimpel, 2018) is the first work that interactively learns a structured energy network and a feedforward network.
It aims to replace the expensive GBI in SPEN with a feedforward network’s forward pass.
During its training with the SSVM loss (Eq.(7)), SPEN maximizes ∆ (˜ y , y ) − E Θ ( x , ˜ y ) with respect to ˜ y by GBI.
InfNet, on the contrary, trains a feedforward network F Φ to generate ˜ y
= F Φ ( x ) that maximizes the term.
This results in an adversarial framework (Goodfellow et al., 2014), where a feedforward network maximizes the margin-based loss while an energy network minimizes it.
The approach, however, suffers from train-test-time objective mismatch (Tu et al., 2020).
While the feedforward network serves as an adversarial sampler during training, the end goal is to learn a feedforward network that predicts outputs with low energy at test time.
Tu et al.
(2020) resolves this by learning two separate feed-forward networks: one adversarial sampler for training and the other for performing inference.
Our proposed SEAL generalizes, and differs from, InfNet by providing a framework that can employ completely independent loss functions for training loss-net and task-net.
SEAL updates task-net in the direction of reducing energy defined by loss-net; loss-net is only used as a tool for evaluation of loss, and how loss-net is trained does not matter as long as it estimates a good loss surface for task-net.
The SEAL framework is not interested in producing the similar test-time behaviour of SPEN as InfNet did.
Despite the difference in the end goal, mechanically speaking, InfNet is a special case of SEAL -dynamic (i.e. SEAL -dynamic with L E − margin ) if the margin is removed from the SSVM loss for the feedforward model training.
In fact, Tu et al.
(2020) reports that this special case leads to more efficient and stable learning of the feedforward network in their ablation study.
We use this special case of InfNet to represent SEAL with L E − margin in the main experiment.
Experiments
Tasks and Datasets.
We demonstrate the effectiveness of SEAL on widely deployed structured prediction tasks: feature and text-based multi-label classification (MLC), semantic role labeling (SRL), and image segmentation.
Feature-based MLC is a standard evaluation task in previous works on SPENs (Belanger & McCallum, 2016; Gygli et al., 2017; Tu & Gimpel, 2018).
We utilizefeature-based MLC datasets which cover various label sizes (ranging from 27 to 4k), training sizes, and input characteristics (binary, real) as shown in Table 5 to extensively compare baselines and proposed SEAL approaches: cross-entropy (CE), energy network, SEAL -static, and SEAL -dynamic.
Then we verify that SEAL -dynamic, which outperforms SEAL -static on feature-based MLC, can also effectively train large pre-trained language models like BERT (Devlin et al., 2019) on text-based MLC and SRL tasks and convolutional networks on image segmentation.
Dataset statistics are detailed in Appendix B.
Feature network, loss-net and task-net.
In MLC and SRL experiments, to represent input x , we utilize the same feature network structures T E and T F for the energy and feedforward networks, which correspond to the loss-net and the task-net for SEAL framework. The energy network utilizes T E in the manner shown in equation 2.
Given a feature network T F that generates features in R h , [Table] the task-net F Φ ( x ) =
G T F ( x ) , where G ∈ R L × h is a matrix consisting of learnable embeddings, one row of embedding for each label.
Specific feature network structures vary by tasks: multi-layer perceptron for feature-based MLC and BERT for text-based MLC and SRL.
In image segmentation experiments, we follow Gygli et al.
and use a Fully Convolutional Network (FCN) as the task-net; the loss-net uses the same convolutional layers in the FCN, followed by fully connected layers which replaces the tranposed convolutional layers in FCN to output scalar scores, instead of the energy formulation in Eq.(2).
Training.
We use separate ADAM optimizers (Kingma & Ba, 2014) for the loss-net ( Θ ) and the task-net ( Φ ) which optimize parameters in alternating fashion 3 .
We defer specific training details such as hyperparameters and gpu environment to the Appendix C.1.
4.1 Multi-label classification for feature-based datasets
Multi-label classification (MLC) is a structured prediction task that does not hold an obvious latent structure in the multivariate space.
In addition to including Bibtex and Delicious that were used in previous works (Belanger & McCallum, 2016; Gygli et al., 2017; Tu & Gimpel, 2018), we carefully picked smaller (Genbase, Cal500) and larger datasets (Eurlex-ev) as well as datasets with known taxonomy (Spo_FUN, Expr_FUN) to examine SEAL in various scenarios.
The experiment results (test F1) across 7 feature-based dataset are presented in Table 1: feedforward network trained with cross-entropy (row 1), energy networks (row 2-4) described in §3 that is evaluated with GBI, task-nets trained with SEAL -static (row 5-7), and task-nets trained with SEAL -dynamic (row 8-11).
In general, SEAL -static and SEAL -dynamic outperform both cross-entropy (CE) and prediction energy networks with GBI on almost all datasets with average gains of SEAL -dynamic over CE ranging from +0.67 to +2.93 F1 points, excluding results on genbase dataset.
While the near-perfect performance of SEAL on genbase is interesting (see Appendix E.2 for details), we exclude this excessive gain for computing the average performance when comparing different models throughout this paper.
Since SEAL uses single forward pass of the task-net for inference, it is as fast as the feedforward model and 3.63x faster (on average) at inference than the corresponding energy network trained using GBI.
However, in training time, SEAL utilizes about twice the parameter to feedforward model as it loads both task-net and loss-net and takes about 5x more time to train compared to feedforward model.
In summary, SEAL is better performing than both energy network and feedforward model and as fast as feedforward model at inference time by utilizing more computations in training time.
The training time, inference time, and parameter size of different methods are reported in Appendix D.
Cross entropy vs SEAL .
While cross entropy cannot capture the dependencies between the labels, we find that global energy of the loss-net visibly captures some of these dependencies.
We plot the gradients of global energy with respect to a pair of labels to analyse pairwise dependencies.
These plot can be found in the Appendix G.
Energy network vs. SEAL -static.
There is a high variability in the performance of the energy network with GBI.
For instance, while SPEN and DVN surpass CE models with large margins for half of the datasets, they underperform on the rest of the datasets with similar margins.
Further, even though DVN has the best performance on three datasets, its average performance (-genbase) is lower than CE.
Nonetheless, in SEAL -static, when the identical energy network is utilized as a loss-net, the performances increase in a stable manner (0.42-2.26 F1 on average) with faster (1.8x-7.4x) inference time.
This trend is most evident with the NCE ranking loss, where the energy network trained with it drastically fails to provide a good energy surface for predicting output with GBI, whereas it provides a useful loss surface for training task-net.
We believe there are two factors for this stable improvement of SEAL -static.
One is that injection of an energy surface as a loss is a softer injection of gradients compared to it being used as a surface for inference.
The second factor is a synergy between cross-entropy and the energy surface.
Even when certain regions of the energy surface are less accurate, the cross-entropy signal could compensate for the erroneous surface.
Thus, we claim that an energy network as a loss is a more stable way of utilizing it compared to GBI.
SEAL -static vs. SEAL -dynamic.
As seen in Table 1, with an average gain of 0.47 to 1.97 F1 points,
SEAL -dynamic is always on par or better than the corresponding SEAL -static.
This confirms our hypothesis that it is helpful for the loss surface to consider the current state of the task-net as opposed to estimating the global energy landscape.
Lastly, SEAL -dynamic not only brings better performance but also simplifies the engineering as both loss-net and task-net can be trained from scratch.
In summary, the SEAL framework outperforms CE loss and energy networks with GBI.
We observe the generality of the SEAL framework as all of the energy loss types benefit working with SEAL , although more effective types of energy loss exist.
SEAL -dynamic brings extra gain in performance to
SEAL -static and among them, the NCE ranking loss shows the most notable improvement.
Lastly, as can be seen with the example of the NCE ranking loss, an energy network that serves as a poor prediction network does not necessarily correlate to a poor energy loss for SEAL .
4.2 Multi-label classification for large text datasets
[Table]
To test the effectiveness of SEAL on large language models, we finetune pre-trained BERT as the feature network T E and T F on text MLC datasets.
Considering the computational expense of running hyper-parameter searches on BERT, we compare the best-performing SEAL from Table 1, SEAL -dynamic with NCE ranking, with baselines: BERT finetuned with cross-entropy and the state-of-the-art text-MLC model (LACO) (Zhang et al., 2021).
Following LACO, we experiment with the Arxiv Academic Paper Dataset (AAPD)
(Yang et al., 2018) which consists of computer science papers with annotations of 54 topics.
Table 2 shows that SEAL -dynamic with the
NCE ranking loss is effective on pre-trained models that utilizes large text datasets as well, surpassing
MLC-specific model performance.
To efficiently work with large pre-trained models, we further studied utilizing BERT-adapter models on three more text-based datsets (Appendix F) which also showed SEAL -dynamic-NCE model outperforming cross-entropy and InfNet.
4.3 Semantic role labeling
To examine SEAL on capturing label dependencies at a sequence level, we perform experiments on semantic role labeling (SRL)
(Palmer et al., 2010) using standard benchmark (CoNLL-12) (Pradhan et al., 2013).
SRL aims to annotate the predicate-argument structure on plain text extracting who did what to whom .
Because the argument roles in a sentence are dependent to each other, injection of structural priors has shown improvements in performance (Lee et al., 2019; Mehta et al., 2020; Li et al., 2020).
To apply SEAL , we utilize pre-trained BERT as the feature network T E and T F .
To capture the energy on variable-length output, we modify E global Θ in equation 2 to use self-attention layers.
We defer more detailed task description, data statistics, and energy architecture to appendix.
We compare SEAL -dynamic (NCE) on BERT-base against two models: one model trained with cross-entropy loss and another model trained with structured tuning framework (Li et al., 2020) where both models utilize BERT-base with CRF layers (Lafferty et al., 2001).
Structured tuning enforces the model to learn the SRL-specific constraint (unique core role) through differentiable constraint loss.
Unique core role violation refers to model predicting a type of core argument more than once when it should be unique.
To examine effectiveness of SEAL in capturing output dependencies, we report unique-core-role violation rate (UCR) in percentage along with F1 score.
While SEAL -dynamic does not enforce any specific constraints nor utilize CRF layers, it shows notable reduction in constraint violation and higher improvement in performance.
[Table] [Table]
4.4 Image segmentation
We evaluate SEAL on binary image segmentation using the Weizmann Horse Image dataset (Borenstein & Ullman, 2004).
This task requires a model to learn to predict detailed structures of horse shapes from a scarce 160 training images of a low 32 × 32 resolution.
Following Gygli et al. , we implement Fully Convolutional Network (FCN)
(Long et al., 2015) as our feedforward baseline and DVN (Gygli et al., 2017) as our energy network baseline.
Given a 3-channel L -pixel RGB image input x , FCN outputs the probability ˜y ∈ [0 , 1] L that each pixel belongs to a horse object; a binary segmentation mask can be obtained through 1 ( ˜y ≥ 0 . 5) .
In contrast, DVN takes a pair of image x and soft segmentation mask ˜y to output their compatibility score and resort to gradient-based inference for prediction.
Following Gygli et al.
(2017), we train DVN by plugging the soft IoU score
[Equation]
, while
[-Mi =
-][Equation]
into the Eq.(8).
For training with the SEAL framework, we adopt the FCN and DVN architectures for the task-net and the loss-net respectively.
In our experiment, we examine SEAL -dynamic with regression which uses the same energy loss as DVN and SEAL -dynamic with the NCE loss because it performs the best in multiple other experiments.
More dataset and model details are in Appendix B; experiment details are in Appendix C.2.
Table 4 shows test performances of FCN trained by cross-entropy, DVN, and FCN trained with
SEAL -dynamic.
Similar to §4.1, we observe that the GBI procedure in DVN training and inference is slow and unstable (Appendix B.3 for more details).
While DVN does not outperform feedforward model learned with cross-entropy, the same energy network used as a loss-net in SEAL -dynamic (regression) improves the mean IoU of FCN by 1.43 IoU point and has less variance across random runs.
Additionally, we find that SEAL -dynamic with the NCE loss proposed for MLC does not improve FCN.
This is likely because many negative samples in NCE are non-realistic as we sample each pixel independently and do not take into account the prior on image that nearby pixels are likely to be in the same class.
We leave the exploration of other energy network architectures and sampling strategies in the image domain as future work.  Further analysis
In this section, we analyze (1) whether gradient of loss-net rightly captures multivariate dependencies, (2) the effect of ranking loss, and (3) we provide pointers to the data-specific analysis and discussion on the effect of samples in Appendix E.
Visualization of gradients In order to inspect the learning signal provided to the task-net by the loss-net, we plot a single component of the gradient from the global energy of the loss-net, i.e.
[Figure]
k ( y i , y k ) .
In Figure 2, we inspect the global energy function of DVN trained on the expr_fun, a dataset that provides a taxonomy on the output space. We first plot the discussed gradient component when l k have positive association with l i , i.e., l i → l k 4 (2b). Contrasting this with the case when l k does not have a significant association with l i (2c), we see that the gradient of the global energy rightly captures the presence and absence of association between the labels. In contrast, binary cross-entropy is incapable of capturing label-to-label association (2a) which is obvious when we inspect its gradient as shown in Appendix A. See Appendix G for more examples and for detailed description of plotting procedure.
[Figure]
of global energy
is also plotted (c) for the case when there is no significant association between the two labels. Effect of applying a ranking loss directly on F Φ With the large improvement that SEAL -NCE achieves, we examine whether the gain is coming from our SEAL framework or simply from the power of a ranking loss itself that can also be applied to the task-net F Φ directly (i.e. s ( x , y ; Θ) = (cid:80) i log P ( y i | x ; Φ t ) in equation 8).
We conduct this ablation study over relatively small datasets: genbase, cal500, and delicious. While we observe a near +10 F1 point in genbase, we see a decrease in performance on cal500 (-1.0 F1) and delicious (-2.3 F1) compared to the plain cross-entropy model. With these experiments, we first reaffirm that genbase is an outlier dataset where capturing structure can significantly increase the performance. Second, we conclude that the ranking loss without an loss-net is not very effective and that the loss-net indeed plays a major role in the SEAL framework.
We propose SEAL , a framework that can adopt a structured energy network as a trainable loss function for training a feedforward network. We show that SEAL is a general framework that can work with various loss functions, such as margin-based, regression-based, and NCE ranking losses, and with different architectures, such as MLPs, CNNs, and pretrained models (BERT). Through extensive experiments on feature-based, text-based MLC, SRL, and image segmentation, we show that SEAL provides an effective way to utilize an energy network and a cross-entropy loss together: performing better, faster, and in a more stable manner when combined.
Lastly, through the ablation study, we examine how the loss-net captures the dependencies among output variables.
This research opens up doors for various future work, including but not limited to, exploration of different architectures of energy network; application of learned loss-nets in semi-supervised setups.
In the long horizon, we are interested in whether general pretrained scoring neural networks, such as BLEURT (Sellam et al., 2020) and BERT-score (Zhang et al., 2019), could serve as a loss function when it can provide backpropagatable gradients to the output space of a feedforward network.
Checklist
For all authors.
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[No] Althgouh we do to some degree, for example, limitation of SEAL -dynamic-NCE in image segmentaiton.
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
We propose a learning framework which does not have direct negative soceital impacts.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[N/A] (b) Did you include complete proofs of all theoretical results?
[N/A] 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] The details are in Appendix.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] The details are in Appendix.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[No] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] In the appendix. 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] (b) Did you mention the license of the assets?
[Yes] (c) Did you include any new assets either in the supplemental material or as a URL?
[No] (d) Did you discuss whether
and how consent was obtained from people whose data you’re using/curating? [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]
Appendix A Gradient to the label representations
As described in section 4, the energy component of loss L F , for the task-net, is the sum of following two parts:
[Equation]
and
[Equation]
The gradient of E global w.r.t y is given by the following vector of length L ∂E [Equation] While the gradient of cross entropy BCE ( y, y ∗
[Equation]
.r.t.
y is given by the following vector of length
L
[Equation]
where ⊗ is element-wise product.
Note the dependency of the gradient for a particular component y i of y on other component in the expressions above.
The gradient of cross entropy ( ∂l∂y i ) only depends on its label dimension i : y i and y ∗ i .
In contrast, in the case of the energy function as loss (SEAL), the gradient ∂ L F ∂y i depends on all L dimensions of y due to the product term My in ∂E global ∂y i .
B Task Details
This Section provides task-specific information including task definitions, descriptions for datasets and models, and additional observations in our experiments.
Please refer to the beginning of Section 4 for a preliminary description.
B.1 Multi-label classification
Task Definition:
Given input x , multi-label classification maps the input to the labels y in the output space, Y = { 0 , 1 } L .
Dataset: In our experiments, we use 11 multi-label classification (MLC) datasets covering various label spaces, label sizes (ranging from 27 to 4k), training sizes, and input characteristics (binary, real, text) as shown in Table 5.
The first seven rows are binary- or real-valued feature-based datasets.
In addition to including Bibtex and Delicious that were used in baseline approaches (Belanger &
McCallum, 2016; Gygli et al., 2017; Tu & Gimpel, 2018), we carefully picked smaller (Genbase,
Cal500) and larger datasets (Eurlex-ev) as well as datasets with known taxonomy (Spo_FUN,
Expr_FUN) to examine SEAL in various scenarios.
For text-based datasets, we select RCV Lewis et al.
(2004) and BGC (Aly et al., 2019) which have label dependencies (DAG and forest respectively),
AAPD to compare with state-of-the-art text-based MLC model (Zhang et al., 2021), and take a subset of the NYT dataset Sandhaus (2008) 5 to experiment with larger label dimensions and training data.
For text-based MLC datasets, we examine whether SEAL can be helpful in the context of large pre-trained models as well.
The links to download the multi-label datasets used in this work are given in Table 6.
Feature network: We use standard feed-forward neural network and pretrained BERT-base as the feature network, T E , for the feature-based and text-based MLC datasets respectively.
[Table]
[Table]
Energy network:
Given the input x and the output y ∈ { 0 , 1 } L , the energy terms are defined as:
[Equation]
where b i , v , M , T E contain learnable parameters.
T E is a feature network to represent an input x in the loss-net, and γ ( z )
=
log(1 + e z ) is the softplus activation function.
B.2 Semantic Role Labeling
Task Definition:
Given a sentence and a predicate pair ( W, v ) , the goal of SRL is to identify the spans of arguments for v in W and assign correct role labels to the identified arguments.
Dataset: We train and evaluate the performance of SEAL and baselines on the standard benchmark (CoNLL-12) (Pradhan et al., 2013) for SRL.
Feature network: We use pretrained BERT-base for the feature network, T E and T F .
Energy network: For the sequence tagging experiments, we take the input, x , a sequence of N tokens and the output, y , a sequence of N labels.
y n denotes the label at the n th position in the sequence and has L dimensions, the number of labels.
Consequently, we define the energy terms as,
[Equation]
where E S applies the self-attention (Vaswani, 2017) over y to capture the long-distance dependencies among labels and gets a L by N dimension output.
We then do max-pooling over the L dimension before taking the sum across the sequence length to get the global energy, E global Θ .
B.3 Image segmentation
Task Definition:
Given a 3-channel L -pixel RGB image x , binary image segmentation requires models to classify each pixel to either foreground or background, thus outputing a binary segmentation mask y ∈ { 0 , 1 } L .
Datasets: The Weizmann Horse Image dataset Borenstein & Ullman (2004) consists of 328 leftoriented horse images annotated with segmentation masks.
Following CHOPPS (Li et al., 2013) and
DVN (Gygli et al., 2017) 6 , we adopt a split of 160 training, 40 validation, and 128 testing examples at the resolution of 32 × 32 pixels.
We train models using the training split, select hyperparameters according to mean image IoU on the validation split, and report mean image IoU on the test split.
Feedforward (FCN): We implement Fully Convolutinal Network described by Gygli et al.
(2017) as our feedforward baseline and task-net in SEAL .
It has 3 convolutional layers, 2 transposed convolutional layers, and 2 skip connections.
The second transposed convolutional layer outputs two channels of logits.
A softmax layer is then applied and we take the normalized horse channel as the probability that each pixel belongs to the horse object.
The channel can be binarized to obtain the segmentation mask.
Energy network (DVN): For segmentation, we do not have separate notions of local and global energies (Eq.(2)).
We implement Deep Value Network described by Gygli et al.
(2017) as our energy network baseline and loss-net in SEAL , referring to this public repository.
In general we find gradient based inference during DVN training and inference unstable and sensitive to configurations like sampling methods and initialization for the segmentation mask.
For example, it achieves < 50 test
IoU if we do not supply ground truth samples during its training.
Nevertheless, using the same energy network architecture as loss for FCN under the SEAL -dynamic learning framework improves the performance of FCN.
We leave sampling strategies and loss-net architectures for further exploration.
Qualitative Results: We observe that segmentation masks predicted by FCN trained with SEALdynamic (regression-s) are better at details (e.g. legs) than those by cross-entropy (Figure 3).
C Experiment Details C.1 Training setup
Our experiments are performed on a mix of TitanX, 1080Ti, 2080ti, M40, and RTX8000 GPUs.
We use separate ADAM optimizers (Kingma & Ba, 2014) for the loss-net ( Θ ) and the task-net ( Φ ) which optimize parameters in alternating fashion 7 .
For each minimization loop, we take n E , n F gradient steps respectively for estimating Θ t and Φ t .
The best hyper-parameters for each model are found using Bayesian search 8 .
We also report the training time, inference time, and parameter size of different MLC methods in Appendix D.
C.2 Hyperparameters
The following tables tabulate the hyper-parameters for SEAL on each dataset.
We fix the λ 2 = 1 in equation 1 to reduce degree of freedom in hyperparameter search.
In MLC and SRL experiments, we fix the task-net number of layers, layer-dimensions, and dropout for the SEAL framework according to the best run of the task-net with cross-entropy.
In segmentation experiments, we run sweeps of about 200 runs and use the same search space for overlapping hyperparameters for different models for fair comparisons; once the best hyperparameters are found by wandb sweeps, we use them to run with 10 random seeds.
C.2.1 Multilabel Classification
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
C.2.2 Semantic Role Labeling
We set batch size to 16 and weight decay to 0.00001 for both loss-net ( Θ ) and the task-net ( Φ ).
For the self-attention used in loss-net ( Θ ), we set the hidden dimension to 256 and number of attention head to 1.
We again fix n E , n F as
[Table]
C.2.3 Weizmann Horse Segmentation
In all experiments, we use batch size 8 and 36 image crops (Gygli et al., 2017).
In DVN with GBI, the found learning rate for GBI to get adversarial samples is 7.82 while that for other GBI is 6.96.
[Table]
D Analysis: Parameter Size and Speed of Inference and Training
We present the analysis on number of parameters (Table 17), inference speed (Table 18)
[Table]
and training speed (Table 19) that different methods utilize for Multilabel Classification datasets
[Table]
In summary
[Table]
SEAL requires approximately twice number of parameters compared to cross-entropy loss on training time, however it requires the same amount of parameter on inference time.
Similarly, train time (sec) per epoch of SEAL is slower than that of cross-entropy and energy network as
SEAL requires multiple backpropagation steps for both energy network and feedforward network.
Nonetheless, again, at inference time, the speed of CE and SEAL is the same and is much faster (2 × -7 × ) than the inference of energy network. The analysis we present below are on the seven feature-based dataset, however the trend is similar for text-based dataset as well: twice parameter size on training time but equal parameter and runtime in inference time with CE method. The only difference in the text-based MLC is, due to running on a much larger BERT-baed models, that we only run one step for each update of feedforward network and energy network. Thus, unlike Table 19, train time of SEAL is approximately twice that of CE.
[Table]
However
[Table]
[Table]
[Table]
[Table]
different types and node status, there are some outliers.
Furthermore,
SEAL of steps energy network and feedforward network in the optimization, direct comparison of training time is not really available.
However, the general trend in terms of training time per epoch is CE < Energy Networks < SEAL. E Further Analysis E.1 Effect of samples Given the notable performance of the NCE ranking loss, we raise the question of whether drawing many samples from the task-net helps better estimation of the energy surface.
To examine this question [Table] we perform additional experiments using the regression loss with sampling. Given a sample set S
[Table]
we define the regression-s loss as L E − regression-s = (cid:80) ˜
y ∈ S L E − regression ( x , y ∗ , ˜ y ; Θ) .
We take two approaches in collecting these samples: (1) discrete binary vectors drawn from probability vector ˜ y as done with the NCE ranking loss and (2) continuous perturbation of ˜ y with Gaussian noise. We find that discrete samples do not make any notable changes between regression and regression-s, while continuous samples result in +0.4 F1 improvements on average.
We report results of regression-s using continuous samples in row 10 of Table 1.
We conclude that sampling helps capturing better energy surfaces for training the task-net.
However, the effect and characteristics of the samples might differ for different types of energy losses.
We believe sampling is more effective for the NCE ranking loss where the group of samples contribute in a relative manner in learning surface, in contrast to the regression-s loss to which each sample contributes independently.
E.2 Data-specific analysis
We analyze the characteristics of selected datasets to reason about the high performances of SEAL on them.
E.2.1 genbase
Here, we analyze the factors behind SEAL achieving almost perfect F1 score on the genbase dataset.
It seems genbase stands out with its small label size (27 which is the smallest among Table 5) and with very clear pattern in the label space.
Upon analysis, out of 27 labels, we found that 6 labels only occur as a singleton (by itself), 10 labels only occur as non-singleton, and 7 labels do not participate at all.
This leaves only 4 labels occurring by themselves as well as with others.
Not only that, only 7% (35/500) of training instances have more than one active label.
In this peculiar setting, we conjecture feedforward networks with cross-entropy (CE) loss can easily learn singletons, and energy networks would easily learn co-occurrence, but learning both might be confusing for these two models at the opposite end of the spectrum.
We believe the synergy of CE and a loss from the energy network enables capturing the best of both worlds and achieving a near-perfect score that none of the approaches by itself achieves.
As can seen in Table 1, neither cross-entropy nor energy networks are nearly as strong as SEAL methods.
E.2.2 Delicious and Cal
In Table 1, it seems SEAL -NCE and SEAL -Ranking are particularly well performing on cal500 and delicious.
It turns out that cal500 and delicious have a very high diversity of 1 and 0.981 (Reference:
MLC data repo).
Diversity of 1 means that each data point holds a unique label set.
It seems that the exposure to multiple samples and evaluations of their relative scores in the NCE ranking loss function can be helpful in a high-diversity setting.
F Additional Experiment: Multi-label Classification on Text Dataset with BERT Adapters
[Table]
To test the effectiveness of SEAL on training large language models, in Table 20, we further experiment with text datasets using pre-trained BERT (Devlin et al., 2019) with adapter (Houlsby et al., 2019;
Pfeiffer et al., 2020) as the feature network T E and T F .
Considering the computational expense of running hyper-parameter searches on BERT, based on Table 1, we choose the best-performing SEAL ( SEAL -dynamic with NCE ranking) to compare with baselines: InfNet and cross-entropy.
Tableshows that SEAL -dynamic with the NCE ranking loss is effective on pre-trained models that utilizes large text datasets as well.
On average, SEAL -NCE gains 0.87 and 0.68 F1 points over cross entropy and SEAL -margin (InfNet) respectively on the text datasets.
G Gradient Plots
In multi-label classification, the global energy E global Θ ( y ) , one of the two constituents of the loss-net, captures the dependence between different labels.
In order to inspect the nature of the dependence captured, we randomly pick a tuple of labels ( l i , l k ) such that l i ▷◁ l k , where ▷◁ is a particular relation and plot samples of ∂E global ∂y k ( y ) in the following manner.
First, we sample each component of y uniformly and independently, which, with a slight abuse of notation can be written as y ∼
Uniform (0 , 1) L ; then heatmap of ∂E global Θ ( y )
∂y k on coordinates ( y i , y k ) are plotted.
Figure 2b (see Fig.
4, 6 for more examples) shows the samples from ∂E global ∂y k
( y i , y k ) when i and k are such that y i =
⇒
y k , i.e. there is a positive association between labels l i and l k .
As seen, larger values of y i makes the gradient w.r.t y k always higher for the same value of y k showing positive association between y i and y k .
Contrast this with Figure 2c (see Fig.
5, 7 for more examples), where i and k are such that there is no relationship between y i and y k .
In this setting, ∂E global Θ ∂y k does not change w.r.t y i .
Furthermore, as expected, Figure 2a, shows that when the same procedure is applied to the cross-entropy loss function, we find no dependence between any pair of labels. For the plots discussed in this section, we analyze energy model trained on DVN (4, 5) and SEAL -dynamic-NCE (6, 7) trained on the expr_fun dataset. 1.0 y i 0. y k
E g y k 0.
1.0 y i 0.
{+y k 
+}
E g y k 0.
1.0 y i 0.
y k
E g y k 0.
1.0 y i 0.
{+y k 
+}
E g y k 0.
1.0 y i 0.
y k
E g y k 0.
1.0 y i 0.
{+y k 
+}
E g y k
1.0 y i 0.
y k
E g y k 0.
1.0 y i 0.
y k E g y k 0. 1.0 y i 0. y k
2 E g y k 0.
1.0 y i 0.
y k
2 E g y k 0.
1.0 y i 0.
y k
2 E g y k 0.
1.0 y i 0.
y k
E g y k
1.0 y i 0.
{+y k 
+}
E g y k 0.
1.0 y i 0.
{+y k 
+}
E g y k 0.
1.0 y i 0.
y k
E g y k 0.
1.0 y i 0.
{+y k 
+}
E g y k 0.
1.0 y i 0.
y k
E g y k 0.
1.0 y i 0.
y k    2 E g y k [Figure] y k
E g y k 0.
1.0 y i 0.
y k
2 E g y k 0.
1.0 y i 0.
{+y k 
+}
E g y k 0.
1.0 y i 0.
y k
2 E g y k 0.
1.0 y i 0.
y k
2 E g y k 0.
1.0 y i 0.
y k
E g y k