Structured Energy Network As a Loss
Authors: Jay-Yoon Lee
Dhruvesh Patel
Purujit Goyal
Wenlong Zhao
Zhiyang Xu
Andrew McCallum
Venue: NeurIPS 2022 Accept (2022)
Keywords: Structured Prediction
Structured Energy network
Energy-based models
Trainable Loss-function
Dynamic loss function
Noise-Contrastive Estimation

Abstract: Belanger & McCallum (2016) and Gygli et al. (2017) have shown that an energy network can capture arbitrary dependencies amongst the output variables in structured prediction; however, their reliance on gradient-based inference (GBI) makes the inference slow and unstable. In this work, we propose Structured Energy As Loss (SEAL) to take advantage of the expressivity of energy networks without incurring the high inference cost. This is a novel learning framework that uses an energy network as a trainable loss function (loss-net) to train a separate neural network (task-net), which is then used to perform the inference through a forward pass. We establish SEAL as a general framework wherein various learning strategies like margin-based, regression, and noise-contrastive, could be employed to learn the parameters of loss-net.  Through extensive evaluation on multi-label classification, semantic role labeling, and image segmentation, we demonstrate that SEAL provides various useful design choices, is faster at inference than GBI, and leads to significant performance gains over the baselines.


Structured Energy Network as a Loss Function
Jay-Yoon Leef
Dhruvesh Patel* (dhruveshpate,
Purujit Goyal
lee. jaeyoonyyonnn ac
purujitgoyal umass. edu
Wenlong Zhao wenlongzhao@cs. umass edu
Zhiyang Xu zhiyangx@vt.cnm edu
Andrew McCallum *
mccallumècs. umass edu
Graduate School of Data Science, Seoul National University Manning College of Information & Computer Sciences, University of Massachusetts Amherst Department Computer Science, Virginia Tech
Abstract
Belanger & McCallum (016) and Gygli et al. (2017) have shown that energy networks can capture arbitrary dependencies amongst the output variables in struc- tured prediction: however, their reliance on gradient based inference (GBI) makes the inference slow and unstable. In this work, we propose Structured Energy As Loss (SEAL) to take advantage of the expressivity of energy networks without incurring the high inference cost. This is novel learning framework that uses an energy network as a trainable loss function (loss-net) train separate neural network (task-net), which is then used to perform inference through a forward pass. We establish SEAL general framework wherein various learning strategies like margin-based, regression, and noise-contrastin could be employed to learn the parameters of loss-net. Through extensive evaluation on multi-label classification, semantic role labeling, and image segmentation, demonstrate that SEAL pro- vides various useful design choices, is faster at inference than GBI, and leads significant performance gains over the baselines.
Introduction
Structured prediction is an important machine learning problem in which, given an input x, the model needs to predict the output y E V1 x 3/2 x ... x V2 where there are dependencies across Vı..... V/- Common structured prediction tasks include multi-label classification (Belanger & McCallum 2016
Gygli et al., 2017), extracting parse trees or semantic role labels from texts (Palmer et al. 2010
and image segmentation Müüller 2014). Traditionally, these tasks are performed using models that explicitly represent the dependencies among output variables, such as directed and undirected graphical models, including Hidden Markov Models and Conditional Random Fields. However, except in cases with very limited dependency patterns (Lafferty et al., 2001 Ghamrawi McCallumi 2005), inference algorithms in these methods difficult engineer and expensive to because structured output spaces are exponentially large. In a multi-label classification task, for example, the size of the label space y is 24 given D/ [0,1), 1).
With their growing expressivity and capacity, is common to use large feedforward neural networks to implicitly capture dependencies among output variables. Given input the elements of an output y are predicted in a conditionally independent manner in parallel, solely relying on the representation of input to capture dependencies among output variables. While the feedforward approach is computationally efficient, the widely used cross- entsspyy loss does not directly capture
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

dependencies in the output space and thus often lacks statistical efficiency in capturing multivariate correlations.
recent line of methods (Belanger & McCallum, 2016 Gygli et al., 2017 Rooshenas et al.
2019)
model the joint space of and multivariate y explicitly with energy function Fig (x. y) LeCun
et al. 2006), which aims to learn arbitrary global dependencies in the output space. They relax the combinatorial output space y' to J = [o, _] and apply gradient-based inference (GBI) directly on ý to search for optima on the relaxed energy surface arg ming Exx, y). These energy models are therefore often called structured prediction energy networks (SPENs).
SPENs show noticeable gains in predictive performance over feedforward models and traditional graphical models such as (Chen et al. 2015 Schwing & Urtasun 2015). Due to the use of GBI, how- ever, inference for SPENs remains relatively inefficient when compared to feedforward approaches (Tu & Gimpel| 2019). Moreover, models using GBI are often difficult to train and sensitive to sampling methods and numerous hyperparameters, including step size, number of iterations, and initialization for GBI. This raises a question: Can energy networks be used in a way that is as expres- sive as SPENs, as efficient inference as feedforward approaches, and also easy train? ? To this end, we propose using structured energy networks as parameterized loss functions for feedforward networks, instead of prediction networks.
In this paper, we introduce the Structured Energy As Loss (SEAL) framework which uses trainable SPENs (loss-ne loss functions to guide the training of other feedforward networks (task-net). The key idea is to provide the feedforward network with access to rich dependencies in the output space through learned loss-net ( 22.1 SEAL-static). We further propose learn the loss-net in dynamic fashion ($2.2) SEAL-dyaayy by adjusting the loss-net to be tailored to the most up- date outputs of the feedforward network; and then propose NCE ranking loss uniquely suited for the SEAL-dynamic framework. Using feature-based multi-label classification as case study, we
extensively analyze the impact of various loss functions for updating the loss-net ( $2.3). We further demonstrate the faster inference and higher performance of SEAL, compared to other baseline methods, over feature-, text-based multi-label classification, semantic role labeling, and image segmentation
{ Additionally, we show that energy models capture the dependencies on the output space through analyzing gradient signals in an ablation study ($5 and Appendix C
(g
(e)
Algorithm I: SEAL-dynamic Algorithm
Ee
Eg loss-net
Require: (x, y): Training Instance
Require: Fg: Feedforward Network
loss-net
Require: sampling: True/False flag optimizere, optimizero Require: No. of steps
Require:
F&
F
On, 2 +- RRandom initialization, +- 0
while T do
task-nel
task-net
if a sampling then
S +- (y i
Kly Fs, (x))
X
X
else
Figure 1: Overview of the SEAL framework. The figure on the left describes the update to the task- net in Eq. 3 and Eq. and the figure the right describes the update to the loss-net in Eq.4
S + (Fo. (x))
b singleton set
+ 1+1 1
Update θ as Eqn.
g Jusing ý E S and optimizere using optimizerg
Update & as Eqn.
2 Structured Energy Network as Loss (SEAL)
Notation. X denotes the input space, y {0, 13 L. the label space, and 5 [0, 1)" the continuous relaxation of y. Let D= (x(0), yli) X' y denote the dataset, and yı E {0, 1) the i-th dimension in the label vector y for i E (1...., L}. and ÿe E [0, 1] its continuous relaxation. The feedforward network Fo: : X + y is a parametrized function that maps an input x E X to an output
We refer energy network as loss-net when it is used trainable loss function for another network.

---

Fx(x) E V, where (x);; is the predicted probability of label occurring, given the input x. The structured energy network Ee A''s 3 - IE is parameterized function that maps an input x and continuously relaxed output y to real-valued scalar energy. For clarity, we refer to the feedforward network as task-net, and the structured energy network as loss-net.
As depicted in Figure the proposed SEAL framework consists of two parametric components. First, the loss-net Ee is trained by an energy loss Lg to capture an energy surface that reflects the latent dependencies among output variables. Low energy indicates a high quality of label prediction given an input. Second, a task-net F& is trained using a loss Lr that involves minimizing the energy Ee estimated by the loss-net. After training finishes, the loss-net no longer utilized; inference time predictions are obtained solely from the task-net as ú; 1 (ar); 2 0.5). Unlike SPEN which requires iterative gradient-based inference, a task-net net trained with the SEAL framework has the same computational efficiency at inference-time as standard feedforward networks (Appendix Dj. while enjoying a loss that captures the label dependencies through the loss-net.
To guide the task-net training with the loss-net Ee, we propose the task-net training loss Lp(#; Q) be weighted sum of estimated energy and a binary-cross entropy (BCE) loss as
( θ) = Ex Fx()) + Aa Σ BCE( Fx())))
(1)
where (x is training instance and BCE(U;: Fo (x)))= - -ly log Fx(x); + (1 25) log (1 Fo (x))). By incorporating the energy term in L,,, we enable the loss function to reflect the latent variable dependencies in the output space, whereas BCE treats L labels in a condi- tionally independent fashion.
We present the formulation of Ee used for multi-label classification to analyze the effect of intro- ducing Ee into Le without loss of generality. We use energy network Eg (x, y) that has the same structure with Belanger McCallum (2016) and Gygli al. (201)) for a fair comparison with them. The energy Feg is the sum of local energy Elocal and global energy Fig Egglobal which are defined as:
Edocal y) Σ (,b,TT(()) Lightall (x,y) = v (My),
(2)
where b;, v, M, contain learnable parameters, Te is feature network to represent input in the loss-net, and 7(2)= = log(1 + e²) is the softplus activation function. We observe from the derivative of the global energy @Egghal
{M (Aag() that all dimensions of decides the gradient signal for each label i. The global energy term Lig
L-global in Lp, specifically, allows the gradient of the task-net loss to capture the dependency of y; on all L dimensions of y. In contrast, the gradient of BCE for each label i only depends on the i-th dimension of y. More details are in Appendix A
2.1 SEAL-static
Since the task-net loss Lp depends on the quality of the loss-net, critical to find the loss-net parameters @ that produce the best loss surface for training the task-net 4. In order achieve this, we first estimate the loss-net parameters & over training data and then optimize the task-net parameters + by plugging in the fixed loss-net parameters Ô into Lr. Formally, given training instances (x,y) D, the net parameters & are trained in the following manner:
min &
L ( θ) D
s.t. θ arg
arggmn min
1
Σ LEx(x,y.::0)
[D]
|D]
Since the loss function LF is static after pretraining θ, we call this approach SEAL-static. We discuss various types of Læ losses in
2.2
SEAL-dynamic
Ideally, we would like to learn a perfect energy surface by estimating θ that is static. However, estimating accurate energy surface over high-dimensional, continuous joint space X y is

---

challenging task. Instead, we hypothesize that using loss-net to dynamically estimate the energy surface that focused around the current input and task-net oulput pair (x, Fa(z)) could be more
beneficial in teaching task-net. To this end, we propose SEAL-dynamic which dynamically adapts loss-net so that the region of interest for training task-net well represented.
Given a batch of training data B,, complete training step of SEAL-dynamic provided as
θ, 0 Ve
Σ B1
LE (e)
(4)
[el
4 P
V
Σ O.)
(5)
BB
B,
where the loss-net parameters θ and the task-net parameters D are optimized in alternating fashion. Note that, from Eq. to Eq. 4 arbitrary label samples y were replaced with Fs, (x) from the task- net during the training of O parameters. Thus the energy surface learned by the loss-net is adaptive to the task-net. On the contrary, in SEAL-static, the training of the static loss-net is independent of the task-net. Further, the update step for the task-net parameters D; relies on dynamic loss-net parameters O, rather than static + In $4 we show the effectiveness of the proposed SEAL framework compared to standard feedforward networks and SPENs, and further that SEAL-dynamic often improves over SEAL-static.
2.3 Energy losses Lg
We now discuss various energy loss functions Lg with different characteristics such as margin-based, regression-basee and moise-contrrastve.. swapesina--a-mmaaaaeeeeeeaeeee-e based, that can be plugged into the SEAL framework to train the loss-net. For brevity, we use (e) and Læ(x, y, θ) interchangeably.
Margin- based ( ) SPEN Bellanger & McCallum 2016) learns structured energy network with the SSVM loss (Taskar et al. 2004 [ sochantaridis et al. 2014
so that energy function learns to predict sufficient energy difference, larger than the margin A (y.y) between an arbitrary output ý and true output y. To examine the effect of the margin- based loss in SEAL, we follow SPEN and utilize the SSVM loss as
Σ max (A (y. Es (x, + Ee (x,y)) x,y
marging
(6)
SPEN resorts to gradient ascent on ý solve the optimization problem arg max, A y) Eg (x. f). InfNet (Tu & Gimpell 2018) proposes an adversarial training of the task-net + together with the energy network O We identify InfNet as a special case of SEAL-dynamic that uses L margin We defer detailed description of InfNet to the related work section
Regression-based (Lg regressio)) Deep Value Network (DVN) Gygli et al. 2017) attempts to learn an energy network which directly outputs score that is similar to the metric s y) of interest, such as F1 score, that compares arbitrary ý with true output y. Following DVN, we let the true score s(.) and o (-E()) be between [o, 1] and express the regression-based loss as cross entropy between
and o( ((-)):
Lg regressssion
-s y) log (- (-Ee(x, 8)) - (1- log (1 + (-Ee(x, (7)
where (-) is a sigmoid function. DVN utilizes two samples to train the model: the model prediction that minimizes the energy and an adversarial sample y that maximizes the loss I. L-g-rsgrssioon- We follow
this approach for training SEAL-static with I_g regresssssioo-- To train SEAL-dynamic with Lg regression
we simply use the output of task-net, Fa.-, (x), as described in Eq.4
To better capture the energy surface in SEAL-dynamic, we further introduce regression-s loss
Ug-regressionn- Eyes Legressssioonnx, y;; O) where the sample set taken by continuously perturbing task-net output Fo.., (x) with Gaussian noise (experimental analysis on the 'effect of samples' can be found at Appendix E.T)
In this paper, we adopt soft score s(y.y) from Gygli et al. 2017 that is defined on the continuous E 1J4.

---

Noise-contrastive ranking (LE-NCE). Lastly, we propose to use noise contrastive estimation (NCE) ranking loss (Ma & Collins 2018) uniquely for SEAL-dynamic. We interested in capturing appropriate energy surface in High-probailltg output spaces of task-net. To reflect such space accurately, we propose to train the loss-net Flg to contrast the true output y with a group of K negative samples drawn from the task-net prediction Fg. (x) motivated by NCE.
Before we discuss Lg- LE-NCE for SEAL-dynamic in more detail, we first review the original form of NCE ranking loss from Ma & Collins 2018). For K samples y(k) PN, k 1,.... K, drawn from noise distribution PN. rewriting ground truth y y()) without loss of generality, the NCE ranking loss is defined as
exp (s(x, y (0)) 0)) Σx= exp (s(x, (()); 0))
LE-NCE = - log
where s(x.y; = - (x. y) - log (x (8)
Given the noise samples g()), minimizing Eq. minimizes the energy of the ground truth sam- ple (() while increasing the energy of the noise samples. In SEAL-dynamic, we propose to use II, P(y][x; &t), which is dynamically changing, in place of the static PN (y) Eq.(8). The NCE
ranking loss is thus
Eq.
8 but with s(x = -Ee((x,)) Σ log P(),(x;;,))
The novelty of this proposal is that task-net, viewed as producing dynamic noise distribution, is in turn trained using loss-net.
The NCE ranking loss Lg-NCE brings two benefits in SEAL-dynamic. First, Lg-Nce captures the output distribution the task-net rather than just a single point. Second, L-E-NCE allows estimation of a more fine-grained loss surface as the training steps proceed. In our SEAL-dynamic learning procedure, we hypothesize that II: P(y x; (.) becomes closer to the true data distribution as training steps proceed. Contrasting these noise samples to the ground truth could lead the loss surface estimated by loss-net to be more fine-grained. This is in line with the known fact that NCE methods work the best when the noise distribution is close to the data distribution but not the same (Gutmann) Hirayama 2012).
To thoroughly experiment with the proposed NCE ranking loss -CeNCE, we also train a static energy network with LE-NCE in order to utilize as a prediction network with GBI and as a loss-net in SEAL-static. We set (y = II, P(ys x; V in Eq. where V is fixed pretrained feedforward network. When we train an energy network Ee with LE-NCE,
exp(-Eo((x.)))
Evey expl exp(-Ee(x.y)) becomes an
unbiased estimator of the true distribution P(y (Ma & Collins, 2018). Given a perfect energy
Ex(x.y), minimizing it with respect to y in SEAL-static and for prediction with GBI is equivalent to minimizing the estimate of log P(y
3 Related Work
Learning with dynamic loss. Wu et al. (2018) and Huang al. (2099) attempt to learn a dynamic loss for multi-class classification tasks. Both approaches try to learn a parametrized loss function that can directly increase the model's metric score m, such as the BLEU or 0-1 accuracy, on the validation set. Since m is usually non-differentiable, Huang et al. (2019) view this problem as reinforcement learning problem and attempt to learn loss function that maximizes the expected reward. On the other hand, Wu et al. (2018) relax the m to be differentiable and try to learn loss parameter (9) that can improve the relaxed score on feed-forward (œ) output.
There are three major differences between SEAL and the previous work. While both Wu et al. 2018) Huang et al. (019) use the parametric form a Oog (x)x) to define the loss function, SEAL utilizes bigger class of functions expressed using general energy network Eg. Moreover, SEAL does not require manually designed state vectors and works directly with the task-net outputs. Lastly, while the SEAL framework is designed for structured prediction, Wu et al. (2018) and Huang et al. (2019) are designed for multi-class classification and it is non-trivial to extend these methods for structured prediction.
5

---

Structured Prediction Energy Network (SPEN).
Structured prediction energy networks (Belanger
McCallum 2016) and its variants (Gygli et al. 2017: Rooshenas et al. 2019) (together referred to as SPENs) learn an energy network Fe X' y - IR and predict the output using gradient-based inference (GBI) with the objective y = arg miny y) The optimization objective for training the SPENs relies on finding adversarial or margin-violating points in the label space using gradient based inference (GBI). This learning process in SPENs is highly sensitive with respect to hyper-parane sepaa-paraeeterss such as the learning rate and initial points for the GBI, resulting in high variance in the model performance. Nonetheless, SPENs are strong baselines for structured prediction tasks and are used as motivations for the design of SEAL. They are further discussed in 2.3
Jointly learning O and Φ. Inference Network (InfNet) (Tu & Gimpel 2018 is the first work that interactively learns a structured energy network and a feedforward network. It aims to replace the expensive GBI in SPEN with feedforward network's 's forward pass. During its training with the SSVM loss (Eq. D. SPEN maximizes A (y. y) - Ee (x, y) with respect to y by GBI. InfNet, on the contrary, trains feedforward network Fg to generate y = Fo (xx) that maximizes the term. This results in an adversarial framework (Goodfellow et al. 2014 where a feedforward network maximizes the bargin-based loss while an energy network minimizes it. The approach, however, suffers from train-test-time objective mismatch (Tu et al. 2020). While the feedforward network serves as an adversarial sampler during training, the end goal is to learn a feedforward network that predicts outputs with low energy at test time. Tu et al. (2020) resolves this by learning two separate networks: one adversarial sampler for training and the other for performing inference.
Our proposed SEAL generalizes, and differs from, InfNet by providing a framework that can employ completely independent loss functions for training loss-net and task-net. SEAL updates task-net in the direction of reducing energy defined by loss-net; loss-net is only used as a tool for evaluation of loss, and how loss-net is trained does not matter as long as it estimates good loss surface for tesk-net. The SEAL framework is not interested in producing the similar test-time behaviour of SPEN as InfNet did. Despite the difference in the end goal, mechanically speaking, InfNet is special case of SEAL-dynamic (i.e. SEAL-dynamic with Lg-marg)n) with the margin removed from the SSVM loss for the feedforward model training. In fact, Tu al. 2020) reports that this special case leads to more efficient and stable learning of the feedforward network. We use this special case of InfNet to represent SEAL with La-man in the main experiment, and by doing so, we reinterpret the success of InfNet as result of utilizing energy network as a loss function, as our formulation shows, rather than the result of task-net mimicking the test-time behaviour of energy network.
Experiments
Tasks and Datasets. We demonstrate the effectiveness of SEAL on widely deployed structured prediction tasks: feature and text-based multi-label classification (MLC), semantic role labeling (SRL), and image segmentation. brature-based MLC is a standard evaluation task in previous works on SPENs (Belanger& & McCallum 2016: Gygli et al., 2017: Tu & Gimpel, 2018). We utilize 7 feature-based MLC datasets which cover various label sizes (ranging from 27 to 4k), training sizes, and input characteristics (binary, real) as shown in Table 5 to extensively compare baselines and proposed SEAL approaches: cross-entropy (CE), energy network, SEAL-static, and SEAL-dynamic. Then we verify that SEAL-dynamic, which outperforms SEAL-static on feature-based MLC, can also effectively train large pre-trained language models like BERT (Devlin et al.) 2019) on text-based MLC and SRL tasks and convolutional networks on image segmentation. Dataset statistics are described in Appendix B
Feature network, loss-net and task-net. In MLC and SRL experiments, to represent input x, we utilize the same feature network structures Te and Tp for the energy and feedforward networks, which correspond to the loss-net and the task-net for SEAL framework. The energy network utilizes TE in the manner shown in equation 2 Given feature network Tp that generates features in R the task-net (z) = GTp (æ)) where G E R..h is matrix consisting of learnable embeddings, one row of embedding for each label. Specific feature network structures vary by tasks: multi-layer perceptron for feature-based MLC and BERT for text-based MLC and SRL. In image segmentation experiments, we follow Gygli et al. (2017) and use a Fully Convolutional Network (FCN) as the task-net; the loss-net uses the same convolutional layers in the FCN, followed by fully connected

---

Table 1: Performance (F1 f) of reature-based MLC datasets. The task-net learned with stAl-static and SEAL-dynamie outperforms cross-entropy- trained model most of the time. The SEAL framework which utilizes energy network loss performs better and faster than an energy network utilized as a prediction network. Best results per dataset are marked with underline; best results within a framework (energy network with GBI, Stal-static, SEAL-dynamie) are marked in bold.
mothod
samples
bibtex
discrete inpul datasets delicions
genhase
cal500
cumiiuous inpul datanees curlesev
expr_fun fun
spo_fun
mean Fll gain over CE overall
genbase
emso-catropy (CF)
42.40 D.I
20.89 + 201
47.37 7.47
33.58
0:13
42,19 -017
37.50 o
27.81 0.33
energy network with GBI
SPEN
42.99
24.30 5007
32.13 1
57.34
41,86
36,74 9.1
27.95 + 0.15
-2.52
-0.40
DVN
45.95
24.77
77.92
47.74
25.49
31.47
29.35
-1.42
NCE ranking
12.95 1L33
12.69 0.15
12.40 0881
0.19 1010
27.74 1
18.00 117
-20.40
-17.98
nargin
43.11 IL15
28.08 0.9
57.45 1177
33.91 D.15
42.15 oiu
38.13
28.15 0.11
1.46
0.31
regression
42:29 ALLS
30,09 9/16
96.68 a:11
37.63 4:4
42,18 2017
38.12 11177
28,42 a 0.89
7.78
0.86
NCE ranking
4300
30.08
96,43
57,82
42.11
37.78
28.29
SEAL-dynumic
margin (lnNee))
29.75 a:11
2021
a:440
41.83 ao1
37.81 1:1.4
28.43 u.133
7.59
regression
23.74:t0.4
29:79 2015
96.95 20.14
37.99 00.33
41.67=000 2018
37.99 2.00
29.02 4.11
8.05
1.13
regressionns
64.53 2..5
29.87 2007
96.81 =0020
38.95 2.13
42.37
37.93=00.1
28.29 x 20.12
8.29
1.43
NCE ranking
44.76
34.67
97.32
42.77
38.28
28,83
9.64
2,03
Inference speed
(exampled/sc)
feedforward
840
2307
1005
574
1080
317
3801
5231
Average specdup
3,68x
layers which replaces the tranposed convolutional layers in FCN to output scalar scores, instead of the energy formulation in Eq. 2
Training. We use separate ADAM optimizers (Kinmma & Ba 2014) for the loss-net (+) and the task-net (4) which optimize parameters in alternating fashion We defer specific training details
such hyperparameters, gpu environment, and number of random seed runs to the Appendix C.T
4.1 Multi-label classification for feature-based datasets
Multi-label classification (MLC) is a structured prediction task that does not hold an obvious latent structure in the multivariate space. In addition including Bibtex and Delicious that were used in previous works (Belanger & McCallum 2016 Gygli et al. 2017 Tu & Gimpel 2018). we carefully picked smaller (Genbase, Ca1500) and larger datasets (Burlex-ev) as well as datasets with known taxonomy (Spo_ FUN, Expr FUN) to examine SEAL in various scenarios. The experiment results (test F1) across 7 feature- based dataset are presented in Table TT feedforward network trained with cross-entropy (row 1), energy networks (row 2-4) described in 3htht is evaluated with GBI, task-nets trained with SEAL-static (row 5-7), and task-nets trained with SEAL-dynamic (row 8-11).
In general, SEAL-static and SEAL-dynamic outperform both cross-entropy (CE) and prediction energy networks with GBI on almost all datasets with average gains of SEAL-dynamic over CE ranging from +0.67 to +2.93 Fl points, excluding results on genbase dataset. While the near-perfect performance of SEAL on genbase is interesting (see Appendix 2 for detailed analysis). we exclude this excessive gain for computing the average performance when comparing different models throughout this paper.
Since SEAL uses single forward pass of the task-net for inference, it is as fast as the feedforward model and 3.63x faster (on average) at inference than the corresponding energy network trained using GBI. However, in training time, SEAL utilizes about twice the parameter to feedforward model as it loads both task-net and loss-net and takes about 5x more time to train compared to feedforward model. In summary, SEAL is better performing than both energy network and feedforward model and as fast as feedforward model at inference time by utilizing more computations in training time. The training time, inference time, and parameter size of different methods are reported in Appendix D
Cross entropy vs SEAL. While cross entropy cannot capture the dependencies between the labels, we find that global energy of the loss-net visibly captures some of these dependencies. We plot the gradients of global energy with respect to a pair of labels to analyse pairwise dependencies. These plots can be found in $ and the Appendix G
'The code we used to train and evaluate our models is available a https://github.com// com/ee// seal- Seal-neurips-000
7

---

Energy network vs. SEAL-static. There is high variability in the performance of the energy network with GBI. For instance, while SPEN and DVN surpass CE models with large margins for half of the datasets, they underperform on the rest of the datasets with similar margins. Further, even though DVN has the best performance on three datasets, its average performance (-genbase) lower than CE. Nonetheless, in SEAL-static, when the identical energy network is utilized as loss-net, the performances increase in a stable manner (0.42-2.26 Fl on average) with faster (1.8x-7.4x) inference time. This trend is most evident with the NCE ranking loss, where the energy network trained with it drastically fails to provide a good energy surface for predicting output with GBI, whereas it provides useful loss surface for training task-net.
We believe there are two factors for this stable improvement of SEAL-static. One is that injection of an energy surface as a loss is softer injection of gradients compared to it being used as surface for inference. The second factor is synergy between cross-entropy and the energy surface. Even when certain regions of the energy surface are less accurate, the cross-entropy signal could compensate for the erroneous surface. Thus, we claim that an energy network as a loss is a more stable way of utilizing it compared to GBI.
SEAL-static vs. SEAL-dynamic. As seen Table [ with an average gain of 0.47 to 1.97 F1 points, SEAL-dynamic is always on par or better than the corresponding SEAL-static. This confirms our hypothesis that it is helpful for the loss surface to consider the current state of the task-net as opposed to estimating the global energy landscape. Lastly, SEAL-dynamic not only brings better performance but also simplifies the engineering as both loss-net and task-net can be trained from scratch. We observed that binary cross entropy in Ly stabilizes the learning signal even in the early stage when loss-net might not be well-trained.
In summary, the SEAL framework outperforms CE loss and energy networks with GBI. We observe the generality of the SEAL framework as all of the energy loss types benefit working with SEAL, although more effective types of energy loss exist. SEAL-dynamic brings extra gain in performance to SEAL-static and among them, the NCE ranking loss shows the most notable improvement. Lastly, as can be seen with the example of the NCE ranking loss, energy network that serves as a poor prediction network does not necessarily correlate to poor energy loss for SEAL.
4.2 Multi-label classification for large text datasets
To test the effectiveness of SEAL on large language models, we finetune pre-trained BERT as the feature network TE and Tr on text MLC datasets. Considering the computational expense of running hyper-paramenn parameter searches on BERT, we compare the best-performing SEAL from Table | SEAL-dynamic with NCE ranking, with baselines: BERT finetuned with cross-entropy and the state-of-the-ar teXt-MLC model (LACO) (Zhag et al.) 202T). Following LACO, we experiment with the Arxiv Aca- demic Paper Dataset (AAPD) (Yang et al. 2018) which consists of computer science papers with annotations of 54 topics. Table 2 shows that SEAL-dynamic with the NCE ranking loss is effective on pre-trained models that utilizes large text datasets as well. SEAL-dynamic again improves task-net trained with cross entropy and performs on par with MI.C- specific model architecture. To efficiently work with large pre-trained models, we further studied utilizing BERT-adapter models on three additional text-based datasets (Appendix F) which also showed SEAL-dynamic-NCE model outperforming cross-entropy and InfNet.
Table Test Fl for AAPD
method
F1 f
BERT (cross-entropy)
73.97
LACO Zhang et al. 2021
74.90
SEAL-dynamic (NCE)
74.95
4.3 Semantic role labeling
To examine SEAL on capturing label dependencies at sequence level, we perform experiments on semantic role labeling (SRL) (Plmer al. 2010) using standard benchmark (CoNLL-112) (Pradhan
ct al. 2013). SRL aims to annotate the predicate-argument argument structure on plain text extracting who did
what to whom. Because the argument roles in a sentence are dependent to each other, injection of
structural priors has shown improvements in performance (Lee et al. 2019) Mchta et al. 2020
et al. 2020). To apply SEAL, we utilize pre-trained BERT as the feature network TE and Tp. To
capture the energy on variable-length output, we modify Eeglobal in equation 2 to use self-attention
layers. We defer more detailed task description, data statistics, and energy architecture to appendix.
8

---

We compare SEAL-dynamic (NCE) on BERT-base against two models: one model trained with cross-entropy loss and another model trained with structured tuning framework (Li al. 2020) where both models utilize BERRT-base with CRF layers (Lafferty et al. (00T). Structured tuning enforces the model to learn the Spec-seecific constraint (unique core role) through differentiable constraint loss. Unique core role violation refers to model predicting a type of core argument more than once when it should be unique. To examine effectiveness of SEAL in capturing output dependencies, we report unique-core-role violation rate (UCR) in percentage along with Fl score. While SEAL-dynamic does not enforce any specific constraints nor utilize CRF layers, it shows notable reduction constraint violation and higher improvement in performance.
Table 3: Results CoNLL-12 12. We report Fl and unique core role violation (UCR) in percentage.
Table 4: Test mean image IoU for Weiz-
mann Horse segmentation.
Dev
Test
method
Mean IoU f
method
F1 f
UCR 4
F1
UCR +
FCN (cross-entropy)
77.80
BERT-CRF
85.92
1.41
85.97 1.32
DVN with GBI
72.93:u0.5
Structured tuning
86.18
0.78
86. 12:006 0.79
SEAL-dynamic (regression)
-9.23
SEAL-dynamic
86.94 = 0,07 0.92+0.02
86.90=0.06 =.91
SEAL-dynamic (NCE)
76.78:00.00
4.4 Image segmentation
We evaluate SEAL on binary image segmentation using the Weizmann Horse Image dataset (Borenstein)
& Ullman 2004). This task requires model to learn to predict detailed structures of horse shapes from scarce 160 training images of a low 32x x 32 resolution. Following Gygli et al. (2017). we implement Fully Convolutional Network (FCN) (Long al., 2015) as our feedforward baseline and DVN (Gylli et al. 2017) as our energy network bascline. Given a 3-channel L-pixel RGB image input x, FCN outputs the probability y E [0. 1|e that each pixel belongs to a horse object; binary segmentation mask can be obtained through 1 2 0.5). In contrast, DVN takes a pair of image x and soft segmentation mask ý to output their compatibility score and resort to gradient-basee inference for prediction. Following Gygli et al. (017)), we train DVN by plugging the soft IoU score
1=1 min(ū..
while loU(y, yny" for y.y E 10, yu
s s(., "Vyy")=
(10)
EN max(U:. v²)'
into the Eq. For training with the SEAL framework, we adopt the FCN and DVN architectures for the task-net and the loss-net respectively. In experiment, we examine SEAL-dynamic with regression which uses the same energy loss as DVN and SEAL-dynamic with the NCE loss because it performs the best in multiple other experiments. More dataset and model details are in Appendix B experiment details are in Appendix
Table 4 shows test performances of FCN trained by cross-entropy, DVN, and FCN trained with SEAL-dynamic. Similar to $4.1 we observe that the GBI procedure in DVN training and inference slow and unstable (Appendix B.3 for more details). While DVN does not outperform feedforward model learned with cross- dynamic (regression) improves the mean IoU of FCN by 1.43 IoU point and has less variance across random-seed runs. Additionally, we find that SEAL-dynamic with the NCE loss proposed for MLC does not improve FCN. This is likely because many negative samples in NCE are non-realistic sample each pixel independently and do not take into account the prior that nearby pixels on image are likely be in the same class. We leave the exploration of other energy network architectures and sampling strategies in the image domain as future work.
the same energy network structure used as a loss-net in SEAL-
5 Further analysis
In this section, analyze (1) whether gradient of loss-net rightly captures multivariate dependencies, (2) the effect of ranking loss, and (3) we provide pointers to the data-specific analysis and discussion on the effect of samples in Appendix E
Visualization of gradients order to inspect the learning signal provided to the task-net by the loss-net, we plot single component of the gradient from the global energy of the loss-net, i.c.

---

BEg
Bys (ye, yk). In Figure we inspect the global energy function of DVN trained on the expr_fun, a dataset that provides a taxonomy on the output space. We first plot the discussed gradient component when have positive association with 44 i.e.,
1 2b). Contrasting this with the when ls does not have a significant association with 4 22). we see that the gradient of the global energy rightly captures the presence and absence of association between the labels. In contrast, binary cross-entropy is incapable of capturing label-to-label association (2) which is obvious when we inspect its gradient as shown in Appendix A See Appendix G for more examples and for detailed description of plotting procedure.
(a) BCE w/ and w/o assoc.
(b) energy w/ positive assoc.
(c) energy w/o significant assoc.
Figure 2: Given there exists positive association 4 + 4s between two labels, the gradient BEg
Zy.
is plotted (b) and compared with the gradient of BCE (a) when true 4 = 1. The gradient of global energy is also plotted (c) for the case when there is no significant association between the two labels.
Effect of applying a ranking loss directly on F& With the large improvement that SEAL-NCE achieves, examine whether the gain is coming from our SEAL framework simply from the power of a ranking loss itself that can also be applied to the task-net F& directly (i.c. s(x, y; 9) = E, log xx (b)) in equation 8). We conduct this ablation study over relatively small datasets: genbase, cal500, and delicious. While we observe a near +10 F1 point in genbase, we see a decrease in performance cal500 (-1.0 F1) and delicious (-2.3 F1) compared to the plain cross-entropy model. With these experiments, we first reaffirm that genbase is an outlier dataset where capturing structure can significantly increase the performance. Second, conclude that the ranking loss without an loss-net is not very effective and that the loss-net indeed plays a major role in the SEAL framework.
Conclusion
We propose SEAL, a framework that can adopt structured energy network as a trainable loss function for training a feedforward network. We show that SEAL a general framework that can work with various loss functions, such as margin- based, regression-basee different architectures, such as MLPs, CNNs, and pretrained models (BERT). Through extensive experiments on feature-based, text-based MLC, SRL, and image segmentation, we show that SEAL provides an effective way to utilize an energy network and cross-entropy loss together: performing better, faster, and in more stable manner when combined. Lastly, through the ablation study, examine how the loss-nct captures the dependencies among output variables. This research opens up doors for various future work, including but not limited to, exploration of different architectures of energy network; application of learned loss-nets in semi-supervised setups. In the long horizon, are interested in whether general pretrained scoring neural networks, such as BLEURT (Sellam) et al. 2020) and BERT-score (Zang et al. 2019), could serve as loss function when it can provide backpropagatable gradients to the output space of feedforward network.
and NCE ranking losses, and with
Acknowledgments and Disclosure of Funding
This work was partially supported by New Faculty Startup Fund from Scoul National University, IBM Research AI through the AI Horizons Network and the Chan Zuckerberg Initiative under the project Scientific Knowledge Base Construction. Additional support was provided by the NSF under Grant Number IIS-2106391, and the Office of Naval Research (ONR) via Contract No. N66001 1924032 under Subaward No. 123875727 from the University of Southern California.
We get label-label dependency from true taxanomy of expr_fun. fun.
10

---

References
Aly, R., Remus, S., and Biemann, C. Hierarchical multi-label classification of text with capsule networks. In ACL 2019 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Student Research Workshop, pp. 323-330, 2019. ISBN 1781950737 doi: 10.18653/v1/p192200
Belanger, D. and McCallum, A. Structured prediction energy networks. In 33rd International Confer- Machine Learning, ICML 2016, volume 3, pp. 1545-15544 2016. ISBN 9781510829008.
Biewald, L. Experiment tracking with weights and biases, 2020. URL htttps://ww... wandb..... com/ Software available from wandb.com.
Borenstein, E. and Ullman, S. Learning to segment. In European conference on computer vision, pp. 315-328. Springer, 2004.
Chen, L.-C., Schwing, A., Yuille, A., and Urtasun, R. Learning deep structured models. In International Conference on Machine Learning, pp. 1785-1794, PMLR, 2015.
Devlin, J., Chang, M. W., Lee, K., and Toutanova, K. BERT: Pre-training of deep bidirectional trans- formers for language understanding, NAACL HLT 2019 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies Pro- ceedings of the Conference, 1: 1:4171-4186, 2019. URL htttt:////axiiii www.aaiiii..ogaaaaaaaaaa aat////ss//111 04805
Ghamrawi, N. and McCallum, A. Collective multi label classification. In Proceedings of the 14th ACM international conference on Information and knowledge management, pp. 195-200, 2005.
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. Generative adversarial nets. Advances neural information processing systems, 27, 2014.
Gutmann, M. and Hirayama, J.-i. Bregman divergence as general framework to estimate unnormalized statistical models. arXiv preprint arXiv: 2oti::::22277.7.,. 2012.
Gygli, M., Norouzi, M., and Angelova, A. Deep value networks learn evaluate and iteratively refine structured outputs. In 34th International Conference on Machine Learning, ICML, 2017, volume 3, pp. 2160-2170, 2017. ISBN 97815 07781500551444 144..
fttttp:://ggo.g1/800..
Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., De Laroussilhe, Q., Gesmundo, A., Attariyan, M., and Gelly. S. Parameter-cfficien transfer learning for NLP. In Chaudhuri, K. and Salakhutdinov, R. (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 2790-2799. PMLR, 09-15 Jun 2019. URL /
https://prooceeddngg mlr. press v97 www.hholllllaahanoa..
Huang, C., Zhai, S., Talbott, W., Martin, M. B., Sun, S.-Y., Guestrin, C., and Susskind, J. Addressing the loss-metric mismatch with adaptive loss alignment. In International Conference on Machine PMLR, 2019.
Learning, pp.
Kingma, D. and Ba, J. Adam: method for stochastic optimization. arXiv preprint 1411v:1412.69880 2014.
Lafferty, J., McCallum, A., and Pereira, C. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. 2001.
LeCun, Y., Chopra, S., Hadsell, R., Ranzato, M., and Huang, F. A tutorial energy-based learning. Predicting structured data, 1(0), 2006.
Lee, J. Y., Mehta, S. V., Wick, M., Tristan, J.-B., and Carbonell, J. Gradient- Inference for Networks with Output Constraints. Proceedings of the AAAI Conference on Artificial Intelligence. 33:4147-4154 2019. ISSN 2159-5399. doi: 10. 10.166091. 160///60/.a..... v33101.33333 33014147. URL www. aaai. org
Lewis, D. D., Yang. Y., Rose, T. G., and Li, Rev1: new benchmark collection for text categorization research. Journal of machine learning research, 5(A99))361-397, 2004.
11

---

Li, T., Jawale, P. A., Palmer, M., and Srikumar, V. Structured tuning for semantic role labeling. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 1402-8412, Online, July 2020. Association for Computational Linguistics. doi: 10. 10.18653/11/2000 acl-main.744. URL htttps///cc..abboooogg.oo///////
acl-main. 744
Li, Y., Tarlow, D., and Zemel, R. Exploring compositional high order pattern potentials for struc- tured output learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 49-56, 2013.
Long, J., Shelhamer, E.,, and Darrell, T. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 3431-3440, 2015.
Ma. Z. and Collins, M. Noise contrastive estimation and negative sampling for conditional models: Consistency and statistical efficiency. In Proceedings of the 20/8 Conference on Empirical Methods in Natural Language Processing, pp.
Brussels, Belgium, October-November 2018. Association for Computational Linguistics. doi: 1/ 1405. URL https /aclanthology Drg/D18-1409
Mehta, V., Lee, Y., and Carbonell, Towards semi-supervised learning for deep semantic role labeling. In Proceedings of the 2018 Conference Empirical Methods in Natural Language Processing, EMNLP 2018, pp. 4958-4963, 2020. ISBN 9781 1781998007841. URL //g giubub com/ hommalllenai/lllennpp
Müller, A. C. Methods for learning structured prediction semantic segmentation natural images. 2014.
Palmer, M., Gildea, D., and Xue, N. Semantic role labeling. Synthesis Lectures on Human Language Technologies, 3(1):1-103, 2010.
Pfeiffer, J., Rücklé, A., Poth, C., Kamath, A., Vulić, I., Ruder, S., Cho, K., and Gurevych, I. Adapterhub: A framework for adapting transformers. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing: System Demonstrations, pp. 46-54, 2020.
Pradhan, S., Moschitti, A., Xue, N., Ng. H. T., Björkelund, A., Uryupina, O., Zhang, Y., and Zhong, Z. Towards robust linguistic analysis using ontonotes. In Hockenmaier, and Riedel, S. (eds.), Proc Proceedings of the Seventeenth Conference on Computational Natural Language Learning, CoNLL 2013, Sofia, Bulgaria, August 8-9, 2013, pp. 143-152. ACL, 2013. URL https: pettps://aclanthollee
org
org/W13- W13 3
Rooshenas, A., Zhang, D., Sharma, G., and McCallum, A. Search-guided, lightly-suerrviedd training of structured prediction energy networks. Advances in Neural Information Processing Systems, 32: 13522-13532, 2019.
Sandhaus, E. The new york times annotated corpus. Linguistic Data Consortium, Philadelphia, 6 (17): 2008.
Schwing, A. G. and Urtasun, R. Fully connected deep structured networks. arXiv preprint 1r330:150. 2015.
Sellam, T., Das, D., and Parikh, A. P. Bleurt: Learning robust metrics for text generation. In Proceedings of ACL, 2020. URL htlphttt////xxvv.rrgg 2004.. 04696
Taskar, B., Guestrin, C., and Koller, D. Max-Margin Markov Networks. Advances in neural information processing systems 16 (2004): 25, 2004.
Tsochantaridis, I., Hofmann, T., Joachims, T., and Altun, Y. Support vector machine learning for interdependent and structured output spaces. In Proceedings of the twenty-first international conference on Machine learning, pp. 104, 2004.
Tu, L. and Gimpel, K. Learning approximate inference networks for structured prediction, 2018. ISSN 23318422.
12

---

Tu, L. and Gimpel, K. Benchmarking approximate inference methods for neural structured prediction. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, Volume (Long and Short Papers), pp. 3313-3324, Minneapolis, Minnesota, June 2019, Association for Computational Linguistics. doi: 10. N0118653/1//119111
org N19- 1335
Tu, I.., Pang, R. Y., and Gimpel, K. Improving joint training of inference networks and structured prediction energy networks. In SPNLP, 2020.
Vaswani, A. Attention Is All You Need. (Nips), 2017.
Wu, L., Tian, F., Xia, Y., Fan, Y., Qin, T., Jian-Huang. L.,, and Liu, T.-Y. Learning to Teach with Dynamic Loss Functions. In Advances in Neural Information Processing Systems, vol- 31. Curran Associates, Inc., 2018. URL pestpa://papere www.::iw......eeeeo. pipps.cc/paaerr/218//aaall 8051ascc4666102223e55556666 433-ff66667406ff-o Abstract htmml
Yang. P., Sun, X., Li, W. Ma, S., Wu, W., and Wang, H. SGM: sequence generation model for multi-label classification. Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018, Santa Fe, New Mexico, USA, August 20-26, 2018, pp. 3915-3926, 2018.
Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., and Artzi, Y. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv: 1XXXv:1904.09000 09675, 2019.
Zhang, X., Zhang, Q.- Yan, Z., Liu, R., and Cao, Y. Enhancing label correlation feed- back in multi-label text classification via multi-task learning. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 1190-1200, Online, August 2021. Association for Computational Linguistics. doi: attps://aclanthology org/2021 wttp//22...ininnn/..o. findings-acl ac1.
122.1..iniinnnn-ae...00000 URL
Checklist
1. For all authors.
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [No] Althgouh we do to some degree, for example, limitation of SEAL-dynamic-NCE in image segmentaiton.
(c) Did you discuss any potential negative societal impacts of your work? [N/A] We propose a learning framework which does not have direct negative soceital impacts. (d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)? [Yes] The details are in Appendix.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] The details are in Appendix.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [No]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] In the appendix.
4. If you using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
13

---

(b) Did you mention the license of the assets? [Yes]
(c) Did you include any assets either in the supplemental material URL? [No]
(d) Did you discuss whether and how consent obtained from people whose data you're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsoureing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
14


Reviews:
[{"review_id": "f1UIzvShO3P", "reviewer": "Reviewer_cyaU", "summary": "The authors propose using the structured energy network as a loss to achieve faster and better performance on structured prediction problems. They demonstrate that their approach achieves competitive results, where a dynamic component of the loss can adapt to specific characteristics of the problem at hand.", "questions": "1) The authors measure the performance of their model using F1 scores for classification. While the results are promising and valid, would it be possible to compute the AUC ROC too? This would provide a measurement agnostic to the cut thresholds, providing a better assessment of the classification performance. Furthermore, clarifications should be provided regarding how the cut threshold is computed to obtain the F1 score.\n\n2) The authors report the mean IoU metric for image segmentation. It would make sense to compute some additional metrics (e.g., https://cocodataset.org/#detection-eval) to make it easier to compare the results against other well-known models? Furthermore, would it make sense to execute additional experiments regarding image segmentation on a well-known benchmark dataset (e.g., PASCAL VOC)? \n\n3) In the description provided for Table 1, please consider: \"are marked with underline\" -> \"are underlined\"\n\n4) The authors may be interested in the following research work, at least for the literature review section: Bechtle, Sarah, et al. \"Meta learning via learned loss.\" 2020 25th International Conference on Pattern Recognition (ICPR). IEEE, 2021.\n\n5) In Table 1, the authors report the inference speed, but no information is provided regarding the hardware on which such models were executed for inference. Could such data be provided? We consider hardware specifications to provide context to put execution times in perspective.", "limitations": "We did not find potential negative societal impacts in the present work. The authors did not provide details regarding the limitations of the proposed approach.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The authors present a novel framework where a trainable loss function is used to train a network to perform the inference through a forward pass. The paper is well structured, and the main concepts are clearly conveyed to the reader. The authors created many experiments and considered the ceteris paribus principle to ensure the results could be attributed to a particular change or aspect of their framework, ensuring the quality of research. They demonstrated their approach on various tasks and datasets, obtaining good results. Nevertheless, we miss code to ensure reproducibility or a statement ensuring such code would be made available upon acceptance.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "F-Y6-Odh_R3", "reviewer": "Reviewer_Y7gJ", "summary": "The paper proposes the combined use of structured prediction energy networks (loss-net) as a loss function that considers relations in the output. This loss function is then used to train a feed forward network (task-net) to speed up inference at test time. They proposed both a static and dynamic approach to train these two networks. ", "questions": "\n- The loss function of task-net is a weighted average with parameters $\\lambda_1$ and $\\lambda_2$, the authors fix the latter to 1 and perform bayesian search to determine the best $\\lambda_1$. The variability of this parameter is high and for certain tasks or energy losses it sets to be quite small. For example, Table 7 in Appendix C.2.1 shows that for genbase (the best performing task for the presented model) the influence of loss-net is quite small compared to simple BCE. Nonetheless, the model performs substantially better than CE. Could authors provide an intuition of why this is happening? It seems to be consistently true for margin as an energy loss across tasks and there are other interesting patterns for different tasks. A discussion on this might help in future usability of the proposed approach. \n- Could the author provide confidence intervals for the results?\n- Given the closeness of the results for SEAL-static and SEAL-dynamic which one would the authors recommend for use? Is there any advantage in one above the other based on context? \n- What’s the behavior in training of SEAL-dynamic? I imagine that given the alternating minimization approach training would be highly variable in early stages. Does it require more epochs to be trained due to its alternating minimization nature? Could the author provide details in this regard? I could not find any mention of it either in the main text nor the appendix. \n- Similar to the question above, how is batch size selected? It seems it should be a crucial parameter in stabilizing training and performances but it is not mentioned. \n", "limitations": "See questions. ", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper is well written and easy to follow, it also does a good job in placing itself in the literature and, as the authors state themselves, it is a generalization of previous work [Tu et al, 2020] aimed at reducing inference time for energy networks. Results show remarkable improvement over the state of the art. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "4hURIvEizhP", "reviewer": "Reviewer_iYhN", "summary": "Energy based networks have to run gradient based inference (GBI), which can be quite expensive. In this paper, the authors propose to use an energy based network to develop a structured prediction loss. Thereafter (or simultaneously for the dynamic approach), a feedforward network is learned that tries to match the energy network's output (modeling structure) and a cross-entropy loss w.r.t. ground truth outputs. Combined, the proposed technique gives the inference efficiency of feedforward networks, yet is able to model structure similar to an energy network. \n", "questions": "* Is the InfNet model in Table 1 from Tu et al. (2020)? That is, is the updated and closer to the current method version of the baseline being used for comparison?\n\n* How expensive is SEAL's training? How does it fare in comparison to other methods? I suspect that twice the model size (due to a Feedforward network and an energy based network) plus having to run GBI to get the energy-based network's output will yield a significant training cost. \n\n* Alternating minimization optimization problems can often be sensitive and hard to tune. Did that cause any difficulty for the dynamic model? Regardless, I think this point should be discussed.\n\n\n#### Minor comments:\n\n* The global energy in Equation 2 does not have a dependence on x. Is this correct?", "limitations": "It will be valuable to discuss the limitations of using a feedforward network to \"learn to do GBI\". ", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n\nThe proposed method is quite simple, yet effective. The idea is fairly obvious, but getting the details right is laudable. The paper is decently well-written and well-motivated. \n\n### Weaknesses\n\n* Limitations should be better discussed: The use of a feedforward network to map the input to the GBI output of the energy based network can be thought of as \"learning to do gradient-based inference\". It is unclear to what extent the strategy can be expected to generalize. \n\n* Relation with InfNet: The authors argue InfNet is a special case of their proposed method. While I understand that there is a train-test mismatch for InfNet, the follow-up by Tu et al. (2020) where two separate networks are used for learning an adversarial sampler and for performing inference gets very close to the proposed method. It would be helpful to focus more on the discussion separating these methods. Currently, the discussion makes it seem like the proposed SEAL method is akin to a slight extension. \n\nPost rebuttal: The authors justifiably addressed my concerns and I have accordingly raised my scores.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept