Learning Energy Networks with Generalized Fenchel-Young Losses
Authors: Mathieu Blondel
Felipe Llinares-López
Robert Dadashi
Leonard Hussenot
Matthieu Geist
Venue: NeurIPS 2022 Accept (2022)
Keywords: energy networks
EBMs
structured prediction
convex analysis
Fenchel conjugates

Abstract: Energy-based models, a.k.a. energy networks, perform inference by optimizing 
an energy function, typically parametrized by a neural network. 
This allows one to capture potentially complex relationships between inputs and
outputs.
To learn the parameters of the energy function, the solution to that
optimization problem is typically fed into a loss function.
The key challenge for training energy networks lies in computing loss gradients,
as this typically requires argmin/argmax differentiation.
In this paper, building upon a generalized notion of conjugate function,
which replaces the usual bilinear pairing with a general energy function,
we propose generalized Fenchel-Young losses, a natural loss construction for
learning energy networks. Our losses enjoy many desirable properties and their
gradients can be computed efficiently without argmin/argmax differentiation.
We also prove the calibration of their excess risk in the case of linear-concave
energies. We demonstrate our losses on multilabel classification and 
imitation learning tasks.

Learning Energy Networks with Generalized Fenchel- Young Losses
Mathieu Blondel, Felipe Llinares-López,, Robert Dadashi, Léonard Hussenot, Matthieu Geist Google Research, Brain team {mblondel, ffaleelllllll filiinarss, dadashi, hussenot, mf mnfgeeistt@rooggl.coomm
Abstract
Energy-based models, a.k.a. energy networks, perform inference by optimizing an energy function, typically parametrized by a neural network. This allows to capture potentially complex relationships between inputs and outputs. To learn the parameters of the energy function, the solution to that optimization problem typically fed into loss function. The key challenge for training energy networks lies in computing loss gradients, as this typically requires argmin/argmax differentiation. In this paper, building upon generalized notion of conjugate function, which replaces the usual bilinear pairing with general energy function, we propose generalized Fenchel-Young losses, a natural loss construction for learning energy networks. Our losses enjoy many desirable properties and their gradients can computed efficiently without argmin/argmax differentiation. We also prove the calibration of their excess risk in the case of linear-concave concave energies. We demonstrate our losses on multilabel classification and imitation learning tasks.
Introduction
Training neural network usually involves finding its parameters by minimizing loss function. which captures how well the network fits the data. A typical example is the multiclass logistic loss (a.k.a. cross-entropy loss) from logistic regression, which the canonical loss associated with the softmax output layer and the categorical distribution. If we replace the softmax with another output layer, what loss function should we use instead? generalized linear models [52, 48], which include logistic regression and Poisson regression as special cases, the negative log-likel gives loss function associated with link function, generalizing the softmax other members of the exponential family [8]. More generally, Fenchel- Fenchel-Young losses [18] provide generic way to construct a canonical convex loss function if the associated output layer can written in a certain argmax form. Besides the aforementioned generalized linear models, models that fall in this family include sparsemax [47], the structured perceptron [27] and conditional random fields [43, 69]. However, the theory of Fenchel- Young losses is currently limited to argmax output layers that use bilinear pairing.
To increase expressivity, energy-based models [44], a.k.a. energy networks, perform inference by optimizing an energy function, typically parametrized by neural network. This leads to an inner optimization problem, which can capture potentially complex relationships between inputs and outputs. A similar approach is taken in SPENs (Structured Prediction Energy Networks) [11, 13], which continuous relaxation the inner optimization problem is used, amenable to projected gradient descent or mirror descent. In input-convex neural networks [4], energy functions are restricted to convex, so as to make the inner optimization problem easy to solve optimally. To learn the parameters of the energy function, the solution of the inner optimization problem is typically fed into a loss function, leading to an outer optimization problem. order to solve that problem by stochastic gradient descent, the main challenge lies in computing the loss gradients. Indeed, when
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

using an arbitrary loss function, by the chain rule, computing gradients requires differentiating through the inner optimization problem solution, often referred argmin or argmax differentiation. This can be done by backpropagation through unrolled algorithm iterates [13] or implicit differentiation through the optimality conditions [4]- This issue can be circumvented by using a generalized perceptron loss [44] or max margin loss [11]. These losses only require max differentiation, for which envelope theorems [49] can be used. However, these losses often fail to satisfy envelope theorem assumptions such unicity of the solution and lack theoretical guarantees.
In this paper, we propose to extend the theory of Fenchel- -ounn losses [18], so as to create a canonical loss function associated with an energy network. Our proposal builds upon generalized conjugate functions, also known as Fenchel-Moreau conjugates [50, 62], allowing us to go beyond bilinear pairings and to support general energy functions such as neural networks. We introduce regulariza- tion, allowing us to obtain unicity of the solutions and to use envelope theorems to compute gradients, without argmin or argmax differentiation. We provide novel guarantees on the calibration of the excess risk, when our loss is used as a surrogate for another discrete loss. To sum up, we obtain a well- motivated loss construction for general energy networks. The rest of the paper is organized as follows.
After providing some background ($2), we describe regularized energy networks, identify a classification of energy functions and give several possible examples that fall in this family ($3).
We define generalized conjugates, an extension of Legendre- Fenchel conjugates, and state their properties ($4). We establish novel conditions for a generalized conjugate be a smooth function.
We then introduce generalized Fenchel- Young losses and show that they enjoy many of the same favorable properties as the regular Fenchel- Young losses ($5). On the theoretical side, we prove calibration of the excess risk for linear-concave energies and strongly-convex regularizers ($6).
We demonstrate our losses multilabel classification and imitation learning tasks ($7).
2 Background
Convex conjugates. Let C App = oo for all p & C, i.e. ( = C. Given v E V C RK, we define the convex conjugate [22] of Q2 also known the Legendre-Fenne Fenchel transform of Ω2 by
R be an output set. Let Q:: C + R be function, such that
O (v) = max (v, Kpp). PEC
(1)
The conjugate Ω is always convex, even if Q is not. We define the corresponding argmax
pa(v) == argmax (v,p) - 2(p) = V2"
(2)
PEC
The latter equality follows from Danskin's theorem [29, 15], under the assumption that Q is strictly convex (otherwise, we obtain subgradient). It is well-known that Q is 1-smooth w.r.t. the dual norm || ||- if and only if is y-strongly convex w.r.t. the norm || [37, 39, 10, 75J.
Fenchel- Tenchel-Young losses. Suppose that -g) R are the logits / scores produced by neural network g, where E X C Rd and 0 E θ are the network's input features and parameters, respectively. What loss function should we use we want to use (2) as output layer? The Fenchel- Young loss [18] generated by Ω: C - IR provides a natural solution. It is defined by
(n v) = 52" (v) + (yv) - -w,,))
where y y C C is the ground-truth label. Earlier instances of this loss were independently proposed in the contexts of ranking [2] and multiclass classification [30]. Among many useful properties, this loss satisfies LR(v, y) 0 and (ạ(v, y) = 0 < y (o()) if Ω is strictly convex [18]. In that sense, it is the canonical loss associated with (2). Interestingly, many existing loss functions are recovered as special cases. For instance, if C R and 1(p) 1 [I2. which is 1-strongly convex w.r.t. || l|2 over RR then we obtain the self-dual, the identity mapping and the squared loss:
Ω2 (v) = |1 pa(v)= = La(e, 3) =
2 |lu ylا..
If is the probability simplex A = {p E Rh: Σl_ Pt = 1} and 12(p) is the scaled Shannon negentropy 7(p, log which is y-strongly convex w.n.t. || ll1 over A then we obtain
(v) (s))(v), Po(x) softmax? La(e, v) KL softmax (x)),
2

---

where we used the lug-sum-exp LSE" (v) == >ogg[ exp(us/>)), softmax"(1) x exp(t/7). and the Kullback-Leible divergence, More generally, when C = conv())), the convex hull of }/ pn corresponds to projection, for which efficient algorithms exist in numerous cases [18, 16]. The calibration of the excess risk of Fenchel-Younr Young losses when Q is strongly convex was established in [56, 16]. However, the theory of Fenchel- Fenchel-Young losses is currently limited to the bilinear pairing (u,P), which restricts their expressivity and scope.
3 Regularized energy networks
Energy networks. Also known energy based models or EBMs [44], of which SPENs [11, 13] are a particular case, these networks compute predictions by solving an optimization problem of the form
(u) = argmax ((u,p),
PEC
where = (1)) V is the energy network's input, $(v, p) is scalar-valued energy function, and C is an output set. Throughout this paper, we use the convention that higher energy indicates higher degree of compatibility between and the prediction p. Since ( p) a general energy function, we emphasize that u and p do not need to have the same dimensions, unlike with the bilinear pairing (v. p).
Any neural network (x)) E R can be written in energy network form, since argiaaxEER* -1)
pll = go (z). The key advantage of energy networks, however, is their ability to capture complex in- teractions between inputs and outputs. This is especially useful in the structured prediction setting [6],
where predictions are made of sub-parts, such as sequences being composed of individual elements.
Introducing regularization.
In this paper, we compute predictions by solving
(u) = argmax ( p) - -((p),
(4)
PEC
where we further added regularization function Q: C - R. We call ((v, {pp regularized energy function and we call the corresponding model a regularized energy network. We obviously recover usual energy networks by setting S to the indicator function of the set C, i.e., s(()) = 0 if p E C, co otherwise. While it is principle possible to absorb into Φ, keeping Ω explicit has several advantages: 1) it allows to introduce generalized conjugate functions and their properties ($4) 2) it allows to introduce generalized Fenchel- Young losses ($5), which mirror the original Younhll-Yuug
Fenchel-Youus losses 3) we obtain closed forms for (4) in certain cases (Table 1).
Solving the maximization problem. The ability to solve the maximization problem in (4) efficiently depends on the properties of Φ(U, p) w.r.t. p. If (vv,)) p) is linear in p. for instance ((v, p) = (v, Up). then we obtain pā(v) (v) = Pn [T) v). Thus, the computation of (4) reduces to (2), for which closed forms are often available. If ( p) is concave in p and C is convex set, then we solve (4) in polynomial time using an iterative algorithm, such as projected gradient ascent. This is the most general energy class for which our loss and its gradient can be computed to arbitrary precision. If ((v, p) is nonconcave in p. then it is not possible to solve (4) in polynomial time in general, unless C is discrete set of small cardinality. In general, we emphasize that ((v, p) can be nonconvex in as is typical with neural networks, since the maximization problem in (4) is with respect to E C. However, in the sequel of this paper, certain properties we establish require ( p) to be convex in v.
We now give examples of regularized energy networks increasing order of expressivity complexity.
Generalized linear models. As a warm up. we consider the case of bilinear energy otv. p). For instance, for probabilistic classification with i classes, where the goal is to predict E A from Rd. we can use ( p) (v. p) with case, we therefore obtain @ (v) = po [w x + b). recovering generalized linear models [52, 48], the structured perceptron [27] and conditional random fields [43, 69].
Wr + b, where W E R and b E R$. In this
Rectifier and maxout networks. We now consider the case of convex-linear energy 2 p). A
first example is a rectifier network [33] with one hidden layer. Indeed, Φ(v,p) (o(v), Up) is convex in e if a is an wise,, convex and non-decreasing activation function, such as the relu or softplus, and if U and p are non-negative. A second example is maxout network [34] (i.e., a max-affine function) or its smoothed counterpart, the log-sum-exp network [24]. Indeed, ((v,p) a(v) p, where is the max or log-sum-exp operator, is convex in if the scalar p neg-negative In general, it is always possible to construct a convex-concave energy from a jointly-convex function using the Legendre-Fenchhn transform in the first argument.
3

---

Table 1: Examples of regularized energy networks. The v and columns indicate the property of the energy function ((u, p) in these variables. The L (u, y) column indicates the property of the loss in v. The linear-quadratic energy uses = (A, where A is negative semi-definite and 12(p) 2 2l
((v,p) p)
p
L y)
pr((v)
GLM
(w, p)
linear linear
linear
convex
po(v)
Luncar-quadratic
(p. Ap) + (p, b)
quadratic
convex
(ry - A)-
Rectifier network
(relu(v), Up)
convex
linear
DC
Pa(U (U" reluu)))())))
Maxout network
p- max (w)
convex
linear
DC
Pa (m((mx(
LSE network
p. U.LSE" (v)
convex
linear
DC nonconvex
peo (LSET (o)) no closed form exp(E(e..)) >y'y exp(E(".y no closed form
ICNN
(CNN(E.P)
nonconvex
concave
Prohabilistic
Σ MEy EyeyP(y)F((M ((v,0)
nonconvex
linear
nоnconvex
Arbitrary
(v,,))
nonconvex
nonconcave
nonconvex
Input-convex neural networks. As an example of nonconvex-concawe energy, consider ((r,p) = - (CNN(c,p)) p), where ICNN is an input-convex neural network [4], i.e., is convex in but can nonconvex in v. (p)) and C are convex, then ( p) is concave in p and (4) can be solved in polynomial time using an iterative algorithm, such as projected gradient ascent.
Probabilistic energy networks.
It is often desirable to define conditional probability distribution exp(E((u,)) y))
P(Y y X x) = Eyer exp( (()())))))))
Such networks are typically trained using a cross-entropy loss (or equivalently, negative log- likelihood). Unfortunately, when y is large or infinite, this loss and its gradients are intractable to compute, due to the normalization constant. Therefore, recent research has been devoted to developing approximate training schemes [67, 35, 51]. Although this is not the focus of this paper, we point out that probabilistic energy networks can also be seen as regularized energy networks in the space of probability distributions
A if we set ( p) Evey E(v, y) and Eyey p(y) log p(y). While the resulting optimization problem intractable in general, does suggest possible approximation schemes, such as the use of Frank-Wolfe methods [12, 42, 55].
(p) =
Existing loss functions for energy networks. Given pair (x,, y) and output e = go (x))., how do measure the discrepancy between (v))) and y? One possibility [4, 13] is use the composition of differentiable loss L: C x y R4 with the argmax output, namely (v,y) However, computing the loss gradient w.r.t. u then requires to differentiate through (1). either through unrolling or implicit diffcrentiation. This is particularly problematic when C is complicated convex differentiating through projection can be challenging. In contrast, our proposed loss completely circumvents this need and enjoys casy-to-compute gradients. For unregularized energy networks, naive idea, called the energy loss, is to use (v. y) ( y). However, this loss only works well if Φ is a similarity measure and works poorly in general [44]. A better choice is the generalized perceptron loss (u,v) + maxpec ( p) $(v, v) [44]. Our loss can be seen principled generalization of this loss to regularized energy networks, with theoretical guarantees.
(P)((v), v).
4 Generalized conjugates
In order to devise generalized Fenchel- Fenchel-Young losses, we build upon a generalization due Moreau [50, Chapter 14] of the convex conjugate 9 Denoted Ω it replaces the bilinear pairing in (I) with more general coupling & [62, Chapter 11, Section L]. In this section, we state their definition, properties, closed-form expressions and connection to the C-transform in optimal transport.
Definition. Let Φ(v, p) be a coupling / energy function. The e-convex conjugate of Ω: C R, also known as Fenchel-Moreau conjugate, is then defined by the value function
(v) = max 4(v.p) - 2(p).
(5)
PEC
The &--covvx conjugate an important tool in abstract convex analysis [66, 63]. Recently, it has been used to provide "Bellman-like like"" equations in stochastic dynamic programming [25] and to

---

provide tropical analogues of reproducing kernels [5]. We assume that the maximum is feasible for all E V RM, meaning that dom( V. We emphasize again that, unlike with (1), and do not need to have compatible dimensions. We denote the argmax solution corresponding to (5) by
(u(() = argmax ( p) - -(p).
PEC
If a function F(e) can be written as F(e) = 52 (v) for some 52 is called d-convex (in analogy, function (v) is convex and closed if and only if it can be written as f(w) = -2" (v) for some f?).
Properties d-convex conjugates enjoy many useful properties, some of them are natural extensions of the usual convex conjugate properties. Proofs are provided Appendix B.1. Proposition (Properties of 4-convex conjugates). Let Ω2: - and 4:
1. Generalized Fenchel- Young inequality: for all v E V and p E C.
Ω 0 (v) + Rp)) ((u 2 0.
2. Convexity: If ((e, p) is convex in then (2(v)
(v) is convex (even if (2(p) is nonconvex).
3. Order reversing: if (pp))= S Ap)) for all E C, then 22 (v) 2 A (v) for all v E.
4. Continuity: Q shares the same continuity modulus as 4.
5. Gradient (envelope theorem): Under assumptions (see paragraph below), we have VΩ" (v) V (1,, (())), where V denotes the gradient in the first argument.
6. Smoothness: C is a compact convex set, ((,,)) p) is 3-smooth in (v, (v.p), concave in and k(p) is 7-troongyy convex in p. then 2 (v) is (8+ +82 (////77-smmi -mmooh and p/ is B/7~Lipschhtzz
The condition & and Ω in item for 2 to be smooth function (i.e., with Lipschitz-continuous gradients) is a novel result and will play crucial role for establishing calibration guarantees in $6.
Assumptions for envelope theorems. The expression in item 5 allows to compute V (v) without argmax differentiation. It is based on envelope theorems, which can be used for max differentiation. Indeed, we have Ω 2d (u) maxpee F(v. p), where F(v. p) = o(2, p) S2(p). We assume that the maximum is unique and C is compact. If F F(v. p) is convex in v. we apply Danskin's 's theorem [29] [15, Proposition B. B.25]. Without convexity assumption in if F(U, is continuously differentiable in o for all p E C, V F is continuous, we apply [62, Theorem 10.31]. When we do not compute the exact solution of (6), we only obtain approximation of the gradient VΩ (v); [1] for approximation guarantees. For other envelope theorem usecases in machine learning, see, e.g., [21].
Closed forms. While (5) and (6) may need to be solved numerically in general, they enjoy closed- form expressions simple cases. Proofs are provided in Appendix B.2.
Proposition 2 (Closed-form expressions). Let Ω: C R and 4: V C
1. Bilinear coupling: If (v, p) (v, Up). then (v)(v) = (U T( v) and ph(v) = pu T)).
Kp) = 3 3|p22 and ((v,)) 2 (p, Ap) + (), 2. Linear-quadratic coupling: If C = R²,
where (A, b) and A is such that (4I A) positive definite, we obtain
Ω (v) 2 (b, (7I ( -)- and ()(e) = I AA-
(7)
3. Metric coupling: V = C, Φb = - where Clv,p) is metric and Q is M M-Lipschitz with M < 1, then ②2 Ω2 and p1 v.
Relation with the C-transtorm.
Given cost function C, may define min counterpart of (5), A(p)).
Ac(v) = min
(8)
PEC
In the optimal transport literature, this is known the C-transform of A [65, 60). When C is bilinear, this recovers the notion of concave conjugate [20]. When C(v, = c(v -p for some c, this recovers the infimal convolution [20]. It is easy to check that f = -A Ω -Ac with C' = - -. Thus, Ω and Ag are the natural extensions of convex and concave conjugates, respectively. We opt for the former in this paper to closely mirror the usual convex conjugates and Fenchel- Fenchel-Young losses.
5

---

Generalized Fenchel- Young losses
Definition. The generalized Fenchel- Young inequality in Proposition 1 leads us to propose the generalized Fenchel-Young loss, the natural extension of (3) to Φ-convex conjugates:
Li v) = 22 (v) + (2y) ( y).
(9)
We also have the relationships L2 (z.
(v y) F(e, F(c, (p))) where F(t, R(p) (vv, (U. p).
and pf(()) argmin,ec (v, We now give an intuitive geometric interpretation of (9).
Geometric interpretation. For the sake of illustra- tion, let us set (p) = 3 and ((v,)) zap² + +bp. From (I), the line p + (u,p) (u), where (u) 2, , is the tightest linear lower bound on R(p). here depicted with
a(p)
(M, (1)39 3)
0.6. It is the tangent of R(p) at p Pa(u). Similarly, from (v) the tightest lower- bound of this form given v. It touches st(p) at p (v), here depicted with v = (a, a
P
(5), p + ((v,p)
(u.p) R2 (w), 0(v.p) (x)
and b = 1, for which we have the closed form (²) (((7-))-18) (Proposition 2). The general- ized Fenchel- Young loss (9) is then the gap between Ap)) and ((v,p) - 52 (v), evaluated at the ground- truth The goal of training is to adjust the parameters 0 of network =g0(z) so to minimize this gap averaged over all (xx y) training pairs.
Geometric interpretation with (2(p) 3r2²² and ((v,p) _pp² + bp.
Figure
Properties. Generalized Fenchel- Younh losses enjoy many desirable properties, we now show.
Proposition 3 (Properties of generalized F-Y losses). Let P:: C + IR and 4: V x R.
1. Non-negativity: (; p) 2 0 for all e E V and p E C.
2. Zero loss: the maximum in (5) exists and unique, L v) = + - (v)
3. Gradient: V LK,, = V2 (v) V,,(u,)).
4. Difference of convex (DC): if (v,p) is convex in then L (v. is difference of convex functions in (v) and (v,,)) p). If (vv,)) is linear in then (v, is convex in v.
5. Smaller output set, smaller loss. C' C C and D' is the restriction of to C", then
T. (v,p) (v,p) for all E V and E C².
6. Quadratic lower-bound. If d(u,
2(p) is y-strongly concave in p w.r.t || over C Cadd C is a closed convex set, then for all v E V and p e
2 1p pf(1i)1 S L)(U.)).
(10)
7. Upper-bounds. ((. ppp) a-Lipschitz in p w.r.t. ||. over C, then (2(p))) S allp (x))) ( p) is concave in p. then ( p) S Ln(Vz((u.p..p)
Proofs are given in Appendix B.3. The non-negativity is a good property for a loss function. The zero-loss property [n 3) = 0 + = (v), which is true for instance if p + 4 - 2(p) is strictly concave, is key as it allows to use (v(v) defined in (6) as the (implicit) output layer associated with an energy network. The gradient V V (v, 3) does not require to differentiate through the argmax problem in (6) needed for computing ph (v). Typically, differentiating through an argmax or argmin, as is done in input-convex neural networks [4], requires either unrolling implicit differentiation [36, 14, 41, 19, 17] and is therefore more costly.
The fact that (v, p) is a difference of convex (DC) functions in v when ((c,p) p) is convex in suggests that we can use DC programming techniques, such as convex-concave procedure [72], for training such energy networks. We leave the investigation of this observation to future work. The "smaller output set, smaller loss" property means that we can achieve the smallest loss by choosing the smallest set C in (6) such that y C C. The smallest such convex set is the convex hull of Y, also

---

known as marginal polytope [71] when J S {0, 1 The quadratic lower-bound relates our loss to using Pn (v) within squared norm loss. The upper-bounds relate our loss using Pa (v) within norm loss and to using regular Fenchel-
loss with a linearized energy.
If S2 is the indicator function of C, i.e., st()) 0 if C, otherwise, then it can be checked that we recover the "generalized perceptron" loss [45, 44] as a special case of (9). Proposition 3 therefore provides new properties understand and analyze this loss.
Regular Fenchel-Youur losses are closely related to Bregman divergences [18]. In Appendix A, we build generalized notion of Bregman divergence using generalized conjugates.
Training. To train regularized energy networks with our framework, the user should choose an energy (u,, regularization (t()), output set C' (from Proposition item 5, the smaller this set the better) and the model v = go (z) with input æ and parameters 0. Given a set of input-output
pairs (I1, (1),,,,.... y1),... (In, Vo) E X' x V. where ) C C. we can find the parameters 0 by minimizing the empirical risk objective regularized by R: O + R,
= argmin BE
Σ 2=1
+ R(0).
(11)
Thanks to the easy-to-compute gradients of L. we can easily solve (11) using any (stochastic) solver.
6
Calibration guarantees
Many times, notably for differentiability reasons, the loss used at training time, here our generalized Fenchel- Young loss (", (1, y), is used as surrogate / proxy for a different (potentially discrete loss)
L: y y - R, used at test time. Calibration guarantees [73, 74, 9, 68] ensure that minimizing the excess of risk of the train loss will also minimize that of the test loss (a.k.a. target loss). We study in this section such guarantees, assuming that satisfies an affine decomposition property [26, 16]:
L( v) (1) V Vy((v) + b) + c(v),
(12)
where svy) + Vy((y) + b is an affine map, syy) a label embedding and c(y) is any function that depends only y. Numerous losses can be written in this form. Examples include the zero-one, Hamming, NDCG and precision at losses [56, 16]. Inference in this setting works two steps. First, we compute "soft" (continuous) prediction
pa(e)) C. Second, we compute "hard" (discrete) prediction by a decoding/ rounding from C to V, calibrated for the loss L:
yz(p) argmin L(ū, p) (r)m Vi(()) + b).
(13)
DEy
gey
The target risk of f: X + y and the surrogate risk of g: X + V are defined by
[(f) = E(x,v)~ L(f(X),Y) and ch(g)= = E(x,r)~ L
where p is typically unknown distribution over X x y. The Bayes predictors defined by f == argminy: x-y C(f) and g == argming: X-v (g))). We now establish calibration of the surrogate excess risk, under the assumption that the energy (vv p) is lincar-concave, i.e., can be written ( p) (v. (p())), for some function 4p
Proposition Calibration of target and surrogate excess risks
Assume (V, u, y) is M-sooohh in u w.r.t the dual norm || Il... C is a compact convex set such that y C and (V, p) = (v, s(p)). Let a = supvey ||V p(y)]l. Then, the generalized Fenchel- Young loss (9) is calibrated with the target loss (12) with decoder d = yr ph:
(C(do g) ((
Vg: X +
chilg)
Ch(l")) -h()")))(o"))
802M
where ph: V + is defined in (6) and yL : - y is defined in (13).
The proof is in Appendix B.4. Proposition item 6 shows that the smoothness of + and strong convexity of Q ensure the smoothness of LĀ- Calibration also implics Fisher consistency, namely
7

---

Table 2: Multilabel classification results using various energies (test accuracy in %6..
Energy
yeast
Scene
mediamill
birds
emotions
cal500
Unary (linear)
79.76
89.14
96.84
86.47
78.22
85.67
Unary (rectifier network)
80.03
91.35
96.91
91.74
79.79
86.25
Pairwise
80.19
91.58
96.95
91.55
80.56
85.73
SPEN
79.99
91.24
96.68
91.41
79.35
86.25
Input-coneave SPEN
80.00
90.64
96.95
91.77
79.73
86.35
Ch(0) - Ch(gg) = Lid o g) = C(*)), when using the decoder - yr. o phy The existing proof technique for the calibration of regular Tenchel-Young losses [56, 16] assumes a bilinear pairing and a loss of the form La, (u, (v)) = % (u) + (yx()) (u, soy)), where S2gp is a strongly-convex regularizer w.r.t. (yy). Our novel proof technique more general, as works with any lincar-concave energy and the regularizer Ω is w.r.t. y, not ( Unlike the existing proof, our proof is valid for the pairwise model we present in the next section.
Experiments
7.1 Multilabel classification
We study in this section the application of generalized Fenchel-Yound losses to multi-label classifica- tion, setting y {0, 1 and C = [0, _] where k is the number of labels. When the loss L in (12) the Hamming loss (1 accuracy), our loss is calibrated for and the decoding (13) is just ūJ = 1 if 0.5 else O. We therefore report our empirical results using the accuracy metric.
Pj
Unary model. We consider neural network u (2r) E RRt, assigning score ny to each label j E [k). With the bilinear pairing +b (u,, p) (u, p), we get
Po (u( argmax(u. p) (1p) po()). pE[o.11=
(14)
That is, (14) is just a normal neural network with as output layer. When 2(p) Q1 (11)))
where Q2 (p) = (p. log p) is Shannon's negentropy, we get pn(v) sigmoid(u) == 11 and (9) is just the usual binary logistic/ / cross cross-yntropy loss. When (2(p) l2) + 92 where (2(p) = 1 (p. p) is Gini's negentropy, we get sparse sigmoid and the binary sparsemax loss [18, $6.21. Because = [0, 11*t is the convex hull of y = {0, 13*, (14) can be intepreted as a marginal probability [71]. Indeed, there exists a probability distribution P(YIX) over Y E y such that
[n())); P(V; 1 = .))
Pairwise model. We now additionally use a network = ha(w)) E BRe*k assigning a score U2, to the pairwise interaction between labels i and j. With (u, U3 and the linear-quadratic coupling ( = (u. p) + 1 (p.
(p. Up), we get
(u( = argmax (u.p) (p. Up) 0(p).
(15)
p=[0.1]a
If U is negative semi-definite, i.e., U AAA for some matrix A E Rkxm then the problem is concave in p and can be solved optimally. Moreover, since ( is linear-concave, the calibration guarantees in $6 hold and (v. y/ is convex in In our experiments, we use a matrix A of rank 1 (cf. Appendix C). Unlike (7), (15) does not enjoy a closed form due to the constraints. We solve by coordinate ascent: if Ω is quadratic, as is the case with Gini's negentropy, the coordinate-wise updates can be computed in closed-form. Again, (15) can be interpreted as a marginal probability, In constrast to (15), marginal inference in the closely related Ising model is known to be #P-hard [32].
SPEN model. Following SPENs [11, Eq. 4 and 5], we also tried the energy ((.up) (u, p) T (w.,P) p). where = (u, w), (z)) and are the weights of the "prior network" vi (independent of x). We also tried a variant where made convex in p. making ((e, p) concave in p. In both cases, compute (c) by solving (6) using projected gradient ascent with backtracking linescarch.
8

---

A2t-v2
Ho 1Ch
Hosperiv2 v2
Wa kerkk
220
2003
0000 2000-
402
Usury
302
Parwecd
11
11
11
Number of demonstration trajectorias
Figure 2: Average performance (higher is better) and standard deviation over 10 seeds.
Experimental setup. We perform experiments on 6 publicly-available datasets, see Appendix C. We use the train-test split from the dataset when provided. When not, we use 80% for training data and 20% for test data. For all models, we solve the outer problem (11) using ADAM. We set (2(p) to the Gini negentropy. We hold out 25% of the training data for hyperparameter validation purposes. We R(O) in (11) ž |101| For the regularization hyper-parameter A, search log-spaced values between 10 and 10 For the learning rate parameter of ADAM, we search 10 fog-spaced valued between 10 5 and 10- Once we selected the best hyperparameters, we refit the model on the entire training set. We average results over runs with different seeds.
Results. Table 2 shows model comparison. We observe improvements with the pairwise model on out datasets, confirming that our losses are able to learn useful models. Input-concavity helps improve SPENs in 4 out of 6 datasets. Table 4 confirms that using the envelope theorem for computing gradients works comparably (if not better than) the implicit function theorem. Table shows that our losses outperform the energy, the cross-entropy and the generalized perceptron losses.
7.2 Imitation learning
In this section, we study the application of generalized Fenchel- Young losses to imitation learning. This setting consists in learning policy X' y, mapping from states X' to actions ) from fixed dataset of expert demonstrations (z1, VI...... (In Vn) E X' x V. In particular, we only consider a Behavior Cloning approach [61]. which essentially reduces imitation learning to supervised learning (as opposed to inverse RL methods [64, 54)). The learned policy æ is evaluated on its performance, which is the expected sum of rewards of the environment [70].
Experimental setup. We consider four MuJoCo Gym locomotion environments [23] together with the demonstrations provided by Orsini al. [58]; Appendix C.2 for details. We evaluate the learned policy π for different number of demonstration trajectories: 1, 4 and 11, consistently with [38, 40, 28). The action space in the demonstrations is included in [-1, 1] where the dimensionality of the action space & corresponds to the torques the actuators. We scale the action space to the hypercube = C [0, 1] learning time and scale it back to the original action space inference time. Similarly to the multilabel classification setup, we evaluate the unary and pairwise models, with the only difference that we use two hidden layers instead of one, since it leads significantly better performance, We specify the hyperparameter selection procedure in Appendix C.2.
Results. We run the best hyperparameters over 10 seeds and report final performance over 100 evaluation episodes. Figure 2 shows a clear improvement of the pairwise model over the unary model for 3 out of 4 tasks. Contrary the unary model, the pairwise model enables to capture the interdependence between the different torques of the action space, translating into better performance.
8 Conclusion
Building upon generalized conjugate functions, we proposed generalized Fenchel-Young losses, natural loss construction for learning energy networks and studied its properties. Thanks to conditions on the energy I and the regularizer Ω ensuring the loss smoothness, we established calibration guarantees for the case of linear-concave energies, more general result than the existing analysis, restricted to bilinear energies. We demonstrated the effectiveness of our losses on multilabel classification and imitation learning tasks. We hope that this paper will help popularize generalized conjugates as powerful tool for machine learning and optimization.

---

Acknowledgments
MB thanks Gabriel Peyré for numerous discussions on C-transforms and Clarice Poon for discussions on envelope theorems, well as Vlad Niculae and André Martins for many fruitful discussions.
References
[1] P. Ablin, G. Peyré, and T. Moreau. Super-efficiency of automatic differentiation for functions defined as a minimum. In Proc. of ICML, pages 32-41. PMLR, 2020.
[2] S. Acharyya. Learning to rank in supervised and unsupervised settings using convexity and monotonicity. 2013.
[3] S. Amari Information Geometry and Its Applications. Springer, 2016.
[4] B. Amos, L. Xu, and J. Z. Kolter. Input convex neural networks. In ICML, pages 146-155. PMLR, 2017.
[5] P.-C. preprint arXiv:2202.1144 2022.
and S. Gaubert. The tropical analogues of reproducing kernels. arXiv
[6] G. Baklr, T. Hofmann, B. Schölkopf, A. Smola, and B. Taskar. Predicting structured data. MIT press, 2007.
[7] A. Banerjee, Merugu, I. S. Dhillon, Ghosh, and J. Lafferty. Clustering with bregman divergences. Journal of machine learning research, 6(10), 2005.
[8] O. Barndorff-Nielsen. Information and Exponential Families: In Statistical Theory, John Wiley Sons, 1978.
[9] L. Bartlett, M. Jordan, and J. D. McAuliffe. Convexity, classification, and risk bounds. Journal of the American Statistical Association, 101(473):138-15) 156, 2006.
[10] A. Beck. First-order methods in optimization. SIAM, 2017.
[11] D. Belanger and McCallum. Structured prediction energy networks. International Conference Machine Learning, pages 983-992. PMIR, 2016.
[12] D. Belanger, D. Sheldon, and A. McCallum. Marginal inference in MRFs using Frank- Wolfe. In NeurIPS Workshop on Greedy Opt., FW and Friends, 2013.
[13] D. Belanger, B. Yang, and A. McCallum. End-to-end learning for structured prediction energy networks. International Conference Machine Learning, pages 429-439. PMLR, 2017,
[14] B. M. Bell and J. V. Burke. Algorithmic differentiation of implicit functions and optimal values. In Advances in Automatic Differentiation, pages 67-77. Springer, 2008.
[15] D. P. Bertsekas. Nonlinear programming. Journal of the Operational Research Society, 48(3):334-333 334, 1997.
[16] M. Blondel. Structured prediction with projection oracles. Advances in neural information processing systems, 32, 2019.
[17] M. Blondel, Q. Berthet, M. Cuturi, R. Frostig, S. Hoyer, F. Llinares-López, F. Pedregosa, and J.-P. Vert. Efficient and modular implicit differentiation. arXiv preprini arXiv:2105. 151883 2021.
[18] M. Blondel, A. Martins, and Niculae. Learning with fenchel-young losses. JMLR, 21(35):1-69, 2020.
[19] F. Bonnans and A. Shapiro. Perturbation analysis of optimization problems. Springer Science & Business Media, 2013.
[20] J. Borwein and A. Lewis. Convex Analysis. 2006.
10

---

[21] L. Bottou, M. Arjovsky, D. Lapez-Paz, and M. Oquab. Geometrical insights for implicit generative modeling. pages 229-268. 2018.
[22] S. Boyd, S. P. Boyd, and L. Vandenberghe. Convex optimizatiio. Cambridge university press, 2004.
[23] G. Brockman, V. Cheung, L. Pettersson, Schneider, J. Schulman, Tang, and W. Zaremba. Openai gym, 2016.
[24] G. C. Calafiore, S. Gaubert, and C. Possieri. Log-sum-exp neural networks and posynomial models for convex and log-log-convex data. IEEE transactions on neural networks and learning systems, 3173)8827-838, 2019.
[25] J.-P. Chancelier and M. De Lara. Fenchel-moreau conjugation inequalities with three couplings and application to stochastic bellman equation. arXiv preprint arXiv:1804000044 2018.
[26] C. Ciliberto, L. Rosasco, and A. Rudi. A consistent regularization approach for structured prediction. In Proc. of NeurIPS, pages 4412-4420, 2016.
[27] M. Collins. Discriminative training methods for Hidden Markov Models: Theory and experi- ments with perceptron algorithms. In Proc. of EMNLP, 2002.
[28] R. Dadashi, L. Hussenot, M. Geist, and O. Pietquin. Primal wasserstein imitation learning. International Conference on Learning Representations, 2021.
[29] J. M. Danskin. The theory of max-min and its application to weapons allocation problems. Springer Science & Business Media, 1967.
[30] C. Duchi, K. Khosravi, and Ruan. Multiclass classification, information, divergence, and surrogate risk. The Annals of Statistics, 46(6B):3246-333 2018.
[31] S. Ghadimi and M. Wang. Approximation methods for bilevel programming. arXiv preprint 17X.v:1800.002246, 2018.
[32] A. Globerson, T. Roughgarden, D. Sontag, and C. Yildirim. How hard is inference for structured prediction? In International Conference on Machine Learning, pages 2181-2190. PMLR, 2015.
[33] X. Glorot, A. Bordes, and Y. Bengio. Deep sparse rectifier neural networks. In AISTATS, pages 315-323, JMLR Workshop and Conference Proceedings. 2011.
[34] I. Goodfellow, D. Warde-Farley, M. Mirza, A. Courville, and Y. Bengio. Maxout networks. In ICML, pages 1919-1327. PMLR, 2013.
[35] W. Grathwohl, J. Kelly, M. Hashemi, M. Norouzi, K. Swersky, and D. Duvenaud. No meme for me: Amortized samplers for fast and stable training of energy-based models. 2021.
[36] A. Griewank and A. Walther. Evaluating derivatives: principles and techniques of algorithmic differentiation. SIAM, 2008
[37] J.-B. Hiriart-Urruty and C. Lemaréchal. Convex analysis and minimization algorithms II, volume 305. Springer science & business media, 1993.
[38] J. Ho and S. Ermon. Generative adversarial imitation learning. In Advances in Neural Informa- tion Processing Systems, 2016.
[39] S. Kakade, Shalev-Shwartz, A. Tewari, al. On the duality of strong convexity and strong smoothness: Learning applications and matrix regularization. Tech report, 2(1):35, 2009.
[40] I. Kostrikov, K. K. Agrawal, D. Dwibedi, S. Levine, and J. Tompson. Discriminator-actr--e Addressing sample inefficiency and reward bias in adversarial imitation learning. International Conference Learning Representations, 2019.
[41] G. Krantz and H. R. Parks. The implicit function theorem: history, theory, and applications. Springer Science & Business Media, 2012.

---

[42] R. G. Krishnan, S. Lacoste-Julien, and D. Sontag, Barrier Frank- for marginal inference. In Proc. of NeurIPS, 2015.
[43] D. Lafferty, A. McCallum, and F. C. Pereira. Conditional Random Fields: Probabilistic
models for segmenting and labeling sequence data. In Proc. of ICML, 2001.
[44] Y. LeCun, Chopra, R. Hadsell, M. Ranzato, and Huang. A tutorial on energy-based learning. Predicting structured data, 1(0), 2006.
[45] Y. LeCun and F. J. Huang. Loss functions for discriminative training of energy-based models. In International Workshop on Artificial Intelligence and Statistics, pages 206-213, PMLR, 2005.
[46] T. Lin, C. Jin, and M. Jordan. On gradient descent ascent for nonconvex-coneat minimax problems. In International Conference Machine Learning, pages 6093. PMLR, 2020.
[47] A. Martins and R. F. Astudillo. From softmax to sparsemax: A sparse model of attention and multi-label classification. In Proc. of ICML, 2016.
[48] P. McCullagh and J. A. Nelder. Generalized Linear Models, volume 37. CRC press, 1989.
[49] P. Milgrom and I. Segal. Envelope theorems for arbitrary choice sets, Econometrica, 70(2):883- 601, 2002.
[50] J.-J Moreau. Fonctionnelles convexes. Séminaire Jean Leray, (2):1-108, 1966.
[51] C. Nash and C. Durkan. Autoregressive energy machines. In International Conference Machine Learning, pages 1744-1744. PMLR, 2019.
[52] J. A. Nelder and R. J. Baker. Generalized Linear Models. Wiley Online Library, 1972.
[53] Y. Nesterov. Introductory lectures on convex optimization: A basic course, volume 87. Springer Science Business Media, 2003.
[54] A. Y. Ng. S. J. Russell, et al. Algorithms for inverse reinforcement learning. In International Conference Machine Learning. 2000.
[55] V. Niculae, A. F. Martins, M. Blondel, and C. Cardie. SparseMAP: Differentiable sparse structured inference. In Proc. of ICML, 2018.
[56] A. Nowak-Vile F. Bach, and A. Rudi. A general theory for structured prediction with smooth convex surrogates. arXiv preprint arXiv: 1902.01958, 2019.
[57] Orabona. modern introduction to online learning. arXiv preprint
2019.
[58] M. Orsini, A. Raichuk, L. Hussenot, D. Vincent, R. Dadashi, S. Girgin, M. Geist, O. Bachem. O. Pietquin, and M. Andrychowicz. What matters for adversarial imitation learning? Advances in Neural Information Processing Systems, 2021.
[59] A. Osokin, F. Bach, and S. Lacoste-Julien. On structured prediction theory with calibrated convex surrogate losses. In Proc. of NIPS, pages 302-313, 2017.
[60] G. Peyré, M. Cuturi, et al. Computational optimal transport: With applications to data science. Foundations and Trends@ in Machine Learning, 11(5-6):355667, 2019.
[61] D. A. Pomerleau. Efficient training of artificial neural networks for autonomous navigation. Neural compulation, 1991.
[62] R. T. Rockafellar and R. J.-B. Wets. Variational analysis, volume 317. Springer Science & Business Media, 2009.
[63] A. M. Rubinov. Abstract convexity and global optimization. Springer Science & Business Media, 2000.
[64] S. Russell. Learning agents for uncertain environments. In Conference on Computational learning theory, 1998.
12

---

[65] F. Santambrogio. Optimal transport for applied mathematicians. Birkäuser, NY, 55(58-63):94, 2015.
[66] I. Singer. Abstract convex analysis, volume 25. John Wiley & Sons, 1997.
[67] Y. Song and D. P. Kingma. How to train your energy-based models. arXiv preprint arXiv:2101.03288, 2021.
[68] I. Steinwart. How compare different loss functions and their risks. Constructive Approxima- lion, 26(2):225-277, 2007.
[69] C. Sutton, A. McCallum, et al. An introduction to conditional random fields. Foundations and Trends in Machine Learning, 4(4):267-373, 2012.
[70] R. Sutton and A. G. Barto. Reinforcement learning: An introduction. 2018.
[71] M. J. Wainwright, M. I. Jordan, et al. Graphical models, exponential families, and variational inference. Foundations and Trends@ in Machine Learning. 1(1-2):1-305, 2008.
[72] A. L. Yuille and A. Rangarajan. The concave-convex procedure. Neural computation, 15(4):115- 936, 2003.
[73] T. Zhang. Statistical analysis of some multi-category large margin classification methods. Journal of Machine Learning Research, 5(05t):1225-2 1251, 2004.
[74] Zhang. Statistical behavior and consistency of classification methods based on convex risk minimization. The Annals of Statistics, 32(1):56-85, 2004.
[75] X. Zhou. On the fenchel duality between strong convexity and lipschitz continuous gradient. arXiv preprint arXiv: 1rXiv:1803.05577 06573, 2018.
13


Reviews:
[{"review_id": "yedKwILEYkh", "reviewer": "Reviewer_8vqD", "summary": "The most common approaches to training EBMs are approximate approaches, where the approximation comes from using MCMC to sample from the EBM. This is because training an EBM using the exact gradients is intractable, due to differentiation through argmax/argmin. This paper constructs new loss functions that can be used to tractably train an EBM using exact gradients.\n\nThe authors apply their approach to imitation learning and multilabel classification.\n\nNote: I am not very familiar with theory, so I did not read the math sections closely.\n", "questions": "None at this time", "limitations": "I do not see any potential negative societal impact of this work. ", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The experiments are done with great thoroughness. The details for reproducibility are included, and the experiments are done in a robust manner (e.g. averaging over multiple seeds and properly tuning hyperparameters on validation sets).\n\nThe authors show that they are able to achieve competitive performance on multilabel classification, despite using energy networks that aren’t arbitrarily nonlinear. In other words, they show that the losses arising from the energy networks they use in these experiments are expressive enough to be accurate.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "U2kZSpPghLk", "reviewer": "Reviewer_w577", "summary": "The main contribution of the paper is to introduce and study generalized Fenchel-Young losses, which are losses for regularized energy networks that mirror Fenchel-Young losses, but where the Euclidean inner product is replaced by the energy function \\Phi. They compare their losses with existing losses in the settings of multilevel classification and imitation learning.", "questions": "- There are certain points which could be made clearer and should be expanded: (a) the paragraph in lines 140-148 (“Existing loss functions for energy networks”) is too condensed and hard to parse, it should be expanded, (b) section 6 is also hard to follow, it would be useful to explain at the beginning of the section what calibration is in this setting. On the other hand, sections 4 and 5 are reasonably clear.\n\n- In section 6, Proposition 3, the list of properties is a bit long. Some of the properties are commented on below the proposition, but the attention of the reader gets lost, as it is not apparent which properties are most relevant. One suggestion is to defer to the appendix the properties that do not add much to the explanation.\n\n- Computationally, how do generalized FY losses compare to the existing alternatives? It would be good to compare time.\n", "limitations": "Everything ok.", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Originality: The work can be considered a novel combination of existing techiques (Fenchel-Young losses and \\Phi-conjugates). It is not strikingly original but still valuable. It is clear how the work differs from previous works, which are appropriately cited for the most part.\n\nQuality: The work seems technically sound (I haven’t checked most proofs in the appendix).\n\nClarity: Some parts need to be clearer. See questions and suggestions.\n\nSignificance: The results provided are relevant as energy networks trained with generalized FY losses have better test accuracy in 4 of the 6 multilevel classification settings, and on 3 out of 4 tasks in imitation learning.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "U2JYUW2mcvw", "reviewer": "Reviewer_jRDG", "summary": "The paper introduces a class of losses based on a generalized Fenchel-Young inequality for learning energy models. The idea is to maximize the agreement between  the target $y$ and a feature $v = f(x)$ obtained using a model $f$ applied to an input $x$. The agreement is measured by minimizing an energy function $\\phi(v,y)$ to which a regularization $\\Omega(y)$ is added. Hence, for a given a feature $v$, the corresponding prediction $p(v)$ is given as the maximizer of the objective $\\max_p F(v,p) := \\phi(v,p)-\\Omega(p)$. The model $f$ learned by minimizing the discrepancy between   $F(v ,y)$ and the optimal value $F(v,p(v))$. In other words, minimizing $L(v,y) = F(v,p(v))-F(v ,y)$.\n\nThe paper shows, amongst other properties, that such objectives $L$ can be optimized without the need to differentiate wrt the optimal $p(v)$ and can thus be implemented easily. \n\nSimple experiments show that the approach allows to consider more general losses that can yield improvement on classification and imitation learning tasks.\n", "questions": "- The paragraph in L140-148 is a bit confusing: what do the authors mean by measuring the discrepancy between p_{\\Omega} and $y$ without regularization Omega? Isn’t p_{\\Omega} defined by a choice of a function Omega? \n\n\n- Envelope thm: The authors refer to some assumptions in the appendix under which the envelope theorem holds. These assumptions are distilled in the text of the proof. They should be explicitly stated somewhere (ideal in the main text).\n\n\n\n\n\n\n\n", "limitations": "The proposed approach has a rather limited novelty (both technical and conceptual). However, this can be mitigated if the authors provide more evidence for the significance of such generalization in practice.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strenghts:\nThe paper is clearly written and the results and derivations are sound. The experiments, although basic, show a marginal improvement, especially in the context of imitation learning. \n\nWeaknesses: Originality and Significance\nOriginality: The idea of extending Fenchel-Young loss, which uses an energy function given by a scalar product $ \\phi(v,y)= <v,y>$ is rather direct and straightforward and does not represent a significant technical challenge. For instance, the proofs are often direct consequences of well-known results such as the envelop theorem in proposition 1.\n\nSignificance: Given that most of the experimental results show only a marginal improvement compared to more standard objectives, I am not fully convinced by the significance of the results. Do the authors have situations in mind where one requires such more general losses and where simpler approaches fail? \n\n\n\n \n\n\n\n\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept