Learning Energy Networks with Generalized Fenchel-Young Losses
Abstract
Energy-based models, a.k.a.
energy networks, perform inference by optimizingan energy function, typically parametrized by a neural network.
This allowsone to capture potentially complex relationships between inputs and outputs.
Tolearn the parameters of the energy function, the solution to that optimizationproblem is typically fed into a loss function.
The key challenge for training energynetworks lies in computing loss gradients, as this typically requires argmin/argmaxdifferentiation.
In this paper, building upon a generalized notion of conjugatefunction, which replaces the usual bilinear pairing with a general energy function,we propose generalized Fenchel-Young losses, a natural loss construction forlearning energy networks.
Our losses enjoy many desirable properties and theirgradients can be computed efﬁciently without argmin/argmax differentiation.
Wealso prove the calibration of their excess risk in the case of linear-concave energies.
We demonstrate our losses on multilabel classiﬁcation and imitation learning tasks.
Introduction
Training a neural network usually involves ﬁnding its parameters by minimizing a loss function,which captures how well the network ﬁts the data.
A typical example is the multiclass logistic loss(a.k.a.
cross-entropy loss) from logistic regression, which is the canonical loss associated with thesoftmax output layer and the categorical distribution.
If we replace the softmax with another outputlayer, what loss function should we use instead?
In generalized linear models [49, 46] which includelogistic regression and Poisson regression as special cases, the negative log-likelihood gives a lossfunction associated with a link function, generalizing the softmax to other members of the exponentialfamily [8].
More generally, Fenchel-Young losses [17] provide a generic way to construct a canonicalconvex loss function if the associated output layer can be written in a certain argmax form.
Besidesthe aforementioned generalized linear models, models that fall in this family include sparsemax[45], the structured perceptron [26] and conditional random ﬁelds [41, 66].
However, the theory of
Fenchel-Young losses is currently limited to argmax output layers that use a bilinear pairing.
To increase expressivity, energy-based models [42], a.k.a.
energy networks , perform inferenceby optimizing an energy function , typically parametrized by a neural network.
This leads to aninner optimization problem, which can capture potentially complex relationships between inputs andoutputs.
A similar approach is taken in SPENs (Structured Prediction Energy Networks)
[10, 12],in which a continuous relaxation of the inner optimization problem is used, amenable to projectedgradient descent or mirror descent.
In input-convex neural networks [4], energy functions arerestricted to be convex, so as to make the inner optimization problem easy to solve optimally.
Tolearn the parameters of the energy function, the solution of the inner optimization problem is typicallyfed into a loss function, leading to an outer optimization problem.
In order to solve that problem bystochastic gradient descent, the main challenge lies in computing the loss gradients.
Indeed, whenusing an arbitrary loss function, by the chain rule, computing gradients requires differentiating throughthe inner optimization problem solution, often referred to as argmin or argmax differentiation .
This can be done by backpropagation through unrolled algorithm iterates [12] or implicit differentiationthrough the optimality conditions [4].
This issue can be circumvented by using a generalizedperceptron loss [42] or a max margin loss [10].
However, these losses lack strong motivation andtheoretical guarantees for learning energy networks.
In this paper, we propose to extend the theory of Fenchel-Young losses [17], so as to create a canonicalloss function associated with an energy network.
Our proposal builds upon two main ingredients. First, we use generalized conjugate functions , also known as Fenchel-Moreau conjugates [47, 59],allowing us to go beyond bilinear pairings and to support general energy functions such as neuralnetworks.
Second, we use envelope theorems to obtain easy-to-compute gradients, without argminor argmax differentiation.
We provide novel guarantees on the calibration of the excess risk, whenour loss is used as a surrogate for another discrete loss.
To sum up, we obtain a well-motivated lossconstruction for general energy networks.
The rest of the paper is organized as follows.
• After providing some background (§2), we describe regularized energy networks , identify aclassiﬁcation of energy functions and give several possible examples that fall in this family (§3).• We deﬁne generalized conjugates , an extension of Legendre-Fenchel conjugates, and state theirproperties (§4).
We establish novel conditions for a generalized conjugate to be a smooth function.• We then introduce generalized Fenchel-Young losses and show that they enjoy many of the same
favorable properties as the regular Fenchel-Young losses (§5).
On the theoretical side, we prove
calibration of the excess risk for linear-concave energies and strongly-convex regularizers (§6).
• We demonstrate our losses on multilabel classiﬁcation and imitation learning tasks (§7).
Background
Convex conjugates.
Let C ⊆
R k be an output set.
Let Ω: C → R be a function, such thatΩ( p ) =
∞ for all p (cid:54)∈ C , i.e., dom(Ω) =
C .
Given v ∈ V ⊆ R k , we deﬁne the convex conjugate [20]of Ω , also known as the Legendre-Fenchel transform of Ω , by
[Equation]
The conjugate Ω ∗ is always convex, even if Ω is not.
We deﬁne the corresponding argmax as
[Equation]
The latter equality follows from Danskin’s theorem [28, 14], under the assumption that Ω is strictlyconvex (otherwise, we obtain a subgradient).
It is well-known that Ω ∗ is 1 γ -smooth w.r.t.
the dualnorm (cid:107) · (cid:107) ∗ if and only if Ω is γ -strongly convex w.r.t.
the norm (cid:107) · (cid:107) [35, 37, 9, 70].
Fenchel-Young losses.
Suppose that v = g θ ( x
) ∈ R k are the logits / scores produced by aneural network g , where x ∈ X ⊆ R d and θ ∈ Θ are the network’s input features and parameters,respectively.
What loss function should we use if we want to use (2) as output layer?
The FenchelYoung loss [17] generated by Ω: C → R provides a natural solution.
It is deﬁned by
[Equation]
where y ∈ Y ⊆
C is the ground-truth label.
Earlier instances of this loss were independently proposedin the contexts of ranking [2] and multiclass classiﬁcation [29].
Among many useful properties, thisloss satisﬁes L Ω ( v, y ) ≥ 0 and L Ω ( v, y ) = 0 ⇔ y = p Ω ( v ) if Ω is strictly convex [17].
In that sense,it is the canonical loss associated with (2).
Interestingly, many existing loss functions are recoveredas special cases.
For instance, if C = R k and Ω( p ) = 12 (cid:107) p (cid:107) 22 , which is 1 -strongly convex w.r.t.
(cid:107) · (cid:107) 2over R k , then we obtain the self-dual, the identity mapping and the squared loss:
[Equation]
If C is the probability simplex (cid:52) k := { p ∈ R k
+ : (cid:80) ki
=1 p i = 1 } and Ω( p ) is the scaled Shannonnegentropy γ (cid:104) p, log p (cid:105) , which is γ -strongly convex w.r.t.
(cid:107) · (cid:107) 1 over (cid:52) k , then we obtainΩ ∗ ( v ) =
LSE γ ( v ) , p Ω
[Equation]
where we used the log-sum-exp LSE γ ( v ) :
= γ log( (cid:80) ki
=1 exp( v i /γ )) , softmax γ ( v ) ∝ exp( v/γ ) ,and the Kullback-Leibler divergence.
More generally, when C = conv( Y ) , the convex hull of Y , p Ωcorresponds to a projection, for which efﬁcient algorithms exist in numerous cases [17, 15].
Thecalibration of the excess risk of Fenchel-Young losses when Ω is strongly convex was establishedin [53, 15].
However, the theory of Fenchel-Young losses is currently limited to the bilinear pairing(cid:104) v, p (cid:105) , which restricts their expressivity and scope.
Regularized energy networks
Energy networks.
Also known as energy-based models or EBMs [42], of which SPENs [10, 12] area particular case, these networks compute predictions by solving an optimization problem of the form
[Equation]
where v = g θ ( x ) ∈ V is the energy network’s input, Φ( v, p ) is a scalar-valued energy function, and
C is an output set.
Throughout this paper, we use the convention that higher energy indicates higherdegree of compatibility between v and the prediction p .
Since Φ( v, p ) is a general energy function, weemphasize that v and p do not need to have the same dimensions, unlike with the bilinear pairing (cid:104) v, p (cid:105) .
Any neural network g θ ( x ) ∈ R k can be written in energy network form, since argmax
p ∈ R k −(cid:107)
g θ ( x ) −p (cid:107) 2
= g θ ( x ) .
The key advantage of energy networks, however, is their ability to capture complex interactions between inputs and outputs.
This is especially useful in the structured prediction setting [6],where predictions are made of sub-parts, such as sequences being composed of individual elements.
Introducing regularization.
In this paper, we compute predictions by solving
[Equation]
where we further added a regularization function Ω: C → R .
We call Φ( v, p ) − Ω( p ) a regularizedenergy function and we call the corresponding model a regularized energy network .
We obviouslyrecover usual energy networks by setting Ω to the indicator function of the set C , i.e., Ω( p ) = 0 ifp ∈ C , ∞ otherwise.
While it is in principle possible to absorb Ω into Φ , keeping Ω explicit hasseveral advantages: 1) it allows to introduce generalized conjugate functions and their properties(§4) 2) it allows to introduce generalized Fenchel-Young losses (§5), which mirror the original
Fenchel-Young losses 3) we obtain closed forms for (4) in certain cases (Table 1).
Solving the maximization problem.
The ability to solve the maximization problem in (4) efﬁcientlydepends on the properties of Φ( v, p ) w.r.t.
p .
If Φ( v, p ) is linear in p , for instance Φ( v, p )
= (cid:104) v, Up (cid:105) ,then we obtain p ΦΩ ( v ) =
p Ω ( U (cid:62) v ) .
Thus, the computation of (4) reduces to (2), for which closedforms are often available.
If Φ( v, p ) is concave in p and C is a convex set, then we can solve (4)in polynomial time using an iterative algorithm, such as projected gradient ascent.
If Φ( v, p ) isnonconcave in p , then it is not possible to solve (4) in polynomial time in general, unless C is adiscrete set of small cardinality.
In general, we emphasize that Φ( v, p ) can be nonconvex in v , asis typical with neural networks, since the maximization problem in (4) is with respect to p ∈ C .
However, in the sequel of this paper, certain properties we establish require Φ( v, p ) to be convex in v .
We now give examples of regularized energy networks in increasing order of expressivity / complexity.
Generalized linear models.
As a warm up, we consider the case of bilinear energy Φ( v, p ) .
Forinstance, for probabilistic classiﬁcation with k classes, where the goal is to predict p ∈ (cid:52) k fromx ∈ R d , we can use Φ( v, p ) =
(cid:104) v, p (cid:105) with v =
Wx + b , where W ∈ R k × d and b ∈ R k .
In thiscase, we therefore obtain p ΦΩ ( v ) =
p Ω ( Wx + b ) , recovering generalized linear models [49, 46], thestructured perceptron [26] and conditional random ﬁelds [41, 66].
Rectiﬁer and maxout networks.
We now consider the case of convex-linear energy Φ( v, p ) .
Aﬁrst example is a rectiﬁer network [31] with one hidden layer.
Indeed, Φ( v, p ) =
(cid:104) σ ( v ) , Up (cid:105) isconvex in v if σ is an element-wise, convex and non-decreasing activation function, such as therelu or softplus, and if U and p are non-negative.
A second example is a maxout network [32](i.e., a max-afﬁne function) or its smoothed counterpart, the log-sum-exp network [23].
Indeed,Φ( v, p ) =
σ ( v ) · p , where σ is the max or log-sum-exp operator, is convex in v , if the scalar pis non-negative.
In general, it is always possible to construct a convex-concave energy from ajointly-convex function using the Legendre-Fenchel transform in the ﬁrst argument.
[Table]
Input-convex neural networks.
As an example of nonconvex-concave energy, we considerΦ( v, p ) = − ICNN ( v, p ) , where ICNN is an input-convex neural network [4], i
[Table]
e., it is convex in pbut can be nonconvex in v .
If Ω( p ) and C are convex, then Φ( v, p ) is concave in p and (4) can besolved in polynomial time using an iterative algorithm, such as projected gradient ascent.
Probabilistic energy networks.
It is often desirable to deﬁne a conditional probability distribution
[Equation]
Such networks are typically trained using a cross-entropy loss (or equivalently, negative loglikelihood).
Unfortunately, when Y is large or inﬁnite, this loss and its gradients are intractableto compute, due to the normalization constant.
Therefore, recent research has been devoted todeveloping approximate training schemes [64, 33, 48].
Although this is not the focus of this paper,we point out that probabilistic energy networks can also be seen as regularized energy networksin
the space of probability distributions C = (cid:52) |Y| , if we set Φ( v, p )
=
(cid:80) y ∈Y p ( y ) E ( v, y )
andΩ( p ) =
(cid:80) y ∈Y p ( y ) log p ( y ) .
While the resulting optimization problem is intractable in general, itdoes suggest possible approximation schemes, such as the use of Frank-Wolfe methods [11, 40, 52].
Existing loss functions for energy networks.
Given a pair ( x, y ) and output v = g θ ( x ) , howdo we measure the discrepancy between p ΦΩ ( v ) and y ?
Without regularization Ω , a naive idea,called the energy loss [42], is to use ( v, y )
(cid:55)→ − Φ( v, y ) .
However, this loss is known to workpoorly as it can only push the model in one direction.
A better choice is the generalized perceptronloss ( v, y ) (cid:55)→ max p ∈C Φ( v, p ) − Φ( v, y ) [42]. Another possibility is ( v, y )
(cid:55)→ L ( p ΦΩ ( v ) , y ) , where
L : C × Y → R + [4, 12]. However, computing the loss gradient w.r.t.
v then requires to differentiatethrough p ΦΩ ( v ) , either through unrolling or implicit differentiation.
This is particularly problematicwhen C is a complicated convex set, as differentiating through a projection can be challenging.
Incontrast, our proposed loss completely circumvents this need and enjoys easy-to-compute gradients.
Generalized conjugates
In order to devise generalized Fenchel-Young losses, we build upon a generalization due to Moreau[47, Chapter 14] of the convex conjugate Ω ∗ .
Denoted Ω Φ , it replaces the bilinear pairing in (1)with a more general coupling Φ [59, Chapter 11, Section L].
In this section, we state their deﬁnition,properties, closed-form expressions and connection to the C -transform in optimal transport.
Deﬁnition.
Let Φ( v, p ) ∈
R be a coupling / energy function.
The Φ -convex conjugate of Ω: C →
R , also known as Fenchel-Moreau conjugate , is then deﬁned by the value function
[Equation]
The Φ -convex conjugate is an important tool in abstract convex analysis [63, 60].
Recently, it hasbeen used to provide “Bellman-like” equations in stochastic dynamic programming [24] and to provide tropical analogues of reproducing kernels [5].
We assume that the maximum is feasible forall v ∈ V = R d , meaning that dom(Ω Φ ) =
V .
We emphasize again that, unlike with (1), v and p donot need to have compatible dimensions.
We denote the argmax solution corresponding to (5) by
[Equation]
If a function F ( v ) can be written as F ( v ) =
Ω Φ ( v ) for some Ω , it is called Φ -convex (in analogy, afunction f ( v ) is convex and closed if and only if it can be written as f ( v ) =
Ω ∗ ( v ) for some Ω ).
Properties.
Φ -convex conjugates enjoy many useful properties, some of them are natural extensionsof the usual convex conjugate properties.
Proofs are provided in Appendix B.1.
Proposition 1 (Properties of Φ -convex conjugates) .
Let Ω: C → R and Φ:
V × C → R .
Generalized Fenchel-Young inequality: for all v ∈ V and p ∈ C , Ω Φ ( v ) + Ω( p ) − Φ( v, p ) ≥ 0 .
Convexity: If Φ( v, p ) is convex in v , then Ω Φ ( v ) is convex (even if Ω( p ) is nonconvex).
Order reversing: if Ω( p ) ≤ Λ( p ) for all p ∈ C , then Ω Φ ( v ) ≥ Λ Φ ( v ) for all v ∈ V . 4.
Continuity: Ω Φ shares the same continuity modulus as Φ . 5.
Gradient (envelope theorem): Under mild assumptions (see proof), we have ∇ Ω Φ ( v ) = ∇ 1 Φ( v, p ΦΩ ( v )) , where ∇ 1 denotes the gradient in the ﬁrst argument.
Smoothness:
If C is a compact convex set, Φ( v, p ) is β -smooth in ( v, p ) , concave in p and Ω( p ) is γ -strongly convex in p , then Ω Φ ( v ) is ( β
The condition on Φ and Ω in item 6 for Ω Φ to be a smooth function (i.e., with Lipschitz-continuousgradients) is a novel result and will play a crucial role for establishing calibration guarantees in §6.
Envelope theorems.
The gradient expression in item 5 is based on envelope theorems. It is crucial,as it allows to compute gradients with respect to v without differentiating through p ΦΩ ( v ) .
In practice,this is implemented using the stop_gradient function available in JAX [21].
For Φ( v, p ) convex inv , we use Danskin’s classical theorem [28, 14].
For Φ( v, p ) nonconvex in v , we use the lesser knowntheorem of Rockafellar [59, Theorem 10.31], which requires additional continuity assumptions on Φand unicity of the maximum in (6).
Note that when we do not compute the exact solution of (6), weonly obtain an approximation of the gradient ∇ Ω Φ ( v ) .
Approximations guarantees are studied in [1].
Closed forms.
While (5) and (6) may need to be solved numerically in general, they enjoy closedform expressions in simple cases.
Proofs are provided in Appendix B.2.
Proposition 2 (Closed-form expressions) .
Let Ω: C → R and Φ:
V × C → R .
Bilinear coupling: If Φ( v, p ) = (cid:104) v, Up (cid:105) , then Ω Φ ( v ) =
Ω ∗ ( U (cid:62) v ) and p ΦΩ ( v ) =
p Ω ( U (cid:62) v ) .
Linear-quadratic coupling: If C = R k , Ω( p ) =
γ 2 (cid:107) p (cid:107) 2 2 and Φ( v, p ) =
12 (cid:104) p, Ap (cid:105) + (cid:104) p, b (cid:105) , where v = ( A, b ) and A is such that ( γI − A ) is positive deﬁnite , we obtain
[Equation]
[Equation]
[Equation]
(cid:105) and p
[Equation]
Metric coupling: If V = C , Φ = − C where C ( v, p ) is a metric and Ω is M -Lipschitz with M ≤ 1 , then Ω Φ = − Ω and p ΦΩ ( v )
= v .
Relation with the C -transform.
Given a cost function C , we may deﬁne a min counterpart of (5),Λ C ( v ) :
= min p ∈C
C ( v, p ) − Λ( p ) .
In the optimal transport literature, this is known as the C -transform of Λ [62, 57].
When C is bilinear,this recovers the notion of concave conjugate [19].
When C ( v, p ) =
c ( v − p ) for some c , this recoversthe inﬁmal convolution [19].
It is easy to check that Ω = − Λ ⇔ Ω Φ = − Λ C with C = − Φ .
Thus,Ω Φ and Λ C are the natural extensions of convex and concave conjugates, respectively.
We opt for theformer in this paper to closely mirror the usual convex conjugates and Fenchel-Young losses.
Generalized Fenchel-Young losses
Deﬁnition.
The generalized Fenchel-Young inequality in Proposition 1 leads us to propose thegeneralized Fenchel-Young loss, the natural extension of (3) to Φ -convex conjugates:
[Equation]
We also have the relationships L ΦΩ ( v, y )
=
F ( v, y ) − F ( v, p ΦΩ ( v )) , where F ( v, p ) =
Ω( p ) − Φ( v, p ) ,and p ΦΩ ( v ) = argmin p ∈C L ΦΩ ( v, p ) .
We now give an intuitive geometric interpretation of (9).
[Figure]
Geometric interpretation.
For the sake of illustration, let us set Ω( p ) =
γ 2 p 2 and Φ( v, p ) =
12 ap 2 + bp .
From (1), the line p (cid:55)→ (cid:104) u, p (cid:105) − Ω ∗ ( u ) , whereΩ ∗ ( u ) = 12 γ u 2 , is the tightest linear lower boundon Ω( p ) , here depicted with u = 0 .
It is thetangent of Ω( p ) at p =
p Ω ( u ) .
Similarly, from(5), p (cid:55)→ Φ( v, p ) − Ω Φ ( v ) is the tightest lowerbound of this form given v .
It touches Ω( p ) atp = p Φ Ω ( v ) , here depicted with v = ( a, b ) , a = − 1and b = 1 , for which we have the closed formΩ Φ ( v ) =
12 ( γ − a ) − 1 b 2 (Proposition 2).
The generalized Fenchel-Young loss (9) is then the gap betweenΩ( p ) and Φ( v, p ) − Ω Φ ( v ) , evaluated at the groundtruth p =
y .
The goal of training is to adjust the parameters θ of a network v = g θ ( x ) so as tominimize this gap averaged over all ( x, y ) training pairs.
Properties.
Generalized Fenchel-Young losses enjoy many desirable properties, as we now show.
Proposition 3 (Properties of generalized F-Y losses) .
Let Ω: C → R and Φ: V × C → R .
Non-negativity: L ΦΩ ( v, p ) ≥ 0 for all v ∈ V and p ∈ C . 2.
Zero loss: if the maximum in (5) exists and is unique, L ΦΩ ( v, y ) = 0 ⇔ y =
p ΦΩ ( v ) .
Difference of convex (DC): if Φ( v, p ) is convex in v , then L ΦΩ ( v, p ) is a difference of convex functions in v : Ω Φ ( v ) and Φ( v, p ) .
If Φ( v, p ) is linear in v , then L ΦΩ ( v, p ) is convex in v .
Smaller output set, smaller loss.
If C (cid:48) ⊆ C and Ω (cid:48) is the restriction of Ω to C (cid:48) , then L ΦΩ (cid:48) ( v, p ) ≤ L ΦΩ ( v, p ) for all v ∈ V and p ∈ C (cid:48) .
Quadratic lower-bound.
If Φ( v, p ) − Ω( p ) is γ -strongly concave in p w.r.t.
(cid:107) · (cid:107) over C and C is a closed convex set, then for all v ∈ V and p ∈ C [Equation]
[Equation]
Upper-bounds.
If Φ( v, p ) − Ω( p ) is α -Lipschitz in p w.r.t.
(cid:107) · (cid:107) over C , then L Φ Ω ( v, p ) ≤ α (cid:107) p − p ΦΩ ( v ) (cid:107) .
If Φ( v, p ) is concave in p , then L ΦΩ ( v, p ) ≤
L Ω ( ∇ 2 Φ( v, p ) , p ) . Gradient: ∇ 1 L ΦΩ ( v, p ) = ∇ Ω Φ ( v ) − ∇ 1 Φ( v, p ) .
Proofs are given in Appendix B.3.
The property L ΦΩ ( v, y ) = 0 ⇔ y =
p ΦΩ ( v ) , which is true forinstance if p (cid:55)→ Φ( v, p ) − Ω( p ) is strictly concave, is key as it allows to use p ΦΩ ( v ) deﬁned in (6) asthe (implicit) output layer associated with an energy network.
The gradient ∇ 1 L ΦΩ ( v, y ) does not require to differentiate through the argmax problem in (6) neededfor computing p ΦΩ ( v ) .
Typically, differentiating through an argmax or argmin, as is done in inputconvex neural networks [4], requires either unrolling or implicit differentiation [34, 13, 39, 18, 16]and is therefore more costly.
Our loss construction circumvents this need completely and enjoyseasy-to-compute gradients.
The fact that L ΦΩ ( v, p ) is a difference of convex (DC) functions in v when Φ( v, p ) is convex in vsuggests that we can use DC programming techniques, such as the convex-concave procedure [69],for training such energy networks.
We leave the investigation of this observation to future work.
If Ω is the indicator function of C , i.e., Ω( p ) = 0 if p ∈ C , ∞ otherwise, then it can be checked thatwe recover the “generalized perceptron” loss [43, 42] as a special case of (9).
Proposition 3 thereforeprovides new properties to understand and analyze this loss.
Regular Fenchel-Young losses are closely related to Bregman divergences [17].
In Appendix A, webuild a generalized notion of Bregman divergence using generalized conjugates.
Training.
To train regularized energy networks with our framework, the user should choose anenergy Φ( v, p ) , a regularization Ω( p ) , an output set C (from Proposition 3 item 4, the smaller thisset the better) and the model v = g θ ( x ) with input x and parameters θ .
Given a set of input-outputpairs ( x 1 , y 1 ) , . . .
, ( x n , y n ) ∈ X × Y , where Y ⊆ C ,
we can ﬁnd the parameters θ by minimizingthe empirical risk objective regularized by R : Θ → R ,
[Equation]
Thanks to the easy-to-compute gradients of L ΦΩ , we can easily solve (11) using any (stochastic) solver.
Calibration guarantees
We study in this section calibration guarantees for when L ΦΩ ( v, y ) is used as a surrogate for a (typicallydiscrete) loss L : Y × Y → R .
We assume that L satisﬁes an afﬁne decomposition property [25, 15]:
[Equation]
(12) where ϕ ( y ) (cid:55)→ V ϕ
( y )
+ b is an afﬁne map, ϕ
( y ) is a label embedding and c ( y ) is any function thatdepends only on y .
Numerous losses can be written in this form.
Examples include the zero-one,
Hamming, NDCG and precision at k losses [53, 15].
Inference in this setting works in two steps.
First, we compute a “soft” (continuous) prediction p = p ΦΩ ( v ) ∈ C .
Second, we compute a “hard”(discrete) prediction by a decoding / rounding from C to Y , calibrated for the loss L :
[Equation]
The target risk of f : X → Y and the surrogate risk of g : X → V are deﬁned by
[Equation]
where ρ is a typically unknown distribution over X × Y .
The Bayes predictors are deﬁned byf [Equation]
:= argmin f : X→Y L ( f ) and g (cid:63) :
= argmin g : X→V L ΦΩ ( g ) .
We now establish calibration of thesurrogate excess risk, under the assumption that the energy Φ( v, p ) is linear-concave , i.e., it can bewritten as Φ( v, p )
=
(cid:104) v, ϕ ( p ) (cid:105) , for some function ϕ .
Proposition 4.
Calibration of target and surrogate excess risks
Assume L Φ Ω ( v, y ) is M -smooth in v w.r.t.
the dual norm (cid:107) · (cid:107) ∗ , C is a compact convex set such that Y ⊆ C and Φ( v, p ) =
(cid:104) v, ϕ ( p ) (cid:105) .
Let σ := sup y ∈Y (cid:107) V (cid:62) ϕ
( y ) (cid:107) ∗ .
Then, the generalized
Fenchel-Young loss (9) is calibrated with the target loss (12) with decoder d = y L ◦ p ΦΩ :
[Equation]
where p ΦΩ : V → C is deﬁned in (6) and y L : C → Y is deﬁned in (13) .
The proof is in Appendix B.4.
Proposition 1 item 6 shows that the smoothness of Φ and strongconvexity of Ω ensure the smoothness of L ΦΩ .
Calibration also implies Fisher consistency, namely
L Φ Ω ( g ) =
L Φ Ω ( g (cid:63) )
⇒ L ( d ◦ g )
=
L ( f (cid:63) ) , when using the decoder d = y L ◦ p Φ Ω .
The existing prooftechnique for the calibration of regular Fenchel-Young losses [53, 15] assumes a bilinear pairing anda loss of the form L Ω ϕ ( u, ϕ ( y ))
=
Ω ∗ ϕ ( u ) + Ω ϕ ( ϕ ( y )) − (cid:104) u, ϕ ( y ) (cid:105) , where Ω ϕ is a strongly-convexregularizer w.r.t.
ϕ ( y ) .
Our novel proof technique is more general, as it works with any linear-concaveenergy and the regularizer Ω is w.r.t.
y , not ϕ ( y ) .
Unlike the existing proof, our proof is valid for thepairwise model we present in the next section.
[Table]
Experiments7.1 Multilabel classiﬁcation
We study in this section the application of generalized Fenchel-Young losses to multi-label classiﬁcation, setting Y = { 0 , 1 } k and C = [0 , 1] k , where k is the number of labels.
When the loss L in (12)is the Hamming loss (1 - accuracy), our loss is calibrated for L and the decoding (13) is just (cid:98)
y j = 1 ifp j > 0 .
5 else 0 .
We therefore report our empirical results using the accuracy metric.
Unary model.
We consider a neural network u
= g θ ( x ) ∈ R k , assigning a score
u j to each labelj ∈ [ k ] .
With the bilinear pairing Φ( u, p ) = (cid:104) u, p (cid:105) , we getp ΦΩ ( u ) =
argmax p ∈
[0 , 1] k (cid:104) u, p (cid:105) − Ω( p ) =
p Ω ( u ) .
That is, (14) is just a normal neural network with p Ω as output layer.
When Ω( p ) =
Ω 1 ( p )+Ω 1 (1 − p ) ,where Ω 1 ( p ) :
= (cid:104) p, log p (cid:105) is Shannon
’s negentropy, we get p Ω ( v )
=
sigmoid ( v ) :
= 1 / (1+exp( − v ))and (9) is just the usual binary logistic / cross-entropy loss.
When Ω( p ) =
Ω 2 ( p ) + Ω 2 (1 − p ) , whereΩ 2 ( p ) :
= 12 (cid:104) p, 1 − p (cid:105) is Gini’s negentropy, we get a sparse sigmoid and the binary sparsemax loss [17,§6.2].
Because C = [0 , 1] k is the convex hull of Y = { 0 , 1 } k , (14) can be intepreted as a marginalprobability [68].
Indeed, there exists a probability distribution P ( Y | X ) over Y ∈ Y such that
[Equation]
Pairwise model.
We now additionally use a network U
=
h θ ( x ) ∈ R k × k , assigning a score
U i,j tothe pairwise interaction between labels i and j .
With v = ( u, U ) and the linear-quadratic couplingΦ( v, p ) =
(cid:104) u, p (cid:105) + 12 (cid:104) p, Up (cid:105) , we get
[Equation]
If U is negative semi-deﬁnite, i.e., U = − AA (cid:62) for some matrix A ∈ R k × m , then the problem isconcave in p and can be solved optimally.
Moreover, since Φ( v, p ) is linear-concave, the calibrationguarantees in §6 hold and L ΦΩ ( v, y ) is convex in v .
In our experiments, we use a matrix A of rank 1(cf.
Appendix C).
Unlike (7), (15) does not enjoy a closed form due to the constraints.
We solve itby coordinate ascent: if Ω is quadratic, as is the case with Gini’s negentropy, the coordinate-wiseupdates can be computed in closed-form.
Again, (15) can be interpreted as a marginal probability.
Inconstrast to (15), marginal inference in the closely related Ising model is known to be #P-hard [30].
SPEN model.
Following SPENs [10, Eq. 4 and 5], we also tried the energy Φ( v, p ) =
(cid:104) u, p (cid:105) −Ψ( w, p ) , where v = ( u, w ) , u
= g θ ( x ) and w are the weights of the “prior network” Ψ (independentof x ).
We also tried a variant where Ψ is made convex in p , making Φ( v, p ) concave in p .
In bothcases, we compute p ΦΩ ( v ) by solving (6) using projected gradient ascent with backtracking linesearch.
Experimental setup.
We perform experiments on 6 publicly-available datasets, see Appendix C.
For all datasets, we normalize samples to have zero mean unit variance.
We use the train-test splitfrom the dataset when provided.
When not, we use 80% for training data and 20% for test data.
Forall models, we solve the outer problem (11) using ADAM.
We set Ω( p ) to the Gini negentropy.
Wehold out 25% of the training data for hyperparameter validation purposes.
We set R ( θ ) in (11) to
[Figure]
[Figure] [Figure] [Figure] λ 2 (cid:107) θ (cid:107) 22 .
For the regularization hyper-parameter λ , we search 5 log-spaced values between 10 − 4 and10 1 .
For the learning rate parameter of ADAM, we search 10 log-spaced valued between 10 − 5 and10 − 1 .
Once we selected the best hyperparameters, we reﬁt the model on the entire training set.
Weaverage results over 3 runs with different seeds.
Results.
Table 2 shows a model comparison.
We observe slight improvements with the pairwisemodel on 4 out of 6 datasets, conﬁrming that generalized FY losses are able to learn useful models.
Input-concavity helps improve SPENs in 4 out of 6 datasets.
Table 4 conﬁrms that using the envelopetheorem for computing gradients works comparably to (if not better than) the implicit functiontheorem.
Table 5 shows that the generalized Fenchel-Young loss outperforms the energy loss, thebinary cross-entropy loss and the generalized perceptron loss.
7.2 Imitation learning
In this section, we study the application of generalized Fenchel-Young losses to imitation learning.
This setting consists in learning a policy π : X (cid:55)→ Y , a mapping from states X to actions Y ,from a ﬁxed dataset of expert demonstrations ( x 1 , y 1 ) , . . .
, ( x n , y n ) ∈
X × Y .
In particular, weonly consider a Behavior Cloning approach [58], which essentially reduces imitation learning tosupervised learning (as opposed to inverse RL methods [61, 51]).
The learned policy π is evaluatedon its performance, which is the expected sum of rewards of the environment [67].
Experimental setup.
We consider four MuJoCo
Gym locomotion environments [22] togetherwith the demonstrations provided by Orsini et al.
[55]; see Appendix C.2 for environment anddemonstration details.
We evaluate the learned policy π for different number of demonstrationtrajectories: 1, 4 and 11, consistently with [36, 38, 27].
The action space in the demonstrations isincluded in [ − 1 , 1] k , where the dimensionality of the action space k corresponds to the torques of theactuators.
We scale the action space to the hypercube Y = C
= [0 , 1] k at learning time and scale itback to the original action space at inference time.
Similarly to the multilabel classiﬁcation setup,we evaluate the unary and pairwise models, with the only difference that we use two hidden layersinstead of one, since it leads to signiﬁcantly better performance.
We specify the hyperparameterselection procedure in Appendix C.2.
Results.
We run the best hyperparameters over 10 seeds and report ﬁnal performance over 100evaluation episodes.
Figure 2 shows a clear improvement of the pairwise model over the unarymodel for 3 out of 4 tasks.
Contrary to the unary model, the pairwise model enables to capture theinterdependence between the different torques of the action space, translating into better performance.
Conclusion
Building upon generalized conjugate functions, we proposed generalized Fenchel-Young losses,a natural loss construction for learning energy networks and studied its properties.
Thanks toconditions on the energy Φ and the regularizer Ω ensuring the loss smoothness, we establishedcalibration guarantees for the case of linear-concave energies, a more general result than the existinganalysis, restricted to bilinear energies.
We demonstrated the effectiveness of our losses on multilabelclassiﬁcation and imitation learning tasks.
We hope that this paper will help popularize generalizedconjugates as a powerful tool for machine learning and optimization. Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work? [Yes] (c) Did you discuss any potential negative societal impacts of your work? [N/A](d)
Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] for data and
instructions, code will be released upon publication
(b) Did you specify all the training details (e
g., data splits, hyperparameters, how theywere chosen)?
[Yes]
(c) Did you report error bars (e
g., with respect to the random seed after running experiments multiple times)?
[Yes]
(d) Did you include the total amount of compute and the type of resources used (e
g., typeof GPUs, internal cluster, or cloud provider)?
[Yes]
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]
A Generalized Bregman divergences
The expression Ω ∗ ( v ) + Ω( p ) − (cid:104) v, p (cid:105) at the heart of Fenchel-Young losses is closely related to
Bregman divergences; see [3, Theorem 1.1] and [17].
In this section, we develop a similar relationshipbetween generalized Fenchel-Young losses and a new generalized notion of Bregman divergence.
Biconjugates.
We begin by recalling well-known results on biconjugates [20].
Applying theconjugate (1) twice, we obtain the biconjugate Ω ∗∗ ( p ) .
It is well-known that a function Ω is convexand closed (i.e., lower-semicontinuous)
if and only if Ω = Ω ∗∗ .
This therefore provides a variationalcharacterization of lower-semicontinuous convex functions.
This characterization naturally motivatesthe class of Φ -convex functions (§4).
If Ω is nonconvex, Ω ∗∗ is Ω ’s tightest convex lower bound.
Generalized biconjugates.
The generalized conjugate in (5) uses maximization w.r.t.
the secondargument p ∈ C .
To obtain a generalized conjugate whose maximization is w.r.t.
the ﬁrst argumentv ∈ V instead, we deﬁne Ψ( p, v ) :
=
Φ( v, p ) .
Note that if Φ is symmetric, the distinction between Φand Ψ is not necessary.
Similarly to (5), we can then deﬁne the Ψ -conjugate of Λ: V → R as
[Equation]
i.e., the maximization is over the left argument of Φ .
We deﬁne the corresponding argmax as
[Equation]
−
[Equation]
In particular, with Λ := Ω Φ , we can deﬁne the generalized biconjugate Λ Ψ = Ω ΦΨ : C → R .
Generalized biconjugates enjoy similar properties as regular biconjugates, as we now show.
Proposition 5 (Properties of generalized biconjugates) .
Let Ω: C → R , Φ:
V × C
→ R and Ψ( p, v ) :
=
Φ( v, p ) .
Lower-bound: Ω ΦΨ ( p ) ≤ Ω( p ) for all p ∈ C . 2.
Equality: Ω ΦΨ ( p ) =
Ω( p ) if and only if Ω is Ψ -convex.
Tightest lower-bound: Ω ΦΨ ( p ) is the tightest Ψ -convex lower-bound of Ω( p ) .
Proofs are given in Appendix B.5.
Similar results hold for Λ ΨΦ , where Λ: V → R .
Deﬁnition.
We can now deﬁne the generalized Bregman divergence D ΦΩ : C × C → R + as
[Equation]
(18) where we recall that Ψ( p, v ) :
= Φ( v, p ) and using (17) we have
[Equation]
We have therefore obtained a notion of Bregman divergence parametrized by a coupling Φ( v, p ) , suchas a neural network.
As shown in the proposition below, generalized Bregman divergences enjoy similar properties asregular Bregman divergences.
Proofs are given in Appendix B.6.
Proposition 6 (Properties of generalized Bregman divergences) .
Let Ω be a Ψ -convex function, where Ψ( p, v ) :
=
Φ( v, p ) .
Link with generalized FY loss: Denoting v := v ΨΩ Φ ( p (cid:48) ) , we have D ΦΩ ( p, p (cid:48) )
=
L ΦΩ ( v, p ) .
Non-negativity: D ΦΩ ( p, p (cid:48) ) ≥ 0 , for all p, p (cid:48) ∈ C .
Identity of indiscernibles: D ΦΩ ( p, p (cid:48) )
= 0 ⇔ p = p (cid:48) if p (cid:55)→ Φ( v, p ) − Ω( p ) is strictly concave for all v ∈ V . 4.
Convexity: if p (cid:55)→ Φ( v, p ) − Ω( p ) is concave for all v ∈ V , then D ΦΩ ( p, p (cid:48) ) is convex in p .
Recovering Bregman divergences: If Φ( v, p ) is the bilinear coupling (cid:104) v, p (cid:105) , then we recover the usual Bregman divergence D Ω : C × C
→ R +
[Equation]
Some remarks:
• The generalized Bregman divergence is between objects p and p (cid:48) of the same space C , while thegeneralized Fenchel-Young loss is between objects v and p of mixed spaces V and C .
• If Φ( v, p ) − Ω( p ) is concave in p , then D Φ Ω ( p, p (cid:48) ) is convex in p , as is the case of the usualBregman divergence D Ω ( p, p (cid:48) ) .
However, (19) is not easy to solve globally in general, as it is themaximum of a difference of convex functions in v .
This can be done approximately by using theconvex-concave procedure [69], linearizing the left part.
We have the opposite situation with thegeneralized Fenchel-Young loss: if Φ( v, p ) − Ω( p ) is convex-concave, L ΦΩ ( v, y ) is easy to computebut it is a difference of convex functions in v .•
Similarly, we may also deﬁne D [Equation] Φ
[Equation]
where we recall that p ΦΩ ( v ) :
= argmax p ∈C Φ( v, p ) − Ω( p ) .
We have thus obtained a divergencebetween two objects v and v (cid:48) in the same space V .
B Proofs
B.1 Proofs for Proposition 1 (properties of generalized conjugates)
Generalized Fenchel-Young inequality.
From (5), we immediately obtain Ω Φ ( v ) ≥ Φ( v, p ) −Ω( p ) for all v ∈ V and all p ∈ C .2.
Convexity.
If Φ( v, p ) is convex in v , then Ω Φ ( v ) is the maximum of a family of convex functionsindexed by p .
Therefore, Ω Φ ( v ) is convex.
Note that this is the case even if Ω( p ) is nonconvex.
Order reversing.
Since Ω ≤ Λ , we have
[Equation]
Continuity.
See [62, Box 1.8].5.
Gradient.
We have Ω Φ ( v ) = max p ∈C F ( v, p ) , where F ( v, p ) : = Φ( v, p ) − Ω( p ) .
If F ( v, p ) is convex in v (but not necessarily differentiable in v ), this is Danskin’s theorem [28],
see also [14, Proposition B.25].
Without convexity assumption in v , if F ( v, p ) is continuously differentiable in v for all p ∈ C , F
and ∇ 1 F are continuous on V × C , p ΦΩ ( v ) =
argmax p ∈C F ( v, p ) is single-valued, C is compact
and V is an open set, then this follows from [59, Theorem 10.31].
See also [59, Theorem 10.58]
for a minimum counterpart.
Smoothness.
We follow the proof technique of [44, Lemma 4.3], which states that v (cid:55)→ max p ∈C E ( v, p ) is ( β + β 2 /γ )
-smooth if E is β -smooth in ( v, p ) and γ -strongly concave in p over C .
We show here that if E ( v, p ) decomposes as E ( v, p ) =
Φ( v, p ) − Ω( p ) , where Ω is γ -strongly convex over C but possibly nonsmooth (as is the case for instance of Shan- non’s negentropy), and Φ( v, p ) is β -smooth in ( v, p ) and concave in p , then we still have that
Ω Φ ( v ) = max p ∈C Φ( v, p ) − Ω( p ) is ( β + β 2 /γ )
-smooth.
For brevity, let us deﬁne the shorthand p ΦΩ ( v ) :
= p (cid:63) ( v ) .
From [44, Section A.3], we have
[Equation]
[Equation]
[Equation] [Equation]
[Equation]
[Equation]
[Equation] [Table]
for all v 1 , v 2 ∈ V .
With E ( v, p ) =
Φ( v, p ) − Ω( p ) , we obtainγ
[Equation]
Since Φ is β -smooth, we have for all p ∈ C
[Equation]
Combined with the Holder inequality (cid:104) a, b (cid:105) ≤ (cid:107) a (cid:107) ∗ (cid:107) b (cid:107) , this gives
[Equation]
Simplifying, we get
[Equation]
Therefore, p ΦΩ ( v ) = p (cid:63) ( v ) is β/γ -Lipschitz.
From Rockafellar’s envelope theorem [59, Theorem 10.31], ∇ Ω Φ ( v ) = ∇ 1 Φ( v, p (cid:63) ( v )) .
Since Φis β -smooth, we therefore have
[Equation]
where we used (21) and βγ ≥ 1 .
B.2 Proofs for Proposition 2 (closed forms)
Bilinear coupling.
This follows from
[Equation]
and similarly for p Ω ( v ) .2.
Linear-quadratic coupling.
Let us deﬁne the function
[Equation]
Its gradient is ∇ F ( p ) =
( γI − A ) p − b .
Setting ∇ F ( p (cid:63) )
= 0 , we obtain( γI − A
[Equation]
where we assumed that ( γI − A ) is positive deﬁnite.
We therefore get
[Equation]
Metric coupling.
From (8), it is easy to check that Ω = − Λ ⇔ Ω
Φ = − Λ C with C = − Φ .
From[57, Proposition 6.1], we have Λ C = − Λ .
Therefore, Ω Φ = − Ω .
B.3 Proofs for Proposition 3 (Properties of generalized Fenchel-Young losses)
Non-negativity.
This follows immediately from the generalized Fenchel- Young
inequality
Zero loss.
If p ΦΩ ( v ) =
y , then using L ΦΩ ( v, y )
=
F ( v, y ) − F ( v, p ΦΩ ( v )) , where F ( v, p ) =
Ω( p ) − Φ( v, p ) , we obtain L ΦΩ ( v, y )
= 0 .
Let’s now prove the reverse direction.
Since the maximumin (5) exists, we have F ( v, p ΦΩ ( v )) ≤ F ( v, y ) .
If L ΦΩ ( v, y ) = 0 , we have F ( v, p ΦΩ ( v ))
=
F ( v, y ) .
Since the maximum is unique by assumption, this proves that y = p ΦΩ ( v ) .
Difference of convex (DC).
If Φ( v, p ) is convex in p , Ω Φ ( v ) is convex (Proposition 1).
Therefore,v (cid:55)→ Ω Φ ( v ) − Φ( v, p ) for all p ∈ C .4.
Smaller output set, smaller loss.
Let Ω: C → R and let Ω (cid:48) be the restriction of Ω to C (cid:48) ⊆
C , i.e.,Ω (cid:48) ( p ) :
= Ω + I C (cid:48) , where I C (cid:48) is the indicator function of C (cid:48) .
From (5), we have (Ω (cid:48) )
Φ ( v ) ≤ Ω Φ ( v )for all v ∈ V .
From (9), we therefore have L ΦΩ (cid:48) ( v, p ) ≤ L ΦΩ ( v, p ) for all v ∈ V and all p ∈ C (cid:48) .5.
Quadratic lower-bound.
If a function F is γ -strongly convex over C w.r.t.
a norm (cid:107) · (cid:107) , then
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
(22) for all p, p (cid:48) ∈ C .
If C is a closed convex set, we also have that p (cid:63) =
argmin p ∈C F ( p ) satisﬁes theoptimality condition
[Equation]
F ( p (cid:63) ) , p − p (cid:63) (cid:105) ≥(23) for all p ∈ C [50, Eq.
Combining (22) with p (cid:48)
= p (cid:63) and (23), we obtain
[Equation]
Applying the above with F ( p )
=
Ω( p ) − Φ( v, p ) and using p (cid:63) =
p ΦΩ ( v ) , we obtain
[Equation]
Upper
to (cid:107) · (cid:107) then (cid:48) C
[Equation]
Applying the above with F ( p )
=
Ω( p ) − Φ( v, p ) and p (cid:48) =
p ΦΩ ( v ) , we obtain
[Equation]
If Φ( v, p ) is concave in p , then its linear approximation always lies above:Φ [Equation] p [Equation]
≤
[Equation]
p )
[Equation]
2 Φ
[Equation]
v, p ) , p
[Equation]
p
[Equation]
, for all p, p (cid:48) ∈ C .
We then have
[Equation]
Gradient.
This follows directly from the deﬁnition (9)
B.4 Proof of Proposition 4 (calibration)
Background.
The pointwise target risk of (cid:98) y ∈ Y according to q ∈ (cid:52) |Y| is
[Equation]
We also deﬁne the corresponding excess of pointwise risk, the difference between the pointwise riskand the pointwise Bayes risk:
[Equation]
We can then write the risk of f : X → Y in terms of the pointwise risk
[Equation]
Let us deﬁne the Bayes predictorf
[Equation]
argmin
[Equation]
L ( f ) .
The Bayes risk is then
[Equation]
Combining the above, we can write the excess of risk of f : X → Y as
[Equation]
Similarly, with the generalized Fenchel-Young loss (9), the pointwise surrogate risk of v ∈ Vaccording to q ∈ (cid:52) |Y| is
[Equation]
and the excess of pointwise surrogate risk is
[Equation]
Let us deﬁne the surrogate risk of g : X → V by
[Equation]
and the corresponding Bayes predictor by
[Equation]
We can then write the excess of surrogate risk of g : X → V as
[Equation]
A calibration function [65] ξ : R + → R + is a function relating the excess of pointwise target risk andpointwise surrogate risk.
It should be non-negative, convex and non-decreasing on R + , and satisfyξ (0) = 0 .
Formally, given a decoder d : V → Y , ξ should satisfy
[Equation]
for all v ∈ V and q ∈ (cid:52) |Y| .
By Jensen’s inequality, this implies that the target and surrogate risks arecalibrated for all g : X → V [56, 53]
[Equation]
From now on, we can therefore focus on proving (24).
Upper-bound on the pointwise target excess risk.
We now make use of the afﬁne decomposition(12).
Let σ := sup y ∈Y (cid:107) V (cid:62) ϕ
( y ) (cid:107) ∗ , where (cid:107) · (cid:107) ∗ denotes the dual norm of (cid:107) · (cid:107) .
Let us deﬁne˜ y L ( u ) := argmin (cid:98) y ∈Y (cid:104) ϕ
( (cid:98) y ) , V u + b (cid:105) and µ ϕ ( q ) :
=
E Y ∼ q [ ϕ ( Y )]
∈ conv( ϕ ( Y )) ⊆
ϕ ( R k ) .
From [15, Lemma 2],δ(cid:96)
(˜ y L ( u ) , q ) ≤ 2 σ (cid:107) µ ϕ ( q ) − u
(cid:107) ∀ u ∈ ϕ
( R k ) , q ∈ (cid:52) |Y| .
Using y L ( p ) =
˜
y L ( u ) with u = ϕ ( p ) , we thus getδ(cid:96)
[Equation]
Bound on the pointwise surrogate excess risk (bilinear case).
To highlight the difference withour proof technique, we ﬁrst prove the result in the bilinear case, assuming Ω is γ -strongly convex.
We follow the same proof technique as [53, 15] but unlike these works we do not require any
Legendre-type assumption on Ω .
If Φ( v, p ) =
(cid:104) v, ϕ ( p ) (cid:105) =
(cid:104) v, p (cid:105) , we have
[Equation]
where, when ϕ ( y ) =
y , we denote µ ( q ) :=
E Y ∼ q
[ Y ] ∈ conv( Y ) ⊆
R k for short.
The quantity
E Y ∼ q Ω( Y ) − Ω( µ ( q )) is called Bregman information [7] or Jensen gap, and is non-negative if Ω isconvex.
This term cancels out in the excess of pointwise surrogate risk
[Equation]
Since the Fenchel-Young loss achieves its minimum at 0 , we have
[Equation]
Therefore, the excess of pointwise surrogate risk can be written in Fenchel-Young loss form.
By thequadratic lower-bound (10) and the upper-bound (25), we have
[Equation]
Therefore the calibration function with the decoder d = y L ◦ p Ω is
[Equation]
Bound on the pointwise surrogate excess risk (linear-concave case).
We now prove the boundassuming that L ΦΩ is smooth and Φ( v, p ) =
(cid:104) v, ϕ ( p ) (cid:105) .
This includes the previous proof as specialcase because when Φ( v, p ) = (cid:104) v, p (cid:105) , then L ΦΩ ( v, y ) =
L Ω ( v, y ) is 1 γ -smooth in v if and only if Ω( p )is γ -strongly convex in p (cf.
By Theorem 4.22 in [54], if a function f ( v ) is M -smooth in v w.r.t.
the dual norm (cid:107) · (cid:107) ∗ and isbounded below, then
[Equation]
With f ( v ) =
(cid:96) ΦΩ ( v, q ) =
E Y ∼ q L Ω ( v, Y ) , which is non-negative, we obtain
[Equation]
We therefore have
[Equation]
Therefore the calibration function with the decoder d = y L ◦ p ΦΩ is
[Equation]
B.5 Proofs for Proposition 5 (Properties of generalized biconjugates)
The proofs are similar to that for C -transforms [62, Proposition 1.34].1.
Lower bound.
Let Ω: C → R .
We have
[Equation]
Therefore, Ω ΦΨ ( p ) ≤ Ω( p ) for all p ∈ C .
Analogously, for a function Λ: V → R , we haveΛ ΨΦ ( v ) ≤ Λ( v ) for all v ∈ V .2.
Equality.
Since Ω is Ψ -convex, there exits Λ: V → R such that Ω = Λ Ψ .
We then haveΩ Φ = Λ ΨΦ .
Using the lower bound property, we get Ω Φ ( v ) =
Λ ΨΦ ( v ) ≤ Λ( v ) for all v ∈ V .
Bythe order reversing property, we have Ω ΦΨ ( p ) ≥ Λ Ψ ( p ) =
Ω( p ) for all p ∈ C .
However, we alsohave Ω ΦΨ ( p ) ≤ Ω( p ) for all p ∈ C .
Therefore, Ω ΦΨ ( p ) = Ω( p ) for all p ∈ C .3.
Tightest lower-bound.
Let Ω (cid:48) be any lower bound of Ω that is Ψ -convex.
Therefore, there existsΛ:
V → R such that Ω (cid:48) ( p ) =
Λ Ψ ( p ) ≤ Ω( p ) for all p ∈ C .
By the order reversing property, wehave Λ ΨΦ ( v ) ≥ Ω Φ ( v ) for all v ∈ V .
By the lower bound property, we also have Λ ΨΦ ( v ) ≤ Λ( v )for all v ∈ V and therefore, Λ( v ) ≥ Ω Φ ( v ) for all v ∈ V .
Applying the order reversing propertyonce more, we get Ω (cid:48) ( p ) =
Λ Ψ ( p ) ≤ Ω ΦΨ ( p ) for all p ∈ C .
Therefore Ω ΦΨ is the tightest lowerbound of Ω .
B.6 Proofs for Proposition 6 (Properties of generalized Bregman divergences)
Link with generalized Fenchel-Young losses.
From (16) and (17), we have
[Equation]
for any Λ: V → R .
With Λ = Ω Φ , if Ω is Φ -convex, using Proposition 5, we have
[Equation]
Plugging Ω( p (cid:48) ) in (18) and using the shorthand v := v ΨΩ Φ ( p (cid:48) ) , we get
[Equation]
Non-negativity.
This
follows
directly
from the non-negativity of L ΦΩ ( v, p )
Identity of indiscernibles .
If p = p (cid:48) , we immediately obtain D ΦΩ ( p, p (cid:48) )
=
from (18).
Let’sprove the reverse direction.
If D ΦΩ ( p, p (cid:48) )
= 0 , we have F ( v ΨΩ Φ ( p (cid:48) ) , p ) =
F ( v ΨΩ Φ ( p (cid:48) ) , p (cid:48) )
where
F ( v, p ) =
Ω( p ) − Φ( v, p ) .
Since by assumption F is strictly convex in p , we
obtain p = p (cid:48) .
Convexity.
This follows from D ΦΩ ( p, p (cid:48) )
=
Ω( p ) − Φ( v, p ) + const.
Recovering Bregman divergences.
If Φ( v, p ) =
(cid:104) v, p (cid:105) , we have
[Equation]
Plugging back in (18), we obtain the Bregman divergence (20).
C Experimental details
C.1 Multilabel classiﬁcation
Datasets.
We used public datasets available at https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/ .
Dataset statistics are summarized in Table 3.
[Table]
Experimental details.
In all experiments, we set the activation σ to relu ( a ) :
= max { 0 , a } .
For the unary model, we use a neural network with one hidden-layer, i.e., g θ ( x ) =
W 2 σ ( W 1 x + b 1 )+b 2 , where θ =
( W 2 , b 2 , W 1 , b 1 ) , W 2 ∈ R k × m , b 2 ∈ R k , W 1 ∈ R m × d , b 1 ∈ R m , and m is the numberof hidden units.
We use the heuristic m =
min { 100 , d/ 3 } , where d is the dimensionality of x .
For the pairwise model, in order to obtain a negative semi-deﬁnite matrix U , we parametrize
U = − AA (cid:62) with A =
[ W 1 x + b 1 , . .
, W m x + b m ] , where W j ∈ R k × d and b j ∈ R k
In ourexperiments, we choose a rank-one model, i.e., m = 1 .
Note that we use distinct parameters for theunary and pairwise models but sharing parameters would be possible.
For the SPEN model, following [10, Eq. 4 and 5], we set the energy to Φ( v, p ) =
(cid:104) u, p (cid:105) − Ψ
( w, p ) ,where v = ( u, w ) , u
= g θ ( x ) and w are the weights of the “prior network” Ψ (independent of x ).
Weparametrize Ψ( w, p ) =
W 2 σ ( W 1 p + b 1 ) + b 2 , where w = ( W 2 , b 2 , W 1 , b 1 ) , W 2 ∈ R 1 × m , b 2 ∈ R 1and W 1 ∈ R m × k , b 1 ∈ R m .
To further impose convexity of Ψ in p , W 2 needs to be non-negative.
Todo so without using constrained optimization, we use the change of variable W 2 =
softplus ( W (cid:48) 2 ) ,where softplus ( a ) :
= log(1 + exp( a )) is used as an element-wise bijective mapping.
GPU results were obtained using a single NVIDIA P100 GPU with 16GB of memory.
Additional results.
The gradient of Ω Φ ( v ) can be computed using the envelope theorem (Proposition 1), which does not require to differentiate through p ΦΩ ( v ) .
Alternatively, since Ω Φ ( v ) =
Φ( v, p ΦΩ ( v )) − Ω( p ΦΩ ( v )) , we can also compute the gradient of Ω Φ ( v ) by using the implicit functiontheorem, differentiating through p ΦΩ ( v ) .
To do so, we use the approach detailed in [16].
Results in
Table 4 show that the envelope theorem performs comparably to the implicit function theorem, if notslightly better.
Differentiating through p ΦΩ ( v ) using the implicit function theorem requires to solve ak × k system and its implementation is more complicated than the envelope theorem.
Therefore, wesuggest to use the envelope theorem in practice.
model (test accuracy in %).
[Table] [Table]
In addition, we also compared the proposed generalized Fenchel-Young loss with the energy loss, thebinary cross-entropy loss and the generalized perceptron loss, which corresponds to setting Ω( p ) = 0 .
As explained in §3, the cross-entropy loss requires to differentiate through p ΦΩ ( v ) ; we do so by implicitdifferentiation.
Table 5 shows that the generalized Fenchel-Young loss outperforms these losses.
As expected, the energy loss performs very poorly, as it can only push the model in one direction[42].
Using regularization Ω , as advocated in this paper, is empirically conﬁrmed to be beneﬁcial foraccuracy.
[Table]
C.2 Imitation learning
Experimental details.
We run a hyperparameter search over the learning rate of the ADAMoptimizer, the number of hidden units in the layers, the weight of the L2 parameters regularizationterm and the scale of the energy regularization term Ω .
We run the hyperparameter search for4 demonstration trajectories and select the best performing ones based on the ﬁnal performance(averaged over 3 seeds).
[Table]
Environments.
We also provide performance of the expert agent as detailed by Orsini et al.
[55] aswell as the description of the observation and action spaces for each environment.
[Table]