Learning Energy Networks with Generalized Fenchel-Young Losses
Abstract
Energy-based models, a.k.a.
energy networks, perform inference by optimizingan energy function, typically parametrized by a neural network.
This allowsone to capture potentially complex relationships between inputs and outputs.
Tolearn the parameters of the energy function, the solution to that optimizationproblem is typically fed into a loss function.
The key challenge for training energynetworks lies in computing loss gradients, as this typically requires argmin/argmaxdifferentiation.
In this paper, building upon a generalized notion of conjugatefunction, which replaces the usual bilinear pairing with a general energy function,we propose generalized Fenchel-Young losses, a natural loss construction forlearning energy networks.
Our losses enjoy many desirable properties and theirgradients can be computed efﬁciently without argmin/argmax differentiation.
Wealso prove the calibration of their excess risk in the case of linear-concave energies.
We demonstrate our losses on multilabel classiﬁcation and imitation learning tasks.
Introduction
Training a neural network usually involves ﬁnding its parameters by minimizing a loss function,which captures how well the network ﬁts the data.
A typical example is the multiclass logistic loss(a.k.a.
cross-entropy loss) from logistic regression, which is the canonical loss associated with thesoftmax output layer and the categorical distribution.
If we replace the softmax with another outputlayer, what loss function should we use instead?
In generalized linear models [52, 48] which includelogistic regression and Poisson regression as special cases, the negative log-likelihood gives a lossfunction associated with a link function, generalizing the softmax to other members of the exponentialfamily [8].
More generally, Fenchel-Young losses [18] provide a generic way to construct a canonicalconvex loss function if the associated output layer can be written in a certain argmax form.
Besidesthe aforementioned generalized linear models, models that fall in this family include sparsemax[47], the structured perceptron [27] and conditional random ﬁelds [43, 69].
However, the theory of
Fenchel-Young losses is currently limited to argmax output layers that use a bilinear pairing.
To increase expressivity, energy-based models [44], a.k.a.
energy networks , perform inferenceby optimizing an energy function , typically parametrized by a neural network.
This leads to aninner optimization problem, which can capture potentially complex relationships between inputs andoutputs.
A similar approach is taken in SPENs (Structured Prediction Energy Networks)
[11, 13],in which a continuous relaxation of the inner optimization problem is used, amenable to projectedgradient descent or mirror descent.
In input-convex neural networks [4], energy functions arerestricted to be convex, so as to make the inner optimization problem easy to solve optimally.
Tolearn the parameters of the energy function, the solution of the inner optimization problem is typicallyfed into a loss function, leading to an outer optimization problem.
In order to solve that problem bystochastic gradient descent, the main challenge lies in computing the loss gradients.
Indeed, whenusing an arbitrary loss function, by the chain rule, computing gradients requires differentiating throughthe inner optimization problem solution, often referred to as argmin or argmax differentiation .
This can be done by backpropagation through unrolled algorithm iterates [13] or implicit differentiationthrough the optimality conditions [4].
This issue can be circumvented by using a generalizedperceptron loss [44] or a max margin loss [11].
These losses only require max differentiation , forwhich envelope theorems [49] can be used.
However, these losses often fail to satisfy envelopetheorem assumptions such as unicity of the solution and lack theoretical guarantees.
In this paper, we propose to extend the theory of Fenchel-Young losses [18], so as to create a canonicalloss function associated with an energy network.
Our proposal builds upon generalized conjugatefunctions , also known as Fenchel-Moreau conjugates [50, 62], allowing us to go beyond bilinearpairings and to support general energy functions such as neural networks.
We introduce regularization , allowing us to obtain unicity of the solutions and to use envelope theorems to compute gradients,without argmin or argmax differentiation.
We provide novel guarantees on the calibration of theexcess risk, when our loss is used as a surrogate for another discrete loss.
To sum up, we obtain a wellmotivated loss construction for general energy networks.
The rest of the paper is organized as follows
• After providing some background (§2), we describe regularized energy networks , identify a classiﬁcation of energy functions and give several possible examples that fall in this family (§3). • We deﬁne generalized conjugates , an extension of Legendre-Fenchel conjugates, and state theirproperties (§4).
We establish novel conditions for a generalized conjugate to be a smooth function. • We then introduce generalized Fenchel-Young losses and show that they enjoy many of the same
favorable properties as the regular Fenchel-Young losses (§5).
On the theoretical side, we prove
calibration of the excess risk for linear-concave energies and strongly-convex regularizers (§6).
• We demonstrate our losses on multilabel classiﬁcation and imitation learning tasks (§7).
Background
Convex conjugates.
Let C ⊆
R k be an output set.
Let Ω: C → R be a function, such thatΩ( p ) =
∞ for all p (cid:54)∈ C , i.e., dom(Ω) =
C .
Given v ∈ V ⊆ R k , we deﬁne the convex conjugate [22]of Ω , also known as the Legendre-Fenchel transform of Ω , by
[Equation]
The conjugate Ω ∗ is always convex, even if Ω is not.
We deﬁne the corresponding argmax as
[Equation]
The latter equality follows from Danskin’s theorem [29, 15], under the assumption that Ω is strictlyconvex (otherwise, we obtain a subgradient).
It is well-known that Ω ∗ is 1 γ -smooth w.r.t.
the dualnorm (cid:107) · (cid:107) ∗ if and only if Ω is γ -strongly convex w.r.t.
the norm (cid:107) · (cid:107) [37, 39, 10, 75].
Fenchel-Young losses.
Suppose that v = g θ ( x
) ∈ R k are the logits / scores produced by aneural network g , where x ∈ X ⊆ R d and θ ∈ Θ are the network’s input features and parameters,respectively.
What loss function should we use if we want to use (2) as output layer?
The FenchelYoung loss [18] generated by Ω: C
→ R provides a natural solution.
It is deﬁned by
[Equation]
where y ∈ Y ⊆
C is the ground-truth label.
Earlier instances of this loss were independently proposedin the contexts of ranking [2] and multiclass classiﬁcation [30].
Among many useful properties, thisloss satisﬁes L Ω ( v, y ) ≥ 0 and L Ω ( v, y ) = 0 ⇔ y = p Ω ( v ) if Ω is strictly convex [18].
In that sense,it is the canonical loss associated with (2).
Interestingly, many existing loss functions are recoveredas special cases.
For instance, if C = R k and Ω( p ) = 12 (cid:107) p (cid:107) 22 , which is 1 -strongly convex w.r.t.
(cid:107) · (cid:107) 2over R k , then we obtain the self-dual, the identity mapping and the squared loss:
[Equation]
If C is the probability simplex (cid:52) k := { p ∈ R k
+ : (cid:80) ki
=1 p i = 1 } and Ω( p ) is the scaled Shannonnegentropy γ (cid:104) p, log p (cid:105) , which is γ -strongly convex w.r.t.
(cid:107) · (cid:107) 1 over (cid:52) k , then we obtainΩ ∗ ( v ) =
LSE γ ( v ) ,
[Equation]
where we used the log-sum-exp LSE γ ( v ) :
= γ log( (cid:80) ki
=1 exp( v i /γ )) , softmax γ ( v ) ∝ exp( v/γ ) ,and the Kullback-Leibler divergence.
More generally, when C = conv( Y ) , the convex hull of Y , p Ωcorresponds to a projection, for which efﬁcient algorithms exist in numerous cases [18, 16].
Thecalibration of the excess risk of Fenchel-Young losses when Ω is strongly convex was establishedin [56, 16].
However, the theory of Fenchel-Young losses is currently limited to the bilinear pairing(cid:104) v, p (cid:105) , which restricts their expressivity and scope.
Regularized energy networks
Energy networks.
Also known as energy-based models or EBMs [44], of which SPENs [11, 13] area particular case, these networks compute predictions by solving an optimization problem of the form
[Equation]
where v = g θ ( x ) ∈ V is the energy network’s input, Φ( v, p ) is a scalar-valued energy function, and
C is an output set.
Throughout this paper, we use the convention that higher energy indicates higherdegree of compatibility between v and the prediction p .
Since Φ( v, p ) is a general energy function, weemphasize that v and p do not need to have the same dimensions, unlike with the bilinear pairing (cid:104) v, p (cid:105) .
Any neural network g θ ( x ) ∈ R k can be written in energy network form, since argmax
p ∈ R k −(cid:107)
g θ ( x ) −p (cid:107) 2
= g θ ( x ) .
The key advantage of energy networks, however, is their ability to capture complex interactions between inputs and outputs.
This is especially useful in the structured prediction setting [6],where predictions are made of sub-parts, such as sequences being composed of individual elements.
Introducing regularization.
In this paper, we compute predictions by solving
[Equation]
where we further added a regularization function Ω: C → R .
We call Φ( v, p ) − Ω( p ) a regularizedenergy function and we call the corresponding model a regularized energy network .
We obviouslyrecover usual energy networks by setting Ω to the indicator function of the set C , i.e., Ω( p ) = 0 ifp ∈ C , ∞ otherwise.
While it is in principle possible to absorb Ω into Φ , keeping Ω explicit hasseveral advantages: 1) it allows to introduce generalized conjugate functions and their properties(§4) 2) it allows to introduce generalized Fenchel-Young losses (§5), which mirror the original
Fenchel-Young losses 3) we obtain closed forms for (4) in certain cases (Table 1).
Solving the maximization problem.
The ability to solve the maximization problem in (4) efﬁcientlydepends on the properties of Φ( v, p ) w.r.t.
p .
If Φ( v, p ) is linear in p , for instance Φ( v, p )
= (cid:104) v, Up (cid:105) ,then we obtain p ΦΩ ( v ) =
p Ω ( U (cid:62) v ) .
Thus, the computation of (4) reduces to (2), for which closedforms are often available.
If Φ( v, p ) is concave in p and C is a convex set, then we can solve (4) inpolynomial time using an iterative algorithm, such as projected gradient ascent.
This is the mostgeneral energy class for which our loss and its gradient can be computed to arbitrary precision.
IfΦ( v, p ) is nonconcave in p , then it is not possible to solve (4) in polynomial time in general, unless Cis a discrete set of small cardinality.
In general, we emphasize that Φ( v, p ) can be nonconvex in v ,as is typical with neural networks, since the maximization problem in (4) is with respect to p ∈ C .
However, in the sequel of this paper, certain properties we establish require Φ( v, p ) to be convex in v .
We now give examples of regularized energy networks in increasing order of expressivity / complexity.
Generalized linear models.
As a warm up, we consider the case of bilinear energy Φ( v, p ) .
Forinstance, for probabilistic classiﬁcation with k classes, where the goal is to predict p ∈ (cid:52) k fromx ∈ R d , we can use Φ( v, p ) =
(cid:104) v, p (cid:105) with v =
Wx + b , where W ∈ R k × d and b ∈ R k .
In thiscase, we therefore obtain p ΦΩ ( v ) =
p Ω ( Wx + b ) , recovering generalized linear models [52, 48], thestructured perceptron [27] and conditional random ﬁelds [43, 69].
Rectiﬁer and maxout networks.
We now consider the case of convex-linear energy Φ( v, p ) .
Aﬁrst example is a rectiﬁer network [33] with one hidden layer.
Indeed, Φ( v, p ) =
(cid:104) σ ( v ) , Up (cid:105) isconvex in v if σ is an element-wise, convex and non-decreasing activation function, such as therelu or softplus, and if U and p are non-negative.
A second example is a maxout network [34](i.e., a max-afﬁne function) or its smoothed counterpart, the log-sum-exp network [24].
Indeed,Φ( v, p ) =
σ ( v ) · p , where σ is the max or log-sum-exp operator, is convex in v , if the scalar pis non-negative.
In general, it is always possible to construct a convex-concave energy from ajointly-convex function using the Legendre-Fenchel transform in the ﬁrst argument.
[Table]
Input-convex neural networks.
As an example of nonconvex-concave energy, we considerΦ( v, p ) = − ICNN ( v, p ) , where ICNN is an input-convex neural network [4], i
[Table]
e., it is convex in pbut can be nonconvex in v .
If Ω( p ) and C are convex, then Φ( v, p ) is concave in p and (4) can besolved in polynomial time using an iterative algorithm, such as projected gradient ascent.
Probabilistic energy networks.
It is often desirable to deﬁne a conditional probability distribution
[Equation]
Such networks are typically trained using a cross-entropy loss (or equivalently, negative loglikelihood).
Unfortunately, when Y is large or inﬁnite, this loss and its gradients are intractableto compute, due to the normalization constant.
Therefore, recent research has been devoted todeveloping approximate training schemes [67, 35, 51].
Although this is not the focus of this paper,we point out that probabilistic energy networks can also be seen as regularized energy networksin
the space of probability distributions C = (cid:52) |Y| , if we set Φ( v, p )
=
(cid:80) y ∈Y p ( y ) E ( v, y )
andΩ( p ) =
(cid:80) y ∈Y p ( y ) log p ( y ) .
While the resulting optimization problem is intractable in general, itdoes suggest possible approximation schemes, such as the use of Frank-Wolfe methods [12, 42, 55].
Existing loss functions for energy networks.
Given a pair ( x, y ) and output v = g θ ( x ) , how dowe measure the discrepancy between p ΦΩ ( v ) and y ?
One possibility [4, 13] is to use the compositionof a differentiable loss L : C × Y → R + with the argmax output, namely ( v, y )
(cid:55)→ L ( p ΦΩ ( v ) , y ) .
However, computing the loss gradient w.r.t.
v then requires to differentiate through p ΦΩ ( v ) , eitherthrough unrolling or implicit differentiation.
This is particularly problematic when C is a complicatedconvex set, as differentiating through a projection can be challenging.
In contrast, our proposed losscompletely circumvents this need and enjoys easy-to-compute gradients.
For unregularized energynetworks, a naive idea, called the energy loss, is to use ( v, y )
(cid:55)→ − Φ( v, y ) .
However, this loss onlyworks well if Φ is a similarity measure and works poorly in general [44].
A better choice is thegeneralized perceptron loss ( v, y )
(cid:55)→ max p ∈C Φ( v, p ) − Φ( v, y ) [44].
Our loss can be seen as aprincipled generalization of this loss to regularized energy networks, with theoretical guarantees.
Generalized conjugates
In order to devise generalized Fenchel-Young losses, we build upon a generalization due to Moreau[50, Chapter 14] of the convex conjugate Ω ∗ .
Denoted Ω Φ , it replaces the bilinear pairing in (1)with a more general coupling Φ [62, Chapter 11, Section L].
In this section, we state their deﬁnition,properties, closed-form expressions and connection to the C -transform in optimal transport.
Deﬁnition.
Let Φ( v, p ) ∈
R be a coupling / energy function.
The Φ -convex conjugate of Ω: C →
R , also known as Fenchel-Moreau conjugate , is then deﬁned by the value function
[Equation]
The Φ -convex conjugate is an important tool in abstract convex analysis [66, 63].
Recently, it hasbeen used to provide “Bellman-like” equations in stochastic dynamic programming [25] and to provide tropical analogues of reproducing kernels [5].
We assume that the maximum is feasible forall v ∈ V = R d , meaning that dom(Ω Φ ) =
V .
We emphasize again that, unlike with (1), v and p donot need to have compatible dimensions.
We denote the argmax solution corresponding to (5) by
[Equation]
If a function F ( v ) can be written as F ( v ) =
Ω Φ ( v ) for some Ω , it is called Φ -convex (in analogy, afunction f ( v ) is convex and closed if and only if it can be written as f ( v ) =
Ω ∗ ( v ) for some Ω ).
Properties.
Φ -convex conjugates enjoy many useful properties, some of them are natural extensionsof the usual convex conjugate properties.
Proofs are provided in Appendix B.1.
Proposition 1 (Properties of Φ -convex conjugates) .
Let Ω: C → R and Φ:
V × C → R .
Generalized Fenchel-Young inequality: for all v ∈ V and p ∈ C , Ω Φ ( v ) + Ω( p ) − Φ( v, p ) ≥ 0 .
Convexity: If Φ( v, p ) is convex in v , then Ω Φ ( v ) is convex (even if Ω( p ) is nonconvex).
Order reversing: if Ω( p ) ≤ Λ( p ) for all p ∈ C , then Ω Φ ( v ) ≥ Λ Φ ( v ) for all v ∈ V . 4.
Continuity: Ω Φ shares the same continuity modulus as Φ . 5.
Gradient (envelope theorem): Under assumptions (see paragraph below), we have ∇ Ω Φ ( v ) = ∇ 1 Φ( v, p ΦΩ ( v )) , where ∇ 1 denotes the gradient in the ﬁrst argument.
Smoothness:
If C is a compact convex set, Φ( v, p ) is β -smooth in ( v, p ) , concave in p and Ω( p ) is γ -strongly convex in p , then Ω Φ ( v ) is ( β
The condition on Φ and Ω in item 6 for Ω Φ to be a smooth function (i.e., with Lipschitz-continuousgradients) is a novel result and will play a crucial role for establishing calibration guarantees in §6.
Assumptions for envelope theorems.
The expression in item 5 allows to compute ∇ Ω Φ ( v ) withoutargmax differentiation.
It is based on envelope theorems, which can be used for max differentiation.
Indeed, we have Ω Φ ( v ) =
max p ∈C F ( v, p ) , where F ( v, p ) :
=
Φ( v, p ) − Ω( p ) .
We assume that themaximum is unique and C is a compact.
If F ( v, p ) is convex in v , we apply Danskin’s theorem [29][15, Proposition B.25].
Without convexity assumption in v , if F ( v, p ) is continuously differentiablein v for all p ∈ C , ∇ 1 F is continuous, we apply [62, Theorem 10.31].
When we do not computethe exact solution of (6), we only obtain an approximation of the gradient ∇ Ω Φ ( v ) ; see [1] forapproximation guarantees.
For other envelope theorem usecases in machine learning, see, e.g., [21].
Closed forms.
While (5) and (6) may need to be solved numerically in general, they enjoy closedform expressions in simple cases.
Proofs are provided in Appendix B.2.
Proposition 2 (Closed-form expressions) .
Let Ω: C → R and Φ:
V × C → R .
Bilinear coupling: If Φ( v, p ) = (cid:104) v, Up (cid:105) , then Ω Φ ( v ) =
Ω ∗ ( U (cid:62) v ) and p Φ Ω ( v ) =
p Ω ( U (cid:62) v ) .
Linear-quadratic coupling: If C = R k , Ω( p ) =
γ 2 (cid:107) p (cid:107) 22 and Φ( v, p ) = 12 (cid:104)
p
[Equation]
+
[Equation]
where v = ( A, b ) and A is such that ( γI − A ) is positive deﬁnite, we obtain
[Equation]
Metric coupling: If V = C , Φ = − C where C ( v, p ) is a metric and Ω is M -Lipschitz with M ≤ 1 , then Ω Φ = − Ω and p ΦΩ ( v )
= v .
Relation with the C -transform.
Given a cost function C , we may deﬁne a min counterpart of (5),
[Equation] [Equation]
min
[Equation]
In the optimal transport literature, this is known as the C -transform of Λ [65, 60].
When C is bilinear,this recovers the notion of concave conjugate [20].
When C ( v, p ) =
c ( v − p ) for some c , this recoversthe inﬁmal convolution [20].
It is easy to check that Ω = − Λ ⇔ Ω Φ = − Λ C with C = − Φ .
Thus,Ω Φ and Λ C are the natural extensions of convex and concave conjugates, respectively.
We opt for theformer in this paper to closely mirror the usual convex conjugates and Fenchel-Young losses.
Generalized Fenchel-Young losses
Deﬁnition.
The generalized Fenchel-Young inequality in Proposition 1 leads us to propose thegeneralized Fenchel-Young loss, the natural extension of (3) to Φ -convex conjugates:
[Equation]
We also have the relationships L ΦΩ ( v, y )
=
F ( v, y ) − F ( v, p ΦΩ ( v )) , where F ( v, p ) =
Ω( p ) − Φ( v, p ) ,and p ΦΩ ( v ) = argmin p ∈C L ΦΩ ( v, p ) .
We now give an intuitive geometric interpretation of (9).p
[Equation]
[Figure]
[Equation]
[Figure]
[Equation]
[Figure]
Geometric interpretation.
For the sake of illustration, let us set Ω( p ) =
γ 2 p 2 and Φ( v, p ) =
12 ap 2 + bp .
From (1), the line p (cid:55)→ (cid:104) u, p (cid:105) − Ω ∗ ( u ) , whereΩ ∗ ( u ) = 12 γ u 2 , is the tightest linear lower boundon Ω( p ) , here depicted with u = 0 .
It is thetangent of Ω( p ) at p =
p Ω ( u ) .
Similarly, from(5), p (cid:55)→ Φ( v, p ) − Ω Φ ( v ) is the tightest lowerbound of this form given v .
It touches Ω( p ) atp
= p ΦΩ ( v ) , here depicted with v = ( a, b ) , a = − 1and b = 1 , for which we have the closed formΩ Φ ( v ) =
12 ( γ − a ) − 1 b 2 (Proposition 2).
The generalized Fenchel-Young loss (9) is then the gap betweenΩ( p ) and Φ( v, p ) − Ω Φ ( v ) , evaluated at the groundtruth p =
y .
The goal of training is to adjust the parameters θ of a network v = g θ ( x ) so as tominimize this gap averaged over all ( x, y ) training pairs.
Properties.
Generalized Fenchel-Young losses enjoy many desirable properties, as we now show.
Proposition 3 (Properties of generalized F-Y losses) .
Let Ω: C → R and Φ: V × C → R .
Non-negativity: L ΦΩ ( v, p ) ≥ 0 for all v ∈ V and p ∈ C . 2.
Zero loss: if the maximum in (5) exists and is unique, L ΦΩ ( v, y ) = 0 ⇔ y =
p ΦΩ ( v ) .
Gradient: ∇ 1 L ΦΩ ( v, p ) =
∇ Ω Φ ( v ) − ∇ 1 Φ( v, p ) .
Difference of convex (DC): if Φ( v, p ) is convex in v , then L ΦΩ ( v, p ) is a difference of convex functions in v : Ω Φ ( v ) and Φ( v, p ) .
If Φ( v, p ) is linear in v , then L ΦΩ ( v, p ) is convex in v .
Smaller output set, smaller loss.
If C (cid:48) ⊆ C and Ω (cid:48) is the restriction of Ω to C (cid:48) , then L ΦΩ (cid:48) ( v, p ) ≤ L ΦΩ ( v, p ) for all v ∈ V and p ∈ C (cid:48) .
Quadratic lower-bound.
If Φ( v, p ) − Ω( p ) is γ -strongly concave in p w.r
t. (cid:107) · (cid:107) over C and C is a closed convex set, then for all v ∈ V and p ∈ C
[Equation]
Upper-bounds.
If Φ( v, p ) − Ω( p ) is α -Lipschitz in p w.r.t.
(cid:107) · (cid:107) over C , then L ΦΩ ( v, p ) ≤ α (cid:107) p − p ΦΩ ( v ) (cid:107) .
If Φ( v, p ) is concave in p , then L ΦΩ ( v, p ) ≤
L Ω ( ∇ 2 Φ( v, p ) , p )
Proofs are given in Appendix B.3.
These two paragraphs are now better connected to
Proposition 3.
The non-negativity is a good property for a loss function.
Thezero-loss property L ΦΩ ( v, y ) = 0 ⇔ y =
p ΦΩ ( v ) , which is true for instance if p (cid:55)→ Φ( v, p ) − Ω( p ) isstrictly concave, is key as it allows to use p ΦΩ ( v ) deﬁned in (6) as the (implicit) output layer associatedwith an energy network.
The gradient ∇ 1 L ΦΩ ( v, y ) does not require to differentiate through theargmax problem in (6) needed for computing p ΦΩ ( v ) .
Typically, differentiating through an argmaxor argmin, as is done in input-convex neural networks [4], requires either unrolling or implicitdifferentiation [36, 14, 41, 19, 17] and is therefore more costly.
The fact that L ΦΩ ( v, p ) is a difference of convex (DC) functions in v when Φ( v, p ) is convex in vsuggests that we can use DC programming techniques, such as the convex-concave procedure [72],for training such energy networks.
We leave the investigation of this observation to future work.
The“smaller output set, smaller loss” property means that we can achieve the smallest loss by choosingthe smallest set C in (6) such that Y ⊆ C .
The smallest such convex set is the convex hull of Y , also known as marginal polytope [71] when Y ⊆ { 0 , 1 } k .
The quadratic lower-bound relates our loss tousing p ΦΩ ( v ) within a squared norm loss.
The upper-bounds relate our loss to using p ΦΩ ( v ) within anorm loss and to using a regular Fenchel-Young loss with a linearized energy.
If Ω is the indicator function of C , i.e., Ω( p ) = 0 if p ∈ C , ∞ otherwise, then it can be checked thatwe recover the “generalized perceptron” loss [45, 44] as a special case of (9).
Proposition 3 thereforeprovides new properties to understand and analyze this loss.
Regular Fenchel-Young losses are closely related to Bregman divergences [18].
In Appendix A, webuild a generalized notion of Bregman divergence using generalized conjugates.
Training.
To train regularized energy networks with our framework, the user should choose anenergy Φ( v, p ) , a regularization Ω( p ) , an output set C (from Proposition 3 item 5, the smaller thisset the better) and the model v = g θ ( x ) with input x and parameters θ .
Given a set of input-outputpairs ( x 1 , y 1 ) , . . .
, ( x n , y n ) ∈ X × Y , where Y ⊆ C ,
we can ﬁnd the parameters θ by minimizingthe empirical risk objective regularized by R : Θ → R ,(cid:98)
[Equation]
[Equation]
Thanks to the easy-to-compute gradients of L ΦΩ , we can easily solve (11) using any (stochastic) solver.
Calibration guarantees
Many times, notably for differentiability reasons, the loss used at training time, here our generalized
Fenchel-Young loss L ΦΩ ( v, y ) , is used as a surrogate / proxy for a different (potentially discrete loss)
L : Y × Y → R , used at test time.
Calibration guarantees [73, 74, 9, 68] ensure that minimizing theexcess of risk of the train loss will also minimize that of the test loss (a.k.a.
target loss).
We study inthis section such guarantees, assuming that L satisﬁes an afﬁne decomposition property [26, 16]:
[Equation]
) where ϕ ( y )
(cid:55)→ V ϕ
( y )
+ b is an afﬁne map, ϕ
( y ) is a label embedding and c ( y ) is any function thatdepends only on y .
Numerous losses can be written in this form.
Examples include the zero-one,
Hamming, NDCG and precision at k losses [56, 16].
Inference in this setting works in two steps.
First, we compute a “soft” (continuous) prediction p = p ΦΩ ( v ) ∈ C .
Second, we compute a “hard”(discrete) prediction by a decoding / rounding from C to Y , calibrated for the loss L :
[Equation]
The target risk of f : X → Y and the surrogate risk of g : X → V are deﬁned by
L ( f ) :
= E ( X,Y ) ∼ ρ L ( f ( X ) , Y ) and L ΦΩ ( g ) :
= E ( X,Y ) ∼ ρ L ΦΩ ( g ( X ) , Y ) , where ρ is a typically unknown distribution over X × Y .
The Bayes predictors are deﬁned byf (cid:63) :
= argmin f : X→Y L ( f ) and g (cid:63) :
= argmin g : X→V L ΦΩ ( g ) .
We now establish calibration of thesurrogate excess risk, under the assumption that the energy Φ( v, p ) is linear-concave , i.e., it can bewritten as Φ( v, p )
=
(cid:104) v, ϕ ( p ) (cid:105) , for some function ϕ .
Proposition 4.
Calibration of target and surrogate excess risks
Assume L ΦΩ ( v, y ) is M -smooth in v w.r.t.
the dual norm (cid:107) · (cid:107) ∗ , C is a compact convex set such that Y ⊆ C and Φ( v, p ) =
(cid:104) v, ϕ ( p ) (cid:105) .
Let σ := sup y ∈Y (cid:107) V (cid:62) ϕ
( y ) (cid:107) ∗ .
Then, the generalized
Fenchel-Young loss (9) is calibrated with the target loss (12) with decoder d = y L ◦ p ΦΩ :
[Equation]
where p ΦΩ : V → C is deﬁned in (6) and y L : C → Y is deﬁned in (13) .
The proof is in Appendix B.4.
Proposition 1 item 6 shows that the smoothness of Φ and strongconvexity of Ω ensure the smoothness of L ΦΩ .
Calibration also implies Fisher consistency, namely
[Table]
L ΦΩ ( g ) =
L ΦΩ ( g (cid:63) )
⇒ L ( d ◦ g )
=
L ( f (cid:63) ) , when using the decoder d = y L ◦ p ΦΩ .
The existing prooftechnique for the calibration of regular Fenchel-Young losses [56, 16] assumes a bilinear pairing anda loss of the form L Ω ϕ ( u, ϕ ( y ))
=
Ω ∗ ϕ ( u ) + Ω ϕ ( ϕ ( y )) − (cid:104) u, ϕ ( y ) (cid:105) , where Ω ϕ is a strongly-convexregularizer w.r.t.
ϕ ( y ) .
Our novel proof technique is more general, as it works with any linear-concaveenergy and the regularizer Ω is w.r.t.
y , not ϕ ( y ) .
Unlike the existing proof, our proof is valid for thepairwise model we present in the next section.
Experiments
7.1 Multilabel classiﬁcation
We study in this section the application of generalized Fenchel-Young losses to multi-label classiﬁcation, setting Y = { 0 , 1 } k and C = [0 , 1] k , where k is the number of labels.
When the loss L in (12)is the Hamming loss (1 - accuracy), our loss is calibrated for L and the decoding (13) is just (cid:98)
y j = 1 ifp j > 0 .
5 else 0 .
We therefore report our empirical results using the accuracy metric.
Unary model.
We consider a neural network u
= g θ ( x ) ∈ R k , assigning a score
u j to each labelj ∈ [ k ] .
With the bilinear pairing Φ( u, p ) = (cid:104) u, p (cid:105) , we getp ΦΩ ( u ) =
argmax p ∈
[0 , 1] k (cid:104) u, p (cid:105) − Ω( p ) =
p Ω ( u ) .
That is, (14) is just a normal neural network with p Ω as output layer.
When Ω( p ) =
Ω 1 ( p )+Ω 1 (1 − p ) ,where Ω 1 ( p ) :
= (cid:104) p, log p (cid:105) is Shannon
’s negentropy, we get p Ω ( v )
=
sigmoid ( v ) :
= 1 / (1+exp( − v ))and (9) is just the usual binary logistic / cross-entropy loss.
When Ω( p ) =
Ω 2 ( p ) + Ω 2 (1 − p ) , whereΩ 2 ( p ) :
= 12 (cid:104) p, 1 − p (cid:105) is Gini’s negentropy, we get a sparse sigmoid and the binary sparsemax loss [18,§6.2].
Because C = [0 , 1] k is the convex hull of Y = { 0 , 1 } k , (14) can be intepreted as a marginalprobability [71].
Indeed, there exists a probability distribution P ( Y | X ) over Y ∈ Y such that
[Equation]
Pairwise model.
We now additionally use a network U
=
h θ ( x ) ∈ R k × k , assigning a score
U i,j tothe pairwise interaction between labels i and j .
With v = ( u, U ) and the linear-quadratic couplingΦ( v, p ) =
(cid:104) u, p (cid:105) + 1 2 (cid:104) p, Up (cid:105) , we get
[Equation]
If U is negative semi-deﬁnite, i.e., U = − AA (cid:62) for some matrix A ∈ R k × m , then the problem isconcave in p and can be solved optimally.
Moreover, since Φ( v, p ) is linear-concave, the calibrationguarantees in §6 hold and L ΦΩ ( v, y ) is convex in v .
In our experiments, we use a matrix A of rank 1(cf.
Appendix C).
Unlike (7), (15) does not enjoy a closed form due to the constraints.
We solve itby coordinate ascent: if Ω is quadratic, as is the case with Gini’s negentropy, the coordinate-wiseupdates can be computed in closed-form.
Again, (15) can be interpreted as a marginal probability.
Inconstrast to (15), marginal inference in the closely related Ising model is known to be #P-hard [32].
SPEN model.
Following SPENs [11, Eq. 4 and 5], we also tried the energy Φ( v, p ) =
(cid:104) u, p (cid:105) −Ψ( w, p ) , where v = ( u, w ) , u
= g θ ( x ) and w are the weights of the “prior network” Ψ (independentof x ).
We also tried a variant where Ψ is made convex in p , making Φ( v, p ) concave in p .
In bothcases, we compute p ΦΩ ( v ) by solving (6) using projected gradient ascent with backtracking linesearch.
[Figure]
Experimental setup.
We perform experiments on 6 publicly-available datasets, see Appendix C.
We use the train-test split from the dataset when provided.
When not, we use 80% for training dataand 20% for test data.
For all models, we solve the outer problem (11) using ADAM.
We set Ω( p ) tothe Gini negentropy.
We hold out 25% of the training data for hyperparameter validation purposes.
We set R ( θ ) in (11) to λ 2 (cid:107) θ (cid:107) 22 .
For the regularization hyper-parameter λ , we search 5 log-spacedvalues between 10 − 4 and 10 1 .
For the learning rate parameter of ADAM, we search 10 log-spacedvalued between 10 − 5 and 10 − 1 .
Once we selected the best hyperparameters, we reﬁt the model onthe entire training set.
We average results over 3 runs with different seeds.
Results.
Table 2 shows a model comparison.
We observe improvements with the pairwise modelon 4 out of 6 datasets, conﬁrming that our losses are able to learn useful models.
Input-concavityhelps improve SPENs in 4 out of 6 datasets.
Table 4 conﬁrms that using the envelope theorem forcomputing gradients works comparably to (if not better than) the implicit function theorem.
Table 5shows that our losses outperform the energy, the cross-entropy and the generalized perceptron losses.
7.2 Imitation learning
In this section, we study the application of generalized Fenchel-Young losses to imitation learning.
This setting consists in learning a policy π : X (cid:55)→ Y , a mapping from states X to actions Y ,from a ﬁxed dataset of expert demonstrations ( x 1 , y 1 ) , . . .
, ( x n , y n ) ∈
X × Y .
In particular, weonly consider a Behavior Cloning approach [61], which essentially reduces imitation learning tosupervised learning (as opposed to inverse RL methods [64, 54]).
The learned policy π is evaluatedon its performance, which is the expected sum of rewards of the environment [70].
Experimental setup.
We consider four MuJoCo
Gym locomotion environments [23] together withthe demonstrations provided by Orsini et al.
[58]; see Appendix C.2 for details.
We evaluate thelearned policy π for different number of demonstration trajectories: 1, 4 and 11, consistently with[38, 40, 28].
The action space in the demonstrations is included in [ − 1 , 1] k , where the dimensionalityof the action space k corresponds to the torques of the actuators.
We scale the action space to thehypercube Y =
C =
[0 , 1] k at learning time and scale it back to the original action space at inferencetime.
Similarly to the multilabel classiﬁcation setup, we evaluate the unary and pairwise models, withthe only difference that we use two hidden layers instead of one, since it leads to signiﬁcantly betterperformance.
We specify the hyperparameter selection procedure in Appendix C.2.
Results.
We run the best hyperparameters over 10 seeds and report ﬁnal performance over 100evaluation episodes.
Figure 2 shows a clear improvement of the pairwise model over the unarymodel for 3 out of 4 tasks.
Contrary to the unary model, the pairwise model enables to capture theinterdependence between the different torques of the action space, translating into better performance.
Conclusion
Building upon generalized conjugate functions, we proposed generalized Fenchel-Young losses,a natural loss construction for learning energy networks and studied its properties.
Thanks toconditions on the energy Φ and the regularizer Ω ensuring the loss smoothness, we establishedcalibration guarantees for the case of linear-concave energies, a more general result than the existinganalysis, restricted to bilinear energies.
We demonstrated the effectiveness of our losses on multilabelclassiﬁcation and imitation learning tasks.
We hope that this paper will help popularize generalizedconjugates as a powerful tool for machine learning and optimization.
A Generalized Bregman divergences
The expression Ω ∗ ( v ) + Ω( p ) − (cid:104) v, p (cid:105) at the heart of Fenchel-Young losses is closely related to
Bregman divergences; see [3, Theorem 1.1] and [18].
In this section, we develop a similar relationshipbetween generalized Fenchel-Young losses and a new generalized notion of Bregman divergence.
Biconjugates.
We begin by recalling well-known results on biconjugates [22].
Applying theconjugate (1) twice, we obtain the biconjugate Ω ∗∗ ( p ) .
It is well-known that a function Ω is convexand closed (i.e., lower-semicontinuous)
if and only if Ω = Ω ∗∗ .
This therefore provides a variationalcharacterization of lower-semicontinuous convex functions.
This characterization naturally motivatesthe class of Φ -convex functions (§4).
If Ω is nonconvex, Ω ∗∗ is Ω ’s tightest convex lower bound.
Generalized biconjugates.
The generalized conjugate in (5) uses maximization w.r.t.
the secondargument p ∈ C .
To obtain a generalized conjugate whose maximization is w.r.t.
the ﬁrst argumentv ∈ V instead, we deﬁne Ψ( p, v ) :
=
Φ( v, p ) .
Note that if Φ is symmetric, the distinction between Φand Ψ is not necessary.
Similarly to (5), we can then deﬁne the Ψ -conjugate of Λ: V → R as [Equation]
[Equation]
i.e., the maximization is over the left argument of Φ .
We deﬁne the corresponding argmax as
[Equation]
Λ
[Equation]
In particular, with Λ := Ω Φ , we can deﬁne the generalized biconjugate Λ Ψ = Ω ΦΨ : C → R .
Generalized biconjugates enjoy similar properties as regular biconjugates, as we now show.
Proposition 5 (Properties of generalized biconjugates) .
Let Ω: C → R , Φ:
V × C
→ R and Ψ( p, v ) :
=
Φ( v, p ) .
Lower-bound: Ω ΦΨ ( p ) ≤ Ω( p ) for all p ∈ C . 2.
Equality: Ω ΦΨ ( p ) =
Ω( p ) if and only if Ω is Ψ -convex.
Tightest lower-bound: Ω ΦΨ ( p ) is the tightest Ψ -convex lower-bound of Ω( p ) .
Proofs are given in Appendix B.5.
Similar results hold for Λ ΨΦ , where Λ: V → R .
Deﬁnition.
We can now deﬁne the generalized Bregman divergence D ΦΩ : C × C → R + as
[Equation] [Equation]
where we recall that Ψ( p, v ) :
= Φ( v, p ) and using (17) we have
[Equation]
We have therefore obtained a notion of Bregman divergence parametrized by a coupling Φ( v, p ) , suchas a neural network.
As shown in the proposition below, generalized Bregman divergences enjoy similar properties asregular Bregman divergences.
Proofs are given in Appendix B.6.
Proposition 6 (Properties of generalized Bregman divergences) .
Let Ω be a Ψ -convex function, where Ψ( p, v ) :
=
Φ( v, p ) .
Link with generalized FY loss: Denoting v := v ΨΩ Φ ( p (cid:48) ) , we have D ΦΩ ( p, p (cid:48) )
=
L ΦΩ ( v, p ) .
Non-negativity: D ΦΩ ( p, p (cid:48) ) ≥ 0 , for all p, p (cid:48) ∈ C .
Identity of indiscernibles: D ΦΩ ( p, p (cid:48) )
= 0 ⇔ p = p (cid:48) if p (cid:55)→ Φ( v, p ) − Ω( p ) is strictly concave for all v ∈ V . 4.
Convexity: if p (cid:55)→ Φ( v, p ) − Ω( p ) is concave for all v ∈ V , then D ΦΩ ( p, p (cid:48) ) is convex in p .
Recovering Bregman divergences: If Φ( v, p ) is the bilinear coupling (cid:104) v, p (cid:105) , then we recover the usual Bregman divergence D Ω : C × C
→ R +
[Equation]
Some remarks:
• The generalized Bregman divergence is between objects p and p (cid:48) of the same space C , while thegeneralized Fenchel-Young loss is between objects v and p of mixed spaces V and C .
• If Φ( v, p ) − Ω( p ) is concave in p , then D Φ Ω ( p, p (cid:48) ) is convex in p , as is the case of the usualBregman divergence D Ω ( p, p (cid:48) ) .
However, (19) is not easy to solve globally in general , as it is the
maximum of a difference of convex functions in v .
This can be done
approximately by using theconvex-concave procedure [72], linearizing the left part.
We have the opposite situation with thegeneralized Fenchel-Young loss if Φ( v, p ) − Ω( p ) is convex concave L ΦΩ ( v y ) is easy to compute
but it is a difference of convex functions in v .
• Similarly, we may also deﬁne D ΨΩ Φ
[Equation]
where we recall that p ΦΩ ( v ) :
= argmax p ∈C Φ( v, p ) − Ω( p ) .
We have thus obtained a divergence
between two objects v and v (cid:48) in the same space V
B Proofs
B.1 Proofs for Proposition 1 (properties of generalized conjugates)
Generalized Fenchel-Young inequality.
From (5), we immediately obtain Ω Φ ( v ) ≥ Φ( v, p ) − Ω( p ) for all v ∈ V and all p ∈ C .
Convexity.
If Φ( v, p ) is convex in v , then Ω Φ ( v ) is the maximum of a family of convex functions
indexed by p .
Therefore, Ω Φ ( v ) is convex.
Note that this is the case even if Ω( p ) is nonconvex.
Order reversing.
Since Ω ≤ Λ , we have
[Equation]
Continuity.
See [65, Box 1.8].
Gradient.
See “Assumptions for envelope theorems” in §4.
Smoothness.
We follow the proof technique of [46, Lemma 4.3], which states that v (cid:55)→max p ∈C E ( v, p ) is ( β + β 2 /γ )
-smooth if E is β -smooth in ( v, p ) and γ -strongly concave inp over C .
We show here that if E ( v, p ) decomposes as E ( v, p ) =
Φ( v, p ) − Ω( p ) , whereΩ is γ -strongly convex over C but possibly nonsmooth (as is the case for instance of Shannon’s negentropy), and Φ( v, p ) is β -smooth in ( v, p ) and concave in p , then we still have thatΩ Φ ( v ) =
max p ∈C Φ( v, p ) − Ω( p ) is ( β + β 2 /γ )
-smooth.
For brevity, let us deﬁne the shorthand p ΦΩ ( v ) :
= p (cid:63) ( v ) .
From [46, Section A.3], we haveγ
[Equation]
[Equation]
[Equation]
[Equation]
[Equation]
[Equation]
∇
[Equation]
E
[Equation]
, p
[Equation] [Equation]
for all v 1 , v 2 ∈ V .
With E ( v, p ) =
Φ( v, p ) − Ω( p ) , we obtainγ (cid:107)
[Equation]
−
[Equation] [Equation] [Equation] [Equation]
[Equation]
Since Φ is β -smooth, we have for all p ∈ C
[Equation]
Combined with the Holder inequality (cid:104) a, b
[Equation]
≤
[Equation]
a
[Equation]
b (cid:107) , this gives
[Equation]
Simplifying, we get
[Equation]
[Equation]
Therefore, p ΦΩ ( v ) = p (cid:63) ( v ) is β/γ -Lipschitz.
From Rockafellar’s envelope theorem [62, Theorem 10.31], ∇ Ω Φ ( v ) = ∇ 1 Φ( v, p (cid:63) ( v )) .
Since Φis β -smooth, we therefore have
[Equation]
, where we used (21) and βγ ≥ 1 .
A related result in the context of bilevel programming but with different assumptions and proof isstated in [31, Lemma 2.2].
More precisely, the proof of that result requires twice differentiabilityof E ( v, p ) =
Φ( v, p ) − Ω( p ) while we do not.
Moreover, applying that result to our setting wouldrequire E ( v, p ) to be smooth in ( v, p ) while we only assume Φ( v, p ) to be the case.
We emphasizeagain that Ω( p ) is nonsmooth when it is the negentropy.
B.2 Proofs for Proposition 2 (closed forms
Bilinear coupling.
This follows from
[Equation]
and similarly for p Ω ( v ) .2.
Linear-quadratic coupling.
Let us deﬁne the function
[Equation]
Its gradient is ∇ F ( p ) =
( γI − A ) p − b .
Setting ∇ F ( p (cid:63) )
= 0 , we obtain(
[Equation]
( γI [Equation] ) − 1 b, where we assumed that ( γI − A ) is positive deﬁnite.
We therefore get
[Equation]
p (cid:63) , b (cid:105) = − 1 2 (cid:104) p (cid:63) , b (cid:105)
= − 1 2 (cid:104) b, ( γI − A )
− 1 b (cid:105) .
Metric coupling.
From (8), it is easy to check that Ω = − Λ ⇔ Ω
Φ = − Λ C with C = − Φ .
From[60, Proposition 6.1], we have Λ C = − Λ .
Therefore, Ω Φ = − Ω .
B.3 Proofs for Proposition 3 (Properties of generalized Fenchel-Young losses)1.
Non-negativity.
This follows immediately from the generalized Fenchel-Young inequality.2.
Zero loss.
If p Φ Ω ( v ) =
y , then using L Φ Ω ( v, y )
=
F ( v, y ) − F ( v, p Φ Ω ( v )) , where F ( v, p ) =
Ω( p ) − Φ( v, p ) , we obtain L ΦΩ ( v, y )
= 0 .
Let’s now prove the reverse direction.
Since the maximumin (5) exists, we have F ( v, p ΦΩ ( v )) ≤ F ( v, y ) .
If L ΦΩ ( v, y ) = 0 , we have F ( v, p ΦΩ ( v ))
=
F ( v, y ) .
Since the maximum is unique by assumption, this proves that y = p ΦΩ ( v ) .
Gradient.
This follows directly from the deﬁnition (9).
Difference of convex (DC).
If Φ( v, p ) is convex in p , Ω Φ ( v ) is convex (Proposition 1).
Therefore,v (cid:55)→ Ω Φ ( v ) − Φ( v, p ) for all p ∈ C .5.
Smaller output set, smaller loss.
Let Ω: C → R and let Ω (cid:48) be the restriction of Ω to C (cid:48) ⊆
C , i.e.,Ω (cid:48) ( p ) :
= Ω + I C (cid:48) , where I C (cid:48) is the indicator function of C (cid:48) .
From (5), we have (Ω (cid:48) )
Φ ( v ) ≤ Ω Φ ( v )for all v ∈ V .
From (9), we therefore have L ΦΩ (cid:48) ( v, p ) ≤ L ΦΩ ( v, p ) for all v ∈ V and all p ∈ C (cid:48) .6.
Quadratic lower-bound.
If a function F is γ -strongly convex over C w.r.t.
a norm (cid:107) · (cid:107) , thenγ
[Equation]
(22) for all p, p (cid:48) ∈ C .
If C is a closed convex set, we also have that p (cid:63) =
argmin p ∈C F ( p ) satisﬁes theoptimality condition
[Equation]
F ( p (cid:63) ) , p − p (cid:63) (cid:105)
[Equation]
(23) for all p ∈ C [53, Eq.
Combining (22) with p (cid:48)
= p (cid:63) and (23), we obtain
[Equation]
Applying the above with F ( p )
=
Ω( p ) − Φ( v, p ) and using p (cid:63) =
p ΦΩ ( v ) , we obtain
[Equation]
Upper-bounds.
If F is α -Lipschitz over C with respect to a norm (cid:107) · (cid:107) , then for all p, p (cid:48) ∈ C
[Equation]
Applying the above with F ( p )
=
Ω( p ) − Φ( v, p ) and p (cid:48) =
p ΦΩ ( v ) , we obtain
[Equation]
If Φ( v, p ) is concave in p , then its linear approximation always lies above:
[Equation]
for all p, p (cid:48) ∈ C .
We then have
[Equation]
B.4 Proof of Proposition 4 (calibration)
Background.
The pointwise target risk of (cid:98) y ∈ Y according to q ∈ (cid:52) |Y| is
[Equation]
We also deﬁne the corresponding excess of pointwise risk, the difference between the pointwise riskand the pointwise Bayes risk:
[Equation]
We can then write the risk of f : X → Y in terms of the pointwise risk
[Equation]
Let us deﬁne the Bayes predictor
[Equation]
The Bayes risk is then
[Equation]
Combining the above, we can write the excess of risk of f : X → Y as
[Equation]
Similarly, with the generalized Fenchel-Young loss (9), the pointwise surrogate risk of v ∈ Vaccording to q ∈ (cid:52) |Y| is
[Equation]
and the excess of pointwise surrogate risk is
[Equation]
Let us deﬁne the surrogate risk of g : X → V by
[Equation]
and the corresponding Bayes predictor by
[Equation]
We can then write the excess of surrogate risk of g : X → V as
[Equation]
A calibration function [68] ξ : R + → R + is a function relating the excess of pointwise target risk andpointwise surrogate risk.
It should be non-negative, convex and non-decreasing on R + , and satisfyξ (0) = 0 .
Formally, given a decoder d : V → Y , ξ should satisfy
[Equation]
for all v ∈ V and q ∈ (cid:52) |Y| .
By Jensen’s inequality, this implies that the target and surrogate risks arecalibrated for all g : X → V [59, 56]
[Equation]
From now on, we can therefore focus on proving (24).
Upper-bound on the pointwise target excess risk.
We now make use of the afﬁne decomposition(12).
Let σ := sup y ∈Y (cid:107) V (cid:62) ϕ
( y ) (cid:107) ∗ , where (cid:107) · (cid:107) ∗ denotes the dual norm of (cid:107) · (cid:107) .
Let us deﬁne˜ y L ( u ) := argmin (cid:98) y ∈Y (cid:104) ϕ
( (cid:98) y ) , V u + b (cid:105) and µ ϕ ( q ) :
=
E Y ∼ q [ ϕ ( Y )]
∈ conv( ϕ ( Y )) ⊆
ϕ ( R k ) .
From [16, Lemma 2],δ(cid:96)
(˜ y L ( u ) , q ) ≤ 2 σ (cid:107) µ ϕ ( q ) − u
(cid:107) ∀ u ∈ ϕ
( R k ) , q ∈ (cid:52) |Y| .
Using y L ( p ) =
˜
y L ( u ) with u = ϕ ( p ) , we thus getδ(cid:96) ( y L ( p ) , q ) ≤ 2 σ (cid:107) µ ϕ ( q ) − ϕ
( p ) (cid:107) ∀ p ∈ R k ,
[Equation]
[Equation]
Bound on the pointwise surrogate excess risk (bilinear case).
To highlight the difference withour proof technique, we ﬁrst prove the result in the bilinear case, assuming Ω is γ -strongly convex.
We follow the same proof technique as [56, 16] but unlike these works we do not require any
Legendre-type assumption on Ω .
If Φ( v, p ) =
(cid:104) v, ϕ ( p ) (cid:105) =
(cid:104) v, p (cid:105) , we have
[Equation]
where, when ϕ ( y ) =
y , we denote µ ( q ) :=
E Y ∼ q
[ Y ] ∈ conv( Y ) ⊆
R k for short.
The quantity
E Y ∼ q Ω( Y ) − Ω( µ ( q )) is called Bregman information [7] or Jensen gap, and is non-negative if Ω isconvex.
This term cancels out in the excess of pointwise surrogate risk
[Equation]
Since the Fenchel-Young loss achieves its minimum at 0 , we have
[Equation]
Therefore, the excess of pointwise surrogate risk can be written in Fenchel-Young loss form.
By thequadratic lower-bound (10) and the upper-bound (25), we have
[Equation]
Therefore the calibration function with the decoder d = y L ◦ p Ω is
[Equation]
Bound on the pointwise surrogate excess risk (linear-concave case).
We now prove the boundassuming that L ΦΩ is smooth and Φ( v, p ) =
(cid:104) v, ϕ ( p ) (cid:105) .
This includes the previous proof as specialcase because when Φ( v, p ) = (cid:104) v, p (cid:105) , then L ΦΩ ( v, y ) =
L Ω ( v, y ) is 1 γ -smooth in v if and only if Ω( p )is γ -strongly convex in p (cf.
By Theorem 4.22 in [57], if a function f ( v ) is M -smooth in v w.r.t.
the dual norm (cid:107) · (cid:107) ∗ and isbounded below, then
[Equation]
With f ( v ) =
(cid:96) ΦΩ ( v, q ) =
E Y ∼ q L Ω ( v, Y ) , which is non-negative, we obtain
[Equation]
We therefore have
[Equation]
Therefore the calibration function with the decoder d = y L ◦ p ΦΩ is
[Equation]
B.5 Proofs for Proposition 5 (Properties of generalized biconjugates)
The proofs are similar to that for C -transforms [65, Proposition 1.34].1.
Lower bound.
Let Ω: C → R .
We have
[Equation]
Therefore, Ω ΦΨ ( p ) ≤ Ω( p ) for all p ∈ C .
Analogously, for a function Λ: V → R , we haveΛ ΨΦ ( v ) ≤ Λ( v ) for all v ∈ V .2.
Equality.
Since Ω is Ψ -convex, there exits Λ: V → R such that Ω = Λ Ψ .
We then haveΩ Φ = Λ ΨΦ .
Using the lower bound property, we get Ω Φ ( v ) =
Λ ΨΦ ( v ) ≤ Λ( v ) for all v ∈ V .
Bythe order reversing property, we have Ω ΦΨ ( p ) ≥ Λ Ψ ( p ) =
Ω( p ) for all p ∈ C .
However, we alsohave Ω ΦΨ ( p ) ≤ Ω( p ) for all p ∈ C .
Therefore, Ω ΦΨ ( p ) = Ω( p ) for all p ∈ C .3.
Tightest lower-bound.
Let Ω (cid:48) be any lower bound of Ω that is Ψ -convex.
Therefore, there existsΛ:
V → R such that Ω (cid:48) ( p ) =
Λ Ψ ( p ) ≤ Ω( p ) for all p ∈ C .
By the order reversing property, wehave Λ ΨΦ ( v ) ≥ Ω Φ ( v ) for all v ∈ V .
By the lower bound property, we also have Λ ΨΦ ( v ) ≤ Λ( v )for all v ∈ V and therefore, Λ( v ) ≥ Ω Φ ( v ) for all v ∈ V .
Applying the order reversing propertyonce more, we get Ω (cid:48) ( p ) =
Λ Ψ ( p ) ≤ Ω ΦΨ ( p ) for all p ∈ C .
Therefore Ω ΦΨ is the tightest lowerbound of Ω .
B.6 Proofs for Proposition
(Properties of generalized Bregman divergences)
Link with generalized Fenchel-Young losses.
From (16) and (17), we have
[Equation]
for any Λ: V → R .
With Λ = Ω Φ , if Ω is Φ -convex, using Proposition 5, we have
[Equation]
Plugging Ω( p (cid:48) ) in (18) and using the shorthand v := v ΨΩ Φ ( p (cid:48) ) , we get
[Equation]
Non-negativity.
This follows
directly
from the non-negativity of L ΦΩ ( v p
Identity of indiscernibles .
If p = p (cid:48) , we immediately obtain D ΦΩ ( p, p (cid:48) )
=
from (18).
Let’sprove the reverse direction.
If D ΦΩ ( p, p (cid:48) )
= 0 , we have F ( v ΨΩ Φ ( p (cid:48) ) , p ) =
F ( v ΨΩ Φ ( p (cid:48) ) , p (cid:48) )
where
F ( v, p ) =
Ω( p ) − Φ( v, p ) .
Since by assumption F is strictly convex in p ,
we obtain p = p (cid:48) .
Convexity.
This follows from D ΦΩ ( p, p (cid:48) )
=
Ω( p ) − Φ( v, p ) + const.
Recovering Bregman divergences.
If Φ( v, p ) =
(cid:104) v, p (cid:105) , we have
[Equation]
Plugging back in (18), we obtain the Bregman divergence (20).
C Experimental details
C.1 Multilabel classiﬁcation
Datasets.
We used public datasets available at https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/ .
Dataset statistics are summarized in Table 3.
For all datasets, wenormalize samples to have zero mean unit variance.
[Table]
Experimental details.
In all experiments, we set the activation σ to relu ( a ) :
= max { 0 , a } .
For the unary model, we use a neural network with one hidden-layer, i.e., g θ ( x ) =
W 2 σ ( W 1 x + b 1 )+b 2 , where θ =
( W 2 , b 2 , W 1 , b 1 ) , W 2 ∈ R k × m , b 2 ∈ R k , W 1 ∈ R m × d , b 1 ∈ R m , and m is the numberof hidden units.
We use the heuristic m =
min { 100 , d/ 3 } , where d is the dimensionality of x .
For the pairwise model, in order to obtain a negative semi-deﬁnite matrix U , we parametrize
U = − AA (cid:62) with A =
[ W 1 x + b 1 , . .
, W m x + b m ] , where W j ∈ R k × d and b j ∈ R k
In ourexperiments, we choose a rank-one model, i.e., m = 1 .
Note that we use distinct parameters for theunary and pairwise models but sharing parameters would be possible.
For the SPEN model, following [11, Eq. 4 and 5], we set the energy to Φ( v, p ) =
(cid:104) u, p (cid:105) − Ψ
( w, p ) ,where v = ( u, w ) , u
= g θ ( x ) and w are the weights of the “prior network” Ψ (independent of x ).
Weparametrize Ψ( w, p ) =
W 2 σ ( W 1 p + b 1 ) + b 2 , where w = ( W 2 , b 2 , W 1 , b 1 ) , W 2 ∈ R 1 × m , b 2 ∈ R 1and W 1 ∈ R m × k , b 1 ∈ R m .
To further impose convexity of Ψ in p , W 2 needs to be non-negative.
Todo so without using constrained optimization, we use the change of variable W 2 =
softplus ( W (cid:48) 2 ) ,where softplus ( a ) :
= log(1 + exp( a )) is used as an element-wise bijective mapping.
Additional results.
The gradient of Ω Φ ( v ) can be computed using the envelope theorem (Proposition 1), which does not require to differentiate through p ΦΩ ( v ) .
Alternatively, since Ω Φ ( v ) =
Φ( v, p ΦΩ ( v )) − Ω( p ΦΩ ( v )) , we can also compute the gradient of Ω Φ ( v ) by using the implicit functiontheorem, differentiating through p ΦΩ ( v ) .
To do so, we use the approach detailed in [17].
Results in
Table 4 show that the envelope theorem performs comparably to the implicit function theorem, if notslightly better.
Differentiating through p Φ Ω ( v ) using the implicit function theorem requires to solve ak × k system and its implementation is more complicated than the envelope theorem.
Therefore, wesuggest to use the envelope theorem in practice.
accuracy in %).
yeast scene mediamill birds emotions cal
Envelope theorem 80.
Implicit function theorem 80.
In addition, we also compared the proposed generalized Fenchel-Young loss with the energy loss, thebinary cross-entropy loss and the generalized perceptron loss, which corresponds to setting Ω( p ) = 0 .
As explained in §3, the cross-entropy loss requires to differentiate through p ΦΩ ( v ) ; we do so by implicitdifferentiation.
Table 5 shows that the generalized Fenchel-Young loss outperforms these losses.
As expected, the energy loss performs very poorly, as it can only push the model in one direction[44].
Using regularization Ω , as advocated in this paper, is empirically conﬁrmed to be beneﬁcial foraccuracy.
[Table]
C.2 Imitation learning
[Table]
Experimental details.
We run a hyperparameter search over the learning rate of the ADAMoptimizer, the number of hidden units in the layers, the weight of the L2 parameters regularizationterm and the scale of the energy regularization term Ω .
We run the hyperparameter search for4 demonstration trajectories and select the best performing ones based on the ﬁnal performance(averaged over 3 seeds).
[Table]
Environments.
We also provide performance of the expert agent as detailed by Orsini et al.
[58] aswell as the description of the observation and action spaces for each environment.
[Table]