Learning Energy Networks with Generalized Fenchel-Young Losses
Abstract
Energy-based models, a.k.a.
energy networks, perform inference by optimizing an energy function, typically parametrized by a neural network.
This allows one to capture potentially complex relationships between inputs and outputs.
To learn the parameters of the energy function, the solution to that optimization problem is typically fed into a loss function.
The key challenge for training energy networks lies in computing loss gradients, as this typically requires argmin/argmax differentiation.
In this paper, building upon a generalized notion of conjugate function, which replaces the usual bilinear pairing with a general energy function, we propose generalized Fenchel-Young losses, a natural loss construction for learning energy networks.
Our losses enjoy many desirable properties and their gradients can be computed efﬁciently without argmin/argmax differentiation.
We also prove the calibration of their excess risk in the case of linear-concave energies.
We demonstrate our losses on multilabel classiﬁcation and imitation learning tasks.
Introduction
Training a neural network usually involves ﬁnding its parameters by minimizing a loss function, which captures how well the network ﬁts the data.
A typical example is the multiclass logistic loss (a.k.a.
cross-entropy loss) from logistic regression, which is the canonical loss associated with the softmax output layer and the categorical distribution.
If we replace the softmax with another output layer, what loss function should we use instead?
In generalized linear models [52, 48], which include logistic regression and Poisson regression as special cases, the negative log-likelihood gives a loss function associated with a link function, generalizing the softmax to other members of the exponential family [8].
More generally, Fenchel-Young losses [18] provide a generic way to construct a canonical convex loss function if the associated output layer can be written in a certain argmax form.
Besides the aforementioned generalized linear models, models that fall in this family include sparsemax [47], the structured perceptron [27] and conditional random ﬁelds [43, 69].
However, the theory of
Fenchel-Young losses is currently limited to argmax output layers that use a bilinear pairing.
To increase expressivity, energy-based models [44], a.k.a.
energy networks , perform inference by optimizing an energy function , typically parametrized by a neural network.
This leads to an inner optimization problem, which can capture potentially complex relationships between inputs and outputs.
A similar approach is taken in SPENs (Structured Prediction Energy Networks)
[11, 13], in which a continuous relaxation of the inner optimization problem is used, amenable to projected gradient descent or mirror descent.
In input-convex neural networks [4], energy functions are restricted to be convex, so as to make the inner optimization problem easy to solve optimally.
To learn the parameters of the energy function, the solution of the inner optimization problem is typically fed into a loss function, leading to an outer optimization problem.
In order to solve that problem by stochastic gradient descent, the main challenge lies in computing the loss gradients.
Indeed, when using an arbitrary loss function, by the chain rule, computing gradients requires differentiating through the inner optimization problem solution, often referred to as argmin or argmax differentiation .
This can be done by backpropagation through unrolled algorithm iterates [13] or implicit differentiation through the optimality conditions [4].
This issue can be circumvented by using a generalized perceptron loss [44] or a max margin loss [11].
These losses only require max differentiation , for which envelope theorems [49] can be used.
However, these losses often fail to satisfy envelope theorem assumptions such as unicity of the solution and lack theoretical guarantees.
In this paper, we propose to extend the theory of Fenchel-Young losses [18], so as to create a canonical loss function associated with an energy network.
Our proposal builds upon generalized conjugate functions , also known as Fenchel-Moreau conjugates [50, 62], allowing us to go beyond bilinear pairings and to support general energy functions such as neural networks.
We introduce regularization , allowing us to obtain unicity of the solutions and to use envelope theorems to compute gradients, without argmin or argmax differentiation.
We provide novel guarantees on the calibration of the excess risk, when our loss is used as a surrogate for another discrete loss.
To sum up, we obtain a wellmotivated loss construction for general energy networks.
The rest of the paper is organized as follows.
• After providing some background (§2), we describe regularized energy networks , identify a classiﬁcation of energy functions and give several possible examples that fall in this family (§3).
• We deﬁne generalized conjugates , an extension of Legendre-Fenchel conjugates, and state their properties (§4).
We establish novel conditions for a generalized conjugate to be a smooth function.
• We then introduce generalized Fenchel-Young losses and show that they enjoy many of the same favorable properties as the regular Fenchel-Young losses (§5).
On the theoretical side, we prove calibration of the excess risk for linear-concave energies and strongly-convex regularizers (§6).
• We demonstrate our losses on multilabel classiﬁcation and imitation learning tasks (§7).
Background
Convex conjugates.
Let C ⊆
R k be an output set.
Let Ω: C → R be a function, such that Ω( p )
= ∞ for all p (cid:54)∈ C , i.e., dom(Ω) =
C .
Given v ∈ V ⊆ R k , we deﬁne the convex conjugate [22] of Ω , also known as the Legendre-Fenchel transform of Ω , by
[Equation]
The conjugate Ω ∗ is always convex, even if Ω is not.
We deﬁne the corresponding argmax as
[Equation]
The latter equality follows from Danskin’s theorem [29, 15], under the assumption that Ω is strictly convex (otherwise, we obtain a subgradient).
It is well-known that Ω ∗ is 1 γ -smooth w.r.t.
the dual norm (cid:107) · (cid:107) ∗ if and only if Ω is γ -strongly convex w.r.t.
the norm (cid:107) · (cid:107) [37, 39, 10, 75].
Fenchel-Young losses.
Suppose that v = g θ ( x
) ∈ R k are the logits / scores produced by a neural network g , where x ∈ X ⊆ R d and θ ∈ Θ are the network’s input features and parameters, respectively.
What loss function should we use if we want to use (2) as output layer?
The FenchelYoung loss [18] generated by Ω: C
→ R provides a natural solution.
It is deﬁned by
[Equation]
where y ∈ Y ⊆
C is the ground-truth label.
Earlier instances of this loss were independently proposed in the contexts of ranking [2] and multiclass classiﬁcation [30].
Among many useful properties, this loss satisﬁes L Ω ( v, y ) ≥ 0 and L Ω ( v, y ) = 0 ⇔ y = p Ω ( v ) if Ω is strictly convex [18].
In that sense, it is the canonical loss associated with (2).
Interestingly, many existing loss functions are recovered as special cases.
For instance, if C = R k and Ω( p ) = 12 (cid:107) p (cid:107) 22 , which is 1 -strongly convex w.r.t.
(cid:107) · (cid:107)over R k , then we obtain the self-dual, the identity mapping and the squared loss:
[Equation]
If C is the probability simplex (cid:52) k := { p ∈ R k
+ : (cid:80) ki
=1 p i = 1 } and Ω( p ) is the scaled Shannon negentropy γ (cid:104) p, log p (cid:105) , which is γ -strongly convex w.r.t.
(cid:107) · (cid:107) 1 over (cid:52) k , then we obtain Ω ∗ ( v [Equation]
= LSE γ ( v ) ,
[Equation]
where we used the log-sum-exp LSE γ ( v ) :
= γ log( (cid:80) ki
=1 exp( v i /γ )) , softmax γ ( v ) ∝ exp( v/γ ) , and the Kullback-Leibler divergence.
More generally, when C = conv( Y ) , the convex hull of Y , p Ω corresponds to a projection, for which efﬁcient algorithms exist in numerous cases [18, 16].
The calibration of the excess risk of Fenchel-Young losses when Ω is strongly convex was established in [56, 16].
However, the theory of Fenchel-Young losses is currently limited to the bilinear pairing (cid:104) v, p (cid:105) , which restricts their expressivity and scope.
Regularized energy networks
Energy networks.
Also known as energy-based models or EBMs [44], of which SPENs [11, 13] are a particular case, these networks compute predictions by solving an optimization problem of the form
[Equation]
where v = g θ ( x ) ∈ V is the energy network’s input, Φ( v, p ) is a scalar-valued energy function, and
C is an output set.
Throughout this paper, we use the convention that higher energy indicates higher degree of compatibility between v and the prediction p .
Since Φ( v, p ) is a general energy function, we emphasize that v and p do not need to have the same dimensions, unlike with the bilinear pairing (cid:104) v, p (cid:105) .
Any neural network g θ ( x ) ∈ R k can be written in energy network form, since argmax
p ∈ R k −(cid:107)
g θ ( x ) − p (cid:107) 2
= g θ ( x ) .
The key advantage of energy networks, however, is their ability to capture complex interactions between inputs and outputs.
This is especially useful in the structured prediction setting [6], where predictions are made of sub-parts, such as sequences being composed of individual elements.
Introducing regularization.
In this paper, we compute predictions by solving
[Equation]
where we further added a regularization function Ω: C → R .
We call Φ( v, p ) − Ω( p ) a regularized energy function and we call the corresponding model a regularized energy network .
We obviously recover usual energy networks by setting Ω to the indicator function of the set C , i.e., Ω( p ) = {+
 if p+} ∈ C , ∞ otherwise.
While it is in principle possible to absorb Ω into Φ , keeping Ω explicit has several advantages: 1) it allows to introduce generalized conjugate functions and their properties (§4) 2) it allows to introduce generalized Fenchel-Young losses (§5), which mirror the original
Fenchel-Young losses 3) we obtain closed forms for (4) in certain cases (Table 1).
Solving the maximization problem.
The ability to solve the maximization problem in (4) efﬁciently depends on the properties of Φ( v, p ) w.r.t.
p .
If Φ( v, p ) is linear in p , for instance Φ( v, p )
= (cid:104) v, Up (cid:105) , then we obtain p ΦΩ ( v ) =
p Ω ( U (cid:62) v ) .
Thus, the computation of (4) reduces to (2), for which closed forms are often available.
If Φ( v, p ) is concave in p and C is a convex set, then we can solve (4) in polynomial time using an iterative algorithm, such as projected gradient ascent.
This is the most general energy class for which our loss and its gradient can be computed to arbitrary precision.
If Φ( v, p ) is nonconcave in p , then it is not possible to solve (4) in polynomial time in general, unless C is a discrete set of small cardinality.
In general, we emphasize that Φ( v, p ) can be nonconvex in v , as is typical with neural networks, since the maximization problem in (4) is with respect to p ∈ C .
However, in the sequel of this paper, certain properties we establish require Φ( v, p ) to be convex in v .
We now give examples of regularized energy networks in increasing order of expressivity / complexity.
Generalized linear models.
As a warm up, we consider the case of bilinear energy Φ( v, p ) .
For instance, for probabilistic classiﬁcation with k classes, where the goal is to predict p ∈ (cid:52) k from x ∈ R d , we can use Φ( v, p ) =
(cid:104) v, p (cid:105) with v =
Wx + b , where W ∈ R k × d and b ∈ R k .
In this case, we therefore obtain p ΦΩ ( v ) =
p Ω ( Wx + b ) , recovering generalized linear models [52, 48], the structured perceptron [27] and conditional random ﬁelds [43, 69].
Rectiﬁer and maxout networks.
We now consider the case of convex-linear energy Φ( v, p ) .
A ﬁrst example is a rectiﬁer network [33] with one hidden layer.
Indeed, Φ( v, p ) =
(cid:104) σ ( v ) , Up (cid:105) is convex in v if σ is an element-wise, convex and non-decreasing activation function, such as the relu or softplus, and if U and p are non-negative.
A second example is a maxout network [34] (i.e., a max-afﬁne function) or its smoothed counterpart, the log-sum-exp network [24].
Indeed, Φ( v, p ) =
σ ( v ) · p , where σ is the max or log-sum-exp operator, is convex in v , if the scalar p is non-negative.
In general, it is always possible to construct a convex-concave energy from a jointly-convex function using the Legendre-Fenchel transform in the ﬁrst argument.
[Table]
Input-convex neural networks.
As an example of nonconvex-concave energy, we consider Φ( v, p ) = − ICNN ( v, p ) , where ICNN is an input-convex neural network [4], i
[Table]
e., it is convex in p but can be nonconvex in v .
If Ω( p ) and C are convex, then Φ( v, p ) is concave in p and (4) can be solved in polynomial time using an iterative algorithm, such as projected gradient ascent.
Probabilistic energy networks.
It is often desirable to deﬁne a conditional probability distribution
[Equation]
Such networks are typically trained using a cross-entropy loss (or equivalently, negative loglikelihood).
Unfortunately, when Y is large or inﬁnite, this loss and its gradients are intractable to compute, due to the normalization constant.
Therefore, recent research has been devoted to developing approximate training schemes [67, 35, 51].
Although this is not the focus of this paper, we point out that probabilistic energy networks can also be seen as regularized energy networks in the space of probability distributions
C = (cid:52) |Y| , if we set Φ( v, p ) =
(cid:80) y ∈Y p ( y ) E ( v, y ) and Ω( p ) =
(cid:80) y ∈Y p ( y ) log p ( y ) .
While the resulting optimization problem is intractable in general, it does suggest possible approximation schemes, such as the use of Frank-Wolfe methods [12, 42, 55].
Existing loss functions for energy networks.
Given a pair ( x, y ) and output v = g θ ( x ) , how do we measure the discrepancy between p ΦΩ ( v ) and y ?
One possibility [4, 13] is to use the composition of a differentiable loss L : C × Y → R + with the argmax output, namely ( v, y )
(cid:55)→ L ( p ΦΩ ( v ) , y ) .
However, computing the loss gradient w.r.t.
v then requires to differentiate through p ΦΩ ( v ) , either through unrolling or implicit differentiation.
This is particularly problematic when C is a complicated convex set, as differentiating through a projection can be challenging.
In contrast, our proposed loss completely circumvents this need and enjoys easy-to-compute gradients.
For unregularized energy networks, a naive idea, called the energy loss, is to use ( v, y )
(cid:55)→ − Φ( v, y ) .
However, this loss only works well if Φ is a similarity measure and works poorly in general [44].
A better choice is the generalized perceptron loss ( v, y )
(cid:55)→ max p ∈C Φ( v, p ) − Φ( v, y ) [44].
Our loss can be seen as a principled generalization of this loss to regularized energy networks, with theoretical guarantees.
Generalized conjugates
In order to devise generalized Fenchel-Young losses, we build upon a generalization due to Moreau [50, Chapter 14] of the convex conjugate Ω ∗ .
Denoted Ω Φ , it replaces the bilinear pairing in (1) with a more general coupling Φ [62, Chapter 11, Section L].
In this section, we state their deﬁnition, properties, closed-form expressions and connection to the C -transform in optimal transport.
Deﬁnition.
Let Φ( v, p ) ∈
R be a coupling / energy function.
The Φ -convex conjugate of Ω: C →
R , also known as Fenchel-Moreau conjugate , is then deﬁned by the value function
[Equation]
The Φ -convex conjugate is an important tool in abstract convex analysis [66, 63].
Recently, it has been used to provide “Bellman-like” equations in stochastic dynamic programming [25] and to provide tropical analogues of reproducing kernels [5].
We assume that the maximum is feasible for all v ∈ V = R d , meaning that dom(Ω Φ ) =
V .
We emphasize again that, unlike with (1), v and p do not need to have compatible dimensions.
We denote the argmax solution corresponding to (5) by
[Equation]
If a function F ( v ) can be written as F ( v ) =
Ω Φ ( v ) for some Ω , it is called Φ -convex (in analogy, a function f ( v ) is convex and closed if and only if it can be written as f ( v ) =
Ω ∗ ( v ) for some Ω ).
Properties.
Φ -convex conjugates enjoy many useful properties, some of them are natural extensions of the usual convex conjugate properties.
Proofs are provided in Appendix B.1.
Proposition 1 (Properties of Φ -convex conjugates) .
Let Ω: C → R and Φ:
V × C → R .
Generalized Fenchel-Young inequality: for all v ∈ V and p ∈ C , [Equation]
Convexity: If Φ( v, p ) is convex in v , then Ω Φ ( v ) is convex (even if Ω( p ) is nonconvex).
Order reversing: if Ω( p ) ≤ Λ( p ) for all p ∈ C , then Ω Φ ( v ) ≥ Λ Φ ( v ) for all v ∈ V . 4.
Continuity: Ω Φ shares the same continuity modulus as Φ . 5.
Gradient (envelope theorem): Under assumptions (see paragraph below), we have ∇ Ω Φ ( v ) = ∇ 1 Φ( v, p ΦΩ ( v )) , where ∇ 1 denotes the gradient in the ﬁrst argument.
Smoothness:
If C is a compact convex set, Φ( v, p ) is β -smooth in ( v, p ) , concave in p and Ω( p ) is γ -strongly convex in p , then Ω Φ ( v ) is ( β + β 2 /γ ) -
smooth and p ΦΩ ( v ) is β/γ -Lipschitz.
The condition on Φ and Ω in item 6 for Ω Φ to be a smooth function (i.e., with Lipschitz-continuous gradients) is a novel result and will play a crucial role for establishing calibration guarantees in §6.
Assumptions for envelope theorems.
The expression in item 5 allows to compute ∇ Ω Φ ( v ) without argmax differentiation.
It is based on envelope theorems, which can be used for max differentiation.
Indeed, we have Ω Φ ( v ) =
max p ∈C F ( v, p ) , where F ( v, p ) :
=
Φ( v, p ) − Ω( p ) .
We assume that the maximum is unique and C is a compact.
If F ( v, p ) is convex in v , we apply Danskin’s theorem [29] [15, Proposition B.25].
Without convexity assumption in v , if F ( v, p ) is continuously differentiable in v for all p ∈ C , ∇ 1 F is continuous, we apply [62, Theorem 10.31].
When we do not compute the exact solution of (6), we only obtain an approximation of the gradient ∇ Ω Φ ( v ) ; see [1] for approximation guarantees.
For other envelope theorem usecases in machine learning, see, e.g., [21].
Closed forms.
While (5) and (6) may need to be solved numerically in general, they enjoy closedform expressions in simple cases.
Proofs are provided in Appendix B.2.
Proposition 2 (Closed-form expressions) .
Let Ω: C → R and Φ:
V × C → R .
Bilinear coupling: If Φ( v, p ) = (cid:104) v, Up (cid:105) , then Ω Φ ( v ) =
Ω ∗ ( U (cid:62) v ) and p Φ Ω ( v ) =
p Ω ( U (cid:62) v ) .
Linear-quadratic coupling: If C = R k , Ω( p ) =
γ 2 (cid:107) p (cid:107) 22 and Φ( v, p ) = 12 (cid:104) p, Ap (cid:105) + (cid:104) p, b (cid:105) , where v =
( A, b ) and A is such that ( γI − A ) is positive deﬁnite, we obtain
[Equation]
Metric coupling: If V = C , Φ = − C where C ( v, p ) is a metric and Ω is M -Lipschitz with M ≤ 1 , then Ω Φ = − Ω and p ΦΩ ( v )
= v .
Relation with the C -transform.
Given a cost function C , we may deﬁne a min counterpart of (5),
[Equation]
In the optimal transport literature, this is known as the C -transform of Λ [65, 60].
When C is bilinear, this recovers the notion of concave conjugate [20].
When C ( v, p ) =
c ( v − p ) for some c , this recovers the inﬁmal convolution [20].
It is easy to check that Ω = − Λ ⇔ Ω Φ = − Λ C with C = − Φ .
Thus, Ω Φ and Λ C are the natural extensions of convex and concave conjugates, respectively.
We opt for the former in this paper to closely mirror the usual convex conjugates and Fenchel-Young losses.
Generalized Fenchel-Young losses
Deﬁnition.
The generalized Fenchel-Young inequality in Proposition 1 leads us to propose the generalized Fenchel-Young loss, the natural extension of (3) to Φ -convex conjugates:
[Equation]
We also have the relationships L ΦΩ ( v, y )
=
F ( v, y ) − F ( v, p ΦΩ ( v )) , where F ( v, p ) =
Ω( p ) − Φ( v, p ) , and p ΦΩ ( v ) = argmin p ∈C L ΦΩ ( v, p ) .
We now give an intuitive geometric interpretation of (9).
[Figure]
[Equation]
[Figure]
Geometric interpretation.
For the sake of illustration, let us set Ω( p ) =
γ 2 p 2 and Φ( v, p ) =
12 ap 2 + bp .
From (1), the line p (cid:55)→ (cid:104) u, p (cid:105) − Ω ∗ ( u ) , where Ω ∗ ( u ) = 12 γ u 2 , is the tightest linear lower bound on Ω( p ) , here depicted with u = 0 .
It is the tangent of Ω( p ) at p =
p Ω ( u ) .
Similarly, from (5), p (cid:55)→ Φ( v, p ) − Ω Φ ( v ) is the tightest lowerbound of this form given v .
It touches Ω( p ) at p = p ΦΩ ( v ) , here depicted with v = ( a, b ) , a = −and b = 1 , for which we have the closed form Ω Φ ( v ) =
12 ( γ − a ) − 1 b 2 (Proposition 2).
The generalized Fenchel-Young loss (9) is then the gap between Ω( p ) and Φ( v, p ) − Ω Φ ( v ) , evaluated at the groundtruth p = y .
The goal of training is to adjust the parameters θ of a network v = g θ ( x ) so as to minimize this gap averaged over all ( x, y ) training pairs.
Properties.
Generalized Fenchel-Young losses enjoy many desirable properties, as we now show.
Proposition 3 (Properties of generalized F-Y losses) .
Let Ω: C → R and Φ: V × C → R .
Non-negativity: L ΦΩ ( v, p ) ≥ 0 for all v ∈ V and p ∈ C . 2.
Zero loss: if the maximum in (5) exists and is unique, L ΦΩ ( v, y ) = 0 ⇔ y =
p ΦΩ ( v ) .
Gradient: ∇ 1 L ΦΩ ( v, p ) =
∇ Ω Φ ( v ) − ∇ 1 Φ( v, p ) .
Difference of convex (DC): if Φ( v, p ) is convex in v , then L ΦΩ ( v, p ) is a difference of convex functions in v : Ω Φ ( v ) and Φ( v, p ) .
If Φ( v, p ) is linear in v , then L ΦΩ ( v, p ) is convex in v .
Smaller output set, smaller loss.
If C (cid:48) ⊆ C and Ω (cid:48) is the restriction of Ω to C (cid:48) , then L ΦΩ (cid:48) ( v, p ) ≤ L ΦΩ ( v, p ) for all v ∈ V and p ∈ C (cid:48) .
Quadratic lower-bound.
If Φ( v, p ) − Ω( p ) is γ -strongly concave in p w.r.t.
(cid:107) · (cid:107) over C and C is a closed convex set, then for all v ∈ V and p ∈ C
[Equation]
Upper-bounds.
If Φ( v, p ) − Ω( p ) is α -Lipschitz in p w.r.t.
(cid:107) · (cid:107) over C , then L ΦΩ ( v, p ) ≤ α (cid:107) p − p ΦΩ ( v ) (cid:107) .
If Φ( v, p ) is concave in p , then L ΦΩ ( v, p ) ≤
L Ω ( ∇ 2 Φ( v, p ) , p ) .
Proofs are given in Appendix B.3.
The non-negativity is a good property for a loss function.
The zero-loss property L ΦΩ ( v, y ) = 0 ⇔ y = p ΦΩ ( v ) , which is true for instance if p (cid:55)→ Φ( v, p ) − Ω( p ) is strictly concave, is key as it allows to use p ΦΩ ( v ) deﬁned in (6) as the (implicit) output layer associated with an energy network.
The gradient ∇ 1 L ΦΩ ( v, y ) does not require to differentiate through the argmax problem in (6) needed for computing p ΦΩ ( v ) .
Typically, differentiating through an argmax or argmin, as is done in input-convex neural networks [4], requires either unrolling or implicit differentiation [36, 14, 41, 19, 17] and is therefore more costly.
The fact that L ΦΩ ( v, p ) is a difference of convex (DC) functions in v when Φ( v, p ) is convex in v suggests that we can use DC programming techniques, such as the convex-concave procedure [72], for training such energy networks.
We leave the investigation of this observation to future work.
The “smaller output set, smaller loss” property means that we can achieve the smallest loss by choosing the smallest set C in (6) such that Y ⊆ C .
The smallest such convex set is the convex hull of Y , also known as marginal polytope [71] when Y ⊆ { 0 , 1 } k .
The quadratic lower-bound relates our loss to using p ΦΩ ( v ) within a squared norm loss.
The upper-bounds relate our loss to using p ΦΩ ( v ) within a norm loss and to using a regular Fenchel-Young loss with a linearized energy.
If Ω is the indicator function of C , i.e., Ω( p ) = 0 if p ∈ C , ∞ otherwise, then it can be checked that we recover the “generalized perceptron” loss [45, 44] as a special case of (9).
Proposition 3 therefore provides new properties to understand and analyze this loss.
Regular Fenchel-Young losses are closely related to Bregman divergences [18].
In Appendix A, we build a generalized notion of Bregman divergence using generalized conjugates.
Training.
To train regularized energy networks with our framework, the user should choose an energy Φ( v, p ) , a regularization Ω( p ) , an output set C (from Proposition 3 item 5, the smaller this set the better) and the model v = g θ ( x ) with input x and parameters θ .
Given a set of input-output pairs ( x 1 , y 1 ) , . . .
, ( x n , y n ) ∈ X × Y , where Y ⊆ C , we can ﬁnd the parameters θ by minimizing the empirical risk objective regularized by R : Θ → R ,
[Equation]
[Equation]
Thanks to the easy-to-compute gradients of L ΦΩ , we can easily solve (11) using any (stochastic) solver.
Calibration guarantees
Many times, notably for differentiability reasons, the loss used at training time, here our generalized
Fenchel-Young loss L ΦΩ ( v, y ) , is used as a surrogate / proxy for a different (potentially discrete loss)
L : Y × Y → R , used at test time.
Calibration guarantees [73, 74, 9, 68] ensure that minimizing the excess of risk of the train loss will also minimize that of the test loss (a.k.a.
target loss).
We study in this section such guarantees, assuming that L satisﬁes an afﬁne decomposition property [26, 16]:
[Equation]
where ϕ ( y ) (cid:55)→ V ϕ
( y )
+ b is an afﬁne map, ϕ
( y ) is a label embedding and c ( y ) is any function that depends only on y .
Numerous losses can be written in this form.
Examples include the zero-one,
Hamming, NDCG and precision at k losses [56, 16].
Inference in this setting works in two steps.
First, we compute a “soft” (continuous) prediction p = p ΦΩ ( v ) ∈ C .
Second, we compute a “hard” (discrete) prediction by a decoding / rounding from C to Y , calibrated for the loss L :
[Equation]
The target risk of f : X → Y and the surrogate risk of g : X → V are deﬁned by
[Equation]
∼ [Equation] L ( f ( X ) , Y ) and L ΦΩ ( g ) := E ( X,Y ) ∼ ρ L ΦΩ ( g ( X ) , Y ) , where ρ is a typically unknown distribution over X × Y .
The Bayes predictors are deﬁned by f (cid:63) :
= argmin f : X→Y L ( f ) and g (cid:63) :
= argmin g : X→V L ΦΩ ( g ) .
We now establish calibration of the surrogate excess risk, under the assumption that the energy Φ( v, p ) is linear-concave , i.e., it can be written as Φ( v, p )
=
(cid:104) v, ϕ ( p ) (cid:105) , for some function ϕ .
Proposition 4.
Calibration of target and surrogate excess risks
Assume L ΦΩ ( v, y ) is M -smooth in v w.r.t.
the dual norm (cid:107) · (cid:107) ∗ , C is a compact convex set such that Y ⊆ C and Φ( v, p ) =
(cid:104) v, ϕ ( p ) (cid:105) .
Let σ := sup y ∈Y (cid:107) V (cid:62) ϕ
( y ) (cid:107) ∗ .
Then, the generalized
Fenchel-Young loss (9) is calibrated with the target loss (12) with decoder d = y L ◦ p ΦΩ :
[Equation]
where p ΦΩ : V → C is deﬁned in (6) and y L : C → Y is deﬁned in (13) .
The proof is in Appendix B.4.
Proposition 1 item 6 shows that the smoothness of Φ and strong convexity of Ω ensure the smoothness of L ΦΩ .
Calibration also implies Fisher consistency, namely
[Table]
L ΦΩ ( g ) =
L ΦΩ ( g (cid:63) )
⇒ L ( d ◦ g )
=
L ( f (cid:63) ) , when using the decoder d = y L ◦ p ΦΩ .
The existing proof technique for the calibration of regular Fenchel-Young losses [56, 16] assumes a bilinear pairing and a loss of the form L Ω ϕ ( u, ϕ ( y ))
=
Ω ∗ ϕ ( u ) + Ω ϕ ( ϕ ( y )) − (cid:104) u, ϕ ( y ) (cid:105) , where Ω ϕ is a strongly-convex regularizer w.r.t.
ϕ ( y ) .
Our novel proof technique is more general, as it works with any linear-concave energy and the regularizer Ω is w.r.t.
y , not ϕ ( y ) .
Unlike the existing proof, our proof is valid for the pairwise model we present in the next section.
Experiments 7.1 Multilabel classiﬁcation
We study in this section the application of generalized Fenchel-Young losses to multi-label classiﬁcation, setting Y = { 0 , 1 } k and C = [0 , 1] k , where k is the number of labels.
When the loss L in (12) is the Hamming loss (1 - accuracy), our loss is calibrated for L and the decoding (13) is just (cid:98)
y j = 1 if p j > 0 .
5 else 0 .
We therefore report our empirical results using the accuracy metric.
Unary model.
We consider a neural network u
= g θ ( x ) ∈ R k , assigning a score
u j to each label j ∈ [ k ] .
With the bilinear pairing Φ( u, p ) = (cid:104) u, p (cid:105) , we get
[Equation]
That is, (14) is just a normal neural network with p Ω as output layer.
When Ω( p ) =
Ω 1 ( p )+Ω 1 (1 − p ) , where Ω 1 ( p ) :
= (cid:104) p, log p (cid:105) is Shannon
’s negentropy, we get p Ω ( v )
=
sigmoid ( v ) :
= 1 / (1+exp( − v )) and (9) is just the usual binary logistic / cross-entropy loss.
When Ω( p ) =
Ω 2 ( p ) + Ω 2 (1 − p ) , where Ω 2 ( p ) :
= 12 (cid:104) p, 1 − p (cid:105) is Gini’s negentropy, we get a sparse sigmoid and the binary sparsemax loss [18, §6.2].
Because C = [0 , 1] k is the convex hull of Y = { 0 , 1 } k , (14) can be intepreted as a marginal probability [71].
Indeed, there exists a probability distribution P ( Y | X ) over Y ∈ Y such that
[Equation]
Pairwise model.
We now additionally use a network U
=
h θ ( x ) ∈ R k × k , assigning a score
U i,j to the pairwise interaction between labels i and j .
With v = ( u, U ) and the linear-quadratic coupling Φ( v, p ) =
(cid:104) u, p (cid:105) + 12 (cid:104) p, Up (cid:105) , we get
[Equation]
If U is negative semi-deﬁnite, i.e., U = − AA (cid:62) for some matrix A ∈ R k × m , then the problem is concave in p and can be solved optimally.
Moreover, since Φ( v, p ) is linear-concave, the calibration guarantees in §6 hold and L ΦΩ ( v, y ) is convex in v .
In our experiments, we use a matrix A of rank(cf.
Appendix C).
Unlike (7), (15) does not enjoy a closed form due to the constraints.
We solve it by coordinate ascent: if Ω is quadratic, as is the case with Gini’s negentropy, the coordinate-wise updates can be computed in closed-form.
Again, (15) can be interpreted as a marginal probability.
In constrast to (15), marginal inference in the closely related Ising model is known to be #P-hard [32].
SPEN model.
Following SPENs [11, Eq. 4 and 5], we also tried the energy Φ( v, p ) =
(cid:104) u, p (cid:105) − Ψ ( w, p ) , where v = ( u, w ) , u
= g θ ( x ) and w are the weights of the “prior network” Ψ (independent of x ).
We also tried a variant where Ψ is made convex in p , making Φ( v, p ) concave in p .
In both cases, we compute p ΦΩ ( v ) by solving (6) using projected gradient ascent with backtracking linesearch.
[Figure]
Experimental setup.
We perform experiments on 6 publicly-available datasets, see Appendix C.
We use the train-test split from the dataset when provided.
When not, we use 80% for training data and 20% for test data.
For all models, we solve the outer problem (11) using ADAM.
We set Ω( p ) to the Gini negentropy.
We hold out 25% of the training data for hyperparameter validation purposes.
We set R ( θ ) in (11) to λ 2 (cid:107) θ (cid:107) 22 .
For the regularization hyper-parameter λ , we search 5 log-spaced values between 10 − 4 and 10 1 .
For the learning rate parameter of ADAM, we search 10 log-spaced valued between 10 − 5 and 10 − 1 .
Once we selected the best hyperparameters, we reﬁt the model on the entire training set.
We average results over 3 runs with different seeds.
Results.
Table 2 shows a model comparison.
We observe improvements with the pairwise model on 4 out of 6 datasets, conﬁrming that our losses are able to learn useful models.
Input-concavity helps improve SPENs in 4 out of 6 datasets.
Table 4 conﬁrms that using the envelope theorem for computing gradients works comparably to (if not better than) the implicit function theorem.
Tableshows that our losses outperform the energy, the cross-entropy and the generalized perceptron losses.
7.2 Imitation learning
In this section, we study the application of generalized Fenchel-Young losses to imitation learning.
This setting consists in learning a policy π : X (cid:55)→ Y , a mapping from states X to actions Y , from a ﬁxed dataset of expert demonstrations ( x 1 , y 1 ) , . . .
, ( x n , y n ) ∈
X × Y .
In particular, we only consider a Behavior Cloning approach [61], which essentially reduces imitation learning to supervised learning (as opposed to inverse RL methods [64, 54]).
The learned policy π is evaluated on its performance, which is the expected sum of rewards of the environment [70].
Experimental setup.
We consider four MuJoCo
Gym locomotion environments [23] together with the demonstrations provided by Orsini et al.
[58]; see Appendix C.2 for details.
We evaluate the learned policy π for different number of demonstration trajectories: 1, 4 and 11, consistently with [38, 40, 28].
The action space in the demonstrations is included in [ − 1 , 1] k , where the dimensionality of the action space k corresponds to the torques of the actuators.
We scale the action space to the hypercube Y = C
= [0 , 1] k at learning time and scale it back to the original action space at inference time.
Similarly to the multilabel classiﬁcation setup, we evaluate the unary and pairwise models, with the only difference that we use two hidden layers instead of one, since it leads to signiﬁcantly better performance.
We specify the hyperparameter selection procedure in Appendix C.2.
Results.
We run the best hyperparameters over 10 seeds and report ﬁnal performance overevaluation episodes.
Figure 2 shows a clear improvement of the pairwise model over the unary model for 3 out of 4 tasks.
Contrary to the unary model, the pairwise model enables to capture the interdependence between the different torques of the action space, translating into better performance.
Conclusion
Building upon generalized conjugate functions, we proposed generalized Fenchel-Young losses, a natural loss construction for learning energy networks and studied its properties.
Thanks to conditions on the energy Φ and the regularizer Ω ensuring the loss smoothness, we established calibration guarantees for the case of linear-concave energies, a more general result than the existing analysis, restricted to bilinear energies.
We demonstrated the effectiveness of our losses on multilabel classiﬁcation and imitation learning tasks.
We hope that this paper will help popularize generalized conjugates as a powerful tool for machine learning and optimization.
Acknowledgments
MB thanks Gabriel Peyré for numerous discussions on C -transforms and Clarice Poon for discussions on envelope theorems, as well as Vlad Niculae and André Martins for many fruitful discussions.
A Generalized Bregman divergences
The expression Ω ∗ ( v ) + Ω( p ) − (cid:104) v, p (cid:105) at the heart of Fenchel-Young losses is closely related to
Bregman divergences; see [3, Theorem 1.1] and [18].
In this section, we develop a similar relationship between generalized Fenchel-Young losses and a new generalized notion of Bregman divergence.
Biconjugates.
We begin by recalling well-known results on biconjugates [22].
Applying the conjugate (1) twice, we obtain the biconjugate Ω ∗∗ ( p ) .
It is well-known that a function Ω is convex and closed (i.e., lower-semicontinuous)
if and only if Ω = Ω ∗∗ .
This therefore provides a variational characterization of lower-semicontinuous convex functions.
This characterization naturally motivates the class of Φ -convex functions (§4).
If Ω is nonconvex, Ω ∗∗ is Ω ’s tightest convex lower bound.
Generalized biconjugates.
The generalized conjugate in (5) uses maximization w.r.t.
the second argument p ∈ C .
To obtain a generalized conjugate whose maximization is w.r.t.
the ﬁrst argument v ∈ V instead, we deﬁne Ψ( p, v ) :
=
Φ( v, p ) .
Note that if Φ is symmetric, the distinction between Φ and Ψ is not necessary.
Similarly to (5), we can then deﬁne the Ψ -conjugate of Λ: V → R as
[Equation]
i.e., the maximization is over the left argument of Φ .
We deﬁne the corresponding argmax as
[Equation]
In particular, with Λ := Ω Φ , we can deﬁne the generalized biconjugate Λ Ψ = Ω ΦΨ : C → R .
Generalized biconjugates enjoy similar properties as regular biconjugates, as we now show.
Proposition 5 (Properties of generalized biconjugates) .
Let Ω: C → R , Φ:
V × C
→ R and Ψ( p, v ) :
=
Φ( v, p ) .
Lower-bound: Ω ΦΨ ( p ) ≤ Ω( p ) for all p ∈ C . 2.
Equality: Ω ΦΨ ( p ) =
Ω( p ) if and only if Ω is Ψ -convex.
Tightest lower-bound: Ω ΦΨ ( p ) is the tightest Ψ -convex lower-bound of Ω( p ) .
Proofs are given in Appendix B.5.
Similar results hold for Λ ΨΦ , where Λ: V → R .
Deﬁnition.
We can now deﬁne the generalized Bregman divergence D ΦΩ : C × C → R + as
[Equation] [Equation]
where we recall that Ψ( p, v ) :
= Φ( v, p ) and using (17) we have
[Equation]
We have therefore obtained a notion of Bregman divergence parametrized by a coupling Φ( v, p ) , such as a neural network.
As shown in the proposition below, generalized Bregman divergences enjoy similar properties as regular Bregman divergences.
Proofs are given in Appendix B.6.
Proposition 6 (Properties of generalized Bregman divergences) .
Let Ω be a Ψ -convex function, where Ψ( p, v ) :
=
Φ( v, p ) .
Link with generalized FY loss: Denoting v := v ΨΩ Φ ( p (cid:48) ) , we have D ΦΩ ( p, p (cid:48) )
=
L ΦΩ ( v, p ) .
Non-negativity: D ΦΩ ( p, p (cid:48) ) ≥ 0 , for all p, p (cid:48) ∈ C .
Identity of indiscernibles: D ΦΩ ( p, p (cid:48) )
= 0 ⇔ p = p (cid:48) if p (cid:55)→ Φ( v, p ) − Ω( p ) is strictly concave for all v ∈ V . 4.
Convexity: if p (cid:55)→ Φ( v, p ) − Ω( p ) is concave for all v ∈ V , then D ΦΩ ( p, p (cid:48) ) is convex in p .
Recovering Bregman divergences: If Φ( v, p ) is the bilinear coupling (cid:104) v, p (cid:105) , then we recover the usual Bregman divergence D Ω : C × C
→ R +
[Equation]
Some remarks:
• The generalized Bregman divergence is between objects p and p (cid:48) of the same space C , while the generalized Fenchel-Young loss is between objects v and p of mixed spaces V and C .
• If Φ( v, p ) − Ω( p ) is concave in p , then D Φ Ω ( p, p (cid:48) ) is convex in p , as is the case of the usual Bregman divergence D Ω ( p, p (cid:48) ) .
However, (19) is not easy to solve globally in general, as it is the maximum of a difference of convex functions in v .
This can be done approximately by using the convex-concave procedure [72], linearizing the left part.
We have the opposite situation with the generalized Fenchel-Young loss: if Φ( v, p ) − Ω( p ) is convex-concave, L ΦΩ ( v, y ) is easy to compute but it is a difference of convex functions in v .
Similarly, we may also deﬁne D [Equation] Φ
[Equation]
where we recall that p ΦΩ ( v ) :
= argmax p ∈C Φ( v, p ) − Ω( p ) .
We have thus obtained a divergence between two objects v and v (cid:48) in the same space V .
B Proofs B.1 Proofs
for Proposition (properties of generalized conjugates)
Generalized Fenchel-Young inequality.
From (5), we immediately obtain Ω Φ ( v ) ≥ Φ( v, p ) − Ω( p ) for all v ∈ V and all p ∈ C . 2.
Convexity.
If Φ( v, p ) is convex in v , then Ω Φ ( v ) is the maximum of a family of convex functions indexed by p .
Therefore, Ω Φ ( v ) is convex.
Note that this is the case even if Ω( p ) is nonconvex.
Order reversing.
Since Ω ≤ Λ , we have
[Equation]
Continuity.
See [65, Box 1.8].
Gradient.
See “Assumptions for envelope theorems” in §4. 6.
Smoothness.
We follow the proof technique of [46, Lemma 4.3], which states that v (cid:55)→ [Equation] v, p ) is ( β + β 2 /γ )
-smooth if E is β -smooth in ( v, p ) and γ -strongly concave in p over C .
We show here that if E ( v, p ) decomposes as E ( v, p ) =
Φ( v, p ) − Ω( p ) , where Ω is γ -strongly convex over C but possibly nonsmooth (as is the case for instance of Shannon’s negentropy), and Φ( v, p ) is β -smooth in ( v, p ) and concave in p , then we still have that Ω Φ ( v ) =
max p ∈C Φ( v, p ) − Ω( p ) is ( β + β 2 /γ )
-smooth.
For brevity, let us deﬁne the shorthand p ΦΩ ( v ) :
= p (cid:63) ( v ) .
From [46, Section A.3], we have
[Equation] [Equation]
p
[Equation] [Equation]
[Equation]
∇
[Equation]
E ( v 1 , p (cid:63) ( v 2 ))
(cid:105) for all v 1 , v 2 ∈ V .
With E ( v, p ) =
Φ( v, p ) − Ω( p ) , we obtain
[Equation]
Since Φ is β -smooth, we have for all p ∈ C (cid:107)∇
[Equation]
Combined with the Holder inequality (cid:104) a, b
[Equation]
≤
[Equation]
a
[Equation]
b (cid:107) , this gives
[Equation]
Simplifying, we get
[Equation] [Equation]
Therefore, p ΦΩ ( v ) = p (cid:63) ( v ) is β/γ -Lipschitz.
From Rockafellar’s envelope theorem [62, Theorem 10.31], ∇ Ω Φ ( v ) = ∇ 1 Φ( v, p (cid:63) ( v )) .
Since Φ is β -smooth, we therefore have
[Equation]
where we used (21) and βγ ≥ 1 .
A related result in the context of bilevel programming but with different assumptions and proof is stated in [31, Lemma 2.2].
More precisely, the proof of that result requires twice differentiability of E ( v, p ) =
Φ( v, p ) − Ω( p ) while we do not.
Moreover, applying that result to our setting would require E ( v, p ) to be smooth in ( v, p ) while we only assume Φ( v, p ) to be the case.
We emphasize again that Ω( p ) is nonsmooth when it is the negentropy.
B.2 Proofs for Proposition 2 (closed forms)
Bilinear coupling.
This follows from
[Equation]
and similarly for p Ω ( v ) .
Linear-quadratic coupling.
Let us deﬁne the function
[Equation]
Its gradient is ∇ F ( p ) =
( γI − A ) p − b .
Setting ∇ F ( p (cid:63) )
= 0 , we obtain
[Equation]
where we assumed that ( γI − A ) is positive deﬁnite.
We therefore get
[Equation]
=
[Equation]
A ) [Equation]
Metric coupling.
From (8), it is easy to check that Ω = − Λ ⇔ Ω
Φ = − Λ C with C = − Φ .
From [60, Proposition 6.1], we have Λ C = − Λ .
Therefore, Ω Φ = − Ω .
(Properties of generalized Fenchel-Young losses
Non-negativity
This follows immediately from the generalized Fenchel-Young inequality
Zero loss
If p Φ Ω ( v ) =
y , then using L Φ Ω ( v, y )
=
F ( v, y ) − F ( v, p Φ Ω ( v )) , where F ( v, p ) =
Ω( p ) − Φ( v, p ) , we obtain L ΦΩ ( v, y )
= 0 .
Let’s now prove the reverse direction.
Since the maximum in (5) exists, we have F ( v, p ΦΩ ( v )) ≤ F ( v, y ) .
If L ΦΩ ( v, y ) = 0 , we have F ( v, p ΦΩ ( v ))
=
F ( v, y ) .
Since the maximum is unique by assumption, this proves that y = p ΦΩ ( v ) .
Gradient.
This follows directly from the deﬁnition (9).
Difference of convex (DC).
If Φ( v, p ) is convex in p , Ω Φ ( v ) is convex (Proposition 1).
Therefore, v [Equation] ) − Φ( v, p ) for all p ∈ C .
Smaller output set, smaller loss.
Let Ω: C → R and let Ω (cid:48) be the restriction of Ω to C (cid:48) ⊆
C , i.e., Ω (cid:48) ( p ) :
= Ω + I C (cid:48) , where I C (cid:48) is the indicator function of C (cid:48) .
From (5), we have (Ω (cid:48) )
Φ ( v ) ≤ Ω Φ ( v ) for all v ∈ V .
From (9), we therefore have L ΦΩ (cid:48) ( v, p ) ≤ L ΦΩ ( v, p ) for all v ∈ V and all p ∈ C (cid:48) .
Quadratic lower-bound.
If a function F is γ -strongly convex over C w.r.t.
a norm (cid:107) · (cid:107) , then
[Equation]
(22) for all p, p (cid:48) ∈ C .
If C is a closed convex set, we also have that p (cid:63) =
argmin p ∈C F ( p ) satisﬁes the optimality condition
[Equation]
(23) for all p ∈ C [53, Eq.
Combining (22) with p (cid:48)
= p (cid:63) and (23), we obtain
[Equation]
Applying the above with F ( p )
=
Ω( p ) − Φ( v, p ) and using p (cid:63) =
p ΦΩ ( v ) , we obtain
[Equation]
Upper-bounds.
If F is α -Lipschitz over C with respect to a norm (cid:107) · (cid:107) , then for all p, p (cid:48) ∈ C
[Equation]
Applying the above with F ( p )
=
Ω( p ) − Φ( v, p ) and p (cid:48) =
p ΦΩ ( v ) , we obtain
[Equation]
If Φ( v, p ) is concave in p , then its linear approximation always lies above:
[Equation]
for all p, p (cid:48) ∈ C .
We then have
[Equation]
p ) .
B.4 Proof of Proposition 4 (calibration)
Background.
The pointwise target risk of (cid:98) y ∈ Y according to q ∈ (cid:52) |Y| is
[Equation]
We also deﬁne the corresponding excess of pointwise risk, the difference between the pointwise risk and the pointwise Bayes risk:
[Equation]
We can then write the risk of f : X → Y in terms of the pointwise risk
[Equation]
Let us deﬁne the Bayes predictor f (cid:63) :=
argmin f : X→Y
L ( f ) .
The Bayes risk is then
[Equation]
Combining the above, we can write the excess of risk of f : X → Y as
[Equation]
Similarly, with the generalized Fenchel-Young loss (9), the pointwise surrogate risk of v ∈ V according to q ∈ (cid:52) |Y| is
[Equation]
and the excess of pointwise surrogate risk is
[Equation]
Let us deﬁne the surrogate risk of g : X → V by
[Equation]
and the corresponding Bayes predictor by
[Equation]
We can then write the excess of surrogate risk of g : X → V as
[Equation]
A calibration function [68] ξ : R + → R + is a function relating the excess of pointwise target risk and pointwise surrogate risk.
It should be non-negative, convex and non-decreasing on R + , and satisfy ξ (0) = 0 .
Formally, given a decoder d : V → Y , ξ should satisfy
[Equation]
for all v ∈ V and q ∈ (cid:52) |Y| .
By Jensen’s inequality, this implies that the target and surrogate risks are calibrated for all g : X → V [59, 56]
[Equation]
From now on, we can therefore focus on proving (24).
Upper-bound on the pointwise target excess risk.
We now make use of the afﬁne decomposition (12).
Let σ := sup y ∈Y (cid:107) V (cid:62) ϕ
( y ) (cid:107) ∗ , where (cid:107) · (cid:107) ∗ denotes the dual norm of (cid:107) · (cid:107) .
Let us deﬁne ˜ [Equation] (cid:98) [Equation]
+ [Equation] and µ ϕ ( q ) :
=
E Y ∼ q [ ϕ ( Y )]
∈ conv( ϕ ( Y )) ⊆
ϕ ( R k ) .
From [16, Lemma 2], δ(cid:96) (˜ y L ( u ) , q ) ≤ 2 σ (cid:107) µ ϕ ( q ) − u (cid:107) ∀ u ∈ ϕ
( R k ) , q ∈ (cid:52) |Y| .
Using y L ( p ) =
˜
y L ( u ) with u = ϕ ( p ) , we thus get δ(cid:96) ( y L ( p ) , q ) ≤ 2 σ (cid:107) µ ϕ ( q ) − ϕ
( p ) (cid:107) ∀ p ∈ R k , q ∈ (cid:52) |Y| .
Bound on the pointwise surrogate excess risk (bilinear case).
To highlight the difference with our proof technique, we ﬁrst prove the result in the bilinear case, assuming Ω is γ -strongly convex.
We follow the same proof technique as [56, 16] but unlike these works we do not require any
Legendre-type assumption on Ω .
If Φ( v, p ) =
(cid:104) v, ϕ ( p ) (cid:105) =
(cid:104) v, p (cid:105) , we have
[Equation]
where, when ϕ ( y ) =
y , we denote µ ( q ) :=
E Y ∼ q
[ Y ] ∈ conv( Y ) ⊆
R k for short.
The quantity
E Y ∼ q Ω( Y ) − Ω( µ ( q )) is called Bregman information [7] or Jensen gap, and is non-negative if Ω is convex.
This term cancels out in the excess of pointwise surrogate risk
[Equation]
Since the Fenchel-Young loss achieves its minimum at 0 , we have
[Equation]
Therefore, the excess of pointwise surrogate risk can be written in Fenchel-Young loss form.
By the quadratic lower-bound (10) and the upper-bound (25), we have
[Equation]
Therefore the calibration function with the decoder d = y L ◦ p Ω is
[Equation]
Bound on the pointwise surrogate excess risk (linear-concave case).
We now prove the bound assuming that L ΦΩ is smooth and Φ( v, p ) =
(cid:104) v, ϕ ( p ) (cid:105) .
This includes the previous proof as special case because when Φ( v, p ) = (cid:104) v, p (cid:105) , then L ΦΩ ( v, y ) =
L Ω ( v, y ) is 1 γ -smooth in v if and only if Ω( p ) is γ -strongly convex in p (cf.
By Theorem 4.22 in [57], if a function f ( v ) is M -smooth in v w.r.t.
the dual norm (cid:107) · (cid:107) ∗ and is bounded below, then
[Equation]
With f ( v ) =
(cid:96) ΦΩ ( v, q ) =
E Y ∼ q L Ω ( v, Y ) , which is non-negative, we obtain
[Equation]
We therefore have
[Equation]
Therefore the calibration function with the decoder d = y L ◦ p ΦΩ is
[Equation]
B.5 Proofs for Proposition 5 (Properties of generalized biconjugates)
The proofs are similar to that for C -transforms [65, Proposition 1.34].
Lower bound.
Let Ω: C → R .
We have
[Equation]
Therefore, Ω ΦΨ ( p ) ≤ Ω( p ) for all p ∈ C .
Analogously, for a function Λ: V → R , we have Λ ΨΦ ( v ) ≤ Λ( v ) for all v ∈ V .
Equality.
Since Ω is Ψ -convex, there exits Λ: V → R such that Ω = Λ Ψ .
We then have Ω Φ = Λ ΨΦ .
Using the lower bound property, we get Ω Φ ( v ) =
Λ ΨΦ ( v ) ≤ Λ( v ) for all v ∈ V .
By the order reversing property, we have Ω ΦΨ ( p ) ≥ Λ Ψ ( p ) =
Ω( p ) for all p ∈ C .
However, we also have Ω ΦΨ ( p ) ≤ Ω( p ) for all p ∈ C .
Therefore, Ω ΦΨ ( p ) = Ω( p ) for all p ∈ C .
Tightest lower-bound.
Let Ω (cid:48) be any lower bound of Ω that is Ψ -convex.
Therefore, there exists Λ: V → R such that Ω (cid:48) ( p ) =
Λ Ψ ( p ) ≤ Ω( p ) for all p ∈ C .
By the order reversing property, we have Λ ΨΦ ( v ) ≥ Ω Φ ( v ) for all v ∈ V .
By the lower bound property, we also have Λ ΨΦ ( v ) ≤ Λ( v ) for all v ∈ V and therefore, Λ( v ) ≥ Ω Φ ( v ) for all v ∈ V .
Applying the order reversing property once more, we get Ω (cid:48) ( p ) =
Λ Ψ ( p ) ≤ Ω ΦΨ ( p ) for all p ∈ C .
Therefore Ω ΦΨ is the tightest lower bound of Ω .
B.6 Proofs for Proposition 6 (Properties of generalized Bregman divergences)
Link with generalized Fenchel-Young losses.
From (16) and (17), we have
[Equation]
for any Λ: V → R .
With Λ = Ω Φ , if Ω is Φ -convex, using Proposition 5, we have
[Equation]
Plugging Ω( p (cid:48) ) in (18) and using the shorthand v := v ΨΩ Φ ( p (cid:48) ) , we get
[Equation]
Non-negativity.
This follows directly from the non-negativity of L ΦΩ ( v, p )
Identity of indiscernibles .
If p = p (cid:48) , we immediately obtain D ΦΩ ( p, p (cid:48) )
=
from (18).
Let’s prove the reverse direction.
If D ΦΩ ( p, p (cid:48) )
= 0 , we have F ( v ΨΩ Φ ( p (cid:48) ) , p ) =
F ( v ΨΩ Φ ( p (cid:48) ) , p (cid:48) )
where
F ( v, p ) =
Ω( p ) − Φ( v, p ) .
Since by assumption F is strictly convex in p , we obtain p = p (cid:48) .
Convexity.
This follows from D ΦΩ ( p, p (cid:48) )
=
Ω( p ) − Φ( v, p ) + const
Recovering Bregman divergences.
If Φ( v, p ) =
(cid:104) v, p (cid:105) , we have
[Equation]
Plugging back in (18), we obtain the Bregman divergence (20).
C Experimental details C.1 Multilabel classiﬁcation
Datasets.
We used public datasets available at https://www.csie.ntu.edu.tw/~cjlin/ libsvmtools/datasets/ .
Dataset statistics are summarized in Table 3.
For all datasets, we normalize samples to have zero mean unit variance.
[Table]
Experimental details.
In all experiments, we set the activation σ to relu ( a ) :
= max { 0 , a } .
For the unary model, we use a neural network with one hidden-layer, i.e., g θ ( x ) =
W 2 σ ( W 1 x + b 1 )+ b 2 , where θ =
( W 2 , b 2 , W 1 , b 1 ) , W 2 ∈ R k × m , b 2 ∈ R k , W 1 ∈ R m × d , b 1 ∈ R m , and m is the number of hidden units.
We use the heuristic m =
min { 100 , d/ 3 } , where d is the dimensionality of x .
For the pairwise model, in order to obtain a negative semi-deﬁnite matrix U , we parametrize
U = − AA (cid:62) with A =
[ W 1 x + b 1 , . .
, W m x + b m ] , where W j ∈ R k × d and b j ∈ R k
In our experiments, we choose a rank-one model, i.e., m = 1 .
Note that we use distinct parameters for the unary and pairwise models but sharing parameters would be possible.
For the SPEN model, following [11, Eq. 4 and 5], we set the energy to Φ( v, p ) =
(cid:104) u, p (cid:105) − Ψ
( w, p ) , where v = ( u, w ) , u
= g θ ( x ) and w are the weights of the “prior network” Ψ (independent of x ).
We parametrize Ψ( w, p ) =
W 2 σ ( W 1 p + b 1 ) + b 2 , where w = ( W 2 , b 2 , W 1 , b 1 ) , W 2 ∈ R 1 × m , b 2 ∈ Rand W 1 ∈ R m × k , b 1 ∈ R m .
To further impose convexity of Ψ in p , W 2 needs to be non-negative.
To do so without using constrained optimization, we use the change of variable W {+
+}
= softplus ( W (cid:48) 2 ) , where softplus ( a ) :
= log(1 + exp( a )) is used as an element-wise bijective mapping.
Additional results.
The gradient of Ω Φ ( v ) can be computed using the envelope theorem (Proposition 1), which does not require to differentiate through p ΦΩ ( v ) .
Alternatively, since Ω Φ ( v ) =
Φ( v, p ΦΩ ( v )) − Ω( p ΦΩ ( v )) , we can also compute the gradient of Ω Φ ( v ) by using the implicit function theorem, differentiating through p ΦΩ ( v ) .
To do so, we use the approach detailed in [17].
Results in
Table 4 show that the envelope theorem performs comparably to the implicit function theorem, if not slightly better.
Differentiating through p Φ Ω ( v ) using the implicit function theorem requires to solve a k × k system and its implementation is more complicated than the envelope theorem.
Therefore, we suggest to use the envelope theorem in practice.
[Table]
[Table]
[Table] [Table] [Table]
In addition, we also compared the proposed generalized Fenchel-Young loss with the energy loss, the binary cross-entropy loss and the generalized perceptron loss, which corresponds to setting Ω( p ) = 0 .
As explained in §3, the cross-entropy loss requires to differentiate through p ΦΩ ( v ) ; we do so by implicit differentiation.
Table 5 shows that the generalized Fenchel-Young loss outperforms these losses.
As expected, the energy loss performs very poorly, as it can only push the model in one direction [44].
Using regularization Ω , as advocated in this paper, is empirically conﬁrmed to be beneﬁcial for accuracy.
[Table]
C.2 Imitation learning
Experimental details.
We run a hyperparameter search over the learning rate of the ADAM optimizer, the number of hidden units in the layers, the weight of the L2 parameters regularization term and the scale of the energy regularization term Ω .
We run the hyperparameter search for 4 demonstration trajectories and select the best performing ones based on the ﬁnal performance (averaged over 3 seeds).
[Table]
Environments.
We also provide performance of the expert agent as detailed by Orsini et al.
[58] as well as the description of the observation and action spaces for each environment
[Table]
[Table]