B ACKPROPAGATION THROUGH C OMBINATORIAL A LGORITHMS : I DENTITY WITH P ROJECTION W ORKS
Anonymous authors
Paper under double-blind review
A BSTRACT
Embedding discrete solvers as differentiable layers has given modern deep learning architectures combinatorial expressivity and discrete reasoning capabilities.
The derivative of these solvers is zero or undeﬁned, therefore a meaningful replacement is crucial for effective gradient-based learning.
Prior works rely on smoothing the solver with input perturbations, relaxing the solver to continuous problems, or interpolating the loss landscape with techniques that typically require additional solver calls, introduce extra hyper-parameters, or compromise performance.
We propose a principled approach to exploit the geometry of the discrete solution space to treat the solver as a negative identity on the backward pass and further provide a theoretical justiﬁcation.
Our experiments demonstrate that such a straightforward hyper-parameter-free approach is able to compete with previous more complex methods on numerous experiments such as backpropagation through discrete samplers, deep graph matching, and image retrieval.
Furthermore, we substitute the previously proposed problem-speciﬁc and label-dependent margin with a generic regularization procedure that prevents cost collapse and increases robustness.
I NTRODUCTION
Deep neural networks have achieved astonishing results in solving problems on raw inputs.
However, in key domains such as planning or reasoning, deep networks need to make discrete decisions, which can be naturally formulated via constrained combinatorial optimization problems.
In many settings— including shortest path ﬁnding (Vlastelica et al., 2020; Berthet et al., 2020), optimizing rank-based objective functions (Rolínek et al., 2020a), keypoint M ATCHING (Rolínek et al., 2020b; Paulus et al., 2021), Sudoku solving (Amos and Kolter, 2017; Wang et al., 2019), solving the knapsack problem from sentence descriptions (Paulus et al., 2021)—neural models that embed optimization modules as part of their layers achieve improved performance, data-efﬁciency, and generalization (Vlastelica et al., 2020;
Amos and Kolter, 2017; Ferber et al., 2020; P. et al., 2021).
This paper explores the end-to-end training of deep neural network models with embedded discrete combinatorial algorithms ( solvers , for short) and derives simple and efﬁcient gradient estimators for these architectures.
Deriving an informative gradient through the solver constitutes the main challenge, since the true gradient is, due to the discreteness, zero almost everywhere.
Most notably,
Blackbox Backpropagation (BB) by Vlastelica et al.
(2020) introduces a simple method that yields an informative gradient by applying an informed perturbation to the solver input and calling the solver one additional time.
This results in a gradient of an implicit piecewise-linear loss interpolation, whose locality is controlled by a hyperparameter.
We propose a fundamentally different strategy by dropping the constraints on the solver solutions and simply propagating the incoming gradient through the solver
, effectively treating the discrete block as a negative identity on the backward pass.
While our gradient replacement is simple and cheap to compute
, it comes with important considerations , as its naïve application can result in unstable learning behavior as described in the following.
Our considerations are focused on invariances of typical combinatorial problems under speciﬁc transformations of the cost vector.
These transformations usually manifest as projections or normalizations , e.g. as an immediate consequence of the linearity of the objective, the combinatorial solver is agnostic to normalization of the cost vector.
Such invariances, if unattended, can hinder fast convergence due to the noise of spurious irrelevant updates, or can result in divergence and
[Figure]
cost collapse (Rolínek et al., 2020a).
We propose to exploit the knowledge of such invariances by including the respective transformations in the computation graph.
On the forward pass this leaves the solution unchanged, but on the backward pass removes the malicious part of the update.
We also provide an intuitive view on this as differentiating through a relaxation of the solver.
In our experiments, we show that this technique is crucial to the success of our proposed method.
In addition, we improve the robustness of our method by adding noise to the cost vector, which induces a margin on the learned solutions and thereby subsumes previously proposed ground-truthinformed margins (Rolínek et al., 2020a).
With these considerations taken into account, our simple method achieves strong empirical performance.
Moreover, it avoids a costly call to the solver on the backward pass and does not introduce additional hyperparameters in contrast to previous methods.
Our contributions can be summarized as follows:
(i)
A hyperparameter-free method for linear-cost solver differentiation that does not require any additional calls to the solver on the backward pass.
(ii) Exploiting invariances via cost projections tailored to the combinatorial problem.
(iii) Increasing robustness and preventing cost collapse by replacing the previously proposed informed margin with a noise perturbation.
(iv) Analysis of the robustness of differentiation methods to perturbations during training.
R ELATED W ORK
Optimizers as Model Building Blocks.
It has been shown in various application domains that optimization on prediction is beneﬁcial for model performance and generalization.
One such area is meta-learning, where methods backpropagate through multiple steps of gradient descent for few-shot adaptation in a multi-task setting (Finn et al., 2017; Raghu et al., 2020).
Along these lines, algorithms that effectively embed more general optimizers into differentiable architectures have been proposed such as convex optimization (Agrawal et al., 2019a; Lee et al., 2019), quadratic programs (Amos and
Kolter, 2017), conic optimization layers (Agrawal et al., 2019b), and more.
Combinatorial Solver Differentiation.
Many important problems require discrete decisions and hence, using combinatorial solvers as layers have sparked research interest (Domke, 2012; Elmachtoub and Grigas, 2022).
Methods, such as SPO (Elmachtoub and Grigas, 2022) and MIPaaL
Ferber et al.
(2020), assume access to true target costs, a scenario we are not considering.
Berthet et al.
(2020) differentiate through discrete solvers by sample-based smoothing.
Blackbox Backpropagation (BB) by Vlastelica et al.
(2020) returns the gradient of an implicitly constructed piecewise-linear interpolation.
Modiﬁed instances of this approach have been applied in various settings such as ranking (Rolínek et al., 2020a), keypoint matching (Rolínek et al., 2020b), and imitation learning (Vlastelica et al., 2020).
I-MLE by Niepert et al.
(2021) adds noise to the solver to model a discrete probability distribution and uses the BB update to compute informative gradients.
Previous works have also considered adding a regularization to the linear program, including differentiable top- k -selection (Amos et al., 2019) and differentiable ranking and sorting (Blondel et al., 2020).
Another common approach is to differentiate a softened solver, for instance in (Wilder et al., 2019) or (Wang et al., 2019) for M AX S AT .
Finally, Paulus et al.
(2021) extend approaches for learning the cost coefﬁcients to learning also the constraints of integer linear programs.
Learning to Solve Combinatorial Problems.
An orthogonal line of work to ours is differentiable learning of combinatorial algorithms or their improvement by data-driven methods.
Examples of such algorithms include learning branching strategies for MIPs (Balcan et al., 2018; Khalil et al., 2016; Alvarez et al., 2017), learning to solve SMT formulas (Balunovic et al., 2018), and learning to solve linear programs (Mandi and Guns, 2020; Tan et al., 2020).
A natural way of dealing with the lack of gradient information in combinatorial problems is reinforcement learning which is prevalent among these methods (Khalil et al., 2016; Bello et al., 2017; Nazari et al., 2018; Zhang and Dietterich, 2000).
Further progress has been made in applying graph neural networks for learning classical programming algorithms (Velickovic et al., 2018; 2020) and latent value iteration (Deac et al., 2020).
Further work in this direction can be found in the review Li et al.
Another interesting related area of work is program synthesis, or “learning to program” (Ellis et al., 2018;
Inala et al., 2020).
Straight-through Estimator (STE).
The STE (Bengio et al., 2013; Hinton, 2012; Rosenblatt, 1958) differentiates the thresholding function by treating it as an identity function.
Jang et al. (2017) extended the STE to differentiating through samples from parametrized discrete distributions. The term STE has also been used more loosely to describe the replacement of any computational block with an identity function (Bengio et al., 2013) or other differentiable replacements (Yin et al., 2019) on the backward pass. Our proposed method can be seen as a generalization of differentiating through the thresholding function to general arg max / arg min problems with linear objectives, and it is therefore closely connected to the STE.
We give a detailed discussion of the relationship in Suppl. A.  M ETHOD We consider architectures that contain differentiable blocks, such as neural network layers, and combinatorial blocks, as sketched in Fig.
In this work, a combinatorial block uses an algorithm (called solver ) to solve an optimization problem of the form
[Equation]
where ω ∈ W ⊆ R n is the cost vector produced by a previous block, Y ⊂ R n is any ﬁnite set of possible solutions and y ( ω ) ∈
Y is the solver’s output.
Without loss of generality, Y consists only of extremal points of its convex hull, as no other point can be a solution of optimization (1).
This formulation covers linear programs as well as integer linear programs.
3. D IFFERENTIATING THROUGH C OMBINATORIAL S OLVERS
We consider the case in which the solver is embedded inside the neural network, meaning that the costs ω are predicted by a backbone, the solver is called, and the solution y ( ω ) is post-processed before the ﬁnal loss (cid:96) is computed.
For instance, this is the case when a speciﬁc choice of loss is crucial, or the solver is followed by additional learnable components.
We aim to train the entire architecture end-to-end, which requires computing gradients in a layer-wise manner during backpropagation.
However, the true derivative of the solver y ( ω ) is either zero or undeﬁned , as the relation between the optimal solution y ( ω ) and the cost vector ω is piecewise constant.
Thus, it is crucial to contrive a meaningful replacement for the true zero Jacobian of the combinatorial block.
See Fig.
1 for an illustration.
Note, that the linearity of the objective in (1) makes differentiating the optimization problem more challenging than the non-linear counterparts considered in Amos and Kolter (2017);
Agrawal et al.
(2019a), due to the inherent discreteness of Y .
See Suppl.
C.1 for a discussion.
3. I DENTITY U PDATE : I NTUITION IN S IMPLE C ASE
On the backward pass, the negated incoming gradient − d (cid:96)/ d y gives us the local information of where we expect solver solutions with a lower loss.
We ﬁrst consider the simple scenario in which − d (cid:96)/ d y points directly toward another solution y ∗ , referred to as the “target”.
This means that there is some η > 0 such that − η d (cid:96) d y =
y ∗ − y ( ω ) .
This happens, for instance, if the ﬁnal layer coincides with the (cid:96) 2 loss (then η = 1 / 2 ), or for (cid:96) 1 loss with Y being a subset of { 0 , 1 } n (then η = 1 ). [Figure]
Our aim is to update ω in a manner which decreases the objective value associated with y ∗ , i.e.
(cid:104) ω, y ∗ (cid:105) , and increases the objective value associated with the current solution y ( ω ) , i.e. (cid:104) ω, y ( ω ) (cid:105) .
Therefore, y ∗ will be favoured over y ( ω ) as the solution in the updated optimization problem.
This motivates us to set the replacement for the true zero gradient d (cid:96)/ d ω to
[Equation]
The scaling factor η is subsumed into the learning rate, therefore we propose the update
[Equation]
This corresponds to simply treating the solver as a negated identity on the backward pass, hence we call our method “Identity”.
An illustration of how the repeated application of this update leads to the correct solution is provided in Figures 2a and 2b.
Comparison to Blackbox Backpropagation.
To strengthen the intuition of why update (3) results in a sensible update, we offer a comparison to BB by Vlastelica et al.
(2020) that proposes an update
[Equation]
where y ( ω ) is the solution from the forward pass and, on the backward pass, the solver is called again on a perturbed cost to get y λ ( ω )
=
y ( ω + λ d (cid:96)/ d y ) .
Here, λ > 0 is a hyperparameter that controls the locality of the implicit linear interpolation of the loss landscape.
Observe that y λ ( ω ) serves as a target for our current y ( ω ) , similar to the role of y ∗ in computation (2).
If λ in BB coincides with a steps size that Identity needs to reach the target y ∗ with update ∆ I ω , then
[Equation]
[Equation] Therefore, if − d (cid:96)/ d y points directly toward a neighboring solution, the Identity update is equivalent to the BB update with the smallest λ that results in a non-zero gradient in (4).
This situation is illustrated in Fig.
2c (cf.
Fig.
2a).
However, Identity does not require an additional call to the solver on the backward pass, nor does it have an additional hyperparameter that needs to be tuned.
3. I DENTITY U PDATE : G ENERAL C ASE
We will now consider the general case, in which we do not expect the solver to ﬁnd a better solution in the direction ∆ I ω
= − d (cid:96)/ d y due to the constraints on Y .
We show that if we ignore the constraints on Y and simply apply the Identity method, we still achieve a sensible update.
Let ω = ω 0 be a ﬁxed initial cost.
For a ﬁxed step size α > 0 , we iteratively update the cost using
Identity, i.e. we set ω
k +1 =
ω
k − α ∆ I ω k for k ∈ N .
We aim to show that performing these updates leads to a solution with a lower loss.
As gradient-based methods cannot distinguish between a nonlinear loss (cid:96) and its linearization f ( y ) =
(cid:96) ( y ( ω ))
+ (cid:104)
y − y ( ω ) , d (cid:96)/ d y (cid:105) at the point y ( ω ) , we can safely work with f in our considerations.
We desire to ﬁnd solutions with lower linearized loss than our current solution y ( ω ) , i.e. points in the set
[Equation]
Our result guarantees that a solution with a lower linearized loss is always found if one exists.
The proof is in Suppl.
C.
Theorem 1.
For sufﬁciently small α > 0 , either Y ∗ (cid:0) y ( ω ) (cid:1) is empty and y ( ω k ) =
y ( ω ) for every k ∈ N , or there is n ∈ N
such that y ( ω n ) ∈
Y ∗ (cid:0) y ( ω ) (cid:1) and y ( ω k ) =
y ( ω ) for all k < n .
3. E XPLOITING S OLVER I NVARIANTS
In practice, Identity can lead to problematic cost updates when the optimization problem is invariant to a certain transformation of the cost vector ω .
For instance, adding the same constant to every component of ω will not affect its rank or top- k indices.
Formally, this means that there exists a mapping P : R n → R n of the cost vector ω that does not change the optimal solver solution, i.e.
[Equation]
Linear Transforms.
Let us demonstrate why invariants can be problematic in a simpliﬁed case assuming that P is linear.
Consider an incoming gradient d (cid:96)/ d y , for which the Identity method suggests the cost update ∆ I ω
=
− d (cid:96)/ d y .
We can uniquely decompose ∆ I ω into ∆ I ω =
∆ I ω 1 + ∆ I ωwhere ∆ I ω 1 = P (
∆ I ω ) ∈
Im P and ∆ I ω
= ( I − P )
∆ I ω ∈ ker P .
Now observe that only the parallel update ∆ I ω 1 affects the updated optimization problem, as
[Equation]
for every ω ∈ W and for any step size α > 0 .
In the second equality, we used
[Equation]
exploiting linearity and idempotency of P .
In the case when a user has no control about the incoming gradient ∆ I ω = − d (cid:96)/ d y , the update ∆ I ωin ker P can be much larger in magnitude than ∆ I ω 1 in Im P .
In theory, if updates were applied directly in cost space, this would not be very problematic, as updates in ker P do not affect the optimization problem.
However, in practice, the gradient is further backpropagated to update the weights in the components before the solver. Spurious irrelevant components in the gradient can therefore easily overshadow the relevant part of the update, which is especially problematic as the gradient is computed from stochastic mini-batch samples.
Therefore, it is desirable to discard the irrelevant part of the update.
Consequently, for a given incoming gradient d (cid:96)/ d y , we remove the irrelevant part ( ker P ) and return only its projected part
[Equation]
Nonlinear Transforms.
For general P , we use the chain rule to differentiate the composed function y ◦ P and set the solver’s Jacobian to identity.
Therefore, for a given ω and an incoming gradient d (cid:96)/ d y , we return
[Equation]
[Equation] If P is linear, then P (cid:48) ( ω ) =
P and hence update (11) is consistent with linear case (10).
Intuitively, if we replace P ( ω − α ∆ I ω ) in the above-mentioned considerations by its afﬁne approximation
P ( ω ) − αP (cid:48) ( ω )∆ I ω , the term P (cid:48) ( ω ) plays locally the role of the linear projection.
Another view on invariant transforms is the following.
Consider our combinatorial layer as a composition of two sublayers.
First, given ω , we simply perform the map P ( ω ) and then pass it to the arg min solver.
Clearly, on the forward pass the transform P does not affect the solution y ( ω ) .
However, the derivative of the combinatorial layer is the composition of the derivatives of its sublayers, i.e. P (cid:48) ( ω ) for the transform and negative identity for the arg min .
Consequently, we get the very same gradient (11) on the backward pass.
In conclusion, enforcing guarantees on the forward pass by a mapping P is in this sense dual to projecting gradients onto Im P (cid:48) .
Examples.
In our experiments, we encounter two types of invariant mappings.
The ﬁrst one is the standard projection onto a hyperplane.
It is always applicable when all the solutions in Y are contained in a hyperplane, i.e. there exists a unit vector a ∈ R n and a scalar b ∈ R such that (cid:104) a, y (cid:105)
=
b for all y ∈ Y .
Consider the projection of ω onto the subspace orthogonal to a given by
P plane ( ω | a ) =
ω − (cid:104) a, ω (cid:105) a .
This results in
[Equation]
thereby fulﬁlling assumption (7).
This projection is relevant for the ranking and top- k experiment, in which the solutions live on a hyperplane with the normal vector a = 1 / √ n .
Therefore, the projection
[Equation]
is applied on the forward pass which simply amounts to subtracting the mean from the cost vector.
The other invariant mapping arises from the stability of the arg min solution to the magnitude of the cost vector.
Due to this invariance, the projection onto the unit sphere P norm ( ω ) = ω/ (cid:107) ω (cid:107) also fulﬁlls assumption (7).
As the invariance to the cost magnitude is independent of the solutions Y , normalization P norm is always applicable and we, therefore, test it in every experiment.
Observe that
[Equation]
) and the ﬁrst order approximation of P norm corresponds to the projection onto the tangent hyperplane given by a = ω and b = 1 .
When both P norm and P mean are applicable, we speak about standardization
P std =
P norm ◦ P mean .
An illustration of how differentiating through projections affects the resulting gradients is provided in Fig. 9 in Suppl.
D. An experimental comparison of the incoming gradient − d (cid:96)/ d y and the Identity update ∆ I ω 1 = − P (cid:48) d (cid:96)/ d y is examined in Suppl. B.1.
The Identity method can also be viewed as differentiating a continuous relaxation of the combinatorial solver on the backward pass.
To see this connection, we relax the solver by a) relaxing the polytope Y to another smooth set (cid:101) Y containing Y and b) adding a quadratic regularizer in case (cid:101) Y is unbounded:
[Equation]
Using this viewpoint, we can interpret the vanilla Identity method as the loosest possible relaxation (cid:101) Y = R n , and the projections P mean , P norm , and P std as relaxations of Y to a hyperplane, a sphere, and an intersection thereof, respectively. For a more detailed discussion, see Suppl.
C.2.
3. P REVENTING C OST C OLLAPSE AND I NDUCING M ARGIN
Any arg min solver (1) is a piecewise constant mapping that induces a partitioning of the cost space
W into convex cones W y
=
{ ω ∈ W : y ( ω ) =
y } , y ∈ Y , on which the solution does not change.
The backbone network attempts to suggest a cost ω ∈ W y that leads to a correct solution y .
However, if the predicted cost ω ∈ W y lies close to the boundary of W y , it is potentially sensitive to small perturbations.
The more partition sets { W y 1 , . . .
, W y k } meet at a given boundary point ω , the more brittle such a prediction ω is, since all the solutions { y 1 , . . .
, y k } are attainable in any neighbourhood of ω .
For example, the zero cost ω = 0 is one of the most prominent points, as it belongs to the boundary of all partition sets.
However, the origin is not the only problematic point, for example in the ranking problem, every point ω
= λ 1 , λ > 0 , is equally bad as the origin.
The goal is to achieve predictions that are far from the boundaries of these partition sets.
For Y = { 0 , 1 } n , Rolínek et al.
(2020a)
propose modifying the cost to ω (cid:48)
=
ω + α 2 y ∗ − α 2 (1 − y ∗ ) before the solver to induce an informed margin α .
However, this requires access to the ground-truth solution y ∗ .
We propose to instead add a symmetric noise ξ ∼ p ( ξ ) to the predicted cost before feeding it to the solver.
Since all the partition sets’ boundaries have zero measure (as they are of a lower dimension), this almost surely induces a margin from the boundary of the size E [ | ξ | ] .
Indeed, if the cost is closer to the boundary, the expected outcome will be inﬂuenced by the injected noise and incorrect solutions will increase the expected loss giving an incentive to push the cost further away from the boundary.
In experiments, the noise is sampled uniformly from {− α 2 , α 2 } n , where α > 0 is a hyperparameter.
In principle, careful design of the projection map P from Sec.
3.4 can also prevent instabilities.
This would require that Im P avoids the boundaries of the partition sets, which is difﬁcult to fully achieve in practice.
However, even reducing the size of the problematic set by a projection is beneﬁcial.
For example, the normalization P norm avoids brittleness around the origin, and P mean avoids instabilities around every ω
= λ 1 for ranking.
For the other—less signiﬁcant, but still problematic—boundaries, the noise injection still works in general without any knowledge about the structure of the solution set.
E XPERIMENTS
We present multiple experiments that show under which circumstances Identity achieves performance that is on par with or better than competing methods.
The experiments include backpropagating through discrete sampling processes, differentiating through a quadratic assignment solver in deep graph matching, optimizing for rank-based metrics in image retrieval, and a synthetic problem including a traveling salesman solver. The runtimes for all experiments are reported in Sec.
B.6. 4. B ACKPROPAGATING THROUGH D ISCRETE S AMPLERS
[Figure]
The sampling process of distributions of discrete random variables can often be reparameterized approximately as the solution to a noisy arg max optimization problem, see e.g. Paulus et al.
Sampling from the discrete probability distribution y ∼ p ( y ; ω ) can then be formulated as
[Equation]
with appropriate noise distribution (cid:15) .
Typically a Gumbel distribution is used for (cid:15) , but in certain situations, other distributions may be more suitable, e.g. Niepert et al.
(2021) use a sum-of-gamma noise distribution to model a top- k distribution on the solutions.
Therefore, sampling ﬁts our general hybrid architecture as illustrated in Fig.
Discrete Variational Auto-Encoder (DVAE).
In a DVAE (Rolfe, 2016), the network layers before the sampling solver represent the encoder and the layers after the sampling solver the decoder.
We consider the task of training a DVAE on the M NIST dataset where the encoder maps the input image to a discrete distribution of k -hot binary vector of length 20 in the latent space and the decoder reconstructs the image.
The loss is the Negative Evidence Lower Bound (N-ELBO), which is computed as the sum of the reconstruction losses (binary cross-entropy loss on output pixels) and the KL divergence between the marginals of the discrete latent variables and the uniform distribution.
We use the same setup and hyperparameters as Jang et al.
(2017) and Niepert et al.
(2021), details can be found in Suppl.
B.1.
Figure 3 shows a comparison of Identity to I-MLE (Niepert et al., 2021), which uses the BB update.
We observe that Identity (with standardization P std ) achieves a signiﬁcantly lower N-ELBO.
[Table]
Learning to Explain.
We consider the BeerAdvocate dataset (McAuley et al., 2012) that consists of reviews of different aspects of beer: appearance, aroma, taste, and palate and their rating scores.
The goal is to identify a subset k of the words in the text that best explains a given aspect rating (Chen et al., 2018;
Sahoo et al., 2021).
We follow the experimental setup of Niepert et al.
(2021), details can be found in
Suppl.
B.1.
As in the DVAE experiment, the problem is modeled as a k -hot binary latent representation with k = 5 , 10 , 15 .
Identity also uses the P std .
We compare Identity against I-MLE, L2X
(Chen et al., 2018), and Softsub (Xie and Ermon, 2019) in Tab. 1.
Softsub is a relaxation-based method designed speciﬁcally for subset sampling, in contrast to Identity and I-MLE which are generic.
We observe that Identity outperforms
I-MLE and L2X, and is on par with Softsub for all k .
Evaluating Identity without projection shows that projection is indispensable in this experiment.
4. D EEP G RAPH M ATCHING
[Table]
Given a source and a target image showing an object of the same class (eg. cat), each annotated with a set of keypoints (e.g. right ear), the task is to match the sets of keypoints from visual information without any access to the true keypoint annotation.
We follow the standard experimental setup from Rolínek et al.
(2020b), in which the cost matrices are predicted by a neural network from the visual information around the keypoints, for details see Suppl.
B.2.
We report results for the two benchmark datasets Pascal VOC (with Berkeley annotations)
(Everingham et al., 2010) and SPair-71k (Min et al., 2019).
Table 2 lists the average matching accuracy across classes.
We see that the use of a margin is important for the success of both evaluated methods, which is in line with the results reported by Rolínek et al.
(2020b).
We observe that the noise-induced margin achieves a similar effect as the informed margin from Rolínek et al.
(2020b).
Analogous results for SPair-71k are in the
Suppl.
B.2.
Our results show that Identity without the projection step yields performance that is on par with BB, while
Identity with the normalization step performs worse.
Such behaviour is not surprising.
Since the Hamming loss between the true and predicted matching is used for training, the incoming gradient for the solver directly points toward the desired solution, as discussed in 3.2.
Consequently, any projection step is rendered unnecessary, and possibly even harmful.
Interestingly, the evaluation of BB with normalization, a combination of methods that we only evaluated for completeness, appears to produce results that slightly outperform the standard BB.
4. O PTIMIZING R ANK B ASED M ETRICS – I MAGE R ETRIEVAL
[Table] [Table]
In image retrieval, the task is to return images from a database that are relevant to a given query.
The performance is typically measured with the recall metric, which involves computing the ranking of predicted scores that measure the similarity of image pairs.
However, directly optimizing for the recall is challenging, as the ranking operation is nondifferentiable.
We follow Rolínek et al.
(2020a) in formulating ranking as an arg min optimization problem with linear cost, i.e. rk ( ω ) =
arg min y ∈ Π n (cid:104) ω, y (cid:105) , where Π n is the set of all rank permutations and ω ∈ R n the vector of scores.
This allows us to use Identity to directly optimize an architecture for recall.
We perform experiments on the image retrieval benchmark CUB-200-2011 (Welinder et al., 2010), and follow the experimental setup used by Rolínek et al.
(2020a).
[Figure]
The results are shown in Tab. 3.
We report only the noise-induced margin here as the informed one performed similarly, see Suppl.
B.3.
Note that without the standardization P std , Identity exhibits extremely low performance, demonstrating that the projection step is a crucial element of the pipeline.
The explanation is that, for ranking-based loss, the incoming gradient does not point toward achievable solutions, as discussed in Sec. 3.2.
For more details, including additional evaluation of other applicable projections, see Suppl.
B.3.
We conclude that for the retrieval experiment, Identity does not match the performance of BB.
Presumably, this is because the crude approximation of the permutahedron by a sphere ignores too much structural information.
4. G LOBE T RAVELING S ALESMAN P ROBLEM
Finally, we consider the Traveling salesman Problem (TSP) experiment from Vlastelica et al.
Given a set of k country ﬂags, the goal for TSP( k ) is to predict the optimal tour on the globe through the corresponding country capitals, without access to the ground-truth location of the capitals.
A neural network predicts the location of a set of capitals on the globe, and after computing the pairwise distances of the countries, a TSP solver is used to compute the predicted optimal TSP tour, which is then compared to the ground-truth optimal tour.
Setup details can be found in Suppl.
B.4.
Corrupting Gradients.
We study the robustness of Identity and BB to potential perturbations during training in two scenarios: noisy gradients and corrupted labels.
We inﬂict the gradient with noise to simulate additional layers after the solver.
In this setting we add ξ ∼ N (0 , σ 2 ) to d (cid:96)/ d y .
In real-world scenarios, incorrect labels are inevitably occurring.
We study random ﬁxed corruptions of labels in the training data, by ﬂipping entries in y ∗ with probability ρ/k .
In Fig.
4a and Fig.
4b we observe that Identity performs on par with BB for the standard non-corrupted setting, and outperforms it in the presence of gradient perturbations and label corruptions.
Figure 4c shows the average norm of the cost vector w in the course of training under gradient noise with σ = 0 .
After epoch 50, as soon as α is set to 0 , cost collapse occurs quickly for BB, but not for
Identity.
In all the TSP experiments Identity uses standardization P std .
C ONCLUSION
We present a simple approach for gradient backpropagation through discrete combinatorial solvers with a linear cost function, by treating the solver as a negative identity mapping during the backward pass.
This approach, in conjunction with the exploitation of invariances of the solver via projections, makes up the Identity method.
It is hyperparameter-free and does not require any additional computationally expensive call to the solver on the backward pass.
We demonstrate in numerous experiments from various application areas that Identity achieves performance that is competitive with more complicated methods and is thereby a viable alternative.
We also propose and experimentally verify the use of noise to induce a margin on the solutions and ﬁnd that noise in conjunction with the projection step effectively increases robustness and prevents cost collapse.
Finally, we analyze the robustness of methods to perturbations during training that can come from subsequent layers or incorrect labels and ﬁnd that Identity is more robust than competing methods.
Promising areas of future work include the investigation of more complex post-processing steps after the solver, as well as a more detailed theoretical investigation of the effect of speciﬁc cost projections and their relationship to relaxations of the solver.
R EPRODUCIBILITY STATEMENT
We provide extensive experimental details and all used hyperparameters for Discrete Samplers in
Suppl.
B.1, Deep Graph Matching in Suppl.
B.2, Rank Based Metrics in Suppl.
B.3, Globe TSP in Suppl.
B.4, and Warcraft Shortest Path in Suppl.
B.5.
The datasets used in the experiments, i e.
BeerAdvocate (McAuley et al., 2012), M NIST (LeCun et al., 2010), SPair-71k (Min et al., 2019),
Globe TSP and Warcraft Shortest Path (Vlastelica et al., 2020), CUB-200-2011 (Welinder et al., 2010) are publicly available.
We further include Jupyter notebooks to reproduce Fig.
3 in the source code.
A curated Github repository for reproducing all other results will be published upon acceptance.
A S TRAIGHT T HROUGH E STIMATOR AND I DENTITY
The origins of the Straight-through estimator (STE) are in the perceptron algorithm (Rosenblatt, 1958) for learning single-layer perceptrons.
If the output function is a binary variable, the authors use the identity function as a proxy for the zero derivative of the hard thresholding function.
Hinton (2012) extended this concept to train multi-layer neural networks with binary activations, and Hubara et al.
(2016) used the identity to backpropagate through the sign function, which is known as the saturated STE. Bengio et al.
(2013) considered training neural networks with stochastic binary neurons modelled as random Bernoulli variables y ( p ) ∈ { 0 , 1 } , with p ∈ [0 , 1] , which are active (equal to one) with probability p and inactive (equal to zero) with probability 1 − p .
This can be reparametrized using the hard thresholding function as y ( p ) =
I [0 , ∞ ) ( (cid:15) − p ) , with (cid:15) ∼ Unif ([0 , 1]) .
In the next step, Bengio et al.
(2013) follow previous work by employing the identity as a replacement for the uninformative
Jacobian of the hard thresholding function, and coin the term Straight-through estimator for it. This application of the STE for differentiating through a discrete sampling process allows for an interesting probabilistic interpretation of the STE. To see this, note that the same Jacobian replacement as the one suggested by the STE, i.e. the identity function, is computed by treating the sample ˆ y on the backward pass as if it were the expected value of the stochastic neuron, i.e.
[Equation]
This interpretation has also been considered by Niepert et al.
; Jang et al.
(2017), as it allows the extension of the STE concept to more complicated distributions.
Most prominently, it has been applied in the setting of categorical distributions by Jang et al.
Encoding d classes as one hot vectors Y = { e i } di =1 , they ﬁrst deﬁne a distribution over classes as a discrete exponential family distribution
[Equation]
Following the STE assumption, we now want to sample from the distribution on the forward pass, and treat it on the backward pass as if it were the expectation of the distribution.
We can easily calculate the expectation as
[Equation]
and therefore the generalization of the STE gives the Jacobian replacement
[Equation]
Jang et al.
(2017) arrive at this result through a different reasoning by using the Gumbel-Argmax trick to reparametrize the categorical distribution as a stochastic argmax problem, and then relax the argmax to a softmax (see Paulus et al.
for a more detailed discussion).
Due to the derivation involving the Gumbel distribution, they refer to this estimator as the Straight-through Gumbel Estimator.
Note, that for general distributions, the expectation involved in applying an STE according to the previously described scheme is unfortunately intractable to compute, as exponentially many structures need to be considered in the expectation.
This is the case even for distributions as simple as the top- k distribution considered in Sec. 4.1.
Note, that this is the starting point for the motivation of
Niepert et al. (2021), who proceed by applying the concepts observed in Vlastelica et al. (2020) to differentiating through general constrained discrete exponential family distributions. From the previous examples, we see that our Identity method and the STE are closely related.
In fact, treating a hard thresholding function as identity on the backward pass is a special case of our method.
To see this, we only need to phrase the thresholding function as the solution to an optimization problem
[Equation]
for which the Identity method suggests the identity function as a Jacobian replacement, thereby matching the STE. From the discussion above, we know that this can also be interpreted in a probabilistic way for sampling from stochastic Bernoulli processes. However, it is also apparent that this correspondence does not extend to other distributions. To see this, we reconsider the case of a discrete exponential family distribution over the set of one-hot vectors Y .
Following Jang et al. (2017) and Maddison et al. (2017), we can simulate the sampling process by computing the solutions to a stochastic argmax problem, i.e. [Equation]
Here, G d (0 , 1) describes the d -dimensional Gumbel distribution. As described above, applying the STE yields the Jacobian of the softmax function as the replacement for the true zero Jacobian. In contrast, the Identity method (without projections), still returns the identity as the replacement.
The beneﬁt of this simplicity is that we can still apply our method to general distributions, where computing the expectation over all structures is intractable. As a ﬁnal remark, note that the term STE is somewhat overloaded, as in addition to the formulation above, it has been used to refer to any kind of operation in which some block is treated as an identity function on the backward pass. For example, Bengio et al. (2013) also consider the case of replacing on the backward pass not only the thresholding function, but also a preceding sigmoid computation with the identity function.
Yin et al. (2019) even refer to the STE for cases of replacing a thresholding function with a leaky relu function on the backward pass.
In this much looser sense, our Identity method also falls into the category of STEs, as we also replace a computational block with the identity function on the backward pass.
B E XPERIMENTAL D ETAILS B. B ACKPROPAGATING T HROUGH D ISCRETE S AMPLERS
DVAE on MNIST. Consider the models described by the equations θ = f e ( x ) , y ∼ p ( y ; θ ) , z
= f d ( y ) where x ∈ X is the input, o ∈ O is the output, and f e : X → θ is the encoder neural network that maps the input x to the logits θ and f d : Y → Z is the decoder neural network, and where Y is the set of all k -hot vectors.
Following Niepert et al.
(2021), we set (cid:15) in sampling (16) as the Sum-of-Gamma distribution given by
[Equation]
where s is a positive integer and Gamma( α, β ) is the Gamma distribution with ( α, β ) as the shape and scale parameters. [Figure]
We follow the same training procedure as Niepert et al.
The encoder and the decoder were feedforward neural networks with the architectures: 512 - 256 - 20 ×and 256 - 512 - 784 respectively.
We used M NIST (LeCun et al., 2010) dataset for the problem which consisted of 50 , 000 training examples and 10 , 000 validation and test examples each.
We train the model for 100 epochs and record the loss on the test data.
In this experiment, we use k = 10 i.e. sample 10 -hot binary vectors from the latent space.
We sample the noise from SoG with s = 10 and τ = 10 .
Additional Experimental Results. To assess the effect of the individual projections that make up P std , we evaluate our method using only P mean or P norm . The results reported in Fig. 5 show that both projections increase the performance individually (with a much larger effect of
P norm than P mean ), but combined they perform the best.
This is intuitively understandable in terms of viewing projections as solver relaxations (as described in Suppl. C.2).
No projection, mean projection, normalization, and standardization correspond to relaxations of Y to R n , a hyperplane H , a hypersphere S , and the intersection S ∩ H , respectively, which increasingly better approximate the true Y .
Additionally, we examine how the spurious irrelevant part of the update direction returned by the
Identity method without projection can lead to problematic behaviour.
First, we test whether adaptive optimizers contribute to the problematic behaviour by adapting the learning rate to the irrelevant component of the update.
As reported in Fig.
5, we observe a large performance gap between using projections and not using projections in the DVAE experiment.
Therefore we rerun this experiment with an SGD optimizer instead of Adam, to check whether a non-adaptive optimizer is able to outperform the adaptive one.
The results are reported in Fig.
6, where we observe a slightly improved performance from the SGD optimizer, but no major difference between the the optimizers.
Next, we examine how much differentiating through the projection affects the returned gradient in terms of magnitude and direction.
In Fig.
7 we compare the incoming gradient − d (cid:96)/ d y and the returned update direction ∆ I ω after differentiating through the projection in terms of norm ratio and cosine similarity.
We observe that the gradient is signiﬁcantly altered by the projection, i.e. the cosine similarity between the original and the projected gradient is smaller than 0 . 05 .
Given that with the projection the loss is optimized well (in contrast to not using projections), we can conclude that relevant information is indeed retained whereas spurious directions are removed.
[Figure]
over epoch.
[Figure]
Learning to Explain.
The entire dataset consists of 80k reviews for the aspect Appearance and 70k reviews for all the remaining aspects.
Following the experimental setup of Niepert et al.
(2021), the dataset was divided into 10 different evenly sized validation/test splits of the 10k held out set.
For this experiment, the neural network from Chen et al.
(2018) was used which had 4 convolutional anddense layer.
The neural network outputs the parameters θ of the pdf p( y ; θ ) over k -hot binary latent masks with k = 5 , 10 , 15 .
hyperparameter conﬁguration was the as that of Niepert et
We compute mean and standard deviation over 10 models, each trained on one split.
B. D EEP G RAPH M ATCHING
We closely follow the experimental setup in Rolínek et al.
(2020b), including the architecture of the neural network and the hyperparameter conﬁguration to train the network.
The architecture consists of a pre-trained backbone that, based on the visual information, predicts feature vectors for each of the keypoints in an image.
The keypoints are then treated as nodes of a graph, which is computed via Delauney triangulation.
A graph neural network then reﬁnes the keypoint features by employing a geometry-aware message passing scheme on the graph.
A matching instance is established by computing similarity scores between the edges and nodes of the keypoint graphs of two images via a learnable afﬁnity layer.
The negated similarity scores are then used as inputs to the graph matching solver, which produces a binary vector denoting the predicted matching.
This matching is ﬁnally compared to the ground-truth matching via the Hamming loss.
For a more detailed description of the architecture, see Rolínek et al.
(2020b).
We run experiments on SPair-71k (Min et al., 2019) and Pascal VOC (with Berkeley annotations) (Everingham et al., 2010).
The Pascal VOC dataset consists of images from 20 classes with up toannotated keypoints.
To prepare the matching instances we use intersection ﬁltering of keypoints, i.e. we only consider keypoints that are visible in both images of a pair.
For more details on this, we refer the reader to Rolínek et al.
(2020b).
The SPair-71k dataset consists of 70 , 958 annotated image pairs, with images from the Pascal VOC 2012 and Pascal 3D+ datasets.
It comes with a pre-deﬁned train–validation–test split of 53 , 340 – 5 , 384 – 12 , 234 .
We train all models for 10 epochs of 2000 training iterations each, with image pairs processed in batches of 8 .
As reported in Rolínek et al.
(2020b), we set the BB hyperparameter λ = 80 .
We use the Adam optimizer with an initial learning rate of 2 × 10 − 3 which is halved every 2 epochs.
The learning rate for ﬁnetuning the VGG weights is multiplied by 10 − 2 .
As in Rolínek et al.
(2020b)
we use a state-of-the-art dual block coordinate ascent solver (Swoboda et al., 2017) based on Lagrange decomposition to solve the combinatorial graph matching problem. {+We report the results from Tab. 
+}
again with their corresponding numbers in Tab. 4.
[Table]
B. R ANK - BASED M ETRICS – I MAGE R ETRIEVAL E XPERIMENTS
Ranking as a Solver.
Ranking can be cast as a combinatorial arg min optimization problem with linear cost by a simple application of the permutation inequality as described in Rolínek et al.
(2020a).
It holds that
[Equation]
where Π n is the set of all rank permutations and ω ∈ R n is the vector of individual scores.
A popular metric in the retrieval literature is recall at K , denoted by r @ K , which can be formulated using ranking as
[Equation]
where K ∈ N , ω ∈ R n are the scores, y ∗ ∈ { 0 , 1 } n are the ground-truth labels and rel( y ∗ )
= { i : y ∗ i = 1 } is the set of relevant label indices (positive labels).
Consequently, r @
K is going to be 1 if a positively labeled element is among the predicted top K ranked elements.
Due to computational restrictions, at training time, instead of computing r @
K over the whole dataset a sample-based version is used.
Recall Loss.
Since r @
K depends only on the top-ranked element from y ∗ , the supervision for r @
K is very sparse.
To circumvent this, Rolínek et al.
(2020a) propose a loss
[Equation]
where rk ( ω + ) i denotes the rank of the i -th element only within the relevant ones.
This loss is called loglog in the paper that proposed it.
We follow their approach and use this loss in all image retrieval experiments.
Experimental Conﬁguration.
We closely follow the training setup and procedure reported in
Rolínek et al.
(2020a).
The model consists of a pre-trained ResNet50 (He et al., 2016) backbone in which the ﬁnal softmax is replaced with a fully connected embedding layer, producing a 512dimensional embedding vector for each batch element.
These vectors are then used to compute pairwise similarities between batch elements.
The ranking of these similarities is then used to compute the recall loss (26), for additional details we refer the reader to Rolínek et al.
(2020a).
We train all models for 80 epochs using the Adam optimizer, using a learning rate of 5 × 10 − 7 , which was the best performing learning rate for both Identity and BB out of a grid search over 5 learning rates.
In all experiments a weight decay of 4 × 10 − 4 is used, as well as a drop of the learning rate by 70% after 35 epochs.
The learning rate of the embedding layer is multiplied by 3 , following previous work.
The BB parameter λ was set to 0 .
2 for all retrieval experiments.
Images are processed in batches of 128 , and three previous batches are always kept in memory to compute the recall to better approximate the recall over the whole dataset, see Rolínek et al.
(2020a) for more details.
Sparse Gradient & Difference between Identity and BB.
As described in Sec. 4.3, Identity performs worse than BB in the retrieval experiments.
The reason is the sparsity of the gradient generated from the loss (26): it only yields a nonzero gradient for the relevant images (positive labels), and provides no information for the images irrelevant to the query.
This sparseness makes training very inefﬁcient.
The BB update circumvents this by producing a new ranking y λ for a perturbed input.
The difference between the two rankings is then returned as the gradient, which contains information for both positive and negative examples.
This process is illustrated in Figure 8a. Ablations.
To assess the effect of the individual projections that make up P std , we also evaluate our method using only P mean or P norm and report them in Tab. 5.
We observe the same trend as for the DVAE ablations reported in Fig.
B.1, and therefore refer to the same explanation in terms of increasingly tight relaxations of the solution set.
B. G LOBE TSP
Dataset.
The training dataset for TSP( k ) consists of 10 000 examples where each datapoint has a k element subset sampled from 100 country ﬂags as input and the output is the optimal TSP tour represented by an adjacency matrix.
We consider datasets for k = 5 , 10 , 20 .
For additional details, see Vlastelica et al.
Architecture.
A set of k ﬂags is presented to a convolutional neural network that outputs k 3dimensional coordinates.
These points are then projected onto a 3D unit sphere and then used to construct the k × k distance matrix which is fed to a TSP solver.
Then the solver outputs the adjacency matrix indicating the edges present in the TSP tour.
The loss function is an L1 loss between the predicted tour and the true tour.
The neural network is expected to learn the correct coordinates of the countries’ capitals on Earth, up to rotations of the sphere.
We use the gurobi (Gurobi Optimization, [Figure]
BB Identity     100 epochs      t r a i n l o ss (b) Training for TSP( 5 ) for gradient noise level 0 . [Table]
For the Globe TSP experiment we use a convolutional neural network with 2 conv.
layers ((channels, kernel_size, stride) =
[[20, 4, 2], [50, 4, 2]]) and 1 fully connected layer of size 500 that predicts vector of dimension 3 k containing the k 3-dimensional representations of the respective countries’ capital cities.
These representations are projected onto the unit sphere and the matrix of pairwise distances is fed to the TSP solver.
The network was trained using Adam optimizer with a learning rate 10 − 4 for 100 epochs and a batch size of 50 .
For BB, the hyper-parameter λ was set to 20 .
As described in Sec.
3.6 adding noise to the cost vector can avoid cost collapse and act as a margininducing procedure.
We apply noise, ξ = 0 .
5 , for the ﬁrst 50 epochs (of 100 in total) to prevent cost collapse.
For this dataset, it was observed that ﬁnetuning the weights after applying noise helped improve the accuracy.
Margin is only important initially as it allows not getting stuck in a local optimum around zero cost.
Once avoided, margin does not play a useful role anymore because there are no large distribution shifts between the train and test set.
Hence, noise was not applied for the entirety of the training phase.
We veriﬁed the beneﬁts of adding noise experimentally in Tab. 6.
A difference between BB and Identity can be observed when applying gradient noise, simulating larger architectures where the combinatorial layer is followed by further learnable layers.
In Fig.
8b, we present the training loss curves in this case.
As we see, BB starts to diverge right after the [Table] margin-inducing noise is not added anymore i.e. after epoch 50 .
However, Identity converges as the training progresses.
B. W ARCRAFT S HORTEST P ATH
work similarly well.
Normalization P norm or noise does not affect the performance signiﬁcantly.
[Table]
We additionally present an evaluation of the Warcraft Shortest Path experiment from Vlastelica et al.
Here the aim is to predict the shortest path between the top-left and bottom-right vertex in a k × k Warcraft terrain map.
The path is represented by an indicator matrix of vertices that appear along the path.
The non-negative vertex costs of the k × k grid are computed by a modiﬁed Resnet18 (He et al., 2016) architecture using softplus on the output.
The network receives supervision in form of L loss between predicted and target paths.
We consider only the hardest case from the dataset with map sizes 32 × 32 .
Due to the non-uniqueness of solutions, we use the ratio between true and predicted path costs as an evaluation metric: (cid:104) ω ∗ , y ( ω ) (cid:105) /
(cid:104) ω ∗ , y ∗ (cid:105) , with ω ∗ being the ground truth cost vector.
Note that lower scores are better and 1 . 0 is the perfect score.
Table 7 shows that Identity performs comparatively to BB.
We follow the same experimental design as Vlastelica et al.
and do the same modiﬁcation to the ResNet18 architecture, except that we use softplus to make the network output positive.
The model is trained with Adam for 50 epochs with learning rate 5 × 10 − 3 .
The learning rate is divided by 10 after 30 epochs.
For the BB method we use λ = 20 .
The noise, when used, is applied for the whole duration of training. B. R UNTIMES
We report the runtimes of our method and BB for all experiments in Table 8.
Without surprise, we see the largest improvement of Identity over BB in terms of runtime when the underlying combinatorial problem is difﬁcult.
Finding the top- k indices or the ranking of a vector is extremely simple, and therefore this procedure is typically not the computational bottleneck of the architecture.
Consequently, we do not see any difference between the methods in terms of runtime for the discrete sampling experiments (DVAE and L2X) and the retrieval experiment.
In the matching experiment, the matching procedure is makes up for a relevant portion of training, which also observed in the runtime, as Identity requires less training time for the same number of training iterations. The runtime of the quadratic assignment solver scales with the number of keypoints in the image pairs, which is between between 9 and 30 in our experiments, and therefore the difference in runtime between the methods will also increase for instances with more keypoints to match. Finally, the TSP problem poses the hardest of the considered combinatorial problems, which is reﬂected in the almost halved runtime of the architecture employing Identity compared to using BB. Overall, we conclude that choosing between Identity and BB depends both on the difﬁculty of the combinatorial problem, and the performance of the two methods, which is affected by the quality of the incoming gradient as discussed in Sec. 3.3.
[Table]
C M ETHOD C. R ELATION TO N ON - LINEAR I MPLICIT D IFFERENTIATION
In the case of a linear program (1), the optimal solution is always located at one of the corners of the conex hull of the possible solutions, where the Jacobian of the optimal solution with respect to the ocost vector is zero.
Note, that this is in contrast to the case of optimization problems with non-linear objectives over convex regions, such as OptNet (Amos and Kolter, 2017) and Cvxpy (Agrawal et al., 2019a).
Intuitively, here the non-linearity often leads to solutions that are not located at such problematic corner points, and therefore the optimal solution usually has a continuously differentiable dependence on the objective parameters with non-zero Jacobian.
This Jacobian can be calculated by solving the KKT system on the backward pass, and therefore no informative gradient replacement is required, given a sufﬁciently strong regularization through the non-linearity.
C. P ROJECTION AS S OLVER R ELAXATION
As described in Sec. 3.5, we can also view the Identity method in terms of relaxations of the combinatorial solver.
From this perspective, the approach is to use the unmodiﬁed discrete solver on the forward pass but differentiate through a continuous relaxation on the backward pass.
To see this connection, we modify the solver by a) relaxing the polytope Y to another smooth set (cid:101)
Y containing
Y and b) adding a quadratic regularizer in case (cid:101) Y is unbounded, i.e.
[Equation]
[Equation]
The loosest possible relaxation is to set (cid:101) Y = R n .
This gives
[Equation]
(28) as the closed form solution of the relaxed problem, and differentiating through it corresponds to the re-scaled vanilla Identity method.
We can improve upon this by making the relaxation tighter.
As an example, we can restrict the solution space to a hyperplane H = { y ∈ R n |(cid:104) a, y (cid:105) =
b } for some unit vector a ∈ R n and scalar b ∈ R , and setting (cid:101) Y =
H leads to the closed-form solution (cid:101)
[Equation]
The Jacobian of this expression matches that of Identity with P plane up to scale.
In practice, we also often encounter the case in which Y is a subset of a sphere S = c + r S n − 1 , for some c ∈ R n , r > 0 .
This is the case in all of our experiments, as both the binary hypercube and the permutahedron are subsets of certain spheres.
Speciﬁcally, that is
[Equation]
for the binary hypercube, and
[Equation]
for the permutahedron.
Setting (cid:101) Y
= S gives the closed form solution
[Equation]
[Equation]
The Jacobian of this relaxed solution is proportional to that of Identity with projection P norm .
In our experiments, we also encounter the case in which the solutions are located on the intersection of a hyperplane H and a sphere S .
We can therefore tighten the relaxation further by (cid:101) Y
= S ∩ H .
Assuming without loss of generality that the sphere center is located on the hyperplane, we get the closed form solution [Equation] In the case of Y being the set of k -hot vectors or the permutahedron, we have a = 1 / √ n , which amounts to [Equation]
The Jacobian of this expression matches that of Identity using P std projection, which we derived originally by considering solver invariants in Sec. 3.4. This further strengthens the intuitive connection between projections in cost-space and relaxations in solution-space. This view of Identity is also related to existing work on structured prediction with projection oracles (Blondel, 2019), which uses (potentially non-differentiable) projections onto convex super-sets on both the forward- and backwards pass. C.
For an initial cost ω and a step size α > 0 , we set ω 0 = ω and ω
[Equation]
k ∈ N , (35) in which ∆ I ω k denotes the Identity update at the solution point y ( ω k ) , i.e.
[Equation]
We shall simply write d (cid:96)/ d y when no confusion is likely to happen.
Recall that the set of better solutions is deﬁned as
[Equation]
where f is the linearization of the loss (cid:96) at the point y deﬁned by [Equation]
[Equation]
In principle, ties in the solver may occur and hence the mapping ω (cid:55)→ y ( ω ) is not well-deﬁned for all cost ω unless we specify, how the ties are resolved.
Typically, this is not an issue in most of the considerations.
However, in our exposition, we need to avoid certain rare discrepancies.
Therefore, we assume that the solver will always favour the solution from the previous iteration if possible, i.e. from (cid:104) y ( ω k ) , ω k
(cid:105)
=
(cid:10) y ( ω k − 1 ) , ω k (cid:11) it follows that y ( ω k ) =
y ( ω k − 1 ) .
Theorem 2.
Assume that ( ω k ) ∞ k
=0 is the sequence as in (35) for some initial cost ω and step size α > 0 .
Then the following holds:
(i) Either Y ∗ (cid:0) y ( ω ) (cid:1) is non-empty and there is some α max > 0 such that for every α < α max there is n ∈ N
such that y ( ω n ) ∈
Y ∗ (cid:0) y ( ω ) (cid:1) and y ( ω k ) =
y ( ω ) for all k < n , (ii) or Y ∗ (cid:0) y ( ω ) (cid:1) is empty and for every α it is y ( ω k )
=
y ( ω ) for all k ∈ N .
We prove this statement in multiple parts.
Proposition 1.
Let α > 0 and ( ω k ) ∞
k
=0 be as in (35) .
If Y ∗ (cid:0) y ( ω ) (cid:1) is non-empty, then there exists n ∈ N such that y ( ω n ) (cid:54) =
y ( ω ) .
Proof.
We proceed by contradiction.
Assume that Y ∗ (cid:0) y ( ω ) (cid:1) is non-empty and y ( ω k )
=
y ( ω ) for all k ∈ N .
Take any y ∗ ∈
Y ∗ (cid:0) y ( ω ) (cid:1) .
By deﬁnition of Y ∗ (cid:0) y ( ω ) (cid:1) , we have that
[Equation]
As y ( ω k ) =
y ( ω ) for all k ∈ N , it is [Equation]
[Equation]
and therefore
[Equation] [Equation]
Since ξ < 0 , the latter term tends to minus inﬁnity.
Consequently, there exists some n ∈ N for which
[Equation]
contradicting the fact that y ( ω ) is the minimizer for ω n .
Let us now make a simple auxiliary observation about arg min solvers.
The mapping
[Equation]
is a piecewise constant function and hence induces a partitioning of its domain W into nonoverlapping sets on which the solver is constant.
Let us denote the pieces by
[Equation]
We claim that W y is a convex cone.
Indeed, if ω ∈ W y , clearly λω ∈ W y for any λ > 0 .
Next, if ω 1 , ω 2 ∈ W y and λ ∈ (0 , 1) , then y ( λω 1 ) =
y and y (cid:0) (1 − λ ) ω 2 (cid:1)
=
y and hence y is also a minimizer for λω 1 + (1 − λ ) ω 2 .
Proposition 2.
Assume that ( ω k ) ∞ k
=0 is the sequence as in (35) for some initial cost ω and step size α .
Then either for every α it is y ( ω k ) =
y ( ω ) for all k ∈ N , or there is some α max > 0 such that for every
α < α max there is n ∈ N such that y ( ω k ) =
y ( ω ) for all k < n and y ( ω n ) (cid:54) =
y ( ω ) .
Proof.
Let us deﬁne w : [0 , ∞ )
→ W and γ : [0 , ∞ )
→ Y as
[Equation]
respectively.
As γ is a composition of an afﬁne function w and a piecewise constant solver, it is itself a piecewise constant function.
Therefore, γ induces a partitioning of its domain [0 , ∞ ) into disjoint sets on which γ is constant.
In fact, these sets are intervals, say I 1 , . .
, I m , as intersections of the line segment { w ( α ) : α > 0 } and convex cones W y .
Consequently, m ≤ | Y | .
Now, If m = 1 , then I 1 = [0 , ∞ ) and y ( ω k ) is constant y ( ω ) for all k ∈ N whatever α > 0 is.
In the rest of the proof, we assume that m ≥ 2 .
Assume that the intervals I 1 , . .
, I m are labeled along increasing α , i.e. if α 1 ∈
I
i and α 2 ∈
I j then α 1 < α 2 if
and only if i < j .
We deﬁne the upper bound on the step size to α max =
| I 2 | .
Assume that α < α max and ( ω k ) ∞ k
=1 is given.
Let n = min { k ∈ N : w ( αk ) ∈
I 2 } , i.e. the ﬁrst index when the sequence y ( ω k ) switches.
Clearly y ( ω k ) = γ ( αk ) =
y ( ω ) for k = 0 , . .
. , n − 1 and y ( ω n ) =
γ ( αn ) (cid:54) =
y ( ω ) .
Proposition 3.
Let α > 0 and ( ω k ) ∞
k
=0 be as in (35) .
Assume that y ( ω k ) =
y ( ω ) for all k < n and y ( ω n ) (cid:54) =
y ( ω ) .
Also assume that from (cid:104) y ( w k ) , ω k (cid:105)
= (cid:10) y
( ω k − 1 ) , ω k (cid:11) it follows that y ( w k ) =
y ( ω k − 1 ) .
Then
[Equation]
46) where f is the linerarized loss at y ( ω ) .
Proof.
As y ( ω k ) =
y ( ω ) for all k < n , it is
[Equation]
Therefore, as y ( ω n ) is the minimizer for the cost ω k , we have
[Equation]
Now, since y ( ω ) is the minimizer for ω , it is (cid:10) y ( ω ) − y ( ω n ) , ω (cid:11) ≤ 0 and therefore
[Equation]
Consequently,
[Equation]
Equality is attained only if
[Equation]
This together with y ( ω n ) (cid:54) =
y ( ω ) =
y ( ω n − 1 ) violates the assumption that the solver ties are broken in favour of the previously attained solution, therefore the inequality in (50) is strict.
Proof of Theorem 2.
Assume Y ∗ (cid:0) y ( ω ) (cid:1) is non-empty.
It follows from Proposition 1 that there exists m ∈ N such that y ( ω m ) (cid:54) =
y ( ω ) .
It follows from Proposition 2 that there is some α max > 0 such that for every
α < α max there is n ∈ N such that y ( ω k ) =
y ( ω ) for all k < n and y ( ω n ) (cid:54) =
y ( ω ) .
From Proposition 3 it also follows that f (cid:0) y ( ω n ) (cid:1) <
f (cid:0) y ( ω ) (cid:1) .
Combining these results we have y ( ω n ) ∈
Y ∗ (cid:0) y ( ω ) (cid:1) by deﬁnition.
This proves the ﬁrst part of the theorem.
We prove the second part of the theorem by contradiction.
Assume that Y ∗ (cid:0) y ( ω ) (cid:1) is empty and there exists some α such that y ( ω k )
=
y ( ω ) for all k ∈ N and y ( ω n ) (cid:54) =
y ( ω ) .
From Propositionwe know that f (cid:0) y ( ω n ) (cid:1) <
f (cid:0) y ( ω ) (cid:1) and therefore y ( ω n ) ∈
Y ∗ (cid:0) y ( ω ) (cid:1) .
This is in contradiction to
Y ∗ (cid:0) y ( ω ) (cid:1) being empty.
D A DDITIONAL I LLUSTRATIONS
In addition to the illustrations for the ’simple’ case in Fig. 2, we provide additional illustrations for the more challenging case in Fig.
9, in which the incoming gradient does not point towards an attainable solution.
The illustrations compare the resulting update directions for Identity without projection,
Identity with P std projection, and Blackbox Backpropagation.