Fit The Right NP-Hard Problem: End-to-end Learning of Integer Programming Constraints
Abstract
Bridging logical and algorithmic reasoning with modern machine learning techniques is a fundamental challenge with potentially transformative impact.
On the algorithmic side, many NP-Hard problems can be expressed as integer programs, in which the constraints play the role of their “combinatorial speciﬁcation”.
In this work, we aim to integrate integer programming solvers into neural network architectures by providing loss functions for both the objective and the constraints.
The resulting end-to-end trainable architectures have the power of jointly extracting features from raw data and of solving a suitable (learned) combinatorial problem with state-of-the-art integer programming solvers.
We experimentally validate our approach on artiﬁcial datasets created from random constraints, and on solving
K NAPSACK instances from their description in natural language.
Introduction
It is becoming increasingly clear that in order to advance artiﬁcial intelligence, we need to dramatically enhance the reasoning, algorithmic, logical and symbolic capabilities of data-driven models.
Only then can we aspire to match humans in their astonishing capability to perform complicated abstract tasks such as playing chess only based on visual input.
While there are decades worth of research directed at solving complicated abstract tasks from their abstract formulation , it seems very difﬁcult to align these methods with deep learning architectures needed for processing raw inputs.
Deep learning methods often struggle to implicitly acquire the abstract reasoning capabilities to solve and generalize to new tasks.
Recent work has investigated more structured paradigms that have more explicit reasoning components, such as layers capable of convex optimization.
In this paper, we focus on combinatorial optimization, which has been well-studied and captures non-trivial reasoning capabilities over discrete objects.
Enabling its unrestrained usage in machine learning models should fundamentally enrich the set of available components.
On the technical level, the main challenge of incorporating combinatorial optimization into the model typically amounts to non-differentiability of methods that operate with discrete inputs or outputs.
Three basic approaches to overcome this are to a) develop “soft” continuous versions of the discrete algorithms [41, 43]; b) adjust the topology of neural network architecture in order to express certain algorithmic behaviour [8, 22, 23]; c) provide an informative gradient approximation for the discrete algorithm [10, 38].
While the last strategy requires non-trivial theoretical considerations, when successful, it resolves the non-differentiability in the strongest possible sense; without any compromise on the performance of the original discrete algorithm.
This is the approach we take in this work.
Integrating integer linear programs (ILPs) as a modeling primitive is challenging because of the non-trivial dependency of the solution on the objective and constraints.
The parametrized objective has been addressed in Berthet et al.
[10], Ferber et al.
[20], Vlastelica et al.
[38], learnability of constraints is, however, unexplored.
At the same time the constraints of an ILP are of critical interest due to their remarkable expressive power .
Only by modifying the constraints one can formulate a number of diverse combinatorial problems ( SHORTEST - PATH , MATCHING , MAX - CUT , KNAPSACK ,
TRAVELLING SALESMAN ).
In that sense, learning ILP constraints corresponds to learning the combinatorial nature of the problem at hand.
More relevant literature can be found in Section A.1.
In this paper, we extend the loss functions of ILPs to cover their full speciﬁcation , effectively allowing a blackbox ILP solver as the last layer of a neural network.
This layer can jointly learn the objective and the constraints of the integer program, and as such it aspires to achieve universal combinatorial expressivity .
We validate this method on an inverse optimization task of learning synthetic constraints and on a knapsack modeling problem that learns constraints from word descriptions.
Background
Integer programming.
We assume the following general form of a bounded integer program:
[Equation]
where Y is a bounded subset of Z n , n ∈ N , c ∈ R n is the cost vector, y are the variables,
A =
[ a 1 , . .
a k ] ∈ R k × n is the matrix of constraint coefﬁcients and b ∈ R k is the bias term.
The point where the minimum is attained is denoted by y ( A , b , c ) .
Example.
The ILP formulation of the K NAPSACK problem can be written as
[Equation]
where c =
[ c 1 , . .
c n ] ∈ R are the prices of the items, w =
[ w 1 , . .
, w n ] ∈ R their weights and
W ∈ R the knapsack capacity.
Similar encodings can be found for many more - potentially NP- HARD - combinatorial optimization problems including those mentioned in the introduction.
Despite the apparent difﬁculty of solving
ILPs, modern highly optimized solvers [13, 24] can routinely ﬁnd optimal solutions to instances with thousands of variables.
Method
The task at hand is to provide gradients for the mapping ( A , b , c ) → y ( A , b , c ) , in which the triple ( A , b , c ) is the speciﬁcation of the ILP solver containing both the cost and the constraints, and y ( A , b , c ) ∈ Y is the optimal solution of the instance.
The main difﬁculty First of all, since there are ﬁnitely many available values of y , the mapping ( A , b , c ) →
y ( A , b , c ) , is piecewise constant; and as such, its true gradient is zero almost everywhere.
Indeed, a small perturbation of the costs or of the constraints does typically not cause a change in the optimal ILP solution.
The zero gradient has to be replaced by a suitable proxy.
In case of only interacting with the costs c , their direct inﬂuence (via the objective function) on the solution y allows for approaches with strong theoretical justiﬁcation [10, 38].
However, the inﬂuence of the constraints on the solution is much more intricate and requires different considerations.
Loss design Similarly to [10, 19, 35], we will focus on the case when the ILP solver is the ﬁnal layer of a network and we have access to the ground truth solutions y ∗ .
At this stage, we propose a loss function, where we distinguish the cases when y ∗ is feasible, i.e. Ay ∗ ≤ b , or if it is not.
For the cost, we rely on a known loss [19], which also has similarities to [10, 35, 38], and set
L cost ( y , y ∗ ) = (cid:26) c · ( y ∗ − y ) if y ∗ is feasible if y ∗ is infeasible.
From now on, we set y =
y ( A , b , c ) .
Note that L cost is non-negative since c · y ( c ) ≤ c · y ∗ .
For the difﬁcult part, the constraints, our loss is based on geometric understanding of the situation.
To that end, we use the Euclidean distance of a point y from a given hyperplane, parametrized by vector a and scalar b as x (cid:55)→ a · x − b : dist( a , b [Equation] =
[Equation] b | / (cid:107) a (cid:107) .
The infeasibility of y ∗ can be caused by one or more constraints.
To identify them, the indicator function I ( a · x > b ) equals to 1 if a · x ≤ b is violated and 0 otherwise.
We then deﬁne
L constraints ( y , y [Equation]
if y ∗ is feasible and y [Equation] is infeasible
[Equation]
The ﬁnal loss then takes the form of the sum of L cost and L constraints .
We capture the intuition behind the suggested constraints loss in Figure 1 and its caption.
Experiments
We validate our method on two experiments.
First we learn a static set of randomly initialized constraints from solved instances, while using access to the ground-truth cost vector c .
Then we approach the harder task of learning an input-dependent set of constraints and the cost vector jointly from the natural language description of solved KNAPSACK instances.
In all experiments we use
G UROBI [24] to solve the ILPs during training and evaluation.
Baselines We compare to a simple MLP baseline and to the differentiable linear programming solver provided by the CVXPy [15] implementation of the update rules presented in [4].
It solves the
LP relaxation of the ILP problem and rounds the solution to the nearest integer solution.
4.1 Random Constraints
Problem formulation The task is to learn constraints ( A , b ) corresponding to an ILP that, when applied to the ground-truth cost vector c , produces the ground-truth solutions y .
Dataset We generate 10 datasets for each cardinality of the ground truth constraint set while keeping the dimensionality of the ILP ﬁxed to n = 16 .
Each dataset ﬁxes a set of (randomly sampled) constraints ( A , b ) specifying the ground-truth feasible region.
Inputs of the dataset are sampled cost vectors c and the labels are corresponding solutions y precomputed ofﬂine by the solver.
The solution space Y is either constrained to [ − 5 , 5] n ( dense ) or [0 , 1] n ( binary ). [Figure]
Results The results are reported in Figure 2.
In the binary case we outperform both baselines signiﬁcantly, as the LP relaxation in the CVXPy baseline is far from tight and proposes many fractional solutions.
In the dense case, the LP relaxation becomes tighter and increases its performance, while the MLP baseline becomes signiﬁcantly worse.
4.2 Knapsack from Sentence Description
Problem formulation The task is inspired by a vintage text-based PC game called The Knapsack
Problem [31] in which a collection of 10 items is presented to a player including their prices and weights.
The player’s goal is to maximize the total price of selected items without exceeding the ﬁxed 100 kg capacity of their knapsack.
The aim is to solve instances of the NP-Hard KNAPSACK problem (2) from their word descriptions.
Here, the cost c and the constraint w is learned simultaneously.
Dataset Similarly to the game, we created a dataset containing 5000 instances.
An instance consists of 10 sentences, each describing one item.
The sentences are preprocessed via the sentence embedding [12] and the 10 resulting 4096 -dimensional vectors constitute the input of the dataset.
We rely on the ability of natural language embedding models to capture numerical values, as the other words in the sentence are uncorrelated with them (see an analysis in [40]).
The corresponding label is the indicator vector of the optimal solution to the described knapsack instance.
Results The results are listed in Figure 3.
The CVXPy baseline solves a standard LP relaxation with a greedy procedure for rounding.
We also evaluate the LP relaxation on the ground truth weights and prices, providing an upper bound for the result achievable by any method that relies on an LP relaxation, which underlines the NP-Hardness of KNAPSACK .
The ability to embed and differentiate through a dedicated ILP solver leads to surpassing this threshold even when learning from imperfect raw inputs.
A Appendix A.1 Related Work
Learning for combinatorial optimization Learning methods can powerfully augment classical combinatorial optimization methods with data-driven knowledge.
This includes work that learns how to solve combinatorial optimization problems to improve upon traditional solvers that are otherwise computationally expensive or intractable, e.g. by using reinforcement learning [9, 25, 29, 44], learning graph-based algorithms [36, 37, 42], learning to branch [6], solving SMT formulas [7] and TSP instances [26].
In a more general computational paradigm, Graves et al.
[22, 23] parameterize and learn Turing machines.
Optimization-based modeling for learning In the other direction, optimization serves as a useful modeling paradigm to improve applications of machine learning models and to add domain-speciﬁc structures and priors.
In the continuous setting differentiating through optimization problems is a foundational topic as it enables optimization to be seen as a layer and end-to-end learned [17, 21].
This approach has been recently studied in the convex setting in OptNet [4] for quadratic programs, and more general cone programs in Amos [3, Section 7.3] and Agrawal et al.
One use of this paradigm is to incorporate the knowledge of a downstream optimization-based task into a predictive model [18, 19].
Extending beyond the convex setting, optimization-based modeling and differentiable optimization is used for sparse structured inference [30], MAXSAT [41], submodular optimization [16] mixed integer programming [20], and discrete and combinational settings [10, 38].
Applications of optimization-based modeling include computer vision [32, 33], reinforcement learning [5, 14, 39], game theory [28], and inverse optimization [34], and meta learning [11, 27].