Improving Learning to Branch via Reinforcement Learning
Introduction
Mixed Integer Programming (MIP) has been applied widely in many real-world problems [1, 2].
Branch and Bound (B&B) is a general and widely used paradigm for solving MIP problems [3].
B&B recursively partitions the solution space into a search tree and compute relaxation bounds along the way to prune subtrees that provably can not contain an optimal solution.
This iterative process requires sequential decision makings: node selection : selecting the next solution space to evaluate, variable selection : selecting the variable by which to partition the solution space [4].
In this work, we focus on learning a variable selection strategy, which is the core of the B&B algorithm [5].
In this work, we argue that strong branching is not a good expert to imitate (More discussion in section B).
To obtain effective and non-myopic policies, we use reinforcement learning (RL) and model the variable selection process as a Markov Decision Process (MDP).
We design a policy network inspired by primal-dual iteration and employing a novelty search evolutionary strategy (NS-ES) to improve the policy.
For efﬁciency-effectiveness trade-off, the primal-dual policy ignores the redundant information and makes high-quality decisions on the ﬂy.
For reducing variance, the ES algorithm is an attractive choice as its gradient estimation is independent of the trajectory length [6].
For exploration, we introduce a new representation of the B&B solving process employed by novelty search [7] to encourage visiting new states.
We evaluate our RL trained agent over a range of problems (namely, set covering, maximum independent set, capacitated facility location).
The experiments show that our approach signiﬁcantly outperforms state-of-the-art human-designed heuristics [4] as well as imitation based learning methods [8, 9].
In the ablation study, we compare our primal-dual policy net with GCN [9], our novelty based ES with vanilla ES [6].
The results conﬁrm that both our policy network and the novelty search evolutionary strategy are indispensable for the success of the RL agent.
In summary, our main contributions are the followings:
• We point out the long-term overestimation of strong branching and suggest that methods other than imitating strong branching are needed to ﬁnd better variable selection policy.
• We model the variable selection process as MDP and design a novel policy net based on primal-dual iteration over reduced LP relaxation.
• We introduce a novel set representation and optimal transport distance for the branching process associated with a policy, based on which we train our RL agent using novelty search evolution strategy and obtain substantial improvements in empirical evaluation.
Method
Due to the pruning in B&B, a good variable selection policy can signiﬁcantly improve solving efﬁciency.
To illustrate how to improve variable selection policy, we organize this section in three parts.
First, we present our formulation of the variable selection process as a RL problem.
Next, we introduce the primal-dual based policy network.
Then, we introduce our branching process representation and the corresponding NS-ES training algorithm.
2.1 RL Formulation
Let the B&B algorithm and problem distribution D be the environment.
The sequential decision making of variable selection can be formulated as a Markov decision process.
We specify state space
S , action space A , transition P and reward r as follows
• State Space .
At iteration t , node selection policy will pop out a LP relaxation P LP from the problem set S .
We set the representation of the state to s t = { P LP , J, S } , where J is the index set of integer variables. • Action Space .
At iteration t , the action space is the index set of non-ﬁxed integer variables determined by the relaxation: A ( s t ) =
{ j ∈ J : (cid:96)
j < u j } .
• Transition .
Given state s t and action a t , the new state is determined by the node selection policy.
• Reward .
As our target is solving the problem faster, we set the reward r t = − 1 with discount γ = 1 .
Maximizing the cumulative reward encourages the agent solving problems with less steps.
2.2 Primal Dual Policy Net
Reduced LP .
In the solving process, the variable bounds keep changing due to branching.
Thus, we obtain our reduced LP relaxation by the following two steps: 1) remove ﬁxed variables x j , where (cid:96) j = u j , and plug their values into the constraints; 2) remove trivial constraints, where max (cid:96) ≤ x ≤ u (cid:80)
j A ij x j ≤ b
i .
In the view of primal-dual iteration, the LP relaxation has Lagrangian form:
[Equation]
where variables and constraints naturally form a bipartite graph.
Fixed variables and trivial constraints are nodes having no interaction with other nodes on the graph.
A more accurate description of the state can be obtained after removing these redundant nodes.
PD policy net .
We parameterize our policy network π θ ( a t | s t ) as a primal-dual iteration over the reduced LP relaxation by message passing
[Equation]
where f C , f C are two-layers neural networks, m C , m V are one layer neural networks, A ij is the entry in the reduced constraint matrix A and X , Y are the embedding for variables and constraints initialized by P LP and J .
After two iterations of Equation 2, the variable embedding X is passed to a two layer neural network score function f S and the output is the ﬁnal score for each variable.
Since the state reduction and message passing are both inspired by primal-dual iteration, we call it
PD policy.
A more detailed discussion and comparison with GCN [9] can be found at section C.2.
2.3 Set Representation for Policy and Optimal Transport Distance
We train the RL agent using evolution strategy similar to NSR-ES [7] and we need to deﬁne the novelty score for B&B process.
In the general B&B algorithm, the solving process can be represented by a search tree, where each leaf is a solved subproblem.
Given a branch policy π and an instance Q , we deﬁne our representation b ( π, Q ) =
{ R 1 , · · · , R H } as the collection of those leaf subproblems.
Focusing on MIP, a subproblem R i is a LP relaxation which can be represented by its feasible region, a polytope.
For example, in Figure 1, b 1 , b 2 and b 3 are the set of polytopes produced by three different policies π 1 , π 2 and π 3 respectively.
And b 1 = { A 1 , B 1 } is a set of two polytopes (leaf [-subproblems),b 
-]subproblems), b 2 = { A 2 , B 2 , C 2 } is a set of three polytopes, and b 3 = { A 3 , B 3 } is a set of two polytopes.
For computational efﬁciency, we ignore the constraints and only consider variable bounds such that every polytope is a box.
For each polytope R i (leaf subproblem), we deﬁne the weight function w ( · ) and distance function d ( · , · ) between two polytopes R i and R j as
• w ( R i ) :
= # { x ∈ R i : x is the feasible solution for Q }
A
B
C
A
B
A
B
Under review as a conference paper at ICLR
B
C 2 A
B
[Table] [Table]
p ( b 1 ) ,p ( b 2 ) ,W ) istheWassersteindistance.
Equippedmetric D betweencharacterization,wecandeﬁnethenoveltyscorefollowingContietal. (2018).
Given a policy memory M and an instance Q sampled from the problem distribution D , noveltyscoreiscomputedas: N ( ✓ ,Q,M ) = 1 k
X ⇡ j 2 kNN ( M, ✓ ) D ( b ( ⇡ ✓ ,Q ) ,b ( ⇡ j ,Q )) (6) where kNN ( M, ✓ ) is the k nearest neighbor of ⇡ ✓ in M .
In this deﬁnition, the novelty score encouragesthepolicieswithcharacterizationfarfromthecharacterizationsinpolicymemory.
FocusingonMIP,inthiswork,werepresentthesubproblem R
i asapolyhedronwhichisthefeasi- ble region
the integer variables of the LP relaxation.
The weight function w is deﬁned as counting number of feasible integer points and the distance function d is deﬁned as the 1  norm distance between the gravity center of two polyhedrons.
For computational efﬁciency, we ignore the con- straintsandonlyconsidervariableboundssuchthateverypolyhedronisabox.
Asimpleillustration ofcharacterizationisgiveninFigure2.
Wecanseethat p ( b 1 ) =
✓ 12 16 , 4 16 ◆ = ✓ 3 4 , 1 4 ◆ p ( b 2 ) =
✓ 8 16 , 4 16 , 4 16 ◆ = ✓ 1 2 , 1 4 , 1 4 ◆
A 1 B
C 1 A 2 B
[Figure]
We now present
[Figure]
where kNN ( M, ✓ ) is the k nearest neighbor of ⇡ ✓ in M .
In this deﬁnition, the novelty score encourages the policies with characterization far from the characterizations in policy memory.
[ Rephrase this paragraph.
Make it easier to understand.]
[Equation]
We have introduced the characterization and its metric for general B&B algorithm.
Focusing on MIP, a subproblem R i can be represented by a polytope, the feasible region of the LP relaxation.
We deﬁne the weight function w as counting the number of feasible integer points in the polytope and the distance function d as the 1  norm distance between the gravity center of two polytopes.
For computational efﬁciency, we ignore the constraints and only consider variable bounds such that every polytope is a box.
A simple illustration of characterizations is given in Figure 2.
We can compute the distance D ( b 1 ,b 2 )
=
Another observation is the polyhedron may degenerate to lower dimension after branching, for example, the B 1 in b 1 .
Hence, we choose the counting weight function w in our work [ Not so sure what this sentence means.] .
This ﬁnishes the deﬁnition of the novelty score.
To put everything together, we summarize the training in Algorithm 2.
5 E XPERIMENTS
We now present a comparative experiments against two competing machine learning approach and three SCIP’s branching rules to assess the value of our RL agent, as well as an ablation study to validate the our choice of state representation and training algorithm.
5.1 S ETUP
Benchmarks : We consider three classes of instances, Set Covering, Maximum Independent Set and Capacitated facility location, that are not only challenging for state-of-the-art solvers, but also representative for problems encountered in practice.
For each class, we set up a backbone based on which we randomly generate the dataset.
[ Need to justify why backbone idea is good: in
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
N ( ✓ ,Q,M ) = 1 k
X ⇡ j 2 kNN ( M, ✓ ) D ( b ( ⇡ ✓ ,Q ) ,b ( ⇡ j ,Q )) (7) where kNN ( M, ✓ ) is the k nearest neighbor of ⇡ ✓ in M .
In this deﬁnition, the novelty score encourages the policies with representation far from the representations in policy memory.
( b 1 ,b 2 )
[Equation]
[Table]
[Equation]
[Table] [Equation] [Table] [Figure]
For example, in Figure
[Figure] we have w ( A 1 ) = 12 , d ( A 1 , A 2 ) = 32 .
Then we can map the representation b
= { R 1 , · · · , R H } to a simplex p ( b ) ∈ ∆ H − 1 by normalizing the weights p ( R j ) = w ( R j ) / (cid:80) Hi =1 w ( R i ) , and compute a cost matrix W ij = d ( R i , R j ) (See Figure 1 for examples).
Then, we can deﬁne the metric D between two representations as the Wasserstein distance (or optimal transport distance)
[Equation]
For example, in Figure 1, the distance D ( b 1 , b 2 ) = 32 , D ( b 1 , b 3 ) =
34 meaning b 3 is closer to b 1 than b 2 .
Hence the corresponding policy π 3 is closer to π 1 than π 2 .
2.4 Novelty Search Evolutionary Strategy
Equipped with metric D between representations, we can deﬁne the novelty score following [7].
Given a policy memory M (a collection of older policies) and an instance Q sampled from the problem distribution D , novelty score is computed as:
[Equation]
where k NN ( M, θ ) is the k nearest neighbor of π θ in M .
Back to Algorithm 1, B&B algorithm recursively splits the feasible region and obtains a set of polytopes when ﬁnishing solving an instance.
Notice that a polytope in the set representation is invariant with the generating order, i.e. branching xthen x 2 will give the same polytope with branching x 2 then x 1 .
As a result, our metric D and novelty score N is mostly determined by the pruning behavior during the solving process.
Put everything together, we summarize the training algorithm in section D.
Experiments
We now present comparative experiments against two competing machine learning approaches and three SCIP’s branching rules to assess the value of our RL agent, as well as an ablation study to validate our choice of policy representation and training algorithm.
3.1 Setup
We evaluate the policies on three data set: set covering, maximum independent set, capacitated facility loacation.
We compare against: Reliability Pseudocost Branch (RPB) [4]; Full Strong Branching (FSB) [12]; Vanilla
Full Strong Branching (VFS) [12]; Support Vector Machine (SVM) rank [8] and
GCN approach [9].
We denote our method as RL, which is the primal-dual net trained by NS-ES.
More details can be found in section E.1.
3.2 Decision Quality
We evaluate the variable selection quality by solving 100 test instances under clean setting.
We say a method wins in this experiment if it results in the least number of solving nodes.
As FSB and RPB beneﬁt a lot from branching LP information (section B), we do not include them when counting Wins.
Table.
1 shows our RL agent leads the win times on all datasets and the average solving nodes on set covering, and independent set are signiﬁcantly better than other methods.
Our method can also generalize to larger unseen instances.
The result can be found in section [-E.
-]E.
[Table]
[Figure]
3.3 Improvement Analysis
Having seen the improvements brought by RL, we would like to ask what kind of decisions our agent learns.
We answer this question in two aspects: ﬁnding lower primal bound c ∗ and obtaining higher dual value ˆ c that allows pruning in line 5 Algorithm 1.
We compare our RL agent with GCN, SVM,
VFC on 100 maximum independent set test instances under clean setting.
We ﬁrst examine primal bound c ∗ .
Figure 2 plots the feasible solutions founded during the solving process.
A point ( n, y ) means we ﬁnd a feasible solution c ∗ =
y in a subproblem containing n branch constraints.
Figure 2 shows that our RL agent is able to detect small c ∗ at the early stage.
Hence, it can prune more subproblems and solve the MIP faster.
Then, we check dual value ˆ c . To eliminate the inﬂuence in primal bound c ∗ changing, we initialize c ∗ = c opt with the optimal value like [8]. We plot the curve of average width versus the depth in
Figure 3.
The area under the curve equals the average number of solving nodes, and we report it in the legend. Figure 3 shows that although our RL agent has the worst width in the beginning, it has the lowest peak and leads the overall performance.
This means our RL agent successfully employs a non-myopic policy to maximize ˆ c in the long term.
[Figure]
[Figure]
3.4 Ablation Study
We present an ablation study of our method on maximum independent set problem by comparing four types of RL agents: (1) PD policy + ES; (2) PD policy + NS-ES; (3) GCN + ES; (4) GCN + NS-ES.
We sample V = 200 instances as our validation set in and plot the average number of solving nodes on the validation set during the training process for ﬁve random seeds.
The results are plotted in Figure 4.
(1) and (2) having larger rewards than (3) and (4) shows that PD policy can obtain more improvement than GCN.
Also, (2) and (4) having larger rewards than (1) and (3) shows that novelty search helps to ﬁnd better policies.
The results suggest that both PD policy and NS-ES are indispensable in the success of RL agent.
Conclusion
We present an NS-ES framework to automatically learn the variable selection policy for MIP.
Central to our approach is the primal-dual policy network and the set representation of the B&B process.
We demonstrate our RL agent makes high-quality variable selection across different problems types and sizes.
Our results suggest that with carefully designed policy networks and learning algorithms, reinforcement learning has the potential to advance algorithms for solving MIP. A Background Mixed Integer Programming . MIP is an optimization problem, which is typically formulated as
[Equation]
where c ∈ R n is the objective vector, A ∈ R m × n is the constraint coefﬁcient matrix, b ∈ R m is the constraint vector, (cid:96) , u ∈ R n are the variable bounds.
The set J ⊆ { 1 , · · · , n } is an index set for integer variables. We denote the feasible region of x as X . Linear Programming Relaxation . LP relaxation is an important building block for solving MIP problems, where the integer constraints are removed:
[Equation]
Branch and Bound .
LP based B&B is the most successful method in solving MIP.
A typical
LP based B&B algorithm for solving MIP looks as Algorithm 1 [13].
It consists of two major decisions: node selection , in line 2, and variable selection , in line 7.
In this paper, we will focus on the variable selection .
Given a LP relaxation and its optimal solution ˆ x , the variable selection means selecting an index j .
Then, branching splits the current problem into two subproblems, each representing the original LP relaxation with a new constraint x j ≤ (cid:98) ˆ x j (cid:99) for Q − j and x j ≥ (cid:100)
ˆ x j (cid:101) for Q + j respectively.
This procedure can be visualized by a binary tree, which is commonly called search tree.
6 If ˆ x ∈ X , set x ∗ =
ˆ x , c ∗ =
ˆ c , go to 2 ; 7 Select variable j , split Q into two subproblems Q + j and Q − j , add them to S and go to 3 ;
Here we gives a simple illustration of B&B algorithm in Figure 5.
Given the LP relaxation, the polytope represents the feasible region of the LP relaxation and the red arrow represents the objective vector.
We ﬁrst solve the LP relaxation and obtain the solution ˆ x as the red point.
Noticing it is not feasible for MIP, we branch the LP relaxation into two subproblems.
In (a) we select to split variable x 1 and in (b) we select to split variable x 2 .
The subproblems obtained after branching are displayed by the shaded purple regions.
After ﬁnishing solve these two MIPs, we obtain the search trees t 1 and t 2 .
We can see that a wise selection of variable x 2 can solve the problem faster.
[Table] • divide into (disjoint) subproblems • improve local bounds Techniques • branching on variables • most infeasible • least infeasible • random branching • strong branching • pseudocost • reliability • VSIDS • hybrid reliability/inference • branching on constraints • SOS • SOS
47/ ( x 1 , x 2 ) =
(1,2) ( x 1 , x 2 ) =
(3,2) Infeasible x 2 ≥ 3 x 2 ≤x 1 ≥ 2 x 1 ≤xx1     (
a) split on x 1 and search tree t
Branching Rules
[Table] [Table] bounds Techniques • [Table]
on variables • most infeasible • least infeasible • random branching •
[Table]
branching [Table] • reliability • VSIDS • hybrid reliability/inference • branching on constraints • [Table]
Hendel – [Table] Introduction 47/ x 2 ≤x 2 ≥ [Table]
( x 1 , x 2 )
=
Infeasible xx1     (b) split on x 2 and search tree t
Evolution Strategy .
Evolution Strategies (ES) is a class of black box optimization algorithm [14].
In this work, we refer to the deﬁnition in Natural Evolution Strategies (NES) [15].
NES represents the population as a distribution of parameter vectors θ characterized by parameters φ : p φ ( θ ) .
Under a ﬁtness function, f ( θ ) , NES optimizes φ to maximize the average ﬁtness of the population E θ ∼ p φ [ f ( θ )] .
In recent work, [6] outlines a version of NES applied to standard RL benchmark problems.
In their work, θ parameterizes the policy π θ , p φ ( θ ) is a Gaussian distribution
N ( θ t , σ 2 I ) and f ( θ ) is deﬁned as the cumulative reward R ( θ ) over a full agent interaction.
At every iteration, [6] apply n additive Gaussian noises to the current parameter and update the population as
[Equation]
To encourage exploration, [7] propose Novelty Search Evolution Strategy (NS-ES).
In NS-ES, the ﬁtness function f ( θ ) =
λN ( θ )+(1 − λ ) R ( θ ) is selected as a combination of domain speciﬁc novelty score N and cumulative reward R , where λ is the balancing weight.
B Why Imitating Strong Branching is Not Good
Strong branching is a human-designed heuristic, which solves all possible branch LPs Q + j , Q − j ahead of branching.
As strong branching usually produces the smallest B&B search trees [16], many learning-based variable selection policy are trained by mimicking strong branching [9, 8, 17, 18].
However, we claim that strong branching is not a good expert: the reason strong branching can produce a small search tree is the reduction obtained in solving branch LP, rather than its decision quality.
Speciﬁcally, (i)
Strong branching can check lines 5, 6 in Algorithm 1 before branching.
If the pruning condition is satisﬁed, strong branching does not need to add the subproblem into the problem set S .
(ii)
Strong branching can strengthen other LP relaxations in the problem set S .
For example, if strong branching ﬁnds x 1 ≥ 1 and x 2 ≥ 1 can be pruned during solving branch LP, then any other LP relaxations containing x 1 ≥ 1 can be strengthened by adding x 2 ≤ 0 .
These two reductions are the direct consequence of solving branch LP, and they can not be learned by a variable selection policy.
To examine the decision quality of strong branching, we employ vanilla full strong branching [12], which takes the same decision as full strong branching, while the side-effect of solving branch LP is switched off.
Experiments in Section 3.2 show that vanilla full strong branching has poor decision quality.
Hence, imitating strong branching is not a wise choice for learning variable selection policy.
C Implementation C.1 Hardware
All the experiments were run at a Ubuntu 18.04 machine with Intel(R)
Xeon(R)
Silver 4116 CPU @ 2.10GHz, 256 GB Memory and Nvidia RTX 2080Ti graphic cards.
C.2 PD Policy
Comparison .
PD policy is similar to the GCN in [9] but has two major differences.
First, we use a dynamic reduced graph where ﬁxed variables and trivial constraints are removed due to the variable bounds changing during the solving process while [9] do not consider it.
The reduced graph can not only save computation, but also give a more accurate description of the solving state by ignoring the redundant information.
The ablation in Section 3.4 shows it is indispensable in the success of RL.
Second, we use a simple matrix multiplication in our PD policy while [9] use a complicated edge embedding in GCN.
In some sense, GCN can be seen as an overparameterized version of our method.
And our success reveals that message passing on the LP relaxation is the true helpful structure. detail .
We implement our primal dual policy net using dgl [19], with hidden dimension h = 64 and
ReLU activation.
The feature X for variable is a 17 dimension vector and feature Y for constraint is a 5 dimension vector.
We list the detail of feature in Table.
C.3 Baseline
FSB .
We use the implementation in SCIP [12]
VFS .
We use the implementation in SCIP [12]
RPB .
We use the implementation in SCIP [12]
GCN .
We tried to implement GCN in dgl [19], however, it is signiﬁcantly slower than the original implementation in [9].
Hence, we still use the implementation in [9].
[Table]
SVM .
We use the implementation in [9] [Table]
D Training
We have two settings clean , default .
In experiments, we always train and test under the same setting.
Imitation Learning .
We initialize our PD policy using imitation learning similar to [9].
The difference is we only use 10000 training samples, 2000 validation samples and 10 training epochs as a warm start.
In our setting, a policy from scratch can hardly solve an instance in a reasonable time, hence a warm start is necessary.
Novelty Search Evolution Strategy .
We improve our RL agent using Algorithm 2.
The parameters are set as α = 1 e − 4 , σ = 1 e − 2 , n = 40 , V = 200 , w = 0 .
25 , β = 0 .
99 , T = 1000 , k = 10 .
E Experiments E.1 Setup
Benchmarks : We consider three classes of instances, Set Covering [20], Maximum Independent
Set [21] and Capacitated facility location [22], those are not only challenging for state-of-the-art solvers, but also representative for problems encountered in practice.
For each class, we set up a backbone based on which we randomly generate the dataset as many real-world problems also share the same backbone.
For example, a logistics company frequently solves instances on very similar transportation networks with different customer demands.
We generate set covering instances using 1000 columns.
We train on instances with 500 rows and evaluate on instances with 500 rows (test), 1000 rows (medium transfer), 1500 rows (hard transfer).
We train maximum independent set on graphs with 400 nodes and evaluate on graphs with 400 nodes (test), 1000 nodes (medium transfer), and 1500 nodes (hard transfer).
We generate capacitated facility location with 100 facilities.
We train on instances with 40 customers (test) and evaluate on instances with 40 customers (test),customers (medium transfer), and 400 customers (hard transfer).
More details are provided in [-thesection E.
-]the section E.
Settings : Throughout all experiments, we use SCIP 7.0.1 as the backend solver, with a time limit ofhour.
For SCIP parameters, we have two settings: clean , a pure B&B algorithm with deep-ﬁrst-search as the node-selection policy and all other functions are disabled so as to eliminate the interference between variable selection and other components of the solver; default , the default setting in SCIP where all components are kept to default.
It is a state-of-the-art solver (environment) that is able to solve challenging problems.
Input
M ,
Output: Best parameter θ best 1 Sample validation instances Q 1 , · · · , Q V ∼ D 2 Set R best = 1 V (cid:80)
Vj
=1 R ( θ 0 , Q j ) , θ best = θ3 for t=0 to T do
Sample instances P 1 , · · · , P M ∼ D
Sample (cid:15) 1 , · · · , (cid:15) n ∼ N (0 , I ) and compute θ it =
θ t + σ(cid:15)
i
Set M = { θ 1 t , · · · , θ nt } for i=1 to n do
Compute R i =
1 m (cid:80)
Mm =1 R ( θ it , P m )
Compute N i = 1 m (cid:80) Mm =1 N ( θ it , P m , M ) end
Set θ t +
=
θ t + α 1 nσ (cid:80) ni
=1 λ · N i (cid:15) i + (1 − λ )
· R i (cid:15) i
Compute R ( t +1) =
1 V (cid:80) V j =1 R ( θ t +1 , Q j ) if R ( t +1) >
R best then
Set R best =
R ( t +1) , θ best = θ t +1 , λ =
β ∗ λ end 16 end
Baselines : We compare against: Reliability Pseudocost Branch (RPB) [4], the human-designed state-of-the-art branching rule, which computes strong branching in the beginning and gradually switches to simpler heuristics; Full Strong Branching (FSB), a full version of strong branching;
Vanilla Full Strong Branching (VFS), strong branching with branch LP information muted [12]; and two recent machine learning policies support vector machine (SVM) rank approach [8] and GCN approach [9] 1 .
We denote our method as RL, which is the primal-dual net trained by NS-ES.
Metrics .
To minimize the expected solving cost, metrics are selected as the average solving times ( T avg ) and average solving nodes ( N avg ).
Since MIP instances could vary a lot in difﬁculty, we count the number of times each method leads the performance over the number of times each method solves the instance within timelimit (Wins) as a third robust metric.
Implementation .
The detail of implementation is provided in section C
E.2 Generalization to Larger Instances
It is very important for RL agents to transfer to larger unseen instances as training on large instances is very expensive in the real world.
We investigate the generalization ability of our RL agent by solving 100 transfer instances under default setting.
To meet the needs in practice, we say a method wins in this experiment if it results in the fastest solving time.
As VFS is not able to solve any transfer instance in time limit, we do not list its results in Table.
We can see, except for RPB and SVM having comparable performance on hard set covering and hard facility location, respectively, the RL agent leads the performance.
[Table]
E.3 Data Set
Set Covering .
We generate a weighted set covering problem following [20].
The problem is formulated as the following ILP.
[Equation]
where U is the universe of elements, S is the universe of the sets, w is a weight vector.
For any e ∈ U and S ∈ S , e ∈ S with probability 0.05.
And we guarantee that for any e , it is contained by at least two sets in S .
Each w S is uniformly sampled from integer from 1 to 100 .
We ﬁrst generate a set covering problem with U
= { e 1 , · · · , e 400 } and S
= { S 1 , · · · , S 1000 } and set it as our backbone.
Then, every time we want to generate a new problem with m elements, we let
U =
U 0 ∪ { e 401 , e 402 , · · · , e m } add new e i into S ∈ S following the pipeline mentioned above.
Maximum Independent Set .
We generate maximum independent set problem using BarabasiAlbert [21] graphs.
The problem is formulated as the following ILP.
[Equation]
where V is the set of vertices and E is the set of edges.
We generate the BA graph using a preferential attachment with afﬁnity coefﬁcient 4.
We ﬁrst generate a BA graph G 0 with 350 nodes.
Then, every time we want to generate a new problem with n variables, we expand G 0 using preferential attachment.
Capacitated Facility Location .
We generate the capacitated facility location problem following [22].
The problem with m customers and n facilities is formulated as the following MIP.
min n (cid:88) i = m (cid:88)
j = c ij d j y ij + n (cid:88) i
=
f
i x i subject to n (cid:88) i =
y ij = 1 , ∀ j = 1 , · · · , m m (cid:88)
j = d j y ij ≤ u i x i , ∀ i = 1 · · · , n y ij ≥ 0 , ∀ i = 1 , · · · , n and j = 1 , · · · , m x i ∈ { 0 , 1 } , ∀ i = 1 , · · · , n where x i = 1 indicates facility i is open, and x i =
otherwise; f i is the ﬁxed cost if facility i is open; d j is the demand for customer j ; c ij is the transportation cost between facility j and customer i ; y ij is the fraction of the demand of customer j ﬁlled by facility i .
Following [22], where we ﬁrst sample the location of facility and customers on a 2 dimension map.
Then c i j is determined by the
Euclidean distance between facility i and customer j and other parameters are sampled from the distribtuion given in [22].
We ﬁrst generate the location of 100 facilities and 40 customers as our backbone.
Then, every time we want to generate a new problem with m customers, we generate new m − 40 locations for customers and follow the pipeline mentioned above.
F Discussion
An interesting phenomenon is that GCN can easily beat VFS after imitation learning (Or our PD policy can obtain similar result).
One possible explanation is that the primal-dual message passing is a principle structure that naturally learns the good decisions and ignores the noise brought by strong branching.
Another possible reason is the biased sampling.
To keep the diversity of the sample, [9] employs a mixed policy of RPB and VFS to sample the training data.
It is possible that VFS is a powerful expert on the state distribution determined by the mixed policy.
More studies are needed before we can answer this question.
Another point is our set representation is compatible with general B&B algorithm.
Once the weight function w and distance function d are deﬁned, we can compute the distance between two B&B solving processes.