Original: I MPROVING L EARNING TO B RANCH VIA R EINFORCEMENT L EARNING
Revised: I MPROVING L EARNING TO B RANCH VIA R EINFORCEMENT L EARNING

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Branch-and-Bound (B&B) is a general and widely used algorithm paradigm for solving Mixed Integer Programming (MIP).
Revised: Branch-and-Bound (B&B) is a general and widely used algorithm paradigm for solving Mixed Integer Programming (MIP).

Original: Recently there is a surge of interest in designing learning-based branching policies as a fast approximation of strong branching, a humandesigned heuristic.
Revised: Recently there is a surge of interest in designing learning-based branching policies as a fast approximation of strong branching, a humandesigned heuristic.

Original: In this work, we argue that strong branching is not a good expert to imitate for its poor decision quality when turning off its side effects in solving branch linear programming.
Revised: In this work, we argue that strong branching is not a good expert to imitate for its poor decision quality when turning off its side effects in solving branch linear programming.

Original: To obtain more effective and non-myopic policies than a local heuristic, we formulate the branching process in MIP as reinforcement learning (RL) and design a novel set representation and distance function for the B&B process associated with a policy.
Revised: To obtain more effective and non-myopic policies than a local heuristic, we formulate the branching process in MIP as reinforcement learning (RL) and design a novel set representation and distance function for the B&B process associated with a policy.

Original: Based on such representation, we develop a novelty search evolutionary strategy for optimizing the policy.
Revised: Based on such representation, we develop a novelty search evolutionary strategy for optimizing the policy.

Original: Across a range of NP-hard problems, our trained RL agent signiﬁcantly outperforms expert-designed branching rules and the state-of-the-art learning-based branching methods in terms of both speed and effectiveness.
Revised: Across a range of NP-hard problems, our trained RL agent signiﬁcantly outperforms expert-designed branching rules and the state-of-the-art learning-based branching methods in terms of both speed and effectiveness.

Original: Our results suggest that with carefully designed policy networks and learning algorithms, reinforcement learning has the potential to advance algorithms for solving MIPs.
Revised: Our results suggest that with carefully designed policy networks and learning algorithms, reinforcement learning has the potential to advance algorithms for solving MIPs.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Mixed Integer Programming (MIP) has been applied widely in many real-world problems, such as scheduling (Barnhart et al., 2003) and transportation (Melo & Wolsey, 2012).
Revised: Mixed Integer Programming (MIP) has been applied widely in many real-world problems, such as scheduling (Barnhart et al., 2003) and transportation (Melo & Wolsey, 2012).

Original: Branch and Bound (B&B) is a general and widely used paradigm for solving MIP problems (Wolsey & Nemhauser, 1999).
Revised: Branch and Bound (B&B) is a general and widely used paradigm for solving MIP problems (Wolsey & Nemhauser, 1999).

Original: B&B recursively partitions the solution space into a search tree and compute relaxation bounds along the way to prune subtrees that provably can not contain an optimal solution.
Revised: B&B recursively partitions the solution space into a search tree and compute relaxation bounds along the way to prune subtrees that provably can not contain an optimal solution.

Original: This iterative process requires sequential decision makings: node selection : selecting the next solution space to evaluate, variable selection : selecting the variable by which to partition the solution space (Achterberg & Berthold, 2009).
Revised: This iterative process requires sequential decision makings: node selection : selecting the next solution space to evaluate, variable selection : selecting the variable by which to partition the solution space (Achterberg & Berthold, 2009).

Original: In this work, we focus on learning a variable selection strategy, which is the core of the B&B algorithm (Achterberg & Wunderling, 2013).
Revised: In this work, we focus on learning a variable selection strategy, which is the core of the B&B algorithm (Achterberg & Wunderling, 2013).

Original: Very often, instances from the same MIP problem family are solved repeatedly in industry, which gives rise to the opportunity for learning to improve the variable selection policy (Bengio et al., 2020).
Revised: Very often, instances from the same MIP problem family are solved repeatedly in industry, which gives rise to the opportunity for learning to improve the variable selection policy (Bengio et al., 2020).

Original: Based on the human-designed heuristics, Di Liberto et al.
Revised: Based on the human-designed heuristics, Di Liberto et al.

Original: (2016) learn a classiﬁer that dynamically selects an existing rule to perform variable selection; Balcan et al.
Revised: (2016) learn a classiﬁer that dynamically selects an existing rule to perform variable selection; Balcan et al.

Original: (2018) consider a weighted score of multiple heuristics and analyse the sample complexity of ﬁnding such a good weight.
Revised: (2018) consider a weighted score of multiple heuristics and analyse the sample complexity of ﬁnding such a good weight.

Original: The ﬁrst step towards learning a variable selection policy was taken by Khalil et al.
Revised: The ﬁrst step towards learning a variable selection policy was taken by Khalil et al.

Original: (2016), who learn an instance customized policy in an online fashion, as well as Alvarez et al.
Revised: (2016), who learn an instance customized policy in an online fashion, as well as Alvarez et al.

Original: (2017) and Hansknecht et al.
Revised: (2017) and Hansknecht et al.

Original: (2018) who learn a branching rule ofﬂine on a collection of similar instances.
Revised: (2018) who learn a branching rule ofﬂine on a collection of similar instances.

Original: Those methods need extensively feature engineering and require strong domain knowledge in MIP.
Revised: Those methods need extensively feature engineering and require strong domain knowledge in MIP.

Original: To avoid that, Gasse et al.
Revised: To avoid that, Gasse et al.

Original: (2019) propose a graph convolutional neural network approach to obtain competitive performance, only requiring raw features provided by the solver.
Revised: (2019) propose a graph convolutional neural network approach to obtain competitive performance, only requiring raw features provided by the solver.

Original: In each case, the branching policy is learned by imitating the decision of strong branching as it consistently leads to the smallest B&B trees empirically (Achterberg et al., 2005).
Revised: In each case, the branching policy is learned by imitating the decision of strong branching as it consistently leads to the smallest B&B trees empirically (Achterberg et al., 2005).

Original: In this work, we argue that strong branching is not a good expert to imitate.
Revised: In this work, we argue that strong branching is not a good expert to imitate.

Original: The excellent performance (the smallest B&B tree) of strong branching relies mostly on the information obtained in solving branch linear programming (LP) rather than the decision it makes.
Revised: The excellent performance (the smallest B&B tree) of strong branching relies mostly on the information obtained in solving branch linear programming (LP) rather than the decision it makes.

Original: This factor prevents learning a good policy by imitating only the decision made by strong branching.
Revised: This factor prevents learning a good policy by imitating only the decision made by strong branching.

Original: To obtain more effective and non-myopic policies, we use reinforcement learning (RL) and model the variable selection process as a Markov Decision Process (MDP).
Revised: To obtain more effective and non-myopic policies, we use reinforcement learning (RL) and model the variable selection process as a Markov Decision Process (MDP).

Original: Though the MDP formulation for MIP has been mentioned in the previous works (Gasse et al., 2019; Etheve et al., 2020), the advantage of RL has not been demonstrated clearly in literature.
Revised: Though the MDP formulation for MIP has been mentioned in the previous works (Gasse et al., 2019; Etheve et al., 2020), the advantage of RL has not been demonstrated clearly in literature.

Original: The challenges of using RL are multi-fold.
Revised: The challenges of using RL are multi-fold.

Original: First, the state space is a complex search tree, which can involve hundreds or thousands of nodes (with a linear program on each node) and evolve over time.
Revised: First, the state space is a complex search tree, which can involve hundreds or thousands of nodes (with a linear program on each node) and evolve over time.

Original: In the meanwhile, the objective of MIP is to solve problems faster.
Revised: In the meanwhile, the objective of MIP is to solve problems faster.

Original: Hence a trade-off between decision quality and computation time is required when representing the state and designing a policy based on this state representation.
Revised: Hence a trade-off between decision quality and computation time is required when representing the state and designing a policy based on this state representation.

Original: Second, learning a branching policy by RL requires rolling out on a distribution of instances.
Revised: Second, learning a branching policy by RL requires rolling out on a distribution of instances.

Original: Moreover, for each instance, the solving trajectory could contain thousands of steps and actions can have long-lasting effects.
Revised: Moreover, for each instance, the solving trajectory could contain thousands of steps and actions can have long-lasting effects.

Original: These result in a large variance in gradient estimation.
Revised: These result in a large variance in gradient estimation.

Original: Third, each step of variable selection can have hundreds of candidates.
Revised: Third, each step of variable selection can have hundreds of candidates.

Original: The large action set makes the exploration in MIP very hard.
Revised: The large action set makes the exploration in MIP very hard.

Original: In this work, we address these challenges by designing a policy network inspired by primal-dual iteration and employing a novelty search evolutionary strategy (NS-ES) to improve the policy.
Revised: In this work, we address these challenges by designing a policy network inspired by primal-dual iteration and employing a novelty search evolutionary strategy (NS-ES) to improve the policy.

Original: For efﬁciency-effectiveness trade-off, the primal-dual policy ignores the redundant information and makes high-quality decisions on the ﬂy.
Revised: For efﬁciency-effectiveness trade-off, the primal-dual policy ignores the redundant information and makes high-quality decisions on the ﬂy.

Original: For reducing variance, the ES algorithm is an attractive choice as its gradient estimation is independent of the trajectory length (Salimans et al., 2017).
Revised: For reducing variance, the ES algorithm is an attractive choice as its gradient estimation is independent of the trajectory length (Salimans et al., 2017).

Original: For exploration, we introduce a new representation of the
Revised: For exploration, we introduce a new representation of the

Original: B&B solving process employed by novelty search (Conti et al., 2018) to encourage visiting new states.
Revised: B&B solving process employed by novelty search (Conti et al., 2018) to encourage visiting new states.

Original: We evaluate our RL trained agent over a range of problems (namely, set covering, maximum independent set, capacitated facility location).
Revised: We evaluate our RL trained agent over a range of problems (namely, set covering, maximum independent set, capacitated facility location).

Original: The experiments show that our approach signiﬁcantly outperforms stateof-the-art human-designed heuristics (Achterberg & Berthold, 2009) as well as imitation based learning methods (Khalil et al., 2016; Gasse et al., 2019).
Revised: The experiments show that our approach signiﬁcantly outperforms stateof-the-art human-designed heuristics (Achterberg & Berthold, 2009) as well as imitation based learning methods (Khalil et al., 2016; Gasse et al., 2019).

Original: In the ablation study, we compare our primal-dual policy net with GCN (Gasse et al., 2019), our novelty based ES with vanilla ES (Salimans et al., 2017).
Revised: In the ablation study, we compare our primal-dual policy net with GCN (Gasse et al., 2019), our novelty based ES with vanilla ES (Salimans et al., 2017).

Original: The results conﬁrm that both our policy network and the novelty search evolutionary strategy are indispensable for the success of the RL agent.
Revised: The results conﬁrm that both our policy network and the novelty search evolutionary strategy are indispensable for the success of the RL agent.

Original: In summary, our main contributions are the followings:
Revised: In summary, our main contributions are the followings:

Original: • We point out the overestimation of the decision quality of strong branching and suggest that methods other than imitating strong branching are needed to ﬁnd better variable selection policy.
Revised: • We point out the overestimation of the decision quality of strong branching and suggest that methods other than imitating strong branching are needed to ﬁnd better variable selection policy.

Original: • We model the variable selection process as MDP and design a novel policy net based on primal-dual iteration over reduced LP relaxation.
Revised: • We model the variable selection process as MDP and design a novel policy net based on primal-dual iteration over reduced LP relaxation.

Original: • We introduce a novel set representation and optimal transport distance for the branching process associated with a policy, based on which we train our RL agent using novelty search evolution strategy and obtain substantial improvements in empirical evaluation.
Revised: • We introduce a novel set representation and optimal transport distance for the branching process associated with a policy, based on which we train our RL agent using novelty search evolution strategy and obtain substantial improvements in empirical evaluation.

Original:  B ACKGROUND
Revised:  B ACKGROUND

Original: Mixed Integer Programming .
Revised: Mixed Integer Programming .

Original: MIP is an optimization problem, which is typically formulated as
Revised: MIP is an optimization problem, which is typically formulated as

Original: [Equation]
Revised: [Equation]

Original: where c ∈ R n is the objective vector, A ∈ R m × n is the constraint coefﬁcient matrix, b ∈ R m is the constraint vector, (cid:96) , u ∈ R n are the variable bounds.
Revised: where c ∈ R n is the objective vector, A ∈ R m × n is the constraint coefﬁcient matrix, b ∈ R m is the constraint vector, (cid:96) , u ∈ R n are the variable bounds.

Original: The set J ⊆ { 1 , · · · , n } is an index set for integer variables.
Revised: The set J ⊆ { 1 , · · · , n } is an index set for integer variables.

Original: We denote the feasible region of x as X .
Revised: We denote the feasible region of x as X .

Original: Linear Programming Relaxation .
Revised: Linear Programming Relaxation .

Original: LP relaxation is an important building block for solving MIP problems, where the integer constraints are removed:
Revised: LP relaxation is an important building block for solving MIP problems, where the integer constraints are removed:

Original: [Equation]
Revised: [Equation]

Original: Input: A MIP P in form Equation
Revised: Input: A MIP P in form Equation

Original: Output: An optimal solution set x ∗ and optimal value c ∗ 1 Initialize the problem set S :
Revised: Output: An optimal solution set x ∗ and optimal value c ∗ 1 Initialize the problem set S :

Original: = { P LP } .
Revised: = { P LP } .

Original: where
Revised: where

Original: P LP is in form Equation 2.
Revised: P LP is in form Equation 2.

Original: Set x ∗ =
Revised: Set x ∗ =

Original: φ, c ∗ = ∞ ; 2 If S = φ , exit by returning x ∗ and c ∗ ; 3 Select and pop a LP relaxation Q ∈ S ; 4 Solve Q with optimal solution ˆ x and optimal value ˆ c ; 

Revised: φ, c ∗ = ∞ ; 2 If S = φ , exit by returning x ∗ and c ∗ ; 3 Select and pop a LP relaxation Q ∈ S ; 4 Solve Q with optimal solution ˆ x and optimal value ˆ c ;

Original: If ˆ c ≥ c ∗ , go to 2 ; 6 If ˆ x ∈ X , set x ∗
Revised: If ˆ c ≥ c ∗ , go to 2 ; 6 If ˆ x ∈ X , set x ∗

Original: =
Revised: =

Original: ˆ x , c ∗ =
Revised: ˆ x , c ∗ =

Original: ˆ c , go to 2 ; 7 Select variable j , split Q two 
Revised: ˆ c , go to 2 ; 7 Select variable j , split Q two subproblems

Original: Q + j and Q − j , add them to S and go to 3 ;
Revised: Q + j and Q − j , add them to S and go to 3 ;

Original: Branch and Bound .
Revised: Branch and Bound .

Original: LP based B&B is the most successful method in solving MIP.
Revised: LP based B&B is the most successful method in solving MIP.

Original: A typical LP based B&B algorithm for solving MIP looks as Algorithm 1 (Achterberg et al., 2005).
Revised: A typical LP based B&B algorithm for solving MIP looks as Algorithm 1 (Achterberg et al., 2005).

Original: It consists of two major decisions: node selection , in line 3 , and variable selection , in line 7.
Revised: It consists of two major decisions: node selection , in line 3, and variable selection , in line 7.

Original: In this paper, we will focus on the variable selection .
Revised: In this paper, we will focus on the variable selection .

Original: Given a LP relaxation and its optimal solution ˆ x , the variable selection means selecting an index j .
Revised: Given a LP relaxation and its optimal solution ˆ x , the variable selection means selecting an index j .

Original: Then, branching splits the current problem into two subproblems, each representing the original LP relaxation with a new constraint x j ≤ (cid:98) ˆ x j (cid:99) for Q − j and x j ≥ (cid:100)
Revised: Then, branching splits the current problem into two subproblems, each representing the original LP relaxation with a new constraint x j ≤ (cid:98) ˆ x j (cid:99) for Q − j and x j ≥ (cid:100)

Original: ˆ x j (cid:101) for Q + j respectively.
Revised: ˆ x j (cid:101) for Q + j respectively.

Original: This procedure can be visualized by a binary tree, which is commonly called search tree.
Revised: This procedure can be visualized by a binary tree, which is commonly called search tree.

Original: We give a simple visualization in Section A.1.
Revised: We give a simple visualization in Section A.1.

Original: Evolution Strategy .
Revised: Evolution Strategy .

Original: Evolution Strategies (ES) is a class of black box optimization algorithm (Rechenberg, 1978).
Revised: Evolution Strategies (ES) is a class of black box optimization algorithm (Rechenberg, 1978).

Original: In this work, we refer to the deﬁnition in Natural Evolution Strategies (NES)
Revised: In this work, we refer to the deﬁnition in Natural Evolution Strategies (NES)

Original: (Wierstra et al., 2008).
Revised: (Wierstra et al., 2008).

Original: NES represents the population as a distribution of parameter vectors θ characterized by parameters φ : p φ ( θ ) .
Revised: NES represents the population as a distribution of parameter vectors θ characterized by parameters φ : p φ ( θ ) .

Original: NES optimizes φ to maximize the expectation of a ﬁtness f ( θ ) over the population E θ ∼ p φ [ f ( θ )] .
Revised: NES optimizes φ to maximize the expectation of a ﬁtness f ( θ ) over the population E θ ∼ p φ [ f ( θ )] .

Original: In recent work, Salimans et al.
Revised: In recent work, Salimans et al.

Original: (2017) outlines a version of NES applied to standard RL benchmark problems, where θ parameterizes the policy π θ , φ t =
Revised: (2017) outlines a version of NES applied to standard RL benchmark problems, where θ parameterizes the policy π θ , φ t =

Original: ( θ t , σ ) parameterizes a Gaussian distribution p φ ( θ )
Revised: ( θ t , σ ) parameterizes a Gaussian distribution p φ ( θ )

Original: =
Revised: =

Original: N ( θ t , σ 2 I ) and f ( θ ) is the cumulative reward R ( θ ) over a full agent interaction.
Revised: N ( θ t , σ 2 I ) and f ( θ ) is the cumulative reward R ( θ ) over a full agent interaction.

Original: At every iteration, Salimans et al.
Revised: At every iteration, Salimans et al.

Original: (2017) apply n additive Gaussian noises to the current parameter and update the population as
Revised: (2017) apply n additive Gaussian noises to the current parameter and update the population as

Original: [Equation]
Revised: [Equation]

Original: To encourage exploration, Conti et al.
Revised: To encourage exploration, Conti et al.

Original: (2018) propose Novelty Search Evolution Strategy (NS-ES).
Revised: (2018) propose Novelty Search Evolution Strategy (NS-ES).

Original: In NSES, the ﬁtness function f ( θ ) =
Revised: In NSES, the ﬁtness function f ( θ ) =

Original: λN ( θ )+(1 − λ ) R ( θ ) is selected as a combination of domain speciﬁc novelty score N and cumulative reward R , where λ is the balancing weight.
Revised: λN ( θ )+(1 − λ ) R ( θ ) is selected as a combination of domain speciﬁc novelty score N and cumulative reward R , where λ is the balancing weight.

Original:  W HY I MITATING S TRONG B RANCHING IS N OT G OOD
Revised:  W HY I MITATING S TRONG B RANCHING IS N OT G OOD

Original: Strong branching is a human-designed heuristic, which solves all possible branch LPs Q + j , Q − j ahead of branching.
Revised: Strong branching is a human-designed heuristic, which solves all possible branch LPs Q + j , Q − j ahead of branching.

Original: As strong branching usually produces the smallest B&B search trees (Achterberg, 2009), many learning-based variable selection policy are trained by mimicking strong branching (Gasse et al., 2019;
Revised: As strong branching usually produces the smallest B&B search trees (Achterberg, 2009), many learning-based variable selection policy are trained by mimicking strong branching (Gasse et al., 2019;

Original: Khalil et al., 2016; Alvarez et al., 2017; Hansknecht et al., 2018).
Revised: Khalil et al., 2016; Alvarez et al., 2017; Hansknecht et al., 2018).

Original: However, we claim that strong branching is not a good expert: the reason strong branching can produce a small search tree is the reduction obtained in solving branch LP, rather than its decision quality.
Revised: However, we claim that strong branching is not a good expert: the reason strong branching can produce a small search tree is the reduction obtained in solving branch LP, rather than its decision quality.

Original: Speciﬁcally, (i)
Revised: Speciﬁcally, (i)

Original: Strong branching can check lines 5, 6 in Algorithm 1 before branching.
Revised: Strong branching can check lines 5, 6 in Algorithm 1 before branching.

Original: If the pruning condition is satisﬁed, strong branching does not need to add the subproblem into the problem set S .
Revised: If the pruning condition is satisﬁed, strong branching does not need to add the subproblem into the problem set S .

Original: (ii)
Revised: (ii)

Original: Strong branching can strengthen other LP relaxations in the problem set S .
Revised: Strong branching can strengthen other LP relaxations in the problem set S via domain propagation (Rodosek et al., 1999) and conﬂict analysis (Achterberg, 2007).

Original: For example, if strong branching ﬁnds x 1 ≥ 1 and x 2 ≥ 1 can be pruned during solving branch LP, then any other LP relaxations containing x 1 ≥ 1 can be strengthened by adding x 2 ≤ 0 .
Revised: For example, if strong branching ﬁnds x 1 ≥ 1 and x 2 ≥ 1 can be pruned during solving branch LP, then any other LP relaxations containing x 1 ≥ 1 can be strengthened by adding x 2 ≤ 0 .

Original: These two reductions are the direct consequence of solving branch LP, and they can not be learned by a variable selection policy.
Revised: These two reductions are the direct consequence of solving branch LP, and they can not be learned by a variable selection policy.

Revised: (iii)

Revised: Strong branching activates primal heuristics (Berthold, 2006) after solving LPs.

Original: To examine the decision quality of strong branching, we employ vanilla full strong branching (Gamrath et al., 2020), which takes the same decision as full strong branching, while the side-effect of solving branch
Revised: To examine the decision quality of strong branching, we employ vanilla full strong branching (Gamrath et al., 2020), which takes the same decision as full strong branching, while the side-effect of solving branch

Original: LP is switched off.
Revised: LP is switched off.

Original: Experiments in Section 5.2 show that vanilla full strong branching has poor decision quality.
Revised: Experiments in Section 5.2 show that vanilla full strong branching has poor decision quality.

Original: Hence, imitating strong branching is not a wise choice for learning variable selection policy.
Revised: Hence, imitating strong branching is not a wise choice for learning variable selection policy.

Original:  M ETHOD
Revised:  M ETHOD

Original: Due to line 5 in Algorithm 1, a good variable selection policy can signiﬁcantly improve solving efﬁciency.
Revised: Due to line 5 in Algorithm 1, a good variable selection policy can signiﬁcantly improve solving efﬁciency.

Original: To illustrate how to improve variable selection policy, we organize this section in three parts.
Revised: To illustrate how to improve variable selection policy, we organize this section in three parts.

Original: First, we present our formulation of the variable selection process as a RL problem.
Revised: First, we present our formulation of the variable selection process as a RL problem.

Original: Next, we introduce the LP relaxation based state representation and the primal-dual based policy network.
Revised: Next, we introduce the LP relaxation based state representation and the primal-dual based policy network.

Original: Then, we introduce our branching process representation and the corresponding NS-ES training algorithm.
Revised: Then, we introduce our branching process representation and the corresponding NS-ES training algorithm.

Original: 4. RL F ORMULATION
Revised: 4. RL F ORMULATION

Original: Let the B&B algorithm and problem distribution D be the environment.
Revised: Let the B&B algorithm and problem distribution D be the environment.

Original: The sequential decision making of variable selection can be formulated as a Markov decision process.
Revised: The sequential decision making of variable selection can be formulated as a Markov decision process.

Original: We specify state space S , action space
Revised: We specify state space S , action space

Original: A , transition P and reward r as follows
Revised: A , transition P and reward r as follows

Original: • State Space .
Revised: • State Space .

Original: At iteration t , node selection policy will pop out a LP relaxation P LP from the problem set S .
Revised: At iteration t , node selection policy will pop out a LP relaxation P LP from the problem set S .

Original: We set the representation of the state to s t = { P LP , J, S } , where J is the index set of integer variables.
Revised: We set the representation of the state to s t = { P LP , J, S } , where J is the index set of integer variables.

Original: • Action Space .
Revised: • Action Space .

Original: At iteration t , the action space is the index set of non-ﬁxed integer variables determined by the relaxation: A ( s t ) =
Revised: At iteration t , the action space is the index set of non-ﬁxed integer variables determined by the relaxation: A ( s t ) =

Original: { j ∈ J : (cid:96)
Revised: { j ∈ J : (cid:96)

Original: j < u j } .
Revised: j < u j } .

Original: • Transition .
Revised: • Transition .

Original: Given state s t and action a t , the new state is determined by the node selection policy.
Revised: Given state s t and action a t , the new state is determined by the node selection policy.

Original: • Reward .
Revised: • Reward .

Original: As our target is solving the problem faster, we set the reward r t = − 1 with discount γ = 1 .
Revised: As our target is solving the problem faster, we set the reward r t = − 1 with discount γ = 1 .

Original: Maximizing the cumulative reward encourages the agent solving problems with less steps.
Revised: Maximizing the cumulative reward encourages the agent solving problems with less steps

Original: In commercial solver, the solving process is much more complicated than the B&B stated in Algorithm 1.
Revised: In commercial solver, the solving process is much more complicated than the B&B stated in Algorithm 1.

Original: For example, between line 3 and line 4, primal heuristics could be used to detect feasible solutions, and cutting planes could be applied to strengthen the LP relaxation.
Revised: For example, between line 3 and line 4, primal heuristics could be used to detect feasible solutions, and cutting planes could be applied to strengthen the LP relaxation.

Original: These components in solver introduce more randomness in transition, but our formulation is still valid
Revised: These components in solver introduce more randomness in transition, but our formulation is still valid

Original: . 4. P RIMAL D UAL P OLICY N ET
Revised: . 4. P RIMAL D UAL P OLICY N ET

Original: Reduced LP .
Revised: Reduced LP .

Original: In the solving process, the variable bounds keep changing due to branching.
Revised: In the solving process, the variable bounds keep changing due to branching.

Original: Thus, we obtain our reduced LP relaxation by the following two steps: 1) remove ﬁxed variables x j , where (cid:96) j = u j , and plug their values into the constraints; 2) remove trivial constraints, where max (cid:96) ≤ x ≤ u (cid:80)
Revised: Thus, we obtain our reduced LP relaxation by the following two steps: 1) remove ﬁxed variables x j , where (cid:96) j = u j , and plug their values into the constraints; 2) remove trivial constraints, where max (cid:96) ≤ x ≤ u (cid:80)

Original: j A ij x j ≤ b
Revised: j A ij x j ≤ b

Original: i .
Revised: i .

Original: In the view of primal-dual iteration, the LP relaxation has Lagrangian form:
Revised: In the view of primal-dual iteration, the LP relaxation has Lagrangian form:

Original: [Equation]
Revised: [Equation]

Original: where variables and constraints naturally form a bipartite graph.
Revised: where variables and constraints naturally form a bipartite graph.

Original: Fixed variables and trivial constraints are nodes having no interaction with other nodes on the graph.
Revised: In the primal-dual iteration over Equation 4, ﬁxed variables and trivial constraints always pass zero and have no interaction with other variables.

Original: A more accurate description of the state can be obtained after removing these redundant nodes.

Original: PD policy net .
Revised: PD policy net .

Original: We parameterize our policy network π θ ( a t | s t ) as a primal-dual iteration over the reduced
Revised: We parameterize our policy network π θ ( a t | s t ) as a primal-dual iteration over the reduced

Original: LP relaxation by message passing
Revised: LP relaxation by message passing

Original: [Equation]
Revised: [Equation]

Original: where f C , f V are two-hidden-layers neural networks, m C , m V are one hidden layer neural networks, A ij is the entry in the reduced constraint matrix A and X , Y are the embedding for variables and constraints initialized by P LP and J .
Revised: where f C , f V are two-hidden-layers neural networks, m C , m V are one hidden layer neural networks, A ij is the entry in the reduced constraint matrix A and X , Y are the embedding for variables and constraints initialized by P LP and J .

Original: For efﬁciency, we do not include problem set S , which makes it a partial observable MDP (Astrom, 1965). After two iterations of Equation 5, the variable embedding X is passed to a two layer neural network score function f S and the output is the ﬁnal score for each variable. Since the state reduction and message passing are both inspired by primal-dual iteration, we call it PD policy.
Revised: As mentioned above, the original primal-dual iterations only occurs on the reduced LP hence, our message passing in Equation 5 is deﬁned only on the reduced graph.

Original: A more detailed discussion and comparison with GCN (Gasse et al., 2019) can be found at section A.2.2.
Revised: For efﬁciency, we do

Original: A
Revised: A

Original: B
Revised: B

Original: C
Revised: C

Original: A
Revised: A

Original: B
Revised: B

Original: A
Revised: A

Original: B
Revised: B

Original: Underreview paper at ICLR
Revised: Under paper at ICLR

Original: [Table]

Original: A B 
Revised: B C

Original: [Table]
Revised: [Table]

Original:  D ( b 1 ,b 2 ) =
Revised: that D ( b 1 ,b 2 ) =

Original: OT ( p ( b 1 ) ,p ( b 2 ) ,W ) istheWassersteindistance.
Revised: OT ( p ( b 1 ) ,p ( b 2 ) ,W ) istheWassersteindistance.

Original: Equippedmetric D betweencharacterization,wecandeﬁnethenoveltyscorefollowingContietal. (2018).
Revised: Equippedmetric D betweencharacterization,wecandeﬁnethenoveltyscorefollowingContietal. (2018).

Original: Given a policy memory M and an instance Q sampled from the problem distribution D , noveltyscoreiscomputedas: N ( ✓ ,Q,M ) = 1 k
Revised: Given a policy memory M and an instance Q sampled from the problem distribution D , noveltyscoreiscomputedas: N ( ✓ ,Q,M ) = 1 k

Original: X ⇡ j 2 kNN ( M, ✓ ) D ( b ( ⇡ ✓ ,Q ) ,b ( ⇡ j ,Q )) (6) where kNN ( M, ✓ ) is the k nearest neighbor of ⇡ ✓ in M .
Revised: X ⇡ j 2 kNN ( M, ✓ ) D ( b ( ⇡ ✓ ,Q ) ,b ( ⇡ j ,Q )) (6) where kNN ( M, ✓ ) is the k nearest neighbor of ⇡ ✓ in M .

Original: In this deﬁnition, the novelty score encouragesthepolicieswithcharacterizationfarfromthecharacterizationsinpolicymemory.
Revised: In this deﬁnition, the novelty score encouragesthepolicieswithcharacterizationfarfromthecharacterizationsinpolicymemory.

Original: Focusing on MIP, in this work, we represent the subproblem R i as a polyhedron which is the feasi- ble region the integer variables of the LP relaxation.
Revised: Focusing on MIP, in this work, we represent the subproblem R i as a polyhedron which is the feasi- ble region the integer variables of the LP relaxation.

Original: The weight function w is deﬁned as counting number of feasible integer points and the distance function d is deﬁned as the 1  norm distance between the gravity center of two polyhedrons.
Revised: The weight function w is deﬁned as counting number of feasible integer points and the distance function d is deﬁned as the 1  norm distance between the gravity center of two polyhedrons.

Original: For computational efﬁciency, we ignore the con- straintsandonlyconsidervariableboundssuchthateverypolyhedronisabox.
Revised: For computational efﬁciency, we ignore the con- straintsandonlyconsidervariableboundssuchthateverypolyhedronisabox.

Original: Asimpleillustration ofcharacterizationisgiveninFigure2.
Revised: Asimpleillustration ofcharacterizationisgiveninFigure2.

Original: Wecanseethat p ( b 1 ) =
Revised: Wecanseethat p ( b 1 ) =

Original: ✓ 12 16 , 4 16 ◆ = ✓ 3 4 , 1 4 ◆ p ( b 2 ) =
Revised: ✓ 12 16 , 4 16 ◆ = ✓ 3 4 , 1 4 ◆ p ( b 2 ) =

Original: ✓ 8 16 , 4 16 , 4 16 ◆ = ✓ 1 2 , 1 4 , 1 4 ◆
Revised: ✓ 8 16 , 4 16 , 4 16 ◆ = ✓ 1 2 , 1 4 , 1 4 ◆

Revised: W ( b 1 ,b 2 ) = " 32 32 52 72 52 32 #

Revised: A

Original: B
Revised: B

Original: C 1 A 2 B
Revised: C 1 A 2 B

Original: [Figure]
Revised: Figure2: Captionforimage

Original: We now
Revised: Toputeverythingtogether,welayoutthetrainingalgorithmin Algorithm2.

Original: [Figure]
Revised: [Figure]

Revised: We now present a comparative experiments against two competing machine learning approach and three SCIP’s branching rules to assess the value of our RL agent, as well as an ablation study to validatetheourchoiceofstaterepresentationandtrainingalgorithm.

Original:  where kNN ( M, ✓ ) is the k nearest neighbor of ⇡ ✓ in M .
Revised: , (m) characterization b 2 , (r) distribution and the cost matrix where kNN ( M, ✓ ) is the k nearest neighbor of ⇡ ✓ in M .

Original: In this deﬁnition, the novelty score encourages the policies with characterization far from the characterizations in policy memory.
Revised: In this deﬁnition, the novelty score encourages the policies with characterization far from the characterizations in policy memory.

Original: [ Rephrase this paragraph.
Revised: [ Rephrase this paragraph.

Original: Make it easier to understand.]
Revised: Make it easier to understand.]

Original: [Equation]
Revised: [Equation]

Original: We have introduced the characterization and its metric for general B&B algorithm.
Revised: We have introduced the characterization and its metric for general B&B algorithm.

Original: Focusing on MIP, a subproblem R i can be represented by a polytope, the feasible region of the LP relaxation.
Revised: Focusing on MIP, a subproblem R i can be represented by a polytope, the feasible region of the LP relaxation.

Original: We deﬁne the weight function w as counting the number of feasible integer points in the polytope and the distance function d as the 1  norm distance between the gravity center of two polytopes.
Revised: We deﬁne the weight function w as counting the number of feasible integer points in the polytope and the distance function d as the 1  norm distance between the gravity center of two polytopes.

Original: For computational efﬁciency, we ignore the constraints and only consider variable bounds such that every polytope is a box.
Revised: For computational efﬁciency, we ignore the constraints and only consider variable bounds such that every polytope is a box.

Original: A simple illustration of characterizations is given in Figure 2.
Revised: A simple illustration of characterizations is given in Figure 2.

Original: We can compute the distance D ( b 1 ,b 2 )
Revised: We can compute the distance D ( b 1 ,b 2 )

Original: =
Revised: =

Original: Another observation is the polyhedron may degenerate to lower dimension after branching, for example, the B 1 in b 1 .
Revised: Another observation is the polyhedron may degenerate to lower dimension after branching, for example, the B 1 in b 1 .

Original: Hence, we choose the counting weight function w in our work [ Not so sure what this sentence means.] .
Revised: Hence, we choose the counting weight function w in our work [ Not so sure what this sentence means.] .

Original: This ﬁnishes the deﬁnition of the novelty score.
Revised: This ﬁnishes the deﬁnition of the novelty score.

Original: To put everything together, we summarize the training in Algorithm 2.
Revised: To put everything together, we summarize the training in Algorithm 2.

Original: 5 E XPERIMENTS
Revised: 5 E XPERIMENTS

Original: We now present a comparative experiments against two competing machine learning approach and three SCIP’s branching rules to assess the value of our RL agent, as well as an ablation study to validate the our choice of state representation and training algorithm.
Revised: We now present a comparative experiments against two competing machine learning approach and three SCIP’s branching rules to assess the value of our RL agent, as well as an ablation study to validate the our choice of state representation and training algorithm.

Original: 5.1 S ETUP
Revised: 5.1 S ETUP

Original: Benchmarks : We consider three classes of instances, Set Covering, Maximum Independent Set and Capacitated facility location, that are not only challenging for state-of-the-art solvers, but also representative for problems encountered in practice.
Revised: Benchmarks : We consider three classes of instances, Set Covering, Maximum Independent Set and Capacitated facility location, that are not only challenging for state-of-the-art solvers, but also representative for problems encountered in practice.

Original: For each class, we set up a backbone based on which we randomly generate the dataset.
Revised: For each class, we set up a backbone based on which we randomly generate the dataset.

Original: [ Need to justify why backbone idea is good: in many real word problems, different problem instance may share a backbone.
Revised: [ Need to justify why backbone idea is good: in many real word problems, different problem instance may share a backbone.

Original: For instance .... .
Revised: For instance .... .

Original: Do you have more information about the backbone, and details about the generative process in appendix?]
Revised: Do you have more information about the backbone, and details about the generative process in appendix?]

Original: [Sun: I will add the generating process in Appendix] We generate set covering instances using 1000 columns.
Revised: [Sun: I will add the generating process in Appendix] We generate set covering instances using 1000 columns.

Original: We train on instances with 500 rows and we evaluate on instances with 500 rows (test), 1000 rows (medium transfer), 1500 rows (hard transfer).
Revised: We train on instances with 500 rows and we evaluate on instances with 500 rows (test), 1000 rows (medium transfer), 1500 rows (hard transfer).

Original: We generate maximum independent set on graphs with 400 nodes and we evaluate on graphs withnodes (test), 1000 nodes (medium transfer) and 1500 nodes (hard transfer).
Revised: We generate maximum independent set on graphs with 400 nodes and we evaluate on graphs withnodes (test), 1000 nodes (medium transfer) and 1500 nodes (hard transfer).

Original: We generate capacitated facility location with 100 facilities.
Revised: We generate capacitated facility location with 100 facilities.

Original: We train on instances with 40 customers (test) and we evaluate on instances with 200 customers (medium transfer) and 400 customers (hard transfer).
Revised: We train on instances with 40 customers (test) and we evaluate on instances with 200 customers (medium transfer) and 400 customers (hard transfer).

Original: More details are provided in the appendix.
Revised: More details are provided in the appendix.

Original: [Table]
Revised: [Table]

Original: Under review as a conference paper at ICLR 2021 4.4 N OVELTY S EARCH E VOLUTIONARY S TRATEGY Withmetric D betweenrepresentations, wecandeﬁnethenoveltyscorefollowingContietal.(2018).
Revised: Under review as a conference paper at ICLR 2021 4.4 N OVELTY S EARCH E VOLUTIONARY S TRATEGY Withmetric D betweenrepresentations, wecandeﬁnethenoveltyscorefollowingContietal.(2018).

Original: Given a policy memory M and an instance Q sampled from the problem distribution D , novelty score is computed as:
Revised: Given a policy memory M and an instance Q sampled from the problem distribution D , novelty score is computed as:

Original: N ( ✓ ,Q,M ) = 1 k
Revised: N ( ✓ ,Q,M ) = 1 k

Original: X ⇡ j 2 kNN ( M, ✓ ) D ( b ( ⇡ ✓ ,Q ) ,b ( ⇡ j ,Q )) (7) where kNN ( M, ✓ ) is the k nearest neighbor of ⇡ ✓ in M .
Revised: X ⇡ j 2 kNN ( M, ✓ ) D ( b ( ⇡ ✓ ,Q ) ,b ( ⇡ j ,Q )) (7) where kNN ( M, ✓ ) is the k nearest neighbor of ⇡ ✓ in M .

Original: In this deﬁnition, the novelty score encourages the policies with representation far from therepresentations in policy memory.
Revised: In this deﬁnition, the novelty score encourages the policies with representation far from the in policy memory.

Original:  ( b 1 ,b 2 ) =

Original:  1 / 2 1 /

Original: [Equation]

Original: [Equation]

Original:

Original: [Equation]
Revised: [Equation]

Original: [ cite and summarize algorithm 1 here in this section!!!
Revised: [ cite and summarize algorithm 1 here in this section!!!

Original: Consider wrap ﬁgure and make it half column.]
Revised: Consider wrap ﬁgure and make it half column.]

Original: Algorithm 2: Evolutionary Strategy with Novelty Score.
Revised: Algorithm 2: Evolutionary Strategy with Novelty Score.

Original: Input: Learning rate ↵ , Noise std  , number of workers n , Validation size N , Batch size M , Initial weight  , Weight decay rate  , Iterations T, Parameter ✓ 0 , Policy memory M K with capacity K , Instance distribution
Revised: Input: Learning rate ↵ , Noise std  , number of workers n , Validation size N , Batch size M , Initial weight  , Weight decay rate  , Iterations T, Parameter ✓ 0 , Policy memory M K with capacity K , Instance distribution

Original: D Output: Best parameter ✓ best 1 Sample valid instances Q 1 , ··· ,Q N ⇠ D 2 Set F best = 1 N
Revised: D Output: Best parameter ✓ best 1 Sample valid instances Q 1 , ··· ,Q N ⇠ D 2 Set F best = 1 N

Original: P
Revised: P

Original: Nj =1 f ( ✓ 0 ,Q j ) , ✓ best = ✓ 0 and push ✓ 0 into M K 3 for t=0 to T do
Revised: Nj =1 f ( ✓ 0 ,Q j ) , ✓ best = ✓ 0 and push ✓ 0 into M K 3 for t=0 to T do

Original: Sample instances P 1 , ··· ,P M ⇠ Dfor i=1 to N do
Revised: Sample instances P 1 , ··· ,P M ⇠ Dfor i=1 to N do

Original: Sample ✏ 1 , ··· , ✏ n ⇠ N (0 ,I )
Revised: Sample ✏ 1 , ··· , ✏ n ⇠ N (0 ,I )

Original: Compute F i = 1 m
Revised: Compute F i = 1 m

Original: P
Revised: P

Original: Mm =1 f ( ✓ t + ✏ i ,P m )
Revised: Mm =1 f ( ✓ t + ✏ i ,P m )

Original: Compute N i = 1 m
Revised: Compute N i = 1 m

Original: P
Revised: P

Original: Mm =1 N ( ✓ t + ✏ i ,P m ,M )
Revised: Mm =1 N ( ✓ t + ✏ i ,P m ,M )

Original: Send N i and F i from each worker to coordinatorend
Revised: Send N i and F i from each worker to coordinatorend

Original: Set ✓ t +1 =
Revised: Set ✓ t +1 =

Original: ✓ t + ↵ 1 n 
Revised: ✓ t + ↵ 1 n

Original: P ni =1  · N i ✏ i + (1   ) · F i ✏ i
Revised: P ni =1  · N i ✏ i + (1   ) · F i ✏ i

Original: Compute F ( t +1) =
Revised: Compute F ( t +1) =

Original: 1 N
Revised: 1 N

Original: P
Revised: P

Original: Nj =1 f ( ✓ t +1 ,Q j )
Revised: Nj =1 f ( ✓ t +1 ,Q j )

Original: if F ( t +1) >
Revised: if F ( t +1) >

Original: F best then
Revised: F best then

Original: Set F best = F ( t +1) , ✓ best = ✓ t +1 ,  =  ⇤ end
Revised: Set F best = F ( t +1) , ✓ best = ✓ t +1 ,  =  ⇤ end

Original: Push ✓ t +1 into M K 17 end 
Revised: Push ✓ t +1 into M K 17 end not include problem set S , which makes it a partial observable MDP (Astrom, 1965). After two iterations of Equation 5, the variable embedding X is passed to a two-hidden-layer neural network score function f S and the output is the ﬁnal score for each variable. Since the state reduction and message passing are both inspired by primal-dual iteration, we call it PD policy. A more detailed discussion and comparison with

Revised: GCN (Gasse et al., 2019) can be found at section A.2.2.

Original: 4. S ET R EPRESENTATION FOR P OLICY AND O PTIMAL T RANSPORT D ISTANCE
Revised: 4. S ET R EPRESENTATION FOR P OLICY AND O PTIMAL T RANSPORT D ISTANCE

Original: We train the RL agent using evolution strategy similar to NSR-ES (Conti et al., 2018) and we need to deﬁne the novelty score for B&B process.
Revised: We train the RL agent using evolution strategy similar to NSR-ES (Conti et al., 2018) and we need to deﬁne the novelty score for B&B process.

Original: In the general B&B algorithm, the solving process can be represented by a search tree, where each leaf is a solved subproblem.
Revised: In the general B&B algorithm, the solving process can be represented by a search tree, where each leaf is a solved subproblem.

Original: Given a branch policy π and an instance Q , we deﬁne our representation b ( π, Q ) =
Revised: Given a branch policy π and an instance Q , we deﬁne our representation b ( π, Q ) =

Original: { R 1 , · · · , R H } as the collection of leaf subproblems on the complete search tree .
Revised: { R 1 , · · · , R H } as the collection of leaf subproblems on the complete search tree .

Original: Focusing on MIP, a subproblem R i is a LP relaxation which can be represented by its feasible region, a polytope.
Revised: Focusing on MIP, a subproblem R i is a LP relaxation which can be represented by its feasible region, a polytope.

Original: For example, in Figure 1, b 1 , b 2 and b 3 are the set of polytopes produced by three different policies π 1 , π 2 and π 3 respectively.
Revised: For example, in Figure 1, b 1 , b 2 and b 3 are the set of polytopes produced by three different policies π 1 , π 2 and π 3 respectively.

Original: And b 1 = { A 1 , B 1 } is a set of two polytopes (leaf subproblems), b 2 = { A 2 , B 2 , C 2 } is a set of three polytopes, and b 3 = { A 3 , B 3 } is a set of two polytopes.
Revised: And b 1 = { A 1 , B 1 } is a set of two polytopes (leaf subproblems), b 2 = { A 2 , B 2 , C 2 } is a set of three polytopes, and b 3 = { A 3 , B 3 } is a set of two polytopes.

Original: For computational efﬁciency, we ignore the constraints and only consider variable bounds such that every polytope is a box.
Revised: For computational efﬁciency, we ignore the constraints and only consider variable bounds such that every polytope is a box.

Original: For each polytope R i (leaf subproblem), we deﬁne the weight function w ( · ) and distance function d ( · , · ) between two polytopes R i and R j as
Revised: For each polytope R i (leaf subproblem), we deﬁne the weight function w ( · ) and distance function d ( · , · ) between two polytopes R i and R j as

Original: • w ( R i ) :
Revised: • w ( R i ) :

Original: = # { x ∈ R i : x is a feasible solution for Q } .
Revised: = # { x ∈ R i : x is a feasible solution for Q } .

Original: • d ( R i , R j ) :
Revised: • d ( R i , R j ) :

Original: = (cid:107) g
Revised: = (cid:107) g

Original: i − g j (cid:107) 1 , where g i and g j are the center of mass for R i and R j respectively.
Revised: i − g j (cid:107) 1 , where g i and g j are the center of mass for R i and R j respectively.

Original: For example, in Figure 1, we have w ( A 1 ) =
Revised: For example, in Figure 1, we have w ( A 1 ) =

Original: 12 , d
Revised: 12 , d

Original: ( A 1 , A 2 ) = 32 .
Revised: ( A 1 , A 2 ) = 32 .

Original: Then we can map the representation b
Revised: Then we can map the representation b

Original: = { R 1 , · · · , R H } to a simplex p ( b ) ∈ ∆ H − 1 by normalizing the weights p ( R j ) =
Revised: = { R 1 , · · · , R H } to a simplex p ( b ) ∈ ∆ H − 1 by normalizing the weights p ( R j ) =

Original: w ( R j ) / (cid:80)
Revised: w ( R j ) / (cid:80)

Original: Hi =1 w ( R i ) , and compute a cost matrix W ij =
Revised: Hi =1 w ( R i ) , and compute a cost matrix W ij =

Original: d ( R i , R j ) (See Figure 1 for examples).
Revised: d ( R i , R j ) (See Figure 1 for examples).

Original: Then, we can deﬁne the metric
Revised: Then, we can deﬁne the metric

Original: D between two representations as the Wasserstein distance (or optimal transport distance) (Villani, 2008;
Revised: D between two representations as the Wasserstein distance (or optimal transport distance) (Villani, 2008;

Original: Peyr ´ e et al., 2019):
Revised: Peyr´e et al., 2019):

Original: [Equation]
Revised: [Equation]

Original: For example, in Figure 1, the distance D ( b 1 , b 2 ) = 32 , D ( b 1 , b 3 ) =
Revised: For example, in Figure 1, the distance D ( b 1 , b 2 ) = 32 , D ( b 1 , b 3 ) =

Original: 34 meaning b 3 is closer to b 1 than b 2 .
Revised: 34 meaning b 3 is closer to b 1 than b 2 .

Original: Hence the corresponding policy π 3 is closer to π 1 than π 2 .
Revised: Hence the corresponding policy π 3 is closer to π 1 than π 2 .

Revised: Here, we provide a concrete method to measure the distance between two solving processes.

Revised: It is also provides a framework for general B&B algorithm.

Revised: We can choose weight function w and distance function d depending on the property of the solution space and compute the distance between two B&B solving processes.

Original: 4. N OVELTY S EARCH E VOLUTIONARY S TRATEGY
Revised: 4. N OVELTY S EARCH E VOLUTIONARY S TRATEGY

Original: Equipped with metric D between representations, we can deﬁne the novelty score following Conti et al.
Revised: Equipped with metric D between representations, we can deﬁne the novelty score following Conti et al.

Original: Given a policy memory M (a collection of older policies) and an instance Q sampled from the problem distribution D , novelty score is computed as:
Revised: Given a policy memory M (a collection of older policies) and an instance Q sampled from the problem distribution D , novelty score is computed as:

Original: [Equation]
Revised: [Equation]

Original: where k NN ( M, θ ) is the k nearest neighbor of π θ in M .
Revised: where k NN ( M, θ ) is the k nearest neighbor of π θ in M .

Original: Back to Algorithm 1, B&B algorithm recursively splits the feasible region and obtains a set of polytopes when ﬁnishing solving an instance.
Revised: Back to Algorithm 1, B&B algorithm recursively splits the feasible region and obtains a set of polytopes when ﬁnishing solving an instance.

Original: Notice that a polytope in the set representation is invariant with the generating order, i.e. branching x 1 then x 2 will give the same polytope with branching x 2 then x 1 .
Revised: Notice that a polytope in the set representation is invariant with the generating order, i.e. branching x 1 then x 2 will give the same polytope with branching x 2 then x 1 .

Original: As a result, our metric D and novelty score N is mostly determined by the pruning behavior during the solving process.
Revised: As a result, our metric D and novelty score N is mostly determined by the pruning behavior during the solving process.

Original: Put everything together, we summarize the training algorithm in section A.3.
Revised: Put everything together, we summarize the training algorithm in section A.3.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We now present comparative experiments against two competing machine learning approaches and three
Revised: We now present comparative experiments against two competing machine learning approaches and three

Original: SCIP’s branching rules to assess the value of our RL agent, as well as an ablation study to validate our choice of policy representation and training algorithm.
Revised: SCIP’s branching rules to assess the value of our RL agent, as well as an ablation study to validate our choice of policy representation and training algorithm.

Original: 5. S ETUP
Revised: 5. S ETUP

Original: Benchmarks : We consider three classes of instances, Set Covering (Balas & Ho, 1980), Maximum Independent Set (Albert & Barab´asi, 2002) and Capacitated facility location (Cornu´ejols et al., 1991), those are not only challenging for state-of-the-art solvers, but also representative for problems encountered in practice.
Revised: Benchmarks : We consider three classes of instances, Set Covering (Balas & Ho, 1980), Maximum Independent Set (Albert & Barab ´ asi, 2002) and Capacitated facility location (Cornu ´ ejols et al., 1991), those are not only challenging for state-of-the-art solvers, but also representative for problems encountered in practice.

Original: For each class, we set up a backbone based on which we randomly generate the dataset as many real-world problems also share the same backbone.
Revised: For each class, we set up a backbone based on which we randomly generate the dataset as many real-world problems also share the same backbone.

Original: For example, a logistics company frequently solves instances on very similar transportation networks with different customer demands.
Revised: For example, a logistics company frequently solves instances on very similar transportation networks with different customer demands.

Original: We generate set covering instances using 1000 columns.
Revised: We generate set covering instances using 1000 columns.

Original: We train on instances with 500 rows and evaluate on instances with 500 rows (test), 1000 rows (medium transfer), 1500 rows (hard transfer).
Revised: We train on instances with 500 rows and evaluate on instances with 500 rows (test), 1000 rows (medium transfer), 1500 rows (hard transfer).

Original: We train maximum independent set on graphs with 400 nodes and evaluate on graphs with 400 nodes (test), 1000 nodes (medium transfer), and 1500 nodes (hard transfer).
Revised: We train maximum independent set on graphs with 400 nodes and evaluate on graphs with 400 nodes (test), 1000 nodes (medium transfer), and 1500 nodes (hard transfer).

Original: We generate capacitated facility location with 100 facilities.
Revised: We generate capacitated facility location with 100 facilities.

Original: We train on instances withcustomers (test) and evaluate on instances with 40 customers (test), 200 customers (medium transfer), and 400 customers (hard transfer).
Revised: We train on instances withcustomers (test) and evaluate on instances with 40 customers (test), 200 customers (medium transfer), and 400 customers (hard transfer).

Original: More details are provided in the section A.
Revised: More details are provided in the section A.

Original: Settings : Throughout all experiments, we use SCIP 7.0.1 as the backend solver, with a time limit of 1 hour.
Revised: Settings : Throughout all experiments, we use SCIP 7.0.1 as the backend solver, with a time limit of 1 hour.

Original: For SCIP parameters, we have two settings: clean, a pure B&B algorithm with deep-ﬁrst-search as the nodeselection policy and all other functions are disabled so as to eliminate the interference between variable selection and other components of the solver; default , the default setting in SCIP where all components are kept to default.
Revised: For SCIP parameters, we have two settings: clean and default . The clean setting switches off other SCIP components, such as estimate node selection, cutting plane and primal heuristics. This way, the evaluation eliminates the interference from other components of the solver to variable selection policy. Under the clean setting, the solving nodes reﬂect the decision quality of variable selection policies only.

Original: It is a state-of-the-art solver (environment) that is able to solve challenging problems.
Revised: So, we compare the decision quality of different methods under the clean setting. The default setting of SCIP will turn on all components inside SCIP, which is tuned for solving real problems. So, We compare the ability to solve challenging problems of different methods under the default setting.

Original: Baselines : We compare against: Reliability Pseudocost Branch (RPB) (Achterberg & Berthold, ), the human-designed state-of-the-art branching rule, which computes strong branching in the beginning and gradually switches to simpler heuristics; Full Strong Branching (FSB), a full version of strong branching;
Revised: Baselines : We compare against: Reliability Pseudocost Branch (RPB) (Achterberg & Berthold, 2009), the human-designed state-of-the-art branching rule, which computes strong branching in the beginning and gradually switches to simpler heuristics; Full Strong Branching (FSB), a full version of strong branching;

Original: Vanilla Full Strong Branching (VFS), strong branching with branch LP information muted (Gamrath et al., 2020); and two recent machine learning policies support vector machine (SVM) rank approach (Khalil et al.,2016) and GCN approach (Gasse et al., 2019) 1 .
Revised: Vanilla Full Strong Branching (VFS), strong branching with branch LP information muted (Gamrath et al., 2020); and two recent machine learning policies support vector machine (SVM) rank approach (Khalil et al., and GCN approach (Gasse et al., 2019) 1 .

Original: We denote our method as RL, which is the primal-dual net trained by NS-ES.
Revised: We denote our method as RL, which is the primal-dual net trained by NS-ES.

Original: Metrics .
Revised: Metrics .

Original: To minimize the expected solving cost, metrics are selected as the average solving times ( T avg ) for all instances and average solving nodes ( N avg ) for instances solved by all methods.
Revised: To minimize the expected solving cost, metrics are selected as the average solving times ( T avg ) for all instances and average solving nodes ( N avg ) for instances solved by all methods.

Original: Since MIP instances could vary a lot in difﬁculty, we count the number of times each method leads the performance over the number of times each method solves the instance within timelimit (Wins) as a third robust metric.
Revised: Since MIP instances could vary a lot in difﬁculty, we count the number of times each method leads the performance over the number of times each method solves the instance within timelimit (Wins) as a third robust metric.

Original: Implementation .
Revised: Implementation .

Original: The detail of implementation is provided in section A.
Revised: The detail of implementation is provided in section A.

Original: 5. D ECISION Q UALITY
Revised: 5. D ECISION Q UALITY

Original: We evaluate the variable selection quality by solving 100 test instances under clean setting.
Revised: We evaluate the variable selection quality by solving 100 test instances under clean setting.

Original: Since we are comparing the decision quality, we say a method wins in this experiment if it results in the least number of solving nodes.
Revised: Since we are comparing the decision quality, we say a method wins in this experiment if it results in the least number of solving nodes.

Original: As FSB and RPB beneﬁt a lot from branching LP information (section 3), we do not include them when counting Wins.
Revised: As FSB and RPB beneﬁt a lot from branching LP information (section 3), we do not include them when counting Wins.

Original: Table.
Revised: Table.

Original: 1 shows our RL agent leads the win times on all datasets and the average solving nodes on set covering, and independent set are signiﬁcantly better than other methods.
Revised: 1 shows our RL agent leads the win times on all datasets and the average solving nodes on set covering, and independent set are signiﬁcantly better than other methods.

Original: [Table]
Revised: [Table]

Original: 5. G ENERALIZATION TO L ARGER I NSTANCES
Revised: 5. G ENERALIZATION TO L ARGER I NSTANCES

Original: It is very important for RL agents to transfer to larger unseen instances as training on large instances is very expensive in the real world.
Revised: It is very important for RL agents to transfer to larger unseen instances as training on large instances is very expensive in the real world.

Original: We investigate the generalization ability of our RL agent by solving 100 transfer instances under default setting.
Revised: We investigate the generalization ability of our RL agent by solving 100 transfer instances under default setting.

Original: To meet the needs in practice [Table] we say a method wins in this experiment if it results in the fastest solving time.
Revised: To meet the needs in practice, we say a method wins in this experiment if it results in the fastest solving time.

Original: As VFS is not able to solve any transfer instance in time limit, we do not list its results in Table.
Revised: As VFS is not able to solve any transfer instance in time limit, we do not list its results in Table.

Original: We can see, except for RPB and SVM having comparable performance on hard set covering and hard facility location, respectively, the RL agent leads the performance
Revised: We can see, except for RPB and SVM having comparable performance on hard set covering and hard facility location, respectively, the RL agent leads the performance.

Original: [Table] [Table]
Revised: In set covering (hard) and maximum independent set (hard), we do not compute the average number of nodes for full strong branching as it solves too limited instances.

Original: 5. I MPROVEMENT A NALYSIS
Revised: 5. I MPROVEMENT A NALYSIS

Original: Having seen the improvements brought by RL [Table] we would like to ask what kind of decisions our agent learns.
Revised: Having seen the improvements brought by RL, we would like to ask what kind of decisions our agent learns.

Original: We answer this question in two aspects: ﬁnding lower primal bound c ∗ and obtaining higher dual value ˆ c that allows pruning in line 5 Algorithm 1.
Revised: We answer this question in two aspects: ﬁnding lower primal bound c ∗ and obtaining higher dual value ˆ c that allows pruning in line 5 Algorithm 1.

Original: We compare our RL agent with GCN, SVM, VFC on 100 maximum independent set test instances under clean setting. 
Revised: We compare our RL agent with GCN, SVM, VFC on 100 maximum independent set test instances under clean setting. [Table] [Figure]

Revised: bounds versus the depth in search tree (number of branch constraints) they are found

Original: We ﬁrst examine primal bound c ∗ .
Revised: We ﬁrst examine primal bound c ∗ .

Original: Figure 2 plots the feasible solutions found during the solving process.
Revised: Figure 2 plots the feasible solutions found during the solving process.

Original: A point ( n, y ) means we ﬁnd a feasible solution c ∗ =
Revised: A point ( n, y ) means we ﬁnd a feasible solution c ∗ =

Original: y in a subproblem containing n branch constraints.
Revised: y in a subproblem containing n branch constraints.

Original: Figure 2 shows that our RL agent is able to detect small c ∗ at the early stage.
Revised: Figure 2 shows that our RL agent is able to detect small c ∗ at the early stage.

Original: Hence, it can prune more subproblems and solve the MIP faster.
Revised: Hence, it can prune more subproblems and solve the MIP faster.

Revised: On the contrary, VFS fails to detect feasible solutions efﬁciently.

Original: Traditionally, strong branching or other human-designed heuristics[Figure] they are found are mainly on the purpose of obtaining higher ˆ c .
Revised: One reason is, traditionally, strong branching or other human-designed heuristics are mainly on the purpose of obtaining higher ˆ c .

Original: Our result suggests a new possibility for researchers to  variable selection method good at detecting feasible solutions.
Revised: Our result suggests a new possibility for researchers to ﬁnd variable selection method good at detecting feasible solutions.

Original: Then, we check local dual value ˆ c .
Revised: Then, we check local dual value ˆ c .

Original: To eliminate the inﬂuence in primal bound c ∗ changing, we initialize c ∗ =
Revised: To eliminate the inﬂuence in primal bound c ∗ changing, we initialize c ∗ =

Original: c opt with the optimal value like Khalil et al.
Revised: c opt with the optimal value like Khalil et al.

Original: We plot the curve of average width versus the depth in Figure 3.
Revised: We plot the curve of average width versus the depth in Figure 3.

Original: The area under the curve equals the average number of solving nodes, and we report it in the legend.
Revised: The area under the curve equals the average number of solving nodes, and we report it in the legend.

Original: Also, when c ∗ is ﬁxed, a smaller width indicates a faster increase of ˆ c .
Revised: Also, as c ∗ is ﬁxed, the width versus depth plot characterizes how many branches are needed to increase the local dual value ˆ c to c ∗ so as to close a subproblem. A smaller width indicates the variable selection policy closes the gap faster. VFS performs better under this setting than in Figure 2 while it is still beat by learning based methods.

Original: Figure 3 shows that although our RL agent has the worst width in the beginning, it has the lowest peak and leads the overall performance.
Revised: Figure 3 shows that although our RL agent has the worst width in the beginning, it has the lowest peak and leads the overall performance.

Original: This means our RL agent successfully employs a non-myopic policy to maximize ˆ c in the long term.
Revised: This means our RL agent successfully employs a non-myopic policy to maximize ˆ c in the long term.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: of 4 RL agents 5. A BLATION S TUDY
Revised: 5. A BLATION S TUDY

Original: We present an ablation study of our method on maximum independent set problem by comparing four types of RL agents: (1) PD policy + ES; (2) PD policy + NS-ES; (3) GCN + ES; (4) GCN + NS-ES.
Revised: We present an ablation study of our method on maximum independent set problem by comparing four types of RL agents: (1) PD policy + ES; (2) PD policy + NS-ES; (3) GCN + ES; (4) GCN + NS-ES.

Original: We sample
Revised: We sample

Original: V = 200 instances as our validation set in and plot the average number of solving nodes under clean setting on the validation set during the training process for ﬁve random seeds.
Revised: V = 200 instances as our validation set in and plot the average number of solving nodes under clean setting on the validation set during the training process for ﬁve random seeds.

Original: All agents are initialized by imitation learning.
Revised: All agents are initialized by imitation learning.

Original: The results are plotted in Figure 4.
Revised: The results are plotted in Figure 4.

Original: All curves obtain higher rewards shows that RL improves the variable selection policy.
Revised: All curves obtain higher rewards shows that RL improves the variable selection policy.

Original: (1) and (2) having larger rewards than (3) and (4) shows that PD policy can obtain more improvement than GCN.
Revised: (1) and (2) having larger rewards than (3) and (4) shows that PD policy can obtain more improvement than GCN.

Original: Also, (2) and (4) having larger rewards than (1) and (3) shows that novelty search helps to ﬁnd better policies.
Revised: Also, (2) and (4) having larger rewards than (1) and (3) shows that novelty search helps to ﬁnd better policies.

Original: The results suggest that RL improves learning to branch and both PD policy, NS-ES are indispensable in the success of RL agent.
Revised: The results suggest that RL improves learning to branch and both PD policy, NS-ES are indispensable in the success of RL agent.

Original:  C ONCLUSION
Revised:  D ISCUSSION

Revised: In this work, we point out the overestimation of the decision quality of strong branching.

Revised: The evidence in

Revised: Table 1 shows VFS performs poor on synthetic dataset under clean setting.

Revised: An interesting phenomenon is that GCN can easily beat VFS after imitation learning (or our PD policy can obtain similar result).

Revised: One possible explanation is that the primal-dual message passing structure naturally learns the good decisions and ignores the noise brought by strong branching.

Revised: Another possible reason is the biased sampling.

Revised: To keep the diversity of the samples, Gasse et al.

Revised: (2019) employs a mixed policy of RPB and VFS to sample the training data.

Revised: VFS probably performs good on most of the states while has poor decision quality when trapped in some regions.

Revised: As a result, VFS has poor overall performance.

Revised: Fortunately, using the mixing policy as the behavior policy helps to escape from these regions hence, the collected data have good decision quality.

Revised: More studies are needed before we can give a conﬁdent answer for this question.

Revised:  C ONCLUSION

Original: We present an NS-ES framework to automatically learn the variable selection policy for MIP.
Revised: We present an NS-ES framework to automatically learn the variable selection policy for MIP.

Original: Central to our approach is the primal-dual policy network and the set representation of the B&B process.
Revised: Central to our approach is the primal-dual policy network and the set representation of the B&B process.

Original: We demonstrate our RL agent makes high-quality variable selection across different problems types and sizes.
Revised: We demonstrate our RL agent makes high-quality variable selection across different problems types and sizes.

Original: Our results suggest that with carefully designed policy networks and learning algorithms, reinforcement learning has the potential to advance algorithms for solving MIP.
Revised: Our results suggest that with carefully designed policy networks and learning algorithms, reinforcement learning has the potential to advance algorithms for solving MIP.

Original: A A PPENDIX A. B RANCH AND B OUND
Revised: A A PPENDIX A. B RANCH AND B OUND

Original: Here we gives a simple illustration of B&B algorithm in Figure 5.
Revised: Here we gives a simple illustration of B&B algorithm in Figure 5.

Original: Given the LP relaxation, the polytope represents the feasible region of the LP relaxation and the red arrow represents the objective vector.
Revised: Given the LP relaxation, the polytope represents the feasible region of the LP relaxation and the red arrow represents the objective vector.

Original: We ﬁrst solve the LP relaxation and obtain the solution ˆ x as the red point.
Revised: We ﬁrst solve the LP relaxation and obtain the solution ˆ x as the red point.

Original: Noticing it is not feasible for MIP, we branch the LP relaxation into two subproblems.
Revised: Noticing it is not feasible for MIP, we branch the LP relaxation into two subproblems.

Original: In (a) we select to split variable x 1 and in (b) we select to split variable x 2 .
Revised: In (a) we select to split variable x 1 and in (b) we select to split variable x 2 .

Original: The subproblems obtained after branching are displayed by the shaded purple regions.
Revised: The subproblems obtained after branching are displayed by the shaded purple regions.

Original: After ﬁnishing solve these two MIPs, we obtain the search trees t 1 and t 2 .
Revised: After ﬁnishing solve these two MIPs, we obtain the search trees t 1 and t 2 .

Original: We can see that a wise selection of variable x 2 can solve the problem faster.
Revised: We can see that a wise selection of variable x 2 can solve the problem faster.

Original: Branching Rules
Revised: Branching Rules

Original: [Table]
Revised: [Table]

Original: • divide into (disjoint) subproblems • improve local bounds Techniques • branching on variables • most infeasible • least infeasible • random branching • strong branching • pseudocost • reliability • VSIDS • hybrid reliability/inference • branching on constraints • SOS • SOS
Revised: • divide into (disjoint) subproblems • improve local bounds Techniques • branching on variables • most infeasible • least infeasible • random branching • strong branching • pseudocost • reliability • VSIDS • hybrid reliability/inference • branching on constraints • SOS • SOS

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: (a) split on x 1 and search tree t
Revised: (a) split on x 1 and search tree t

Original: Branching Rules
Revised: Branching Rules

Original: [Table]
Revised: [Table]

Original: • divide into (disjoint) subproblems • improve local bounds
Revised: • divide into (disjoint) subproblems • improve local bounds

Original: Techniques
Revised: Techniques

Original: • branching on variables • most infeasible • least infeasible • random branching • strong branching • pseudocost • reliability • VSIDS • hybrid reliability/inference • branching on constraints • SOS • SOS
Revised: • branching on variables • most infeasible • least infeasible • random branching • strong branching • pseudocost • reliability • VSIDS • hybrid reliability/inference • branching on constraints • SOS • SOS

Original: Gregor
Revised: Gregor

Original: [Table]
Revised: [Table]

Original: 47/ x 2 ≤x 2 ≥( x 1 , x 2 ) = (3,2)
Revised: 47/ x 2 ≤x 2 ≥( x 1 , x 2 ) = (3,2)

Original: Infeasible xx

Revised: Infeasible xx

Original: [Figure]
Revised: [Figure]

Original: A. I MPLEMENTATION A.2. H ARDWARE
Revised: A. I MPLEMENTATION A.2. H ARDWARE

Original: All the experiments were run at a Ubuntu 18.04 machine with Intel(R)
Revised: All the experiments were run at a Ubuntu 18.04 machine with Intel(R)

Original: Xeon(R)
Revised: Xeon(R)

Original: Silver 4116 CPU @ 2.10GHz, 256 GB Memory and Nvidia RTX 2080Ti graphic cards.
Revised: Silver 4116 CPU @ 2.10GHz, 256 GB Memory and Nvidia RTX 2080Ti graphic cards.

Original: A.2. PD P OLICY
Revised: A.2. PD P OLICY

Original: Comparison .
Revised: Comparison .

Original: PD policy is similar to the GCN in Gasse et al.
Revised: PD policy is similar to the GCN in Gasse et al.

Original: but has two major differences.
Revised: but has two major differences.

Original: First, we use a dynamic reduced graph where ﬁxed variables and trivial constraints are removed due to the variable bounds changing during the solving process while Gasse et al.
Revised: First, we use a dynamic reduced graph where ﬁxed variables and trivial constraints are removed due to the variable bounds changing during the solving process while Gasse et al.

Original: (2019) do not consider it.
Revised: (2019) do not consider it.

Original: The reduced graph can not only save computation, but also give a more accurate description of the solving state by ignoring the redundant information.
Revised: The reduced graph can not only save computation, but also give a more accurate description of the solving state by ignoring the redundant information.

Original: The ablation in Section 5.5 shows it is indispensable in the success of RL.
Revised: The ablation in Section 5.5 shows it is indispensable in the success of RL.

Original: Second, we use a simple matrix multiplication in our PD policy while Gasse et al.
Revised: Second, we use a simple matrix multiplication in our PD policy while Gasse et al.

Original: (2019) use a complicated edge embedding in GCN.
Revised: (2019) use a complicated edge embedding in GCN.

Original: In some sense, GCN can be seen as an overparameterized version of our method.
Revised: In some sense, GCN can be seen as an overparameterized version of our method.

Original: And our success reveals that message passing on the LP relaxation is the true helpful structure.
Revised: And our success reveals that message passing on the LP relaxation is the true helpful structure.

Original: detail .
Revised: detail .

Original: We implement our primal dual policy net using dgl (Wang et al., 2019), with hidden dimension h = 64 and ReLU activation.
Revised: We implement our primal dual policy net using dgl (Wang et al., 2019), with hidden dimension h = 64 and ReLU activation.

Original: The feature X for variable is a 17 dimension vector and feature Y for constraint is a 5 dimension vector.
Revised: The feature X for variable is a 17 dimension vector and feature Y for constraint is a 5 dimension vector.

Original: We list the detail of feature in Table.
Revised: We list the detail of feature in Table.

Original: [Table]
Revised: [Table]

Original: A.2. B ASELINE
Revised: A.2. B ASELINE

Original: FSB .
Revised: FSB .

Original: We use the implementation in SCIP Gamrath et al.
Revised: We use the implementation in SCIP Gamrath et al.

Original: VFS .
Revised: VFS .

Original: We use the implementation in SCIP Gamrath et al.
Revised: We use the implementation in SCIP Gamrath et al.

Original: RPB .
Revised: RPB .

Original: We use the implementation in SCIP Gamrath et al.
Revised: We use the implementation in SCIP Gamrath et al.

Original: GCN .
Revised: GCN .

Original: We tried to implement GCN in dgl (Wang et al., 2019), however, it is signiﬁcantly slower than the original implementation in Gasse et al.
Revised: We tried to implement GCN in dgl (Wang et al., 2019), however, it is signiﬁcantly slower than the original implementation in Gasse et al.

Original: Hence, we still use the implementation in Gasse et al.
Revised: Hence, we still use the implementation in Gasse et al.

Original: SVM .
Revised: SVM .

Original: We use the implementation in Gasse et al.
Revised: We use the implementation in Gasse et al.

Original: A. T RAINING
Revised: A. T RAINING

Original: We have two settings clean , default .
Revised: We have two settings clean , default .

Original: In experiments, we always train and test under the same setting.
Revised: In experiments, we always train and test under the same setting.

Original: Imitation Learning .
Revised: Imitation Learning .

Original: We initialize our PD policy using imitation learning similar to Gasse et al.
Revised: We initialize our PD policy using imitation learning similar to Gasse et al.

Original: The difference is we only use 10000 training samples, 2000 validation samples and 10 training epochs as a warm start.
Revised: The difference is we only use 10000 training samples, 2000 validation samples and 10 training epochs as a warm start.

Original: In our setting, a policy from scratch can hardly solve an instance in a reasonable time, hence a warm start is necessary.
Revised: In our setting, a policy from scratch can hardly solve an instance in a reasonable time, hence a warm start is necessary.

Original: Novelty Search Evolution Strategy .
Revised: Novelty Search Evolution Strategy .

Original: We improve our RL agent using Algorithm 2.
Revised: We improve our RL agent using Algorithm 2.

Original: The parameters are set as α = 1 e − 4 , σ = 1 e − 2 , n = 40 , V = 200 , w = 0 .
Revised: The parameters are set as α = 1 e − 4 , σ = 1 e − 2 , n = 40 , V = 200 , w = 0 .

Original: 25 , β =
Revised: 25 , β =

Original: [Table]
Revised: [Table]

Original: Input: Learning rate α , Noise std σ , number of workers n , Validation size V , Batch size M , Initial weight λ , Weight decay rate β , Iterations T, Parameter θ 0 , Policy memory M , Instance distribution D , Neighborhood size k .
Revised: Input: Learning rate α , Noise std σ , number of workers n , Validation size V , Batch size M , Initial weight λ , Weight decay rate β , Iterations T, Parameter θ 0 , Policy memory M , Instance distribution D , Neighborhood size k .

Original: Output: Best parameter θ best 1 Sample validation instances Q 1 , · · · , Q V ∼ D 2 Set R best = 1 V (cid:80)
Revised: Output: Best parameter θ best 1 Sample validation instances Q 1 , · · · , Q V ∼ D 2 Set R best = 1 V (cid:80)

Original: Vj
Revised: Vj

Original: =1 R ( θ 0 , Q j ) , θ best = θ3 for t=0 to T do
Revised: =1 R ( θ 0 , Q j ) , θ best = θ3 for t=0 to T do

Original: Sample instances P 1 , · · · , P M ∼ D
Revised: Sample instances P 1 , · · · , P M ∼ D

Original: Sample (cid:15) 1 , · · · , (cid:15) n ∼ N (0 , I ) and compute θ it =
Revised: Sample (cid:15) 1 , · · · , (cid:15) n ∼ N (0 , I ) and compute θ it =

Original: θ t + σ(cid:15)
Revised: θ t + σ(cid:15)

Original: i
Revised: i

Original: Set M = { θ 1 t , · · · , θ nt } for i=1 to n do
Revised: Set M = { θ 1 t , · · · , θ nt } for i=1 to n do

Original: Compute R i =
Revised: Compute R i =

Original: 1 m (cid:80)
Revised: 1 m (cid:80)

Original: Mm =1 R ( θ it , P m )
Revised: Mm =1 R ( θ it , P m )

Original: Compute N i = 1 m (cid:80) Mm =1 N ( θ it , P m , M ) end
Revised: Compute N i = 1 m (cid:80) Mm =1 N ( θ it , P m , M ) end

Original: Set θ t +

Revised: Set θ t +

Original: =
Revised: =

Original: θ t + α 1 nσ (cid:80) ni
Revised: θ t + α 1 nσ (cid:80) ni

Original: =1 λ · N i (cid:15) i + (1 − λ )
Revised: =1 λ · N i (cid:15) i + (1 − λ )

Original: · R i (cid:15) i
Revised: · R i (cid:15) i

Original: Compute R ( t +1) =
Revised: Compute R ( t +1) =

Original: 1 V (cid:80) Vj
Revised: 1 V (cid:80) Vj

Original: =1 R ( θ t +1 , Q j ) if R ( t +1) >
Revised: =1 R ( θ t +1 , Q j ) if R ( t +1) >

Original: R best then
Revised: R best then

Original: Set R best =
Revised: Set R best =

Original: R ( t +1) , θ best = θ t +1 , λ =
Revised: R ( t +1) , θ best = θ t +1 , λ =

Original: β ∗ λ end 16 end
Revised: β ∗ λ end 16 end

Original: A.
Revised: A.

Original: D ATA S ET
Revised: D ATA S ET

Original: Set Covering .
Revised: Set Covering .

Original: We generate a weighted set covering problem following Balas & Ho (1980).
Revised: We generate a weighted set covering problem following Balas & Ho (1980).

Original: The problem is formulated as the following ILP.
Revised: The problem is formulated as the following ILP.

Original: [Equation]
Revised: [Equation]

Original: subject to (cid:88)
Revised: subject to (cid:88)

Original: [Equation]
Revised: [Equation]

Original: where U is the universe of elements, S is the universe of the sets, w is a weight vector.
Revised: where U is the universe of elements, S is the universe of the sets, w is a weight vector.

Original: For any e ∈ U and
Revised: For any e ∈ U and

Original: S ∈ S , e ∈ S with probability 0.05.
Revised: S ∈ S , e ∈ S with probability 0.05.

Original: And we guarantee that for any e , it is contained by at least two sets in
Revised: And we guarantee that for any e , it is contained by at least two sets in

Original: S .
Revised: S .

Original: Each w S is uniformly sampled from integer from 1 to 100 .
Revised: Each w S is uniformly sampled from integer from 1 to 100 .

Original: We ﬁrst generate a set covering problem with U 

Revised: We ﬁrst generate a set covering problem with U

Original: = { e 1 , · · · , e 400 } and S 

Revised: = { e 1 , · · · , e 400 } and S

Original: = { S 1 , · · · , S 1000 } and set it as our backbone.
Revised: = { S 1 , · · · , S 1000 } and set it as our backbone.

Original: Then, every time we want to generate a new problem with m elements, we let U =
Revised: Then, every time we want to generate a new problem with m elements, we let U =

Original: U 0 ∪ { e 401 , e 402 , · · · , e m } add new e i into S ∈ S following the pipeline mentioned above.
Revised: U 0 ∪ { e 401 , e 402 , · · · , e m } add new e i into S ∈ S following the pipeline mentioned above.

Original: Maximum Independent Set .
Revised: Maximum Independent Set .

Original: We generate maximum independent set problem using Barabasi-Albert (Albert & Barab´asi, 2002) graphs.
Revised: We generate maximum independent set problem using Barabasi-Albert (Albert & Barab´asi, 2002) graphs.

Original: The problem is formulated as the following ILP. max (cid:88) v ∈ V x v subject to x u + x v ≤ 1 ,
Revised: The problem is formulated as the following ILP. max (cid:88) v ∈ V x v subject to x u + x v ≤ 1 ,

Original: ∀ e uv ∈ E x v ∈ { 0 , 1 } , ∀ v ∈ V where V is the set of vertices and E is the set of edges.
Revised: ∀ e uv ∈ E x v ∈ { 0 , 1 } , ∀ v ∈ V where V is the set of vertices and E is the set of edges.

Original: We generate the BA graph using a preferential attachment with afﬁnity coefﬁcient 4.
Revised: We generate the BA graph using a preferential attachment with afﬁnity coefﬁcient 4.

Original: We ﬁrst generate a BA graph G 0 with 350 nodes.
Revised: We ﬁrst generate a BA graph G 0 with 350 nodes.

Original: Then, every time we want to generate a new problem with n variables, we expand G 0 using preferential attachment.
Revised: Then, every time we want to generate a new problem with n variables, we expand G 0 using preferential attachment.

Original: Capacitated Facility Location .
Revised: Capacitated Facility Location .

Original: We generate the capacitated facility location problem following Cornu´ejols et al.
Revised: We generate the capacitated facility location problem following Cornu´ejols et al.

Original: The problem with m customers and n facilities is formulated as the following MIP.
Revised: The problem with m customers and n facilities is formulated as the following MIP.

Original: minn (cid:88) i =
Revised: min

Original: [Equation]
Revised: [Equation]

Original:  =c
Revised: i =

Revised: f i x i subject to n (cid:88) i =

Original: [Equation]
Revised: [Equation]

Original: + n (cid:88) i

Original: =

Original: f i x i subject to n (cid:88) i = y ij = 1 , ∀ j = 1 , · · · , mm (cid:88) j = d
Revised: m

Original: [Equation]
Revised: [Equation]

Original: u
Revised: n

Original: [Equation]
Revised: [Equation]

Original: x i , ∀ i = 1 · · · , n y
Revised: n and j [Equation]

Original: [Equation]
Revised: [Equation]

Original: 0 , ∀ i = 1 , · · · , n and j = 1 , · · · , m
Revised: m

Original: [Equation]
Revised: [Equation]

Original: n where x i = 1 indicates facility i is open, and x i = 

Revised: n where x i = 1 indicates facility i is open, and x i =

Original: otherwise; f i is the ﬁxed cost if facility i is open; d j is the demand for customer j ; c ij is the transportation cost between facility j and customer i ; y ij is the fraction of the demand of customer j ﬁlled by facility i .
Revised: otherwise; f i is the ﬁxed cost if facility i is open; d j is the demand for customer j ; c ij is the transportation cost between facility j and customer i ; y ij is the fraction of the demand of customer j ﬁlled by facility i .

Original: Following Cornu ´ ejols et al.
Revised: Following Cornu ´ ejols et al.

Original: (1991), where we ﬁrst sample the location of facility and customers on a 2 dimension map.
Revised: (1991), where we ﬁrst sample the location of facility and customers on a 2 dimension map.

Original: Then c i j is determined by the Euclidean distance between facility i and customer j and other parameters are sampled from the distribtuion given in
Revised: Then c i j is determined by the Euclidean distance between facility i and customer j and other parameters are sampled from the distribtuion given in

Original: Cornu ´ ejols et al.
Revised: Cornu ´ ejols et al.

Original: We ﬁrst generate the location of 100 facilities and 40 customers as our backbone.
Revised: We ﬁrst generate the location of 100 facilities and 40 customers as our backbone.

Original: Then, every time we want to generate a new problem with m customers, we generate new m − 40 locations for customers and follow the pipeline mentioned above.
Revised: Then, every time we want to generate a new problem with m customers, we generate new m − 40 locations for customers and follow the pipeline mentioned above.

Original: A. D ISCUSSION

Original: An interesting phenomenon is that GCN can easily beat VFS after imitation learning (Or our PD policy can obtain similar result).

Original: One possible explanation is that the primal-dual message passing is a principle structure that naturally learns the good decisions and ignores the noise brought by strong branching.

Original: Another possible reason is the biased sampling.

Original: To keep the diversity of the sample, Gasse et al.

Original: (2019) employs a mixed policy of RPB and VFS to sample the training data.

Original: It is possible that VFS is a powerful expert on the state distribution determined by the mixed policy.

Original: More studies are needed before we can answer this question.

Original: Another point is our set representation is compatible with general B&B algorithm.

Original: Once the weight function w and distance function d are deﬁned, we can compute the distance between two B&B solving processes.

Original: A. E XPERIMENTS ON BENCHMARK FROM G ASSE ET AL
Revised: A. E XPERIMENTS ON BENCHMARK FROM G ASSE ET AL

Original: We are mostly interested in improving the variable selection policy on similar problems hence, we generate our benchmark based on a backbone.
Revised: We are mostly interested in improving the variable selection policy on similar problems hence, we generate our benchmark based on a backbone.

Original: The backbone allows the instances share some common structures such that there exists a good policy for the given distribution of problems.
Revised: The backbone allows the instances share some common structures such that there exists a good policy for the given distribution of problems.

Original: Our experiments show that NS-ES is able to learn a good policies on this purpose.
Revised: Our experiments show that NS-ES is able to learn a good policies on this purpose.

Original: However, it is also interesting to check the performance of our method on a more random distribution.
Revised: However, it is also interesting to check the performance of our method on a more random distribution.

Original: Here, we conduct experiments on benchmark from Gasse et al.
Revised: Here, we conduct experiments on benchmark from Gasse et al.

Original: We employ the same instance generator and SCIP setting as Gasse et al.
Revised: We employ the same instance generator and SCIP setting as Gasse et al.

Original: For each category, we evaluate the policy on 20 instances with 5 random seeds.
Revised: For each category, we evaluate the policy on 20 instances with 5 random seeds.

Original: We report the average solving time T avg and shifted geometric mean solving time T geo on all instances, average solving nodes N avg and shifted geometric mean solving nodes N geo on instances solved by all instances
Revised: We report the average solving time T avg and shifted geometric mean solving time T geo on all instances, average solving nodes N avg and shifted geometric mean solving nodes N geo on instances solved by all instances, and Wins for the times one methods leads

Original: [Table]
Revised: [Table]

Original: and Wins for the times one methods leads over the number of instance solving to optimal
Revised: over the number of instance solving to optimal. As VFS is too slow to solve challenge instances

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: we only report its performance on easy instances.

Original: We can see that, in table 4, the improvement from RL method is less than that in the main text.
Revised: We can see that, in Table 4, the improvement from RL method is less than that in the main text.

Original: Intuitively, the randomly generated instances have less shared structure and leave less room for RL to improve the policy.
Revised: Intuitively, the randomly generated instances have less shared structure and leave less room for RL to improve the policy.

Original: How can we improve branch policies for randomly generated problems is still a question needs more explorations in the future.
Revised: How can we improve branch policies for randomly generated problems is still a question needs more explorations in the future.