Original: L EARNING C UT S ELECTION FOR M IXED -I NTEGER L INEAR P ROGRAMMING VIA H IERARCHICAL S EQUENCE M ODEL
Revised: L EARNING C UT S ELECTION FOR M IXED -I NTEGER L INEAR P ROGRAMMING VIA H IERARCHICAL S EQUENCE M ODEL

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Cutting planes (cuts) are important for solving mixed-integer linear programs (MILPs), which formulate a wide range of important real-world applications.
Revised: Cutting planes (cuts) are important for solving mixed-integer linear programs (MILPs), which formulate a wide range of important real-world applications.

Original: Cut selection—which aims to select a proper subset of the candidate cuts to improve the efficiency of solving MILPs—heavily depends on (P1) which cuts should be preferred, and (P2) how many cuts should be selected.
Revised: Cut selection—which aims to select a proper subset of the candidate cuts to improve the efficiency of solving MILPs—heavily depends on (P1) which cuts should be preferred, and (P2) how many cuts should be selected.

Original: Although many modern
Revised: Although many modern

Original: MILP solvers tackle (P1)-(P2) by manually designed heuristics, machine learning offers a promising approach to learn more effective heuristics from MILPs collected from specific applications.
Revised: MILP solvers tackle (P1)-(P2) by manually designed heuristics, machine learning offers a promising approach to learn more effective heuristics from MILPs collected from specific applications.

Original: However, many existing learning-based methods focus on learning which cuts should be preferred, neglecting the importance of learning the number of cuts that should be selected.
Revised: However, many existing learning-based methods focus on learning which cuts should be preferred, neglecting the importance of learning the number of cuts that should be selected.

Original: Moreover, we observe from extensive empirical results that (P3) what order of selected cuts should be preferred has a significant impact on the efficiency of solving MILPs as well.
Revised: Moreover, we observe from extensive empirical results that (P3) what order of selected cuts should be preferred has a significant impact on the efficiency of solving MILPs as well.

Original: To address this challenge, we propose a novel h ierarchical s e quence m odel (HEM) to learn cut selection policies via reinforcement learning.
Revised: To address this challenge, we propose a novel h ierarchical s e quence m odel (HEM) to learn cut selection policies via reinforcement learning.

Original: Specifically, HEM consists of a two-level model: (1) a higher-level model to learn the number of cuts that should be selected, (2) and a lower-level model—that formulates the cut selection task as a sequence to sequence learning problem—to learn policies selecting an ordered subset with the size determined by the higher-level model.
Revised: Specifically, HEM consists of a two-level model: (1) a higher-level model to learn the number of cuts that should be selected, (2) and a lower-level model—that formulates the cut selection task as a sequence to sequence learning problem—to learn policies selecting an ordered subset with the size determined by the higher-level model.

Original: To the best of our knowledge, HEM is the first method that can tackle (P1)-(P3) in cut selection simultaneously from a data-driven perspective.
Revised: To the best of our knowledge, HEM is the first method that can tackle (P1)-(P3) in cut selection simultaneously from a data-driven perspective.

Original: Experiments show that HEM significantly improves the efficiency of solving MILPs compared to human-designed and learning-based baselines on both synthetic and large-scale real-world MILPs, including MIPLIB 2017.
Revised: Experiments show that HEM significantly improves the efficiency of solving MILPs compared to human-designed and learning-based baselines on both synthetic and large-scale real-world MILPs, including MIPLIB 2017.

Original: Moreover, experiments demonstrate that HEM well generalizes to MILPs that are significantly larger than  seen during training.
Revised: Moreover, experiments demonstrate that HEM well generalizes to MILPs that are significantly larger than those seen during training.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Mixed-integer linear programming (MILP) is a general optimization formulation for a wide range of important real-world applications, such as supply chain management (Paschos, 2014), production planning (J¨unger et al., 2009), scheduling (Chen, 2010), facility location (Farahani & Hekmatfar, 2009), bin packing (Nair et al., 2020), etc.
Revised: Mixed-integer linear programming (MILP) is a general optimization formulation for a wide range of important real-world applications, such as supply chain management (Paschos, 2014), production planning (J¨unger et al., 2009), scheduling (Chen, 2010), facility location (Farahani & Hekmatfar, 2009), bin packing (Nair et al., 2020), etc.

Original: A standard MILP takes the form of z ∗ ≜
Revised: A standard MILP takes the form of z ∗ ≜

Original: min x { c ⊤ x | Ax ≤ b , x ∈ R n , x j ∈ Z for all j ∈ I} , (1) where c ∈ R n , A ∈ R m × n , b ∈ R m , x j denotes the j -th entry of vector x , I ⊆ { 1 , . . . , n } denotes the set of indices of integer variables, and z ∗ denotes the optimal objective value of the problem in 1.
Revised: min x { c ⊤ x | Ax ≤ b , x ∈ R n , x j ∈ Z for all j ∈ I} , (1) where c ∈ R n , A ∈ R m × n , b ∈ R m , x j denotes the j -th entry of vector x , I ⊆ { 1 , . . . , n } denotes the set of indices of integer variables, and z ∗ denotes the optimal objective value of the problem in (1).

Original: However, MILPs can be extremely hard to solve as they are NP -hard problems (Bengio et al., 2021).
Revised: However, MILPs can be extremely hard to solve as they are NP -hard problems (Bengio et al., 2021).

Original: To solve MILPs, many modern MILP solvers (Gurobi, 2021; Bestuzheva et al., 2021;FICO
Revised: To solve MILPs, many modern MILP solvers (Gurobi, 2021; Bestuzheva et al., 2021;

Original:  Xpress, 2020) employ a branch-and-bound tree search algorithm (Land & Doig, 2010), in which a linear programming (LP) relaxation of a MILP (the problem in (1) or its subproblems) is solved at each node.
Revised: FICO Xpress, 2020) employ a branch-and-bound tree search algorithm (Land & Doig, 2010), in which a linear programming (LP) relaxation of a MILP (the problem in (1) or its subproblems) is solved at each node.

Original: To further enhance the  tree search algorithm, cutting planes (cuts) (Gomory, 1960) are introduced to tighten the LP relaxations (Achterberg, 2007; Bengio et al., 2021).
Revised: To further enhance the performance of the tree search algorithm, cutting planes (cuts) (Gomory, 1960) are introduced to tighten the LP relaxations (Achterberg, 2007; Bengio et al., 2021).

Original: Existing work on cuts falls into two categories: cut generation and cut selection (Turner et al., 2022).
Revised: Existing work on cuts falls into two categories: cut generation and cut selection (Turner et al., 2022).

Original: Cut generation aims to generate cuts, i.e., valid linear inequalities that tighten the LP relaxations (Achterberg, 2007).
Revised: Cut generation aims to generate cuts, i.e., valid linear inequalities that tighten the LP relaxations (Achterberg, 2007).

Original: However, adding all the generated cuts to the LP relaxations can pose a computational problem Wesselmann & Stuhl (2012).
Revised: However, adding all the generated cuts to the LP relaxations can pose a computational problem (Wesselmann & Stuhl, 2012).

Original: To further improve the efficiency of solving MILPs, cut selection is proposed to select a proper subset of the generated cuts (Wesselmann & Stuhl, 2012).
Revised: To further improve the efficiency of solving MILPs, cut selection is proposed to select a proper subset of the generated cuts (Wesselmann & Stuhl, 2012).

Original: In this paper, we focus on the cut selection problem, which has a significant impact on the overall solver performance (Achterberg, 2007; Tang et al., 2020; Paulus et al., 2022).
Revised: In this paper, we focus on the cut selection problem , which has a significant impact on the overall solver performance (Achterberg, 2007; Tang et al., 2020; Paulus et al., 2022).

Original: Cut selection heavily depends on (P1) which cuts should be preferred, and (P2) how many cuts should be selected (Achterberg, 2007; Dey & Molinaro, 2018b).
Revised: Cut selection heavily depends on (P1) which cuts should be preferred, and (P2) how many cuts should be selected (Achterberg, 2007; Dey & Molinaro, 2018b).

Original: Many modern MILP solvers (Gurobi, 2021; Bestuzheva et al., 2021; FICO Xpress, 2020) tackle (P1)-(P2) by hard-coded heuristics designed by experts.
Revised: Many modern MILP solvers (Gurobi, 2021; Bestuzheva et al., 2021; FICO Xpress, 2020) tackle (P1)-(P2) by hard-coded heuristics designed by experts.

Original: However, hard-coded heuristics do not take into account underlying patterns among MILPs collected from certain types of real-world applications, e.g., day-to-day production planning, bin packing, and vehicle routing problems (Pochet & Wolsey, 2006; Laporte, 2009;
Revised: However, hard-coded heuristics do not take into account underlying patterns among MILPs collected from certain types of real-world applications, e.g., day-to-day production planning, bin packing, and vehicle routing problems (Pochet & Wolsey, 2006; Laporte, 2009;

Original: Nair et al., 2020).
Revised: Nair et al., 2020).

Original: To further improve the efficiency of MILP solvers, recent methods (Tang et al., 2020; Paulus et al., 2022; Huang et al., 2022) propose to learn cut selection policies via machine learning, which offers promising approaches to learn more effective heuristics by capturing underlying patterns among MILPs from specific applications (Lodi & Zarpellon, 2017; Nair et al., 2020).
Revised: To further improve the efficiency of MILP solvers, recent methods (Tang et al., 2020; Paulus et al., 2022; Huang et al., 2022) propose to learn cut selection policies via machine learning, especially reinforcement learning. They offer promising approaches to learn more effective heuristics by capturing underlying patterns among MILPs from specific applications (Bengio et al., 2021).

Original: However, many existing learning-based methods (Tang et al., 2020; Paulus et al., 2022; Huang et al., 2022)—which learn a scoring function to measure cut quality and select a fixed ratio/number of cuts with high scores—suffer from two weaknesses.
Revised: However, many existing learning-based methods (Tang et al., 2020; Paulus et al., 2022; Huang et al., 2022)—which learn a scoring function to measure cut quality and select a fixed ratio/number of cuts with high scores—suffer from two limitations.

Original: First, they learn which cuts should be preferred by learning a scoring function, neglecting the importance of learning the number of cuts that should be selected (Dey & Molinaro, 2018b).
Revised: First, they learn which cuts should be preferred by learning a scoring function, neglecting the importance of learning the number of cuts that should be selected (Dey & Molinaro, 2018b).

Original: Moreover, we observe from extensive empirical results that (P3) what order of selected cuts should be preferred significantly impact the efficiency of solving MILPs as well (see Section 3).
Revised: Moreover, we observe from extensive empirical results that (P3) what order of selected cuts should be preferred significantly impacts the efficiency of solving MILPs as well (see Section 3).

Original: Second, they do not take into account the interaction among cuts when learning which cuts should be preferred, as they score each cut independently .
Revised: Second, they do not take into account the interaction among cuts when learning which cuts should be preferred, as they score each cut independently .

Original: As a result, they struggle to select cuts that complement each other nicely, which could severely hinder the quality of selected cuts (Dey & Molinaro, 2018b).
Revised: As a result, they struggle to select cuts that complement each other nicely, which could severely hinder the efficiency of solving MILPs (Dey & Molinaro, 2018b).

Original: Indeed, we empirically show that they tend to select many similar cuts with high scores (see  Section 5).
Revised: Indeed, we empirically show that they tend to select many similar cuts with high scores (see Experiment 4 in Section 5).

Original: To address the aforementioned challenges, we propose a novel h ierarchical s e quence m odel (HEM) to learn cut selection policies via reinforcement learning.
Revised: To address the aforementioned challenges, we propose a novel h ierarchical s e quence m odel (HEM) to learn cut selection policies via reinforcement learning.

Original: To  tackle (P1)-(P3) in cut selection simultaneously, HEM leverages a two-level model: (1) a higher-level model to learn the number of cuts that should be selected, (2) and a lower-level model to learn policies selecting an ordered subset with the size determined by the higher-level model.
Revised: To the best of our knowledge, HEM is the first learning-based method that can tackle (P1)-(P3) simultaneously by proposing a two-level model. Specifically, HEM is comprised of (1) a higher-level model to learn the number of cuts that should be selected, (2) and a lower-level model to learn policies selecting an ordered subset with the size determined by the higher-level model.

Original: Specifically, the lower-level model formulates the cut selection task as a sequence to sequence learning problem, which has two main advantages.
Revised: The lower-level model formulates the cut selection task as a sequence to sequence learning problem, leading to two major advantages.

Original: First, the sequence model naturally captures the  order of selected cuts, as it selects cuts sequentially.
Revised: First, the sequence model is popular in capturing the underlying order information (Vinyals et al., 2016), which is critical for tackling (P3) .

Original: Second, the sequence model captures the interaction among cuts, as it models the joint conditional probability of the selected cuts given an input sequence of the candidate cuts.
Revised: Second, the sequence model can well capture the interaction among cuts, as it models the joint conditional probability of the selected cuts given an input sequence of the candidate cuts.

Original: Experiments show that HEM significantly outperforms human-designed and learning-based baselines in terms ofthe solving efficiency on threeclassical synthetic MILP problems and seven challenging MILP problems, including some benchmarks from MIPLIB 2017 (Gleixner et al., 2021) and  real-world production planning problems.
Revised: As a result, experiments show that HEM significantly outperforms human-designed and learning-based baselines in terms of solving efficiency on three synthetic MILP problems and seven challenging MILP problems. The challenging MILP problems include some benchmarks from MIPLIB 2017 (Gleixner et al., 2021) and large-scale real-world production planning problems.

Original: Our results highlight the  ability to enhance modern MILP solvers with our proposed HEM in real-world applications.
Revised: Our results demonstrate the strong ability to enhance modern MILP solvers with our proposed HEM in real-world applications.

Original: Moreover, experiments demonstrate that HEM can well generalize to MILPs that are significantly larger than  seen during training.
Revised: Moreover, experiments demonstrate that HEM can well generalize to MILPs that are significantly larger than those seen during training.

Original: We summarize our major contributions as follows.
Revised: We summarize our major contributions as follows.

Original: (1) We observe from extensive empirical results that the order of selected cuts has a significant impact on the efficiency of solving MILPs (see Section 3).
Revised: (1) We observe from extensive empirical results that the order of selected cuts has a significant impact on the efficiency of solving MILPs (see Section 3).

Original: (2) To the best of our knowledge, our proposed HEM is the first method that is able to tackle (P1)- (P3) in cut selection simultaneously from a data-driven perspective.
Revised: (2) To the best of our knowledge, our proposed HEM is the first method that is able to tackle (P1)-(P3) in cut selection simultaneously from a data-driven perspective.

Original: (3) We propose to formulate the cut selection task as a sequence to sequence learning problem, which not only captures the  order of selected cuts, but alsocan well capture the interaction among cuts to select cuts that complement each other nicely.
Revised: (3) We propose to formulate the cut selection task as a sequence to sequence learning problem, which not only can capture the underlying order information, but also well captures the interaction among cuts to select cuts that complement each other nicely.

Original: (4) Experiments demonstrate that HEM achieves significant improvements over competitive baselines on challenging MILP problems, including some benchmarks from MIPLIB 2017 and  real-world production planning problems.
Revised: (4) Experiments demonstrate that HEM achieves significant improvements over competitive baselines on challenging MILP problems, including some benchmarks from MIPLIB 2017 and large-scale real-world production planning problems.

Original:  B ACKGROUND
Revised:  B ACKGROUND

Original: Cutting planes.
Revised: Cutting planes.

Original: Given the MILP problem in (1), we drop all its integer constraints to obtain its linear programming (LP) relaxation , which takes the form of
Revised: Given the MILP problem in (1), we drop all its integer constraints to obtain its linear programming (LP) relaxation , which takes the form of

Original: [Equation]
Revised: [Equation]

Original: Since the problem in (2) expands the feasible set of the problem in (1), we have z ∗ LP ≤ z ∗ .
Revised: Since the problem in (2) expands the feasible set of the problem in (1), we have z ∗ LP ≤ z ∗ .

Original: We denote any lower bound found via a LP relaxation by a dual bound .
Revised: We denote any lower bound found via an LP relaxation by a dual bound .

Original: Given the LP relaxation in (2), cutting planes (cuts) are linear inequalities that are added to the original problem in (1) in the attempt to tighten the LP relaxation without removing any integer feasible solutions of the problem in (1).
Revised: Given the LP relaxation in (2), cutting planes (cuts) are linear inequalities that are added to the LP relaxation in the attempt to tighten it without removing any integer feasible solutions of the problem in (1).

Original: Cuts generated by MILP solvers are added in successive rounds.
Revised: Cuts generated by MILP solvers are added in successive rounds.

Original: Specifically, each round k involves (i) solving the current LP relaxation, (ii) generating a pool of candidate cuts C k , (iii) selecting a subset S k ⊆ C k , (iv) adding S k to the current LP relaxation to obtain the next LP relaxation , (v) and proceeding to the next LP relaxation.
Revised: Specifically, each round k involves (i) solving the current LP relaxation, (ii) generating a pool of candidate cuts C k , (iii) selecting a subset S k ⊆ C k , (iv) adding S k to the current LP relaxation to obtain the next LP relaxation, (v) and proceeding to the next round.

Original: Although adding all the generated cuts to the formulation would maximally strengthen the LP relaxation and improve the lower bound at each round, adding too many cuts could lead to large models, which can increase the computational burden and present numerical instabilities (Wesselmann & Stuhl, 2012; Paulus et al., 2022).
Revised: Adding all the generated cuts to the LP relaxation would maximally strengthen the LP relaxation and improve the lower bound at each round. However, adding too many cuts could lead to large models, which can increase the computational burden and present numerical instabilities (Wesselmann & Stuhl, 2012; Paulus et al., 2022).

Original: Therefore, selecting a proper subset of the candidate cuts, namely cut selection, is significant for improving the efficiency of solving MILPs (Dey & Molinaro, 2018b; Tang et al., 2020).
Revised: Therefore, cut selection is proposed to select a proper subset of the candidate cuts, which is significant for improving the efficiency of solving MILPs (Dey & Molinaro, 2018b; Tang et al., 2020).

Original: Branch-and-cut.
Revised: Branch-and-cut.

Original: In modern MILP solvers, cutting planes are often combined with the branch-andbound algorithm (Land & Doig, 2010), which is known as the branch-and-cut algorithm (Bengio et al., 2021).
Revised: In modern MILP solvers, cutting planes are often combined with the branch-andbound algorithm (Land & Doig, 2010), which is known as the branch-and-cut algorithm (Bengio et al., 2021).

Original: Branch-and-bound techniques perform implicit enumeration by building a search tree, in which every node represents a subproblem of the original problem in (1).
Revised: Branch-and-bound techniques perform implicit enumeration by building a search tree, in which every node represents a subproblem of the original problem in (1).

Original: The solving process begins by selecting a leaf node of the tree and solving its LP relaxation.
Revised: The solving process begins by selecting a leaf node of the tree and solving its LP relaxation.

Original: Let x ∗ be the optimal solution of the LP relaxation.
Revised: Let x ∗ be the optimal solution of the LP relaxation.

Original: If x ∗ violates the original integrality constraints, two subproblems (child nodes) of the leaf node are created by branching .
Revised: If x ∗ violates the original integrality constraints, two subproblems (child nodes) of the leaf node are created by branching .

Original: Specifically, the leaf node is added with constraints x i ≤ ⌊ x ∗ i ⌋ and x i ≥ ⌈ x ∗ i ⌉ , respectively, where x i denotes the i -th variable, x ∗ i denotes the i -th entry of vector x ∗ , and ⌊⌋ and ⌈⌉ denote the floor and ceil functions.
Revised: Specifically, the leaf node is added with constraints x i ≤ ⌊ x ∗ i ⌋ and x i ≥ ⌈ x ∗ i ⌉ , respectively, where x i denotes the i -th variable, x ∗ i denotes the i -th entry of vector x ∗ , and ⌊⌋ and ⌈⌉ denote the floor and ceil functions.

Original: In contrast, if x ∗ is a (mixed-)integer solution of (1), then we obtain an upper bound on the optimal  value of (1), which we denote by primal bound .
Revised: In contrast, if x ∗ is a (mixed-)integer solution of (1), then we obtain an upper bound on the optimal objective value of (1), which we denote by primal bound .

Original: In modern MILP solvers, the addition of cutting planes is alternated with the branching phase.
Revised: In modern MILP solvers, the addition of cutting planes is alternated with the branching phase.

Original: That is, cuts are added atnodes of the search tree  before branching to tighten their LP relaxations.
Revised: That is, cuts are added at search tree nodes before branching to tighten their LP relaxations.

Original: Since strengthening the relaxation before starting to branch is decisive to ensure an efficient tree search (Wesselmann & Stuhl, 2012; Bengio et al., 2021), we focus on only adding cuts at the root node, which follows (Huang et al., 2022; Paulus et al., 2022).
Revised: Since strengthening the relaxation before starting to branch is decisive to ensure an efficient tree search (Wesselmann & Stuhl, 2012; Bengio et al., 2021), we focus on only adding cuts at the root node, which follows Gasse et al. ; Paulus et al. (2022).

Original: Primal-dual gap integral.
Revised: Primal-dual gap integral.

Original: We keep track of two important bounds when running branch-and-cut, including the global primal and dual bound, which are the best upperbound and lower bound on the optimal  value of (1), respectively.
Revised: We keep track of two important bounds when running branch-and-cut, i.e., the global primal and dual bounds, which are the best upper and lower bounds on the optimal objective value of (1), respectively.

Original: We define the primal-dual gap integral (PD integral) by the area between the curve of the solver’s global primal bound and the curve of the solver’s global dual bound.
Revised: We define the primal-dual gap integral (PD integral) by the area between the curve of the solver’s global primal bound and the curve of the solver’s global dual bound.

Original: In the NeurIPS 2021 ML4CO competition (Bowly et al., 2021), the primal-dual gal integral isset as a primary evaluation metric.
Revised: The PD integral is a widely used metric for evaluating solver performance (Bowly et al., 2021; Cao et al., 2022).

Original: We provide additional details in Appendix C.1.
Revised: We provide more details in Appendix C.1.

Original:  M OTIVATING RESULTS
Revised:  M OTIVATING R ESULTS

Original: [Figure]
Revised: [Figure]

Original: RandomNV (see Section 3 for details), which always select the same subset of cuts for a given MILP and then vary the order of the selected cuts. The results in (a) and (b) show that adding cuts with different order leads to variable overall solver performance. We first empirically show that the order of selected cuts , i.e., the selected cuts are added to the problem in (1) in this order, significantly impact the efficiency of solving MILPs.
Revised: [Figure]

Original: We then empirically show that the ratio of selected cuts matters significantlywhen solving MILPs as well. Due to limited space, we present the latter results in
Revised: cuts cuts to LP in order MILPs ratio of selected cuts matters significantly

Original:  Appendix F.1.
Revised: MILPs (see Appendix G.1)

Original: Please refer to Appendix D.2 for details of the datasets used in this section.
Revised: Appendix D

Original: Order matters.
Revised: Order matters.

Original: Previous work (Bixby, 1992; Maros, 2002; Li et al., 2022) has shown that the order of constraints for a given linear program (LP) greatly impact its constructed initial basis, which is significant forthe efficiency of solving LPs.
Revised: Previous work (Bixby, 1992; Maros, 2002; Li et al., 2022) has shown that the order of constraints for a given linear program (LP) significantly impacts its constructed initial basis, which is important for solving the LP.

Original: Due to the fact that a cut is a linear constraint, adding cuts to the LP relaxation is equivalent to adding constraints to the LP relaxation.
Revised: As a cut is a linear constraint, adding cuts to the LP relaxations is equivalent to adding constraints to the LP relaxations.

Original: Therefore,we hypothesize that the order of added cuts should have a large impact on solving the LP relaxation, thereby being significant for solving MILPs.
Revised: Therefore, the order of added cuts could have a significant impact on solving the LP relaxations as well, thus being important for solving MILPs.

Original: (1) To evaluate whether the order of selected cuts affects the efficiency of solving MILPs, we first design a RandomAll cut selection rule, which randomly permutes all the candidate cuts and adds all the cuts to the formulation. RandomAll always selects all the cuts, but the order of selected cuts is very variable.
Revised: Indeed, our empirical results show that this is the case. (1) We design a RandomAll cut selection rule, which randomly permutes all the candidate cuts, and adds all the cuts to the LP relaxations in the random order.

Original: We evaluate RandomAll on five challenging datasets that are inspired by real-world applications.
Revised: We evaluate RandomAll on five challenging datasets, namely D1, D2, D3, D4, and D5.

Original: Wemeasure the solver performance with the average PD integral within a time limit of 300 seconds, and use  SCIP 8.0.0 (Bestuzheva et al., 2021) as the backend solver.
Revised: We use the SCIP 8.0.0 (Bestuzheva et al., 2021) as the backend solver, and evaluate the solver performance by the average PD integral within a time limit.

Revised: We evaluate RandomAll on each dataset over ten random seeds, and each bar in Figure 1a shows the mean and standard deviation (stdev) of its performance on each dataset.

Original: As shown in
Revised: As shown in

Original: Figure 1a, the performance ofthe solver with RandomAll on each dataset varies  with the order of selected cuts.
Revised: Figure 1a, the performance of RandomAll on each dataset varies widely with the order of selected cuts.

Original: (2) To furtherevaluate the impact of the order of selected cuts with different given ratios, we then design a RandomNV cut selection rule, which selects a given ratio of  cuts based on the Normalized Violation(NV) method (Huang et al., 2022) and  randomly permutes the selected cuts.
Revised: (2) We further design a RandomNV cut selection rule. RandomNV is different from RandomAll in that it selects a given ratio of the candidate cuts rather than all the cuts. RandomNV first scores each cut using the Normalized Violation (Huang et al., 2022) and selects a given ratio of cuts with high scores. It then randomly permutes the selected cuts.

Original: The NV method scores each cut via the extent to which it is violated by the optimal solution of the LP relaxation and selects a given ratio ofcuts with high scores. RandomNV with a given ratio always selects the same subset, but the order of selected cutsis variable. The results in Figure 1b show that RandomNV with a given ratio achieves variable solver performance on the same dataset.
Revised: Each bar in Figure 1b shows the mean and stdev of the performance of RandomNV with a given ratio on the same dataset. Figures 1a and 1b show that adding the same selected cuts in different order leads to variable solver performance, which demonstrates that the order of selected cuts is important for solving MILPs.

Original:  LEARNING CUT SELECTION VIA HIERARCHICAL SEQUENCE MODEL
Revised:  L EARNING C UT S ELECTION VIA H IERARCHICAL S EQUENCE M ODEL

Original: Although the optimal labels in the cut selection task are inaccessible,  one can assess the quality of selected subsets using a solver and provide some reward feedbacks.
Revised: In the cut selection task, the optimal subsets that should be selected are inaccessible, but one can assess the quality of selected subsets using a solver and provide the feedbacks to learning algorithms.

Original: Therefore, we leverage reinforcement learning (RL) to learn cut selection policies as Tang et al.
Revised: Therefore, we leverage reinforcement learning (RL) to learn cut selection policies.

Original: In this section, we provide a detailed description of our proposed RL framework for learning cut selection.
Revised: In this section, we provide a detailed description of our proposed RL framework for learning cut selection.

Original: First, we present our formulation of the cut selection as a Markov decision process (MDP) (Sutton & Barto, 2018).
Revised: First, we present our formulation of the cut selection as a Markov decision process (MDP) (Sutton & Barto, 2018).

Original: Then, we present a detailed description of our proposed h ierarchical s e quence m odel (HEM).
Revised: Then, we present a detailed description of our proposed h ierarchical s e quence m odel (HEM).

Original: Finally, we derive a hierarchical policy gradient for training HEM efficiently. 
Revised: Finally, we derive a hierarchical policy gradient for training HEM efficiently. [Equation]

Original: 𝑀𝐼𝐿𝑃𝑠𝑜𝑙𝑣𝑒𝑟

Original: [Figure]
Revised: [Figure]

Revised: 𝑠𝑡𝑎𝑡𝑒 𝑠

Original: Reinforcement Learning Formulation
Revised: Reinforcement Learning Formulation

Original: As shown in Figure 2, we regard a MILP solver as an environment and our proposed hierarchical sequence model as an agent. 
Revised: As shown in Figure 2, we formulate a MILP solver as the environment and our proposed HEM as the agent. We consider an MDP defined by the tuple ( S , A , r, f ) .

Original: Specifically, we specify the state space S , the action space A , the reward function r : S ×
Revised: Specifically, we specify the state space S , the action space A , the reward function r : S ×

Original: A → R , the transition function f , and the terminal state in the following.
Revised: A → R , the transition function f , and the terminal state in the following.

Original: (1) The state space S .
Revised: (1) The state space S .

Original: Since the current LP relaxation and the generated cuts contain the core information for cut selection, we define a state s by ( M LP , C , x ∗ LP ) , where M LP denotes the mathematical model of the current LP relaxation, C denotes the set of the candidate cuts, and x ∗ LP denotes the optimal solution of the LP relaxation.
Revised: Since the current LP relaxation and the generated cuts contain the core information for cut selection, we define a state s by ( M LP , C , x ∗ LP ) . Here M LP denotes the mathematical model of the current LP relaxation, C denotes the set of the candidate cuts, and x ∗ LP denotes the optimal solution of the LP relaxation.

Original: To encode the state information, we follow (Achterberg, 2007; Huang et al., 2022) to design thirteen features for each candidate cut based on the information of ( M LP , C , x ∗ LP ) .
Revised: To encode the state information, we follow Achterberg (2007); Huang et al. (2022) to design thirteen features for each candidate cut based on the information of ( M LP , C , x ∗ LP ) .

Original: That is, we  represent a state s by a sequence of thirteen-dimensional feature vectors.
Revised: That is, we actually represent a state s by a sequence of thirteen-dimensional feature vectors .

Original: We present details of the designed features in Appendix E.1.
Revised: We present details of the designed features in Appendix F.1.

Original: The action space A .
Revised: The action space A .

Original: Previous work (Achterberg, 2007;

Original: Turner et al., 2022) defines the action space as all the subsets of the candidate cuts C , which does not take into account the order of selected cuts.

Original: However, we empirically show that the order of selected cuts matters significantly when solving MILPs in Section 3.

Original: To capture the effect of the order, we define the action space by all the ordered subsets of the candidate cuts C .
Revised: To take into account the ratio and order of selected cuts, we define the action space by all the ordered subsets of the candidate cuts C .

Original: It can be challenging to explore the action space efficiently, as the cardinality of the action space can be extremely large due to its combinatorial structure.
Revised: It can be challenging to explore the action space efficiently, as the cardinality of the action space can be extremely large due to its combinatorial structure.

Original: The reward function r .
Revised: The reward function r .

Original: To evaluate the effect of the added cuts on solving MILPs, we design the reward function by (i) measures collected at the end of solving LP relaxations such as the dual bound improvement, (ii) and end-of-run statistics, such as the solving time and the primal-dual gap integral.
Revised: To evaluate the impact of the added cuts on solving MILPs, we design the reward function by (i) measures collected at the end of solving LP relaxations such as the dual bound improvement, (ii) or end-of-run statistics, such as the solving time and the primal-dual gap integral.

Original: For the first, the reward r ( s, a ) can be defined as the  dual bound improvement at each step.
Revised: For the first, the reward r ( s, a ) can be defined as the negative dual bound improvement at each step.

Original: For the second, the reward r ( s, a ) can be defined as zero except  the last  ( s T , a T ) in a trajectory, i.e.,we define r ( s T , a T )  by the  solving time or the  primal-dual gap integral.
Revised: For the second, the reward r ( s, a ) can be defined as zero except for the last step ( s T , a T ) in a trajectory, i.e., r ( s T , a T ) is defined by the negative solving time or the negative primal-dual gap integral.

Original: The transition function f .
Revised: The transition function f .

Original: The transition function maps the current state s and the action a to the next state s ′ , where s ′ represents the next LP relaxation generated by adding the selected cuts at the current LP relaxation.
Revised: The transition function maps the current state s and the action a to the next state s ′ , where s ′ represents the next LP relaxation generated by adding the selected cuts at the current LP relaxation.

Original: The terminal state.
Revised: The terminal state.

Original: There is no standard and unified criterion to  terminate the cutseparation, i.e., how many times the separation procedureshould be repeated (Paulus et al., 2022).
Revised: There is no standard and unified criterion to determine when to terminate the cut separation procedure (Paulus et al., 2022).

Original: In this paper, we set the cut separation rounds T as one following (Huang et al., 2022).
Revised: In this paper, we set the cut separation rounds T as one following Huang et al.

Original: Nevertheless, we emphasize that our proposed RL framework can be easily applied to the multiple rounds setting, i.e., T > 1 .
Revised: That is, we terminate the cut separation after one round. Nevertheless, our proposed RL framework can be easily applied to the multiple rounds setting, i.e., T > 1 .

Original: If T = 1 , then the decision problem is a one-step decision problem.
Revised: If T = 1 , then the decision problem is a one-step decision problem.

Original: More precisely, it is a contextual bandit (Sutton & Barto, 2018).
Revised: More precisely, it is a contextual bandit (Sutton & Barto, 2018).

Original: Hierarchical Sequence Model
Revised: Hierarchical Sequence Model

Original: Motivation.
Revised: Motivation.

Original: Based on the aforementioned formulation, we define the cut selection policyby π : S → P ( A ) , where P ( A ) denotes the probability distribution over the action space and π ( ·| s ) denotes the probability distribution over the action space given the state s .
Revised: Let π denote the cut selection policy π : S → P ( A ) , where P ( A ) denotes the probability distribution over the action space, and π ( ·| s ) denotes the probability distribution over the action space given the state s .

Original: We emphasize that learning such policies can tackle (P1)-(P3) in cut selection simultaneously.
Revised: We emphasize that learning such policies can tackle (P1)-(P3) in cut selection simultaneously.

Original: However, directly learning such policies can be challenging for the following reasons.
Revised: However, directly learning such policies is challenging for the following reasons.

Original: First, it is difficult to explore the action space efficiently, as the cardinality of the action space can be extremely large due to its combinatorial structure.
Revised: First, it is challenging to explore the action space efficiently, as the cardinality of the action space can be extremely large due to its combinatorial structure.

Original: Second, the length and max length of actions (i.e., ordered subsets) are variable across different MILPs.
Revised: Second, the length and max length of actions (i.e., ordered subsets) are variable across different MILPs.

Original: However, traditional reinforcement learning usually deal with problems whose actions have a fixed length.
Revised: However, traditional RL usually deals with problems whose actions have a fixed length.

Original: Instead of directly learning the aforementioned policy, many existing learning-based methods (Tang et al., 2020; Huang et al., 2022; Paulus et al., 2022) learn a scoring function to measure cut quality and select a fixed ratio/number of cuts with high scores.
Revised: Instead of directly learning the aforementioned policy, many existing learning-based methods (Tang et al., 2020; Huang et al., 2022; Paulus et al., 2022) learn a scoring function that outputs a score given a cut, and select a fixed ratio/number of cuts with high scores.

Original: However, they have two main drawbacks as mentioned in Section 1.
Revised: However, they suffer from two limitations as mentioned in Section 1.

Original: Policy network architecture.
Revised: Policy network architecture.

Original: To tackle the  problems, we propose a novel hierarchical sequence model (HEM) to learn cut selection policies.
Revised: To tackle the aforementioned problems, we propose a novel hierarchical sequence model (HEM) to learn cut selection policies.

Original: To promote efficient exploration, HEM leverages the hierarchical structure of the cut selection task to decompose the policy into two sub-policies, i.e., a higher-level policy π h and a lower-level policy π l (see Figure 2).
Revised: To promote efficient exploration, HEM leverages the hierarchical structure of the cut selection task to decompose the policy into two sub-policies, i.e., a higher-level policy π h and a lower-level policy π l . The policy network architecture of HEM is also illustrated in Figure 2.

Original: First , the higher-level policy learns the number of cuts that should be selected by predicting a proper ratio.
Revised: First , the higher-level policy learns the number of cuts that should be selected by predicting a proper ratio.

Original: Suppose the length of the state is N and the predicted ratio is k , then the predicted number of cuts that should be selected is ⌊ N ∗ k ⌋ , where ⌊·⌋ denotes the floor function.
Revised: Suppose the length of the state is N and the predicted ratio is k , then the predicted number of cuts that should be selected is ⌊ N ∗ k ⌋ , where ⌊·⌋ denotes the floor function.

Original: We define the higher-level policy by π h : S → P ([0 , 1]) , where π h ( ·| s ) denotes the probability distribution over [0 , 1] given the state s .
Revised: We define the higher-level policy by π h : S → P ([0 , 1]) , where π h ( ·| s ) denotes the probability distribution over [0 , 1] given the state s .

Original: Second , the lower-level policy learns to select an ordered subset with the size  determined by the higher-level policy.
Revised: Second , the lower-level policy learns to select an ordered subset with the size 1 determined by the higher-level policy.

Original: We define the lower-level policy by π l : S × [0 , 1] → P ( A ) , where π l ( ·| s, k ) denotes the probability distribution over the action space given the state s and the ratio k .
Revised: We define the lower-level policy by π l : S × [0 , 1] → P ( A ) , where π l ( ·| s, k ) denotes the probability distribution over the action space given the state s and the ratio k .

Original: To capture the interaction among cuts and the order of selected cuts, we formulate the lower-level policy as a sequence model.
Revised: Specifically, we formulate the lower-level policy as a sequence model, which can capture the interaction among cuts.

Original: Finally , we derive the cut selection policy via the law of total probability, i.e., π ( a k | s ) =
Revised: Finally , we derive the cut selection policy via the law of total probability, i.e., π ( a k | s ) =

Original: E k ∼ π h ( ·| s )
Revised: E k ∼ π h ( ·| s )

Original: [ π l ( a k | s, k )] , where k denotes the given ratio and a k denotes the action.
Revised: [ π l ( a k | s, k )] , where k denotes the given ratio and a k denotes the action.

Original: The policy is computed by an expectation, as a k cannot determine the ratio k .
Revised: The policy is computed by an expectation, as a k cannot determine the ratio k .

Original: For example, suppose that N = 100 and the length of a k is 10 , then the ratio k can be any number in the interval [0 . 1 , 0 . 11) .
Revised: For example, suppose that N = 100 and the length of a k is 10 , then the ratio k can be any number in the interval [0 . 1 , 0 . 11) .

Revised: Actually, we sample an action from the policy π by first sampling a ratio k from π h and then sampling an action from π l given the ratio.

Original: For the higher-level policy, we first model the higher-level policy as a tanh-Gaussian, i.e., a Gaussian distribution with an invertible squashing function ( tanh ), which is commonly seen in deep reinforcement learning (Schulman et al., 2017; Haarnoja et al., 2018).
Revised: For the higher-level policy, we first model the higher-level policy as a tanh-Gaussian, i.e., a Gaussian distribution with an invertible squashing function ( tanh ), which is commonly used in deep reinforcement learning (Schulman et al., 2017; Haarnoja et al., 2018).

Original: The mean and covariance of the Gaussian are given by neural networks.
Revised: The mean and variance of the Gaussian are given by neural networks.

Original: The support of the tanh-Gaussian is [ − 1 , 1] , but avalid ratio of selected cuts should belong to [0 , 1] .
Revised: The support of the tanh-Gaussian is [ − 1 , 1] , but a ratio of selected cuts should belong to [0 , 1] .

Original: Thus, we further perform a linear transformation on the tanh-Gaussian.
Revised: Thus, we further perform a linear transformation on the tanh-Gaussian.

Original: Specifically, we define the parameterized higher-level policy by π hθ 1 ( ·| s ) = 0 .
Revised: Specifically, we define the parameterized higher-level policy by π hθ 1 ( ·| s ) = 0 .

Original: 5 ∗ tanh ( K )
Revised: 5 ∗ tanh ( K )

Original: + 0 .
Revised: + 0 .

Original: 5 , where K ∼ N ( µ θ 1 ( s ) , σ θ 1 ( s )) .
Revised: 5 , where K ∼ N ( µ θ 1 ( s ) , σ θ 1 ( s )) .

Original: Since a state is a sequence  of feature vectors whose length is variable across different instances, we use a long-short term memory (LSTM) (Hochreiter & Schmidhuber, 1997) network to embed the sequence of candidate cuts.
Revised: Since the sequence lengths of states are variable across different instances (MILPs), we use a long-short term memory (LSTM) (Hochreiter & Schmidhuber, 1997) network to embed the sequence of candidate cuts.

Original: We then use a multi-layer perceptron (MLP)
Revised: We then use a multi-layer perceptron (MLP)

Original: (Goodfellow et al., 2016) to predict the mean and covariance from the last hidden state of the LSTM encoder.
Revised: (Goodfellow et al., 2016) to predict the mean and variance from the last hidden state of the LSTM.

Original: For the lower-level policy, we formulate it as a sequence model.
Revised: For the lower-level policy, we formulate it as a sequence model.

Original: That is, its input is a sequence of candidate cuts, and its output can be any ordered subset of candidate cuts with the size determined by the higher-level policy.
Revised: That is, its input is a sequence of candidate cuts, and its output is the probability distribution over ordered subsets of candidate cuts with the size determined by the higher-level policy.

Original: Specifically, given a state action pair ( s, k, a k ) , the sequence model computes the conditional probability π lθ 2 ( a k | s, k ) using a parametric model to estimate the terms of the probability chain rule, i.e., π lθ 2 ( a k | s, k ) =
Revised: Specifically, given a state action pair ( s, k, a k ) , the sequence model computes the conditional probability π lθ 2 ( a k | s, k ) using a parametric model to estimate the terms of the probability chain rule, i.e., π lθ 2 ( a k | s, k ) =

Original: (cid:81) mi
Revised: (cid:81) mi

Original: =1 p θ 2 ( a ik | a 1 k , . .
Revised: =1 p θ 2 ( a ik | a 1 k , . .

Original: , a i − 1 k , s, k ) .
Revised: , a i − 1 k , s, k ) .

Original: Here s = { s 1 , . . .
Revised: Here s = { s 1 , . . .

Original: , s N } is a sequence of N cut feature vectors, m =
Revised: , s N } is the input sequence, m =

Original: ⌊ N ∗ k ⌋ is the length of the output sequence, and a k = { a 1 k , . . .
Revised: ⌊ N ∗ k ⌋ is the length of the output sequence, and a k = { a 1 k , . . .

Original: , a mk } is a sequence of m indices, each corresponding a position in the input sequence s .
Revised: , a mk } is a sequence of m indices, each corresponding a position in the input sequence s .

Original: Such policy can be parametrized by  vanilla sequence model commonly used in machine translation (Sutskever et al., 2014; Vaswani et al., 2017).
Revised: Such policy can be parametrized by the vanilla sequence model commonly used in machine translation (Sutskever et al., 2014; Vaswani et al., 2017).

Original: However, thenumber of candidate cuts varies on different instances, even if the instances are from the same class of problems. Thus, the vanilla sequence model can only be applied to learning on a single instance.
Revised: However, the vanilla sequence model can only be applied to learning on a single instance, as the number of candidate cuts varies on different instances.

Original: To generalize across different instances, we use a pointer network (Vinyals et al., 2015; Bello* et al., 2017)—which uses attention as a pointer to select a member of the input sequence as the output at each decoder step—to parametrize π lθ 2 (see Appendix E.4 for details).
Revised: To generalize across different instances, we use a pointer network (Vinyals et al., 2015; Bello* et al., 2017)—which uses attention as a pointer to select a member of the input sequence as the output at each decoder step—to parametrize π lθ(see Appendix F.4.1 for details).

Original: To the best of our knowledge, we are the first to formulate the cut selection task as a sequence to sequence learning problem and apply the pointer network to cut selection, which has two main advantages as mentioned in Section 1.
Revised: To the best of our knowledge, we are the first to formulate the cut selection task as a sequence to sequence learning problem and apply the pointer network to cut selection. This leads to two major advantages: (1) capturing the underlying order information, (2) and the interaction among cuts. This is also illustrated through an example in Appendix E.

Original: Training: hierarchical policy gradient
Revised: Training: hierarchical policy gradient

Original: For the cut selection task, we aim to find θ that maximizes the expected reward over all trajectories
Revised: For the cut selection task, we aim to find θ that maximizes the expected reward over all trajectories

Original: [Equation]
Revised: [Equation]

Original: where θ =
Revised: where θ =

Original: [ θ 1 , θ 2 ] with [ θ 1 , θ 2 ] denoting the concatenation of the two vectors, π θ ( a k | s )
Revised: [ θ 1 , θ 2 ] with [ θ 1 , θ 2 ] denoting the concatenation of the two vectors, π θ ( a k | s )

Original: =
Revised: =

Original: E k ∼ π hθ 1 ( ·| s )
Revised: E k ∼ π hθ 1 ( ·| s )

Original: [ π lθ 2 ( a k | s, k )] , and µ denotes the initial state distribution.
Revised: [ π lθ 2 ( a k | s, k )] , and µ denotes the initial state distribution.

Original: To train the policy with a hierarchical structure, we derive a hierarchical policy gradient following the well-known policy gradient theorem (Sutton et al., 1999a; Sutton & Barto, 2018).
Revised: To train the policy with a hierarchical structure, we derive a hierarchical policy gradient following the well-known policy gradient theorem (Sutton et al., 1999a; Sutton & Barto, 2018).

Original: Proposition 1.
Revised: Proposition 1.

Original: Given the cut selection policy π θ ( a k | s ) =
Revised: Given the cut selection policy π θ ( a k | s ) =

Original: E k ∼ π hθ 1 ( ·| s )
Revised: E k ∼ π hθ 1 ( ·| s )

Original: [ π lθ 2 ( a k | s, k )] and the training objective (3), the hierarchical policy gradient takes the form of
Revised: [ π lθ 2 ( a k | s, k )] and the training objective (3), the hierarchical policy gradient takes the form of

Original: [Equation]
Revised: [Equation]

Original: We provide detailed proof in Appendix A.
Revised: We provide detailed proof in Appendix A.

Original: We use the derived hierarchical policy gradient to update the parameters of the higher-level and lower-level policies.
Revised: We use the derived hierarchical policy gradient to update the parameters of the higher-level and lower-level policies.

Original: We implement the training algorithm in a parallel manner that is closely related to the asynchronous advantage actor-critic (A3C) (Mnih et al., 2016).
Revised: We implement the training algorithm in a parallel manner that is closely related to the asynchronous advantage actor-critic (A3C) (Mnih et al., 2016).

Original: Due to limited space, we summarize the procedure of the training algorithm in Appendix
Revised: Due to limited space, we summarize the procedure of the training algorithm in Appendix

Original: E.6.
Revised: F.3.6.

Original: Moreover, we discuss some additional advantages of HEM (see Appendix E.7 for details).
Revised: Moreover, we discuss some more advantages of HEM (see Appendix F.4.3 for details).

Original: HEM leverages the hierarchical structure of the cut selection task, which is important for efficient exploration in complex decision-making tasks (Sutton et al., 1999b).
Revised: HEM leverages the hierarchical structure of the cut selection task, which is important for efficient exploration in complex decision-making tasks (Sutton et al., 1999b).

Original: (2)  HEMcan be trained via gradient-based algorithms, which is sample efficient (Sutton & Barto, 2018).
Revised: (2) We train HEM via gradientbased algorithms, which is sample efficient (Sutton & Barto, 2018).

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: Our experiments have five main parts: #Experiment 1.
Revised: Our experiments have five main parts: Experiment 1.

Original: Evaluate our approach on three classicalsynthetic MILP problems and six challenging MILP problem benchmarks from diverse application areas.
Revised: Evaluate our approach on three classical MILP problems and six challenging MILP problem benchmarks from diverse application areas.

Original: #Experiment 2.
Revised: Experiment 2.

Original: Perform carefully designed ablation studies of our approach.
Revised: Perform carefully designed ablation studies to provide further insight into HEM.

Original: #Experiment 3.
Revised: Experiment 3.

Original: Test whether HEM can generalize to instances significantly larger than  seen during training.
Revised: Test whether HEM can generalize to instances significantly larger than those seen during training.

Original: #Experiment 4.
Revised: Experiment 4.

Original: Visualize the cuts selected by our method compared to the baselines.
Revised: Visualize the cuts selected by our method compared to the baselines.

Original: #Experiment 5.
Revised: Experiment 5.

Original: Deploy our approach to real-world production planning problems.
Revised: Deploy our approach to real-world production planning problems.

Original: Benchmarks.
Revised: Benchmarks.

Original: We evaluate our approach on nine NP -hard MILP problem benchmarks, which consist of three classical synthetic MILP problems and six challenging MILP problems from diverse application areas.
Revised: We evaluate our approach on nine NP -hard MILP problem benchmarks, which consist of three classical synthetic MILP problems and six challenging MILP problems from diverse application areas.

Original: We divide the nine problem benchmarks into three categories according to the difficulty to solve them using the SCIP 8.0.0 solver (Bestuzheva et al., 2021).
Revised: We divide the nine problem benchmarks into three categories according to the difficulty of solving them using the SCIP 8.0.0 solver (Bestuzheva et al., 2021).

Original: We call the three categories easy, medium, and hard datasets, respectively.
Revised: We call the three categories easy, medium, and hard datasets, respectively.

Original: (1) Easy datasets consist of three widely used synthetic MILP problem benchmarks: Set Covering (Balas & Ho, 1980), Maximum Independent Set (Bergman et al., 2016), and Multiple Knapsack (Scavuzzo et al., 2022).
Revised: (1) Easy datasets comprise three widely used synthetic MILP problem benchmarks: Set Covering (Balas & Ho, 1980), Maximum Independent Set (Bergman et al., 2016), and Multiple Knapsack (Scavuzzo et al., 2022).

Original: We artificially generate instances following (Gasse et al., 2019; Sun et al., 2020).
Revised: We artificially generate instances following Gasse et al. ; Sun et al.

Original: (2) Medium datasets consist of MIK (Atamt¨urk, 2003) and CORLAT (Gomes et al., 2008), which are widely used benchmarks for evaluating MILP solvers (He et al., 2014; Nair et al., 2020).
Revised: (2) Medium datasets comprise MIK (Atamt¨urk, 2003) and CORLAT (Gomes et al., 2008), which are widely used benchmarks for evaluating MILP solvers (He et al., 2014; Nair et al., 2020).

Original: (3) Hard datasets include the Load Balancing problem, inspired by real-life applications of large-scale systems, and the Anonymous problem, inspired by a large-scale industrial application (Bowly et al., 2021).
Revised: (3) Hard datasets include the Load Balancing problem, inspired by real-life applications of large-scale systems, and the Anonymous problem, inspired by a large-scale industrial application (Bowly et al., 2021).

Original: Moreover, hard datasets contain benchmarks from MIPLIB 2017 (MIPLIB) (Gleixner et al., 2021).
Revised: Moreover, hard datasets contain benchmarks from MIPLIB 2017 (MIPLIB) (Gleixner et al., 2021).

Original: Although Turner et al.
Revised: Although Turner et al.

Original: (2022) has shown that directly learning over the full MIPLIB can be extremely challenging, we propose to learn over subsets of MIPLIB.
Revised: (2022) has shown that directly learning over the full MIPLIB can be extremely challenging, we propose to learn over subsets of MIPLIB.

Original: We construct two subsets, called MIPLIB mixed neos and MIPLIB mixed supportcase.
Revised: We construct two subsets, called MIPLIB mixed neos and MIPLIB mixed supportcase.

Original: Due to limited space, please see Appendix D.1 for details of these datasets.
Revised: Due to limited space, please see Appendix D.1 for details of these datasets.

Original: Experimental setup.
Revised: Experimental setup.

Original: Throughout all experiments, we use SCIP 8.0.0 (Bestuzheva et al., 2021) as the backend solver, which is the state-of-the-art open source solver and is widely used in research of machine learning for combinatorial optimization (Gasse et al., 2019;
Revised: Throughout all experiments, we use SCIP 8.0.0 (Bestuzheva et al., 2021) as the backend solver, which is the state-of-the-art open source solver, and is widely used in research of machine learning for combinatorial optimization (Gasse et al., 2019;

Original: Huang et al., 2022; Turner et al., 2022; Nair et al., 2020).
Revised: Huang et al., 2022; Turner et al., 2022; Nair et al., 2020).

Original: Following (Gasse et al., 2019; Huang et al., 2022; Paulus et al., 2022), we only allow cutting plane generation and selection at the root node and set the cut separationrounds as one.
Revised: Following Gasse et al. ; Huang et al. ; Paulus et al. , we only allow cutting plane generation and selection at the root node, and set the cut separation

Revised: [Table]

Original: All other SCIP parametersare kept to default so as to make comparisons as fair and reproducible as possible.
Revised: rounds as one. We keep all the other SCIP parameters to default so as to make comparisons as fair and reproducible as possible.

Original: We emphasize that all of the SCIP solver’s advanced features, such as presolve and heuristics, are open, which ensures that our setup is consistent with the realistic setting.
Revised: We emphasize that all of the SCIP solver’s advanced features, such as presolve and heuristics, are open, which ensures that our setup is consistent with the practice setting.

Original: Throughout all experiments, we set the default time limit as 300 seconds.
Revised: Throughout all experiments, we set the solving time limit as 300 seconds.

Original: For completeness,[Table] we also evaluate HEM with a much longer time limit of 3 hours.
Revised: For completeness, we also evaluate HEM with a much longer time limit of three hours.

Original: The results are given inAppendix
Revised: The results are given in

Original: F.5.
Revised: Appendix G.6.

Original: We train HEM with ADAM (Kingma & Ba, 2014) using the PyTorch (Paszke et al., 2019) and MindSpore (Chen, 2021).We use a single machine that contains eight GPU devices (NVidia
Revised: We train HEM with ADAM (Kingma & Ba, 2014) using the PyTorch (Paszke et al., 2019). Additionally, we also provide another implementation using the MindSpore (Chen, 2021).

Original: GeForce GTX 3090 Ti) and 32 Intel Gold 6246R CPUs. For simplicity, we split each dataset into the training and test sets without the valid set.
Revised: For simplicity, we split each dataset into the train and test sets with 80% and 20% instances. To further improve HEM, one can construct a valid set for hyperparameters tuning.

Original: We train our model on the training set, and select themodel that achieves the best performance on the training set to evaluate on the test set.
Revised: We train our model on the train set, and select the best model on the train set to evaluate on the test set.

Original: Please refer to Appendix E.3 for implementation details and hyperparameters of HEM.
Revised: Please refer to Appendix F.3 for implementation details, hyperparameters, and hardware specification.

Original: Baselines.
Revised: Baselines.

Original: Our baselines include five widely-used human-designed cut selection rules and a stateof-the-art learning-basedcut selection method.
Revised: Our baselines include five widely used human-designed cut selection rules and a state-ofthe-art (SOTA) learning-based method.

Original: Cut selection rules include NoCuts, Random, Normalized Violation (NV), Efficacy (Eff), and Default.
Revised: Cut selection rules include NoCuts, Random, Normalized Violation (NV), Efficacy (Eff), and Default.

Original: NoCuts does not add any cuts.
Revised: NoCuts does not add any cuts.

Original: Default denotes the default cut selection rule used in SCIP 8.0.0.
Revised: Default denotes the default cut selection rule used in SCIP 8.0.0.

Original: For learning-based methods, we implement a  variant of the state-of-the-art learning-based methods (Tang et al., 2020; Huang et al., 2022), namely score-based policy (SBP).
Revised: For learning-based methods, we implement a slight variant of the SOTA learning-based methods (Tang et al., 2020; Huang et al., 2022), namely scorebased policy (SBP).

Original: Please see Appendix E.2 for implementation details of these baselines.
Revised: Please see Appendix F.2 for implementation details of these baselines.

Original: Evaluation metrics.
Revised: Evaluation metrics.

Original: We use two primary evaluation metrics, i.e., the average solving time (Time, lower is better), and the average primal-dual gap integral (PD integral, lower is better).
Revised: We use two widely used evaluation metrics, i.e., the average solving time (Time, lower is better), and the average primal-dual gap integral (PD integral, lower is better).

Original: We provide more results in terms of another two metrics, i.e., the average number of nodes and the average primal-dual gap, in Appendix F.2.
Revised: Additionally, we provide more results in terms of another two metrics, i.e., the average number of nodes and the average primal-dual gap, in Appendix G.2.

Original: Furthermore, to evaluate different cut selection strategies compared to pure branch-and-bound without cutting plane separation, we propose an Improvement metric.
Revised: Furthermore, to evaluate different cut selection methods compared to pure branch-and-bound without cutting plane separation, we propose an Improvement metric.

Original: Specifically, we define the metric by Im M ( · ) =
Revised: Specifically, we define the metric by Im M ( · ) =

Original: M ( NoCuts ) − M ( · ) M ( NoCuts ) , where M ( NoCuts ) represents the performance of NoCuts, and M ( · ) represents a map from a method to its performance.
Revised: M ( NoCuts ) − M ( · ) M ( NoCuts ) , where M ( NoCuts ) represents the performance of NoCuts, and M ( · ) represents a mapping from a method to its performance.

Original: The improvement metric represents the improvement of a given method compared to the NoCuts method.
Revised: The improvement metric represents the improvement of a given method compared to NoCuts.

Original: We mainly focus on the Time metric on  easy datasets , as the solver can solve all instances to optimality within the  time limit.
Revised: We mainly focus on the Time metric on the easy datasets , as the solver can solve all instances to optimality within the given time limit.

Original: However, HEM and  baselines cannot solve all instances to optimality within the time limit on  medium and hard datasets.
Revised: However, HEM and the baselines cannot solve all instances to optimality within the time limit on the medium and hard datasets.

Original: As a result, the average solving time of those unsolved instances is the same, which makes it difficult to distinguish the performance of different cut selection methods using the Time metric.
Revised: As a result, the average solving time of those unsolved instances is the same, which makes it difficult to distinguish the performance of different cut selection methods using the Time metric.

Original: Therefore, we mainly focus on the PD integral metric on  medium and hard datasets .
Revised: Therefore, we mainly focus on the PD integral metric on the medium and hard datasets .

Original: Note that the PD integral is also a well-recognized metric for measuring the solver performance (Bowly et al., 2021; Cao et al., 2022).
Revised: The PD integral is also a well-recognized metric for evaluating the solver performance (Bowly et al., 2021; Cao et al., 2022).

Original: #Experiment 1.
Revised: Experiment 1.

Original: Comparative evaluation  (1) Easy datasets.
Revised: Comparative evaluation The results in Table 1 suggest the following. (1) Easy datasets.

Original: HEM significantly outperforms  the baselines on  easy datasets, especially on Max Independent Set and Multiple Knapsack.
Revised: HEM significantly outperforms all the baselines on the easy datasets, especially on Max- [Table]

Original: HEM and
Revised: [Table]

Original:  SBP achieve better performance than all  rule-based baselines, demonstrating that it is promising to enhance modern MILP solvers by learning cut selection.
Revised: imum Independent Set and Multiple Knapsack. SBP achieves much better performance than all the rule-based baselines, demonstrating that our implemented SBP is a strong baseline.

Original: Compared to SBP, HEM improves the Time by 7 . 7% and 16 .
Revised: Compared to SBP, HEM improves the Time by up to 16 .

Original: 4% on Max Independent Set and Multiple Knapsack, respectively, which demonstrates the superiority of our method over  learning-based baselines.
Revised: 4% on the three datasets, demonstrating the superiority of our method over the SOTA learning-based method.

Original: (2) Medium datasets.
Revised: (2) Medium datasets.

Original: On MIK [Table]
Revised: On MIK and CORLAT,

Original: [Table] and CORLAT, HEM still outperforms  the baselines.
Revised: HEM still outperforms all the baselines.

Original: Especially on CORLAT, HEM achieves at least 33 .
Revised: Especially on CORLAT, HEM achieves at least 33 .

Original: 48% improvement in terms of the PD integral compared to the baselines.
Revised: 48% improvement in terms of the PD integral compared to the baselines.

Original: (3) Hard datasets.
Revised: (3) Hard datasets.

Original: HEM significantly outperforms the baselines in terms of the PD integral on several problems in  hard datasets.
Revised: HEM significantly outperforms the baselines in terms of the PD integral on several problems in the hard datasets.

Original: We emphasize that HEM achieves outstanding performance on two challenging benchmarks from MIPLIB 2017 and challenging real-world problems (Load Balancing and Anonymous), highlighting the potential of applying HEM to  large-scale real-world applications.
Revised: HEM achieves outstanding performance on two challenging datasets from MIPLIBand real-world problems (Load Balancing and Anonymous), demonstrating the powerful ability to enhance MILP solvers with HEM in large-scale real-world applications.

Original: Moreover, SBP performs extremely poor on several medium and hard datasets, which implies that it can be difficult to learn good cut selection policies on challenging MILP problems.
Revised: Moreover, SBP performs extremely poorly on several medium and hard datasets, which implies that it can be difficult to learn good cut selection policies on challenging MILP problems.

Original: #Experiment 2.
Revised: Experiment 2.

Original: Ablation study We present ablation studiesof HEM on Max Independent Set,
Revised: Ablation study We present ablation studies on Maximum Independent Set (MIS),

Original: CORLAT, and MIPLIB mixed neos, which are representative datasets from the easy, medium, and hard datasets, respectively.
Revised: CORLAT, and MIPLIB mixed neos, which are representative datasets from the easy, medium, and hard datasets.

Original: We provide additional results on more datasets in Appendix F.3.
Revised: We provide more results on the other datasets in Appendix G.3.

Original: Contribution of each component.
Revised: Contribution of each component.

Original: First, we perform ablation studies to understand the contribution of eachindividual component in HEM.
Revised: We perform ablation studies to understand the contribution of each component in HEM.

Original: HEM contains two components: the lower-level model and the higher-level model.

Original: We report the performance of HEM and HEM without the higher-level model (HEM w/o H) in Table 2.
Revised: We report the performance of HEM and HEM without the higher-level model (HEM w/o H) in Table 2.

Original: HEM w/o H is essentially a pointer network.
Revised: HEM w/o H is essentially a pointer network.

Original: Note that it can still implicitly predicts the number of cuts that should be selected by predicting an end token as used in language tasks (Sutskever et al., 2014).
Revised: Note that it can still implicitly predicts the number of cuts that should be selected by predicting an end token as used in language tasks (Sutskever et al., 2014).

Original: Please see Appendix E.5 for details.
Revised: Please see Appendix F.4.2 for details.

Original: The results in
Revised: First , the results in

Original: Table 2 show that HEM w/o H outperforms all  baselinesin terms of all metrics on Max Independent
Revised: Table 2 show that HEM w/o H outperforms all the baselines on MIS and CORLAT, demonstrating the advantages of the lower-level model.

Original: Set and CORLAT, which demonstrates the advantages of formulating the cut selection task as a sequence to sequence learning problem.

Original: Although HEM w/o H outperforms Default on MIPLIB mixed neos, HEM w/o H performs on par with SBP.
Revised: Although HEM w/o H outperforms Default on MIPLIB mixed neos, HEM w/o H performs on par with SBP.

Original: A possible reason is thatthe cardinality of the action space is large, which makes it  difficult for the sequence model to explore the action space efficiently. Therefore, HEM w/o H tends to be trapped to  local optimal.
Revised: A possible reason is that it is difficult for HEM w/o H to explore the action space efficiently, and thus HEM w/o H tends to be trapped to the local optimum.

Original: Furthermore, the results in Table 2 show that HEM significantly outperforms HEM w/o H and  baselines onthree datasets, demonstrating the effectiveness of the higher-level model  for efficient exploration in complex tasks.
Revised: Second , the results in Table 2 show that HEM significantly outperforms HEM w/o H and the baselines on the three datasets. The results demonstrate that the higher-level model is important for efficient exploration in complex tasks, thus significantly improving the solving efficiency.

Original: The importance of tackling (P1)-(P3) 1 .
Revised: The importance of tackling (P1)-(P3).

Original: Second, we perform ablation studies to understand the importance of tackling (P1)-(P3) in cut selection.
Revised: We perform ablation studies to understand the importance of tackling (P1)-(P3) in cut selection.

Original: (1) HEM.
Revised: (1) HEM.

Original: HEM tackles (P1)-(P3) in cut selection simultaneously.
Revised: HEM tackles (P1)-(P3) in cut selection simultaneously.

Original: (2) HEM-ratio.
Revised: (2) HEM-ratio.

Original: In order not to learn how many cuts should be selected, we remove the higher-level model of HEM and force the lower-level model to select a fixed ratio of cuts .
Revised: In order not to learn how many cuts should be selected, we remove the higher-level model of HEM and force the lower-level model to select a fixed ratio of cuts .

Original: We denote it by HEM-ratio.
Revised: We denote it by HEM-ratio.

Original: Note that HEM-ratio is different from HEM w/o H (see Appendix E.5).
Revised: Note that HEM-ratio is different from HEM w/o H (see Appendix F.4.2).

Original: HEM-ratio tackles (P1) and (P3) in cut selection.
Revised: HEMratio tackles (P1) and (P3) in cut selection.

Original: (3) HEM-ratio-order.
Revised: (3) HEM-ratio-order.

Original: To further mute the effect of the order of selected cuts, we reorder the selected cuts given by HEM-ratio with the original index of the generated cuts, which we denote by HEM-ratio-order.
Revised: To further mute the effect of the order of selected cuts, we reorder the selected cuts given by HEM-ratio with the original index of the generated cuts, which we denote by HEM-ratio-order.

Original: HEM-ratio-order mainly tackles (P1) in cut selection.
Revised: HEM-ratio-order mainly tackles (P1) in cut selection.

Original: The results in Table 3 suggest the following.
Revised: The results in Table 3 suggest the following.

Original: HEM-ratio-order significantly outperforms Default and NoCuts, demonstrating that tackling (P1) by data-driven methods is crucial.
Revised: HEM-ratio-order significantly outperforms Default and NoCuts, demonstrating that tackling (P1) by data-driven methods is crucial.

Original: HEM significantly outperforms HEM-ratio in terms of the PD integral, demonstrating the significance of tackling (P2) .
Revised: HEM significantly outperforms HEM-ratio in terms of the PD integral, demonstrating the significance of tackling (P2) .

Original: HEM-ratio outperforms HEM-ratio-order in terms of the Time and the PD integral, which demonstrates the importance of tackling (P3) .
Revised: HEM-ratio outperforms HEM-ratio-order in terms of the Time and the PD integral, which demonstrates the importance of tackling (P3) .

Original: Moreover, HEM-ratio and HEM-ratio-order perform better than SBP on Max Independent Set and CORLAT, demonstrating the advantages of using  sequence model to learn cut selection over SBP.
Revised: Moreover, HEM-ratio and HEM-ratio-order perform better than SBP on MIS and CORLAT, demonstrating the advantages of using the sequence model to learn cut selection over SBP.

Original: HEM-ratio and HEM-ratio-order perform on par with SBP on MIPLIB mixed neos.
Revised: HEM-ratio and HEM-ratio-order perform on par with SBP on MIPLIB mixed neos.

Original: We provide possible reasons in Appendix F.3.
Revised: We provide possible reasons in Appendix G.3.1.

Original: [Table]
Revised: [Table]

Original: [Table]

Original: [Table]

Original: [Table] [Table] [Table] [Table]
Revised: [Figure]

Original: #Experiment 3.
Revised: Experiment 3.

Original: Generalization We evaluate the ability of HEM to generalize across different sizes of MILPs.
Revised: Generalization We evaluate the ability of HEM to generalize across different sizes of MILPs.

Original: Let n × m denotes the size of MILP instances.
Revised: Let n × m denote the size of MILP instances.

Original: Following (Gasse et al., 2019; Sun et al., 2020), we test the generalization ability of HEM on Set Covering and Max Independent Set, as we can artificially generate instances with arbitrary sizes for synthetic MILP problems.
Revised: Following Gasse et al. ; Sun et al. (2020), we test the generalization ability of HEM on Set Covering and Maximum Independent Set (MIS), as we can artificially generate instances with arbitrary sizes for synthetic MILP problems.

Original: On Max Independent Set, we test HEM on four times and nine times larger instances than  seen during training.
Revised: On MIS, we test HEM on four times and nine times larger instances than those seen during training.

Original: The results in Table 4 (Left) show that HEMgeneralizes well to instances that are significantly larger than seen during training. In particular, HEM significantly outperforms all baselines in terms of the
Revised: The results in Table 4 ( Left ) show that HEM significantly outperforms the baselines in terms of the

Original: Time  on 4 ×Max Independent Set. Moreover, HEM consistently outperforms SBP in terms of the Time and PD integral, demonstrating the superiority of HEM.
Revised: Time and the PD integral on 4 × and 9 × MIS, demonstrating the superiority of HEM in terms of the generalization ability.

Original: Interestingly, SBP also generalizes well to large instances, highlighting the potential for good generalization by learning cut selection.
Revised: Interestingly, SBP also generalizes well to large instances, demonstrating that SBP is a strong baseline. We provide more results on Set Covering in Appendix G.4.

Original: [Figure]
Revised: [Figure]

Original: To evaluate whether HEM can select cuts that complement each other nicely, we visualize the cuts selected by  HEM-ratio and SBP on a randomly sampled instance from Max Independent
Revised: Colored points illustrate the reduced cut features. The area covered by the dashed lines represents the diversity of selected cuts. The results show that HEM-ratio selects much more diverse cuts than SBP.

Original:  Set and CORLAT, respectively.
Revised: Experiment 4. Visualization of selected cuts We visualize the diversity of selected cuts, an important metric for evaluating whether the selected cuts complement each other nicely (Dey & Molinaro, 2018b). We visualize the cuts selected by HEM-ratio and SBP on a randomly sampled instance from Maximum Independent Set and CORLAT, respectively.

Original: We evaluate HEM-ratio instead of HEM as HEM-ratio selects the same number of cuts as SBP.
Revised: We evaluate HEM-ratio rather than HEM, as HEM-ratio selects the same number of cuts as SBP.

Original: Specifically, we perform principal component analysis(Mohri et al., 2018) on  selected cuts to reduce the cut features to two-dimensional space as shown in Figure 3.
Revised: Furthermore, we perform principal component analysis on the selected cuts to reduce the cut features to two-dimensional space.

Original: Colored points illustrate  reduced cut features.
Revised: Colored points illustrate the reduced cut features.

Original: To visualize the diversity of selected cuts, we use dashed lines to connect the points with the smallest and largest x,y coordinates.
Revised: To visualize the diversity of selected cuts, we use dashed lines to connect the points with the smallest and largest x,y coordinates.

Original: That is, the area covered by the dashed lines represents the diversity of selected cuts.
Revised: That is, the area covered by the dashed lines represents the diversity .

Original: The results show that SBP tends to select many similar cuts that are possibly redundant, especially on Max Independent Set.
Revised: Figure 3 shows that SBP tends to select many similar cuts that are possibly redundant, especially on Maximum Independent Set.

Original: In contrast, HEM-ratio selects much more diverse cutsthan SBP, which demonstrates that HEM tends to select cuts that  complement each other nicely.
Revised: In contrast, HEM-ratio selects much more diverse cuts that can well complement each other.

Original: Please refer to Appendix F.4 for results on more datasets.
Revised: Please refer to Appendix G.5 for results on more datasets.

Original: #Experiment 5.
Revised: Experiment 5.

Original: Deployment in real-world production planning problems To further evaluate the effectiveness of our proposed HEM, we deploy HEM to  real-world production planning problems  at an anonymous enterprise, which is one of the largest global commercial technology enterprise.
Revised: Deployment in real-world production planning problems To further evaluate the effectiveness of our proposed HEM, we deploy HEM to large-scale real-world production planning problems 2 at an anonymous enterprise, which is one of the largest global commercial technology enterprises.

Original: Please refer to Appendix D.3 for more details of the problems.
Revised: Please refer to Appendix D.3 for more details of the problems.

Original: The results in Table 4 (Right) show that HEM significantly outperforms all  baselines in terms of the Time and PD integral.
Revised: The results in Table( Right ) show that HEM significantly outperforms all the baselines in terms of the Time and PD integral.

Original: Specifically,

Original: HEM improves the Time by 13 . 5% , and 12 . 5% compared to NoCuts, Default, and SBP, respectively. HEM improves the PD integral by 11 .
Revised: The results demonstrate the strong ability to enhance modern MILP solvers with our proposed

Original: 84% , 10 . 93% , and 6 .

Original: 73% compared to NoCuts, Default, and SBP, respectively. The results highlight the ability to improve the efficiency of modern MILP solvers with our proposed HEM inlarge-scale real-world applications.
Revised: HEM in real-world applications.

Original: Interestingly, Default performs poorer than NoCuts, which implies that an improper cut selection policy could significantly degrade the efficiency of MILP solvers.
Revised: Interestingly, Default performs poorer than NoCuts, which implies that an improper cut selection policy could significantly degrade the performance of MILP solvers.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, wefirst observe from extensive empirical results that the order of selected cuts has a large impact on the efficiency of solving MILPs as well.
Revised: In this paper, we observe from extensive empirical results that the order of selected cuts has a significant impact on the efficiency of solving MILPs.

Original: Wethen propose a novel h ierarchical s e quence m odel (HEM) to learn cut selection policies via reinforcement learning.
Revised: We propose a novel h ierarchical s e quence m odel (HEM) to learn cut selection policies via reinforcement learning.

Original: Specifically,

Original:  HEM consists of a two-level model: (1) a higher-level modelaims to learn how many cuts  should be selected, (2) and a lower-level model formulates the cut selection task as a sequence to sequence learning problem to learn to select an ordered subset with the size determined by the higher-level model.
Revised: Specifically, HEM consists of a two-level model: (1) a higher-level model to learn the number of cuts that should be selected, (2) and a lower-level model—that formulates the cut selection task as a sequence to sequence learning problem—to learn policies selecting an ordered subset with the size determined by the higher-level model.

Original: Experiments show that HEM significantly improves the efficiency of solving MILPs compared to human-designed and learning-based baselines on both synthetic and large-scale real-world
Revised: Experiments show that HEM significantly improves the efficiency of solving MILPs compared to human-designed and learning-based baselines on both synthetic and large-scale real-world

Original: MILPs.
Revised: MILPs.

Original: We believe that our proposed approach will bring new insights into learning cut selection.  
Revised: We believe that our proposed approach brings new insights into learning cut selection.  E THICS S TATEMENT

Original: A P ROOF A. P ROOF OF P ROPOSITION
Revised: As our work is fundamental research in solving mixed-integer linear programs (MILPs) there are no direct ethical risks. Modern MILP solvers with expert-designed cut selection heuristics aim to solve general MILPs, while modern MILP solvers with our proposed HEM can learn more effective cut selection heuristics from MILPs collected from certain types of real-world applications  R EPRODUCIBILITY S TATEMENT To ensure reproducibility we describe our proposed HEM in Section 4 and provide more implementation details in Appendix F. Moreover, we provide the pseudocode of our proposed HEM in

Original: Proposition 2.
Revised: Appendix F.3.6.

Original: Given the cut selection policy π θ ( a k | s ) = E k ∼ π hθ 1 ( ·| s )
Revised: We will release all the source codes publicly once the paper is accepted.

Original: [ π lθ 2 ( a k | s, k )] and the training objective (3), the hierarchical policy gradient takes the form of 
Revised: All datasets except the real-world production planning problems used in the experiments are publicly available. Please refer to Appendix D for details of these datasets.

Original: [Equation]
Revised: We will also publicly release the dataset from the real-world production planning problems once the paper is accepted. A P ROOF A. P ROOF OF P ROPOSITION

Original: Proof.
Revised: Proof.

Original: The optimization objective takes the form of
Revised: The optimization objective takes the form of

Original: [Equation]
Revised: [Equation]

Original: where θ = 
Revised: where θ = [ θ

Revised: [Equation]

Revised: θ 2 ] with [ θ

Revised: [Equation]

Original: [ θ 1 , θ 2 ] with [ θ 1 , θ 2 ] denoting the concatenation of the two vectors, π θ ( a k | s )
Revised: θ 2 ] denoting the concatenation of the two vectors [Equation] π θ ( a k | s [Equation]

Original: =
Revised: =

Original: E k ∼ π hθ1 ( ·| s )
Revised: E k ∼ π hθ

Original:  [ π hθ 2 ( a k | s, k )] , and µ denotes the initial state distribution.
Revised: [Equation] ·| s ) [ π hθ 2 ( a k | s, k )] [Equation] and µ denotes the initial state distribution.

Original: We first compute the policy gradient for θ 1 :
Revised: We first compute the policy gradient for θ 1 :

Original: [Equation]
Revised: [Equation]

Original: Let
Revised: Let

Original: [Equation]
Revised: [Equation]

Original: then we have that
Revised: then we have that

Original: [Equation]
Revised: [Equation]

Original: Therefore
Revised: Therefore

Original: [Equation]
Revised: [Equation]

Original: we have that
Revised: we have that

Original: [Equation]
Revised: [Equation]

Original: We then compute the policy gradient for θ 2 :
Revised: We then compute the policy gradient for θ 2 :

Original: [Equation]
Revised: [Equation]

Original: which completes the proof
Revised: which completes the proof

Original: [Equation]
Revised: [Equation]

Original: B R ELATED WORK B. M ACHINE LEARNING FOR MILP
Revised: B R ELATED W ORK

Revised: Machine learning for MILP.

Original: The use of machine learning methods to help improve the MILP solver performance has been an active topic of significant interest in recent years (Bengio et al., 2021; Lodi & Zarpellon, 2017; Bowly et al., 2021; Gasse et al., 2019).
Revised: The use of machine learning methods to help improve the MILP solver performance has been an active topic of significant interest in recent years (Bengio et al., ; Lodi & Zarpellon, 2017; Bowly et al., 2021; Gasse et al., 2019).

Original: During the solving process of the solvers, many crucial decisions that significantly impact the solver performance are based on heuristics (Achterberg, 2007).
Revised: During the solving process of the solvers, many crucial decisions that significantly impact the solver performance are based on heuristics (Achterberg, 2007).

Original: Recent methods propose to replace these hand-crafted heuristics with machine learning models (Bengio et al., 2021).
Revised: Recent methods propose to replace these hand-crafted heuristics with machine learning models (Bengio et al., 2021).

Original: This line of research has shown significant improvement on the solver performance, including cut selection (Tang et al., 2020; Paulus et al., 2022; Turner et al., 2022; Baltean-Lugojan et al., 2019), variable selection (Khalil et al., 2016; Gasse et al., 2019; Balcan et al., 2018; Zarpellon et al., 2021), node selection (
Revised: This line of research has shown significant improvement on the solver performance, including cut selection (Tang et al., 2020; Paulus et al., 2022; Turner et al., 2022; Baltean-Lugojan et al., 2019), variable selection (Khalil et al., 2016; Gasse et al., 2019; Balcan et al., 2018; Zarpellon et al., 2021), node selection (

Original: He et al., 2014; Sabharwal et al., 2012), column generation (Morabit et al., 2021), and primal heuristics selection (Khalil et al., 2017; Hendel et al., 2019).
Revised: He et al., 2014; Sabharwal et al., 2012), column generation (Morabit et al., 2021), and primal heuristics selection (Khalil et al., 2017; Hendel et al., 2019).

Original: In this paper, we focus on cut selection, which plays a significant role in modern MILP solvers (Dey & Molinaro, 2018a; Tang et al., 2020).
Revised: In this paper, we focus on cut selection, which plays a significant role in modern MILP solvers (Dey & Molinaro, 2018a; Tang et al., 2020).

Original: For cut selection, many existing learning-based methods (Tang et al., 2020; Paulus et al., 2022;
Revised: For cut selection, many existing learning-based methods (Tang et al., 2020; Paulus et al., 2022;

Original: Huang et al., 2022) focus on learning which cuts should be preferred by learning a scoring function to measure cut quality.
Revised: Huang et al., 2022) focus on learning which cuts should be preferred by learning a scoring function to measure cut quality.

Original: Specifically, (Tang et al., 2020) proposes a reinforcement learning approach to learn to score Gomory cuts (Gomory, 1960) and select a Gomory cut with the best scores.
Revised: Specifically, (Tang et al., 2020) proposes a reinforcement learning approach to learn to score Gomory cuts (Gomory, 1960) and select a Gomory cut with the best scores.

Original: Furthermore, (Paulus et al., 2022) designs a lookahead selection rule which selects a cut that yield the best dual bound improvement, and proposes to learn the expert rule via imitation learning.
Revised: Furthermore, (Paulus et al., 2022) designs a lookahead selection rule which selects a cut that yields the best dual bound improvement, and proposes to learn the expert rule via imitation learning.

Original: Instead of selecting the best cut, (Huang et al., 2022) frames cut selection as multiple instance learning to learn a scoring function and selects a fixed ratio of cuts with high scores.
Revised: Instead of selecting the best cut, (Huang et al., 2022) frames cut selection as multiple instance learning to learn a scoring function and selects a fixed ratio of cuts with high scores.

Original: However, they neglect the importance of learning how many cuts should be selected.
Revised: However, they neglect the importance of learning how many cuts should be selected.

Original: Moreover, we empirically show that the order of selected cuts has a large impact on the efficiency of solving MILPs (see Section 3).
Revised: Moreover, we empirically show that the order of selected cuts has a large impact on the efficiency of solving MILPs (see Section 3).

Original: Moreover, (Turner et al., 2022) proposes to learn the weightings of four existing scoring rules designed by experts.
Revised: Moreover, (Turner et al., 2022) proposes to learn the weightings of four existing scoring rules designed by experts.

Original: For the theoretical analysis, (Balcan et al., 2021) provides some provable guarantees for learning cut selection policies.
Revised: For the theoretical analysis, (Balcan et al., 2021) provides some provable guarantees for learning cut selection policies.

Original: B. S EQUENCE MODEL
Revised: Sequence model.

Original: Sequence model such as long-short term memory and Transformer has achieved outstanding performance in language tasks such as machine translation (Hochreiter & Schmidhuber, 1997; Sutskever et al., 2014; Vaswani et al., 2017).
Revised: Sequence model such as long-short term memory and Transformer has achieved outstanding performance in language tasks such as machine translation (Hochreiter & Schmidhuber, 1997; Sutskever et al., 2014; Vaswani et al., 2017).

Original: For combinatorial optimization, recent works (Vinyals et al., 2015; Bello* et al., 2017) propose a variant of the traditional sequence model, namely pointer network, which is applied to directly finding solutions for specific combinatorial optimization problems, such as the Travelling Salesman Problem (Lenstra & Shmoys, 2009).
Revised: For combinatorial optimization, recent works (Vinyals et al., 2015; Bello* et al., 2017) propose a variant of the traditional sequence model, namely pointer network, which is applied to directly finding solutions for specific combinatorial optimization problems, such as the Travelling Salesman Problem (Lenstra & Shmoys, 2009).

Original: Instead of finding solutions directly, we propose to use the pointer network for cut selection in modern MILP solvers.
Revised: Instead of finding solutions directly, we propose to use the pointer network for cut selection in modern MILP solvers.

Original: To the best of our knowledge, we are the first to apply the pointer network to cut selection, which not only captures the order of selected cuts, but also can well capture the interaction among cuts to select cuts that complement each other nicely.
Revised: To the best of our knowledge, we are the first to apply the pointer network to cut selection, which not only captures the order of selected cuts, but also can well capture the interaction among cuts to select cuts that complement each other nicely.

Original: C M ORE DETAILS OF BACKGROUND C. M ORE DETAILS OF THE PRIMAL - DUAL GAP INTEGRAL
Revised: C M ORE D ETAILS OF B ACKGROUND C. M ORE DETAILS OF THE PRIMAL - DUAL GAP INTEGRAL

Original: We keep track of two important bounds when running branch-and-cut, including the global primal and dual bound.
Revised: We keep track of two important bounds when running branch-and-cut, including the global primal and dual bound.

Original: The global primal bound corresponds to the value of the best feasible solution found so far, which is the best upper bound of the problem in (1).
Revised: The global primal bound corresponds to the value of the best feasible solution found so far, which is the best upper bound of the problem in (1).

Original: The global dual bound corresponds to the minimum dual bound across all leaves of the search tree, which is the best lower bound of the problem in (1).
Revised: The global dual bound corresponds to the minimum dual bound across all leaves of the search tree, which is the best lower bound of the problem in (1).

Original: We define the primal-dual gap integral by the area between the curve of the solver’s global primal bound and the curve of the solver’s global dual bound.
Revised: We define the primal-dual gap integral by the area between the curve of the solver’s global primal bound and the curve of the solver’s global dual bound.

Original: With a time limit T , we define the primal-dual gap integral by
Revised: With a time limit T , we define the primal-dual gap integral by

Original: [Equation]
Revised: [Equation]

Original: where c is the objective coefficient vector as in (1), x ∗ t is the best feasible solution found at time t , z ∗ t is the best dual bound at time t .
Revised: where c is the objective coefficient vector as in (1), x ∗ t is the best feasible solution found at time t , z ∗ t is the best dual bound at time t .

Original: We define the primal-dual gap by the difference between the global primal bound and the global dual bound.
Revised: We define the primal-dual gap by the difference between the global primal bound and the global dual bound.

Original: In SCIP 8.0.0 (Bestuzheva et al., 2021), the initial value of the primal-dual gap is set to a constant 100.
Revised: In SCIP 8.0.0 (Bestuzheva et al., 2021), the initial value of the primal-dual gap is set to a constant 100.

Original: The primal-dual gap integral is a well-recognized metric for measuring the solver performance.
Revised: The primal-dual gap integral is a well-recognized metric for evaluating solver performance.

Original: For example, the primal-dual gal integral isset as a primary evaluation metric in the NeurIPS 2021 ML4CO competition (Bowly et al., 2021).
Revised: For example, the primal-dual gal integral is a primary evaluation metric in the NeurIPS 2021 ML4CO competition (Bowly et al., 2021).

Original: D D ETAILS OF DATASETS USED IN THIS PAPER D. D ATASETS USED IN THE MAIN EVALUATION
Revised: D D ETAILS OF THE D ATASETS U SED IN THIS P APER D. T HE DATASETS USED IN THE MAIN EVALUATION

Original: Easy datasets.
Revised: Easy datasets.

Original:  Easy datasets consist of  Set Covering, Max Independent Set, and Multiple Knapsack.
Revised: The SCIP 8.0.0 solver needs one minute to solve the MILP instances in the easy datasets to optimality. Easy datasets are comprised of three synthetic MILP problems: Set Covering (Balas & Ho, 1980), Maximum Independent Set (Bergman et al., 2016), and Multiple Knapsack (Scavuzzo et al., 2022). We choose these three classes of problems for the following reasons. First, they are widely used benchmarks for evaluating MILP solvers (Gasse et al., 2019;

Original: We artificially generate the synthetic MILP problems following (Gasse et al., 2019; Scavuzzo et al., 2022). 
Revised: Huang et al., 2022; Sun et al., 2020; Gupta et al., 2022). Second, they represent a wide collection of MILP problems encountered in practice.

Revised: Third, for each class of these problems, the average number of generated cuts is at least twenty, which ensures that proper cut selection strategies are significant for improving the solver performance.

Revised: Similarly to (Gasse et al., 2019; Scavuzzo et al., 2022; Sun et al., 2020; Gupta et al., 2022), we generate set covering instances with 500 rows and 1000 columns, Maximum Independent Set instances with graphs of 500 nodes and affinity set to 4, multiple knapsack instances with 60 items and 12 knapsacks.

Revised: For each benchmark, we generate a training set of 10,000 instances, and a test set of 100 instances that are never seen during training.

Original: Specifically, readers can refer to https://github.com/ds4dm/learn2branch or https://github.com/lascavana/rl2branch for code to generate the easy datasets.
Revised: Specifically, readers can refer to https://github.com/ds4dm/learn2branch or https://github.com/lascavana/rl2branch for code to generate the easy datasets.

Original: We will also release our code once the paper is accepted to be published.
Revised: We will also release our code once the paper is accepted to be published.

Original: Medium datasets.
Revised: Medium datasets.

Revised: The SCIP 8.0.0 solver needs at least five minutes to solve the instances in the medium datasets to optimality.

Original: Medium datasets consist of MIK (He et al., 2014) and CORLAT (Nair et al., 2020).
Revised: Following He et al. ; Hutter et al. ; Nair et al. (2020), medium datasets comprise MIK (Atamt ¨ urk, 2003), a set of MILP problems with knapsack constraints, and CORLAT (Gomes et al., 2008), a real dataset used for the construction of a wildlife corridor for grizzly bears in the Northern Rockies region.

Revised: Each problem set is split into training and test sets with 80% and 20% of the instances.

Original: Readers can refer to https://atamturk.ieor.berkeley.edu/data/ mixed.integer.knapsack/ for MIK.
Revised: Readers can refer to https://atamturk. ieor.berkeley.edu/data/mixed.integer.knapsack/ for MIK.

Original: Readers can refer to https://bitbucket.org/ mlindauer/aclib2/src/master/ for CORLAT.
Revised: Readers can refer to https://bitbucket.org/mlindauer/aclib2/src/master/ for CORLAT.

Original: [Table]
Revised: [Table]

Original: Hard datasets.
Revised: Hard datasets.

Revised: The SCIP 8.0.

Revised: solver needs at least one hour to solve the instances in the hard datasets to optimality.

Original: (1) Benchmarks from MIPLIB 2017.
Revised: (1) Benchmarks from MIPLIB 2017.

Original: Note that MIPLIB 2017 (MIPLIB) (Gleixner et al., 2021) contains instances of MILPs across many different application areas and has been used as a long-standing standard benchmark for MILP solvers (Nair et al., 2020; Turner et al., 2022; Gleixner et al., 2021).
Revised: Note that MIPLIB 2017 (MIPLIB) (Gleixner et al., 2021) contains instances of MILPs across many different application areas and has been used as a long-standing standard benchmark for MILP solvers (Nair et al., 2020; Turner et al., 2022; Gleixner et al., 2021).

Original: Previous work (Turner et al., 2022) has shown that directly learning over the full MIPLIB can be extremely challenging, as these instances are heterogeneous but machine learning  has difficulty in learning from heterogeneous datasets.
Revised: Previous work (Turner et al., 2022) has shown that directly learning over the full MIPLIB can be extremely challenging, as these instances are heterogeneous but machine learning [Table] has difficulty in learning from heterogeneous datasets.

Original: Despite this challenge, we take the first step towards learning over subsets of MIPLIB.
Revised: Despite this challenge, we take the first step towards learning over subsets of MIPLIB.

Original: Specifically, we construct two subsets by selecting similar instances from MIPLIB.
Revised: Specifically, we construct two subsets by selecting similar instances from MIPLIB.

Original: We measure the similarity between instances by 100 human-designed instance features (Gleixner et al., 2021).
Revised: We measure the similarity between instances by 100 human-designed instance features (Gleixner et al., 2021).

Original: Following (Turner et al., 2022), we first discard instances from MILLIB that satisfy any of the criteria in Table 5.
Revised: Following Turner et al. , we first discard instances from MILLIB that satisfy any of the criteria in Table 5.

Original: This ensures that a good cut selection strategy can significantly improve the dual bound on the remaining instances.
Revised: This ensures that a good cut selection policy can significantly improve the dual bound on the remaining instances.

Original: Note that we only use three of seven criteria that are used in (Turner et al., 2022) to preserve as many instances as possible.
Revised: Note that we only use three of seven criteria that are used in (Turner et al., 2022) to preserve as many instances as possible.

Original: To select similar instances from MIPLIB 2017, we first choose a representative instance that has knapsack constraints (neos-1456979), and a representative instance that has set covering constraints (supportcase40).
Revised: To select similar instances from MIPLIB 2017, we first choose a representative instance with knapsack constraints (neos-1456979), and a representative instance with set covering constraints (supportcase40).

Original: Then we construct the dataset MIPLIB mixed neos following the procedure in Algorithm xx.
Revised: Then we construct the dataset MIPLIB mixed neos following the procedure in Algorithm 1 with the initial instance neos-1456979.

Original: Furthermore, we construct the dataset MIPLIB mixed supportcaseby selecting instances following the procedure in Algorithm xx.
Revised: We construct the dataset MIPLIB mixed supportcase following the procedure in Algorithm 1 with the initial instance supportcase40.

Original: Note that We measure the similarity between instances by 100 human-designed instance features (Gleixner et al., 2021).
Revised: Note that We measure the similarity between instances by 100 human-designed instance features (Gleixner et al., 2021).

Revised: Each dataset is split into training and test sets with 80% and 20% of the instances.

Original: Specifically, MIPLIB mixed neos contains 20 instances: neos-1456979, ic97 tension, icir97 tension, l2p12, lectsched-4-obj, lectsched-5-obj, loopha13, neos-686190, neos-2294525-abba, neos3009394-lami, neos-3046601-motu, neos-3046615-murg, neos-3610173-itata, neos-4338804snowy, neos-5221106-oparau, neos-5260764-orauea, neos-5261882-treska, neos-5266653-tugela, neos16, and timtab1CUTS.
Revised: Specifically, MIPLIB mixed neos contains 20 instances: neos-1456979, ic97 tension, icir97 tension, l2p12, lectsched-4-obj, lectsched-5-obj, loopha13, neos-686190, neos-2294525-abba, neos3009394-lami, neos-3046601-motu, neos-3046615-murg, neos-3610173-itata, neos-4338804snowy, neos-5221106-oparau, neos-5260764-orauea, neos-5261882-treska, neos-5266653-tugela, neos16, and timtab1CUTS.

Original: Moreover, MIPLIB mixed supportcase contains 40 instances: supportcase40, 30 70 45 05 100, 30 70 45 095 100, acc-tight2, acc-tight4, acc-tight5, comp07-2idx, comp08-2idx, comp122idx, comp21-2idx, decomp1, decomp2, gus-sch, istanbul-no-cutoff, mkc, mkc1, neos-555343, neos-555424, neos-738098, neos-872648, neos-933562, neos-933638, neos-933966, neos935234, neos-935769, neos-983171, neos-1330346, neos-1337307, neos-1396125, neos-3209462rhin, neos-3755335-nizao, neos-3759587-noosa, neos-4300652-rahue, neos18, physiciansched6-1, physiciansched6-2, piperout-d27, qiu, reblock354, and supportcase37.do b do 5: D 6: Take action k a k at state s policy π 7:
Revised: Moreover, MIPLIB mixed supportcase contains 40 instances: supportcase40, 30 70 45 05 100, 30 70 45 095 100, acc-tight2, acc-tight4, acc-tight5, comp07-2idx, comp08-2idx, comp122idx, comp21-2idx, decomp1, decomp2, gus-sch, istanbul-no-cutoff, mkc, mkc1, neos-555343, neos-555424, neos-738098, neos-872648, neos-933562, neos-933638, neos-933966, neos935234, neos-935769, neos-983171, neos-1330346, neos-1337307, neos-1396125, neos-3209462rhin, neos-3755335-nizao, neos-3759587-noosa, neos-4300652-rahue, neos18, physiciansched6-1, physiciansched6-2, piperout-d27, qiu, reblock354, and supportcase37.

Original: Receive reward r and add ( s 0 , k, a k , r ) to D train 8: end for 9: Compute hierarchical policy gradient using D train as in proposition10: Update the parameters, θ 1 = θ 1 + α ∇ θ 1 J ([ θ 1 , θ 2 ]) , θ 2 = θ 2 + α ∇ θ 2 J ([ θ 1 , θ 2 ]) 11: end for (2) Benchmarks used in NeurIPS 2021 ML4CO competition The Load Balancing and Anonymous problems used in the main text are from the NeurIPS 2021 ML4CO competition (Bowly et al., 2021).
Revised: (2) Benchmarks used in NeurIPS 2021 ML4CO competition The Load Balancing and Anonymous problems used in the main text are from the NeurIPS 2021 ML4CO competition (Bowly et al., 2021).

Original: Readers can refer to https://www.ecole.ai/2021/ml4co-competition/ for details of the competition.
Revised: Readers can refer to https://www.ecole.ai/2021/ml4co-competition/ for details of the competition.

Original: The competition releases three challenging datasets, but we only use two of the three datasets.
Revised: The competition releases three challenging datasets, but we only use two of the three datasets.

Original: The major reason is that the average number of the candidate cuts on  instances from the third dataset (Item Placement) is less than five, which makes cut selection has little impact on the overall solver performance.
Revised: The major reason is that the average number of the candidate cuts on the instances from the third dataset (Item Placement) is less than five, which makes cut selection has little impact on the overall solver performance.

Original: D. D ATASETS USED IN S ECTION : M OTIVATING RESULTS
Revised: D.1. D ETAILED DESCRIPTION OF THE AFOREMENTIONED DATASETS

Revised: In this part, we provide detailed description of the aforementioned datasets. Note that all datasets we use except MIPLIB 2017 are application-specific, i.e., they contain instances from only a single application. We summarize the statistical description of the used datasets in this paper in Table 6.

Revised: Let n, m denote the average number of variables and constraints in the MILPs.

Revised: Let m × n denote the size of the MILPs.

Revised: We emphasize that the largest size of our used datasets is up to two orders of magnitude larger than that used in previous learning-based cut selection methods (Tang et al., 2020;

Revised: Paulus et al., 2022), which demonstrates the superiority of our proposed HEM.

Revised: Moreover, we test the inference time of our proposed HEM given the average number of candidate cuts.

Revised: The results in

Revised: Table 6 show that the computational overhead of the HEM is very low.

Revised: D. D ATASETS USED IN S ECTION 3 IN THE MAIN TEXT

Original: (1) In Figure 1a in the main text, we use five  datasets, namely D1, D2, D3, D4, and D5, respectively.
Revised: In Figure 1a in the main text, we use five challenging datasets, namely D1, D2, D3, D4, and D5, respectively.

Original: Specifically, D1 represents MIPLIB mixed supportcase, D2 represents the single instance neos1456979 from MIPLIB 2017, D3 represents MIPLIB mixed neos, D4 represents Anonymous, and
Revised: Specifically, D1 represents MIPLIB mixed supportcase, D2 represents the single instance 𝛼 1 𝑇 𝑥 ≤ 𝛽𝑐𝑢𝑡 𝛼 2𝑇 𝑥 ≤ 𝛽𝛼 3𝑇 𝑥 ≤ 𝛽 [Figure] 𝑐𝑢𝑡 𝑠𝑐𝑜𝑟𝑒1:0.8 𝑠𝑐𝑜𝑟𝑒2:0.78 𝑠𝑐𝑜𝑟𝑒3:0.

Original: D5 represents the single instance lectsched-5-obj from MIPLIB 2017.
Revised: 𝛼 1 𝑇 𝑥 ≤ 𝛽𝑐𝑢𝑡 𝛼 2𝑇 𝑥 ≤ 𝛽𝑐𝑢𝑡 𝛼 1 𝑇 𝑥 ≤ 𝛽𝑐𝑢𝑡 𝛼 2 𝑇 𝑥 ≤ 𝛽𝛼 3 𝑇 𝑥 ≤ 𝛽𝑐𝑢𝑡 𝑐𝑢𝑡 𝛼 1𝑇 𝑥 ≤ 𝛽 [Figure] 3𝑇 𝑥 ≤ 𝛽 [Figure]

Revised: 𝑐𝑢𝑡 𝑒𝑛𝑑

Revised: Sequence model based policy

Revised: Score-based policy

Revised: [Figure]

Revised: neos-1456979 from MIPLIB 2017, D3 represents MIPLIB mixed neos, D4 represents Anonymous, and D5 represents the single instance lectsched-5-obj from MIPLIB 2017.

Original: In Figure 1b in the main text, we use the dataset MIPLIB mixed neos.
Revised: In Figure 1b in the main text, we use the dataset MIPLIB mixed neos.

Original: D. R EAL - WORLD SUPPLY CHAIN PROBLEMS
Revised: D. L ARGE - SCALE REAL - WORLD PRODUCTION PLANNING PROBLEMS

Original: The supply chain problem aims to find the optimal production planning for hundreds of factories according to the daily order demand.
Revised: The production planning problem aims to find the optimal production planning for thousands of factories according to the daily order demand.

Original: The constraints include the production capacity for each production line in each factory, transportation limit, the order rate, etc.
Revised: The constraints include the production capacity for each production line in each factory, transportation limit, the order rate, etc.

Original: The optimization objective is to minimize the production cost and lead time simultaneously.
Revised: The optimization objective is to minimize the production cost and lead time simultaneously.

Original: We split the dataset into training and test sets with 80% and 20% of the instances.
Revised: We split the dataset into training and test sets with 80% and 20% of the instances.

Revised: The average size of the production planning problems is approximately equal to 3500 ×

Revised: 5000 = 1 .

Revised: 75 × 10 8 , which are large-scale real-world problems.

Original: To promote thecommunity of machine learning  for MILP, we will release the dataset once the paper is accepted to be published.
Revised: To promote the machine learning community for MILP, we will release the dataset once the paper is accepted to be published.

Revised: E I LLUSTRATION OF A DVANTAGES OF U SING A S EQUENCE M ODEL

Revised: Figure 4 illustrate two major advantages of using the sequence model to learn cut selection.

Revised: First, the sequence model takes into account the order of selected cuts by modeling the selected cuts as an output sequence.

Revised: As shown in Figure 4, the order of cuts determined by the sequence model is better than the score-based method, thus leading to a better initial basis for solving the LP relaxation faster.

Revised: Second, the sequence model captures the interaction among cuts, as it models the joint conditional probability of the selected cuts given an input sequence of the candidate cuts.

Original: E A LGORITHM IMPLEMENTATION AND EXPERIMENTAL SETTINGS E. D ESIGNED CUT FEATURES 
Revised: As shown in Figure 4, the sequence model selects cuts that complement each other nicely, thus leading to a more tightened LP relaxation and speeding up solving the MILP. F A LGORITHM I MPLEMENTATION AND E XPERIMENTAL S ETTINGS F. D ESIGNED CUT FEATURES (Suppose c denotes the objective coefficient.)

Original: [Table]
Revised: [Table]

Original: Following (Huang et al., 2022; Wesselmann &
Revised: Following Huang et al.

Original: Stuhl, 2012; Dey & Molinaro, 2018b; Achterberg, 2007), we design 13 cut features for the cut selection task, such as the extent to which a cut is violated by the current LP solution and the proportion of non-zero coefficients of a cut.
Revised: ; Wesselmann & Stuhl (2012); Dey & Molinaro (2018b); Achterberg (2007), we design thirteen cut features for the cut selection task, such as the extent to which a cut is violated by the current LP solution and the proportion of non-zero coefficients of a cut.

Original: We present a detailed description of the designed cut features in Table 6.
Revised: We present a detailed description of the designed cut features in Table 7.

Original: We emphasize that we do not tune the cut features.
Revised: We emphasize that we do not tune the cut features.

Original: Therefore, it is promising to further improve ourproposed method by design better cut features or using graph neural network to learn better features inthe future work.
Revised: Therefore, it is promising to further improve our method by designing better cut features or using graph neural networks to learn better features in future work.

Original: E. I MPLEMENTATION DETAILS OF  BASELINES
Revised: F. I MPLEMENTATION DETAILS OF THE BASELINES

Original: In this part, we present a detailed description of all  baselines used in this paper.
Revised: In this part, we present a detailed description of all the baselines used in this paper.

Original: We denote a cut by α T x ≤ β and the  current LP solution by x ∗ .
Revised: We denote a cut by α T x ≤ β and the optimal solution of the current LP relaxation by x ∗ .

Original: Throughout all experiments, we set the ratio of selected cuts  0 .
Revised: Throughout all experiments, we set the ratio of selected cuts as 0 .

Original: 2 for all score-based rules and learning baselines.
Revised: 2 for all score-based rules and learning baselines.

Original: Random Random selects a fixed ratio of  candidate cuts stochastically.
Revised: Random. Random selects a fixed ratio of the candidate cuts stochastically.

Revised: The ratio is set as 0 .

Revised: 2 in this paper.

Original: Normalized Violation (NV) NV is a score-based rule.
Revised: Normalized Violation (NV). NV is a score-based rule.

Original: It score each cut based on the normalized violation of the cut to the current LP solution, and select a  ratio of0 . 2 cuts with high scores.
Revised: It scores each cut based on the normalized violation of the cut to the current LP solution, and selects a fixed ratio of cuts with high scores.

Original: The normalized violation is defined by max { 0 , α T x ∗ LP − β
Revised: The normalized violation is defined by max { 0 , α T x ∗ LP − β

Original: | β | } .
Revised: | β | } .

Revised: The ratio is set as 0 .

Revised: 2 in this paper.

Original: Efficacy (Eff)
Revised: Efficacy (Eff).

Original: Eff is a score-based rule.
Revised: Eff is a score-based rule.

Original: It score each cut based on the Euclidean distance of the cut hyperplane to the current LP solution, and select a  ratio of  0 .
Revised: It scores each cut based on the Euclidean distance of the cut hyperplane to the current LP solution, and selects a fixed ratio of cuts with high scores. The ratio is set as 0 .

Original: 2 cuts with high scores.
Revised: 2 in this paper.

Original: Default Default is the default cut selection rule used in SCIP 8.0.0 (Bestuzheva et al., 2021).
Revised: Default. Default is the default cut selection rule used in SCIP 8.0.0 (Bestuzheva et al., 2021).

Original: Please refer to (Achterberg, 2007) for a detailed description of the SCIP’s default cut selection rule.
Revised: Please refer to (Achterberg, 2007) for a detailed description of the SCIP’s default cut selection rule.

Original: Noet that Default tackles the two problems: (1) which cuts should be preferred, and (2) how many cuts should be selected, in cut selection by human-designed heuristics. 
Revised: Note that Default tackles the two problems: (1) which cuts should be preferred, and (2) how many cuts should be selected, in cut selection by human-designed heuristics. That is, Default selects variable ratios of cuts rather than a fixed ratio.

Original: Score-based policy (SBP)
Revised: Score-based policy (SBP).

Original: Since the state-of-the-art (SOTA) reinforcement learning based methods for cut selection (Tang et al., 2020) is deigned for the setting that selects a best cut in each round, we implement a  variant of the SOTA to adapt to our setting that selects a subset of cuts in each round, namely SBP.
Revised: Since the state-of-the-art (SOTA) reinforcement learning based method for cut selection (Tang et al., 2020) is designed for the setting that selects the best cut in each round, we implement a slight variant of the SOTA to adapt to our setting that selects a subset of cuts in each round, namely SBP.

Original: Specifically, the core idea of SBP is learning a scoring function to measure cut quality as well.
Revised: Specifically, the core idea of SBP is learning a scoring function to measure cut quality as Tang et al. ; Huang et al. ; Paulus et al.

Original:  For a fair comparison, SBP uses the same cut features as HEM and we train SBP via reinforcement learning as well.
Revised: (2022) do. For a fair comparison, SBP uses the same cut features as HEM and we train SBP via reinforcement learning as well.

Revised: Our implemented SBP is also a slight variant of the method proposed in Huang et al.

Revised: We emphasize that experiments in the main text show that our implemented SBP is a strong baseline.

Original: We implement the scoring function with a multi-layer perceptron that predicts the score of a given cut.
Revised: Specifically, we implement the scoring function with a multi-layer perceptron that predicts the score of a given cut.

Original: That is, the scoring function predicts a cut’s score based on the features of the cut.
Revised: That is, the scoring function predicts a cut’s score based on the features of the cut.

Revised: The MLP network contains two hidden layers with 128 units.

Original: Moreover, we train the scoring function via evolutionary strategies as (Tang et al., 2020) 
Revised: Moreover, we train the scoring function via evolutionary strategies as (Tang et al., 2020) does.

Revised: We will also release the code for implementing SBP once the paper is accepted to be published.

Revised: do do 5:

Revised: D 6:

Revised: Take action a state s policy π 7:

Revised: Receive reward add a to D train 8: end for 9:

Original: . E. I MPLEMENTATION DETAILS AND HYPERPARAMETERS OF HEM E.3. R EWARD FUNCTION
Revised: Compute policy D train Update the parameters, θ α 11: end for

Revised: F

Revised:  I MPLEMENTATION DETAILS AND HYPERPARAMETERS F.3. H ARDWARE SPECIFICATION

Revised: Throughout all experiments, we use a single machine that contains eight GPU devices (NVidia

Revised: GeForce GTX 3090 Ti) and two Intel Gold 6246R CPUs.

Revised: F.3. S OLVER SETUP

Revised: For reproducibility, we emphasize that all results in the main text are obtained by averaging results over the SCIP random seeds { 1 , 2 , 3 } .

Revised: F.3. R EWARD FUNCTION

Original: For easy datasets, we set the reward as the negative solving time.
Revised: [Table] On the easy datasets, we set the reward as the negative solving time.

Original: For medium and hard datasets, we set the reward as the negative primal-dual gap integral within a time limit  300 seconds.
Revised: On the medium and hard datasets, we set the reward as the negative primaldual gap integral within a time limit of 300 seconds.

Original: For the real-world production planning problems, we set the reward as the negative primal-dual gap integral within a time limit  600 seconds or the negative dual bound improvement.
Revised: For the real-world production planning problems, we set the reward as the negative primal-dual gap integral within a time limit of 600 seconds or the negative dual bound improvement.

Original: The resultsof HEM reported in the main text are based on the negative dual bound improvement reward.
Revised: The results reported in the main text are achieved by HEM with the negative dual bound improvement reward.

Original: We provide the results of HEM that uses the negative primal-dual gap integral in Table xx.
Revised: We provide the performance of HEM that uses the negative primal-dual gap integral in Table 8.

Revised: The results still show that HEM significantly outperforms all the baselines in terms of the Time and PD integral.

Original: We emphasize that we can set the reward according to our objective in real-world problems.
Revised: We emphasize that we can set the reward according to our objective in real-world problems.

Original: For example, suppose we aim to minimize the primal-dual gap within a time limit, then we can set the reward as the primal-dual gap within the time limit.
Revised: For example, suppose we aim to minimize the primal-dual gap within a time limit, then we can set the reward as the primal-dual gap within the time limit.

Original: E.3. P OLICY NETWORK ARCHITECTURE
Revised: F.3. P OLICY NETWORK ARCHITECTURE

Original: The higher-level model contains a LSTM encoder and a MLP.
Revised: The higher-level model contains an LSTM encoder and an MLP.

Original: The LSTM network encodes variable sized inputs into hidden vector with dimension 128.
Revised: The LSTM network encodes variable-sized inputs into hidden vectors with dimension 128.

Original: The MLP network cantains two hidden layers with 128 units.
Revised: The MLP network contains two hidden layers with 128 units.

Original: The lower-level model is essentially a pointer network.
Revised: The lower-level model is essentially a pointer network.

Original: We keep the hyperparameters of the pointer network as that used in (Bello* et al., 2017).
Revised: We keep the hyperparameters of the pointer network as that used in (Bello* et al., 2017).

Original: E.3. O PTIMIZATION
Revised: F.3. O PTIMIZATION

Original: Throughout all experiments, we apply Adam optimizer with learning rate α 1 = 1 e − 4 to optimize the lower-level model, and learning rate α 2 = 5 e − 4 to optimize the higher-level model.
Revised: Throughout all experiments, we apply Adam optimizer with learning rate α 1 = 1 × 10 − 4 to optimize the lower-level model, and learning rate α 2 = 5 × 10 − 4 to optimize the higher-level model.

Original: For each epoch, we collect 32 samples for training, and we set the total epochs  100.
Revised: For each epoch, we collect 32 samples for training, and we set the total epochs as 100.

Original: It is interesting that learning a good cut selection strategy does not need too much data as shown in (Tang et al., 2020).
Revised: It is surprising that learning a good cut selection policy does not need too much data as shown in Tang et al.

Original: For training stability, we delay the higher-level policy update.
Revised: For training stability, we delay the higher-level policy update.

Original: This creares a two-timescale algorithm, as often required for convergence in the linear setting (Fujimoto et al.,2018; Konda & Take Compute Update α ∇
Revised: This creates a two-timescale algorithm, as often required for convergence in the linear setting (Fujimoto et al.,

Original: J 11: end for Tsitsiklis, 2003).
Revised: Konda & Tsitsiklis, 2003).

Original: We set the delay  freq 2.
Revised: We set the delay update freq as two.

Original: That is, we first train the lower-level policy twice, then train the higher-level policy once.
Revised: That is, we first train the lower-level policy twice, then train the higher-level policy once.

Revised: Additionally, the results in Appendix G.3.4 show that the convergence performance of HEM is insensitive to the hyperparameter delay update freq.

Revised: F.3. T HE TRAINING ALGORITHM

Revised: We provide the procedure of the training algorithm of HEM in Algorithm 2.

Original: E. D ETAILS OF THE POINTER NETWORK
Revised: F. M ORE DETAILS OF HEM F.4. D ETAILS OF THE POINTER NETWORK

Original: The pointer network is first introduced by (Vinyals et al., 2015) for directly finding solutions of specific combinatorial optimization problems, such as  Travelling Salesman Problems. 
Revised: The pointer network is first introduced by (Vinyals et al., 2015) for directly finding solutions of specific combinatorial optimization problems, such as the Travelling Salesman Problems. The pointer network architecture is illustrated in Figure 5.

Original: The pointer network consists of a Long Short-Term Memory encoder, a Long Short-Term Memory decoder, and an attention that is used as a pointer to select a member of the input sequence as the output (Vinyals et al., 2015).
Revised: The pointer network consists of a Long Short-Term Memory encoder, a Long Short-Term Memory decoder, and an attention that is used as a pointer to select a member of the input sequence as the output (Vinyals et al., 2015).

Original: Specifically, we implement the pointer network following (Bello* et al., 2017).
Revised: Specifically, we implement the pointer network following Bello* et al.

Original: Please refer to (Bello* et al., 2017) for implementation details of the pointer network.
Revised: Please refer to (Bello* et al., 2017) for implementation details of the pointer network.

Original: The major difference between our used pointer network and the pointer network used in (Bello* et al., 2017) is that we use the pointer network to select ordered subsets of input sequences, but (Bello* et al., 2017) use the pointer network to output permutations of input sequences.
Revised: The major difference between our used pointer network and the pointer network used in (Bello* et al., 2017) is that we use the pointer network to select ordered subsets of input sequences, but (Bello* et al., 2017) use the pointer network to output permutations of input sequences.

Original: E. D IFFERENCE BETWEEN HEM- RATIO AND HEM W / O H
Revised: F.4. D IFFERENCE BETWEEN HEM- RATIO AND HEM W / O H

Revised: Details of HEM w/o H To implement HEM w/o H, we augment each input sequence with an end token, i.e., a thirteen-dimensional tensor with values all being one.

Revised: The end token is at the end position of the input sequence.

Revised: Once the decoder of HEM w/o H outputs the end token, then the decoding ends.

Revised: That is, HEM w/o H can implicitly predict the number of cuts that should be selected by predicting whether to decode the end token at the current decoding step.

Original: The policy network of HEM-ratio and HEM w/o H are both essentially a pointer network (Vinyals et al., 2015), a variant of the sequence model.
Revised: The policy network of HEM-ratio and HEM w/o H are both essentially a pointer network (Vinyals et al., 2015), a variant of the sequence model.

Original: We present the major difference between HEM-ratio  and HEM w/o H in the following.
Revised: We present the major difference between HEM-ratio [Table] ) and HEM w/o H in the following.

Original: HEM w/o H predicts an end token as used in language tasks (Sutskever et al., 2014; Vaswani et al., 2017) toimplicitly determine the number of cuts that should be selected.
Revised: HEM w/o H predicts an end token as used in language tasks (Sutskever et al., 2014; Vaswani et al., 2017) to determine the number of cuts that should be selected implicitly.

Original: In contrast, HEM-ratio always selects a fixed ratio of cuts, i.e, it always ends decoding at a pre-determined position.
Revised: In contrast, HEM-ratio always selects a fixed ratio of cuts, i.e, it always ends decoding at a pre-determined position.

Original: Therefore, HEM w/o H can learn how many cuts  should be selected, but HEM-ratio cannot.
Revised: Therefore, HEM w/o H can learn the number of cuts that should be selected, but HEM-ratio cannot.

Original: Details of HEM w/o H To implement HEM w/o H, we augment each input sequence with an end token, i.e., a thirteen dimensional tensor with values all being one.

Original: The end token is at the end position of the input sequence. Once the decoder of HEM w/o H outputs the end token, then the decoding ends. That is, HEMw/o H can implicitly predicting the number of cuts that should be selected by predicting whether to decode the end token at current decoding step. E. T HE TRAINING ALGORITHM
Revised: F.4. M ORE DISCUSSION OF HEM

Original: We provide the procedure of the training algorithm of HEM in Table 2.

Original: E. D ISCUSSION OF HEM

Original: In this part, we provide details of some additional advantages of HEM.
Revised: In this part, we provide details of some more advantages of HEM.

Original: (1) Inspired by hierarchical reinforcement learning (Sutton et al., 1999b; Nachum et al., 2018), HEM leverages the hierarchical structure of the cut selection task, which is important for efficient exploration in complex decisionmaking tasks.
Revised: (1) Inspired by hierarchical reinforcement learning (Sutton et al., 1999b; Nachum et al., 2018), HEM leverages the hierarchical structure of the cut selection task, which is important for efficient exploration in complex decisionmaking tasks.

Original: (2) Previous methods (Tang et al., 2020; Huang et al., 2022) usually train cut selection policies via black-box optimization methods such as evolution strategies (Salimans et al., 2017).
Revised: (2) Previous methods (Tang et al., 2020; Huang et al., 2022) usually train cut selection policies via black-box optimization methods such as evolution strategies (Salimans et al., 2017).

Original: In contrast, HEM is differentiable and we train the HEM via gradient-based algorithms, which is more sample efficient than black-box optimization methods (Sutton & Barto, 2018; Schulman et al., 2015).[Table]
Revised: In contrast, HEM is differentiable and we train the HEM via gradient-based algorithms, which is more sample efficient than black-box optimization methods (Sutton & Barto, 2018; Schulman et al., 2015).

Original: Although we can offline generate training samples as much as possible using a MILP solver, high sample efficiency is significant as generating samples can be extremely time-consuming in practice.
Revised: Although we can offline generate training samples as much as possible using a MILP solver, high sample efficiency is significant as generating samples can be extremely time-consuming in practice.

Original: F A DDITIONAL R ESULTS F. M ORE MOTIVATING RESULTS
Revised: G M ORE R ESULTS G. M ORE MOTIVATING RESULTS

Original: Ratio matters.
Revised: Ratio matters.

Original: To evaluate the effect of the ratio of selected cuts on solving MILPs, we focus on the
Revised: To evaluate the effect of the ratio of selected cuts on solving MILPs, we focus on the

Original: Normalized Violation (NV) cut selection method with different ratios of selected cuts.
Revised: Normalized Violation (NV) cut selection method with different ratios of selected cuts.

Original: (1) We first evaluate the NV methods that select 5% , 10% , 20% , 30% , 40% , 50% , 60% , 70% , and 80% of candidate cuts, respectively, on four datasets.
Revised: (1) We first evaluate the NV methods that select 5% , 10% , 20% , 30% , 40% , 50% , 60% , 70% , and 80% of candidate cuts, respectively, on four datasets.

Original: The results in Fig. 4a show that the NV achieves better solver performance with larger ratios on CORLAT and MIPLIB mixed neos.
Revised: The results in Figure 6a show that the NV achieves better solver performance with larger ratios on CORLAT and MIPLIB mixed neos.

Original: The results demonstrate that the ratio that leads to better solver performance is variable over different datasets, which implies that learning dataset-dependent ratios is important.
Revised: The results demonstrate that the ratio that leads to better solver performance is variable over different datasets, which implies that learning dataset-dependent ratios is important.

Original: (2) We then evaluate the NV methods that select 5% , 10% , 20% , 30% , 40% , 50% , 60% , 70% , and 80% of candidate cuts, respectively, on four instances from the Anonymous dataset.
Revised: (2) We then evaluate the NV methods that select 5% , 10% , 20% , 30% , 40% , 50% , 60% , 70% , and 80% of candidate cuts, respectively, on four instances from the Anonymous dataset.

Original: The results in Fig.
Revised: The results in Figure 6b show that NV achieves better [Figure] li z [Figure]

Revised: p r i m a l ­

Original: 4b show that NV achieves better solver performance with larger ratios on Anonymous 121 and Anonymous 131.
Revised: [Figure] i n [Figure] solver performance with larger ratios on Anonymous 121 and Anonymous 131.

Original: The results demonstrate that the ratio that leads to better solver performance is variable over different instances from the same dataset, which implies that learning instance-dependent ratios is significant as well.
Revised: The results demonstrate that the ratio that leads to better solver performance is variable over different instances from the same dataset, which implies that learning instance-dependent ratios is important as well.

Original: F. M ORE RESULTS OF MAIN EVALUATION
Revised: G. M ORE RESULTS OF MAIN EVALUATION

Original: [Figure]

Original: The In this section, we provide additional results of the main evaluation.
Revised: In this section, we provide more results of the main evaluation.

Original: The results in Table 7 show the performance of HEM and  baselines in terms of the total number of nodes  and the primal-dual gap.
Revised: The results in Table 9 show the performance of HEM and the baselines in terms of the total number of nodes (Nodes) and the primal-dual gap (PD gap).

Original:  On  easy datasets, HEM and most baselines find the optimal solution within the time limit as the primal-dual gap converges to zero.
Revised: (1) Easy datasets. On the easy datasets, HEM and most baselines find the optimal solution within the time limit, as the PD gap converges to zero.

Revised: Additionally,

Original: In terms of the primal-dual gap, HEM  outperforms all the 
Revised: HEM significantly outperforms all the baselines in terms of the Nodes on the easy datasets.

Original: [Table]
Revised: Medium and hard datasets. In terms of the PD gap, HEM outperforms all the baselines on medium and hard datasets, especially on CORLAT. However, the Nodes metric cannot well distinguish the performance of different cut selection methods on the medium and hard datasets for the following two reasons. First, the solving time of the LP relaxation on each node is different, and thus the

Original: baselines on medium and hard datasets, especially on CORLAT.
Revised: Nodes cannot directly determine the solving time (Huang et al., 2022). Second, on those unsolved instances within the time limit, the Nodes metric is not a proper metric, as the Nodes cannot evaluate the quality of the solving process.

Original: F. M ORE RESULTS OF ABLATION STUDY
Revised: G. M ORE RESULTS OF ABLATION STUDY

Original: In this section [Table] we provide addition results of ablation study.
Revised: In this section, we provide more results of ablation studies in the main text

Original: F.3. A NALYSIS OFCOMPARSION BETWEEN HEM- RATIO AND SBP
Revised: . G.3. I N - DEPTH ANALYSIS OF HEM- RATIO AND SBP

Revised: We provide possible reasons for HEM-ratio performing poorer than SBP on several challenging

Revised: MILP problem benchmarks.

Revised: Fundamentally, HEM-ratio formulates the cut selection task as a sequence modeling problem, which has two main advantages over SBP.

Revised: That is, the sequence model can not only capture the underlying order information, but also capture the interaction among cuts.

Revised: However, training a sequence model is more difficult than training a scoring function, as the sequence model aims to learn a much more complex task.

Original: A possible reason is that it is  more difficult to train a sequence model than training a multi-layer perceptron and thus the sequence model may suffer from inefficient exploration and be trapped to the local optimum
Revised: Specifically, the scoring function aims to learn to score each cut, while the sequence model aims to model the joint probability of the selected cuts. The latter is a more challenging learning task. Moreover, we follow the reinforcement learning paradigm instead of supervised learning to train the model, making the training process more unstable. Therefore, the sequence model may suffer from inefficient exploration and be trapped to a local optimum.

Revised: As a result, HEM-ratio can perform poorer than SBP, especially on challenging

Revised: MILP problem benchmarks.

Original: . F.3. C ONTRIBUTION OF EACH COMPONENT
Revised: G.3. C ONTRIBUTION OF EACH COMPONENT

Original: To understand the contribution of each component of HEM, we provide addition results of HEM and
Revised: To understand the contribution of each component of HEM, we provide more results of HEM and

Original: HEM without the higher-level model on Set Covering, Multiple Knapsack, MIK, Load Balancing,
Revised: HEM without the higher-level model on Set Covering, Multiple Knapsack, MIK, Load Balancing,

Original: Anonymous [Table] and MIPLIB mixed supportcase.
Revised: Anonymous, and MIPLIB mixed supportcase.

Original: The results in Table 8 show that HEM outperforms 
Revised: The results in Table 10 show that HEM outperforms [Table]

Original:  HEM w/o H in terms of the solving time, the primal-dual gap, and the primal-dual gap integral on several challenging datasets, demonstrating the importance of our proposed higher-level model.
Revised: [Table] HEM w/o H in terms of the solving time, the primal-dual gap, and the primal-dual gap integral on several challenging datasets, demonstrating the importance of our proposed higher-level model.

Original: Moreover, HEM w/o H significantly outperforms SBP in terms of the solving time, the primal-dual gap, and the primal-dual gap integral on several challenging datasets, demonstrating the significance of our proposed lower-level model.
Revised: Moreover, HEM w/o H significantly outperforms SBP in terms of the solving time, the primal-dual gap, and the primal-dual gap integral on several challenging datasets, demonstrating the significance of our proposed lower-level model.

Original: F.3. T HE IMPORTANCE OF TACKLING P1-P3 IN CUT SELECTION
Revised: G.3. T HE IMPORTANCE OF TACKLING P1-P3 IN CUT SELECTION

Original: To understand the importance of tackling P1-P3 in cut selection, we provide additional results of 
Revised: To understand the importance of tackling P1-P3 in cut selection, we provide more results of HEM,

Original: HEM, HEM-ratio, and HEM-ratio-order on Set Covering, Multiple Knapsack, MIK, Load Balancing, Anonymous, and MIPLIB mixed supportcase.
Revised: HEM-ratio, and HEM-ratio-order on Set Covering, Multiple Knapsack, MIK, Load Balancing, Anonymous, and MIPLIB mixed supportcase.

Original: Here we refresh what HEM, HEM-ratio, HEMratio-order mean.
Revised: Here we refresh what HEM, HEM-ratio, HEM-ratioorder mean.

Original: (1) HEM .
Revised: (1) HEM .

Original: HEM tackles P1-P3 in cut selection simultaneously.
Revised: HEM tackles P1-P3 in cut selection simultaneously.

Original: (2) HEM-ratio .
Revised: (2) HEM-ratio .

Original: In order not to learn how many cuts should be selected, we remove the higher-level model of HEM and force the lower-level model to select a fixed ratio of cuts.
Revised: In order not to learn how many cuts should be selected, we remove the higher-level model of HEM and force the lower-level model to select a fixed ratio of cuts.

Original: We denote it by HEM-ratio.
Revised: We denote it by HEM-ratio.

Original: Note that HEM-ratio is different from HEM w/o H (see Appendix E).
Revised: Note that HEM-ratio is different from HEM w/o H (see Appendix F).

Original: HEM-ratio tackles P1 and P3 in cut selection.
Revised: HEM-ratio tackles P1 and P3 in cut selection.

Original: (3) HEM-ratio-order.
Revised: (3) HEM-ratio-order.

Original: To further mute the effect of the order of selected cuts, we reorder the selected cuts given by HEM-ratio with the original index of the generated cuts, which we denote by HEM-ratio-order.
Revised: To further mute the effect of the order of selected cuts, we reorder the selected cuts given by HEM-ratio with the original index of the generated cuts, which we denote by HEM-ratio-order.

Original: HEM-ratio-order mainly tackles P1 .
Revised: HEM-ratio-order mainly tackles P1 .

Original: The results in Table 9 suggest the following.
Revised: The results in Table 11 suggest the following.

Original: HEM-ratio-order outperforms Default and NoCuts on several datasets, demonstrating that tackling P1 by data-driven methods is crucial.
Revised: HEM-ratio-order outperforms Default and NoCuts on several datasets, demonstrating that tackling P1 by data-driven methods is crucial.

Original: HEM signifi-
Revised: HEM signif-

Original: [Table] [Table]
Revised: [Table] [Table]

Original: cantly outperforms HEM-ratio in terms of the primal-dual gap integral, demonstrating the significance of tackling P2 .
Revised: icantly outperforms HEM-ratio in terms of the primal-dual gap integral, demonstrating the significance of tackling P2 .

Original: HEM-ratio outperforms HEM-ratio-order on several datasets, which demonstrates the importance of tackling P3 .
Revised: HEM-ratio outperforms HEM-ratio-order on several datasets, which demonstrates the importance of tackling P3 .

Original: Moreover, HEM-ratio performs better than SBP in terms of the solving time and the primal-dual gap integral on all six datasets except Set Covering and MIPLIB mixed supportcase, which shows the superiority of formulating the cut selection as a seuqence to sequence learning problem over formulating it as a scoring task.
Revised: Moreover, HEM-ratio performs better than SBP in terms of the solving time and the primal-dual gap integral on all six datasets except Set Covering and MIPLIB mixed supportcase, which shows the superiority of formulating the cut selection as a sequence to sequence learning problem over formulating it as a scoring task.

Original: However, HEM-ratio and HEMratio-order perform a little worse than SBP on Set Covering and MIPLIB mixed supportcase.
Revised: However, HEM-ratio and HEMratio-order perform a little worse than SBP on Set Covering and MIPLIB mixed supportcase.

Original: A possible reason is that it is more difficult to train a sequence model than training a multi-layer perceptron and thus the sequence model may suffer from inefficient exploration and be trapped to the local optimum.
Revised: A possible reason is that it is more difficult to train a sequence model than to train a multi-layer perceptron and thus the sequence model may suffer from inefficient exploration and be trapped to the local optimum.

Original: Please refer to Appendix F.3.1 for a detailed analysis. 
Revised: Please refer to Appendix G.3.1 for a detailed analysis. G.

Revised: . S ENSITIVITY ANALYSIS

Revised: Additionally, we perform ablation studies to test the sensitivity of HEM to the hyperparameter delay update freq d .

Revised: The results in Table 12 show that there is a wide range of d for HEM to achieve comparable performance on Maximum Independent Set, CORLAT, and MIPLIB mixed neos.

Revised: Moreover, we emphasize that we do not tune the hyperparameter d .

Original: F. M OREVISUALIZATION RESULTS 
Revised: As the results shown in Table 12, d = 3 and d = 4 performs the best on CORLAT and MIPLIB mixed neos, respectively. However, we simply set d = 2 throughout all experiments in the main text. G. M ORE RESULTS OF GENERALIZATION Here we provide more results of the generalization experiments on Set Covering.

Revised: On Set Covering, we test HEM on two times and four times larger instances than those seen during training.

Revised: The results in Table 13 show that HEM generalizes well to instances that are significantly larger than seen during training.

Revised: In particular, HEM achieves at least 70% improvement in terms of the Time compared to all the rule-based baselines.

Revised: Moreover, SBP also generalizes well to large instances, demonstrating that SBP is a strong baseline.

Revised: G. M ORE VISUALIZATION RESULTS

Original: In this part, we provide additional visualization results on Set Covering and MIK.
Revised: In this part, we provide more visualization results on Set Covering and MIK.

Original: On MIK, we visualize the cuts selected by HEM-ratio and SBP on a randomly sampled instance.
Revised: On MIK, we visualize the cuts selected by HEM-ratio and SBP on a randomly sampled instance.

Original: We perform principal component analysis (Mohri et al., 2018) on selected cuts to reduce the cut features to two-dimensional space.
Revised: We perform principal component analysis (Mohri et al., 2018) on selected cuts to reduce the cut features to two-dimensional space.

Original: Colored points illustrate reduced cut features.
Revised: Colored points illustrate reduced cut features.

Original: To visualize the diversity of selected cuts, we use dashed lines to connect the points with the smallest and largest x,y coordinates.
Revised: To visualize the diversity of selected cuts, we use dashed lines to connect the points with the smallest and largest x,y coordinates.

Original: [Figure] The results in Figure 6 still show that HEMratio selects much more diverse cuts than SBP on MIK.
Revised: The results in Figure 7 still show that HEM-ratio selects much more diverse cuts than SBP on MIK.

Original: However, HEM-ratio performs poorer than SBP on Set Covering (see Appendix F. for a detailed analysis).
Revised: However, HEM-ratio performs poorer than SBP on Set Covering (see Appendix G.3.1 for a detailed analysis).

Original: Therefore, we visualize the cuts selected by HEM and SBP on a randomly sampled instance from Set Covering.
Revised: Therefore, we visualize the cuts selected by HEM and SBP on a randomly sampled instance from Set Covering.

Original: Although HEM learns the number of cuts that should be selected, we find that HEM selects much fewernumber of cuts than SBP.
Revised: Although HEM learns the number of cuts that should be selected, we find that HEM selects much fewer cuts than SBP.

Original: Specifically, HEM selects 25 cuts, while SBP selects 158 cuts.
Revised: Specifically, HEM selects 25 cuts, while SBP selects 158 cuts.

Original: Interestingly, the results in Figure 
Revised: Interestingly, the results in Figure 7 show that SBP selects 158 similar cuts with high

Original: [Figure]
Revised: [Figure]

Original: t r a i n i n g / p r i m a l - d u a l g a p i n t e g r a l Anonymous HEM 0. 1.0 Epoch 1e2     t r a i n i n g / p r
Revised: Set Covering

Original: i m a l - d u a l g a p i n t e g r a l [Figure] [Figure]
Revised: SBP

Original: Epoch 1e [Figure] [Figure] t r a i n i n g / p r i m
Revised: HEM

Original: [Figure]
Revised: [Figure]

Revised: scores, while HEM selects much more diverse cuts than SBP.

Revised: The results show that HEM tends to select cuts that complement each other nicely.

Original: F. E VALUATION WITH A TIME LIMIT OF THREE HOURS
Revised: G. E VALUATION WITH A TIME LIMIT OF THREE HOURS

Original: In this section, we aim to evaluate whether HEM can generalize well to solving problems within a much longer time limit, which is also common in real-world applications.
Revised: In this section, we aim to evaluate whether HEM can generalize well to solving problems within a much longer time limit.

Original: Specifically, we evaluate HEM on two extremely challenging MIPLIB datasets within a time limit of three hours .
Revised: Specifically, we evaluate HEM on two extremely challenging MIPLIB datasets within a time limit of three hours .

Revised: Note that we still train HEM with a time limit ofseconds, while we test HEM with a time limit of three hours.

Original: The results in Table 10 show that HEM  significantly outperforms all the baselines, especially in terms of the primal-dual gap integral on MIPLIB mixed neos and MIPLIB mixed supportcase.
Revised: The results in Table 14 show that HEM still significantly outperforms all the baselines, especially in terms of the primal-dual gap integral on MIPLIB mixed neos and MIPLIB mixed supportcase.

Original: In terms of the primal-dual gap, HEM also outperforms the baselines.
Revised: In terms of the primal-dual gap, HEM also outperforms the baselines.

Original: Moreover, HEM performsa little better than baselines in terms of the solving time on MIPLIB mixed neos, but HEM performs poor in terms of the solving time on MIPLIB mixed supportcase.
Revised: Moreover, HEM performs better than baselines in terms of the solving time on MIPLIB mixed neos, but HEM performs poorly in terms of the solving time on MIPLIB mixed supportcase.

Original: It is interesting that the primal-dual gap integral is not always consistent with the solving time.
Revised: Interestingly, the primal-dual gap integral is not always consistent with the solving time.

Revised: We emphasize that we train with the negative primal-dual gap integral reward.

Original: To  improve the performance of HEM in terms of the solving time, we can set the reward as the  solving time instead of the  primal-dual gap integral.
Revised: To further improve the performance of HEM in terms of the solving time, we can set the reward as the negative solving time instead of the negative primal-dual gap integral.

Original: F. T RAINING CURVES
Revised: G. T RAINING CURVES

Original: In this section, we provide the training curves of HEM on all nine datasets.
Revised: In this section, we provide the training curves of HEM on all nine datasets.

Original: The results in Fig.show that the performance of our learned policies in terms of the solving time or the primal-dual gap integral drops with the  epochs, demonstrating the effectiveness of our learning process. 
Revised: The results in Figure 8 show that the performance of our learned policies in terms of the solving time or the primal-dual gap integral drops with the training epochs, demonstrating the effectiveness of our learning process. [Figure] t r a i n i n g / p r i m a l - d u a l g a p i n t e g r a l

Revised: [Figure]

Revised: 1e

Revised: [Figure]

Revised: [Figure]

Revised: t r a i n i n g / p r i m a l - d u a l g a p

Revised: i n t e g r a l

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: 1e

Revised: [Figure]