Title: MILP-StuDio: MILP Instance Generation via Block Structure Decomposition
Venue: NeurIPS 2024
Accepted: yes
Keywords: Mixed Integer Linear Programming

MILP Instance Generation
Abstract: Mixed-integer linear programming (MILP) is one of the most popular mathematical formulations with numerous applications. In practice, improving the performance of MILP solvers often requires a large amount of high-quality data, which can be challenging to collect. Researchers thus turn to generation techniques to generate additional MILP instances. However, existing approaches do not take into account specific block structures—which are closely related to the problem formulations—in the constraint coefficient matrices (CCMs) of MILPs. Consequently, they are prone to generate computationally trivial or infeasible instances due to the disruptions of block structures and thus problem formulations. To address this challenge, we propose a novel MILP generation framework, called Block Structure Decomposition (MILP-StuDio), to generate high-quality instances by preserving the block structures. Specifically, MILP-StuDio begins by identifying the blocks in CCMs and decomposing the instances into block units, which serve as the building blocks of MILP instances. We then design three operators to construct new instances by removing, substituting, and appending block units in the original instances, enabling us to generate instances with flexible sizes. An appealing feature of MILP-StuDio is its strong ability to preserve the feasibility and computational hardness of the generated instances. Experiments on the commonly-used benchmarks demonstrate that using instances generated by MILP-StuDio is able to significantly reduce over 10% of the solving time for learning-based solvers.

MILP-StuDio: MILP Instance Generation via Block Structure Decomposition

Abstract

Mixed-integer linear programming (MILP) is one of the most popular mathematical formulations with numerous applications. In practice, improving the performance of MILP solvers often requires a large amount of high-quality data, which can be challenging to collect. Researchers thus turn to generation techniques to generate additional MILP instances. However, existing approaches do not take into account specific block structures-which are closely related to the problem formulationsin the constraint coefficient matrices (CCMs) of MILPs. Consequently, they are prone to generate computationally trivial or infeasible instances due to the disruptions of block structures and thus problem formulations. To address this challenge, we propose a novel MILP generation framework, called Block Structure Decomposition (MILP-StuDio), to generate high-quality instances by preserving the block structures. Specifically, MILP-StuDio begins by identifying the blocks in CCMs and decomposing the instances into block units, which serve as the building blocks of MILP instances. We then design three operators to construct new instances by removing, substituting, and appending block units in the original instances, enabling us to generate instances with flexible sizes. An appealing feature of MILP-StuDio is its strong ability to preserve the feasibility and computational hardness of the generated instances. Experiments on commonly-used benchmarks demonstrate that with instances generated by MILP-StuDio, the learning-based solvers are able to significantly reduce over 10% of the solving time.

1 Introduction

Mixed-integer linear programming (MILP) is a fundamental mathematical optimization problem that finds extensive applications in the real world, such as scheduling [1], planning [2], and chip design [3, 4]. In industrial scenarios, the solving efficiency of MILPs is associated with substantial economic value. To speed up the solving process, a great number of high-quality MILP instances are required to develop or test the solvers. Here we give the following two examples. First, both traditional solvers [5, 6] and learning-based solvers [7-10] rely heavily on a lot of MILP instances for hyperparameter tuning or model training. Second, evaluating the robustness of solvers needs a comprehensive MILP benchmark consisting of numerous instances. However, acquiring many instances is often difficult due to high acquisition costs or privacy concerns [11, 12]. As a result, the limited data availability poses great challenges and acts as a bottleneck for solver performance.

(a) CCMs in FA.

Hardness

Preservation

Stucture

Preservation

Scalable

Generation

Efficient

Generation

Enhancing ML

Solver

Enhancing

Benchmark

Construction

Applications

MILP-StuDio

CCM

"-  using 1:2

Block Decomp

"-  using 1:2

"-  using 1:2

"-  using 1:2

(b) Features and applications of MILP-StuDio.

This challenge motivates a wide range of MILP generation techniques. In the past, researchers relied on problem-specific techniques for generation [13-17]. These methods assumed knowledge of problem types and generated instances based on the corresponding mathematical formulations, such as the satisfiability problem [18], set covering problem [15], and others. However, these techniques require much expert knowledge to design and are limited to specific MILP problems. They also face limitations in practical scenarios where problem formulations are unknown [19].

In recent years, there has been some progress in general MILP generation that does not require explicit knowledge of the problem formulations. These approaches can be broadly classified into statistics-based and learning-based methods. Statistics-based approaches utilize a few instance statistics to sample in the MILP space [20]. More advanced learning-based approaches, exemplified by G2MILP [19], leverage deep learning models to capture global instance features and iteratively modify constraints in the original instances. Though in the early stage, learning-based techniques offer convenience and strong adaptability for MILP generation, making them applicable in a wider range of practical scenarios [19]. However, they still suffer from significant challenges. (1) They fail to account for the inherent problem structures adequately and disrupt instances' mathematical properties. This leads to low-quality instances with degrading computational hardness or infeasible regions. (2) Existing methods fail to generate instances with different sizes from the original ones, limiting instance diversity. (3) The iterative style to modify constraints becomes time-consuming when dealing with large-scale instances.

Therefore, a natural question arises: can we analyze and exploit the problem structures during generation to address the above challenges? Consider a MILP instance with constraints Ax ≤ b , where A is the constraint coefficient matrix (CCM), x is the decision variable and b is a vector. As shown in Figure 1a, we observe that a great number of real-world MILP problems exhibit structures with repeated patterns of block units in their CCMs. In operational research, researchers have long noticed the similar block structures of CCMs across instances from the same problem type, and they have been aware of the critical role of CCMs in determining problem formulation and mathematical properties [21-24]. Although a wide suite of CCM-based techniques have been developed to solve MILPs [25-27], existing works on MILP generation rarely pay attention to CCMs. Consequently, these works fail to preserve the block structures during the generation process.

In light of this, we propose a novel MILP generation framework called Block Structure Decomposition (MILP-StuDio), which takes into account the block structures throughout the generation process and addresses Challenge (1)-(3) simultaneously. Specifically, MILP-StuDio consists of three key steps. We begin by identifying the block structures in CCMs and decomposing the instances into block units, which serve as the building blocks in the MILP instances. We then construct a library of the block units, enabling efficient storage, retrieval, and utilization of the comprehensive block characteristics during the subsequent process. Leveraging this library, we design three block operators on the original instances to generate new ones, including block reduction (eliminating certain blocks from the original instances), block mix-up (substituting some blocks with others sampled from the library), and block expansion (appending selected blocks from the library). These operators enable us to generate instances with flexible sizes, effectively improving the diversity of instances.

Experiments demonstrate that MILP-StuDio has the following advanced features. (1) Hardness preservation. MILP-StuDio can effectively preserve the computational hardness and feasibility in the

(a) CA

(b) IP

(c) MIK

(d) WA

generated instances. (2) Scalable generation. MILP-StuDio can generate instances with flexible sizes. (3) High efficiency. MILP-StuDio can reduce over two-thirds of the generation time in real-world large datasets. We observe an over 10% reduction in the solving time for learning-based solvers using instances generated by MILP-StuDio.

2 Background

2.1 MILP and MILP with Block Structure

A MILP instance takes the form of:

In Formula (1), x denotes the decision variables, c ∈ R n denotes the coefficients in the objective function, A ∈ R m × n is the constraint coefficient matrix (CCM) and b ∈ R m denotes the terms on the right side of the constraints, respectively. The vectors l ∈ ( R ∪{-∞} ) n and u ∈ ( R ∪{ + ∞} ) n denote lower and upper bounds for the variables, respectively.

In real-world applications, a significant portion of MILPs exhibit block structures-consisting of many block units-in their constraint coefficient matrices (CCMs) A . These problems, referred as MILPs with block structures, include many commonly-used and widely-studied datasets in recent papers on learning-based solvers [8-10, 28], such as combinatorial auctions (CA), capacitated facility location (FA), item placement (IP), multiple knapsacks (MIK), and workload balancing (WA). In Figure 2, we visualize the CCMs of MILP instances using a black-and-white digital image representation [29]. In this representation, the rows and columns of the digital images correspond to the constraints and variables in the MILPs, respectively. To construct the digital image, we assign a pixel value of 255 (white) to the entry ( i, j ) if the corresponding entry in the CCM A [ i, j ] is nonzero. Conversely, if A [ i, j ] is zero, we set the pixel value to 0 (black). This mapping allows us to depict the sparsity patterns and structural characteristics of CCMs visually. For each problem, the CCMs of the instances present a similar block structure, characterizing specific mathematical formulations.

The importance of block-structured CCMs in the context of MILP solving has long been acknowledged by operational researchers, where instances with similar block structures share similar mathematical properties [29-31]. Furthermore, the block structures of CCMs are closely related to the problem formulations [30]. Thus, the block matrices have shown great potential in accelerating the solution process for a family of MILP problems [21-24]. One notable technique developed to exploit this structure is Dantzig-Wolfe decomposition [25] for solving large-scale MILP instances.

2.2 Bipartite Graph Representation of MILPs

A MILP instance can be represented as a weighted bipartite graph G = ( W∪V , E ) [8]. The two sets of nodes W = { w 1 , · · · , w m } and V = { v 1 , · · · , v n } in the bipartite graph correspond to the MILP's constraints and variables, respectively. The edge set E = { e ij } comprises edges, each connecting a constraint node w i ∈ W with a variable node v j ∈ V . The presence of an edge e ij is determined by the coefficient matrix, with e ij = ( e ij ) as its edge feature, and an edge e ij does not exist if A [ i, j ] = 0 . Please refer to Appendix I.1 for more details on the graph features we use in this paper.

3 Motivated Experiments

Preserving the mathematical properties of the original instances is a fundamental concern in MILP generation [19]. These properties encompass feasibility, computational hardness, and problem structures, with the latter being particularly crucial. The problem structure directly determines the problem formulation and, consequently, impacts other mathematical properties. However, it

is important to note that the term "problem structure" can be ambiguous and confusing. There are different understandings and definitions of the problem structure-such as the bipartite graph structure [19] and the CCM's block structure [29]-and we are supposed to identify the most relevant and useful ones that contribute to the mathematical properties of MILPs. Analyzing and exploiting these specific structure types become key factors in improving the quality of the generated instances.

3.1 Challenges of Low-Quality Generation

G2MILP is the first learning-based approach for MILP instance generation. While it has shown promising performance, we observe that G2MILP still encounters difficulties in generating highquality instances for MILPs with block structures. To evaluate its performance, we compare the graph structural distributional similarity and solving properties of the original and generated instances from the workload appointment (WA) benchmark [32] using Gurobi [5], a state-of-the-art traditional MILP solver. We set the masking ratio of G2MILP-which determines the proportion of constraints to be modified-to 0.01. The results are summarized in Table 1. In this table, the Similarity metric refers to the graph structural distributional similarity score [19] (defined in Appendix I.3), Time represents the average solving time (with a time limit 1,000s), and Feasible ratio indicates the proportion of feasible instances out of the total instances. Results show that although the generated instances achieve a high similarity score, most of them are infeasible. Furthermore, the feasible instances exhibit a severe degradation in computational hardness.

3.2 Visualization of CCMs

The aforementioned experiments provide evi- dence that by iterative modifications of sampled constraints, G2MILP can generate MILP with high graph structural distributional similarities to the original instances, but may lead to disruptions in mathematical properties. Consequently, it becomes necessary to explore alternative definitions of problem structures that offer stronger correlations to the mathematical properties of the instances and can be effectively preserved during the generation process. One such promising structure is the block structures within CCMs.

The concept of block structures within CCMs originates from traditional operational research and has proven to be effective for problem analysis [21-24]. It is widely recognized that MILPs with similar block structures in CCMs often share similar formulations, resulting in similar mathematical properties [29]. We visualize and compare the CCMs of the original and generated instances in Figure 3. In the middle figure, we observe that G2MILP breaks the block pattern in the left and introduces a noisy block in the bottom right. It becomes evident that the generation operation in G2MILP breaks the block structures presenting in the original instances. Thus, it motivates us that exploring and preserving the block structures in CCMs can hold the potential for generating high-quality instances.

4 Generation Framework Using Block Structure Decomposition

In this section, we introduce the proposed MILP-StuDio framework to generate high-quality instances. MILP-StuDio comprises three steps: block decomposition, construction of structure library, and block manipulations. We begin by presenting the concept of block decomposition for MILP in Section 4.1,

Block Decomposition

Structure library

Block Manipulation

(a) Reduction

(c) Expansion

(b) Mix-up

Manipulation

6

MILP

StuDio

Subgraph

Instance

CCM block

1

Reordering CCMs

MILP instances

3

4

Library Construction

Sub-graphs

Sub-matrices

Block Decomposition

2

5

Sampling

"-  using 1:2

Image Representation

"-  using 1:2

"-  using 1:2

"-  using 1:2

"-  using 1:2

min cTc

nc>1

Dn

E ZP × Rn-p

B1

··

Bn

1

O

Bo

O

O

00

as it forms the core of our method. The subsequent sections, from Section 4.2 to Section 4.4, provide a detailed explanation of each step. The overview of MILP-StuDio is depicted in Figure 4.

4.1 Block Decomposition of MILP

In this part, we specify the block structures that we are interested in. CCMs are often reordered to achieve the well-studied block structures [30], including the block-diagonal (BD), bordered blockdiagonal (BBD), and doubly bordered block-diagonal (DBBD) structures. We highlight the block unit of the decomposition in blue in Equation (2). We can see that the former two structures are special cases of the latter one. Despite the simplicity, they are the building blocks for more complex block structures and are widely used in operational research [30].

c ) Doubly bordered block-diagonal

Formally, a MILP with a DBBD structure can be written as

where the partition c = ( c ⊤ 1 , · · · , c ⊤ k +1 ) ⊤ , x = ( x ⊤ 1 , · · · , x ⊤ k +1 ) ⊤ and b = ( b ⊤ 1 , · · · , b ⊤ k +1 ) ⊤ . To process more complex block structures beyond the three basic ones, we specify different types of constraints and variables in a CCM (and the corresponding MILP). First, we classify variables as block and bordered variables (Bl-Vars and Bd-Vars). The block variables DBBD are x block = ( x ⊤ 1 , · · · , x ⊤ k ) ⊤ , which are involved in the blocks D i , (1 ≤ i ≤ k ) . The bordered variables are

defined to be those in F i , (1 ≤ i ≤ k ) , i.e. the variables x k +1 . Notice that all the variables in BD and BBD are block variables. Then, we classify the constraints in an instance as master , block and doubly block constraints (M-Cons, B-Cons, and DB-Cons), which we have illustrated in Equation (2). As we can see, BD only contains B-Cons, BBD contains B-Cons and M-Cons, and DBBD contains DB-Cons and M-Cons. The classifications of constraints and variables make it possible for us to investigate more delicate structures in the instances-such as the combination of the three basic ones-in the subsequent process (please see Appendix H.2 and H.4).

4.2 Block Decomposition

Reordering Rows and Columns in CCMs Given a MILP instance, the raw orders of rows and columns for a CCM are determined by the orders of constraints and variables respectively, which is defined when we establish the instance. Generally, the block structures are not readily apparent in this raw form. To identify and exploit these block structures, we employ a structure detector implemented in the Generic Column Generation (GCG) solver [33] for CCM reordering. This detector identifies row and column permutations that effectively cluster the nonzero coefficients, thereby revealing distinct block structures within CCMs.

Block Decomposition We employ an enhanced variable partition algorithm based on the image representations of CCMs for block decomposition, using the constraint-variable classification results mentioned in Section 4.1. Specifically, we extract the sub-matrices of the block units BU in CCMs, i.e., which take the form of D i in BD, ( D i B i ) in BBD, and ( D i F i B i ) in DBBD. Finally, we

partition and decompose the CCMs into sub-matrices of block units. The algorithm enables us to handle more complex structures beyond the basic three found by GCG, such as instances in WA with

i

where D (1) i represents the diagonal block with DB-Cons, and D (2) i represents the diagonal block with B-Cons. Please refer to Section H.3 for the detailed implementation of the decomposition process.

4.3 Construction of Structure Library

As we can see in Figure 1a, the block units across instances exhibit striking similarities in terms of the internal structures. These common characteristics indicate that the distribution of block units holds valuable information about the problem formulations, making it an ideal building block for reconstructing new instances. Given block-unit sub-matrices of CCMs obtained in Section 4.2, we proceed to extract the corresponding bipartite sub-graphs within the graph representations of the original instances. Compared to the image representation, graph representation offers more convenience for modifying MILP instances during block manipulation. Specifically, suppose that a sub-matrix contains constraints ˜ W = { w i 1 , · · · , w i k } and variables ˜ V = { v i 1 , · · · , v i l } in the instances, we then extract the sub-graph containing ˜ W , ˜ V and the edges connecting ˜ W and ˜ V in the bipartite graph representation of the original instance. Subsequently, we collect these sub-graphs from all the training instances and utilize them to construct a comprehensive structure library denoted as L . This structure library serves as a repository for the collected sub-graphs, allowing efficient storage, retrieval, and utilization of the block information.

4.4 Scalable Generation via Block Manipulations

With the structure library, we devise three types of generation operators that enable the generation of high-quality MILP instances with flexible sizes. These operators, namely block reduction, block mix-up, and block expansion, play a crucial role in the instance generation process.

Reduction. This operator involves randomly sampling a block unit BU ins from the original instances and then removing it. The reduction operator generates MILP instances with smaller sizes compared to the original ones, reducing the complexity of the problem.

Mix-up. This operator involves randomly sampling one block unit BU ins from the original instances and another block unit BU from the structure library L . We then replace BU ins with BU to generate a new instance. The mix-up operator introduces structural variations through the incorporation of external block units.

Expansion. This operator involves randomly sampling a block unit BU from the structure library L and appending it to the original instances. This process generates new instances of larger sizes compared to the original ones, potentially introducing more complex structures.

To preserve the block structures, the operators should leverage the constraint-variable classification results. Taking the expansion operator as an example, the coefficients of M-Cons in the external block unit should be properly inserted into the M-Cons of the original instances. Meanwhile, we construct new constraints for the B- and DB-Cons of the block using the corresponding right-hand-side terms b i . Finally, we design a coefficient refinement algorithm to align the coefficients of the external blocks during mix-up and expansion (please see Appendix H.4 for details).

5 Experiments

5.1 Experiment Settings

Benchmarks We consider four MILP problem benchmarks: combinatorial auctions (CA) [16], capacitated facility location (FA) [17], item placement (IP) [32] and workload appointment (WA) [32]. The first two benchmarks, CA and FA, are commonly-used benchmarks proposed in [8]. The last two benchmarks, IP and WA, come from two challenging real-world problem families used in NeurIPS ML4CO 2021 competition [32]. The numbers of training, validation, and testing instances are 100, 20, and 50. More details on the benchmarks are in Appendix I.2.

Metrics We leverage three metrics to evaluate the similarity between the original and generated instances. (1) Graph statistics are composed of 11 classical statistics of the bipartite graph [34]. Following [19], we compute the Jensen-Shannon divergence for each statistic between the generated and original instances. We then standardize the metrics into similarity scores ranging from 0 to 1. (2) Computational hardness is measured by the average solving time of the instances using the Gurobi solver [5]. (3) Feasible ratio is the proportion of feasible instances out of the total ones.

Baselines We consider two baselines for MILP generation. The first baseline is the statistics-based MILP generation approach Bowly [20], which generates MILP instances by controlling specific statistical features, including the coefficient density and coefficient mean. We set these features to match the corresponding statistics of the original instances so as to generate instances with high statistical similarity with the original ones. The second baseline is the learning-based approach G2MILP [19]. By leveraging masked VAE, G2MILP iteratively masks one constraint node in the bipartite graph and replaces it with a generated one.

Downstream Tasks We consider three downstream tasks to demonstrate the effectiveness of the generated instances in practical applications. (1) Improving the performance of learning-based solvers, including predict-and-search (PS) [10] in Section 5.3 and the GNN approach for learning-to-branch [8] in Appendix F.1. (2) Hyperparameter tuning for the traditional solver (please see Appendix F.3). In the above two tasks, we use MILP-StuDio and the baselines to generate new instances to enrich the training data. We also consider the task of (3) hard instances generation in Section 5.4, which reflects the ability to construct hard benchmarks.

Variants of MILP-StuDio During the generation process, we use the three operators to generate one-third of the instances, respectively. We can choose different modification ratios, which implies that we will remove (substitute or append) block units when performing the reduction (mix-up or expansion) operator until the proportions of variables modified in the instance reaches η .

5.2 Similarity between the Generated and the Original Instances

For each generation technique, we use 100 original instances to generate 1,000 instances and evaluate the similarity between them. As shown in Table 2, we present the graph structural distributional similarity scores between the original and generated instances. We do not consider Bowly in WA since the instance sizes in WA are so large that the generation time is over 200 hours. The results suggest that MILP-StuDio shows a high graph structural similarity compared to the baselines. In FA, instances generated from G2MILP present a low similarity, while our proposed MILP-StuDio can still achieve a high similarity score. As the modification ratio increases, the similarity scores of G2MILP and MILP-StuDio decrease, whereas G2MILP suffers from a more severe degradation.

We also evaluate the computational hardness and feasibility of the generated instances in Table 3. We report the average solving time and feasibility ratio for each dataset. Results demonstrate that the instances generated by the baselines represent a severe degradation in computational hardness.

Moreover, most of the instances generated by G2MILP in FA and WA datasets are infeasible. Encouragingly, MILP-StuDio is able to preserve the computational hardness and feasibility of the original instances, making it a strong method for maintaining instances' mathematical properties.

We visualize the CCMs of the original and generated instances to demonstrate the effectiveness of MILP-StuDio in preserving block structures in Appendix G. MILP-StuDio is able to preserve the block structures while all the baselines fail to do so.

5.3 Improving the Performance of Learning-based Solvers

GNN branching policy [8] and predict-and-search [10] are two representative approaches of learningbased solvers. Here we report the results of PS built on Gurobi and leave the other in Appendix F.1. In alignment with [10], we consider two additional baselines Gurobi and BKS. For each instance, we run Gurobi on a single-thread mode for 1,000 seconds. We also run Gurobi [5] for 3,600 seconds and denote the obtained objective values as the best-known solution (BKS). For more details on the implementation of PS, please refer to Appendix H.1. Table 4 demonstrates the performance of MILP-StuDio-enhanced PS and the baselines, where we set the modification ratio η = 0 . 05 . We report three metrics to measure the solving performance. (1) Obj represents the objective values achieved by different methods. (2) gap abs is the absolute primal gap defined as gap abs := | Obj -BKS | , where smaller gaps indicate superior primal solutions and, consequently, a better performance. (3) Time denotes the average time used to find the solutions.

In CA and FA, all the approaches can solve the instances to the optimal with gap abs = 0 , thus we compare the Time metric. In IP and WA, all the approaches reach the time limit of 1,000s, thus we mainly focus on the gap abs metric. Methods built on PS does not perform well in CA compared to Gurobi, since CA is easy for Gurobi to solve and PS needs to spend time for network inference. Even so, PS+MILP-StuDio achieves a comparable performance to Gurobi. In the other three datasets, MILP-StuDio-enhanced PS outperforms other baselines, achieving the best solving time or gap abs .

5.4 Hard Benchmark Generation

The hard instance generation task is important as it provides valuable resources for evaluating solvers and thus potentially motivates more efficient algorithms. The objective of this experiment is to test the ability to generate harder instances within a given number of iterations. We use 30 original instances to construct the pool. In each iteration, for each instance in the pool, we employ mix-up or expansion operators to generate two new ones, and We then select the instance among and with the longest solving time to replace in the pool. This setting is to preserve the diversity of the pool. We observe that there exist slight differences in the hardness of the original instances, and the generated instances

0

2

4

6

8

10

Iteration Step

12

14

16

18

20

22

Average Time (s)

Bowly

G2MILP

MILP-StuDio

0

50

100

150

200

Time (h)

Time limit

153.4

39.3

derived from the harder original instances are also harder than those from easier ones. If we had simply generated 60 instances and selected the hardest 30, the proportion of instances generated from the hard original instances would have continuously increased, reducing the diversity of the pool. In Figure 5, we depict the growth curve of the average solving time of instances in the pool during 10 iterations. The solving time of the final set is two times larger than that of the initial set, suggesting MILP-StuDio's ability to generate increasingly harder instances during iterations.

5.5 Extensive Studies

Generation Efficiency WAis a challenging benchmark with large instance sizes. The instances in WAhave over 60,000 constraints and variables, making generation in WA especially time-consuming. We compare the generation time of the generation techniques to generate 1,000 instances. We set the time limit to 200 hours. The results in Table 6 show that MILP-StuDio significantly archives 3 × acceleration compared to G2MILP, demonstrating high generation efficiency.

More Results We conduct ablation studies on three operators and modification ratios in Appendix F.2, and we also try to extend MILP-StuDio to MILPs without block structures in Appendix F.4 and MILPs in the real-world industrial dataset in Appendix F.5.

6 Related Work on MILP Generation

The field of MILP generation encompasses two main categories: problem-specific generation and general MILP generation. Problem-specific generation methods rely on expert knowledge of the mathematical formulation of problems to generate instances. Examples include set covering [15], combinatorial auctions [16], and satisfiability [18]. While these methods can generate instances tailored to specific problem types, they are limited in their applicability and require much modeling expert knowledge. On the other hand, general MILP generation techniques aim to generate MILPs using statistical information [20] or by leveraging neural networks to capture instance distributions [19]. G2MILP [19] is the first learning-based generation framework designed for generating general MILPs. This approach represents MILPs as bipartite graphs and utilizes a masked variational auto-encoder [35] to iteratively corrupt and replace parts of the original graphs to generate new ones.

7 Limitations and Future Avenue

Our method originates from the field of operational research and is designed for instances with block structures. The performance of the detector in GCG influences the overall generation quality. Although the detector can identify a wide range of useful structures in the real world, it is still limited when facing instances with extremely complex structures. The exploration of enhancing the performance of the detector, such as those involving prior knowledge of the instances, represents a promising avenue for future research.

8 Conclusion

In this paper, we propose a novel MILP generation framework (MILP-StuDio) to generate high-quality MILP instances. Inspired by the studies of CCMs in operational research, MILP-StuDio manipulates the block structures in CCMs to preserve the mathematical properties-including the computational hardness and feasibility-of the instances. Furthermore, MILP-StuDio has a strong ability of scalable generation and high generation efficiency. Experiments demonstrate the effectiveness of MILP-StuDio in improving the performance of learning-based solvers.

References

John A Muckstadt and Richard C Wilson. An application of mixed-integer programming duality to scheduling thermal generating systems. IEEE Transactions on Power Apparatus and Systems , (12), 1968.

Yves Pochet and Laurence A Wolsey. Production planning by mixed integer programming , volume 149. Springer, 2006.

Kefan Ma, Liquan Xiao, Jianmin Zhang, and Tiejun Li. Accelerating an fpga-based sat solver by software and hardware co-design. Chinese Journal of Electronics , 28(5):953-961, 2019.

Zhihai Wang, Jie Wang, Yinqi Bai Qingyue Yang, Xing Li, Lei Chen, Jianye Hao, Mingxuan Yuan, Bin Li, Yongdong Zhang, and Feng Wu. Towards next-generation logic synthesis: A scalable neural circuit generation framework. In The Thirty-eighth Annual Conference on Neural Information Processing Systems , 2024.

LLC Gurobi Optimization. Gurobi optimizer. URL http://www. gurobi. com , 2021.

Tobias Achterberg. Scip: solving constraint integer programs. Mathematical Programming Computation , 1:1-41, 2009.

He He, Hal Daume III, and Jason M Eisner. Learning to search in branch and bound algorithms. Advances in neural information processing systems , 27, 2014.

Maxime Gasse, Didier Chételat, Nicola Ferroni, Laurent Charlin, and Andrea Lodi. Exact combinatorial optimization with graph convolutional neural networks. Advances in neural information processing systems , 32, 2019.

Zhihai Wang, Xijun Li, Jie Wang, Yufei Kuang, Mingxuan Yuan, Jia Zeng, Yongdong Zhang, and Feng Wu. Learning cut selection for mixed-integer linear programming via hierarchical sequence model. In The Eleventh International Conference on Learning Representations , 2023.

Qingyu Han, Linxin Yang, Qian Chen, Xiang Zhou, Dong Zhang, Akang Wang, Ruoyu Sun, and Xiaodong Luo. A gnn-guided predict-and-search framework for mixed-integer linear programming. In The Eleventh International Conference on Learning Representations , 2023.

Byoung-Woon Kim and Chong-Min Kyung. Exploiting intellectual properties with imprecise design costs for system-on-chip synthesis. IEEE Transactions on Very Large Scale Integration (VLSI) Systems , 10(3):240-252, 2002. doi: 10.1109/TVLSI.2002.1043327.

Jun Sakuma and Shigenobu Kobayashi. A genetic algorithm for privacy preserving combinatorial optimization. In Proceedings of the 9th annual conference on Genetic and evolutionary computation , pages 1372-1379, 2007.

Russ J Vander Wiel and Nikolaos V Sahinidis. Heuristic bounds and test problem generation for the time-dependent traveling salesman problem. Transportation Science , 29(2):167-183, 1995.

David Bergman, André Augusto Ciré, Willem Jan van Hoeve, and J. Hooker. Decision diagrams for optimization. Constraints , 20:494 - 495, 2015.

Egon Balas and Andrew Ho. Set covering algorithms using cutting planes, heuristics, and subgradient optimization: a computational study . Springer, 1980.

Kevin Leyton-Brown, Mark Pearson, and Yoav Shoham. Towards a universal test suite for combinatorial auction algorithms. In Proceedings of the 2nd ACM Conference on Electronic Commerce , pages 66-76, 2000.

Gérard Cornuéjols, Ramaswami Sridharan, and Jean-Michel Thizy. A comparison of heuristics and relaxations for the capacitated plant location problem. European Journal of Operational Research , 50:280-297, 1991.

Jiaxuan You, Haoze Wu, Clark Barrett, Raghuram Ramanujan, and Jure Leskovec. G2sat: learning to generate sat formulas. Advances in neural information processing systems , 32, 2019.

Zijie Geng, Xijun Li, Jie Wang, Xiao Li, Yongdong Zhang, and Feng Wu. A deep instance generative framework for milp solvers under limited data availability. In Advances in Neural Information Processing Systems , 2023.

Simon Andrew Bowly. Stress testing mixed integer programming solvers through new test instance generation methods . PhD thesis, School of Mathematical Sciences, Monash University, 2019.

Dimitris Bertsimas and John N Tsitsiklis. Introduction to linear optimization , volume 6. Athena Scientific Belmont, MA, 1997.

Guy Desaulniers, Jacques Desrosiers, and Marius M Solomon. Column generation , volume 5. Springer Science & Business Media, 2006.

Marco E Lübbecke and Jacques Desrosiers. Selected topics in column generation. Operations research , 53(6):1007-1023, 2005.

Daniel Bienstock George Nemhauser and L Wolsey. Integer programming and combinatorial optimization, 1993.

George B. Dantzig and Philip Wolfe. Decomposition principle for linear programs. Operations Research , 8:101-111, 1960. URL https://api.semanticscholar.org/CorpusID: 61768820 .

FG Commoner. A sufficient condition for a matrix to be totally unimodular. Networks , 3(4): 351-365, 1973.

Ilias Mitrai, Wentao Tang, and Prodromos Daoutidis. Stochastic blockmodeling for learning the structure of optimization problems. AIChE Journal , 68(6):e17415, 2022.

Jiacheng Lin, Meng XU, Zhihua Xiong, and Huangang Wang. CAMBranch: Contrastive learning with augmented MILPs for branching. In The Twelfth International Conference on Learning Representations , 2024. URL https://openreview.net/forum?id=K6kt50zAiG .

Zachary Steever, Chase C. Murray, Junsong Yuan, Mark H. Karwan, and Marco E. Lübbecke. An image-based approach to detecting structural similarity among mixed integer programs. INFORMS J. Comput. , 34(4):1849-1870, 2022. doi: 10.1287/IJOC.2021.1117. URL https: //doi.org/10.1287/ijoc.2021.1117 .

JAKUB MARE. Exploiting structure in integer programs . PhD thesis, Citeseer, 2011.

Ambros Gleixner, Gregor Hendel, Gerald Gamrath, Tobias Achterberg, Michael Bastubbe, Timo Berthold, Philipp Christophel, Kati Jarck, Thorsten Koch, Jeff Linderoth, et al. Miplib 2017: data-driven compilation of the 6th mixed-integer programming library. Mathematical Programming Computation , 13(3):443-490, 2021.

Maxime Gasse, Simon Bowly, Quentin Cappart, Jonas Charfreitag, Laurent Charlin, Didier Chételat, Antonia Chmiela, Justin Dumouchelle, Ambros Gleixner, Aleksandr M. Kazachkov, Elias Khalil, Pawel Lichocki, Andrea Lodi, Miles Lubin, Chris J. Maddison, Morris Christopher, Dimitri J. Papageorgiou, Augustin Parjadis, Sebastian Pokutta, Antoine Prouvost, Lara Scavuzzo, Giulia Zarpellon, Linxin Yang, Sha Lai, Akang Wang, Xiaodong Luo, Xiang Zhou, Haohan Huang, Shengcheng Shao, Yuanming Zhu, Dong Zhang, Tao Quan, Zixuan Cao, Yang Xu, Zhewei Huang, Shuchang Zhou, Chen Binbin, He Minggui, Hao Hao, Zhang Zhiyu, An Zhiwu, and Mao Kun. The machine learning for combinatorial optimization competition (ml4co): Results and insights. In Douwe Kiela, Marco Ciccone, and Barbara Caputo, editors, Proceedings of the NeurIPS 2021 Competitions and Demonstrations Track , volume 176 of Proceedings of Machine Learning Research , pages 220-231. PMLR, 06-14 Dec 2022. URL https:// proceedings.mlr.press/v176/gasse22a.html .

Gerald Gamrath and Marco E. Lübbecke. Experiments with a generic dantzig-wolfe decomposition for integer programs. In The Sea , 2010. URL https://api.semanticscholar.org/ CorpusID:14380165 .

Nathan Brown, Marco Fiscato, Marwin HS Segler, and Alain C Vaucher. Guacamol: benchmarking models for de novo molecular design. Journal of chemical information and modeling , 59(3):1096-1108, 2019.

Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 , 2013.

Xijun Li, Fangzhou Zhu, Hui-Ling Zhen, Weilin Luo, Meng Lu, Yimin Huang, Zhenan Fan, Zirui Zhou, Yufei Kuang, Zhihai Wang, Zijie Geng, Yang Li, Haoyang Liu, Zhiwu An, Muming Yang, Jianshu Li, Jie Wang, Junchi Yan, Defeng Sun, Tao Zhong, Yong Zhang, Jia Zeng, Mingxuan Yuan, Jianye Hao, Jun Yao, and Kun Mao. Machine learning insides optverse ai solver: Design principles and applications, 2024.

Prateek Gupta, Maxime Gasse, Elias Khalil, Pawan Mudigonda, Andrea Lodi, and Yoshua Bengio. Hybrid models for learning to branch. Advances in neural information processing systems , 33:18087-18097, 2020.

Prateek Gupta, Elias B Khalil, Didier Chetélat, Maxime Gasse, Yoshua Bengio, Andrea Lodi, and M Pawan Kumar. Lookback for learning to branch. arXiv preprint arXiv:2206.14987 , 2022.

Vinod Nair, Sergey Bartunov, Felix Gimeno, Ingrid Von Glehn, Pawel Lichocki, Ivan Lobov, Brendan O'Donoghue, Nicolas Sonnerat, Christian Tjandraatmadja, Pengming Wang, et al. Solving mixed integer programs using neural networks. arXiv preprint arXiv:2012.13349 , 2020.

Jie Wang, Zhihai Wang, Xijun Li, Yufei Kuang, Zhihao Shi, Fangzhou Zhu, Mingxuan Yuan, Jia Zeng, Yongdong Zhang, and Feng Wu. Learning to cut via hierarchical sequence/set model for efficient mixed-integer programming. IEEE Transactions on Pattern Analysis and Machine Intelligence , pages 1-17, 2024. doi: 10.1109/TPAMI.2024.3432716.

Yunhao Tang, Shipra Agrawal, and Yuri Faenza. Reinforcement learning for integer programming: Learning to cut. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning , volume 119 of Proceedings of Machine Learning Research , pages 9367-9376. PMLR, 13-18 Jul 2020. URL https: //proceedings.mlr.press/v119/tang20a.html .

Zeren Huang, Kerong Wang, Furui Liu, Hui-Ling Zhen, Weinan Zhang, Min jie Yuan, Jianye Hao, Yong Yu, and Jun Wang. Learning to select cuts for efficient mixed-integer programming. Pattern Recognit. , 123:108353, 2021. URL https://api.semanticscholar.org/ CorpusID:235247851 .

Haotian Ling, Zhihai Wang, and Jie Wang. Learning to stop cut generation for efficient mixedinteger linear programming. Proceedings of the AAAI Conference on Artificial Intelligence , 38 (18):20759-20767, Mar. 2024. doi: 10.1609/aaai.v38i18.30064. URL https://ojs.aaai. org/index.php/AAAI/article/view/30064 .

Abdel Ghani Labassi, Didier Chetelat, and Andrea Lodi. Learning to compare nodes in branch and bound with graph neural networks. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems , volume 35, pages 32000-32010. Curran Associates, Inc., 2022. URL https://proceedings.neurips.cc/paper_files/paper/2022/file/ cf5bb18807a3e9cfaaa51e667e18f807-Paper-Conference.pdf .

Yufei Kuang, Xijun Li, Jie Wang, Fangzhou Zhu, Meng Lu, Zhihai Wang, Jia Zeng, Houqiang Li, Yongdong Zhang, and Feng Wu. Accelerate presolve in large-scale linear programming via reinforcement learning. arXiv preprint arXiv:2310.11845 , 2023.

Chang Liu, Zhichen Dong, Haobo Ma, Weilin Luo, Xijun Li, Bowen Pang, Jia Zeng, and Junchi Yan. L2p-MIP: Learning to presolve for mixed integer programming. In The Twelfth International Conference on Learning Representations , 2024. URL https://openreview. net/forum?id=McfYbKnpT8 .

Huigen Ye, Hua Xu, and Hongyan Wang. Light-MILPopt: Solving large-scale mixed integer linear programs with lightweight optimizer and small-scale training dataset. In The Twelfth International Conference on Learning Representations , 2024. URL https://openreview. net/forum?id=2oWRumm67L .

Huigen Ye, Hua Xu, Hongyan Wang, Chengming Wang, and Yu Jiang. GNN&GBDT-guided fast optimizing framework for large-scale integer programming. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning , volume 202 of Proceedings of Machine Learning Research , pages 39864-39878. PMLR, 23-29 Jul 2023. URL https: //proceedings.mlr.press/v202/ye23e.html .

Nicolas Sonnerat, Pengming Wang, Ira Ktena, Sergey Bartunov, and Vinod Nair. Learning a large neighborhood search algorithm for mixed integer programs. ArXiv , abs/2107.10201, 2021. URL https://api.semanticscholar.org/CorpusID:236154746 .

Taoan Huang, Aaron Ferber, Yuandong Tian, Bistra N. Dilkina, and Benoit Steiner. Searching large neighborhoods for integer linear programs with contrastive learning. In International Conference on Machine Learning , 2023. URL https://api.semanticscholar.org/ CorpusID:256598329 .

Ted K. Ralphs and Matthew V. Galati. Decomposition methods for integer programming. 2011. URL https://api.semanticscholar.org/CorpusID:122662809 .

George B. Dantzig and Philip Wolfe. Decomposition principle for linear programs. Operations Research , 8:101-111, 1960. URL https://api.semanticscholar.org/CorpusID: 61768820 .

George Bernard Dantzig and Mukund Narain Thapa. Linear programming: Theory and extensions , volume 2. Springer, 2003.

Brian T. Denton, Andrew J. Miller, Hari Balasubramanian, and Todd R. Huschka. Optimal allocation of surgery blocks to operating rooms under uncertainty. Oper. Res. , 58(4-Part-1): 802-816, 2010. doi: 10.1287/OPRE.1090.0791. URL https://doi.org/10.1287/opre. 1090.0791 .

Hans Kellerer, Ulrich Pferschy, and David Pisinger. Multiple Knapsack Problems , pages 285316. Springer Berlin Heidelberg, Berlin, Heidelberg, 2004. ISBN 978-3-540-24777-7. doi: 10.1007/978-3-540-24777-7_10. URL https://doi.org/10.1007/978-3-540-24777-7_ 10 .

Gonzalo E. Constante-Flores and Antonio J. Conejo. Security-constrained unit commitment: A decomposition approach embodying kron reduction. Eur. J. Oper. Res. , 319(2):427-441, 2024. doi: 10.1016/J.EJOR.2023.06.013. URL https://doi.org/10.1016/j.ejor.2023. 06.013 .

T. C. Hu. Multi-commodity network flows. Operations Research , 11:344-360, 1963. URL https://api.semanticscholar.org/CorpusID:121932073 .

Alessandro Lonardi, Mario Putti, and Caterina De Bacco. Multicommodity routing optimization for engineering networks. Scientific Reports , 12(1):7474, May 2022. ISSN 2045-2322. doi: 10.1038/s41598-022-11348-9. URL https://doi.org/10.1038/s41598-022-11348-9 .

Alexander Cowen-Rivers, Wenlong Lyu, Rasul Tutunov, Zhi Wang, Antoine Grosnit, Ryan-Rhys Griffiths, Alexandre Maravel, Jianye Hao, Jun Wang, Jan Peters, and Haitham Bou Ammar. Hebo: Pushing the limits of sample-efficient hyperparameter optimisation. Journal of Artificial Intelligence Research , 74, 07 2022.

Antoine Prouvost, Justin Dumouchelle, Lara Scavuzzo, Maxime Gasse, Didier Chételat, and Andrea Lodi. Ecole: A gym-like library for machine learning in combinatorial optimization solvers. In Learning Meets Combinatorial Algorithms at NeurIPS2020 , 2020. URL https: //openreview.net/forum?id=IVc9hqgibyB .

Table of Contents for Appendix

A More Related Work

A.1 Machine Learning for Solving MILP

In recent years, there has been notable progress in utilizing machine learning approaches to accelerate MILP solvers [36]. The learning-based approaches for solving MILPs can be broadly categorized into two main groups. The first group of works replaces specific components of the solver that greatly impact solving efficiency, such as branching [8, 37-39], cut selection [40, 9, 41-43], node selection [7, 44] and presolve [45, 46]. These approaches are integrated into exact MILP solvers, ensuring that the resulting solutions are optimal. However, they also inherit the limitation of exact solvers, which can be time-consuming for solving large-scale instances.

The second category includes learning-aided search approaches, which encompass techniques such as predict-and-optimize [10, 47, 48], large neighbor search [49, 50] and so on. Among these methods, predict-and-search (PS) [10] has gained significant popularity. In PS, a machine learning model is first employed to predict a feasible solution, which is then used as an initial point for a traditional solver to explore the solution space and find the best possible solution within a defined neighborhood. By leveraging the predicted feasible solution, PS effectively reduces the search space of the solver, leading to accelerated search and the discovery of high-quality solutions.

By leveraging machine learning techniques, these approaches have demonstrated substantial improvements in the solving efficiency and solution quality of MILP problems. However, training such learning-based models requires many MILP instances as training data, which can be challenging to obtain in real-world applications [19]. There is still ongoing research in this area to improve the sample efficiency of these solvers [10] and study the MILP generation techniques [19].

A.2 MILP with Block Structure

Operational researchers have observed that a great number of real-world MILP problems exhibit block structures within their constraint coefficient matrices (CCMs) [30, 27]. These problems are prevalent in various applications, including scheduling, planning, and knapsack scenarios [31, 16, 17]. Thus, many researchers have focused on understanding the impact of these block structures on the mathematical properties of the instances, and how to leverage them to accelerate the solving process[21-24, 26, 51]. Among this stream of research, the Dantzig-Wolfe decomposition (DWD) [52] stands out as one of the most successful applications. Many classical textbooks on linear programming dedicate sections to discussing this decomposition algorithm [53, 21], underscoring its importance. DWD is widely utilized when a CCM contains both block-diagonals and coupling constraints [52], and can significantly accelerate the solving process for large-scale linear programming and mixed-integer linear programming.

To identify the block structures in a raw CCM, the state-of-the-art detecting algorithm is implemented in GCG (Generic Column Generation) [33], distributed with the SCIP framework [6]. This sophisticated algorithm utilizes row and column permutations to effectively cluster the nonzero coefficients in CCMs, thereby revealing distinct block structures, including the useful block-diagonal, bordered block-diagonal, and doubly bordered block-diagonal structures.

B Broader Impacts

This paper introduces a MILP instance generation framework (MILP-StuDio) that aims to generate additional MILP instances with highly similar computational properties to the original instances. Our approach holds significant potential in numerous practical applications and important engineering scenarios, including scheduling, planning, facility location, etc. With instances generated by MILPStuDio, we can improve the performance of block-box learning-based or traditional solvers at a low data acquisition cost. One notable advantage of our proposed method is that it does not rely on the knowledge of MILP formulations and can be applied to general MILP instances, eliminating concerns regarding privacy disclosure. Furthermore, as a plug-and-play module, MILP-StuDio offers great convenience for users.

C The Importance of MILPs with Block Structures

The MILPs with block structures are important in industrial and academic fields. We found that MILP instances with block structures are commonly encountered in practical scenarios and have been an important topic in operations research (OR) with much effort [25, 54-58].

MILP with block structures is an important topic in OR. Analyzing block structures is a critical tool for analyzing the mathematical properties of instances or accelerating the solving process (e.g., Dantzig-Wolfe decomposition [25]) in OR. The MIPLIB dataset also provides visualization results of the constraint coefficient matrices for each instance, highlighting the prevalence of block structures.

The MILP instances with block structures are common and have wide applications in daily production and life. There are many examples where the instances present block structures, including the allocation and scheduling problems [54], the multi-knapsack problem [55], the security-constrained unit commitment problem in electric power systems [56], multicommodity network flow [57], multicommodity transportation problem [58] and so on. In real-world optimization scenarios, there are different types of similar items-such as different workers or machines in planning and scheduling problems, a set of power generation units in the electric power systems, vehicles in the routing problems, and so on-with relevant variables naturally presents a block-structured form in the mathematical models.

The datasets we used in this paper (IP and WA) are from real-world applications. The NeruIPS 2021 Competition of Machine Learning for Combinatorial Optimization [32] released three wellrecognized challenging datasets from real-world applications (IP, WA, and the anonymous dataset). Two of the three competition datasets (IP and WA) have block structures. Moreover, instances from the anonymous dataset are selected from MIPLIB with large parts having block structures. These further reflect the wide application of block structures in real-world applications. Thus, our method works in a wide range of problems in practice.

Researchers have investigated specific MILP problems with block structures. MILP with block structures has a large scope in the optimization field and there has been a wide range of works on specific problems with block structures, and they have developed a suite of optimization problems tailored to these problems. For example, the tailored algorithm for the security-constrained unit commitment problem in electric power systems [4], multicommodity transportation problem [6], vehicle routing problem [7], and so on.

Thus, MILP with block structures has a large scope in production and optimization. It has drawn much attention in the industry and academic fields.

D Explanation: Why do the MILPs Exhibit Block Structures?

The key reasons why MILP instances exhibit block structures can be summarized as follows.

Repeated items or entities with similar attributes. In many real-world applications involving scheduling, planning, and packing problems, we often encounter multiple items or entities that share the same type or attributes. For instance, in a scheduling problem, there may be multiple destinations or vehicles that exhibit similar characteristics. Similarly, in a knapsack problem, there can be multiple packages or items that are interchangeable from the perspective of operational research or mathematical modeling.

Symmetric interactions between different types of items. These repeated items or entities, as well as their interactions, lead to symmetries in the mathematical formulation of the MILP instances. For example, in a scheduling problem, all the vehicles may be able to pick up items from the same set of places and satisfy the demand of the same set of locations.

These symmetries in the problem structure are reflected in the blocks of CCMs, where each block may represent the information associated with a certain vehicle, destination, or other item type.

It is important to note that although MILP-StuDio is designed primarily for MILP instances with block structures, it has a broader application to a wider category of problems, including those with more complex block structures (Appendix H.4), blocks of different sizes (Appendix H.4), and even non-structural MILPs (Appendix F.4).

E Introductions the Underlying Learning-Based Solvers

E.1 Learning to Branch

Branching is a critical component of branch-and-bound (B&B) solvers for mixed-integer linear programming (MILP) problems. It involves selecting a fractional variable to partition the feasible region in each iteration. The effectiveness of the chosen variable and the time required to make the branching decision are crucial factors that heavily impact the size of the branch-and-bound search tree and, consequently, the solver's efficiency. Thus, there have been many efforts to develop effective and efficient branching policies. Among the conventional branching policies, the strong branching policy has been demonstrated to yield the smallest branch-and-bound trees. This policy identifies the fractional variable that provides the largest improvement in the bound before performing the branching operation. However, the evaluation process associated with strong branching involves solving a considerable number of linear relaxations of the original MILP, resulting in unacceptable time and computational costs.

To address the limitations of strong branching, [8] proposes a GNN-based approach that employs behavior cloning to imitate the strong branching policy. In this approach, the B&B process is formulated as a Markov decision process, with the solver acting as the environment. At each step, the branching policy receives the current state s , which includes information on past branching decisions and current solving statistics. Subsequently, the policy selects an action a from the set of integer variables with fractional values in the current state. The researchers parameterize the branching policy as a GNN model, which serves as a fast approximation of the strong branching policy.

During the training process, we first run the strong branching expert on the training and testing instances to collect the state-action pair ( s , a ) , forming the training dataset D train and testing dataset D test. The GNN branching policy π θ is then trained by minimizing the cross-entropy loss

The imitation accuracy refers to the consistency of the learned GNN policy compared to the strong branching expert on the testing dataset D test :

where I is the indicator function defined as I ( c ) = 1 if the condition c is true, and I ( c ) = 0 otherwise. The imitation accuracy of the GNN model reflects the similarity between the learned branching policy and the expert policy, and it significantly impacts the solver's efficiency.

E.2 Predict-and-Search

Different from learning-to-branch, which replaces certain heuristics in a traditional B&B solver, Predict-and-Search (PS) [10] belongs to another category of learning-based solvers that directly predict a feasible solution and subsequently perform the neighborhood search. PS aims to leverage a learning-based model to approximate the solution distribution p ( x | I ) given an instance I from the instance dataset D . Given an instance I , the solution distribution p ( x | I ) is defined as follows:

In the prediction step, PS employs a GNN model to approximate the solution distribution for the binary variables in a MILP instance. To train the GNN predictor p θ ( x | I ) , PS adopts the assumption, as described in [39], that the variables are independent of each other, i.e., p θ ( x | I ) = ∏ n i =1 p θ ( x i | I ) . To calculate the prediction target, PS collects a set of m feasible solutions for each instance I , from which a vector p = ( p 1 , p 2 , . . . , p n ) ⊤ is constructed. Here, p i = p ( x i = 1 |I ) represents the probability of variable x i being assigned the value 1, given the instance I . The GNN predictor then outputs the predicted probability p θ ( x i = 1 |I ) for each variable. Finally, the predictor p θ is trained by minimizing the cross-entropy loss

where N represents the number of instances in the training set.

In the search step, PS performs the neighborhood search based on the predicted partial solution ˆ x . This involves employing a traditional solver, such as SCIP [6] or Gurobi [5], to explore the neighborhood B (ˆ x , △ ) of ˆ x in search of an optimal feasible solution. Here △ represents the trust region radius, and B (ˆ x , △ ) = { x ∈ R n | ∥ ˆ x -x ∥ 1 ≤ △} is the trust region. The neighborhood search process is formulated as the following MILP problem,

It is worth noting that the accuracy of the GNN predictor strongly influences the overall solving performance. A reliable and accurate prediction of a feasible solution can effectively reduce the search time. Conversely, an inaccurate prediction may result in an inferior search neighborhood and sub-optimal solution.

F Extensive Experiment Results

F.1 Experiments on Learning to Branch

In this section, we conduct experiments on the learning-to-branch task to further demonstrate the effectiveness of MILP-StuDio in enhancing the learning-based solvers.

Experiment Setup We conduct the learning-to-branch experiments following the setting described in the original paper [8]. In our experiments, we evaluate the performance of the solvers using two benchmark problems: the capacitated facility location (FA) and item placement (IP) problems. The capacitated facility location problem (FA) is chosen as a representative problem that solvers can successfully solve within the given time limit. This problem serves as a benchmark to test the solving efficiency of solvers. The item placement problem (IP) is selected as a benchmark where the solvers struggle to find the optimal solution within the given time limit. By including this challenging problem, we can gain insights into the solvers' ability to find a high-quality primal solution.

The training, validation, and testing instance sets used in this work are identical to those described in the main paper. Specifically, for each benchmark, we use 100 training instances, 20 validation instances, and 50 testing instances. To generate training samples, we execute the strong branching expert on instances in each benchmark, collecting 1,000 training samples, 200 validation samples, and 200 testing samples. For the final evaluation, we test the solving performance on 50 instances.

We implement the model with the code available at https://github.com/ds4dm/learn2branch . This code leverages the state-of-the-art open-source solver SCIP 8.0.3 [6] as the backend solver. During testing, the solving time limit for each instance is set to 1,000 seconds. Consistent with the setting in [8], we disabled solver restarts and only allowed the cutting plane generation module to be employed at the root node.

Baselines The first baseline we consider is the GNN model [8], which is trained on an initial set of 1,000 samples collected from 100 original instances ( GNN ). To explore the effectiveness of different instance generation techniques, we use the same set of 100 training instances to generate an additional 1,000 instances using each technique. Subsequently, we collect 10,000 training samples by running a strong branching expert again on these instances. In combination with the original 1,000 samples, we have a comprehensive enriched training dataset consisting of 11,000 samples. Thus, the three generation techniques lead to the following approaches: GNN+Bowly , GNN+G2MILP η = x , GNN+MILP-StuDio η = x , where η = x implies that we set the modification ratio to be x . Additionally, we compare our method with a GNN model trained on 11,000 samples collected from 1,100 original instances ( GNN-11000 ). This comparison allows us to demonstrate that the improvement achieved by our approach is not only due to an increase in the number of training samples but also the high quality of the generated instances. To provide a comprehensive evaluation, we also compare the solving performance of our method with that of the SCIP solver [6] ( SCIP ), which serves as our backend solver for comparison purposes.

Experiment Results In this section, we conduct a comprehensive comparison of imitation accuracy and solving performance. As discussed in Appendix E.1, imitation accuracy serves as a crucial

metric for evaluating the performance of the GNN model. Thus, we first compare our methods with other GNN-based baselines, including GNN, GNN-11000, GNN+Bowly, and GNN+G2MILP regarding imitation accuracy. The corresponding results are presented in Table 5, where we observe that MILP-StuDio yields the most significant improvement in imitation accuracy. Interestingly, we find that the instances generated by Bowly in FA are computationally trivial, such that the solver does not need to perform any branching operations to solve them. Consequently, we are unable to collect additional branching samples from these trivial instances to further train the GNN model.

Furthermore, we evaluate the solving performance of the different baselines, as summarized in Table 6. Notably, GNN+MILP-StuDio outperforms all the generation-technique-enhanced baselines and achieves comparable solving performance of GNN-11000. This suggests that incorporating the MILP-StuDio method can boost the performance of the GNN branching policy. We also find that the generated data from MILP-StuDio with different modification ratios can be beneficial for training the GNN model while using instances generated by the G2MILP method may potentially disrupt the training of the GNN model. These findings substantiate the effectiveness of MILP-StuDio in enhancing the GNN branching policy, both in terms of imitation accuracy and solving performance.

F.2 More Results on Different Generation Operators and Modification Ratios

In this part, we investigate the influence of different generation operators and modification ratios of MILP-StuDio. 'MILP-StuDio (Optor) η = x ' denotes the generation approach that uses the operator Optor ('red' means reduction, 'mix' represents mix-up, and 'exp' denotes expansion) and modification ratio η = x .

Influence on the Similarity between the Generated and the Original Instances We conduct experiments on the FA benchmark. We first compute the graph distributional similarity score for the 100 original instances and 1,000 instances generated using different operators and modification ratios. The results presented in Table 7 suggest that the choice of generation operator and modification ratio can impact the values of the similarity scores. (1) MILP-StuDio consistently outperforms G2MILP in similarity scores. (2) Among the three operators, mix-up achieves the highest similarity scores, with values exceeding 0.8 across the modification ratios. Although the reduction and expansion operators yield lower similarity scores, they are still able to achieve comparable performance in the downstream tasks, as shown in the following paragraph. This finding indicates that there is not necessarily a positive correlation between the structural similarity scores and the benefits for the downstream tasks.

We also evaluate the computational properties of the generated instances. The results in Table 8 show that the three operators are able to preserve the computational hardness of the original instances. Importantly, all the operators succeed in generating feasible instances, which is a crucial requirement for the downstream tasks. In contrast, the instances generated by the G2MILP approach are found to have an extremely low feasible ratio.

Moreover, as the modification ratio increases, we observe a decrease in the similarity scores. Considering the good computational properties we have discussed, this finding suggests that the increasing modification ratios lead to the generation of more novel instances. This observation indicates that MILP-StuDio is able to generate instances increasingly distinct from the original data, while still preserving the computational hardness and feasibility properties.

Influence of Generation Operators on the Performance of Predict-and-Search We investigate the influence of instances generated with different operators on the performance of Predict-andSearch (PS). Specifically, we conduct experiments on the FA and IP benchmarks, using 1,000 generated instances based on 100 original instances and different operators. For comparison, we also report the results of PS+MILP-StuDio η = 0 . 05 . In PS+MILP-StuDio η = 0 . 05 , we use the three

↓

operators (red, mix, and exp) to generate 1,000 instances (333, 334, and 333 using the three operators, respectively). The results are presented in Tables 9 and 10, which compare the prediction loss and solving performance on the testing datasets for the different methods. The key findings are as follows. (1) All three generation operators are beneficial for the performance of the PS algorithm. (2) The PS algorithm is not overly sensitive to the choice of generation operator (3) The most beneficial operator may differ across different benchmarks.

Influence of Modification Ratio on the Performance of Predict-and-Search We investigate how the modification ratio impacts the performance of PS. Specifically, we conduct experiments on the FA and IP benchmarks, generating 1,000 instances using 100 original instances with different modification ratios { 0 . 01 , 0 . 05 , 0 . 10 } for both the MILP-StuDio and G2MILP approaches. In PS+MILP-StuDio, we use the three operators (red, mix, and exp) to generate one-third of the 1,000 instances, respectively. The results are presented in Tables 11 and 12, in which we compare the prediction loss and solving performance on the testing datasets for the different methods. Tables 11 and 12 showcase the following results. (1) MILP-StuDio consistently outperforms G2MILP across modification ratios. (2) A smaller modification ratio in MILP-StuDio can lead to a slightly better performance of PS. (3) MILP-StuDio is not sensitive to the masking ratio in general.

F.3 Enhancing the Traditional Solvers via Hyperparameter Tuning

In our Gurobi hyperparameter tuning experiment, We employ the Bayesian optimization framework provided by the HEBO package [59]. Each tuning process involves 100 trials, where different hyperparameter configurations are sampled and evaluated. To strike a balance between tuning efficiency and effectiveness, we focus on optimizing eight key hyperparameters: Heuristics, MIPFocus, VarBranch, BranchDir, Presolve, PrePasses, Cuts, and Method. We list and briefly introduce the key hyperparameters in Table 13.

We utilize the Bayesian optimization package HEBO [59] to perform hyperparameter tuning for the Gurobi solver. We start by using the original dataset consisting of 20 FA instances and generate an additional 200 instances using the MILP-StuDio framework with a modification ratio of η = 0 . 05 . This enriched dataset is then used to run the hyperparameter optimization process, where we perform 100 trials to search for the best parameter configuration. Finally, we evaluate the performance of the tuned Gurobi solver on a testing dataset of 50 instances. We denote the default Gurobi as Gurobi, the Gurobi tuned on the 20 original instances as Gurobi-tuned, and the Gurobi tuned with additional instances generated by Bowly, G2MILP, and MILP-StuDio as tuned-Bowly, tuned-G2MILP, and tuned-MILP-StuDio, respectively. Experiment results in Table 14 demonstrate that Gurobi tuned with

↓

additional instances generated by the MILP-StuDio framework (tuned-MILP-StuDio) consistently outperforms the other baseline methods.

F.4 More Results on Non-Structured MILPs

While MILP-StuDio is primarily designed to handle MILP instances with block structures in their CCMs, it is also important to evaluate its performance on wider classes of MILP problems, including those that may not exhibit such structured characteristics. Evaluating MILP-StuDio's capabilities in this broader context will highlight its potential for broader applications.

To extend our method to general MILP instances, the framework remains unchanged and we just need to adjust the block decomposition and partition Algorithms 1 and 2. Specifically, the constraint

↓

and variable classification Algorithm 1 is tailored to general MILPs. The graph community reflects the connection of the constraints and variables, which can serve as a generalization of blocks. Block partition and community discovery are both graph node clustering. Thus, we cluster the constraints and variables according to the community and classification results to form generalized 'blocks'.

To this end, we provide more results on the SC (set covering) and MIS (maximum independent set) datasets (two popular benchmarks with non-structural instances from [8]) and further investigate the improvement of the ML solvers. We use 100 original instances, following the generation algorithm described in [8] and [15]), and generate 1,000 new instances using G2MILP and MILP-StuDio. The results demonstrate the outstanding performance of MILP-StuDio on general MILP instances without block structures. (1) Tables 15 and 17 show that MILP-StuDio can even outperform existing generation methods in terms of the similarity score and feasibility ratio. (2) Tables 15 and 17 show that MILP-StuDio better preserves instances' computational hardness with closer solving time to the original instances. (3) Tables 16 and 18 show that MILP-StuDio leads to the greatest improvement for the PS solver.

F.5 More Results on Real-world Industrial Dataset

To further demonstrate the effectiveness in real-world applications, we also conduct experiments on a real-world scheduling dataset from an anonymous enterprise, which is one of the largest global commercial technology enterprises. The instances do not present clear block structures. The results in Table 19 show that the extended framework generalizes well on the general MILP datasets and has promising potential for real-world applications. The dataset contains 36 training and 12 testing instances. The few training instances reflect the data inavailability problem in real-world applications. We use different data generation methods to enhance the performance of the MLPS solver and list the solving time, objective value, and node number in Table 19 as follows. MILP-StuDio outperforms other baselines in this dataset, highlighting its strong performance and applicability.

G Visualizations of CCMs

To provide further insights into the characteristics of the instances generated by MILP-StuDio, we visualize the Constraint Coefficient Matrices (CCMs) of both the original and generated instances in Figure 7-10. The CCM visualization is a powerful tool to understand the structural properties of MILP instances, as it captures the patterns and relationships between the constraints and variables. By comparing the CCMs of the original and generated instances, we can understand how well MILP-StuDio is able to preserve the key structural characteristics of the input problems.

As shown in Figure 7-10, MILP-StuDio is able to successfully maintain the block structures observed in the original instances across the various benchmark problems. This indicates that the generated instances share similar underlying problem structures with the original data, which is a desirable property. This ability to preserve the structural properties of MILP instances is crucial for the effective training and evaluation of machine learning-based MILP solvers. In contrast, Bowly struggles to

capture the intricate CCM structures, while G2MILP introduces additional noise that disrupts the block structures in the generated CCMs.

H Implementation Details

H.1 Implementation of Predict-and-Search

For model structure, the PS models used in this paper align with those outlined in the original papers [10]. We use the code in https://github.com/sribdcn/Predict-and-Search MILP method to implement PS. For the PS predictor, we leverage a graph neural network comprising four half-convolution layers. We conducted all the experiments on a single machine with NVidia GeForce GTX 3090 GPUs and Intel(R) Xeon(R) E5-2667 V4CPUs 3.20GHz.

In the training process of MILP-StuDio, we set the initial learning rate to be 0.001 and the training epoch to be 1000 with early stopping. In addition, the partial solution size parameter ( k 0 , k 1 ) and neighborhood parameter ∆ are two important parameters in PS. The partial solution size parameter ( k 0 , k 1 , ∆ ) represents the numbers of variables fixed with values 0 and 1 in a partial solution. The neighborhood parameter ∆ defines the radius of the searching neighborhood. We list these two parameters used in our experiments in Table 20.

CA: original.

CA: Bowly.

CA: G2MILP.

CA: MILP-StuDio.

FA: original.

FA: Bowly.

FA: G2MILP.

FA: MILP-StuDio.

IP: original.

IP: Bowly.

IP: G2MILP.

IP: MILP-StuDio.

WA: original.

WA: Bowly (failed due to time out).

WA: G2MILP.

WA: MILP-StuDio.

H.2 Implementation of Classification Algorithm for Constraints and Variables

In Section 4.1, we classified the constraints into M-Cons, B-Cons, and DB-Cons, and the variables into Bl-Vars and Bd-Vars. As shown in Equation (4), we show the constraint and variable classification results for the bordered block-diagonal and doubly bordered block-diagonal structures.

Identifying these different types of constraints and variables in a CCM can aid the (1) variable partition process during block decomposition and (2) block manipulation process during generation. Considering the CCM, we denote the following attributes for each row (constraint) and column (variable). For a row in CCM A ∈ R m × n , x max is the maximum x -coordinate of nonzero entries, x min is the minimum x -coordinate of nonzero entries, and h x is the number of nonzero entries. Similarly, given a column in CCM, we denote the maximum y -coordinate of the nonzero entries by y max, the minimum y -coordinate of the nonzero entries by y min, and the number of nonzero entries by h y . we first give the following observations.

The nonzero entries in an M-Con often have a wide range of x -indices, i.e., large x max -x min and high standard deviation of x .

The nonzero entries in a B-Con often have a narrow range of x -indices, i.e., small x max -x min and small standard deviation of x .

For a Bd-Var, the nonzero entries in the corresponding column of CCM have a wide range of y -indices and high density (the proportion of nonzero entries in the column), i.e., large y max -y min and h y /m .

DB-Cons are defined as constraints containing Bd-Vars.

Using these observations, we define vector features for each row and column in the CCM. The row features include the standard deviation ( row_feat[0] ), the proportion of nonzeros (density, row_feat[1] ), and ranges of the x -coordinates ( row_feat[2] ). For a column in CCM, we consider the range ( col_feat[0] ) and density ( col_feat[1] ) of the nonzero entries. The pseudo-code of the classification algorithm is presented in Algorithm 1. For the hyperparameter used in this paper, we set ϕ 1 = 0 . 75 , ϕ 2 = 0 . 75 , ϕ 3 = 0 . 5 , ϕ 4 = 0 . 2 , and ϕ 5 = 0 . 5 , and set DB = True only in WA.

H.3 Implementation of Partition Algorithm for Block Units

In Section 4.2, we use GCG to reorder the rows and columns in CCMs and obtain an initial variable partition result from GCG. However, GCG sometimes cannot partition the variables well, such as the situation in CA where GCG fails to identify the blocks since the blocks in CA contain different numbers of variables. Thus, we need to refine the partition to obtain better ones. Specifically, we find that many blocks in CCMs contain regular lines (mainly horizontal and diagonal lines). Thus, we define a line detection for block identification. Splitting these lines by columns we can get the block units from the CCMs. To get block units by lines from the reordered CCM image, we design the following partition algorithm in Algorithm 2. The algorithm iterates over the points in the image and finds the endpoint in the columns of the lines mentioned above. With all the endpoints in columns, we can split one image into block units in columns. To keep the algorithm concise, we introduce two criteria, each representing a different detection method.

Criterion 1: horizontal line detection. This criterion can help to locate the endpoint of horizontal lines in the image precisely.

Criterion 2: diagonal line detection. This criterion can also help to locate the endpoint of diagonal lines in the image precisely.

We set ζ = 4 for CA and ζ = 3 for other benchmarks.

H.4 Details on Block Manipulation

Block manipulation We leverage the classification results for constraints and variables to aid the block manipulation process, in which the results can help us process more complex structures of CCMs beyond the three basic ones. For example, we can process the following structures in Equation 5, which is the combination of DB-Cons, M-Cons, and B-Cons.

Different types of constraints and variables have different manipulation methods. For example, when we apply the reduction and expansion operators, the manipulation of Bl-Vars can change the variable numbers, and the manipulation of B-Cons and DB-Cons can change the constraint numbers.

For the mix-up and expansion operators, the number of the introduced M-Cons m 1 may mismatch that in the original instance m 2 . We define a successful matching if m 1 ≥ m 2 If m 1 = m 2 , the operators can be performed well. Otherwise, we drop the last m 1 -m 2 M-Cons in the Block unit, such that the M-Cons can be perfectly matched.

Coefficient refinement In addition to the block structures, we have observed the presence of specific patterns in the coefficient values of real-world MILP instances. These patterns contribute to the overall problem structure and are worth considering during the instance generation process. For instance, as shown in Equation (6), we write two CCMs from different instances. In certain MILP instances (like FA), the nonzero coefficients within the B-Cons (rows of D ) remain consistent across the block units within a given instance (marked in the same color, blue or red, in the same instance), but may differ across different instances (marked in different colors). However, when applying mix-up or expansion operators to generate new instances, the introduced constraint coefficients from other instances can potentially disrupt this inherent coherence.

To address this issue, we have incorporated a constraint refinement component that focuses on preserving the distribution of coefficient values in such scenarios. Specifically, we define a "nontrivial constraint" as a constraint that contains values other than 0, -1, and 1. For each MILP instance, we identify the non-trivial constraints in the block units. For the k -th non-trivial constraint a k in the block unit, we then compute the mean µ ( i ) k and variance σ ( i ) k across the constraint coefficients and block units in this instance. During the instance generation process, whenever the refinement component is triggered (e.g., after mix-up or expansion operations), it samples the new constraint coefficients for the introduced blocks from a Gaussian distribution N ( µ ( i ) k , σ ( i ) k ) . This ensures that the generated instances maintain a similar distribution of non-trivial coefficients as observed in the original instances. The pseudo code of coefficient refinement is in Algorithm 3 and we activate this component in FA and IP.

I More Details on the Data and Experiments

I.1 Details on Bipartite Graph Representations

The bipartite instance graph representation utilized by MILP-StuDio closely aligns with the approach presented in the G2MILP paper [19]. This representation can be extracted using the observation function provided by the Ecole framework [60]. We list the graph features in Table 21.

I.2 Details on the Benchmarks

The CA and FA benchmark instances are generated following the process described in [8]. Specifically, the CA instances were generated using the algorithm from [16], and the FA instances were generated using the algorithm presented in [17]. The IP and WA instances are obtained from the NeurIPS ML4CO 2021 competition [32]. The statistical and structural information for all the instances is provided in Table 22.

I.3 Details on Graph Distributional Similarity

To evaluate the distributional similarity between the training and generated MILP instances, we compute 11 graph statistics [19], as detailed in Table 23. First, we calculate these statistics for both the original training instances and the generated instances. Then, we compute the JensenShannon divergence (JSD) D JS ,i , for each of the 11 statistics, where i = 1 , · · · , 11 . The JSD ranges from 0 to log 2 , so we standardized the values as follows: D std JS ,i = 1 log 2 (log 2 -D JS ,i ) . Finally, we obtain an overall similarity score by taking the mean of the standardized JSD values, score = 1 11 ∑ 11 i =1 D std JS ,i . The resulting score falls within the range of [0 , 1] , where a higher value indicates stronger distributional similarity between the training and generated instances.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We do so in the abstract and introduction of this paper.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: Please refer to Section 7

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: This paper does not include theoretical results.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Please refer to Appendix H.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [No]

Justification: We will release the code if the paper is accepted.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Please refer to Section 5.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: Please see Section 5.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Please refer to Appendix H.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: Our research conforms with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: Please refer to Appendix B.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: This paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: Please refer to Appendix H and I.2.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details on the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors note the specific block structures in the constraint coefficient matrices (CCMs) of MILP instances that are closely related to problem formulations, and propose a novel MILP generation framework, called MILP-StuDio. MILP-StuDio identifies blocks in CCMs and decomposes instances into block units that serve as building blocks for MILP instances. New instances are constructed by removing, substituting and appending block units from the original instance. This allows MILP-StuDio to generate approximate instances with high feasibility and computational hardness as existing instances. The authors claim that using instances generated by MILP-StuDio, the learning-based solver is able to significantly reduce the solution time by more than 10%.\n\nSTRENGTHS:\n1. The logical coherence and structural integrity of the manuscript are particularly noteworthy. The organization of the paper with a clear and effective division of content that facilitates understanding and engagement with the material.\n\n2. The code repository and appendices provided by the author are notable for their rich content and contribution to the replicability of the research. The codebase is meticulously organized, enabling a systematic understanding and replication of the study's procedures. The supplementary materials effectively supplement the paper's main content.\n\n3. The paper presents a novel and insightful approach to generating high-quality MILP instances through the exploration of block structures in CCMs. The visualisation of CCMs is particularly compelling, and effectively highlights the central idea of the paper by giving the reader an understanding of how MILP-StuDio's property of preserving block structure works.\n\n4. The robustness of MILP-StuDio is commendable. Even in the absence of clear block structures, as seen in non-structured MILP problems like the set covering problem, the method is able to generate instances that are still usable for downstream tasks. This demonstrates the adaptability and general applicability of the proposed framework.\n\n5. The experimental design is meticulously crafted and effectively demonstrates the advantages of MILP-StuDio. The comparison with both Bowly and G2MILP reveals that MILP-StuDio achieves higher similarity scores while preserving computational hardness and feasibility. Furthermore, the impressive results in improving the performance of learning-based solvers.\n\nWEAKNESSES:\nThe significantly lower similarity scores for the red and exp operators (Table 7), despite their successful performance in downstream tasks, raises concerns about the appropriateness of the similarity metric for evaluating MILP instance generation. The consistent decline in similarity scores with increasing modification ratios, while maintaining computational hardness and feasibility, further suggests potential limitations of the similarity metric. Could the authors kindly elaborate on their thoughts regarding the role of similarity in MILP instance generation and whether alternative metrics or considerations might be more suitable for this task?\n\nQUESTIONS:\nPlease reply to my comment listed in 'Weaknesses'.\n\nLIMITATIONS:\nNot applicable.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a method for generating instances for mixed\ninteger programming that mimic the characteristics of existing\ninstances. The main insight is that the performance characteristics of\nMIP solvers relate to the structure of the coefficient matrix of the\ninstance, with a few examples of such structure given in the paper. So\nthe new method is designed to discover such structures and replicate\nthem/combine them in the instances it generates. In experimental\nresults, the generator seems to produce instances whose behavior is\ncloser to that of the rest of the family.\n\nSTRENGTHS:\nI liked the overall idea. The paper identifies a feature of instances\nthat existing generators ignore, proposes a not-too-complicated way to\nuse it, and the results match the expectations.\n\nOverall a solid paper.\n\nWEAKNESSES:\nI think the latter two of the evaluation criteria are\nunder-explored. Specifically:\n\n- Improving the performance of learning-based solvers: I would expect\n  a lot more detail on how these solvers were trained. In general,\n  this aspect of the work is not explained very well, I think, and is\n  presented as if it should self-evident (which it is not, to me).\n\n- Hard benchmark generation: here there is just a very briefly\n  explained generation of a pool of instances that progressively gets\n  harder. I do not understand exactly why this test was performed like\n  this, what the original instances, nor do I find this a convincing\n  argument that the generator can produce instances of increasing\n  difficulty.\n\nFinally, I don't like the title of the paper at all. It feels labored\nand gimmicky. It is not a reason to reject, though.\n\nvarious places: operational research -> operations research\n\n110: motivated -> motivating\n\n(3): what is bold O?\n\nQUESTIONS:\nNo specific questions\n\nLIMITATIONS:\nAddressed sufficiently", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}

comments: SUMMARY:
This work presents a method for generating MILP instances by leveraging block structure decomposition. The primary aim is to address the challenges of generating high-quality MILP instances that preserve computational properties and structures of the original problems, thereby supporting the study and development of both traditional and learning-based MILP algorithms.

STRENGTHS:
Strength:
- The paper is easy to follow
- It is interesting to study instance generation as real-world MIP data is generally limited for the study of learning-based MIP algorithms.
- Although the idea of studying block structure is not new, the focus on block structure decomposition for MILP instance generation makes sense.
- The paper provides numerical experiments on multiple datasets and results looks promising.

WEAKNESSES:
Weakness:
- The paper lacks a detailed discussion on the criteria for selecting subgraphs during block decomposition and manipulation, which could impact the quality of the generated inst
- The proposed method only works on problems with block structures, which limits its broader applicability
- The number of instances on the test bed is very small.
- Code is not provided.

QUESTIONS:
Question:
- How does the choice of subgraphs during block decomposition and manipulation affect the overall quality and difficulty of the generated MILP instances?
- Could the proposed approach be potentially extended for more general MIP problems that without block structure?

LIMITATIONS:
Not applicable.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2024-07-15

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a novel framework for generating high-quality MILP instances. The proposed method, MILP-StuDio, leverages the block structures in constraint coefficient matrices (CCMs) of MILP instances to preserve computational hardness while allowing scalable and efficient generation. The framework consists of three main steps: block decomposition, structure library construction, and block manipulation through reduction, mix-up, and expansion. Experimental results demonstrate MILP-StuDio's ability to generate instances that improve the performance of learning-based solvers and maintain high similarity to original instances in terms of graph structural distribution and solving properties.\n\nSTRENGTHS:\nThe method takes a further look into the block structures to generate MILP instances that share more similarity with the original structures. The empirical results seem better than the previous method G2MILP.\n\nWEAKNESSES:\nThe proposed MILP-StuDio framework is tailored to highly artificial or synthetic problems with explicit block structures. The paper does not provide a convincing argument or empirical evidence showing that the framework can be generalized in real-world applications. Actually, in most of the real-world applications, most MILP instances do not have such clear block structures at all. MILP instances can have varying and overlapping structures that are not neatly decomposable. The paper is cherry-picking on specific synthetic/artificial datasets (all the four datasets) that have very structured blocks. It’s of no use to generate well-structured data that could be easily manually generated. The paper does not address how the method handles variability in block structures, leading to concerns about its robustness.\n\nQUESTIONS:\n1. Can you provide formal mathematical proofs to support the claim that the MILP-StuDio framework preserves the feasibility and computational hardness of generated instances? How do you theoretically ensure that the generated instances are representative of the original distribution?\n2. How does MILP-StuDio handle MILP instances that do not exhibit clear block structures? Can you provide empirical evidence or case studies demonstrating the framework's applicability to a diverse range of MILP problems beyond those with explicit block structures?\n3. How do you ensure that the instances generated by MILP-StuDio do not introduce biases that are absent in naturally occurring MILP problems? Can you provide an analysis comparing the distribution and complexity of generated instances with real-world instances?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-16"}