Paper: DIG-MILP: a Deep Instance Generator for Mixed-Integer Linear Programming with Feasibility Guarantee (ICLR-2024)

Reviewer comment (Experiments):
In the pipeline of the proposed method, I see that one node will be removed and the origin G is changed to G'. I wonder if this one-node change is too small to identify. In common MILP datasets, the size of nodes/constraints is more than thousands, and for the industry area, the size is even larger.

Author response:
We appreciate your insightful suggestion. DIG-MILP currently supports iteratively selecting and altering a single node in the graph. However, we acknowledge that selecting and changing multiple nodes simultaneously might be an alternative and potentially effective approach, and we would leave this as a future work.

Reviewer comment (Theory):
In section 3.2, the authors provide the feasibility guarantee for the proposed methods. But I am curious about unfeasible instances, can the proposed framework generate unfeasible MIP instances? In my view, it is okay when a MIP instance is infeasible and this situation is common in the real world.

Author response:
As mentioned in our universal response, we argue that given that almost all the existing scalable MILP datasets are feasible, it is naturally more interesting to generate feasible-bounded MILP instances. In real-world industrial applications, MILP problems modeled from real-world applications are usually feasible, or infeasible problems are transformed into feasible ones for solver application. Therefore, we argue that training models on feasible datasets to generate feasible MILP instances could be a justified approach.

Reviewer comment (Evaluation):
The metrics in the experiments are confusing to me. The authors mentioned: "The similarity score is derived from the Jensen-Shannon (JS) divergence (the lower the better) between each metric of the generated and original data", while in the caption of table 1 said: "The similarity score ↑ between the original and generated data." The similarity score is lower the better or higher the better? Actually, I wonder what we expect about the similarity score. I mean, more similarity could increase Authenticity while less similarity could increase Diversity. Both cases seem to mean something.

Author response:
We use JS-divergence where a smaller value indicates higher similarity of two distributions. The similarity score is calculated in the way like (1 - JS divergence), implying that a larger score is preferable. Despite aiming for diversity in new data, it's crucial that the new MILP instances represent problem types similar to the original. This relevance ensures their practical utility in tasks like solver tuning and as augmentation in ML training. We measure such similarity to the original problem distribution by comparing various statistical metrics on graphs, leading to our comprehensive scoring metric that incorporates these diverse measures.

Reviewer comment (Experiments):
About the datasets in the experiments, I see that for small datasets SC and CA the amount of instances is 1000, but for large datasets CVS and IIS the amount is less than 10. This gap looks very strange to me. Moreover, in table 2, does training and testing on merely 3 instances lead to an overfitting to these instances? I think the datasets CVS and IIS need to be further refined. I understand that they are selected from MIPLIB17, but MIPLIB17 at least has hundreds of instances. If the MIPLIB17 is too hard, I think NeurIPS 2021 ML4CO[1] datasets could be more suitable.

Author response:
The instances in MIPLIB2017 are of thousands of constraints and variables. In each training iteration, we sample a training batch by randomly choosing a node and delete it from the original bipartite graph, which leads to new training examples during the training process. Also for the downstream task, we train the model with the three original data with 20 newly generated instances by our model. We may consider larger datasets for further experiments.

Reviewer comment (Experiments):
Admittedly, I am not familiar with generative models, so my questions are mainly about this part. However, based on my experience, I think the dataset problem is more severe, as I do not think training and testing on only 3 instances is suitable for meeting the bar of ICLR.

Author response:
The instances in MIPLIB2017 are of thousands of constraints and variables. In each training iteration, we sample a training batch by randomly choosing a node and delete it from the original bipartite graph, which leads to new training examples during the training process. Also for the downstream task, we train the model with the three original data with 20 newly generated instances by our model. We may consider larger datasets for further experiments.

Reviewer comment (Novelty):
It seems that the literature review lacks of the most related paper that recently published in NeurIPS 2023 (Geng, Zijie, Xijun Li, Jie Wang, Xiao Li, Yongdong Zhang, and Feng Wu. "A Deep Instance Generative Framework for MILP Solvers Under Limited Data Availability." arXiv preprint arXiv:2310.02807 (2023).) This submission is highly similar to the above paper. Thus the authors are supposed to highlight the largest difference and improvement from the mentioned paper.

Author response:
1. **Difference and Improvement Over Paper [1]**:
As stated in our universal response, our work fundamentally differs from and is concurrent with [1]. We encourage you to refer to our universal response for a detailed explanation.

Reviewer comment (Presentation):
Please clarify more in-depth about the feature mentioned in Table 1, especially the all 0’s of y, r and the all 1’s about x, s.

Author response:
2. **Clarification on Features in Table 1**:
The all 1’s and all 0’s denote the feature encoding in the first dimension of the node input feature. We use 1 and 0 to classify the variable node and constraint nodes from the bipartite graph.

Reviewer comment (Theory):
It is not that dogmatical to claim that the boundness and feasibility of the generated instances can sure the authenticity of the produced data. The authenticity of MILP dataset can be defined in kinds of perspectives. Can you give more evidences to support your claim?

Author response:
3. **Authenticity of Generated MILP Instances**:
We assert that feasible-bounded MILP instances are 'authentic’ because 1)most existing MILP datasets for training are feasible, and 2) MILP problems in industrial applications are typically feasible. Infeasible problems are generally transformed into feasible ones before being solved. 
Therefore, we believe training models on feasible datasets to generate feasible MILP instances is appropriate. Please refer to the universal response for more detailed explanation.

Reviewer comment (Novelty):
The technical novelty of the proposed method is incremental, as the proposed method primarily use the existing VAE [1, 2] model to generate MILP instances. The authors may want to explain the technical novelty of the proposed methods in detail.

Author response:
We respectfully disagree with this assessment.Our paper is the first to apply a deep generative model for the generation of MILP instances with feasibility and boundedness guarantee. Please refer to the universal response for more details.

Reviewer comment (Theory):
The relationship between the theoretical derivations (i.e., Theorem 1) and the proposed MILP generation pipeline based on the VAE [1, 2] model is unclear.

Author response:
Our core contribution is to extract information from a well-designed generative space for producing new MILP examples with feasibility and boundedness guarantees. Theorem 1. in our paper provides theoretical backing for the guarantee, and the choice of generative model is not the key of our work.

Reviewer comment (Experiments):
The motivation of using the VAE [1, 2] model to generate MILP instances is unclear. The popular generative models include VAE [1, 2], Generative Adversarial Network (GAN) [3], and diffusion model [4]. The authors may want to explain the motivation of using the VAE [1, 2] model rather than the other generative models.

Author response:
We acknowledge the reviewer's point about discussing our choice of generative model. Briefly, generating graphs from scratch, especially larger ones, might face complexity and scalability issues, hence our choice of VAE to modify parts of MILP examples iteratively. We will amend our paper to include comparisons with other models.

Reviewer comment (Novelty):
The proposed method is similar to one recent work at NIPS [5]. The authors may want to explain the differences between their proposed method and the recent work [5] in detail.

Author response:
Our work fundamentally differs from [5] and is concurrent, as detailed in our universal response.

Reviewer comment (Experiments):
The experiments are insufficient. First, it would be more convincing if the authors could evaluate their method on large-scale benchmarks, such as instances from the MIPLIB with over 100,000 variables and 100,000 constraints. Second, the authors may want to evaluate their method on popular downstream tasks, such as learning to cut [6, 7] and learning to branch [8, 9]. Third, the baselines are insufficient. The authors may want to compare their method to G2SAT [10], which is the first deep generative framework that learns to generate Boolean Satisfiability (SAT) problems.

Author response:
Thanks for your advice. We find most studies in the ML for MILP community focus on problems of a hundred variables to a thousand in size. Our experiments on two sizable MIPLIB datasets might sufficiently demonstrate the essence of our method. 
The downstream tasks suggested by the reviewer could they themselves be independent research branches and are far removed from the core issue we address. 
Furthermore, the framework of DIG-MILP supports general MILP generation, unlike G2SAT which works on stratification problems, making it unsuitable as a baseline in our context.
We hope this adequately addresses the concerns and thank you for your valuable input.

Reviewer comment (Experiments):
Do the datasets contain unfeasible MILPs? Can the model learn to generate feasible MILPs without the feasibility gurantee? The necessity of this component is not demonstrated with ablation study.

Author response:
The short answer is the datasets do not contain infeasible MILPs. DIG-MILP indeed requires the training data with at least one set of solutions to generate feasible-bounded new instances. But we argue that such a requirement of training DIG-MILP solely on feasible datasets is both practical and appropriate. Please refer to universal response 3) for more detailed explanation.

Reviewer comment (Evaluation):
The proposed method does not performs better than random significantly.

Author response:
We respectfully disagree with the reviewer’s argument. From the aspect of the structural similarity (15%-20% higher similarity score) and performance on the downstream tasks (10% higher Pearson correlation score in solver tuning and much lower MSE in ML model training), the performance of DIG-MILP is much better than the random baseline.

Reviewer comment (Experiments):
Why not report the hyper-configuration results to show whether this method can benefit this task?

Author response:
Hyper-configuration results can be inferred from the provided figures, where configurations resulting in shorter SCIP solving times on both original and generated data are preferred. Direct comparison of optimal hyper-configurations is challenging, because it may not be proper to evaluate the similarity of two sets of hyper-configurations by their distance directly on Euclidean space.

Reviewer comment (Novelty):
What is the useness of the optimal value prediction task? Can the proposed method help the solving instead of just predicting the optimal value?

Author response:
Tackling the optimal value prediction task is a preliminary step in aiding MILP problems with ML models. We follow a general task setting introduced in [1]. Directly solving MILP is more challenging than predicting the optimal solution. In MILP generation task, one may first do well on the simpler task and then move towards more challenging tasks in the future work.