Title: Principled Data Augmentation for Learning to Solve Quadratic Programming Problems
Venue: NeurIPS 2025
Accepted: yes
Keywords: GNNs

linear programming

data augmentation
Abstract: Linear and quadratic optimization are crucial in numerous real-world applications, ranging from training machine learning models to solving integer linear programs. Recently, learning-to-optimize methods (L2O) for linear (LPs) or quadratic programs (QPs) using message-passing graph neural networks (MPNNs) have gained traction, promising lightweight, data-driven proxies for solving such optimization problems. For example, they replace the costly computation of strong branching scores in branch-and-bound solvers, thereby reducing the need to solve many such optimization problems. However, robust L2O MPNNs remain challenging in data-scarce settings, especially when addressing complex optimization problems such as QPs. This work introduces a principled approach to data augmentation tailored for QPs via MPNNs. Our method leverages theoretically justified data augmentation techniques to generate diverse yet optimality-preserving instances. Furthermore, we integrate these augmentations into a self-supervised contrastive learning framework, thereby pretraining MPNNs for improved performance on L2O tasks. Extensive experiments demonstrate that our approach improves generalization in supervised scenarios and facilitates effective transfer learning to related optimization problems.


Peer reviews:
comments: SUMMARY:
This paper presents a solid and comprehensive analysis of data augmentation for both linear programming (LP) and quadratic programming (QP) problems. The authors propose a rigorous theoretical framework for identifying valid data augmentations, grounded in dual problem formulations. Importantly, the method maintains computational efficiency, making it practical for large-scale applications. Leveraging these theoretical guarantees, the authors generate additional data to enhance both supervised learning and pretraining tasks, effectively addressing the issue of limited original data. Experimental results are compelling, demonstrating substantial performance improvements with comparable computational costs to training without data augmentation.

STRENGTHS AND WEAKNESSES:
Strengths:

1. The theoretical foundation is solid, and the proposed data augmentation method effectively addresses the primal problem under limited data settings.

2. The experimental results are strong, demonstrating the practical effectiveness of the approach across various benchmarks.

Weaknesses:

1. Can the author provide some experimental results on public datasets such as QPLIB?

2. The proposed data augmentation guarantees that the solution remains unchanged. I am curious whether there exists a way to design augmentations that introduce slight variations in the solution, in order to obtain more diverse augmented data?

QUESTIONS:
See Weaknesses.

LIMITATIONS:
None.
TITLE: Official Review
CLARITY: 4
IMPACT: 4
ORIGINALITY: 4
DATE: 2025-06-21

comments: SUMMARY:
This paper proposes a theoretically motivated data augmentation framework for LP/QP tasks by transforming the KKT system, enabling supervised and contrastive training.

STRENGTHS AND WEAKNESSES:
Strengths:

1. Clear theoretical foundation via affine KKT transformations.

2. Augmentations are label-preserving or solution-independent, enabling efficient integration into both learning paradigms.

3. Extensive experiments show strong gains under data scarcity and OOD transfer.

Weaknesses:

1. Experiments are limited to synthetic, fixed-scale problems; scalability remains unclear.

2. The number of augmented instances during training is not analyzed, making it hard to assess the effect of augmentation intensity.

3. In the LP setting, similar augmentation motivations have been explored (e.g., MILP-FBGen [1]), limiting novelty.

[1] MILP-FBGen: LP/MILP Instance Generation with Feasibility/Boundedness

QUESTIONS:
1. Please provide OOD evaluation in the supervised setting under data augmentation to isolate the benefit of augmentation itself, beyond contrastive pretraining.

2. Can the authors release code and data generation scripts to ensure full reproducibility?

3. How does performance vary with the number of augmented instances used during training? I suggest reporting results under different augmentation intensities.

4. How does this work compare empirically and conceptually with MILP-FBGen and similar LP instance generation work?

LIMITATIONS:
See above
TITLE: Official Review
CLARITY: 2
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-06-23

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a method for data augmentation to generate more training data for L2O methods. By solving a relatively small number of problem instances, the authors then use those solutions to generate additional problem instances whose optimal solutions can be efficiently computed using matrix operations on the solution to the original problems. This is accomplished by constructing affine transformations of compact representation of the KKT conditions for the original problem. The authors describe how to efficiently compute these transformations, as well as listing many example transformations that fit the required form. Results are compared against a baseline data augmentation method.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- This paper uses a clever idea to augment the data for L2O methods. Generating training data for these problems is very expensive, and the method is clear, principled, and theoretically grounded.\n- The statement of contributions is clear, and represents a small, but worthwhile step forward in L2O\n- The paper is well-written overall, and easy to follow.\n\nWeaknesses:\n- The related work section is comprehensive but not particularly useful. It fails to contextualize this work with respect to the citations in that section. The authors need to explain how the work is related to their method, not just that it is related.\n\nMinor:\n- Line 93, the edges should be $\\subseteq V(G)\\times V(G)$\n- Line 143 uses $B_{11}$ and $B_{21}$ before they are introduced\n\nQUESTIONS:\nHow does it compare to training on a more diverse data set? It seems as though this method would bias the network towards performing well on problems that are similar to those that have been augmented in the training data. Are there tradeoffs involved, and if so, what are they?\n\nThe focus was not on complete coverage of the space but wouldn’t there be an advantage to focusing on that as a goal? That is, what is the motivation for scaling variable coefficients, scaling constraints, etc.?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a principled approach to data augmentation tailored for QPs via MPNNs. The method leverages theoretically justified data augmentation techniques to generate diverse yet optimality-preserving instances. The paper presents various forms of data augmentation for QP problems and provides proofs to ensure the validity of the generated data. The value of the proposed algorithm is demonstrated from multiple perspectives, including improvements in supervised learning, supervised finetuning, generalization, and practical computational overhead.\n\nSTRENGTHS AND WEAKNESSES:\nThis is a mathematically grounded data augmentation framework for LPs and QPs, and detailed derivations and proofs for their augmentation framework. \nThe paper provides extensive experiments. \nThe framework emphasizes computational efficiency, and the paper discusses how structured matrices (e.g., diagonal matrices) reduce computational overhead, which is supported by runtime experiments.\n\nThe experiments focus on synthetic datasets for LPs and QPs, which may not generalize to real-world optimization problems.\nWhile the authors acknowledge this limitation, the lack of results on benchmarks reduces the practical impact of the work.\nSome proposed transformations require access to the primal-dual optimal solutions of the original problem, which may not always be available in practice. This could limit the framework’s applicability. \nThe mathematical sections (e.g., Section 2.1) are dense and may be difficult for readers without a strong optimization background to follow.\nOnly applied to convex problens. \nExperiments focus on synthetic datasets. \nThe experimental results lack sufficient discussion in the main text.\n\nQUESTIONS:\nHave you considered extending the method to non-convex QPs or other non-convex optimization problems? How does the convexity assumption limit practical applications currently? \n\nAre there plans to test the method’s performance on more real-world datasets?\n\nis there a way to achieve fully solution-independent data augmentation?\n\nWhy was the temperature parameter $\\tau = 0.1$ chosen for contrastive learning? Have you conducted sensitivity analysis on this parameter?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}