Principled Data Augmentation for Learning to Solve Quadratic Programming Problems
Authors: Chendi Qian
Christopher Morris
Venue: NeurIPS 2025 spotlight ()
Keywords: GNNs
linear programming
data augmentation
Primary area: deep_learning
TL;DR: We provide principled data augmentations for learning to optimize.

Abstract: Linear and quadratic optimization are crucial in numerous real-world applications, ranging from training machine learning models to solving integer linear programs. Recently, learning-to-optimize methods (L2O) for linear (LPs) or quadratic programs (QPs) using message-passing graph neural networks (MPNNs) have gained traction, promising lightweight, data-driven proxies for solving such optimization problems. For example, they replace the costly computation of strong branching scores in branch-and-bound solvers, thereby reducing the need to solve many such optimization problems. However, robust L2O MPNNs remain challenging in data-scarce settings, especially when addressing complex optimization problems such as QPs. This work introduces a principled approach to data augmentation tailored for QPs via MPNNs. Our method leverages theoretically justified data augmentation techniques to generate diverse yet optimality-preserving instances. Furthermore, we integrate these augmentations into a self-supervised contrastive learning framework, thereby pretraining MPNNs for improved performance on L2O tasks. Extensive experiments demonstrate that our approach improves generalization in supervised scenarios and facilitates effective transfer learning to related optimization problems.

Principled Data Augmentation for Learning to Solve Quadratic Programming Problems
Chendi Qian
Christopher Morris
Department of Computer Science
RWTH Aachen University
Aachen, Germany
chendi .a www.haaa.......aahee......
Abstract
Linear and quadratic optimization are crucial in numerous real-world applications, ranging from training machine learning models to solving integer linear programs. Recently, peaming-to-oppimizz methods (L20) for linear (LPs) or quadratic programs (QPs) using message- message-passing graph neural networks (MPNNs) have gained traction, promising lightweight, data-driven proxies for solving such optimization problems. For example, they replace the costly computation of strong branching scores in solvers, thereby reducing the need solve many such optimization problems. However, robust 1.20 MPNNs remain challenging in data-scarce settings, especially when addressing complex optimization problems such as QPs. This work introduces principled approach to data augmentation tailored for QPs via MPNNs. Our method leverages theoretically justified data augmentation techniques generate diverse yet optimality- optimality-preserviinn these augmentations into a self-supervised contrastive learning framework, thereby pretraining MPNNs for improved performance on L20 tasks. Extensive experiments demonstrate that our approach improves generalization in supervised scenarios and facilitates effective transfer learning related optimization problems.
preserving instances. Furthermore, we integrate
Introduction
Linear and quadratic optimization problems fundamental problems in machine learning, operations research, and scientific computing [Boyd and Vandenberghe [2004] Nocedal and Wright 1999]. Many real-world applications, such resource allocation, logistics, and training machine learning models, rely on efficiently solving large-scale linear programming (L.s)) and quadratic programming (QPs). In addition, they play key role in state-of-the-art integer-linear optimization solvers, allowing the computation of lower bounds, and are the basis for crucial heuristics such as strong branching for variable selection [Achterberg et al. 2005).
Recently, machine learning techniques, particularly pressage-passing graph neural networks (MPNNs) [Gimmer al. 2017 Scarselli et al.) 2008) have been explored for learning-to-on -t-omiimize (L20) approaches, aiming to learn to solve LPs and QPs data-driven fashion [Bengio et al., 2021 Cappart et al. 2023), enhancing solver efficiency and improving generalization across different problem instances. For example, (asse et al. [2019) used MPNNs to imitate the costly strong branching score for variable selection in integer-linear optimization solvers, which requires solving many LPs during the solving process. However, training robust models for 1.20 remains challenging due to the scarcity of labeled data, especially for complex optimization formulations such QPs.
In response to this challenge, self-super learning (SSL) has emerged as powerful paradigm for pretraining models on large, unlabeled datasets [Liu et al. 202 Contrastive learning, key approach in SSL, has demonstrated significant success in graph- based tasks by leveraging data augmentation
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

techniques to create diverse training examples You et al. [202). Despite advances in contrastive learning and L20 with MPNNs, gap exists in integrating these two paradigms for LPs and QPs. Additionally, designing effective and principled data augmentation strategies for IPss and QPs remains non-trivial due the structural constraints and optimality conditions inherent in these problems.
Present work In this work, we propose principled approach to data augmentation for LPs and QPs, specifically designed to enhance the performance of MPNN- L20 methods for such problems; see Fig. foo a high-level overview. Concretely, our contributions are as follows:
1. We introduce a set of novel and theoretically grounded augmentation techniques for LPs and QPs that preserve optimality while generating diverse training instances;
2. We apply these augmentations in both supervised and self-supervised settings, including contrastive pretraining of MPNNs to enhance downstream performance;
3. We empirically evaluate our approach on synthetic and benchmark datasets, showing that pretraining with our augmentations improves generalization and transferability across problem classes.
By bridging the gap between data augmentation and L2O for LPs and QPs, our work offers new perspective on enhancing neural solvers through supervised learning and self-supervised pretraining. The proposed augmentations improve generalization in data-scarce settings, enabling more efficient and robust learning- based optimization methods.
1.1 Related work
This section reviews relevant literature on L20,, graph data augmentation, graph contrastive learning, and synthetic instance generation for LP and MILP problems.
MPNN and L20
Message Passing Neural Networks (MPNNs) Gilmer et al. 2017 Scarselli et al.
2008 have been extensively studied and are broadly categorized into spatial and spectral variants.
Spatial MPNNs [Bresson and Laurent 2017 Duvenaud et al. 2015 Hamilton et al. 2017 Veličkovid
al. 2017 Xu et al.] follow the pesssag-ppsssing paradigm introduced by Gilmer et al. [2017]
MPNNs have shown strong potential in learning to optimize (L.20). A widely adopted approach
represents MILPs using constraint-variabe bipartite graphs (Chen al. 2022 Ding et al. 2020 Gasse
et al. 2019 Khalil et al. 2022 Qian et al.] [2024a]. Recent work has also aligned MPNNs with various optimization algorithms, including interior-point methods (IPMs) [Qian al., 2024a Qian and Morris
2023), primal-dual hybrid gradient (PDHG) [Li et al., 2024a]. and distributed al algorithms [Li et al.
2024b 2025). From theoretical perspective, several studies have analyzed the expressiveness of
MPNNs in approximating solutions to linear and quadratic programming [Chen et al.) 2022 2023
2024b a
Wu et al. 2024
Graph data augmentation Graph data augmentation is central to learning- -ased optimization,
especially under data scarcity. Common strategies include feature-wise perturbations YYou al.
2020 Zhu al.) 2020, (Suresh et al. |02 Zhu et al. [021 Thakoor et al. 2021 Biclak et al. 2022
Hu et al. 2023). J. structure- wise modifications such as node dropping and edge perturbation RRong
al. 2019]
and Khasahmadi, 2020, Zhu et al. 2020 Qiu et al. 2020 Papp et al. 202 Zhu
ct al.
2021 Bielak et al. 2022 Hu et al.) 2023), J. and spectral-domane augmentations Yang et al.
2024a
Liu et al. 2022a Lin et al. 2022 Wan et al. 2024), the latter aligning with recent works like
S3GCL
LL. Learning-bart based approaches Yang et al. 2020 Zhho et al.] 2021 Liu et al. 2022b] Yin et al.
2022
You et al. T Wu et al. 2023) further enable adaptive augmentation via trainable modules.
Complementary perspectives include graph rewiring [Topping et al. 2021 Karhadkar et al. 2022
Arnaiz-Rodríge
et al. 2022 Qian et al. 2023 Gutteridge et al. 2023 Barbero et al. 2023 Qian
et al.
2024b) and structure learning JJin et al.] 2020 Liu et al. Zou
Zou et al.) 2023
Zhou et al.
2023
Fatemi al. [202). both of which can be viewed as task-specific augmentation techniques. For a
broader overview, including robustness and self-supervised settings, see [Zhao et al. 2022 Ding et al.
2022
Graph contrastive learning Graph augmentations also play central role in graph self-supervised learning (SSI.), which aims to learn transferable representations without labeled supervision. Graph SSL methods are categorized into contrastive, generative, and predictive approaches [Wu et al. 2021
2

---

Q
lide surnigg
NF-Noe
sh.
Asiti
Arats
Anite is
Azota
Anata is
3
Pretis Liggaatoio
Auti
Alats
Andta S
Saoea trancinu
Figure 1: Overview of our framework for principled data augmentation for quadratic optimization problems. Given QP instance, apply transformations (e.g., adding/removing/saling variables constraints; see Section (.2) to generate new instances, thereby augmenting the training dataset. We then use standard supervised learning or contrastive learning train MPNN.
We focus on contrastive ones, which operate at three levels: global-to-glll global (G2G), local-to-locad (L2L), and local-to- global (L2G). G2G methods generate contrasting views of entire graphs in GraphCL You et al., 2020 with structural perturbations, extended by CSSL [Zeng and Xic 2021]
GCC [Qiu al. 2020 using random walks, AutoGCI. YYin al. 2022] with learned augmentations,
and AD-GCL Suresh et al. [021] with an adversarial objective. The only contrastive method
targeting LPs is Li al. [202cc]. which adopts CLIP-style [Radford af., [201] formulation. L.2L
methods contrast node-level views, e.g., GRACE (Zhu et al. 2020). GCA [Zhu et al., 2021]. Graph
Barlow Twins [Bielak al., 2022], BGRL [Thakoor al. [022]], J. and REGCL [Ji et [2024), with
S3GCL [an et 20244 introducing spectral views. L2G methods such as DGI Velicković et al.
2018 MVGRL [Hassani and Khasahmadi 2020). and InfoGraph [Sun al. 2019
contrast local and
global representations to maximize mutual information.
While general-purpoot augmentations well-studied, few works target L20 problems. Duan et al. [022] designs satisfiability -ereeiiii--eeiinnggggg augmentations for SAT problems, and Huang et al. 2023 samples neighborhoods around expert solutions. A concurrent work [Zeng al. [2025] explores constraint permutations in MILPs. Compared with these works, we propose efficient, principled transformations, tailored for supervised and contrastive learning in linear and quadratic programming.
Instance generation While random instance generators exist for LPs and MILPs Gasse et al. 2019 2022]. data scarcity has driven model- approaches, including stress testing [Bowly] 2019].
VAES Geng et al., [2023] and diffusion models (Zhang et al. 2024]. Other works reconstruct or adapt
instances Wang et al.) 2023 Guo et al., 2024 Yang et al.) 2024c , generate them from code
substructures [Li et al. 2024c Liu et al. 2024]. In contrast, our approach is model-free. mathematically
principled, and generates new instances through transformations with analytically tractable solutions.
1.2 Background
We introduce notations and define MPNNs, LPs, and QPs in the following.
Notations Let N = {0, 1, }. For n 2 1, let [n] == to denote multisets, i.e., the generalization of sets allowing for multiple instances of each of its elements. A graph G is a pair (V(G), E(G)) with finite sets of vertices nodes V(G) and edges E(G) C {(,, v) C V(G) x V(G) u v). For ease of notation, we denote the edge (u, v) in E(G) by vu,v) or (v, u). The neighborhood of a node is denoted by N(v) = {u V (r,, E E}, and its degree is [N()] An attributed graph augments each node E V with feature vector (v) E Rd yielding a node feature matrix H E n where H, (v). The adjacency matrix of G is denoted A E (0, xxx with A, = 1 and only (i. E E. Vectors x E Rd are column vectors by default.
C N. We use ft..
LCQP In this work, we focus special QPs, namely linearly-comstrrineet quadratic programming (LCQP), of the following form,
min x Qx + cT s.t. Ax < b. RcR
(I)
Here, an LCQP instance is tuple (Q, A, b. e), where Q E Raxx and e R are the quadratic and linear coefficients of the objective, A E pmmx and b R form the inequality constraints.

---

Note that the bounds of variables, e.g., 2 O, can also be merged into the constraints. We assume that the smmetric and quadratic matrix Q is positive definite (PD), denoted Q > so that the problem is convex and has unique solution. The optimal solution x² is a feasible solution such that -a=QQr + et 2 = Qx' cTx=,, for any feasible æ. The corresponding dual variables for the constraints are denoted as X By setting the matrix Q to zero matrix, we arrive at LPs,
min cº x s.t. Ax S b.
(2)
#ER"
Considering the QP of Eq. (T), the optimal primal-dual solutions must satisfy the Karush- Carush-Kuhn-a Tucker (KKT) conditions (Nocedal and Wright 1999 p. 321],
Qx' ATX
(3a)
Aæ" < b
(3b)
X 0
(3c)
X (A. b.) = 0.
(3d)
Define slack variables s" := b Ax", the inequality constraints become equalities Ax" = b - 8" The KKT conditions can then be compactly written as,
Q A
AT
x
(4)
b 8
where the inequality and complementarity conditions Eqs. (3c) and (3d) implicitly encoded via 8* 2 O and 8 X = O. In practice, we can partition the inequality constraints into active ones a. b, and inactive ones Aaæ < ba. where the slack variables satisfy s 0 and 8 0.
MPNNs for LCQPs Representing LPs and QPs with MPNNs has been explored in prior work. For example, Chen et al. [022] models LPs using bipartite constraint-variable graph, and Chen et al. 2024a extends this to QPs by adding edges between variable nodes. We adopt the setting of Chen al. 2024a Given LCQP instance I, we construct graph G(I) with constraint nodes C(T) and variable nodes (I). Edges between (')) and (1) are defined by nonzero entries of with weights Ace, for V(T),c C(T); and edges between variables defined by nonzero Q U, ((T). Node features are H. == reshape(b) E R=x1 for constraint nodes and H. == reshape(c) E Rxx
for variable nodes. MPNNs learn vectorial representation of each node in graph by aggregating information from its neighbors, i.e.,
( UPD (t) h (t-1)
Σ (EN(C)NV()
Acch(!-1))
(5)
h = UPD( ( (1-1)
Σ
Qu
(t-1)
Σ
Acch
(E
cEN(w)CC()
followed by pooling function and readout function to predict the objective,
z == POOL
Σ (T).
Σ
h neT)):
obj(!) = READOUT(2:).
(6)
EEV(T)
eEC(T)
2 Principled data augmentation for LCQPs
We propose set of principled transformations for LCQPs data augmentation. Let I denote a of LCQP instances. We consider transformations T': I + I that perturb the problem structure while allowing efficient computation optimal solutions through simple algebraag operations, without requiring solver. Our core idea is to construct affine transformations of the KKT system Eq. (4) that yield valid KKT conditions for new problem. We notice that applying linear mapping M Eq. does not change the equality, i.e.,
AT
x A
c b s*
M
M
We consider only feasible and bounded problems where x" and X are not both zeros. If they are, then c = and b 2 O, reducing the problem to unconstrained QP with Q > 0 and trivial solution æ = 0. which we exclude.

---

holds for all choices of transformation matrix M. To make the transformed problem valid, we need a MT the right and fürther require right (pseudo) inverse (MT)² with MT (MT) I. To ensure (MT)² exists, the matrix M needs be full column rank. Now have
M
Q
AT
M
(MT)
x
M
c 8*
X
Finally, we could add a bias term on both sides, and arrive at the general form,
M
AT
M M +B) (MT)
+
x=
8* + B.
(7)
X
Here, the matrices M and B define the transformation on the problem parameters (Q, b, c). with if it exists, recovers the optimal
x²' B. The quantity (MT)²
x
B chosen to satisty B (MT))
X
X
solution of the transformed problem. Thus, compute the new solutions using only matrix operations, without solving the transformed LCQP.
2.1 A framework for valid transformations
An ideal transformation should be expressive, valid, and computationally efficient. Here, expressivity refers to its ability to generate a wide range of problem instances from given one, while validity ensures the transformed system remains valid KKT system. We discuss these properties in detail below.
Expressivity Intuitively, transformation more expressive than another T' it maps an initial problem to a superset of the targets T" can reach. We observe that the transformation in Eq. (7) has maximal expressivity. For example, setting M O, with proper and /3 it can recover any target instance /' = (Q', A', b, c') from any source J, regardless of dimensions. This flexibility relies on the bias term, without which the transformations are limited. For instance, a low-rank A cannot map to a higher-rank A'. In practice, full expressivity is unnecessary, as our practical objective is not to span the entire space of QP instances, but to generate meaningful, diverse variants for data augmentation.
Validity Transformation following Eq. 7 under some conditions, ensure that the transformed system remains a valid KKT system, which we explore below. We can write the matrices in block
MMn
M12
Bn B12
matrix form M
and B
such that dimensions for A and Q
M21
M2
B21
B2
matched. Using straightforward calculations, we have
T
Ti
M1
M12
Q
AT
M
MT
Bu
B
+
Ta
T22
MA
M
A
M
M
BN1
Bzg
T M + M12AM + AT B11
T12 = MIQM,, + M1AAM AT, + B12
T21 Ma1 QM + MzAMM + M A ATM + B21
T22 Mz1 QM,, + MzAM, M AT A + B22-
Q
A
xx
cc'
So that the transformed equation Eq. (7) is valid KKT form
A'
x
= b' " it
requires Ti2 = T and we further require Tn be another positive definite matrix, and T22 must hold.
For inequality constraints, the transformation matrix M22 must satisfy certain conditions to enable efficient computation of the new solution. These conditions not required for feasibility optimality but reflect our goal of avoiding QP solving. To isolate M22. we simplify the system by discarding the bias terms B, B, setting M2 M21 to zero, and fixing M I. The resulting transformed KKT system is,
Q
ATM
22
I
O
I
-c b
(8)
M
(ML)
X
M
s*
5

---

This system indicates that the primal solution x' remains unchanged, while the dual solution is mapped to (MZ) X'. The following proposition establishes the conditions under which this transformation preserves the solution and avoids re-solving the LCQPs; see Appendix [DI a proof.
Proposition 2.1. Let == (Q. c) be a LCQP instance with optimal primal-duct solution x" X* Consider transformation defined by T(T) = (Q, M22A, M22,, c). Then the transformed problem
N11
N12
preserves the primal solution x² if and only if M2 takes the block form
to match the
N21
N22
dimensions of active and inactive constraints, where Nu has full row rank equal to the number of active constraints, N128; 0, N225. 0.
In summary, the conditions on M, B, 3 for the validity of the new problem is as follows,
MnQM] + MºAM] MIATM, + B1 >
12 Q + M24MM
M21 AT ATM + B22
B12 B
T 3, and
(
X
M22 satisfies Proposition 2.1
Computational efficiency In general, satisfying the full transformation structure in Eq. (9) is chal- lenging. particularly when arbitrary M, B and 3 are involved. However, we notice the multiplicative term M and bias terms B, 3 can be decoupled. That is, we can design transformations with the bias terms B, B, e.g. Appendix [C.2 but we mainly focus on dropping them, setting M1 and M21 to zero matrices, and investigate the design space of M1 1 and M22. We will abbreviate the subscripts of M.. and we target at designing transformations of the form
Mi
M2
0
Q A
AT
MI
MI
(MT)
0
(M
0
x
A
=
M
0 M2
c
(10)
We notice the commutativity and the decoupled nature,
M
0
I
0
M.
0
M
I
0 M₂
M2
=
0
M2
0
1
=
0
0
which enables us to design M1, M2 separately, and merge them later.
The design space in Eq. 10 is flexible but constrained by the need to compute pseudo- pseudo-inverses. For M, E RR' there are three cases: (i) n' = n linear reparameterizztiio, (ii) n' < dropping variables, and (iii) n' > n adding variables. While (ii) is often ill ill-posed since MT does not have full column rank and lacks right inverse, some special cases are still feasible, as we will show. More broadly, computing (M,))
M1 (M. M1) requires full column rank and typically costs O(n"). To improve efficiency, we focus on structured matrices, for example. diagonal matrices, as a particular case, enable both efficient inverse calculation and generation (()) time. These considerations motivate the use of diagonal or structured Mj and M, for scalable data augmentation. To formalize efficiency, we introduce two notions of efficiently computable transformations. The first covers transformations whose solutions can be computed in linear time.
Definition 2.2 (Efficiently recoverable transformation). For an LCQP instance 1 := AQ E and its primal-dual solutions x", Xx, a transform T E T is efficiently recoverable, if the new solutions of the transformed instance I' T(I) can be obtained within linear time O(n).
The second one, motivated by unsupervised settings, tightens this by requiring that the transformation be independent of the original solution and focuses on generating structurally consistent instances rather than solving them.
Definition 2.3 (Solution-nndependen transformation). Given LCQP instance I == (Q, ( transformation T E T is solution-independen if the solution of the transformed problem can be obtained without solver, and the transformation parameters do not depend on the original optimal solutions x" X*

---

2.2 Example transformations
In the following, we discuss various transformations that fit into the above framework.
Removing idle variables As discussed above, when M1 does not have full column rank, the pseudo inverse (MF)' typically does not exist, making such transformations ill-posed. However, there is special case which we can safely remove a variable, specifically, when it is idle because its optimal value is zero, resulting in the following proposition.
Proposition 2.4. Let I == (Q, A, b, e) be an LCQP instance with primal-duat solution x", X. Then variable x' can be removed from the problem without affecting the optimal values of the remaining variables if and only if x = O.
Removing inactive constraints Like the variable removal case above, constraint be removed under certain conditions by introducing a wide identity matrix M2 that selectively excludes the corresponding row.
be an LCQP instance with optimal primal-dual solution x", X* Then constraint of the form a' S b' can be removed from the problem without affecting the optimal solution if and only if it is strictly inactive, i.e., axxx'
Proposition 2.5. Let 1= == =
x < b'
The variable and constraint removal transformations are efficiently recoverable, as the remaining solutions are unchanged and need no recomputation. However, they are not solation-indepeedene since identifying removable components requires access the primal solution. Moreover, we apply heuristic on problem instances to select inactive constraints, as described in Appendix C.3
Scaling variable coefficients A natural class of transformations involves scaling the coefficients associated with individual variables. Specifically, scaling the j-th column of A and the j-th entry of e by a nonzero scalar aj, while updating Q2, by agaty i.c., T(1) QM, AM, b, M) with a diagonal Mi preserves the structure of the QP. Under this transformation. the optimal value remains unchanged, and the solution ær is rescaled by 1/a,. This transformation is efficiently recoverable, as the new solution can be obtained directly from the original in (n) time. Moreover, it is also solution- innroonnddddddnen,,. as it does not require access to the original solution, but only knowledge of how to compute the new one.
Adding variables When n' > n, the transformation effectively adds new variables to the problem and linearly combines existing ones. Without loss of generality, we consider adding single new variable
I
by choosing transformation matrix of the form Mi =
with q E IR." being an arbitrary vector. 9-
Q
Qq QTQq]
This yields a new positive definite quadratic matrix
We can find the pseudo inverse
TTQ
I qT
(M))² = Mi (M)
(I --_ which can be computed with Sherman-Morrioon
formula [hhermen and Morrison 1949]. In this case, the primal solution of the original variables does not remain the same. Interestingly, due the structure of M,, another valid pseudo-inverse is
I
which is special case indicating that the added variable has zero contribution to the (MT)* OT
solution. This corresponds to the reverse of the variable removal transformation discussed above. Proposition 2.6. Let I = (Q, c) be an LCQP instance with optimal solution x" X' Define the transformation T(I)= = (MQQ AMI b, Mic), where M1 = qT Then the transformed problem has optimal primal solution xx') if and only if the new variable x²' = 0.
This transformation is both efficiently recoverable and solution-indee indeeendentd... Moreover, there also exist other implementations of variable addition in the form of Eq. (7). with a bias term. Please refer to Appendix C.4
Scaling constraints Similar to the variable scaling transformation above, we can scale the constraint coefficients by fixing Mi I and letting M2 be square diagonal matrix. We assume all diagonal
7

---

entries of M2 are positive. If any entry is zero, the transformation reduces to inactive constraint removal above, or an active constraint is removed, and the problem will be relaxed. If any entry is negative, the corresponding inequality direction will be flipped, and the problem's 's solution may change. Specifically, the transformation would be T(T) = (Q, M2A, M2b,
Under this transformation, the new dual variables given by X M, X which can be computed in linear time since M2 is a diagonal matrix. The primal solution remains unchanged. This transformation is both efficiently recoverable and Dolation-independenn
In Appendix we outline additional transformations, including constraint addition Appendix
2.3 Using data augmentations
Having introduced efficient data augmentation methods for LCQPs and LPs, we now describe how they can be integrated into different training pipelines. Our target task is graph regression predict the objective value from a graph representation. Depending on the setting, we can either (1) use solution information to generate supervised labels for related problems or (2) apply solution-independent augmentations for contrastive pretraining without solutions.
Supervised learning Given training set and set of augmentation methods, we dynamically generate additional training instances during optimization. We randomly apply selected augmentation or a combination of augmentations at each iteration, and train the MPNN using the supervised loss on the predicted objective value.
Contrastive pretraining We perform self-supervised contrastive learning on the entire dataset without touching the solutions of the problems, using the NT-Xent loss (Chen et al. 2020). Specifically, data instances, we generate two augmented views of each instance using for a mini-batch transformations, resulting in 2NV data instances. We consider the two views of the same instance (i as a positive pair, and the other 2(N 1) samples as negative. The similarity between embeddings is measured by 2-norm-normalized cosine similarity sim (u, v) == | For each instance and its positive sample j. we have the NT-Xent loss on the pooled representations 2, #z from Eq.
=T.
exp (sim(z,, 2z)/T)
log
(11)
2N k=
1(k + exp (sim(zi, zx)/T)²
where > 0 is temperature hyperparameter.
3 Experimental setup and results
To empirically validate the effectiveness of our data augmentations, we conduct a series of experiments, answering the following research questions?
Q1 Do our augmentations improve supervised learning, especially under data scarcity?
Q2 Are the augmentations effective in contrastive pretraining followed by supervised finetuning?
Q3 Does the pretrained model enhance generalization on out-of-distributio (OOD) or larger datasets?
Q4 What is the practical computational overhead of the augmentations?
As LPs are a special case of QPs, we evaluate them separately. We generate 100000 instances for each dataset, each with 100 variables and 100 inequality constraints, and split the data into training, validation, and test sets with 8 1 ratio. To study performance under data scarcity, we partition the training set into multiple disjoint subsets in each run, each containing either 10% or 20% of the full training data. Models are trained independently on each subset, and results are aggregated across all partitions. Hyperparameters across all methods. Specifically, we use a 6-layer MPNN followed by 3-layer MLP with 192 hidden dimensions and GraphNorm (Cai al. 2021]. All experiments are conducted on a single NVIDIA L40S GPU. We evaluate performance using the mean relative objective error (in percentage) over the test set, defined as 1/IP) EiED (Tbj obj" (T))/ (7) 100%, where D denotes the set of
tuned using supervised training on the full training set and fixed
'The repository of our source code can be accessed at :ttps://gituee htt////ttthuutueeeg com/ chendiiaang Data- Aitaa-Auggenttaion--ooo wtw.tttttattttoo-o--aoaaaiaaeaa Sesarning-to--ptaaee
8

---

instances, obj" (T) is the optimal objective value, and ob)(T) is the predicted objective value. We repeat the experiments five times with different seeds and report the mean and standard deviation.
Supervised learning To address Q1, we investigate the impact of data augmentation on LPs and QPs in supervised learning setting. We evaluate performance on synthetically generated datasets following the procedure described in Appendix G Models are trained with a batch size of 32 for up to 2000 epochs, with early stopping after 200 epochs of patience. We evaluate performance under data scarcity by training models on subsets containing 1 10%. 20%, and 100% of the training data. As baselines, we apply the augmentations proposed by You et al. (020]: node dropping, edge perturbation, and feature masking. In addition, we evaluate each of our proposed augmentations separately and in combination. Hyperparameter details for all augmentations are provided in Appendix [F] Results, shown in Table [
reveal that the augmentations from You et al. (2020) fail to improve performance consistently and can even be detrimental compared to training without augmentation. In contrast, all of our proposed methods consistently yield better performance, and combining augmentations further amplifies the improvement up to 62.6% on LP with 20% of training data, aligning with empirical evidence observed in You et al. 20200]
Table I: Supervised learning performance on LP/QP datasets with and without data augmentation under different levels of data scarcity. The best- best-performing method is colored in green, the second-best in blue, and third orange. Our proposed augmentations consistently improve performance.
LP
QP
Angmentation
10%
20%
100%
10%
20%
100%
None
8.539_0000
6.149
2.784
5.304.020
3.567
1,240
Drop node
8.618_0112
2.131
6.355 1.65
4.867 1o.1te
2.59410n87
Mask node
8.979_0113
1.
3.21620149
5.865
4.005
1.347
Flip odge
7.794=0115
6,503=0 125
4.358=0m:
5,485
20,13
+0.01
Drop vars.
4.996-00
3.484-0004
1,484
4,232an 195
2.374
a.835
Drop cons.
5.563
2000
1666
3.407 19
2.104..0m
1006
Scale cons.
=,491
3.872 136
1.61220012
3.909
2.448
0.722a0055
Scale vars.
4.682-0.62
3.208=0.116
1.241
3.790as.013
2.468s0.124
0.649
Add cons.
6,245-0011
4,299
1.878
3,731
2.304
0.814
Add vars.
6,565=0.18
4.696=0.01
2.188
4.475
2.885
1,02 #0
Combo
3,465=0045
2.300=a03
1.05 100m
2.434
1.532a0mm
0,542a0013
Contrastive pretraining To address Q2, we evaluate whether contrastive pretraining can improve supervised tine-tuning potentially under data scarcity. We adopt supervipe setting: a small subset (10%, 20%, 100%) of the training data is labeled, while the whole training set available as unlabeled data. During pretraining, we use the complete unlabeled training set and train only an MPNN backbone without a prediction head, following the deployment described in Section 2.3 We pretrain for 800 epochs with batch size of 128, and set 0.1. To assess pretraining quality and pick the best set of hyperparameters, we use linear probing [Veličković ct al. 2018). training only a lincar regression layer top of a frozen MPNN to efficiently evaluate feature quality. For finetuning, we follow Zeng and Xie 2021]. attaching an MLP head and jointly training it with the MPNN using supervised regression loss, essentially the same setup as supervised learning, but initialized from pretrained model.
As baselines, we consider several graph contrastive learning methods. GraphCL You et al. 2020
generates views via node dropping, edge perturbation, and feature masking. GCC [Quu et al. 2020
samples random walk subgraphs. IGSD [Zhang et al. [2023] uses graph diffusion [Gasteiger et al. combined with model distillation. MVGRL Hasssai and Khasahmadi [2200] also employs diffusion- based views but performs contrastive learning at the graph-node level. Additionally, we include mutual information maximization methods such as InfoGraph [Sun et al.) 2019] and DGI /Velčković et al.
2019
2018). Beyond contrastive methods, we evaluate the generative SSL method GAE Kipl and Welling 2016 which reconstructs graph edge weights.
As shown in Table GraphCL and GCC pretraining can improve performance in some cases but do not consistently yield better results. Other baselines even degrade performance. In contrast, our pretraining methods substantially improve, reducing the objective gap by 59.4% on IPP and 54.1% 1% on QP with only 10% of the training data. This highlights the effectiveness of our data augmentations, which are specifically tailored for optimization problem instances and significantly enhance fine-tuning.

---

Table 2: cetrained-finetunee model performance LP/QP datasets under different levels of data scarcity. Our pretraining consistently improves performance and outperforms the baselines.
LP
QP
Pretraining
10%
20%
100%
20%
100%
None
8,53950205
6.149
2.784=008:
5.304
3.567=nm64
1.240
GraphiCL
95694
6.033+as53
2.613=0.26
6,024a6.599
=.253 3.8
GCC
8.248
5.761
2.686
0
6.356_0044
1.472:0055
IGSD
18.097
3.082.0.0 2.631
200
12.799 488
6.306_0068
1.302
MYGRL
20.392:24.8
9.07241854
2.852:0112
8.440
4.932-1.03
1.343:0003
InfoGraaph
18.338117785
7.464
2.956.017
10.306 189
6.246_. 180
1.409
DGI
19,661
9.671=277
3.156=0 184
7.223-0.42
1.425 009
GAE
9.082s0m0
6.032e0241
3,434:2225
5,848
3.759=0.58
1.381
Ours
3.472a00ts
2.794
1,588=0006
3.791
2.427=0083
0.926
Generalization To address Q3, we compare the performance of models trained from scratch versus models initialized with contrastive pretraining and then finetuned. For LPs, generate four types of relaxed LP instances derived from MILPs: Set Cover (SC), Maximum Independent Set (MIS), Combinatorial Auction (CA), and Capacitated Facility Location (CFL), following Gasse et al. [2119]. For QPs, we generate instances of soft-margin SVM, Markowitz portfolio optimization, and LASSO regression following Jung et al. [022). If possible, problem sizes and densities are kept similar the Drngginnnnnggggoooo
pretraining datasets; see more details in Appendix G As shown in Tables 3 and 4 pretrained models
outperform models trained from scratch in almost all cases, demonstrating strong transferability to
OOD tasks. The evaluation on larger datasets can be found in Appendix E.2
Table 3:
Generalization performance of con- trastive pretrained MPNNs on OOD LP instances.
Table Generalization performance of con- trastive pretrained MPNNs on OOD QP instances.
Ratio 20%
Family
Pretrained
10%
100%
Ratio 30%
SC
No
5,297eam
1.965,03w
=.531
1.349_0111
0.752
0.662_coes
Family
Protrained
0.191120es
10%
100%
MIS
CA
Yes
Yes
No
0.674=3mis
0,722
2.381
2,303
0.465=
0.421
1.803=xx27
1.713_emm
0,203=0006
0.753_coMs
0.177
0.906=tu2s
Pottolio
Yes
No
3.7061
0.14
5 1:005
3.351
0.077=139
4.083
=841=1102
1.637 3313
0.353.0011
2.159
s.802
0.024
LASSO
Yos
5.169am
3.726
1.178
CRL
CL
0.367:
0.217:cmm
0.255-6004
0.059=0006
0.064
Regarding Q4, see Appendix
E.1 for results and discussion.
Conclusion
We introduced principled framework for data augmentation in learning to optimize over linear and quadratic programming. By leveraging affine transformations of the KKT system, we designed a family of expressive, colution-preserving and computationally efficient transformations. Our method allows for augmentations that either admit exact solution recovery or preserve key structural properties without requiring access to the original solutions, making them suitable for supervised and contrastive learning. Extensive experiments show that these augmentations consistently improve performance under data scarcity, generalize to larger and out-of-distribution problems, and outperform existing graph augmentation baselines. This work highlights the benefits of optimization-avaa augmentation strategies and opens new directions for robust, scalable L220 under limited supervision.
Acknowledgements
Christopher Morris and Chendi Qian are partially funded by a DFG Emmy Noether grant (468502433) and RWTH Junior Principal Investigator Fellowship under Germany's Excellence Strategy. We thank Erik Müller for crafting the figures.
10

---

References
Achterberg. Koch, and Martin, Branching rules revisited. Operations Research Letters, 33(1): 42-54, 2005.
A. Amaiz-Rodríguez, A. Begga, F. Escolano, and N. Oliver. Diffwire: Inductive graph rewiring via the lovasz bound. arXiv preprint, 2022. 2
Barbero, A. Velingker, A. Saberi, M. Bronstein, and Di Giovanni. Locality-aware graph-rewiring
in gnns. arXiv preprint, 2023. 2
Y. Bengio, A. Lodi, and A. Prouvost. Machine learning for combinatorial optimization: a methodological tour d'horizon. European Journal of Operational Research, 290(2):405-4211, 2021. g
Bielak, T. Kajdanowicz, and N. V. Chawla. Graph barlow twins: A self-supervised representation learning framework for graphs. Knowledge-Based Systems, 256: 106:1,,,3,, 2022.
S. Bowly, Stress testing mixed integer programming solvers through new test instance generation methods. PhD thesis, University of Melbourne, Parkville, Victoria, Australia, 2019.
S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, 2004.
26
X. Bresson and T. Laurent. Residual gated graph convnets. arXiv preprint, 2017.
T. Cai, S. Luo, K. Xu, D. He, T.-y. Liu, and L.. Wang. Graphnorm: A principled approach to accelerating graph neural network training. In International Conference on Machine Learning, pages 1204-1215. PMLR, 2021. 8
Q Cappart, D. Chételat, E. B. Khalil, A. Lodi, C. Morris, and P. Veličković. Combinatorial optimization and reasoning with graph neural networks. Journal of Machine Learning Research, 24(130):1-61, 2023. 1
T. Chen, S. Kornblith, M. Norouzi, and G. Hinton. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning, pages 1597-1607. PMLR, 2020. 8
Z. Chen, J. Liu, X. Wang. J. Lu, and W. Yin. On representing linear programs by graph neural networks. arXiv preprint, 2022.
Z. Chen, J. Liu, X. Wang. Lu, and W. Yin. On representing mixed-integer linear programs by graph neural networks. arXiv preprint, 2023. 2
Z. Chen, X. Chen, J. Liu, X. Wang, and W. Yin. Expressive power of graph neural networks for (mixed-integer) quadratic programs. arXiv preprini, 2024a.
Z. Chen, J. Liu, X. Chen, W. Wang, and W. Yin. Rethinking the capacity of graph neural networks for branching strategy. Advances in Neural Information Processing Systems, 37:123991-124024, 2024b.
J.-Y. Ding, C. Zhang, L. Shen, S. Li, B. Wang, Y. Xu, and L. Song. Accelerating primal solution findings for mixed integer programs based on solution prediction. In Proceedings of the AAAI Conference Artificial Intelligence, volume 34, pages 1452-1459, 2020. 2
K. Ding, Z. Xu, H. Tong, and H. Liu. Data augmentation for deep graph learning: A survey. ACM SIGKDD Explorations Newsletter, 24(2):61-77, 2022. 2
H. Duan, P. Vaezipoor, M. B. Paulus, Y. Ruan, and C. Maddison. Augment with care: Contrastive learning for combinatorial problems. In International Conference Machine Learning, pages 8627-5642 PMLR, 2022. 3
D. Duvenaud, D. Maclaurin, J. Aguilera--paarrguuree, R. Genne--oomarelll Hirzel, A. Aspuru- Guzik, and R. P. Adams. Convolutional networks on graphs for learning molecular fingerprints. arXiv preprint, 2015. 2
B. Fatemi, S. Abu-Fl-Haija, A. Tsitsulin, M. Kazemi, D. Zelle, N. Bulut, J. Halcrow, and B. Perozzi. Ugsl: A unified framework for benchmarking graph structure learning. arXiv preprint, 2023, 2
11

---

F. Furini, E. Traversi, P. Belotti, A. Frangioni, A. Gleixner, N. Gould, L. Liberti, A. Lodi. R. Misener, H. Mittelmann, et al. Qplib: a library of quadratic programming instances. Mathematical Programming Computation, 11:237-265. 2019. 23 29
M. Gasse, D. Chételat, N. Ferroni, L. Charlin, and A. Lodi. Exact combinatorial optimization with graph convolutional neural networks. Advances Neural Information Processing Systems, 32:
15580-15592, 2019.
31
M. Gasse, S. Bowly, Q. Cappart, J. Charfreitag, L. Charlin, D. Chételat, A. Chmicla. J. Dumouchelle, A. Gleixner, A. M. Kazachkov, al. The machine learning for combinatorial optimization competition (ml4co): Results and insights. In NeurlPS 2021 competitions and demonstrations track, pages 220-231 231. PMLR, 2022. 3
J. Gasteiger, S. Weißenbergerr, and Günnemann. Diffusion improves graph learning. Advances in Neural Information Processing Systems, 32:13366-13378 2019. 9
Z. Geng, X. Li, Wang, X. Li, Y. Zhang, and F. Wu. A deep instance generative framework for milp solvers under limited data availability. Advances in Neural Information Processing Systems, 36: 26025-26047, 2023. 3
J. Gilmer, S. S. Schoenholz, P. Riley, O. Vinyals, and G. E. Dahl. Neural message passing for quantum chemistry. In International conference machine learning, pages 1263-1272. PMLR, 2017.
2
A. Gleixner, G. Hendel, G. Gamrath, T. Achterberg, M. Bastubbe, T. Berthold, P. Christophel, K. Jarck,
T. Koch, J. Linderoth, et al. Miplib 2017: data-driven compilation of the 6th mixed-integer programming library. Mathematical Programming Computation, 13(3):443 490, 2021. 23
Z. Guo, Y. Li, C. Liu, W. Ouyang, and J. Yan. Acm-milp: Adaptive constraint modification via grouping and selection for hardness-preserviee preserving milp instance generation. In International Conference
Machine Learning, pages 16869-16890. PMLR, 2024. 3
B. Gutteridge, X. Dong, M. M. Bronstein, and Di Giovanni. Drew: Dynamically rewired message passing with delay. In International Conference on Machine Learning, pages 12252-12267. PMLR, 2023. 2
W. Hamilton, Z. Ying, and J. Leskovec. Inductive representation learning on large graphs. Advances in
Neural Information Processing Systems, 30: 1025-1035, 2017. 2
K. Hassani and A. H. Khasahmadi. Contrastive multi-view representation learning on graphs. In International Conference on Machine Learning, pages 41 16 4126. PMLR, 2020.
Y. Hu, S. Ouyang, J. Liu, G. Chen, Yang. J. Wan, Zhang, Z. Wang, and Y. Liu. Graph ranking contrastive learning: A extremely simple yet efficient method. arXiv preprint, 2023. 2
T. Huang, A. M. Ferber, Y. Tian, B. Dilkina, and B. Steiner. Searching large neighborhoods for integer linear programs with contrastive learning. In International Conference on Machine Learning, pages 13869-13890, PMLR, 2023. 3
C. Ji, Z. Huang, Q. Sun, H. Peng. X. Fu, Q. Li, and
J. Li. Regel: rethinking message passing in graph contrastive learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 8544-8552, 2024. 3
W. Jin, Y. Ma, X. Liu, X. Tang, S. Wang, and J. Tang. Graph structure learning for robust graph neural networks. arXiv preprint, 2020.
H. Jung, J. Park, and J. Park. Learning context-aware adaptive solvers accelerate quadratic programming. arXiv preprint, 2022. 10
K. Karhadkar, P. K. Banerjee, and G. Montúfar. FoSR: First-order spectral rewiring for addressing oversquashing in gnns. arXiv preprint, 2022. 2
E. B. Khalil, C. Morris, and A. Lodi. Mip-gnn: A data-driven framework for guiding combinatorial solvers. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 10219-10227, 2022. 2
12

---

T. N. Kipf and M. Welling. Variational graph auto-encoders. arXiv preprint, 2016. 31
B. Li, L. Yang. Y. Chen, S. Wang, Q. Chen, H. Mao. Y. Ma, A. Wang, T. Ding, Tang, al. PDHG-unrolled learning-to-optimin method for large-scale linear programming. arXiv preprint, 2024a. 2
Q. Li, T. Ding, L.. Yang, M. Ouyang, Q. Shi, and R. Sun. On the power of small-size graph neural networks for linear programming. In Advances in Neural Information Processing Systems, volume 37,
pages 38695-38719, 2024b. 2
Q Li, M. Ouyang. T. Ding, Y. Wang, Q. Shi, and R. Sun. Towards explaining the power of constant-depth graph neural networks for structured linear programming. The Thirteenth International Conference Learning Representations, 2025. URL BRLLtttt:////teeerrrii http:///ppppppooeeeeooooooooo/aa
S. Li, J. Kulkarni, I. Menache, C. Wu, and B. Li. Towards foundation models for mixed integer linear programming. arXiv preprint, 2024c. 3
L. Lin, J. Chen, and H. Wang. Spectral augmentation for self-supervise learning on graphs. arXiv preprint, 2022. 2
H. Liu, J. Wang, W. Zhang, Z. Geng, Y. Kuang, X. Li, B. Li, Y. Zhang, and F. Wu. Milp-studio: Milp instance generation via block structure decomposition. arXiv preprint, 2024. S
N. Liu, X. Wang, D. Bo, Shi, and J. Pei. Revisiting graph contrastive learning from the perspective of graph spectrum. Advances in Neural Information Processing Systems, 35:2972-2956 2983, 2022a.
Q Liu, L. Chen, F. Fang, and X. Qian. Self-supervised learning: Generative or contrastive. IEEE Transactions Knowledge and Data Engineering, 33(8):3 100-3318,-3 2021.
Liu, R. Ying, H. Dong, L. Li, T. Xu, Rong, P. Zhao, Huang, and D. Wu. Local augmentation for graph neural networks. In International Conference on Machine Learning, pages 14054-14072 PMLR, 2022b. 2
Y. Liu, Y. Zheng, D. Zhang, H. Chen, H. Peng, and S. Pan. Towards unsupervised deep graph structure learning. ACM Web Conference 2022, 2022c. 2
Nocedal and S. J. Wright. Numerical optimization. Springer, 1999.
P. A. Papp, K. Martinkus, L. Faber, and R. Wattenhofer. Dropgnn: Random dropouts increase the expressiveness graph neural networks. Advances in Neural Information Processing Systems, 34: 21997-22009, 2021. 2
C. Qian and C. Morris. Towards graph neural networks for provably solving convex optimization problems. arXiv preprint, 2025. 2
C. Qian, A. Manolache, Ahmed, Z. Zeng, G. V. d. Broeck, M. Niepert, and C. Morris. Probabilistically rewired message-passing neural networks. arXiv preprint, 2023. 2
C. Qian, D. Chételat, and C. Morris. Exploring the power of graph neural networks in solving linear optimization problems. In International Conference on Artificial Intelligence and Statistics, pages 1432-1440. PMLR, 2024a.
C. Qian, A. Manolache, C. Morris, and M. Niepert. Probabilistic graph rewiring via virtual nodes. arXiv preprint, 2024b.
Qiu, Q. Chen, Y. Dong, Zhang, H. Yang, M. Ding, K. Wang, and J. Tang. Gcc: Graph contrastive coding for graph neural network pre-training. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery Data Mining, pages 1150-1160. 2020.
A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, Clark, et al. Learning transferable visual models from natural language supervision. In International Conference Machine Learning, pages 8748-8763, PMLR, 2021. 3
13

---

Y. Rong, W. Huang, T. Xu, and J. Huang. Dropedge: Towards deep graph convolutional networks on node classification. arXiv preprint, 2019.
F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini. The graph neural network model. IEEE Transactions on Neural Networks, 20(1):61-88) 2008.
J. Shermen and W. Morrison. Adjustment of an inverse matrix corresponding to changes in the elements of a given column or a given row of the original matrix. Annual Mathmatical Statistics, 20:621-625, 1949.
F-Y. Sun, J. Hoffmann, V. Verma, and J. Tang. Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization. arXiv preprint, 2019.
3
31
S. Suresh, P. Li, C. Hao, and J. Neville. Adversarial graph augmentation improve graph contrastive learning. Advances in Neural Information Processing Systems, 14:15920-15933, 2021.
2
S. Thakoor, C. Tallec, M. G. Azar, M. Azabou, L. Dyer, R. Munos, P. Veličković, and M. Valko. Large-scale representation learning on graphs via bootstrapping. arXiv preprint, 2021. 23
Topping, Di Giovanni, B. Chamberlain, X. Dong, and M. M. Bronstein. Understanding over-squashing and bottlenecks on graphs via curvature. arXiv preprint, 2021. 2
Veličković, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio. Graph attention networks
arXiv preprint, 2017. 2
Veličković, W. Fedus, W. L. Hamilton, Liò, Y. Bengio, and R. D. Hjelm. Deep graph infomax.
arXiv preprint, 2018.
9
31
G. Wan, Y. Tian, W. Huang, N. V. Chawla, and M. Ye. S3gel: Spectral, swift, spatial graph contrastive learning. In International Conference Machine Learning, pages 49973 49990. PMLR, 2024.
H. Wang, J. Liu, X. Chen, X. Wang, P. Li, and W. Yin. Dig-milp: deep instance generator for mixed-integer linear programming with feasibility guarantee. arXiv preprint, 2023.
C. Wu, C. Wang. J. Xu, Z. Liu, K. Zheng, X. Wang, Y. Song, and K. Gai. Graph contrastive learning with generative adversarial network. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2721-2730 2023. 2
C. Wu, Q. Chen, A. Wang, T. Ding, R. Sun, W. Yang, and Q. Shi. On representing convex quadratically constrained quadratic programs via graph neural networks. arXiv preprint, 2024. 2 29
L. Wu, H. Lin, C. Tan, Z. Gao, and S. Z. Li. Self-supervised learning on graphs: Contrastive, generative, or predictive. IEEE Transactions Knowledge and Data Engineering, 2021.
2
K. Xu, W. Hu, J. Leskovee, and S. Jegelka. How powerful are graph neural networks? arXiv preprint, 2018. 2
K. Yang, H. Han, W. Jin, and H. Liu. Spectral-aware augmentation for enhanced graph representation learning. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, pages 2837-2847, 2024a. 2
L. Yang, B. Li, T. Ding. J. Wu, A. Wang, Y. Wang. J. Tang, R. Sun, and X. Luo. An efficient unsupervised framework for convex quadratic programs via deep unrolling. arXiv preprint arXiv:2412.01051, 2024b. 29
T. Yang. H. Ye, and H. Xu. Learning to generate scalable milp instances. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, pages 159-162, 2024c.
Z. Yang, M. Ding, C. Zhou, H. Yang. J. Zhou, and J. Tang. Understanding negative sampling in graph representation learning. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1666-1676, 2020.
14

---

Y. Yin, Q. Wang, S. Huang, H. Xiong, and X. Zhang, Autogel: Automated graph contrastive learning via learnable view generators. In Proceedings the AAAI Conference on Artificial Intelligence volume 36, pages 8892-8900, 2022.
Y. You, T. Chen, Y. Sui, T. Chen, Z. Wang. and Y. Shen. Graph contrastive learning with augmentations. Advances in Neural Information Processing Systems, 33:5812-5823, 2020. 30 31
Y. You, T. Chen, Y. Shen, and Z. Wang. Graph contrastive learning automated. In International Conference Machine Learning, pages 12121-12132. PMLR, 2021.
J. Zeng and P. Xie. Contrastive self-supervised learning for graph classification, In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 10824-10832, 2021.
S. Zeng, M. Zhou, S. Zhang, Y. Hu, F. Wu, and X.-Y. Li. Cler: Contrastive learning-based constraint reordering for efficient milp solving. arXiv preprint, 2025. X
H. Zhang, S. Lin, W. Liu, P. Zhou, J. Tang, X. Liang, and E. P. Xing. Iterative graph celf-distillation. IEEE Transactions Knowledge and Data Engineering, 36(3):1161-1165 2023.
31
Y. Zhang, C. Fan, D. Chen, C. Li, W. Ouyang, M. Zhu, and J. Yan. MILP-FBGen: LP/MILP instance generation with Feasiblity/Boundddnss.. In International Conference on Machine Learning, volume 235, pages 58881-58896. PMLR, 2024. 3
Zhao, Y. Liu, L. Neves, O. Woodford, M. Jiang, and N. Shah. Data augmentation for graph neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 11015-11023, 2021. 2
Zhao, W. Jin, Y. Liu, Wang, G. Liu, Günnemann, Shah, and M. Jiang. Graph data augmentation for graph machine learning: A survey, arXiv preprint, 2022. 2
Z. Zhou, S. Zhou, B. Mao, X. Zhou, J. Chen, Q. Tan, D. Zha, Y. Feng, C. Chen, and C. Wang. Opengsl: A comprehensive benchmark for graph structure learning. Advances in Neural Information Processing Systems, 16:17904--7922 2023. 2
Y. Zhu, Y. Xu, F. Yu, Q. Liu, S. Wu, and L. Wang. Deep graph contrastive representation learning. arXiv preprint, 2020. 2
Y. Zhu, Y. Xu, F. Yu, Q. Liu, S. Wu, and L. Wang. Graph contrastive learning with adaptive augmentation. Proceedings the Web Conference 2027, pages 2069-2088 2021.
D. Zou, H. Peng, Huang, R. Yang, J. Li, J. Wu, Liu, and P. Yu. SE-GSL: A general and effective graph structure learning framework through structural entropy optimization. arXiv preprint, 2023. 2
15

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly state our main contributions, which are supported by both theoretical analysis and empirical results, and are appropriately scoped to reflect the demonstrated capabilities of our method.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We explicitly discuss the limitations of our approach in Appendix A including dataset scale, generalizability to real-world benchmarks, and modeling assumptions.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested on few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images taken in low lighting. Or specch-to-tex system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
16

---

Answer: [Yes]
Justification: All theoretical results are accompanied by clearly stated assumptions and complete proofs, which are provided in Appendix D
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide all necessary details to reproduce our main results in Section 3 Appendix [F] and Appendix G including data generation, model setup, training procedures, and evaluation metrics.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or a way to reproduce
the model (e.g., with open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited some way (c.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
17

---

Question: Does the paper provide open access to the data and code, with sufficient instructions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We provide an anonymized code repository with clear instructions to reproduce all main experimental results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
public/guides:// hotts///oooeSuubbsisiiooooooo
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/piiss.ccpuulic//uii.ee httpbbiii/gguddeee/eeeeuuuueeeee
www.:iiiiissiooooooooooooo for more details.
The authors should provide instructions on data access and preparation, including how to access the data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e-g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide all relevant experimental details, including data splits, training configurations, and hyperparameter settings in Section 3 Appendix [F] and Appendix ensuring clarity and reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We report the numbers in mean and standard deviation of the objective five random seeds. Even considering the error bar, our empirical results are still significant.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
18

---

The method for calculating the error bars should be explained (closed form formula, call library function, bootstrap, etc.)
The assumptions made should given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information the computer resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Answer: [Yes]
Justification: We specify the compute setup, including device type and per-epoch computation time, in Section 3 This includes the use of an NVIDIA L40S GPU and detailed timing in Table sufficient for reproduction.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics httph://////// cctpprpppcclbp//tttiicccccccccc
Answer: [Yes]
Justification: We have thoroughly reviewed the NeurlPS Code of Ethics and confirm that our research complies with its principles in all respects.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there a special considera- tion due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We discuss broader impacts in Appendix B including potential benefits such as improving solver efficiency and optimization-bar based decision-making well as potential risks related to misuse in applications.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
19

---

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairesss considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: The paper does not release any pretrained models or real-world datasets with risk of misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All existing assets used the paper, such as graph contrastive learning baselines and synthetic datasets generation, are open-source and properly cited. No scraped data is used, all datasets are synthetically generated.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, wwwppppappphhaeeec thcode. com com/datasets
20

---

has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We introduce a new suite of synthetic LP and QP datasets along with data augmentation tools for optimization learning tasks. These assets well-documented and will be released under an open-source license.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text instructions given to participants and sereenshots, applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This work does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
This work does not involve research with human subjects or crowdsourcing.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
21

---

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing. editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: This work does not use LLMs as part of the core methods or contributions. Any LLM usage, if any, was limited minor writing assistance and does affect the scientific rigor or originality of the research.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to LLM policy oitepp///eaarrrecccc//ooorrrreee for what should or should not be described.
22

---

A Limitations
While our proposed data augmentation framework for QP and LP instances is efficient and principled, several limitations remain. First, some transformations, e.g., Appendix [ and Section 2.2 require access to the optimal primal-dual solution of the original problem, which may not always be available in practice. Second, our framework assumes convexity (i.e., Q > 0) and problem feasibility, and does not directly extend to non-convex or infeasible cases. Additionally, non-linear quadratic programming is beyond our scope. Also, we train and evaluate solely on synthetic QP and LP instances because real-world datasets are scarce and heterogeneous. Benchmarks such as MIPLIB [Gleixner et al., 2021) and QPLIB [Furini et al., 2019] contain too few samples and exhibit significant variability in problem size and structure, making them unsuitable for training deep models. Finally, while our transformation framework mathematically expressive, we restrict to small subset of efficient, feasible augmentations in practice, leaving its full expressive power unexplored.
B Broader impact
This work introduces principled data augmentation methods for learning- based solvers in convex optimization, focusing on predicting objective values and solutions for LP/QP problems. These methods can also replace heuristics, such as strong branching in mixed-integer optimization. Our approach may benefit logistics, finance, and scientific computing applications by enabling more robust and data-efficient learning. As a foundational contribution, this work does not raise concerns about privacy, security, or fairness. It does not involve sensitive data or end-user interaction, and the proposed techniques are general purposs and not tied to specific domains. Nevertheless, when applied to real-world optimization systems, care should be taken to ensure reliability and fairness.
C Additional transformations
Here, we outline additional transformations, omitted from the main paper for space reasons.
C.1 Adding constraints
we can augment problem instance by introducing additional constraints. This is done by extending the constraint matrix using transformation of
Analogous to adding variables (see Section 2.2
I where m E R is a non-negative vector. This effectively appends a new
the form M2 == m
constraint that is a convex combination of the existing ones. Notably, this transformation, along with the scaling constraints in Section [.2, satisfies the condition derived in Eq. (13). In unsupervised settings where neither the primal-dual solutions the active constraints are known, we store all constraints and add a new one that is linearly combined with them and does not affect the solution. The
added constraint remains inactive, corresponding a zero dual variable, transforming both efficiently recoverable and solution-inddepender
)*)). We define Proposition C.1. Let (Q, A, b, c) be an LCQP instance with optimal solution I
our transformatinn as T(])= = (Q, M2,, Myb, c), where M2 ==
Then the transformed
m
instance has an optimal solution x", |X','" X*) if and only if the new dual variable X = O.
C.2 Biasing the problem
We introduce a data augmentation strategy that leverages nontrivial bias terms while fixing M to the identity. Specifically,
(A
[Q
AT
B
A
x
- c b
s*
+ B.
(12)
Consider the transformation in Eq. [12 B. To satisfy the conditions in Eq. we construct Bn = RRT, where R E Raak random matrix, ensuring that the resulting Q remains positive definite. We then set B21 B
where we fix M = I and design a suitable bias matrix
23

---

Rmxa as a random matrix, and Bz Accordingly, we must have
B
B
x
B21 0
X
ensuring the transformed KKT system is satisfied. This yields the transformed instance T(I) (Q+ B1,, + + B21, b. b+ B2ix", c B1i + ATX*) under which the original primal and dual solutions remain valid; therefore, is efficiently recoverable transformation. However, this transformation not solution- iidduienn-nndeeddentt, since it requires access to x; X", which is an intrinsic drawback of such biasing transformations.
C.3 Finding inactive constraints
We introduce the following heuristics. For all inequality constraints in given problem, including the variable bounds, e.g., 22 2 O, we calculate score h; == a, x + b,. The constraints with lower scores more likely be active. See Appendix C.3 for illustration. Given the number of constraints m
02
b2
a1
03
7
az = bj
(a)
(b)
Figure 2: Illustration of our heuristics. (a). Smaller aTc is more likely to be inactive. (b). Lower b is more likely to be inactive.
and the number variables n, m n (due to variable bounds constraints) and the constraints are not degenerate, constraints will be active. Therefore, pick the constraints with the largest h; as heuristic inactive constraints. Then this could be plugged into the removing constraint transformation.
We evaluate the effectiveness on our datasets, by calculating the accuracy given ground truth inactive constraint set Ag: and heuristic one Aneu.
Agg Abaul
acc
Anen
over all the instances; see Table for summary of results.
Table 5: Accuracy summary.
LP
QP
Size
100
150
200
250
100
150
200
250
Acc. (5)
88.5+29
90.1
89.1
88.3#36
91.8+34
92.9+27
91,6+21
91.4=24
The heuristic's 's performance is satisfactory, with a low false-positive rate. Although is generally designed for IPss, it is also effective for QPs.
C.4 Adding variable
Besides Proposition 2.6 derive another efficient method to add new variable.

---

Proposition C.2. Given an LCQP instance 1 = (Q. (,,,,,,c) c) and its optimal primal and dual solution
Q 0
1A a),b, 2 ) where > 0. a E R is
x",X² the data augmertation T(T) = OT
OT
arbitrary, d' == -a X", allows for an extra variable x' without affecting the original optimal solution, if and only if the new variable x² 0.
Such data transformation construction is efficient and valid. The only drawback that it is not solution- independent. Therefore, we provide an improved augmentation that introduces a new constraint.
Corollary C.2.1. Given QP instance I = (Q, A, c) and its optimal primal and dual solution
x², X", the data augmentation T(T) = ( Q 0
where q > 0, a E R", E R is arbitrary, allows for an extra variable x' an extra dual variable X without affecting the original optimal solution if and only if the new primal variable x'* O.
A
a
0T q
O
Proposition C.3. There exists a set of matrices (M, B), such that the design in Corollary C.2.1 can be realized in the form of Eq. 7
D Omitted proofs
Here, we outline missing proof' from the main paper.
D.1 Proof for Proposition 2.1
N
N
Proof. The rank condition of N1i is straightforward. Calculating the (pseudo) inverse
NT
N 22
12
requires N11 or N22 being invertible. From the KKT condition Eq. (3d). we know
X3 O.
So we
don't have to consider N or its existence; however, N1 must be invertible, otherwise we cannot
solve the optimal solution of the transformed problem without QP solver. Intuitively speaking, if N1 is not of full rank, we will drop some equality constraints, which might cause the solution to be relaxed, thus requiring to solve the transformed QP problem with QP solver.
Let us consider N12. We rewrite the second equation of Eq.
(8)
as
Nu N21
N1 N2
Aa Aa
Nn
N1
b
x=
N21
N 22
b
8
(13)
The first equation leads to An N12 Ni.b,
Ni2b, N128
(14)
which is equivalent to constructing new equality constraints with a linear combination of current constraints. We observe that x' remains for the transformed problem if and only if N128; = 0. A special case N O, which removes the effect of inactive constraints on active ones. Otherwise, we must resolve this equation for x²" Intuitively speaking, the margins of inactive constraints s* might relax or tighten an existing equation constraint, and we will have solve the linear equation Eq. (1) rather than lazily copying the x for the new problem.
The condition of N22 is from the fact that if we multiply negative constant on an inequality S b. the inequality direction will flip. Let us look at the second equation of Eq. (13). We shall have
N21 Aax + A2 N21b. + N22 (ba
(15)
given A.x' by have
N2AA,x = N2 (b. - -)))
(16)
must hold. The matrix N22 gives us combinations for set of new inequality constraints given the existing constraints. Let us take out a row TT from the N22,
T Aa = u - TT
(17)
yields vector aT = TT Aa on the LHS and scalar b = Tb on the RHS. We know that for new inequality constraint, we should guarantee aTx" S b, so T > must be satisfied. Generalizing this to all rows of N22, should have N 0.
25

---

D.2
Proof for Proposition 2.4
Lemma D.1. KKT conditions are sufficient and necessary for LCQPs.
See
Boyd and Vandenberghe 2004
244]. LCQPs satisfy the Slater conditions.
Proof.
We extend the variable set by isolating the target variable x' and rewriting the problem with a: xd
Q
variables
E 1n++, and the corresponding coefficients as Q'
E (+(++))x(++1))
q
a] E [mx(x+1), and c' = [c] E IR 1R* b R
A' = [A
(If direction) Suppose x' = O, we can drop it as well as its corresponding coefficients 9, a, c. We shall have the KKT conditions Eqs. (3a and (3b) satisfied for the optimal solutions and X
Q' x
+
A x + c' =0
A' x
S b.
We can introduce row selection matrix M1 == [I 0] {0, 1]ax(at)), which effectively drops the last row, and projects the KKT conditions as
x 0
M M
M
Mid' Qx +TX +c=0
A'M M
Ax <.b
Thus, removing x' along with its associated coefficients preserves primal and dual feasibility, and x remains optimal for the reduced problem.
(Only if direction) Suppose we can remove x' without affecting the optimality w" Then the reduced and full KKT conditions must be consistent for all possible coefficients associated with x'. That is,
Qx" A
0
Qx" qu"
A
0
Jww + qx"
T +c =0
Ax" b
Ax + ax < b.
For these conditions hold for all choices of q, q, a, it must be that xxx 0: otherwise, the residual terms would depend on those coefficients and the solution would vary. Hence, x"t = is necessary for the removal to be valid without affecting the remaining solution.
D.3 Proof for Proposition 2.5
Proof. We rewrite the problem by appending the target constraint as an additional row. The constraint matrix becomes A'
A
[b'
(m++)) and b'
R (m++)), and the corresponding dual
aT
E
b'
A
variables A'
E
R((+1).
X
(If direction) Suppose the additional constraint is strictly inactive, i.e., X 0. We can introduce row selection matrix M, : [I 0) {0, 1) (m++)) The KKT conditions for the full system are,
Qx" A +c= Qx" ATX a' x
+
x² < i'
a
26

---

Since X = 0, we can write.
Qx' + A M2 M2X' + == + A +c=0
MrA'x* = Ax <b M2b'.
This corresponds the KKT conditions of the original problem with the constraint removed. Thus, removing the inactive constraint does not affect optimality.
(Only if direction) Suppose the constraint is removed and the optimal solution x", X remains valid for possible coefficients a' , t/. Then both of the following must hold,
Qx X + Qu + A + t'x' +c=0
Qx + A +c=0.
Subtracting the two equations yields a' x* O. For this hold for arbitrary a', it must be that X = O. Hence, the constraint must have been inactive.
D.4 Proof for Proposition 2.6
Proof. (If direction) Let x'
with x/" O, verify that
MQQMTT' + MIAXX + Mic= = Mi (Qx* + ATX + c) = 0
aM Ax" < b.
Thus, (x'*, X*) satisfies the KKT conditions for the transformed problem.
(Only if direction) The equation
aM Ax" + Aqr" s
must hold for all choices of q, therefore x' = O.
D.5 Proof for Proposition C11
Proof. (If direction) Suppose X 0, and the transformed KKT conditions
Qx
LAT
m x
+c= Qx + +c=0
MzAv" M2b
hold.
(Only if direction) The equations of pre- and post-transformation KKT must hold
Qx
A +c=
X + = 0
(18)
Qx + ATM
for all choices of m, therefore, X' 0.
D.6 Proof for Section
Proof for Appendix
A, b, c) and its optimal primal and dual solution x", X", new variable x 0, the following holds,
Proof. (If direction) Given LCQP instance I (Q,
Q
0
x
AT
[c
+
X
0r
x'+
az
x
A
a)
8*
(Only if direction) Similar to the proof of Proposition
2.6
27

---

Proof for Corollary C.2.1
Proof. (If direction) Let x'=
with xzx
O, we have
x
Q
0
x²
A
T
0
O
x
a
1
*
x
0
which is satisfiable for some X (Only if direction) The inequality,
x²
[A
a)
x²
S b
must hold for any choices of a, therefore x² 0.
Proof for Proposition C.3
0 a
Proof. Prove by construction, We can have M = I, and B1, B2 = 0, B21 = B =
0
0
0 =
0
0
E
Additional experiments
Here, we provide results for additional experiments.
E.1
Computational overhead
To address Q4, we compare the per-epoch training time for regular training, data augmentation, and CPU and model training GPU, their
contrastive pretraining. Since data augmentation absolute computation times are not directly comparable. Instead, we report the training time (excluding validation) with and without augmentation evaluate practical overhead. Results in Table 6 show the mean and standard deviation over 100 epochs, using batch size 32 (128 for pretraining) and num_wo workers
Table 6: Per-epoch training time (in seconds) over 100 epochs. Data augmentation adds negligible overhead. Pretraining uses a larger batch size and remains efficient.
I.P
QP 100%
Aug.
10%
100%
Pretrain
10%
Pretrain
No
1.005
7.752+0.43
N/A
1.244
10
N/A 9.648_0119
Yes
0.975
7,822::.2
7.699
1.235
10.247-033
E.2 Size generalization
For both LPs and QPs, we generate larger instances with increased numbers of variables and constraints (proportional the original sizes) while maintaining fixed average graph degree. The exact parameters are detailed in Appendix FF As shown in Table 7 increasing instance size improves performance across all methods, suggesting some extent of size generalization. Importantly, models pre-trained consistently outperform models trained from scratch across all problem sizes and levels of data scarcity.
28

---

Table 7: Generalization performance of contrastive pretrained MPNNs larger LP and QP instances. Models pretrained with contrastive learning consistently outperform those trained from scratch.
I.P
QP
Size
Prefrained
10%
20%
100%
10%
20%
100%
No
8.539+0.206
15
2.784+0001
5.304=a229
3.067
1.240==008
100%
Yes
3.472#0.066
2.794=0049
1.58820006
3.791
2.427=0.083
0.926=0081
6.707
00.08
2.401
1.253
2.947:10.070
0.98621002
150%
2.948+0.225
2.304=00029
+0044
2.999=a120
2.021
0.766=20331
No
5.820=n.099
4.173
2.052.20008
3.97
=000s
2.474
0.847=0044
200%
2.624
2.122::: 102
1.188
2.688
1.793
0.654
No
3.27510.00
3.759 L0.009
1.912.10003
3,708 100089
2.382.10:065
0.775
250%
Yes
2.451
1.91220063
#0:00
2.600 2003
1.67 100054
0.605
E.3
Other pretraining
Besides contrastive pretraining, we also pretrain with supervised learning with and without augmentation on the random LP dataset, and test its OOD performance the set cover dataset. As shown in Table 8 supervised pretraining performs surprisingly well, especially with data augmentation.
Table 8: Generalization performance of supervised pretrained MPNNs OOD set cover instances.
Ratio
Pretrained
10%
20%
100%
No
5.297=0.787
2.531 +0.861
0.752
CL
1.965-100.06
1.349
0..02210.055
Su. 1.994+0.115
1.521
0.778
Su.+aug. 0.034:10055
0.848
0.545 0.006
E.4 Ablation on temperature parameter
We conduct experiments on the temperature parameter the contrastive loss Eg. IT). We use same setting in Table 2 pick E {0.01, 0.1, 1). and run it with our method on LP instances. The results are summarized as Table As shown, т = 0.1 consistently the best option.
Table 9: Ablation on the temperature parameter T.
Ratio 20%
10%
100%
0.01
3.483+0.074
2.903+40078
1.67]+0.053
0.1
3.472+t0.886
2.794=10044
1.588
1
4.269
2.802.10.032
1.628.10.06
E.5 QPLIB
We evaluate QPLIB [Furini et al. 2019). Due to the limited size, large scale, and extreme heterogeneity of QPLIB instances, performing standard train/validationneest split is infeasible. Existing works that use QPLIB for evaluation typically rely on perturbing problem coefficients [Wu et al. 2024 Yang et al. 2024b These approaches, however, have significant limitations:
1. The augmentations are not label- and thus require solving each perturbed instance from scratch;
2. Perturbations may break feasibility, which limits the perturbation strength;
3. As a result, weaker perturbations are often used, but they yield training instances that are nearly identical to the test instances, making evaluation less meaningful.
29

---

Our proposed method, on the other hand, includes both structural and feature perturbation and is targeted better generalization performance. For the current perturbation method and ours to work well, one can reduce the perturbation rate to arbitrarily small value to fit the test data perfectly, which appears nonsensical. To study transferability and fitting efficiency on QPLIB, we conduct pre-training experiment.
We generate a foundation dataset of 10000 large-scale random QP instances with sizes ranging from 1000 to 1500 variables and constraints, which are 100-200 times larger than the problems used in Table 2 Since no labels are needed for contrastive pretraining, data generation is efficient. Training takes around 30 seconds per epoch using 4 NVIDIA L40S GPUs.
2.
We select feasible LCQP instances from QPLIB, relax integer constraints, and train an MPNN to fit these problems in a supervised setting. compare models trained from scratch with models initialized from the pretrained weights.
Here are the problem statistics and results:
Table 10: Statistics of selected QPLIB instances.
Name
cons.
vars.
A density
Q density
QPLIB _PI9
3280
3240
0.001 208
0.000313
QPLIB 3708
12917
12930
0.000171
0.000628
QPLIB_3861
4650
4530
0.000856
0.000222
QPLIB 3871
1040
1025
0.003772
0,001000
QPIIB_8553
5000
10000
0.000500
0.000700
We train each model for 100 epochs and compare predicted objectives to the actual optimal value. Longer training will diminish the advantage of pretraining, since testing on the same data used for training. We repeat the training times with random seeds and report the mean.
Table 11: Training performance on selected QPLIB instances. Pretraining substantially improves convergence.
Name
Pretrain Obj.(optimal)
Obj.(predict)
0.000
10.362
QPLIB_366 3694
Yes 0.000
0.142
No
-42.469
-42,469
QPLIB_3708
Yes
-42.469
-42,469
No
0.000
10,895
QPLIB_3861
Yes
0.000
-0.193
No 0.000
7.205
QPLIB 3871
1871
Yes 0.000
-0.193
No 15.793
29.208
QPLIB 8559 8
Yes
15.793
12.331
As shown in Table 11. pretrained models generally fit faster and more accurately within limited training epochs. QPLIB_3IP __70883081110 is an exception, though both models converge very close to the actual objective. These results support the scalability and transferability of our method.
F Hyperparameters
This section lists some crucial hyperparameters except those already described in Section 3
In supervised learning, we generate augmented data on the fly and control the degree of perturbation using an augmentation strength parameter a. For example, node-dropping strength of a = 0.3 removes 30% of the graph's nodes. We treat the variable-constraint graph as homogeneous for baselines from You et al. [2020). For scaling-base based augmentations (e.g., variable or constraint coefficients), we interpret strength a multiplication by e For constraint addition, add fraction of new
30

---

constraints, each convex combination of three existing constraints. To balance original and augmented views, we sample scaled strength a' == ac, where (0 1), ensuring smooth interpolation between unaltered and fully augmented data. The strengths used in our experiments are listed below.
Table 12: Augmentation strength of various augmentations in Table |
Augmentation
LP
QP
Drop node
0.10
0.05
Mask node
0.10
0,05
Flip edge
0.10
0.05
Drop vars.
0.99
0,99
Drop cons.
0.99
0.99
Scale cons,
1.00
1.00
Scale vars.
1.00
1.00
Add cons.
0.50
0.50
Add vars.
0.80
0.60
For the combined augmentations, we use the following combination for both LP and QP, and for each instance, we sample two augmentations from the list below.
Table 13: Augmentation strength of combined augmentations in Table
Augmentation
Strength
Drop vars.
0.00
Drop cons.
0.50
Scale cons.
0.50
Scale vars.
0.50
Add cons.
0.60
Add vars.
0.00
For contrastive pretraining, we omit interpolation apply fixed augmentation strengths. For GraphCI. You et al. 2020]. we tune the strengths of its three components: for LP, we use 0.2 node dropping, 0.1 edge flipping. and 0.13 feature masking; for QP, 0.3, 0.17, and 0.15, respectively, For use random walk length of 50 for LP and 200 for QP. IGSD (Zhang et al.,
GCC Qiu et al. 2020),
2023 and MVGRL Hassani and Khasahmadil 22200] apply graph diffusion with 10% edge addition.
InfoGraph [Sun et al. 2019). DGI [Veličković al. 20118) and GAE (Kipf and Welling 2016 require no additional hyperparameters. For our method, we use the following configuration.
Table 14: Augmentation strength of combined augmentations in Table
Augmentation
LP
QP
Drop (heuristic)
0.05
0.07
Scale cons.
0.40
1.03
Scale vars.
1.07
0.65
Add cons.
0.36
0.33
Add vars.
0.46
0.26
G Data generation
We use the following pseudo code for random L.P and QP generation; see Algorithms 1 and 2
For the instances used in Tables 1 and and Table 7 we generate the instances with sizes described in Table [3]
For the OOD LP instances in Table 3 we follow the generation procedure in Gasse et al. 2019 The idea is to keep the dimensions and the density the matrix A similar those of the problems we pretrain on. Hence, generate with 100 rows and columns and 5% A matrix density for SC; 100
31

---

Algorithm Generate a sparse feasible LP instance.
Require: Number of constraints m, number of variables n, constraint density Pa. random seed
Ensure: Matrices (c, A, b) defining an LP
1: Initialize random number generator rng with the given seed
2: Generate sparse random matrix (0, 1) mxx" with density PA
3: Sample Draw ~ (0, 1)
4: Set a: [raw (elementwise absolute value)
5: Compute Smoise (0, 1) (slack noise)
6: Set b == Ax + |8poise]
7: Sample random linear term e (O0,, 1)"
8: return (c, A,
Algorithm 2 Generate a sparse feasible QP instance.
Require: Number of constraints m, number of variables n, constraint density PA. matrix density pr. random seed
Ensure: Matrices (Q,c, A, b) defining QP
1: Initialize random number generator rng with the given seed
2: Generate sparse random matrix A ~ (0, 1)mx with density PA
3: Sample Uraw (O, 1)"
4: Set æ == [raw| (elementwise absolute value)
5: Compute Snoise (0, 1)"h (slack noise)
6: Set b = Ax + |Soisee
7: Sample random linear term c c~ N', 1)"
8: Generate sparse positive semi-definite matrix Q E
R
using
SciPy
make_ Lake_sparse__teee apparse_sp__matrii with sparsity parameter a pp
9: return (Q, C, b)
nodes and 0.02 edge probability for MIS; 100 items and 100 bids for CA; and 25 customers, three facilities, 0.5 ratio for CFL problem.
For the OOD QP instances, we generate SVM problems with Algorithm with 100 samples and 0.05 density.
Algorithm 3 Generate soft-margin SVM QP instance.
Require: Number of samples n. number of features d, regularization parameter X, feature density random number generator rng
Ensure: Matrices (Q,c, c, A, b) defining QP
Generate positive samples matrix A₁ ~ No ( dpp dp)
xxd
2: Generate negative samples matrix A₂ ~ N (- dp dp
xxd
A
Stack data:
A₂
4: Sparsify A with density
5: Construct label vector y = [1....,
E 1-1,1}"
6: Apply labels to data: A = A yT
7: Form constraint matrix: A == [[A In]
8: Set right-hand side: b == 1,,
9: Define quadratic matrix Q diag [1, 1, o...... O) E {0, (((+)))(()))))
10: Define linear term: c = (0..... 0, X. A| f0, Apatn
return cQ. A,
We generate portfolio problems as Algorithm with 100 assets and 0.05 density.
We generate LASSO problems using Algorithm with 50 samples, 50 features, and density of 0.05.
32

---

Table 15: Random instance generation hyperparameters.
1.P
QP
Size
#Col
#Row
A density
#Col
#Row
A density
Q density
100%
100
100
0.05
100
100
0.05
0.05
150%
150
150
0.03
150
150
0.03
0.03
200%
200
200
0.025
200
200
0,025
0.025
250%
250
250
0.02
250
250
0.02
0.02
Algorithm Generate mean-variance portfolio QP instance.
Require:
Number of assets n, covariance matrix density p. random number generator rng
Ensure: Matrices (Q, c. A, b, Aeg bay) defining QP
1:
Generate sparse positive definite covariance matrix Q R" using seapeee_pare___dd____e
with a = p. and eigenvalues in [0.1, 0.9]
2: Set linear cost vector == On
3: Generate inequality constraint matrix A /V 0.01) 1xn
4: Generate equality constraint matrix A
0.01 1,
5: Set inequality right hand side: =
6: Set equality right hand side: bog == [1]
7: return (Q, c, A, b, Aeq: Deg)
Algorithm 5 Generate LASSO QP instance.
Require: Number of samples n. number of features d, feature density p. regularization parameter A,
random number generator rng
Ensure: Matrices (Q,c. A, b) defining QP
1: Generate sparse design matrix X ~ (0, 1)d with density p
2: Sample true weight vector Wine ~ NO0 1)d
3: Generate noise vector \0, 0.5)
4: Compute target: := X Wimme + 6
5: Compute quadratic term: Qo
X X
6: Compute linear term: Co
Qo
7: Form block matrix: Q
R
2dx2d
0
Co A 111
8: Form cost vector:
9: Construct constraint matrix:
Ia - 11
- Ia
10: Set constraint right-hand side: b = Ozd
11: return (Q, c, A,
33


Reviews:
[{"review_id": "cDUdMOWu97", "reviewer": "Reviewer_55gp", "summary": "This paper introduces a principled approach to data augmentation tailored for QPs via MPNNs. The method leverages theoretically justified data augmentation techniques to generate diverse yet optimality-preserving instances. The paper presents various forms of data augmentation for QP problems and provides proofs to ensure the validity of the generated data. The value of the proposed algorithm is demonstrated from multiple perspectives, including improvements in supervised learning, supervised finetuning, generalization, and practical computational overhead.", "questions": "Have you considered extending the method to non-convex QPs or other non-convex optimization problems? How does the convexity assumption limit practical applications currently? \n\nAre there plans to test the method’s performance on more real-world datasets?\n\nis there a way to achieve fully solution-independent data augmentation?\n\nWhy was the temperature parameter $\\tau = 0.1$ chosen for contrastive learning? Have you conducted sensitivity analysis on this parameter?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "This is a mathematically grounded data augmentation framework for LPs and QPs, and detailed derivations and proofs for their augmentation framework. \nThe paper provides extensive experiments. \nThe framework emphasizes computational efficiency, and the paper discusses how structured matrices (e.g., diagonal matrices) reduce computational overhead, which is supported by runtime experiments.\n\nThe experiments focus on synthetic datasets for LPs and QPs, which may not generalize to real-world optimization problems.\nWhile the authors acknowledge this limitation, the lack of results on benchmarks reduces the practical impact of the work.\nSome proposed transformations require access to the primal-dual optimal solutions of the original problem, which may not always be available in practice. This could limit the framework’s applicability. \nThe mathematical sections (e.g., Section 2.1) are dense and may be difficult for readers without a strong optimization background to follow.\nOnly applied to convex problens. \nExperiments focus on synthetic datasets. \nThe experimental results lack sufficient discussion in the main text.", "was_revised": true, "final_justification": "I have read the authors' rebuttal and believe that the issues that could be addressed have been satisfactorily answered. There are also some issues (e.g., the non-convex case) that I certainly do not expect to be resolved in this paper, which is acceptable. After reviewing the comments from other reviewers and their exchanges with the authors, I still consider this a strong paper and will maintain my original evaluation.", "rebuttal": "We sincerely thank the reviewer for the thoughtful and constructive feedback, which has helped us improve the clarity and depth of our work. Below we provide point-by-point responses to your concerns.  \n### **Regarding real-world dataset**\nWe fully acknowledge this limitation. We do an evaluation on QPLIB. Due to the limited size, large scale, and extreme heterogeneity of QPLIB instances, performing a standard train/validation/test split is infeasible. Existing works that use QPLIB for evaluation typically rely on perturbing problem coefficients [1][2]. These approaches, however, have significant limitations:\n1. The augmentations are not label-preserving and thus require solving each perturbed instance from scratch;\n2. Perturbations may break feasibility, which limits the perturbation strength;\n3. As a result, weaker perturbations are often used, but they yield training instances that are nearly identical to the test instances, making evaluation less meaningful.  \n\nOur proposed method, on the other hand, includes both structural and featural perturbation and is targeted at better generalization performance. For the current perturbation method and ours to work well, one can reduce the perturbation rate to an arbitrarily small value to fit perfectly on the test data, but that is meaningless. To study transferability and fitting efficiency on QPLIB, we conduct a pretraining experiment:\n-  Step 1: We generate a foundation dataset of 10000 large-scale random QP instances with sizes ranging from 1000 to 1500 variables and constraints, being 100–200 times larger than the problems used in Table 2. Since no labels are needed for contrastive pretraining, data generation is efficient. Training takes around 30 seconds per epoch using 4 NVIDIA L40S GPUs.\n-  Step 2: We select feasible LCQP instances from QPLIB that vary a lot in size, relax integer constraints, and train an MPNN to fit these problems in a supervised setting. We compare models trained from scratch with models initialized from the pretrained weights.\n\nHere are the problem statistics and results:\n\n| Name | cons. | vars. | A density | Q density |\n|-------------|--------|-------|-----------|-----------|\n| QPLIB_3694 | 3280 | 3240 | 0.001208 | 0.000313 |\n| QPLIB_3708 | 12917 | 12930 | 0.000171 | 0.000628 |\n| QPLIB_3861 | 4650 | 4530 | 0.000856 | 0.000222 |\n| QPLIB_3871 | 1040 | 1025 | 0.003772 | 0.001000 |\n| QPLIB_8559 | 5000 | 10000 | 0.000500 | 0.000700 |\n\nWe train each model for 100 epochs and compare predicted objectives to the ground-truth optimal value. Longer training will diminish the advantage of pretraining, as we are testing exactly on the training data. We repeat the pretraining and fine-tuning with three random seeds and report the mean.\n\n| ID | Pretrain | Obj.(optimal) | Obj.(predict) |\n|-------------|----------|---------------|---------------|\n| QPLIB_3694 | No | 0.0 | 10.36161 |\n| QPLIB_3694 | Yes | 0.0 | __0.14176__ |\n| QPLIB_3708 | No | -42.46981 | __-42.46981__ |\n| QPLIB_3708 | Yes | -42.46981 | -42.46916 |\n| QPLIB_3861 | No | 0.0 | 10.89544 |\n| QPLIB_3861 | Yes | 0.0 | __0.41946__ |\n| QPLIB_3871 | No | 0.0 | 7.20569 |\n| QPLIB_3871 | Yes | 0.0 | __-0.19335__ |\n| QPLIB_8559 | No | 15.79288 | 29.20809 |\n| QPLIB_8559 | Yes | 15.79288 | __12.33148__ |\n\nAs shown, pretrained models generally fit faster and more accurately within limited training epochs. QPLIB_3708 is an exception, though both models converge very close to the actual objective. These results support the scalability and transferability of our method.  \n[1] Wu, Chenyang, et al. \"On representing convex quadratically constrained quadratic programs via graph neural networks.\" arXiv preprint arXiv:2411.13805 (2024).  \n[2] Yang, Linxin, et al. \"An Efficient Unsupervised Framework for Convex Quadratic Programs via Deep Unrolling.\" arXiv preprint arXiv:2412.01051 (2024).  \n\n### **Regarding label requirement in augmentations**\nWe agree with the reviewer that certain augmentations do rely on access to both primal and dual solutions. Indeed, these augmentations are useful in supervised settings. However, for self-supervised pretraining, we also propose several label-free augmentations, such as scaling the coefficients of variables and scaling constraints, adding redundant constraints, and dummy variables. These augmentations are designed to be solution-independent and are shown to perform well even when applied directly in the supervised setting (see Table 1 in the main paper).  \nThat said, we would like to highlight that achieving fully solution-independent augmentation is intrinsically difficult. First of all, the most universal form is Equation (7) in the paper. If we consider applying the bias terms, they must satisfy $$\\mathbf{B} \\left(\\mathbf{M}^T\\right)^{\\dagger} \\begin{bmatrix} \\mathbf{x}^* \\\\\\ \\boldsymbol{\\lambda}^* \\end{bmatrix} = \\mathbf{\\beta}$$ which requires the primal and dual solutions. Second, the $\\mathbf{M}_{22}$ matrix has its restriction as we wrote in Proposition 2.1, so that we should be careful not to violate the inequalities, which also requires the slack variables solution. If those requirements are met, e.g., the constraints are all equalities, given the form $$\\mathbf{M} \\begin{bmatrix} \\mathbf{Q}  &\\mathbf{A}^T  \\\\\\ \\mathbf{A}  &  \\mathbf{0} \\end{bmatrix} \\mathbf{M}^{\\dagger} \\begin{bmatrix} \\mathbf{x}^* \\\\\\ \\boldsymbol{\\lambda}^* \\end{bmatrix} = \\mathbf{M} \\begin{bmatrix} -\\mathbf{c} \\\\\\ \\mathbf{b} \\end{bmatrix}$$\nwe can retrieve the new solution with $\\mathbf{M}^{\\dagger}$. However, _fully solution-independent_ means it is not to compute the new solution, but instead to be applied in self-supervised learning. In the contrastive learning setting, label-free augmentations must retain the same semantic meaning. Arbitrary transformations using general $\\mathbf{M}$ may distort the instance such that it no longer corresponds to the same underlying solution, undermining the contrastive objective. To this end, we introduce and formally define the notions of _efficiently recoverable transformation_ and _solution-independent transformation_, which help guide augmentation design in a principled, effective, and pragmatic way.\n### **Application on non-convex problems**\nWe agree with the reviewer that our current framework is restricted to linearly-constrained quadratic programs (LCQPs). While this is a limitation, we note that LCQPs encompass a broad range of real-world problems, such as soft-margin SVM, Lasso regression, and portfolio optimization, which we include in our experiments.  \nOur theoretical foundation relies on KKT conditions, which are both necessary and sufficient for convex problems. In non-convex settings, however, the KKT conditions are only necessary (i.e., not sufficient) for local optima and do not apply to global minima. Thus, extending our augmentation theory and instantiations to non-convex optimization is highly non-trivial. Nonetheless, we agree this is an important direction for future work.\n### **Dense mathematics in Section 2.1 and limited discussion of experimental results**\nThank you for pointing this out. We acknowledge that the mathematical exposition in Section 2.1 may be dense and difficult to follow due to space constraints. In the camera-ready version, we will use the additional page to enhance the narrative and provide a clearer and more intuitive explanation of the theory and its connection to the augmentations and experiments. \n### **Temperature tuning in the contrastive loss**\nThank you for the sharp observation. We conducted ablation experiments over the temperature $\\tau$ in the contrastive loss on a random LP dataset. We use the same experiment setting as in Table 2, and run our method with different $\\tau$ in {0.01, 0.1, 1.0}. As can be seen in the table below, 0.1 is consistently the best choice, as shown in our Table 2 in the paper. Since our paper is not focused on contrastive learning per se, we did not explore more advanced techniques such as temperature scheduling, but we agree they could be interesting avenues for future work.\n\n| tau | LP 10% | LP 20% | LP 100% |\n|------|-------------|---------------|------------|\n| 0.01 | 3.483±0.074 | 2.903±0.078 | 1.671±0.053 |\n| 0.1 | __3.472±0.086__ | __2.794±0.049__ | __1.588±0.056__ |\n| 1. | 4.269±0.197 | 2.802±0.032 | 1.628±0.046 |\n\n**Once again, we sincerely appreciate the reviewer’s insightful comments and hope our responses address your concerns.**"}, {"review_id": "aMa0bxokW0", "reviewer": "Reviewer_yftr", "summary": "The authors propose a method for data augmentation to generate more training data for L2O methods. By solving a relatively small number of problem instances, the authors then use those solutions to generate additional problem instances whose optimal solutions can be efficiently computed using matrix operations on the solution to the original problems. This is accomplished by constructing affine transformations of compact representation of the KKT conditions for the original problem. The authors describe how to efficiently compute these transformations, as well as listing many example transformations that fit the required form. Results are compared against a baseline data augmentation method.", "questions": "How does it compare to training on a more diverse data set? It seems as though this method would bias the network towards performing well on problems that are similar to those that have been augmented in the training data. Are there tradeoffs involved, and if so, what are they?\n\nThe focus was not on complete coverage of the space but wouldn’t there be an advantage to focusing on that as a goal? That is, what is the motivation for scaling variable coefficients, scaling constraints, etc.?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- This paper uses a clever idea to augment the data for L2O methods. Generating training data for these problems is very expensive, and the method is clear, principled, and theoretically grounded.\n- The statement of contributions is clear, and represents a small, but worthwhile step forward in L2O\n- The paper is well-written overall, and easy to follow.\n\nWeaknesses:\n- The related work section is comprehensive but not particularly useful. It fails to contextualize this work with respect to the citations in that section. The authors need to explain how the work is related to their method, not just that it is related.\n\nMinor:\n- Line 93, the edges should be $\\subseteq V(G)\\times V(G)$\n- Line 143 uses $B_{11}$ and $B_{21}$ before they are introduced", "was_revised": true, "final_justification": "I think the authors provided reasonable responses to my questions and those of the other reviewers. Therefore, I elect to keep my rating of Accept.", "rebuttal": "We sincerely thank the reviewer for the thoughtful and encouraging review. We address your questions and suggestions below.\n### **Regarding related work**\nThank you for pointing this out. We agree that our related work section could be improved by better connecting prior work to our method, rather than listing related topics in isolation. In the revised version, we will revise this section to highlight the connections to our work. \n- __MPNNs and Learning to Optimize (L2O)__: In Section 1.1 of the main paper, we review relevant work on MPNNs in the context of L2O. This is directly connected to our setting, as we employ an MPNN architecture on the variable-constraint graph of linearly-constrained quadratic programs (LCQPs). Since LCQPs encompass a wide range of real-world applications (e.g., portfolio optimization, SVMs, Lasso regression), understanding how MPNNs can represent optimization problems and mimic algorithmic behavior is central to our formulation. We also discuss theoretical aspects to ground this connection.\n- __Graph data augmentation__: We include a review of classical graph data augmentation techniques, such as edge perturbation, node dropping, and subgraph sampling. However, we point out their limitations in our specific setting: such augmentations often alter the semantic meaning of optimization problems or violate feasibility. This distinction is also verified empirically, as shown in our experiments, where general-purpose augmentations underperform compared to our mathematically principled ones. \n- __Graph representation learning__: In Appendix C, we provide a background on graph contrastive learning and related representation learning frameworks. This is important because our augmentations are ultimately designed to be plugged into a contrastive pretraining pipeline. We compare these approaches in Table 2 and demonstrate that contrastive learning is only effective when paired with augmentations that respect the structure of the optimization problem. Besides, generic graph representation learning methods often fail to capture meaningful inductive biases in the L2O setting. \n- __Instance generation for MILPs__: We also review recent work on generating in-distribution instances for MILPs. While at first glance similar, these methods differ from ours in several key aspects. Their goal is to learn generative models that fit the instance distribution, often disregarding feasibility or boundedness. Some methods rely on strong structural assumptions, e.g., diagonalizable constraint matrices. Besides, solving the generated problems still requires running a solver, incurring additional computational cost. In contrast, our method is solver-free, learning-free, and mathematically grounded. By leveraging known solutions and applying solution-preserving transformations to the KKT conditions, we can recover solutions to new instances analytically. This makes our approach not only computationally efficient but also interpretable and principled. Moreover, our method is tailored to convex problems (specifically LCQPs), while MILP instance generation targets fundamentally different, combinatorial optimization problems.\n\n### **Training on a diverse dataset**\nThank you for the insightful suggestion.  \n__Self-supervised setting__: To explore the effect of pretraining on more diverse datasets, we extended the experiments in Table 2 by introducing additional pretraining setups. These new instances vary in both **instance size** and **problem distribution**. Note that generating data is not expensive, because the pretraining is self-supervised and doesn't require solving for the primal-dual solutions. This means we can generate large datasets without solving any optimization problems. Specifically:\n-   We pretrained one MPNN on large-scale random QP instances with 1000 to 1500 variables and constraints—approximately 100–200 times larger than those used in Table 2. \n-   We pretrained another model on a mixed dataset of similar size, containing randomly generated QPs, soft-margin SVMs, and Lasso regression problems.  \n\nWe observed that contrastive pretraining on diverse datasets __converges significantly faster__, indicating that the contrastive task is easier. This is expected, as in this case, negative samples in contrastive loss are from different classes and are trivially distinguishable, encouraging the model to learn **superficial shortcuts** rather than meaningful representations. Similar phenomena have been observed in [1] as too easy constrastive task is not beneficial.  \nAfter pretraining, we fine-tuned all models on the original random QP dataset used in Table 2. The results are as follows:\n\n| Pretrained | 10% data | 20% data | 100% data |\n|--------|-----|-----|----|\n| No | 5.304±0.229 | 3.567±0.034 | 1.240±0.088 |\n| On QP100 | 3.791±0.097 | 2.427±0.083 | 0.926±0.031 |\n| On QP1000 | 3.632±0.139 | 2.750±0.079 | 1.325±0.159|\n| On mixed |__8.628±0.618__|__4.821±0.503__|__1.419±0.075__|\n\nThe number in bold font are the __worst__ performing. Interestingly, pretraining on a highly diverse dataset results in worse performance than no pretraining at all. This is because the model learns from too easy contrastive task in data heterogeneous setting, and fails to capture useful data representations. This leads to poor transferability.\n\n__Supervised setting__: To evaluate the effect of augmentation supervised learning, we construct a mixed dataset of similar scale to Table 1 (approximately 100 variables and constraints), comprising 10000 diverse QP instances including randomly generated QPs, soft-margin SVMs, and Lasso regression problems. Due to limited time for author response, we employ inactive constraint dropping as a representative augmentation method, without hyper parameter tuning. The results are summarized below:\n\n| Augmentation| 10% data | 100% data |\n|--------|-----|----|\n| No | 12.982±0.701 |  4.372±0.310 |\n| Drop cons. | __11.778±0.519__ |  __3.896±0.275__ |\n\nThis preliminary result indicates that augmentation has a positive effect in the supervised setting as well. We plan to conduct a more comprehensive evaluation for the camera-ready version.\n\n### **Regarding trade-off of our method**\nWe acknowledge the trade-off introduced by our augmentation approach. One immediate drawback is the additional computation required to generate augmented instances. However, as discussed in Appendix F.1, this overhead can be effectively mitigated by asynchronously prefetching the next batch during model training, resulting in minimal impact on overall training time.  Another trade-off lies in training convergence: when performing supervised learning with augmentations—especially under high perturbation rates—convergence can be slower. This is expected, as the model must generalize over a broader distribution of instances. Nevertheless, we believe this added diversity ultimately leads to more robust generalization.\n\n### **Not covering the whole design space**\nThank you for raising this point. Equation (7) in the main paper presents a **universal formulation** that, in principle, can represent a broad class of transformations over the solution space. However, as discussed in the subsequent section, this general form comes with meaningful **practical constraints**. To navigate these trade-offs, we introduce two key concepts: _efficiently recoverable transformation_, transformations for which solutions can be computed efficiently, relying solely on analytic operations; and _solution-independent transformation_, those that preserve feasibility and optimality without requiring access to the original solution. These criteria reflect our design philosophy: to create augmentations that are not only effective, but also computationally efficient, theoretically grounded, and easy to implement in practice.\n\n### **Typos**\nThank you for pointing out the typos and formatting issues. We have corrected them and will ensure that the camera-ready version is clean and refined. \n\n**We sincerely thank the reviewer for the constructive feedback and insightful questions. We hope our responses have addressed the concerns raised and clarified the strengths and contributions of our work.**\n\n[1] You, Yuning, et al. \"Graph contrastive learning with augmentations.\" Advances in neural information processing systems 33 (2020): 5812-5823."}, {"review_id": "d9pVBfPsxJ", "reviewer": "Reviewer_PXtd", "summary": "This paper proposes a theoretically motivated data augmentation framework for LP/QP tasks by transforming the KKT system, enabling supervised and contrastive training.", "questions": "1. Please provide OOD evaluation in the supervised setting under data augmentation to isolate the benefit of augmentation itself, beyond contrastive pretraining.\n\n2. Can the authors release code and data generation scripts to ensure full reproducibility?\n\n3. How does performance vary with the number of augmented instances used during training? I suggest reporting results under different augmentation intensities.\n\n4. How does this work compare empirically and conceptually with MILP-FBGen and similar LP instance generation work?", "limitations": "See above", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. Clear theoretical foundation via affine KKT transformations.\n\n2. Augmentations are label-preserving or solution-independent, enabling efficient integration into both learning paradigms.\n\n3. Extensive experiments show strong gains under data scarcity and OOD transfer.\n\nWeaknesses:\n\n1. Experiments are limited to synthetic, fixed-scale problems; scalability remains unclear.\n\n2. The number of augmented instances during training is not analyzed, making it hard to assess the effect of augmentation intensity.\n\n3. In the LP setting, similar augmentation motivations have been explored (e.g., MILP-FBGen [1]), limiting novelty.\n\n[1] MILP-FBGen: LP/MILP Instance Generation with Feasibility/Boundedness", "was_revised": true, "final_justification": "The rebuttal addresses my concern. So i raise the score.", "rebuttal": "We thank the reviewer for the thoughtful and insightful suggestions, which we address below.\n\n### **Regarding synthetic datasets**\nWe fully acknowledge this limitation. We do an evaluation on QPLIB. Due to the limited size, large scale, and extreme heterogeneity of QPLIB instances, performing a standard train/validation/test split is infeasible. Existing works that use QPLIB for evaluation typically rely on perturbing problem coefficients [1][2]. These approaches, however, have significant limitations:\n1. The augmentations are not label-preserving and thus require solving each perturbed instance from scratch;\n2. Perturbations may break feasibility, which limits the perturbation strength;\n3. As a result, weaker perturbations are often used, but they yield training instances that are nearly identical to the test instances, making evaluation less meaningful.  \n\nOur proposed method, on the other hand, includes both structural and featural perturbation and is targeted at better generalization performance. For the current perturbation method and ours to work well, one can reduce the perturbation rate to an arbitrarily small value to fit perfectly on the test data, but that isn't very meaningful. To study transferability and fitting efficiency on QPLIB, we conduct a pretraining experiment:\n-  Step 1: We generate a foundation dataset of 10000 large-scale random QP instances with sizes ranging from 1000 to 1500 variables and constraints, being 100–200 times larger than the problems used in Table 2. Since no labels are needed for contrastive pretraining, data generation is efficient. Training takes around 30 seconds per epoch using 4 NVIDIA L40S GPUs.\n-  Step 2: We select feasible LCQP instances from QPLIB that vary a lot in size, relax integer constraints, and train an MPNN to fit these problems in a supervised setting. We compare models trained from scratch with models initialized from the pretrained weights.\n\nHere are the problem statistics and results:\n\n| Name | cons. | vars. | A density | Q density |\n|-------------|--------|-------|-----------|-----------|\n| QPLIB_3694 | 3280 | 3240 | 0.001208 | 0.000313 |\n| QPLIB_3708 | 12917 | 12930 | 0.000171 | 0.000628 |\n| QPLIB_3861 | 4650 | 4530 | 0.000856 | 0.000222 |\n| QPLIB_3871 | 1040 | 1025 | 0.003772 | 0.001000 |\n| QPLIB_8559 | 5000 | 10000 | 0.000500 | 0.000700 |\n\nWe train each model for 100 epochs and compare predicted objectives to the actual optimal value. Longer training will diminish the advantage of pretraining, as we are testing exactly on the training data. We repeat the pretraining and fine-tuning with 3 random seeds and report the mean.\n\n| ID | Pretrain | Obj.(optimal) | Obj.(predict) @ 100th epoch |\n|-------------|----------|---------------|---------------|\n| QPLIB_3694 | No | 0.0 | 10.36161 |\n| QPLIB_3694 | Yes | 0.0 | __0.14176__ |\n| QPLIB_3708 | No | -42.46981 | __-42.46981__ |\n| QPLIB_3708 | Yes | -42.46981 | -42.46916 |\n| QPLIB_3861 | No | 0.0 | 10.89544 |\n| QPLIB_3861 | Yes | 0.0 | __0.41946__ |\n| QPLIB_3871 | No | 0.0 | 7.20569 |\n| QPLIB_3871 | Yes | 0.0 | __-0.19335__ |\n| QPLIB_8559 | No | 15.79288 | 29.20809 |\n| QPLIB_8559 | Yes | 15.79288 | __12.33148__ |\n\nAs shown, pretrained models generally fit faster and more accurately within limited training epochs. QPLIB_3708 is an exception, though both models converge very close to the actual objective. These results support the scalability and transferability of our method.  \n\n[1] Wu, Chenyang, et al. \"On representing convex quadratically constrained quadratic programs via graph neural networks.\" arXiv preprint arXiv:2411.13805 (2024).  \n[2] Yang, Linxin, et al. \"An Efficient Unsupervised Framework for Convex Quadratic Programs via Deep Unrolling.\" arXiv preprint arXiv:2412.01051 (2024).  \n\n### **Regarding scalability**\nScalability is well addressed in our framework. We introduced the notion of _efficiently recoverable transformations_, whose concrete instantiations have provably linear time complexity. Empirically, we demonstrate this through our large-scale pretraining experiment on 10000 QP instances with 1000–1500 variables and constraints, as shown above for QPLIB. Since no solver is required during pretraining, the dataset creation and runtime are efficient.\n\n### **Augmentation intensity**\nThis is an excellent question. Without loss of generality, we generate one augmented instance per data point per epoch, as required by contrastive learning. For the supervised setting, we conducted new experiments, using two of the best-performing augmentations on LPs: dropping inactive constraints and scaling variable coefficients. Using the same setup as Table 1, we pick the number of augmentations per instance in {1, 2, 3}.\n\nThe results are quite counterintuitive. Interestingly, more augmentations do **not** help, except slightly in the 100% training data case. In fact, they slightly hurt performance and slow down the convergence, as we have observed in the training curves (unfortunately, we cannot provide external links or plots). From both effectiveness and efficiency standpoints, using a single augmentation per instance seems optimal. \n\n- Dropping inactive constraints\n\n| #Samples | LP 10% | LP 20% | LP 100% |\n|----------|---------------|---------------|---------------|\n| 1 |__5.563±0.097__|__3.691±0.087__| 1.656±0.057 |\n| 2 | 6.135±0.267 | 4.023±0.055 |__1.596±0.054__|\n| 3 | 6.277±0.136 | 4.092±0.029 | 1.754±0.062 |\n\n- Scaling variables\n\n| #Samples | LP 10% | LP 20% | LP 100% |\n|----------|---------------|-----------------|-----------------|\n| 1 |__4.682±0.162__| __3.208±0.116__ | 1.241±0.052 |\n| 2 | 5.027±0.113 | 3.426±0.097 | __1.204±0.033__ |\n| 3 | 5.068±0.130 | 3.518±0.103 | 1.284±0.060 |\n\n### **OOD evaluation in the supervised setting**\nThis is a good suggestion, and we will include the corresponding results in the revised paper. We pretrain on random LP instances with and without augmentation, then transfer the model to OOD LP instances—specifically, the set cover problem. For comparison, we also include results from the original paper: training from scratch on OOD data, and using contrastive pretraining.\n\nThe results show that both supervised pretraining (with/without augmentation) and contrastive pretraining outperform training from scratch. Remarkably, supervised pretraining with augmentation yields the best performance, highlighting the benefit of our proposed augmentations. Nonetheless, the label-free nature of contrastive pretraining still offers practical advantages.\n\n| Pretrain | SC 10% | SC 20% | SC 100% |\n|--------------|---------------|---------------|-----------------|\n| None | 5.297±0.787 | 2.531±0.861 | 0.752±0.042 |\n| CL | 1.965±0.206 | 1.349±0.161 | 0.662±0.055 |\n| Supervised | 1.994±0.115 | 1.521±0.085 | 0.778±0.006 |\n|Supervised+aug|__1.034±0.055__|__0.848±0.045__| __0.545±0.026__ |\n\n### **Comparing to other MILP generation work**\nWe thank the reviewer for pointing out these related works. This is a sharp and knowledgeable observation. Indeed, recent MILP instance generation methods such as MILP-FBGen address data scarcity in learning-based optimization, but their objectives and methodologies are fundamentally different from ours. We have already conducted a literature survey for these methods in Appendix C. \n\nWhile both our work and the related work aim to enrich training data, their approaches focus on generating new instances that are distributionally similar to the original problem set, typically using generative models or with strong assumptions such as diagonalizable matrices. These methods often ignore labels and may not guarantee feasibility or boundedness without strong structural assumptions. Moreover, solving these generated MILPs still requires a solver, adding to the computational cost. These limitations make it hard to compare our method to theirs empirically. \n\nIn contrast, our approach is mathematically grounded, learning-free, and solver-free. We design label-preserving augmentations by transforming the KKT system of a given LP/QP instance, enabling us to generate semantically equivalent problems—i.e., those with the same solution and objective. This is crucial for both supervised learning, where we require efficient label computation, and self-supervised contrastive pretraining, where we need two solution-preserving views of the same instance. These transformations are derived analytically and allow us to recover solutions efficiently, without retraining a model or solving the problem again. Besides, we do not focus on preserving the structural distribution of the generated instances, which can potentially cover a broader spectrum of instances. \n\n### **Code release and data generation**\nYes, we have included a link to our anonymous repository in the main paper, which has been recently updated. However, due to NeurIPS's new regulations, we are not permitted to include it in the author response. Additionally, we have provided a zip file with code and data in the supplementary materials. We kindly ask the reviewer to check again. \n\n**We sincerely appreciate the reviewer’s valuable comments and have carefully addressed all concerns in our response. We hope the revisions address your concerns and kindly request reconsideration of the evaluation scores.**"}, {"review_id": "BPgGSchgbg", "reviewer": "Reviewer_ahYQ", "summary": "This paper presents a solid and comprehensive analysis of data augmentation for both linear programming (LP) and quadratic programming (QP) problems. The authors propose a rigorous theoretical framework for identifying valid data augmentations, grounded in dual problem formulations. Importantly, the method maintains computational efficiency, making it practical for large-scale applications. Leveraging these theoretical guarantees, the authors generate additional data to enhance both supervised learning and pretraining tasks, effectively addressing the issue of limited original data. Experimental results are compelling, demonstrating substantial performance improvements with comparable computational costs to training without data augmentation.", "questions": "See Weaknesses.", "limitations": "None.", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths:\n\n1. The theoretical foundation is solid, and the proposed data augmentation method effectively addresses the primal problem under limited data settings.\n\n2. The experimental results are strong, demonstrating the practical effectiveness of the approach across various benchmarks.\n\nWeaknesses:\n\n1. Can the author provide some experimental results on public datasets such as QPLIB?\n\n2. The proposed data augmentation guarantees that the solution remains unchanged. I am curious whether there exists a way to design augmentations that introduce slight variations in the solution, in order to obtain more diverse augmented data?", "was_revised": true, "final_justification": "I think the work has valuable impact so I raise my score.", "rebuttal": "We sincerely thank the reviewer for the thoughtful and encouraging review. We are grateful for your recognition of both the theoretical contributions and the practical effectiveness of our method. Below, we address your questions in detail. \n### **Experiments on public datasets such as QPLIB**\nThank you for this suggestion. We fully acknowledge this limitation. We do an evaluation on QPLIB. Due to the limited size, large scale, and extreme heterogeneity of QPLIB instances, performing a standard train/validation/test split is infeasible. Existing works that use QPLIB for evaluation typically rely on perturbing problem coefficients [1][2]. These approaches, however, have significant limitations:\n1. The augmentations are not label-preserving and thus require solving each perturbed instance from scratch;\n2. Perturbations may break feasibility, which limits the perturbation strength;\n3. As a result, weaker perturbations are often used, but they yield training instances that are nearly identical to the test instances, making evaluation less meaningful.  \n\nOur proposed method, on the other hand, includes both structural and featural perturbation and is targeted at better generalization performance. For the current perturbation method and ours to work well, one can reduce the perturbation rate to an arbitrarily small value to fit perfectly on the test data, but that isn't very meaningful. To study transferability and fitting efficiency on QPLIB, we conduct a pretraining experiment:\n-  Step 1: We generate a foundation dataset of 10000 large-scale random QP instances with sizes ranging from 1000 to 1500 variables and constraints, being 100–200 times larger than the problems used in Table 2. Since no labels are needed for contrastive pretraining, data generation is efficient. Training takes around 30 seconds per epoch using 4 NVIDIA L40S GPUs.\n-  Step 2: We select feasible LCQP instances from QPLIB that vary a lot in size, relax integer constraints, and train an MPNN to fit these problems in a supervised setting. We compare models trained from scratch with models initialized from the pretrained weights.\n\nHere are the problem statistics and results:\n\n| Name | cons. | vars. | A density | Q density |\n|-------------|--------|-------|-----------|-----------|\n| QPLIB_3694 | 3280 | 3240 | 0.001208 | 0.000313 |\n| QPLIB_3708 | 12917 | 12930 | 0.000171 | 0.000628 |\n| QPLIB_3861 | 4650 | 4530 | 0.000856 | 0.000222 |\n| QPLIB_3871 | 1040 | 1025 | 0.003772 | 0.001000 |\n| QPLIB_8559 | 5000 | 10000 | 0.000500 | 0.000700 |\n\nWe train each model for 100 epochs and compare predicted objectives to the actual optimal value. Longer training will diminish the advantage of pretraining, as we are testing exactly on the training data. We repeat the pretraining and fine-tuning with 3 random seeds and report the mean.\n\n| ID | Pretrain | Obj.(optimal) | Obj.(predict) @ 100th epoch |\n|-------------|----------|---------------|---------------|\n| QPLIB_3694 | No | 0.0 | 10.36161 |\n| QPLIB_3694 | Yes | 0.0 | __0.14176__ |\n| QPLIB_3708 | No | -42.46981 | __-42.46981__ |\n| QPLIB_3708 | Yes | -42.46981 | -42.46916 |\n| QPLIB_3861 | No | 0.0 | 10.89544 |\n| QPLIB_3861 | Yes | 0.0 | __0.41946__ |\n| QPLIB_3871 | No | 0.0 | 7.20569 |\n| QPLIB_3871 | Yes | 0.0 | __-0.19335__ |\n| QPLIB_8559 | No | 15.79288 | 29.20809 |\n| QPLIB_8559 | Yes | 15.79288 | __12.33148__ |\n\nAs shown, pretrained models generally fit faster and more accurately within limited training epochs. QPLIB_3708 is an exception, though both models converge very close to the true objective. These results support the scalability and transferability of our method.\n\n[1] Wu, Chenyang, et al. \"On representing convex quadratically constrained quadratic programs via graph neural networks.\" arXiv preprint arXiv:2411.13805 (2024).  \n[2] Yang, Linxin, et al. \"An Efficient Unsupervised Framework for Convex Quadratic Programs via Deep Unrolling.\" arXiv preprint arXiv:2412.01051 (2024).  \n\n### **Regarding augmentations that introduce slight variations**\nThis is an excellent and insightful idea. Indeed, Equation (7) in the main paper defines a highly expressive framework that, in principle, covers the whole space of linearly-constrained quadratic programs (LCQP), including those that induce slight deviations in the solution. However, incorporating such deviations into data augmentation presents several practical and theoretical challenges:\n- **In the supervised setting**, even a small change in the solution still requires solving the new problem, making it as computationally expensive as generating an entirely new instance. Thus, the advantage of analytic recoverability of our framework is lost. \n- **In the self-supervised setting**, introducing slight solution changes raises the question of **how much** deviation is acceptable. In other words, it is unclear when an augmented instance retains the same semantic meaning. This introduces additional hyperparameter tuning and problem-specific calibration. While minor deviations may be harmless in practice, the boundary is unclear and complicated to formalize.\nWe do explore this direction heuristically in Appendix D.3, where we describe an augmentation strategy that may slightly alter the solution while maintaining approximate correctness. This heuristic is included in our contrastive pretraining experiments (Table 2), and no harm in empirical performance is observed. \nGenerally, perturbing continuous optimization problems is fundamentally different from perturbing problems with discrete labels. In our case, even a small change to the problem data can yield a quantitatively different solution, and solvers are generally not designed to exploit similarity between near-identical instances unless warm-starting is explicitly supported.\nWe believe your suggestion opens an interesting topic for future work. Developing solution-perturbing augmentations with provable guarantees could greatly enrich the current augmentation space, especially for self-supervised learning.\n\n**We appreciate the reviewer’s thoughtful comments and valuable suggestions. We hope our detailed responses have sufficiently addressed the concerns and helped clarify the contributions and scope of our work.**"}]

Author rebuttal:

Meta-review: The paper presents a data augmentation approach for quadratic programming problems for use in machine learning approaches for optimization. Reviewers unanimously agree on the merits of the work, find it clever, with strong experimental results, nice mathematical foundation. The extensive rebuttal and discussion phase is noted, that successfully addressed raised concerns w.r.t. generalizability.
The paper meets the bar for acceptance at NeurIPS and might be of broader interest for the ML4OPT community.
Decision: Accept (spotlight)