MILPnet: A Multi-Scale Architecture with Geometric Feature Sequence Representations for Advancing MILP Problems
Authors: Ruobing Wang
Xin Li
Mingzhong Wang
Venue: ICLR 2026 Poster (2026)
Keywords: MILP;optimal
Primary area: optimization

Abstract: We propose MILPnet, a multi-scale hybrid attention framework that models Mixed Integer Linear Programming (MILP) problems as geometric sequences rather than graphs. This approach directly addresses the challenge of Foldable MILP instances, a class of problems that graph-based models, specifically Graph Neural Networks (GNNs), fail to distinguish due to expressiveness limits imposed by the Weisfeiler-Lehman test. By representing MILPs through sequences of constraint and objective features, MILPnet captures both local and global geometric structure using a theoretically grounded multi-scale attention mechanism. We theoretically prove that MILPnet can approximate feasibility, optimal objective value, and optimal solution mappings over a measurable topological space with arbitrarily small error. Empirically, MILPnet outperforms graph-based methods in feasibility prediction accuracy and convergence speed on Foldable MILPs, while using significantly fewer parameters. It also generalizes effectively among problem scales and demonstrates strong performance on real-world MILP benchmarks when integrated into an end-to-end solver pipeline. Our code is available.

Published as a conference paper at ICLR 2026
MILPNET: A MULTI-SCALE ARCHITECTURE WITH GE- OMETRIC FEATURE SEQUENCE REPRESENTATIONS FOR ADVANCING MILP PROBLEMS
Ruobing Wang', Xin Li Mingzhong Wang* 'Beijing Institute of Technology, Beijing, China University of the Sunshine Coast, Australia (3120245684, xinii @bit. edu. en, mwang@usc.edu.
ABSTRACT
We propose MILPnet, a multi-scale hybrid attention framework that models Mixed Integer Linear Programming (MILP) problems geometric sequences rather than graphs. This approach directly addresses the challenge of Foldable MILP instances, class of problems that graph-base baseed models, specifically Graph Neural Networks (GNNs), fail to distinguish due to expressiveness limits imposed by the Weisfeiler- Lehman test. By representing MILPs through sequences of constraint and objective features, MILPnet captures both local and global geometric structure using a theo- retically grounded multi-scale attention mechanism. We theoretically prove that MILPnet can approximate feasibility, optimal objective value, and optimal solution mappings over measurable topological space with arbitrarily small error. Empiri- cally, MILPnet outperforms graph-bases methods in feasibility prediction accuracy and convergence speed on Foldable MILPs, while using significantly fewer param- eters. It also generalizes effectively among problem scales and demonstrates strong performance on real-word MIIP benehmarks when integrated into end-to-end solver pipeline. The code is available at the link
INTRODUCTION
Mixed-integer linear programming (MILP) is foundational combinatorial optimization problem characterized by linear objective function and linear constraints, with decision variables that can be either continuous discrete. This flexibility makes MILP highly expressive and applicable in diverse theoretical and real-world domains, such as probabilistic optimal transport theory (Goldman Trevisan (203) and transportation systems (Wang et al.) 2023a), route optimization (Bula et al. 2016 Mammeri 2019 Chen et al. 2023a), and power system planning (Zelaschi al. 2025 Zhang ct al. 2020).
However, as well-known NP-hard problem, solving MILP remains significant challenge. Tra- ditional methods, such as Branch-and-Bonn annd-Bound (Land & Doig 1960) and Cutting Planes (Gomory 2010), are commonly employed but become impractical for large-scale instances due to their intensive resource requirements.
Recently, Machine Learning methods have emerged as promising alternative. ML models can approximate solutions efficiently by leveraging the implicit structure and patterns within MILP problems and integrating them with reinforcement learning or MILP Exact solvers. These approaches can significantly reduce computational costs while delivering effective solutions within practical timeframes (Bengio al. 2020 Wu & Yang 2022 Wang al., 2024). Generative models, including diffusion models, have also been explored for solving structured MILP variants, such as the Traveling Salesman Problem (TSP) and the Maximum Independent Set Problem (MIS) (Sun Yang 2023
Sanokowski et al. 2024 Ma et al. 2024), showing strong performance and generalization.
"Corresponding author.
https://githubs jtoo///jkkkae faagkk/kjjaaaaahhh//aaassee

---

Published as a conference paper at ICLR 2026
A recent trend has been to solve MILPs using Graph Neural Networks (GNNs), treating MILP instances as bipartite graphs that link variables and constraints (Han et al. 2023 Ye et al. 2023
Paulus & Krause 2023 Geng al. 2025 Liu et al. 2025). However, Bipartite graphs and GNNs can capture relationships between constraints and variables, missing the interactions between the constraints themselves, which potentially contain crucial features, such feasible regions optimal solutions of the MILP. Thus, GNN-based solutions suffer from fundamental limitation: they cannot distinguish between non-isomorphic MILP instances that differ in feasibility, due the expressive bounds of the Weisfeiler-Lehman (WL) test. As shown in recent work (Chen et al. 2023b), this leads to failure cases known as Foldable MILPs, where multiple distinct MILP instances are indistinguishable to GNNs but differ critically in their feasible regions.
Although, recent MIIP represeettnnnnnnnnn research (Chen et al. 2023b) partially addresses this by injecting random features into graph structures. However, they only bypass WL-test limitations without cap- turing fundamental characteristics of MILP instances themselves. Current graph- based models remain in- adequate for robust feasibility prediction and high- fidelity representation of general MILP problems, especially Foldable MILPs.
Overall Performance Comparison
Inference Time
GB% KAUL
Generalization
Accuracy
To overcome these limitations, we propose novel representation framework that departs from the graph paradigm and instead adopts a computational geome- try perspective. Considering the geometric and topo- logical uniqueness of MILP problems (Huchette & Vielma 2019 Conforti et al., 2010), we encode each MILP instance by extracting spatial geometric fea- ture vectors, including hyperplane vectors from linear constraints, discrete integer point features, and direc- tion vector from the linear objective function, and assemble these into a sequence. To the best of our knowledge, this is the first work to represent MILPs sequences.
GPU
Usage
Pre-train
Time
Figure 1: Overall performance comparison between MILPnet and graph-based models. Larger area indicates better performance.
Given the complexity of MILP problems, we propose a Multi-Scale Hybrid Attention mechanism that enables our model, MILPhet, learn both local and global features from the MILP sequence. Furthermore, we mathematically prove that our model can effectively approximate the feature mappings of any MILP instance with arbitrarily small error.
We validated our approach on Foldable MILP instances, which are specifically challenging for GNN- and Graph-based models. Across feasibility, optimal solution, and optimal object value predictions, MILPnet achieves great improvements of in accuracy and convergence speed, while using significantly fewer parameters and pre-training time. As summarized Figure I MILPnet consistently outperforms baseline models across five key dimensions: generalization. accuracy, model size, inference time, and GPU memory usage. These results highlight the effectiveness of our geometric modeling and its broad potential.
2 RELATED WORK
MILP is a classic NP-hard problem (Karp) 1972). Traditional methods. such as branch-and aad---nn--bdddddd (Land & Doig 1960) and cut-plane methods (Gomory 2010). typically solve MILPs by simplifying or relaxing the problem. However, these methods can grow exponentially with the number of variables, making them computationally prohibitive for real-world or time-sensitive applications.
To address these challenges, machine learning methods have been applied to accelerate the MILP solving process(Bengio al.) 2020). Specifically, most methods model MILP instances with GNNs, framing them as weighted bipartite graphs (Gupta et al., 2020; Nair et al. 2021). These graphs typically consist of two disjoint sets of nodes: variables and constraints (Chen et al. (2233)). Message- passing mechanisms within GNNs are then used to capture their structural relationships. ML-based
methods for MILP solving have been advanced by diffusion-modes model-based solvers (Sun & Yang 2023
2

---

Published as a conference paper at ICLR 2026
Zhao et al.
2025 Sanokowski et al. 2024), predict and search solvers (Han et al.
2023 Huang et al.
2024j. reinforcement learning methods (Wang et al. 20024] Feng & Yang 2025 machine learning
methods which reducing the problem solving complexity combined with heuristic algorithms (He
et al. 2014 Gasse et al. 2019 Chmiela et al. 2021 Paulus & Krause 2023 Huang et al. 2023).
Formally, bipartite graph is denoted as G (V U E, where V and W represent the variable and constraint nodes, respectively. The set Gm,z includes all such graphs with |V] = m and
[W| ni, while the complete graph representation with node features for an MILP is given by (G, H) Gm,m x Hy x Hn,
While these graph-based models effective in capturing variable-constraimg relationships, they inherently miss higher-order interactions, particularly between constraints themselves, which may encode crucial information about feasibility or optimality. Therefore, for a class of MILP instances known as Foldable MILPs (Theorem D. GNN-based models cannot represent their feasibility as the underlying WL-testt cannot distinguish non-isomorphic graphs. Conversely, MILP instances that are not Foldable MILPs are Unfoldable MILPs.
Theorem 1. (Lemma 3.2. in (Chen et al. 2023b)) There exist two MILP problems (G, H) and (G, H). with one being feasible and the other one being infeasible. such that (G, H) ~ (G, H).
This finding underscores a critical gap: graph-based models, regardless of feature augmentation, fundamentally cannot resolve the feasibility of Foldable MILP instances. It signifies the need for alternative representations beyond graph-based modeling to enhance the fcasibility prediction for complex MILP problems.
(a) MILP Sequence with Geometric featuress
(b) Multi-scale Computation
Multi Scale Local
Hyperplane 3
Geometrization
MILP
T/s : The Size of the Local Set
Sequence
Mans
MTLP IIL.Pnsttacce
A
z subjjcci
min
Padding
Aruh a S u. 2 22 E...
Aons A'
t= Buf ou U
A"
A
Σ
Figure 2: Overview of MILPnet. (a) An MILP instance is transformed into a sequence of geometric feature tokens, including constraint hyperplanes, variable bounds, integer indicators, and the objective vector. (b) The sequence is processed by multi-scale hybrid attention architecture, enabling accurate approximation feasibility, optimal objective value, and solution mappings.
PRELIMINARIES
MILP formulation. MILP is an NP-hard optimization problem characterized by a linear objective function and a set of linear constraints, with a subset of variables restricted to integer values. The standard formulation is:
min C x s.t. Arob I5=5u, 2j ZÆEZVEEL I ZER
(I)
whereA R be RT and c R" are the problem parameters, I, (IR U (
} specifies the variable bounds, o E (,,=,>) pm denotes the constraint types, and / C {I,..., n) indexes the variables that required to be integer.
Feasibility, From a geometric perspective, define the continuous feasible region as a polyhedron /} and the integer lattice Z {I E R" x. E Z, Vj E I).
P = { R" Ax o b, L S The MILP feasible set is their intersection S = Pnžy, ie. all points that satisfy the linear constraints and bounds while taking integer values in the specified dimensions. If S 0, the MIIP is infeasible.
3

---

Published as a conference paper at ICLR 2026
Optimal solution and optimal objective value. point x= S is called optimal solution if it minimizes the objective function over all feasible points: c x* I c z; va E S. The corresponding scalar e x" is the optimal objective value. If the objective be decreased arbitrarily, i... for any e > 0, there exists & E S such that e & < -, the MII. unbounded and the optimal value -oo.
SEQUENCE MODELING FOR MILP: MILP-SEQUENCE
This section introduces the first component of MILPnet (Figure 2 (a)), which encodes an MILP instance into sequence representation from a geometric perspective. The MILP instance is decom- posed into multiple components, each modeled in a well-defined topological space, forming what we refer to as the MILP -lLL-eeeeeece.
4.1 GEOMETRIC MODELING OF MILP
We first reformulate the MILP problem from geometric perspective. The MILP space consists of nigh-dimensional hyperplanes or half-spaces (from linear constraints and variable bounds), discrete point sets (from integrality requirements), and direction vector (from the objective function). Together, these are represented as Poonstraint U Prange U UIUc. c. Each of these components is defined within a topological space as follows:
Linear constraints. The linear constraints are represented as the union of hyperplanes: Peonstraint U H4, where H; is the i-th hyperplane half-space defined by the vector hionss (n,, di, b.). where n, E IR" is the normal vector of H;, d, E {-1, 0, 1 denotes its directional type, and b, E R is the bias term. h; is chosen from the topological space Hrons (R") {-1,0,1) R.
Variable bounds. The bounds on variables define upper and lower half-spaces: Prange (U)=1 Hupper.s) U (U (U) Hoowess, whereHupper.s f0 E R 2ÿ S uz} and Hoower = for E R" az 2 l.). We encode the bounds as vectors h=== (1.42 ................................ (n) and h (wI, u Un), drawn from the topological spaces [[ La C TR. and U -11 U, C R"" respectively, each equipped with the standard Euclidean topology. II denotes the Cartesian product. The combined vector é U belongs to the space H I
Integer set. The integrality constraints are encoded using a binary vector: h = (i,, i2, ,g....,, in) E (0, 1)", where is = 1 if x is constrained to be integer, and ig 0 otherwise. This vector resides in the discrete topological space {0, 1
Linear optimization direction. The objective function is represented by a coefficient vector h = (2., E R", drawn from the topological space /1 = 12"
It is worth noting that, although the MILP feasible region is mathematically defined by intersections, we use unions in our geometric reformulation to denote the collection of individual components (c.g., constraint hyperplanes and bounds) treated as sequence tokens. This formulation expands the representational space, enabling MILPnet to more effectively explore the solution structure and approximate optimal outcomes.
4.2 MEASURES THE MILP GEOMETRIC SPACES
To unify component representations and manage dimensionality differences, we apply zero-padding, yielding two related topological spaces MILPO (original) and MIMILP (padded). The padded compo- nent spaces include _ H H I As shown in Theorem 5 Appendix, the spaces padded and non-padded are homeomorphic, ensuring no representation loss.
We equip each continuous space (R" R2n RR with Borel o-algebra and Lebesgue measure, and each discrete component (f-1,0, 1}, [0]. 13") with the counting measure. By product construction, the component measures Plyccous Az- x #(-1.0,1) Ag. Py Apan X 1(0): '' Apn PLo.- By "{oo1)" KGoj- The overall padded topological space is HMILP (HHoons) m HVar) I' Hob', which is measurable by construction.
Measures.
4

---

Published as a conference paper at ICLR 2026
4.3 GEOMETRIC FEATURE INTEGRATION
We now integrate the MILP geometric feature vectors into sequence-based representation, treating each feature vector as token.
MIL.-eeeue Since the linear constraints hooee integer point sets h² and variable ranges h." and 1 are permutation invariant (as formalized in Theorems and 7 in Appendix), we can arrange them in any order. To preserve this invariance and recognize the term objective function's 's unique role, we place the objective token h the end of the sequence. Thus, the Mll.P-sequence is:
[ N
conss .... h cons: h²" he h ho',, h E R
(2)
The topological space corresponding to the MILP-sequence is defined as HMILP C (R((++)(++))
Sequence-based mappings. With this representation, define the mappings used analyze MILP instances: Feasibility mapping fee.. HMMILP - {0, 1] Optimal objective value mapping
ony HMILP + U (oo,- -oo). and Optimal solution mapping H/MILP, O $ Ire (1) - RR. These mappings are formally defined in Definitions to 10 in the Appendix. We prove their measurability in Appendix G
MULTI-SCALE HYBRID ATTENTION FOR MILP-SEQUENCE
This section introduces the second core component of MILPnet (Figure (b)): novel Multi-Scale Hybrid Attention mechanism specifically designed to model MILP sequences. This architecture enables the network to capture both fine-grained local structure and global context by combining multiple levels of attention. It operates directly on the MILP sequence defined in the previous section and is supported by a rigorous approximation theory over the measurable space MMMLP..
5.1 SHIFTED WINDOW MULTI-SCAM ATTENTION
The geometric structure of MILPs suggests that relationships among constraints can reflect topological characteristics the feasible region. To leverage this structure, we employ multi-scale local attention via sliding windows that extract features from various neighborhoods in the MILP-ssqqence.
Shifted- Window Local Attention. Given an embedded MIL.Psseqeen X emmed E IR ()++)xx((+2)))))) derived through linear projection, we define local attention windows that slide across the sequence. For each token at position i E {1, ..,..., 4), window of size 7h is centered at i, covering elements to position imax min(i + ]=--
+ 4). To effectively capture the relationship between multi-level features and the MILP's overall goal, this window also needs to incorporate the embedding of the objective function feature & at position m + 4. Therefore,
from position min max( (i [
the position indices 7) (i) for the window attention position i with the window size me are defined as:
m (i) {j [imin: Ammxx] |} U {m + 4). Then, the local attention at position for scale 77 is computed as:
Q
W Q
W embed.
KTt W Xembod,
V W WyXXeebed.jj
j (1)
(4)
Qthh -K3"
7% softmax
j E (1), Att = The V
Σ
(5)
Vdle
jEm(()
where W Q. W K. Wv v are the linear transformation matrices for Query, Key, and Value, and de is the dimension of the key vectors, which is split from the embedding size daim by the softmax function is used to normalize the Attention weights.
Multi-Scale MILP MILP-sequence Attention. To aggregate information at different granularities, we apply local attention using N window sizes. The resulting outputs averaged to produce the multi-scale representation, wiaien which is like the style in the Figure (b).i.
m+1 N
N m44
Att multi
Att
Σ
Th ag
(6)
N
N
k=1
(x)
5

---

Published as a conference paper at ICLR 2026
where window sizes 7h [2, + 4] allow for comprehensive evaluation of local and global contextual influences, the summation over i represents concatenation along the sequence dimension. Let 7mmax maxx (ms) denote the largest window size.
Hybrid Attention Integration. To capture both multi-scale locality and global context, we define hybrid attention mechanism that integrates multi-scale attention with global attention using learnable parameter a:
At Att. hybrid = 0: Att multi +
a) Att.glohal
As illustrated in Figure 2 each MILP instance is first geometrically encoded as a feature vector in R=+2 and then padded to form MII.P-sequence of length + 4. This sequence undergoes linear projection and position encoding to produce the embedded input:z [Z1, 22. Zx+2) + 2position With this embedded sequence, MILPnet applies its Hybrid Attention (HYA) mechanism, followed by residual connections and layer normalization. The computation at the (-th layer proceeds as:
HYA(Z ()A(zz"))+ +)=='
MLP(LN(Z')) + LN(z')
(8)
where LN denotes layer norm, MLP refers to a position wiie feedforward network. More detailed information is in Figure 5 from Appendix.
Time Complexity. The computational complexity of the hybrid attention is O 4m) (N+1)) where h is the number of attention heads, d is the embedding size, m is the number of the constraints, and N is the number of windows. Detailed analysis is provided in Appendix
5.2 MILPNET REPRESENTATION ON THE MEASURABLE SPACE MILPP
We now formally demonstrate that the multi-scale sequence architecture of MILPnet is capable of approximating the feature mappings of any MILP instance, when represented as sequence in the measurable topological space H We define two function classes: НYA MMl,, HMIPP - R for scalar-valued network mappings, and JHYA, MMLP F for vector-valued network mappings with fixed output dimension n. By leveraging these mappings and the measurability structure introduced in Section 4.2 and 4.3 we can prove that for any MILP instance viewed as sequence:
MILPnet can uniformly approximate MILP feasibility mapping, MILP optimal-solution mapping, and MILP optimal objective value mapping.
The following theorems formally establish MILPnet's approximation capabilities (see Appendix G for detailed proofs and corollary on infinite set):
Theorem 2. Let D C MMP be a funite dataset. For any e > 0, there exists a neural network such that:
FuYA J HYA
P (I # ()oa(()) <6, Vx E D,
Theorem 3. Let D C HIILP be a finite dataset. For any c, O, there exist two neural networks Puet such that for classifying whether the objective value is finite:
FHYA,1 FuYA,2 E HYA
P P( F Loaa(a)eR) <e. <<. .a E D
(10)
where Td is an indicator function that determines whether the objective value is finite. And for the regression problem of predicting the objective value:
P P((FuAA2(Z) ( 6) <c. Vx D 1D
(11)
Theorem 4.
Let D C
1 2-1 [R) c. MILP be finite dataset. For any 6 > 0, there exists Hybrid
attention based network FHVA,Y Mill. such that:
P F(IA.v)))))) - (x)) >))
VE E
(12)
To train any MILPnee Fg to approximate these mappings, minimize the emror between MILPhet and the feature mapping as the loss function L(Øb= = Ellly - [2), where y is ground truth.

---

Published as a conference paper at ICLR 2026
Table 1: Generalization results for feasibility mapping on FOLD(20. *) and FOLD(50, *), with 10,000
foldable instances per setting and pre-training times of 3 and 10 minutes. Full results are provided in Appendix B.5
Method
Type
F0LD(20,6)
FOLD(20,16)
F0LD(50,20)
FOLD(S0),0)
ErroNN Params
MSE
ErrosN
Params
MSE
ErrorN
Params
MSE
ErroNN
Params
SCIP
Exact
0
GCN
Graph
0.3073
5000
1.21M
0,3073
5000
1.21M
0.3073
5000
1.21M
0.3073
5000
1.2TM
GIN
Graph
0.4200
5000
1.63M
0,4209
5000
1.63M
0.4200
5000
1.63M
0.4204
5000
166M
SAGE
Graph
0.4642
5000
0.66M
0.4999
5000
0.66M
0.4714
5000
0.66M
0.4586
5000
0.66M
PGN
Graph
0.2508
5000
1.64M
0,2523
5000
1.64M
0.2511
5000
1.64M
0.2511
5000
1.64M
GraphGPS
Graph
0.2500
5000
0.66M
0.2500
5000
0.66M
0.2500
5000
0.66M
0.2500
5000
0.66M
GCN"
RI Graph
0,2498
5000
1.21M
0,2500
5000
1.21M
0.2476
4334
1.21M
0.5223
5000
1.21M
GIN
Rf Graph
0,2500
5000
1.63M
0.2501
5000
1.63M
0.2500
5000
1.63M
0,4204
5000
1.63M
SAGE
Rf Graph
0,2499
5009
0.66M
(124999
4995
0.66M
0.2500
4997
0.66M
0.2500
5002
0.66M
PGN"
Rí Graph
0,2582
5000
1.64M
0,2560
5000
1.64M
0.2502
5000
1.64M
0.2502
5000
1.4M
GraphGPS
Rf Graph
0.2510
5000
0.66M
0,2510
5000
0.66M
0.2500
5000 0,0005
0.66M
0.2502
5000
0.66M
MII.Pret
(Seq)
0.56M
0.60M
0.60M
Graph based
Graph based
MILPPest Serpeeeec based)
EXXOR number
nunnber
4000
Entrr sumber
4000
4000
2000
2000
2000
2000
2000
4000
2000
4000
2000
4000
2000
4000
(6) FOLD(20.)
10)
(c) FOLD(50,20)
FOLD(50,30)
Figure 3: Representation learning Dynamics on feasibility for FOLD20 and FOLD50. Full results are in Figure in Appendix.
6 EXPERIMENT
We conducted comprehensive experiments to evaluate the effectiveness, efficiency, and generalizabil- ity of MILPnet answer:RQ1: How effectively does MILPnet represent and generalize on Foldable MILP instances? RQ2: How does MILPnet perform on real-world MILP instances? RQ3: How do MII.Pnet's architectural components impact performance?
6.1 EXPERIMENT SETUP
Datasets. We evaluate MILPnet on two categories of datasets: (1) Synthetic Foldable MILP Instances (FOLD(n, m)): Following (Chen et al. 2002b)), we generate Foldable MILP instances where and m are the number of variables and constraints, respectively. (2) Real- World MILP Benchmarks: We select four common MILP benchmarks: IP (Item Placement) from ML4CO competition dataset (Gasse let al.
SC (Feige 1998 Chvátal 1979), CA (Leyton-Brown et al. 2000), and FC (Charikar et al. 1999.. They are all Unfoldable MILP instances in Classical ML4CO paper (Gasse et al. 2019).
2022),
More details on the datasets are provided in Appendix K.3 and Appendix K.5
Baselines. We compare MILPnet against several representative traph-basce learning models, includ- ing GCN (Chen et al.) (023b), GIN (Xu et al.) 22119) (Wu et al.
(Cappart et al.)
2022 Veličković et al. 2020 GraphGPS (Wang al. 2023b). Each baseline has augmented variation with random features (RF), indicated by superscript specifically designed to overcome
if
WL-test limitations in feasibility prediction. Appendix K.I provides more details on the baselines.
Metrics. We use MSE (Mean Squared Error) and ErrorN (number incorrect predictions) (Chen
al. 2023b), Params (number of model parameters) to evaluate performance. Lower is better.
7

---

Published as a conference paper at ICLR 2026
Table 2: Generalization experiments for End-to-
Table 3: Generalization experiments for End-to- End optimal objective value prediction
End optimal solution predict 1-hour of pre-train.
Method
Type
FOLD220)))
FOLD120,
Method
Type
FOLD1/20,6)
20116)20116)
MSE
Params
MSE
Params
MSE
Params
MSE
Params
GCN
Graph
0.0751
117M
0.2000
1.7M
GUN
Graph
1.776e-10
0.86M
3:928c-10
0.58M
GIN
Graph
0.0753
1.59M
0,2000
1.59M
GIN
Graph
6.489c-10
0.12M
6.489c-10
0.16M
SAGE
Graph
0.0750
0.60M
0.2000
0.60M
SAGE
Graph
2.235e-9
0.66M
6.742e 10
0.60M
PGN
Gruph
70.1485
164M
20,5955
1.6MM
PGN
Gruph
1.958e 10 291M
5.4934e 10
2.91M
GraphGPS
Griph
0.0863
0.60M
4.0395
0.60M
GraphIPS
Graph
3.769e-6
0.92M
1. 1.365e-05
0.92M
MlLPPee((urr
Sequence
0,0473
0.56M
0.1964
0.56M
(Ours)
Sequence
1.309e-00
0.56M
2.828e100 10
0,56M
Method
Type
20LD150,20)
50,
Method
Type
FOLD550,20)
FOLDE50,300
MSE
Params
MSE
Params
MSE
Parans
MSE
Params
GCN
Graph
0.1000
1.77M
0.1309
1.17M
GCN
Graph
3.3536-10
0.96M
5.80126-10
0.96M
GIN
Graph
0.1000
1.59M
0,1501
1.59M
GIN
Graph
2.629c-10
1.30M
3,458c-10
1.47M
SAGE
Graph
0.1000
0.86M
0.1500
0.86M
SAGE
Gmaph
1.219e-8
0.66M
7.555e-9
0.92M
PGN
Gruph
55,8530
1.6MM
36.5160
1.6M
PGN
Graph
3.935e-10
10 291M
3.762e
2.30M
GraphGPS
Griph
0.1020
0.67M
0.1551
0.67M
GraphGPS
Graph
2.531e-7
0.92M
3.442e-6
0.92M
(Our))
Sequence
0.1158
0.62M
0.1654
0.63M
(ILrs
Sequence
9,458e-12 0.60M
3,007e-10
0.60M
Table 4: Generalization results for feasibility prediction on larger Foldable instances FOLDD200,20), FOLD(300,40), and FOLD(500,60), each with 10,000 instances and I hour of pre-training.
Method
Type
200)
(00
VOLD(500,60)
MSB
ErroNN
Params
MSE
ErroNN
Params
MSE
ErroNN
Params
SCIP
Exact
GCN
Graph
0.2676
3000
0.03M
0.3073
3000
0.08M
0.2500
4959
0.14M
GIN
Graph
0,2573
5000
0.04M
0,3099
5000
0.10M
0,2500
5000
0.21M
SAGE
Graph
0.2814
5000
0.03M
0.3951
5000
0.05M
0.2500
5000
0.12M
PGN
Graph
0,2508
5000
0.08M
0.2523
5000
0.14M
0,2523
5000
0.29M
GraphGPS
Graph
0,2500
5000
0.03M
0,2500
5000
0.59M
0,2501
5000
0.92M
GCN
Rí Graph
0.03M
0.2500
0.80M
0,2611
0.14M
GIN
Rf Graph
0,2500
4998
0.04M
0,2500
5012
0.10M
0,2501
5003
0.21M
SAGE
Rf Graph
0,2500
5011
0,03M
0,2500 4998
0,05M
0,2500
4981
0
PGN"
Rf Graph
0,2544
5000
0.08M
0,2500
5000
0,14M
0,2506
5000
0
GraphGPS
Rf Graph
0,2502
5000
0.03M
0,2510
5000
0.05M
0,2506
5000
0.92M
Pnett
Seg (Ours)
0.02M
560
0.05M
0,1832
0.11M
6.2 PERFORMANCE ON FOLDABLE MILP INSTANCES
Representation Effectiveness. We evaluate MILPnet on Foldable MILP instances with increasing complexity: FOLD(20,6), F0LD(20,16) F0LD(50,20), and F0LD(50,30). Figure 3 shows that MILPnet converges rapidly to near-zero ErrorN while standard graph-based models fail to improve beyond their initial performance. Models with random feature augmentation show moderate improve- ments on simpler instances but struggle to converge on more complex cases such as FOLD(50.35) Similar trends are observed in MSE (more details in Appendix (..4). These results confirm that MILP. net effectively captures the geometric and combinatorial structure of MILPss, providing empirical support for the theoretical guarantees Theorems 2
Generalization Abilities. We evaluate MILPnet on both small (FOI.D(0,**) FOL.D500 *)) and large (F0,.D(000,20), FOID(300,40)) and FOLD(500,60) Foldable MILP in- stances under equal pre-training time, focusing on three End-to-End prediction tasks: feasibility, optimal solution, and optimal objective value. In feasibility prediction (Ta- bles [ and 4. MILPnet improves MSE by 1-3 orders of magnitude and reduces ErrorN by 90% or more in most cases. For optimal solution and value prediction (Tables 2 33d MIL.nett exceeds baselines while using substantially fewer parameters.
6336 uzz KG1
Figure& 4: Inference time comparisons
Efficiency Analysis and Sparse Variant. Figure 4 shows that MILPnet consistently achieves the fastest inference time across both small and large Foldable MILPs. GPU memory usage (Appendix Figure 12) remains moderate (< 4GB on FOLD300), making wide range of hardware. We also provide a sparse (stride) MILPnet variant that reduces the time complexity to I of the original, with FOLD200 faster inference results shown in Table and details in the AppendixT] indiicttssss its potential computational scalability.
it suitable for deployment
8

---

Published as a conference paper at ICLR 2026
Table Predict + Search performance of MILPnet on four benchmarks. "Rand" denotes randomly generated initial solutions. "gap" is computed as gap (ú y") / (lv*1 + 10-9), where y" is the exact optimal value by SCIP. "PT" means the predicting optimal candidate solution. "ST" total solving time including search. Full Comparisons with advanced methods are in Appendix C.1
Method
Type
IP
SC
CA
FC
gap(4)
PT(s) (D) ST(a) (3) sap([) PT(s) (.)
ST(s) (L)
gap(()
PT(s)
ST()) (
sap(1)
PT(s) (L)
ST())
Rand
Random
36,5397
0.1240
1.7166
2,0173
2.8057
6,7830
1.4939
L7776
GCN
Graph
0.0389
0.2443
0.3774
1.1461 0.2391
0.8386
0.9890
0.2278
3.2328
0.4257
0.2518
14875
MII.Paet
Seq(Ours)
0.0234
0.0625
0.1864
0.3483 0.0137
0.6915
0.7651
0.0139
3.2300
0.3503
0.0177
1,3773
Table 6: Performance impact of MILPnett's architectural components
(a) Ablation results on the impact of HYA and MSA
(b) Impact of Thnax ("/n²" means Tjnax 71)
Method
F0LD
FOLD150,20)
200
Method
FOLD(20
50LD050,30
FOLD(100,20)
MSF
ErreeN
MSE
EmorN
MSF:
ErroNN
MSF
EreoNN
MSF
FrroeN
MSF
EmorN
MiLPPne (ours)
0.0001
0.0001 0
0.0074
MILPnet
0.0005 0
0.0477 235
0.0558 318
woo HYA
0.2503
5036
0.2501
4999
0,2501
4987
MILPnet
0.0006 0
0.0481
259
0,0471
272
MSA
0,2500
5051
0,2500
5008
0.2500
4995
MILPPnct
0.0026 0
0.0841
0.0774 727
Overall Model Comparison. We summarize performance across key metrics, including generaliza- tion, parameter count, inference time, GPU memory, and ErrorN, using radar charts with performance ranks(Figures T for FOLD50 and Appendix Figure [3 for FOLD300). In both cases, MILPnet consistently forms the outermost polygon, indicating superior performance in accuracy, efficiency, and resource usage compared graph- based baselines.
6.3 PERFORMANCE ON REAL-WORLD MILP INSTANCES
While our focus is on sequence-based representation, we further demonstrate the utility of MILPnet in solving real-world MILP problems. We adopt a predict + search approach, where the model is trained predict an optimal solution, which is then refined via lightweight local heuristic. Table compares MILP with graph-based baselines and random search baseline. MILPnet consistently achieves the smallest optimality gap the exact solver, and the lowest overall solving time. These results demonstrate the model's strong representation quality and practical solving effectiveness.
6.4 IMPACT OF ARCHITECTURAL COMPONENTS
Ablation Study on Hybrid and Multi-scale Atten- tion. To evaluate the contribution of MILPnet's atten- tion design, compare the removal of the Hybrid Attention (HYA) and Multi-Scale Attention (MSA) components. Table [oal shows that removing either component significantly degrades feasibility predic- tion accuracy. In particular, models without MSA struggle to capture local structure, while disabling HYA weakens global context integration. These re- sults confirm that both components are essential to MILPnet's representation effectiveness.
Table 7: Permutation invariance Experiments "Or ** represents randomly permutated con- straint order in the MIL---equeecce "Original *"represents the original order.
Order
Method 1 Arch
FOLDI50.20)
FOLD350,0)
MSE
ErroNN
MSE
ErooN
GCN (Onginal) Original
Graph
0.1719
Mil. r Seq
D.000S 0
0.002 0
Or
MILPaet + Seg
0.0006 0
0.0021 0
Or
MIL Poe_
0.0001
0.0014
MILPaet Seg
0.0004
0.0021
Ablation study on multi-scale attention blocks We conduct an ablation study on the depth of the multi-scale attention module by varying the number of blocks from I to 3. The results, which visualize and summarize its representational and generalization capabilities, are provided in Figure ( in the Appendix B and Table 10 key finding is that deeper
ingnriicnnimppoeeemmen multi-scale attention modules offer significant improvement in convergence speed and representation at no cost to generalization ability.
Impact of the Maximum Window Size As shown in Tables 6b (pre-train 5mins) and Appendix Fig the maximum window size Mmax influences the trade-off between convergence speed and representa- tional capacity. Smaller windows accelerate convergence simpler instances like FOLD(20.), while large windows slow training and degrade performance on complex cases such as FOLD(100,20) These results indicate that mmax requires careful tuning and does not follow monotonic pattern.

---

Published as a conference paper at ICLR 2026
Permutation Sensitivity of MILP-sequence. Randomly shuffling the constraint and variable order in FOLD 20 and FOLD 50 keeps MILPnet's s MSE within the same magnitude (Table 7 in Appendix B} and well below that of other models (typical MSE z 0.x, error = 5000), for three seeds. These results demonstrate that ours MII Psequennce construction preserves invariance under both constraint and variable permutations in this scale for long time pre-training.
6.5 MILPNET SOLVING EFFICIENCY ANALYSIS
We analyze the solving efficiency of MILPnee across two dimensions: (i) Large-scale public MILP benchmarks (with 1000 variables, details in the Appendix) to evaluate each framework component; (ii) Heterogencous generalization and scalability on four very large real-world benchmarks (with 10000+ variables). To quantify solution quality in details, we integrate both MILPNet and graph- based baselines as pre-solving method (Nair et al. 2021)) like for predicting candidate solutions by learning from MSE, and use the refined solutions by fast heuristic as the warm- start solution for branch-and and-bound framework in (i).
Impact of Window Size Mmax and Attention Depth L We evaluate the impact of maximum window size Jamu and attention depth L on two large MILP benchmarks. Table show that performance is relatively insensitive to these hyperparameters, with consistent improvements (Nodes, Branching Time, Dual Gap) over graph-based methods and Strong Branching across all configurations.
Table 8: Branch and Bound performance of MILPnet (Neural pre-solving with fast heuristic: H.Seq.) on 1000+ variable benchmarks for 50 instances within 60s solving limit.
Method
Type
SC1000,,000
CA(1000,500)
Obj.(L)
Node(
Time
Gap(.)
(bj
Node(
Time (1)
Dual Gap(J)
SB
Exact
230.50 28.27
39.0
1 41.2
19.1 =
15.0 0.214 1.23(%)
146.0 i 26.6
3,4 1.0
0.00 R6
GIN
Gruph
230.56 28.35
28.35
38.8
+ 40.0
18.6
15,4 0.21 1.23(%)
146.0 + 26.6
1.8 1.8 1.0
0.00=00.0098
GraphGPS
H. Gruph
230.56 + 28.35
38.9
44.8
17.9
15,1
0.211 1.23(%)
146.0 1 26.6
1.8 1.0
0.00=0.00030 96)
MILPneu2
Seq. L=1
230.56 1. 28.35
37.7
39.8
16.5
15.5
0.21 1.23(%)
146.0 1 26.6
1.0.1.0.0
1.7 1.2.1.02
0.0010.0005)
MIL Pnet/2
Seq. 1==2
230.56
28.35
37.7
16.5
16.5+1156
15.6
0,21
1.23())
146.0 + 26.6
1.0+0.0
1.5+-02 0.00+000003)
MILPnet22
H. Seq.
L=3
230.56
+: 28.35
38.5
16.2
16,21154
15.4
0.174 1.21(%)
146.0 + 26.6
26.6
1.0
1.5
0.00=0.00X% %)
MILPneu22
Soq.
L=4
230.56
28.35
16,3
15.5
0.174 1.21(%)
146.0 26.6
1.0
1,5
0.00 10.00
MILPne
H. Seq.
L=]
230.56
+ 28.35
37.7
39.9
16.5
= 15,5
1.230%
146.0 26.6
1.0
1,7
0:00=00.008 S)
MILPneu/4
Soq. L=1
230.56 28.35
37.7
39.8
16.5
= 15.5
0,21 1.23(%)
146.0 + 26.6
1.0
1.2
0.00=00.0x$8
Very Large Heterogeneous solving generalization To assess heterogeneous generalization capability, we integrate simple Heterogeneous MILPnet Variant (Details in Appendix C.2) novel representation framework that replaces graph-based representations in advanced ML algorithms training, such as Con- PASSHuang et al. 2024). Trained on the SC dataset by ConPAS training pipeline, PlI. directly transferred to solve very large MILP instances from open-source benchmarks (Gleixner et al.) 2021) by predicting candidate solution, and refined the rounded solution by a quick heuristic for short time, then constructing the trust-region (Geng et al. 2025 for final solving. The results, including primal bound iterations (Fig Appendix) and solving efficiency (Table demonstrate that MILPnet generalizes stably and scalably across heterogeneous, large-scale benchmarks, achieving consistent performance improvements in all tested cases. Other Details are in Appendix
Table 9: Heterogeneous generalization on very-large benchmarks (Gleixner et al. 2021 within 1500s with ConPAS style training.
Method
30m200188
blpice8 13.6K
bip-ar98
18.4K
16.0K
Time
Obj
Time
Obj
Time
Obj
SCTP
163.96 302.00
1500.15
4620. 13
1500.10 10
6215335
CoMPASIGEN)
175.86
307.00 00
1500.00
4817.66 66
1500.01
6554
ComPAS(MIL
6220.57
CONCLUSION
We propose MILPnet, novel multi-scale hybrid framework for representing MILP problems through sequence modeling, rather than conventional graph methods. We prove that this architecture can approximate essential MILP mappings for arbitrary instances. Empirical evaluation confirms that MILPner outperforms graph-basee based methods in terms of efficiency and performance, while addressing the Foldable MILPPs where graph-based approaches fail.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
The authors would like to thank the anonymous reviewers for their valuable feedback. This work was partially supported by the National Natural Science Foundation of China (Grant No. 62276024), the Beijing Natural Science Foundation (Grant No. 4262066), the Fundamental Research Funds for the Central Universities, Jilin University(Grant No. 93K172025K01), and the Fundamental Research Funds for the Central Universities (Grant No. 2025CX01010).
REPRODUCIRILLITY STATEMENT
All datasets, experiments, and architecture hyperparameters used in our experiments are documented the Appendix K and Appendix B
ETHICS STATEMENT
This work is fundamental research in optimization and machine learning. All experiments were conducted without any potential risks. We do not foresee any direct negative societal impact or ethical concerns arising from this work.
REFERENCES
Charalambos D Aliprantis and Kim C Border. Infinite Dimensional Analysis: A Hitchhiker's Guide, Springer, Berlin, 3rd edition, 2006.
Yoshua Bengio, Andrea Lodi, and Antoine Prouvost. Machine learning for combinatorial optimization: methodological tour d' d'horizon, 2020. URL [tttps::///a org. htth:/bs//8881 htabs///11... 06128
Gustavo Alfredo Bula, Fabio Augusto Gonzalez, Caroline Prodhon, H. Murat Afsar, and Nubia Milena Velasco. Mixed integer linear programming model for vehicle routing problem for hazardous materials transportation" * *universidad nacional de colombia. universite de technologie de troyes. IFAC- PanerOnnne,,, 49(12):538 543,, 2016. ISSN 2405-8963. doi: https::/doi.orr//000
2016.oo.2016.07.99 URL https: //www. sci enced i rect com/ / wwwee/eeearcccco p11/524058 96311186611110 9631613311 6309673 8th IFAC Conference on Manufacturing Modelling, Management
and Control MIM 2016.
Quentin Cappart, Didier Chételat, Elias Khalil, Andrea Lodi, Christopher Morris, and Petar Veličković. Combinatorial optimization and reasoning with graph neural networks, 2022. URL https
//axxiv.orr/abs//222..000
Moses Charikar, Sudipto Guha, Êva Tardos, and David B. Shmoys. A constant- factor approximation algorithm for the k-median problem (extended abstract). In Proceedings of the Thirty-Firy Annual ACM Symposium on Theory of Computing, STOC '99, pp. 1-10, New York. NY, USA, 1999. Association for Computing Machinery. ISBN 1581 130678. doi: 10. 10.1145/301250.30125 URL hhttps://di..oo org/10 111.1...11111111 3012220100000015.7.
Hao Chen, Kai-Chieh Hsu, Walker Turner, Po-Hsuan Wei, Keren Zhu, David Pan, and Haoxing
Ren. Reinforcement learning guided detailed routing for custom circuits. In Proceedings of the 2023 International Symposium on Physical Design, ISPD '23, pp. 26-34, New York, NY, USA, 2023a. Association for Computing Machinery. ISBN 9781 450399784. doi: 10.1 10.1145/266005235577777.
Ziang Chen, Jialin Liu, Xinshang Wang. Jianfeng Lu, and Wotao Yin. On representing mixed-inteeseeeeeeee integer linear programs by graph neural networks, 2023b.
http://aaaavvvvoaaaa///a/aaa//// 2210
10759
Antonia Chmiela, Elias B. Khalil, Ambros Gleixner, Andrea Lodi, and Sebastian Pokutta. Learning to schedule heuristics in branch-and-hound, 2021. URL httpt/ttt///airiiii http://awww..or//////////0//
T0294
V. Chvátal. A greedy heuristic for the set-covering problem. Mathematics of Operations Research, (3):233-235, 1979. doi: 10.2307/3689577.
11

---

Published as a conference paper at ICLR 2026
Michele Conforti, Gérard Coruéjols, and Giacomo Zambelli. Potyhedral Approaches to Mixed Integer Linear Programming. pp. 343-385. Springer Berlin Heidelberg, Berlin, Heidelberg, 2010.
ISBN 978 778-3-540-68279-0, doi: 10. 10.1007/978.3-540-6877777 11. URL /Rltttps://ddi.orgg
10. 1 007/ 978 540 6827
Uriel Feige. A threshold of In for
set cover. http://ag..........aa/ccaaaaaaaa 1998. ISSN
0004-5411. doi: 10. 10.1145/285555.285053 URL 1ttp 285059
Shengyu Feng and Yiming Yang. Sorrel: Subbppiiialdemmmmssttrrooorueede reinoorceeen learning for learning to branch, 2025, URL httpppp://ax....o///////////////
Maxime Gasse, Didier Chetelat, Nicola Ferroni, Laurent Charlin, and Andrea Lodi. Ex- act combinatorial optimization with graph convolutional neural networks. In H. Wal- lach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (eds.), Ad- vances in Neural Information Processing Systems, volume 32. Curran Associates, Inc.. 2019, URL
cc/paper. http:/aaar__ffeeeseeeeeepp fi.es//ppeer///19// www.boddddapp.........
fi
Maxime Gasse, Quentin Cappart, Jonas Charfreitag, Laurent Charlin, Didier Chételat, Antonia Chmiela, Justin Dumouchelle, Ambros Gleixner, Aleksandr M. Kazachkov, Elias Khalil, Pawel Lichocki, Andrea Lodi, Miles Lubin, Chris J. Maddison, Christopher Morris, Dimitri J. Papageor- giou, Augustin Parjadis, Sebastian Pokutta, Antoine Prouvost, Lara Scavuzzo, Giulia Zarpellon, Linxin Yang, Sha Lai, Akang Wang, Xiaodong Luo, Xiang Zhou, Haohan Huang, Shengcheng Shao, Yuanming Zhu, Dong Zhang, Tao Quan, Zixuan Cao, Yang Xu, Zhewei Huang, Shuchang Zhou, Chen Binbin, He Minggui, Hao Hao, Zhang Zhiyu, An Zhiwu, and Mao Kun. The machine learning for combinatorial optimization competition (ml4co): Results and insights, 2022. URL
owg/ abs 2203. 02433
Zijie Geng, Jie Wang, Xijun Li, Fangzhou Zhu, Jianye HAO, Bin Li, and Feng Wu. Differentiable inte- ger linear programming. In The Thirtentth International Conference on Learning Representations, 2025. URL /
Ambros Gleixner, Gregor Hendel, Gerald Gamrath, Tobias Achterberg, Michael Bastubbe, Timo Berthold, Philipp M. Christophel, Kati Jarck, Thorsten Koch, Jeff Linderoth, Marco Lübbecke, Hans Mittelmann, Derya Ozyurt, Ted K. Ralphs, Domenico Salvagnin, and Yuji Shi- Compilation of the 6th Mixed Programming Library. Mathematical Programming Computation, 2021. doi: 10.10077:11232--00-0000999 hittps://do/ org 1007 02011222222200000993000000
nano. MIPLIB 2017:
URL
Michael Goldman and Dario Trevisan. Optimal transport methods for combinatorial optimization random point sets, 2023.
Ralph F. Gomory. Outline of an Algorithm for Integer Solutions to Linear Programs and An Algorithm for the Mixed Integer Problem, pp. 77-103. Springer Berlin Heidelberg, Berlin, Heidelberg, 2010, 4. URL /RL org///
ISBN 978-3-540-68229-00 doi: 10.
1007/978 3- 682
0_
Prateek Gupta, Maxime Gasse, Elias B. Khalil, M. Pawan Kumar, Andrea Lodi, and Yoshua Bengio. Hybrid models for learning branch. CoRR, 0tth/2006.152122 2020. URL htttps///aaxxi
org/ bogg/abs/a 2006. 15212
Qingyu Han, Linxin Yang, Qian Chen, Xiang Zhou, Dong Zhang, Akang Wang, Ruoyu Sun, and Xiaodong Luo. A gnn-guided predict-and-searchh framework for mixed-integer linear programming. 2023.
He He, Hal Daumé, and Jason Eisner. Learning to search in branch and bound algorithms. In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger (eds.), Ad- vances in Neural Information Processing Systems, volume 27. Curran Associates, Inc., 2014. URL attpp:://ppoocecinggg..uuuususg ccppaper files/ files//pperr20147 ofFe
12

---

Published as a conference paper at ICLR 2026
Kurt Hornik, Maxwell Stinchcombe, and Halbert White. Multilayer feedforward networks are uni-
versal approximators. Neural Networks, 2(5):359 -669 1989. ISSN 0893-6080. doi: https:
URL https::/www/ /tw..//ceeeeeieeettttt.t.tceccee com/
sci bssieeecs/rrtrcceeee/11883333383
Taoan Huang, Aaron Ferber, Yuandong Tian, Bistra Dilkina, and Benoit Steiner. Searching large
neighborhoods for integer linear programs with contrastive learning, 2023. URL https://
arxiv.org/abs 015778
Taoan Huang, Aaron M Ferber, Arman Zharmagambetov, Yuandong Tian, and Bistra Dilkina. Contrastive predict-and-search for mixed integer linear programs. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.). Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 19757-19771, PMLR, 21-27 Jul 2024. URL https://proceeccing
mlr www.sress/seeeee wttppse/see///nieeee https//v222hhhaaa22...a3 html
Joey Huchette and Juan Pablo Vielma. A geometric way build strong mixed- integer programming formulations, 2019. URL/attps://axxi///t
10409 10409
Richard M. Karp. Reducibility among Combinatorial Problems, pp. 85-103. Springer US, Boston,
MA. 1972. ISBN 978 1-468--000-22 doi: (0.110077978-144444200000
URL https:
7 7/doi. org. 1TT.111..100001111 -08-
A. H. Land and A. G. Doig. An automatic method of solving discrete programming problems. Econometrica, 28(3):497-520, 1960.
Kevin Leyton-Brown. Mark Pearson, and Yoav Shoham. Towards a universal test suite for combina- torial auction algorithms. In Proceedings of the 2nd ACM Conference on Electronic Commerce, New York, NY, USA, 2000. Association for Computing Machinery. ISBN doi: 10. 11444455871.35258885 URL /ttp::::://o...og/0000001111
EC '00, pp. 66 66-76,
1581 132727.
352879
Haoyang Liu, Jie Wang, Zijie Geng, Xijun Li, Yuxuan Zong, Fangzhou Zhu, Jianye Hao, and Feng Wu. Apollo-milp: An alternating prediction-correction neural solving framework for mixed-integer linear programming. 2025. URL http::////aawvwvo.///.aa//// 220.:/11339333000 01129
Hengyuan Ma. Wenlian Lu, and Jianfeng Feng. Efficient combinattraa coa.binnatrillooiimiittiioo via heat diffusion, 2024.
wtt.oo/////ba223..0003300 08757
Zoubir Mammeri. Reinforeement learning based routing in networks: Review and classification of approaches. IEEE Access, 1155916-55500 2019. doi: 10. (100///CCESS.2219.9999333
Vinod Nair, Sergey Bartunov, Felix Gimeno, Ingrid von Glehn, Pawel Lichocki, Ivan Lobov, Brendan O'Donoghue, Nicolas Sonnerat, Christian Tjandraatmadja, Pengming Wang, Ravichandra Addanki, Tharindi Hapuarachchi, Thomas Keck, James Keeling, Pushmeet Kohli, Ira Ktena, Yujia Li, Oriol Vinyals, and Yori Zwols. Solving mixed integer programs using neural networks, 2021. URL
https://arxiv pttg:b//////2020/ 13349
Max B. Paulus and Andreas Krause. Learning to dive in branch and bound, 2023. URL https
09943
Sebastian Sanokowski, Sepp Hochreiter, and Sebastian Lehner. A diffusion model framework for unsupervised neural combinatorial optimization, 2024. URL
2.406.0665 661
Zhiqing Sun and Yiming Yang. Difusco: Graph-based diffusion solvers for combinatorial optimiza- tion, 2023, URL
08224
Petar Veličković, Lars Buesing, Matthew C. Overlan, Razvan Pascanu, Oriol Vinyals, and Charles Blundell. Pointer graph networks, 2020. URL /ttp:/ttpae//arai...o/////aa//aa
http://wwww.rg/a///////////// 06380
John von Neumann. On rings of operators. reduction theory. Annals of Mathematics, 50:401, 1949. URL httpt:t::/// //attss::/apiii semant i ar htt.://hhoo..oooooooooppsaaaaaaa
13

---

Published as a conference paper at ICLR 2026
Jie Wang, Zhihai Wang, Xijun Li, Yufei Kuang, Zhihao Shi, Fangzhou Zhu, Mingxuan Yuan, Jia Zeng. Yongdong Zhang, and Feng Wu. Learning to cut via hierarchical sequence/set model for efficient mixed-integer programming, 2024.
Shaohua Wang, Junyuan Zhou, Haojian Liang, Zhenbo Wang, Cheng Su, and Xiao Li. A new approach for solving location routing problems with deep reinforcement learning of emergency medical facility. Proceedings of the 8th ACM SIGSPATIAL International Workshop on Security Response Using GIS, EM-GIS '23, pp. 50-53, New York, NY, USA, 2023a. Association for Computing Machinery. ISBN 9798400703461. doi: 10.1145/361 3622449.,
URL, https
7/doi. oag..org/1n 11451155333 01155/665555 1884 3624443662990 9429
Tianze Wang, Amir H. Payberah, and Vladimir Vlassov. Graph representation learning with graph transformers in neural combinatorial optimization. In 2023 International Conference on Machine
Learning and Applications (ICMLA), pp. 488-495, 2023b. doi: 10.1 109//CML 2023. 00074.
Weitian Wu and Xinmin Yang. The hybridization of branch and bound with metaheuristics for nonconvex multiobjective optimization. CoRR, abs/2212.04624, 2022. doi: 10.48550/ARXIV. 2212.04624. URL. htttps::/dattt wttp 48550 2212. 04624.
Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long. Chengqi Zhang, and Philip S. Yu. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 32(1):4 2- 24, January 2021. ISSN 2162-2388. doi: 10.1 00.1009/m/s.22202297755.
URL httth:tt:/// doi org. 11tpg//1..1111/N....... 2020 2978386
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks?, 2019. URL BRLLittps://axxi..o
abs http:aaaas///... 1810.
Huigen Ye, Hua Xu, Hongyan Wang, Chengming Wang, and Yu Jiang. Gnn gbdt-guides fast optimizing framework for large-scale integer programming. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardtt Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning. volume 202 of Proceedings of Machine Learning Research, pp. 39864-39878 PMLR, 23-29 Jul 2023. URL https: http::::s//pooceednng..o mlr w.l.ppress/ee v202 ye23e....... html
Gilad Yehudai, Ethan Fetaya, Eli Meirom, Gal Chechik, and Haggai Maron. From local structures size generalization in graph neural networks, 2021. URL httts:://artttt //tp:///axxvv.../ http//aaxxv...r/.bb..aaa/e 2010
08853
Andrea Zelaschi, Lorenzo Pliotti, Giulio Betti, Giovanni Tonno, Daniele Sgrò, and Emanuele Martelli. An effective milp model for the optimal design of microgrids with high-reliabilltyy requirements. Applied Energy, 125359,,,,, 2025. ISSN 0306-2619. doi: hutps//doi..rr///111666...eppee
1253599 URL https:/ www. wtt.://eeeecceeee...ooooeeeeeeee
$0306261925000892
Zidong Zhang, Dongxia Zhang, and Robert C. Qiu. Deep reinforcement learning for power system applications: An overview. CSEE Journal of Power and Energy Systems, 6(1):213-225, 2020. doi:
10.17755CSEEIPESS2209.099
Hang Zhao, Kexiong Yu, Yuhang Huang, Renjiao Yi, Chenyang Zhu, and Kai Xu. Disco: Efficient diffusion solver for large-scale combinatorial optimization problems, 2025. URL https://
arxiv. arxiv.org/abs /bppr///b///24...... 2406. 19705
14

---

Published as a conference paper at ICLR 2026
A DETAILED ARCHITECTURE FOR MILPNET
Figure 5 illustrates the detailed architecture of MILPnet, which connects a feed-forward network with the core Multi-scale based Hybrid Attention.
MILP
Sequence
Embeddings
Nx
A
Encoding
Linear Projection
Scuti-Scale
Attention
N hf"
Norm
Norm
Output
Global attention
MILPnet block
Figure 5: Detailed Architecture of MILPnet.
B FULL RESULTS ON MAXIMUM WINDOW SIZE, ATTENTION BLOCKS, REPRESENTATION, AND GENERALIZATION
B. ABLATION STUDIES THE 7maz
We conducted study on the maximum sliding-window window size and visualized the resulting representation curves in Figure 6
120
201
4000
100
200
Truch
TONCS
POLLDVEB0.0 E800@rankur
Figure 6: Performance of varied max window size of MILPnet on FOLD(20,16) and FOLD(100,20) for feasibility training. We can observe that the maximum sliding window size affects the convergence speed of the feasibility mapping approximation. The window size does not follow simple pattern, but rather requires balancing.
15

---

Published as a conference paper at ICLR 2026
B.2 PERFORMANCE ON THE NUMBER OF ATTENTION BLOCKS L
We conduct the ablation study on the number of attention blocks on FOLD(20) and FOLD(50).
Errer Rate Comparisen (20)
Error Rate Comparioon
Figure 7: Performance of varied numbers of multi-scale attention blocks in MILP on FOLD(20,16) 16) and FOLD(50,20) for feasibility with the same embedding size and same window size for blocks =1,2,3,
Table 10: Ablation study on the number of multi-scale attention blocks (L).
Number of blocks Method Arch
FOLD(20,16)
FOLD(50,20)
MSE
ErrorN
MSE
ErrorN
GCN (Original)
GNN + Graph
0.3070
5000
0.4719
5000
T=1
MILPhett + Seq
0.0006 0
0.0003 0
L=2
MILPnet Seq
0.0001
0
0.0001
L=3
MILPnet Seq
0.0004 0
0,0001
B.3 PERMUTATION SENSITIVITY ON VARIABLE ORDERS
Table 11:
Experiments on variable permutation invariance. "V-Or represents randomly permutated
variable order in the Mll.--sequenee."OO "Origgina "rrepresents the original order.
Order
Method + Arch F0L)(20,16)
FOLD(500,00)
MSE
ErrorN
MSE
ErrorN
GCN (Original)
GNN Graph 0.3070
5000
0.4719 5000
Original
MILPnet Seq
0.0003
0.0003
V-Or
MILPnet Seq
0.0006
0.0001
V-Or 2
MILPnet Seq
0.0003
0.0003 0
V-Or 3
MILPnet Seq
0.0004 0
0.0005 3
B.4 REPRESENTATION EXPERIMENTS
We conducted representation experiments on the representation of feasible mapping, the optimal solution mapping, and the objective optimal value, for MILP instances. The embedding sizes used in our experiments were chosen from {32, 128, 216, 256, 512]. Figure 8 shows the representation results of feasible mapping, Figure 9 shows the results of optimal solution mapping, and Figure [4] shows the results of the objective optimal value mapping. The dropout is used in this part.
B.5 GENERALIZATION EXPERIMENTS RESULTS
We provide the full results on FOLD(200) to FOLD(50.) in Table 8 following.
16

---

Published as a conference paper at ICLR 2026
Fensible MSE Fold(20,6)
Peasible FRROORuumber Fold(20,6)
Feasible MSE Fold(20,16)
Feasible ERROOnumber Fold(20.16)
4000
0.6
4000
MX 1oe.
Enox staieer
MB
2000
2000
0.0
0.0
1poc
Rpcb
Ipoch
Feasible MSE 200
Feasible ERRORnumber FOLD(50,20)
Feasible MSE 30)
Feasible ERRORaumber FOLD(50,30)
0.8
0.8
MSN low
Eite mandes
0.4
Eme
2000
0.4
2000
0.0
0.0
2000
4000
(c)
2000
4000
2000
4000
(d)
2000
4000
Rpoc
Epadi
Epuch
Spuch
Graph-based
Graph based
Figure 8: Representation experiments on feasibility for FOLD20 ((a) for FOLD(20,6), (b) for FOLD(20,16) and FOLD50 ((c) for FOLD(50,20),c)) (d) for FOLD(50,30). MILPPett approximates the feasible mapping of Foldable MILP instances more efficiently.
Table 12: Generalization experiments for feasibility mapping on FOLD(20,*) to FOLD(50, *), with 10,000 foldable MILP instances from each.3M(20)/ ((203330220)0(555) means pre-train time for FOLD20 is 3mins, FOLD50 is 10mins,5M(20//30M550) means pre-train time for FOLD20 is 5mins, FOLD50 is 30mins.
Method
Type
FOLD120,6)
FOLD120,16)
F0LD250,20
FOLD(50,
MSE
ErroN
Params
MSE
ErrocN
Params
MSE
ErooN
Params
MSE
EmorN
Params
SCIP
Fxact
0
GCN
Graph
0.3073
5000
1.21M
0.3073
5000
1.21M
0.3073
5000
1.21M
0.3073
5000
1.2MM
GCNSM2220MM
Graph
0,3073
5000
1.21M
0.3070
5000
1.21M
0.4719
5000
1.21M
0.3242
5000
1.21M
Graph
0.4200
5000
1.63M
0.4209
5000
1.63M
0.4200
5000
1.63M
0.4204
5000
1.3M
GIN
GIN
Graph
0.4199
5000
1.63M
0.2999
5000
1.63M
0.2939
5000
1.6M
0.4204
5000
1.63M
Graph
0.4642
5000
0.66M
0.4999
5000
0.66M
0.4714
5000
0.66M
0.4586
5000
0.66M
SAG SMG
Graph
0.4642
5000
0.66M
0.4999
5000
0.66M
0.4714
5000
0.66M
0.4856
5000
0.66M
Graph
0.2508
5000
1.64M
0.2523
5000
1.64M
0.2511
5000
1.64M
0,2511
5000
1.64M
PUNmMaa
Graph
0,2511
5000
1.64M
0.2511
5000
1.64M
0.2512
5000
1.64M
0.2512
5000
1.64M
Griph
Graph
0.2500
5000
0.66M
0.2500
5000
0.66M
0.2500
5000
0.66M
0,2500
5000
0.66M
Graph
0,2500
5000
0.66M
0.2500
5000
0.66M
0.2500
5000
0.66M
0.2500
5000
0.66M
GCN" SM000005)))
Graph
1.21M
0.2500
1.21M
5000
GCN" 5M02630055)
Griph
0.2126
2853
1.21M
0.2499
4921
1.21M
0.71
0
1.21M
0,2402
1.21M
GIN" 3M1201100:00
Rf Graph
0.2500
5000
1.63M
0.2501
5000
1.63M
0.2500
5000
1.63M
0.4204
5000
1.63M
GIN" 5M0002063311
Rf Graph
0,2499
4757
1.63M
0.2500
5000
1.63M
0.2500
5000
1.63M
0.2458
2603
1.63M
SAGE" (M13111111)))
Graph
0,2499
5009
0.66M
0.2499
4995
0.66M
0.2500
4997
0.66M
0.2500
5002
0.66M
SAGE" 55629901100
Graph
0.66M
0.2499
0.2500
0.66M
4999
PGN" 20120/106:30
Rí Gruph
0.2582
5000
1.64M
0.2560
5000
1.64M
0.2502
5000
1.64M
0,2502
5000
1.64M
PGN" (010800n61))
Rí Graph
0,2507
5000
1.64M
0.2514
5000
1.64M
0.2512
5000
1.64M
0,2502
5000
L64M
GraphGPS (0:20910155))
Rí Gruph
0,2510
5000
0.66M
0.2510
5000
0.66M
0.2500
5000
0.66M
0.2502
5000
0.66M
GraphGPS" 100000es:9
Graph
0.2510
5000
0,2500
0.66M
5000
MI Pnetayi 20110100
Ours (Seg)
0.00015
0.56V
0.0004
0.56MM
0.7000
0
0.6N
0.0023
12
0.6M
MIL Pn//////////
Ours (Soq)
0.0003
0.56M
8.53e-5
0.56M
0.0005
0.60M
0.0082
0.60M
B.6 CROSS-SIZE ADAPTION
Our approach also enables End to End Cross-Size generalization from FOLD(n,m.) to FOL.(n. which is quite difficult for Bipartite Graph-based MILP End-to-End todEnd representation methods Ychudai al. 202 We directly transfer the network pre-trained FOLD (50,20) on the feasibility mapping to FOLD (50,30). As shown by the heatmap, the pre- pre-trained network also easily achieves good performance FOLD (50,30), which demonstrates that model can still perform inference directly when faced with changes in length, effectively transferring its prior knowledge. This verifies that network effectively extract features; however, we also found that this adaptive capability is
17

---

Published as a conference paper at ICLR 2026
305
1200
Fpach
Ppeck
(4) FOLD(2LL DI MSE
(b) FOLDD22010 D2 MSE
0.4
(C) FOLD(50200) DI MSE
(4) FOLD150,20) D2 MSE
buse
Pact Sequince
Figure 9; Representation experiments optimal solution for FOLD20 and FOLD50 D1 and D2 (details is shown in Appendix F). MILPnet approximates the solution mapping of Foldable MILP instances with smaller errors than graph-based based method.
sensitive factors such model capacity and problem scale, which we plan to investigate further in future work.
ERRORN Heatmap
MSE Heatmap
5000
0.20
ERRORN
71
MSE
0.0023
0.0417
0.3242
0.15
2000
1000
Ourst50,301
Transfen
GCN(50,30)
Curs/50.30)
Transterts,,
GCNI50.301
(o150,30)
ta/50, 30)
Figure 10: Cross-size adaptation for MIL.
18

---

Published as a conference paper at ICLR 2026
C MILPNET SOLVING EFFICIENCY RESULTS
C.I ADDITIONAL EXPERIMENTS ON REAL-WORLD BENCHMARKS
This part, we use MILPnet and advanced graph-based network for predicting the near-optimal solution and refine by a local heuristic algorithm, the results are summarized following Table [3 This result
Table 13: Additional evaluations with more powerful GNN baselines, including GIN, PGN, and GraphGPS, on the Real world practical problems
IP
SC
CA
FA
Time
Gap
Time
Gap
Time
Gap
Time
Gap
0,6254
0.0320
0.8223
0.6616
3,3065
0.7655
8,8688
0.7504
PGN
0,6023
0.1020
1.0492
0.4207
4,8369
1.0759
12,4009
0.7438
GraphGps
0.7324
0.0595
1.9202
0.4292
4,5980
1.0472
8.1973
0.8328
M
0.1864
0.0234
0.6915
0.3483
3,2300
0.7651
1.3773
0,3503
demonstrate that MILPnet consistently outperforms these advanced graph-based models in terms of solving efficiency and inference speed. This further validates the effectiveness of the sequence- based architecture and the representation advantage MeLueseqeenc. For SC and CA with 1000 variables and 500 constraints in ConPAS style training, the number of the dataset are: training 10000 instances, solving on 50 instances.
C.2 CONPAS STYLE HETEROGENFOUS VARIANT ON REAL- WORLD INSTANCES SOLVING
To address complex heterogeneous optimization problems in the real world, we train the hetero- geneous MILPnet as the solution representation predictors on SC problems and construct the trust region solving on three very large benchmarks from (Gleixner et al. 202T). The visualization of the solving results in terms of primal bound trajectories is shown in Figure 11 Notably, because of the very-large scale the instances, we use 5s pre-heuristic refinement based on GCN/MIL rounded prediction. The solving quality depends on the pretrained MILPnet predictions, which provide the trust-region. Since ConPAS's training code has not been released, we reproduce it ourselves. For ComPAS(MILPPn)
the MSE loss based Contrastive loss for our training, which is related to our solution mappings approximation.
Heterogeneous MILPnet Variant. The heterogeneous MILPnet variant focuses transfer capability. We observe that the dataset used for pre-training can limit this. Therefore, we relaxed the direction in the n + 2 dimension, so the MILP-sequence becomes (m + x (n + 1) dimensional, enhance its transfer capability. We use the same-scale Setcover to train this variant and use it as the solution predictor.
30u2165
ce s00.0
016-000:0
6/
62
T5km (mnoen))))
Tuxn (mooak))
Figure 11: Primal bound iterations very-large benchmarks solving with ConPAS(MILPnet)+ Trust Region +SCIP and ComPAS(GCNN+Trus Region SCIP
19

---

Published as a conference paper at ICLR 2026
D GPU ANALYSIS AND OVERALL PERFORMANCE COMPARISON
D.1 GPU MEMORY ANALYSIS
Figure visualizes the GPU usage during the training process.
SAGE
MiLrettOuss
1540
FOLDs F1stan0s
Figure 12: GPU memory usage on FOLD(20 *) to FOLD(300 *). (The lower the better)
D.2 OVERALL PERFORMANCE COMPARISON
We conducted a comprehensive evaluation of various performance dimensions on the instances of FOLD300, and the results are shown in Figure B
Overall Performance Comparison
GDV
Param
Time
Can alaation
LS0OR N
Figure 13: Overall performance comparisons FOLD(300, Broader the better)
20

---

Published as a conference paper at ICLR 2026
E PROOF OF THE SECTION 4.2 AND SECTION 4.3
We establish the results stated in Remark 4.2 and Section 4.3 where the former follows from Theorem
5 and the latter from Theorem as follows.
Theorem 5 (Padding Equivalence). Define the topological space before padding as: HMILPO
(m H Wur) I Hah and the padded topological space H 11 and M are homeomorphic (topologically equivalent):
H1 MILP MILPO
(13)
Proof. Define f as follows:
f h Rws ho))
her
hus O..0..... times
i. 1.O.....0
ho. 0.:يي
times
times
where:
be (Hoons)m
hy E Va
E I,
ho Hobj
O,...,, D {0)* are the padded zero dimensions.
Define l-1 as:
((
1)
hes hy 0.....))
i )
Nor Qu.....) = (hee bus i, ho)-
times
A times
times
Subsequently, we introduce (i) and (ii), which are instrumental in establishing the main theorem.
(i) f is Bijective
Proof.f is Injectivity: Suppose (hi f12) for h1, h2 E MILPO Then, [(h)) = f(h2) - (her, (hor, O, .,..., , 0), (i1, O, ...., 0), (hoI, 0, .,...,,)))) (hoz, (ho2, .,..., 0), (i2, ..... 0), (ho2 ......)). 0))
(14)
(15)
This equality implies:
ha hez, hel hvz: i1 iz.
hol hoz-
Therefore, h1 = h2, establishing injectivity.
is Surjectivity: For any h E MIPP, suppose h' (he, (h,, Po), (1', (h's Po E {0) Then, there exists h E AiMILKO such that:
where
= (he, Ki, (1)))
Applying f to h, obtain:
f(h) (bes (h/, 0, 0), 0), (f, OK..,...., Rt
Thus, f is surjective.
(ii) f and f-1 are both continuous Proof. Both MILPO and HMILP are equipped with the product topology. the product topology, a function is continuous and only if each its component functions is continuous.
Continuity of f: The mapping f involves embedding each component of MILPO into a higher- dimensional space by appending zero vectors. Each such embedding is continuous because it is defined by coordinate- wise inclusion and fixed assignments (adding zeros). Specifically:
f H, H, k0)*
21

---

Published as a conference paper at ICLR 2026
continuous for each component H, {Homss H H.
Continuity of f The inverse mapping f involves projecting each padded component back to its original space by removing the appended zero vectors. Projection maps in the product topology are continuous. Specifically:
IT H. (0)* + H;
continuous for each component H,.
Drawing upon the results presented (i) and (ii), the f bijective and both f and f are continuous, is homeomorphism. Therefore, the topological spaces MIMP and // MILPP are homeomorphic:
HMILP MIPPO
Theorem 6 (Constraint Permutation Invariance). Let (Hioms HH "ism Henus I, H Hob) be topological spaces corresponding respectively to the m constraint feature spaces Hi the integer- index space I' the seriable-boundee space H and the thjective-coeffcien space Hobi' For any permutation a on the index set (1,2.... m, 'T, 'Var', "obj',, J. their product spaces are homeomor- phic:
H
Hroms
I'
Har
Hobj
122
II
Hifactor)s
kEo(().......)) Nar', ooh'))
In other words, the topological structure of the full MILP sequence feature space is invariant under permutation of its component spaces.
Proof The ordered product is
X
H
H
I
H Var Hobj
Let
Xo
II
H (factor),
(fo(()....,, Var (obj'))
be an arbitrary permutation of the factors, where a permutation of the index set. We explicitly construct coordinate-exchunge (or coordinate
mup
+ X
Xa.
Given point
T =
(
cons
conns i', Var 1obb)) E X.
where
hoooss
Ho i' E,,, A E Var, obb e Hobj
define (xx) by rearranging these coordinate components according a. Concretely, if sends the index to position (1), the index 2 to position (2), etc., then
h
(lactor)
=
the factor with index a 1(1)....)
ordered according to
In simpler terms, & reorders the factors (hquus ...., heums i, h Var habb)) into the sequence (...) deter- mined by a.
Bijection. 4 is bijection because:
Injective: If (xx1) = ((x2), then their coordinates in each position of the reordered product are identical. Since a product space comparison equates each factor, it follows that xi = W22-
Surjective: Given an arbitrary point E Xx,, we can "reverse reorder" its factors to form E X. By construction, {(x)
22

---

Published as a conference paper at ICLR 2026
Because & is bijective and both d and & are continuous, & is homeomorphiism. Hence
H X x H x I'
H Var
Hubj"
II
H(lacion),
kEa( Var',obj" obj
Theorem (Variable Permutation Invariance). Let HMIPP be the topological space of MILP- sequences defined in Section 4.3 For any permutation π on the variable indices n the MILP -squeecce space is homeomorphic under variable permutations:
MIPP H
where H denotes the space after permuting variable coordinates according to æ. Consequently, the feasibility mapping 4 jear optimal objective value mapping Dobj. and optimal solution mapping shn are all invariant under variable permutation.
Proof. We explicitly construct
reordering map
V: MMPP
MIPPP
Given an MILP sequence
2 he he hh he1 H
cons
where each component is vector in Ru+2 define (x) by applying the coordinate permutation to each token:
For each token h (hi,.... ... Ans 1 h242) R in the MILP-sequence, define the coordi- nate permutation operator Pr: Rn+2 + I R2+2 by
Pa = (1)(1)).... h(()).)).......))..))...)))))"") hp41, (2+2),,
where permutes only the first variable-related coordinates, leaving the last two padding coordinates unchanged. This operator applies uniformly all tokens: constraint tokens heass integer set token h"', variable bounds tokens h² and h." and objective token h='.
Then
(x) [Pa (...) Pa (h m. (h P,(
), E H.MP.
Then prove that V is bijective and continuous:
1. Bijection. a bijection because:
Injective: (xx) = (x2), then for each token, the permuted coordinates are identical. Since is bijection, the original coordinates must also be identical, hence X2.
X1
Surjective: Given an arbitrary point E MIIP we can apply the inverse permutation to each coordinate of each token obtain MILP such that / (x) = y.
2. Continuity. Both V and - are continuous because coordinate permutation a continuous operation in the product topology of R=+2 Specifically, for each token space IR the permutation map a linear isomorphism, and the composition over all tokens preserves continuity in the product space MMILP I (m=++)((++))
Therefore, V is homeomorphism, and
MILP
MILP
Remark E.1 (Permutation Invariance). Theorems 6 and and and establish that MIPP is invariant under both constraint permutation (reordering tokens) and variable permutation (reordering coordinates within tokens). These homeomorphisms ensure that different representations of the same MILP instance topologically equivalent and define identical optimization problems. Consequently, the mappings Prea. Danj. and Pantu are invariant under both types of permutations, which is crucial for designing permutation-equivarii neural architectures.
23

---

Published as a conference paper at ICLR 2026
F PROOF OF THE MAPPINGS.
We introduced the details of the MeLuesequence mapping definitions: with the defined topological space HMILP, which encapsulates the MILP-sequence, analyzing MILP instances. These mappings assess feasibility, compute optimal objective values, and identify optimal solutions, providing a comprehensive toolkit for MILP analysis.
define key mappings essential for
Definition 8. (Feasibility mapping by sequence) The feasibility mapping is classification function that determines whether a sequence within MiPP represents a feasible solution:
iras
HMILP {0, 1}
(16)
where Picas indicates that the MILP instance is feasible.
Definition 9. (Optimal objective value mapping by sequence) The optimal objective value mapping for each MILP instance defined as:
Pobj H MILP + RU [00, -o0
(17)
which projects feasible sequences to their respective optimal values.
Definition 10. (Optimal solution mapping by sequence) To simplify the discussion, focus settings where all components of the vectors l and u are finite. This assumption ensures the existence of an optimal solution when the MILP problem is feasible. Consequently, we define restricted subset of the MILP topologiaal space, HMP.P H which only has finite variable bounds. The optimal solution mapping for the MIL.P-sequene is defined as: For any F E $ ( (TR), the MILP problem has unique optimal solution with the smallest normm. Let
H MILP
$1111 (1) -> RR.
(18)
Then prove the feature mappings of the MILP-sequence are measurable in the following theorems. Theorem 11. The feasibility mapping for Meeueesequence is measurable.
Proof The target space {0, 1} is equipped with the discrete o-algebra, where every subset is Borel. Specifically, the Borel sets in {0, 1} are:
B1o,1) (0, [0), (1),
We consider each possible Borel set B S {0, 1} and examine $-1 (B):
1. 0:
(0) = 0. T Teas
which is trivially a Borel set.
2. B (1):
TL ((1)) (1)))<(<,,))) MIP
(c,
the MILP instance is feasible
The feasibility MILPP instance is determined by the existence solutions that satisfy all linear constraints and integrality conditions. Specifically, it requires that there exists E TR such that:
Ax < b, 6 u, I E E
where is the set of indices corresponding to integer variables.
The set of feasible MILP instances can be expressed as the intersection of:
finite union closed half-spaces defined by the linear constraints Ax S
Closed intervals defined by the variable bounds &. < x < u.
Discrete conditions x₁ E 2 for integer variables.
Since finite unions and intersections of closed sets are closed (hence Borel), and the discrete conditions correspond to countable intersections, t ({1}) is Borel set in the product
topology of TR (m++)))++) HMILP.
24

---

Published as a conference paper at ICLR 2026
3. B (0):
c teas ((0))
-(0r,))
E HMILP
the MILP instance is infeasible
The infeasibility set is the complement of the feasibility set:
+ 1eas ({0}) = ( ( (f1))))
Since ((1}) is Borel set and the complement of Borel set is also Borel set, + fcas (f0)) is Borel.
4. B 10.1):
leus (10, = MIPP
which is the entire space and hence a Borel set.
Since for every Borel set B S {0, 1}, the preimage & Tre is Borel set in ((m+4)(+22) x MILP.. the feasibility mapping &eass is measurable.
Remark F.1. The measurability of feas is crucial for ensuring that probabilistic and statistical analyses involving MILP instances are well-defined. Since Djeas maps measurable spaces to discrete space with simple o-algebra, its measurability guarantees that feasibility can be reliably incorporated into broader seaasre--theorticc f frameworks.
Remark F.2. Both the domain MIPP and the codomain 1) are equipped with their respective o-algebras. The domain utilizes the product topology. and HMIPP itself is product of measurable spaces as defined earlier. The codomain {0, 1} employs the discrete a-algebra, where all subsets are measurable.
Theorem 12. The optimal objective value mapping for MILP-sequence is measurable.
Proof To prove that the optimal objective value mapping Potj MMP.P + R U foo, -00 is measurable, need demonstrate that for every Borel set B C R (xo, _00). the preimage + (B) is Borel set in ((m+1)(++2)) HMILP.
(B)
The codomain R f0o, -0)
-o) can be equipped with the extended real line topology, where the Borel o-algebra is generated by the open intervals in E along with the points {oo} and {-oo]. The Borel sets f0o, 30) include:
1. BCR
2. B contains and/or 00
We consider each category of Borel sets in R U {oo, -oo} and examine 4 (B).
1. BBCC R: (1,
① (B) (1,,)) E MILP the optimal objective value of the MILP instance is in B
Assuming that the MILP's optimal objective value is determined by a continuous opti- mization process (which holds under certain regularity conditions, such as linearity of the objective function and constraints), Posj can be considered continuous function on the feasible set. Therefore, the preimage of any Borel set C R under Potj is Borel set the domain.
2. B contains oo and/or - -0:
{ (I, H MI.P. (zn h))
obj
(B)
The inclusion of co or -c typically corresponds to the infeasibility or unboundedness of the MILP instance:
If (2, h) oo, the MILP instance is unbounded above.
If Potg(1; /t) -00, the MILP instance is unbounded below.
25

---

Published as a conference paper at ICLR 2026
These conditions define specific subsets of the domain:
bbb ((oc)) (2, HMPP MILP unbounded above
and
+ 2a) (1- = (a, h) HMMPP
MILP is unbounded below
Assuming that the conditions for unboundedness are also defined by Borel sets (similar to feasibility), these preimages are Borel sets the domain.
Since for every Borel set S [o0, -o), the preimage ( HMILP, the mapping ony measurable.
(B) is Borel set (m ()x++))))))))+++)))+++++++)++)))
Before give the proof of the mensurability of the optimal solution mapping for the MILP- sequence, we first introduce the Jankov-von Neumann Measurable Selection Theorem.
Theorem 13 (Jankov-von Neumann Measurable Selection Theorem, (von Neumann
1949) Let
A) and (Y, B) be measurable spaces, and let S X - 2 be measurable set-valued map such that for all
X, S(x) is non-empty and closed in Y. Then, there exists a measurable function f: X + Y such that (x) E S(x) for all x E X.
Remark F.3. The Jankov-von Neumann Measurable Selection Theorem provides crucial guaran- tee in measure theory and its applications. Given measurable space X and set-valued mapping A: X + +2 where each A(x) is non-empty set, the theorem ensures the existence of a measurable function f that selects element from each A(x) in a measurable manner. Specifically, for almost every I E X, the function f assigns a value f(x) that belongs to the set A(x). This result is particularly useful in areas such as optimization, probability theory, and economics, where selecting measurable choices from a set of feasible options is essential.
Then prove the measurblity of the optimal solution mapping for the MILP-sequence Theorem 14. The optimal solution mapping for the MILP-sequence is measurable.
Proof. This part proves the optimal solution mapping for the MILP-sequence measurable. To prove that the optimal solution mapping
solu
H
MILP
Eeas (1) - R
is measurable, we need to demonstrate that for every Borel set B C R", the preimage
+ zo (B)
a Borel in MLP.
Consider the mapping & as a selection function that assigns to each feasible MILP instance its unique optimal solution with the smallest mo-nom.. Formally, for each
(x, h) MI. n n 1 (1),
there exists least x² E R. such that xx optimal solution. We aim select unique x for each instance in a measurable manner. With Theorem [3] for each
(x, h) MILP n T T1 (1),
the set of optimal solutions
S(x, h) {x" Ih
x= is an optimal solution for (æ,1)) h))
non-empty and closed, then there exists a measurable selection function
solu H
Toas (1) - I2"
such that for all
P (x, h) Szz ht)
(x. h) E MIPP n 1 (1).
Then, prove the Non-emptiness and Cloasedness of AMILP n +b -1 (1), detailed
26

---

Published as a conference paper at ICLR 2026
1. Non-emptiness: By definition,
H
(1)
consists of MILP instances that feasible and have finite bounds, ensuring that optimal solution exists. Therefore, S(x, h) is non-empty for all
(x,, h) MILPP n o
2. Closedness: The set of optimal solutions S(x, h) is closed in IR.. This is because optimal solutions to MILP problems, defined by linear constraints and objective functions, form closed sets under standard topologies.
Given that both the closedness and non-emptiness conditions are satisfied, Theorem ensures the existence of measurable selection function P Since ssolu is measurable selection function by theorem, for any Borel set B C RR"
(B) ((r,h) H @ (x,, B}
a Borel set in MMP.P Therefore, the mapping s is measurable.
After establishing the measurabilityy of the feasibility, optimal objective value, and optimal solution mappings, we define the corresponding measurable mapping sets for each of these mappings as follows:
Definition 15. (Measurable Mapping Set for Feasibility Mapping) The Feasibility Mapping Set consists of all measurable functions
HMILPP
(0,1),
where PFess (v. h) indicates that the MILP instance defined by (ar, h) is feasible, and Preas h) 0 indicates infeasibility. Formally, the set is defined as:
JMIPPP feas
1eas MILP -> {0,1} P is measurable }
Definition 16 (Measurable Mapping Set for Optimal Objective Value Mapping). The Optimal Objective Value Mapping Set comprises all measurable functions
Pobj
MILP
R (20,
which assign to each MILP instance (x, /)) its optimal objective value. Specifically,
x-'x"
if the MILP instance is feasible and bounded,
4 Pony h)
0 if the MILP instance unbounded above.
0
if the MILP instance unbounded below.
Formally, the set is defined as:
MIIP
Pobl H MI.P. +
+R (00, -00 Poty is measurable
Dbj
Definition 17 (Measurable Mapping Set for Optimal Solution Mapping). The Optimal Solution Mapping Set consists of all measurable functions
MILP
n 4 Teus (1)
IR
salu
which assign to each feasible and bounded MILP instance (xr, h) its unique optimal solution x' with smallest norm. Formally, the set defined as:
J-IPPP
T solu
MILP
11110 feas
(1) - Rd
o solu is measurable
27

---

Published as a conference paper at ICLR 2026
G THE PROOF OF THE SECTION 5.2
Firstly, theorem Lusin is presented before the proof in this section.
Theorem 18 (Lusin's Theorem, (Alipraniss & Border, 2006)). Let f : RR - R. be a measurable function on a measurable E' C R with finite measure. For every > there exists a closed set C' C E such that the measure of E' \ C is less than e, and f restricted to C is continuous.
Secondly, for the probability measure used in Theorems we would like give the Remark G.1 to clarify the probability using:
Remark G.1 (On the Probability Measure). The probability P(-) in Theorems 2 accounts for the randomness in neural network training, including random initialization and stochastic optimization the space of trained networks induced by the
(SGD). Specifically, P is the probability measure random training process. This formulation aligns with the Probably Approximately Correct learning. For fixed trained network FHYA, the probability can be interpreted the empirical error rate on the finite dataset D.
Then we give the proof the theorems in the Section 5.2
G.1 PROOF AND COROLLARY ON THEOREM 2
Proof of Theorem 2 We leverage the measurability of the feasibility mapping Preas the Jankov-von Neumann Measurable Selection Theorem, and Lusin's Theorem construct neural network FiYA that approximates Peas with high accuracy on the finite dataset D.
From earlier definitions in Definition 15 Preas HMILPP {0, 1} is measurable. This ensures that ieas is compatible with measure-theoretie frameworks. Lusin's 's Theorem states that for any measurable function and any >> 0, there exists compact subset where the function is continuous and the measure of the complement is less than 8. However, since D is finite dataset, we can consider the discrete measure where each point in D has equal probability mass.
Given the finiteness of D, Lusin's Theorem trivially holds as we can define = and select the entire dataset as the compact subset where Peas is continuous (since all functions on finite sets are continuous).
By the Universal Approximation Theorem (Hornik et al., 1989). Since our MILPnet is an architecture that combines at least one-layer feedforward network structure with activation functions, it can approximate any continuous function on a compact subset to arbitrary accuracy. Since D is finite, and ieas is effectively continuous on D, there exists neural network FнYa E HYA that satisfies:
(a) Tienn < 2.2
Vx E D.
This ensures that:
(1 if FHVA 1
T (x)
(0 otherwise,
matches Piems exactly for all x E D. Since D finite, the probability P can be interpreted as uniform distribution over D. Given that FHYA correctly classifies all E D, we have:
P (IFin((E)> # (x)as(x)) 0
= <.
Thus, the constructed neural network FhYA satisfies the required condition for the theorem.
Remark G.2. While the above steps suffice for finite datasel, the framework can be extended using the Jankov-von Neumann Measurable Selection Theorem for more general settings. This theorem ensures the existence of a measurable selection function that can be approximated by neural networks even in measurability and closedness conditions. However, for the scope of this theorem with finite dataset D, the construction above is sufficient to guarantee the existence of the desired neural network FHYA-
spaces, provided the feasibility mapping satisfies the necessary
Then have corollary on the infinite dataset for feasibility mapping:
28

---

Published as a conference paper at ICLR 2026
Corollary 1 (Extension infinite dataset). Let D HMP.P be an infinite continuous dataset with a finite measure pu() < oc. For any e > O, there exists a neural network FHYA E HYA
Prel such that:
([nnna(s)> + (tccu(()) <E
Vr D.
(19)
where x is the MILP-sequence, and ( is the feasibility mapping of the MILP instance.
Proof. We have proven that the feasibility mapping Preas measurable D in Theorem B.I.
HMMILP
{0, 1} is assumed to be
By Lusin's Theorem, for the measurable function Pieas and for any e > 0, there exists a compact (closed and bounded) subset C' D such that:
1dD <C<
and 4 feas restricted to C' is continuous:
Pieas : {0, 1} is continuous.
Since ffeas is simple indicator function, its continuity on C. implles that avoids the boundary cases where the feasibility of the MILP instances changes.
Given that & Peas continuous on C, by the Universal Approximation Theorem, there exists a neural network PHYA E HYA that approximates the indicator function [x) to within error less than 2 C. Specifically:
(F)x(()) Im)))()) < 2 Vx E C
This implies that:
(x)x [l = Vx E
Thus, the neural network FhYA correctly classifies the feasibility of MILP instances in C. Consider the probability that the classification error occurs:
P F * Preas (z)
This event can only occur i E Therefore:
\ C. since for x E C, the classification is guaranteed to be correct.
P P [Eima (I)
1
Preas (2) <M(DiO)<
Thus, the probability that FAYa misclassifies any x E D is bounded by c. By constructing FHvA using Lusin's Theorem ensure continuity on a large compact subset C' of D, and then applying the Universal Approximation Theorem to approximate the feasibility indicator function on C, we have established the existence of neural network within JIYYA bound for the classification task on infinite or continuous datasets D.
JMI..Pnet that satisfies the desired probabilistic
G.2 PROOF AND COROLLARY OF THEOREM 3
Proof of Theorem 3
To prove the theorem, we leverage the Lusin's Theorem alongside the Universal Approximation Theorem for neural networks. The proof is divided into two parts corresponding to the classification and regression problems.
1. Classification Problem:
We aim to construct a neural network FHYA, that accurately classifies whether the objective value Doy (x) is finite for all E D.
First, we have proven that the feature mapping P MIIP - R U (20) is measurable. By Lusin's Theorem, for any measurable function defined on measurable set with finite measure, and for any 0, there exists closed set C' C D such that:
μ(D
< <
and f restricted to continuous. Since finite, the measure ((D) finite, and thus Lusin's Theorem is applicable. Therefore, there exists subset C с D where P is continuous.
29

---

Published as a conference paper at ICLR 2026
Given that Potj is continuous C, the Universal Approximation Theorem, there exists neural network F JIYA that approximates the indicator function (a)EE to within an error less than on C. Specifically:
1
FILYA, (x) (a))E
< 22 Va E
Since ((D \ C) the probability that FuYA,1 misclassifies any E D is less than et
P (T * (()(())) <,, Vr E
2. Regression problems:
Next, construct neural network F predict the objective value Poty (1) with error less than for all x E D where Pusj(z) (z) is finite.
We have proven that the daby is measurable in Theorem B.2. By Lusin's Theorem, for the regression task, there exists closed subset C" Dn on 4 Pbi (IR) such that:
# (D 2 ohj (RR) \ C') <e
and obi is continuous on C".
By the Universal Approximation Theorem, there exists a neural network FHYA, E approximates Pony to within error 5 on C²:
Mill that HYA
(F)) Losy (z)l < 8, e C"
Since the measure of the complement set is less than e, the probability that the prediction error exceeds 8 is bounded by
((FinA.2(x)) (b))(()) > <, Vr E D o T
T- (R)
By constructing FIYAA and FHYA.2 using Lusin's Theorem to ensure measurability and continuity on
large subsets, and then applying Universal Approximation Theorem approximate the respective functions, we have established the existence of neural networks within MiL that satisfy the
IYYA desired probabilistic bounds for both classification and regression tasks on the finite dataset D. O
Then we give the corollary on the infinite dataset for optimal objective mapping as follows:
Corollary 2 (Extension to Compact Infinite Sets). Let K C HMILP be compact subset equipped with the measure pypaa defined in Section Define the probability measure J' on K by normalizing:
P(A) "gM=r(K] [
Pgmur(A A) for any measurable set A C K.
For any 6, > 0, there exist two neural networks FIYA,I, FIYA,2 E
Milt such that: IIYA
Classification of finite objective values:
( [FFm # [T <e E
(20)
2. Regression of objective values:
P P(IPHYA,((T) o <6, Vr K Kenknn
(21)
Proof. The proof parallels Theorem replacing the counting measure on finite D with the probability measure P on compact K.
1. Classification Problem.
Since Pobj MILP + (co) measurable and is compact with [ oo, we apply Lusin's 's Theorem.
For any e O, there exists a closed set C C K such that:
P(K CC) = Hmpar K |00 C)
E
<
Mmprr (K)
2
30

---

Published as a conference paper at ICLR 2026
and Potyle is continuous. By the Universal Approximation Theorem, there exists PHYA,1 such that:
Poet HYA
sup aEC
( - L
This ensures that for all x E C:
FivA (2))> (x))
The set of misclassified points is contained K \ C. therefore:
(IFra, F (1)E))) S P(K)
Regression Problem. Let Kinite-
K 4 To (IR). Define the conditional probability measure on
K finte by:
Piiite (A) P (A)
C Kinite
pepuur K finite)
By Lusin's Theorem applied to Pooj on Kiitee for any e 0, there exists a closed set C" Kinite such that:
Panite Pinite (Kiint
(C*)
MiL and & Ic is continuous. By the Universal Approximation Theorem, there exists FIYA,2 such that:
HYA
sup ((HYA,2(x) (zb) < =EC*
Therefore:
Proite (|FHYA,((I) (oo)(()) > 8) S Panito (Kinit 109
G.3
PROOF AND THE COROLLARY OF THEOREM
Proof of Theorem 4 Since 1) is finite, let denote it
D {x1, T2,
for some integer 1. For each 3; D, 4 watu (v:) is well-defined finite solution in IR
The Universal Approximation Theorem states that a feedforward neural network with at least one hidden layer and sufficient number of neurons approximate any continuous function compact subsets of IR" to any desired degree of accuracy, provided the activation function is non-linear (e.g.. Sigmoid, ReLU).
Given that D finite, it trivially compact. Therefore, there exists a neural network FiYA, that can approximate the mapping sotu on D with arbitrary precision. Specifically, for each Ię D, we can ensure:
|FYAA w (z²) Lunta (I;))l < 6
by appropriately choosing the network architecture and weights W.
Since 1) is finite, the probability P can be interpreted over uniform distribution any probability measure defined on D. However, because we have constructed FHYA, such that the approximation error less than ô for every a E D, the event
[/HYA, (
does occur for any x D. Therefore:
P ((I) (x)l >0) =0<e
for any e 0.
By the Universal Approximation Theorem, we can construct a neural network FAYA, that approx- imates the solution mapping & &aolu on the finite dataset with an eттоr less than 5 for all æ e D. Consequently, the probability that the approximation error exceeds 8 is zero, which is trivially less than any 0. This establishes the existence of such a neural network within Mi.
Then have the corollary the infinite dataset:
31

---

Published as a conference paper at ICLR 2026
Corollary 3 (Extension to infinite dataset). Let D C obyj (R) M be infinite dataset with finite measure j²D) < oc. For any c, > there exists neural network FIYA,W E M such that:
P P(Il/iv,w P [x)] > 8) <e, Va D, where P denotes the probability measure D.
(22)
Proof. To extend the theorem to infinite datasets, we employ Lusin's Theorem in conjunction with the Universal Approximation Theorem.
Assume D is equipped with probability measure such that (DD) = 11 (without loss of generality, as we can normalize the measure). Also, have proven that the solution mapping Psolu MMI... R is measurable on D before.
By Lusin's ' Theorem, for the measurable function 2 and for any O, there exists compact (closed and bounded) subset C' C D such that:
and 4 solu restricted to C' is continuous:
solu Io C R is continuous.
Since C' is compact and P is continuous, the Universal Approximation Theorem ensures that there exists neural network P JMILPpee such that:
sup [|HHYa.(x)) Dsota (x)]l ZEC
This implies that for all a E C:
FFIYA, (x) solu (2)ll 8.
Consider the probability that the approximation error exceeds 6:
P ( ((]), saiu(()) 6).
This event can only occur if x E \ C, since for x E C, the error is guaranteed to be less than Therefore:
P (() sutu (x)( || 6) < pl ((D(C) <)
Thus, we have:
P ((] sx)
<E
By Lusin's s Theorem, we ensure that & is continuous on a large subset C of D. The Universal Approximation Theorem then guarantees the existence of neural network FIYA,W that approximates Pootu within on C. Consequently, the probability that the approximation error exceeds on the entire dataset D is bounded by
This establishes that for infinite or continuous datasets with finite measure, there exists neural network within Muil that satisfies the desired probabilistic bound the approximation error.
H PROOF OF THE STABILITY
In this section, establish the stability properties of the MILP mappings Preus. Labj. and & under small perturbations of problem coefficients. Our analysis builds upon Berge's Maximum Theorem (Aliprantis & Border 2006].
Theorem 19 (Berge's Maximum Theorem Aliprantis & Border (006)). Let X and T be topological spaces, and let f X T R be continuous. Let : X be correspondence (set-valued map) such that:
1. C(t) is non-empty and compact for all t T,
2. C. is continuous, i.e., its graph Gr(C) = {(t, x) : x C(t) closed and is lower hemicontinuous.
32

---

Published as a conference paper at ICLR 2026
Define the value function and argmax correspondence by:
V(t) max f(x,t), (t)))= arg max f(a,t) - (()))) v())) =EC(t)
=EC(t)
Then:
V(t) is continuous in t.
2. (t) is non-empry, compact-valued, and upper hemicontinuous in t.
We first analyze the stability of Pabj. which exhibits the strongest stability properties among the three mappings.
Definition 20 (Continuous Relaxation for MILP sequence). For MILP sequence x
[Ncomss..... Monns ht", he" 10 ,0",,"] E HMMLP, its continuous relaxation is the linear program obtained
by dropping the integrality constraints:
CR(x): min ((, Ax S b, 8 Ssasu,
PER"
where A, b, é, are extracted from the tokens in x.
Proposition 21 (Stability of Continuous Relaxation). Let MMIP be feasible MILP instance whose continuous relaxation has bounded feasible region. Then the optimal value function of the continuous relaxation, denoted @CR is continuous x. Furthermore, the optimal solution is non-degenerate, then &CR obj locally Lipschitz continuous with constant:
L ||c| diam(Feas(x)),
where Fxas(x) denotes the feasible region and diam(-) its diameter.
Proof. The continuous relaxation defines a parametric linear program where the parameter / = (A,b, b,bb,,, G, u) lives in HMILP, Let X = R" and T HMILP. The objective function [xx (c,x) continuous in (xr (). Then the feasible region correspondence C(l) = fxr E IR= Ax S b, S u} satisfies:
. Compactness: By assumption, C(x) is bounded. Combined with the closed constraints, ((x) is compact.
The graph Gr(C) is closed because the constraints define closed set in IR" x HMILP, Moreover, C' is lower hemicontinuous: for any x E C(t) and fn - t. since the constraints are linear and continuous in t,, small perturbations preserve feasibility in neighborhood.
Continuity:
By Berge's Theorem 19 CCR (x) is continuous at Under non-degeneracy, the optimal basis remains unchanged in neighborhood x. The optimal value is given by the basis solution x² B B-1bg. where B is the optimal basis matrix. The objective value is:
&C (x) = (cB, B bB))
which is linear function of (CB, (CB,b)) within the neighborhood where the basis constant.
For perturbation x' x + with | < 8:
C obj
(x') CC (x)
(c', x²t) (c, x²))
5 (c' c, x " + (c 20 -x===")) x²)1
< || - cll2 ||a'*| + ||e|12 ||00 20112
L. 6,
where 7. ||c||2 diam(Feas(x)) bounds both terms.
33

---

Published as a conference paper at ICLR 2026
H.1 STABILITY OF THE OPTIMAL OBJECTIVE MAPPING
Theorem 22 (Conditional Stability of MILP Objective Value), Let x E HMILP be feassibll MILP
sequence with bounded feasible region. Consider perturbation x' x + where ||||| S 6. If the following assumptions hold:
Feasibility preservation: feoos(x) =1
=x
2. Optimal solution stability: The optimal integer solution x²' E 8d of x remains feasible for
x',
3. Bounded diameter: diam(Feas(2)) (23) D.
then:
(x') (xb)(x)) (l/c|12 + 6) D.
Proof. Let xx T be optimal solution to and x²* E Z be an optimal solution to x'.
By condition (2), x² feasible for x', so:
onn (c, S (c,x"))
Similarly, x'* is feasible for its problem, and by condition (1), both xt and x² lie in the bounded integer feasible region. Thus:
Oon (x) x*).
Now:
Paby (x') (xb)(x)
= (e, x²T) - (c,x²)
S [d x²") - (c, x1)) + [(,,'') - (c, x*)|
1(d e, c,,"')] + 1',,,' x²)|
I ||c dll2 || lla ||c||2 -e--=x111
< 8 D + ||c|l2-211 D
([|cl22 + +))
where we used ||e' - clla < 8 (from the perturbation bound) and ||>" - x² 22112, "1,,,112** l|2 S D (from condition 3).
H.2 STABILITY OF THE FEASIBILITY MAPPING
We now analyze the stability of Picas-
Theorem 23 (Conditional Stability of Feasibility). Lei x E HMILP with ( = 1. Suppose
the feasible integer points satisfy strict feasibility condition: there exists p such that for all x E Feas(x) n 2",
p1, where 1 is the all-ones vector. Then for any perturbation x' + with ||Ello 50 < p. we have:
Ax < pl, L e+pl
L (x') = 1.
Proof. x* Feas(x) n be any feasible integer point for x. For the perturbed instance x' with parameters (A', b' f", w'), we have:
|| All Sō bb b 5, 12 Lg s 5, 104 uz1 S 8.
Check constraint satisfaction:
A'r' = (A+ AA)r Ar + AAx"
$ (b p1) + |<
< pl 8 11
34

---

Published as a conference paper at ICLR 2026
If 8 |= then A'x" b -p--=1=="111)) <..
For sufficiently small 6 p. can ensure all integer points remain strictly feasible. Therefore, icas (x') 1.
H.3 STABILITY THE OPTIMAL SOLUTION MAPPING
Finally, we analyze Psolu-
Theorem 24 (Conditional Stability of Solution Mapping). Let H with optimal solution x" E L". Suppose x² is the unique optimal solution and satisfies a strong optimality gap condition: for all x E Feas(x) O T with x * x",
2) (c,x") +7
for some y Then there exists 8 > 0 such that for any perturbation x' + e with ||ell S 8:
① ennuu (x' = x²
That is, the optimal solution remains unchanged.
Proof. For the perturbed instance with objective = c Ac where | S 6:
For any x E Feas(x') n with x: a x²:
(c,x) (c,x") = (c Ac, x) - (c+ Ac,x")
((e,z) x) - (c, )) + (Ac, - x²)
|A|||l|| x
2 D,
where D = max || - 221 over the finite integer feasible set. If < D. then (c,x) > (d, xx)) for all x # x², ensuring x² remains optimal for x'.
I DETAILS OF THE TIME-COMPLEXITY OF MILPNET
Excluding the linear transformation, the time complexity for multi-scale operations across all windows can be simplified to O (d(m== + 4) Σx m7 The global attention mechanism employs a global multi-head self atf-attention mechanism over the entire sequence, resulting in final time complexity of O (m 4) Em= hnd + hmm + 4) for the hybrid attention. Given that the maximum window size satisfies 7/max I m + 4, follows that Σk=1 Me S.N. (m + 4). Therefore, the time complexity can further simplified to (h- (m + 4d + 1)), where represents the linear embedding size and h denotes the number of attention heads.
SPARSE VARIANT
To mitigate the quadratic overhead of global attention, we evaluated a multi-scale sliding sparse attention (with sliding mask) to replace the global attention, and discovered that it can achieve faster inference while maintaining comparable performance. As the sparse sliding attention uses sparse masks to restrict each position's attention only specific locations beyond the step interval the time complexity of the stride attention is:
(o + 4 (
hi(m
Taride
(23)
Thus, the time complexity of multi-scale sliding spare attention becomes:
T Tpaare (dh + 4) E The + (m + 4)²/s))
(24)
35

---

Published as a conference paper at ICLR 2026
Table 14: Experimental results on FOLD(200,20) with 1-hour pre-training and stride of 2
Fold(200,20)
Error
Inference Time
MSE
GCN
5000
0.228
0.268
MILPnet Sparse)
763
0.033
0.091
MII.Pnet(ours)
191
0.117
0.016
We can calculate the speed-up ratio against our original design
Σ Uk + (m 4) Σk= 7k + (m + 4)/s
Speedup
(25)
In practice, (m + 4) >> Σx=1 The (sequence length much larger than window size, large scale MILPs), so the speed-up of the comp-exxl is finally as:
(m 4)
Speedup
(m + 4)/8
This signifies that the model's scalability with engineering optimizations.
K DETAILS OF THE EXPERIMENTS
K.1 DETAILED EXPERIMENT SETTINGS
Baselines. As methods using variant GNNs for MILP representation are still limited, adapted several representative graph algorithms to serve as baselines. To validate the advantages of our sequence-based al algorithm, we compare our model against multiple graph-based representation methods that model MILPs as bipartite graphs. The GNN-based networks include GCN (Chen al. 2203bb,, GIN (uu al. 2 and SAGE (Wu et al. [02I]. The attention-based graph networks include PGN (Cappart et al. 2022) and GraphGPS (Wang et al. (2023b). We also include the random feature graph modeling method proposed in (Chen al. (023b)), which specifically designed to alleviate the feasibility prediction problem on Foldable instances.
Metrics. We established several evaluation metrics for the experimental settings of the various feature mappings discussed in the context of Mixed- Linear Programming (MILP). (1) (1pproxiimtaoon Error: For all feature mappings, we assessed the average approximation error, defined as the mean of prediction eπrоrs across all instances. (2) Feasibility Error Number: Specifically for feasibility mappings, we designed metrics to measure the error prediction rate and the number of prediction errors. (3)Model Params: We evaluated the size of the model parameters to assess the impact of different feature mappings on the model's complexity.
Implementation. The MILPnet module is implemented using PyTorch. Our experiments were conducted on a single NVIDIA 4090Ti GPU (24GB) and a 12th Gen Intel(R) Core(TM) i5-12600KF 3.69GHz CPU for Foldable instance experiments, for real-world MILP experiments, NVIDIA A800 GPU.
K.2 DETAILS OF GENERATING FOLDABLE MILP INSTANCES
We strictly follow the desl-desi foldable dataset generation method and the publicly available code implementation provided by Chen et al. (2023)) adhering to the formal definition of foldable MILP (Definition 4.1) therein. In details, we set the Ci
Cn foldable instances as D1, the C1 Cn. 0.01 foldable instances as D2. We use D1 and D2 worth mentioning that in the representation and generalization experiments on foldable instances, training contains 10,000 foldable instances, while the test set also includes 10,000 foldable instances, with each set containing 5,000 feasible instances,every instance is generated with different LP files (i.e. different integer indices and different lb and ub).
experimental datasets. It
Variable Generation: The lower and upper bounds for each variable 2, are generated from a normal distribution. If the lower bound is greater than the upper bound, they swapped. Some variables
36

---

Published as a conference paper at ICLR 2026
specified integer variables (i.e., Z, E {0, 1)), while others continuous variables, with lower and upper bounds. The continuous and integer variables, construct different MILP variables.
Feasible and Infeasible Problem Constraint Setup: We strictly follow publicly available code implementation provided by Chen et al. (203b) for feasible and infeasible problem construction. For example, for the feasible problem (I:I), the constraints are follows:
Jjj + J222 = 1, T2 + Jo
Fj + Ijj =1
These constraints connect binary variables with equality, forming a cyclic structure that guarantees the problem is feasible; For the infeasible problem (k:2,, the constraints are adjusted to make the problem infeasible. For example, in publicly available code implementation provided by |Chen et al. (2023b), the basic infeasible construction logic from (Chen al. 20236) is:
2j + 32 = 1, Jjj
JJjs = 1, J/ + T2 =1
This setup results in an infeasible problem, which is basic but strong construction.
K.3 DETAILS OF THE REPRESENTATION EXPERIMENTS
In the representation experiments of our MILPnet on Foldable instances. The graph-based methods share the same set of experimental seeds and details MILPnet. And our feasibility dataset consists of 50% feasible instances and 50% infeasible instances, while other datasets are obtained by removing infeasible samples.
K.4 DETAILS OF THE GENERALIZATION EXPERIMENTS
In the generalization experiments of our MILPnet on Foldable instances, the network is firstly pre- trained for same-scale MILPs in training set. For the feasibility generalization experiments, limited the pre-training time, setting it to 3 minutes and minutes on FOLD20, 10 minutes and 30 minutes FOLD50, and hours FOLD100 to FOLD500. We then compared the performance of our model with other baselines under different pre-training time constraints.
0.025
080.8
088
0.004
0.000
18
28
Figure 14: Representation experiments on the optimal objective value for FOLD20. MILPnet approximates the optimal value mapping of Foldable MILP instances with smaller emoors than graph- based method.
K.5 FOUR COMMON REAL- WORLD MILP SOLVING BENCHMARKS
This section introduces the details of constructing specific MILP instances. In particular, for the SC, CA, and FC problems, we follow the instance construction method described in Learn2Branch Gasse al. 2019). and the numbers of variables and constraints shown in the table below.
37

---

Published as a conference paper at ICLR 2026
Table 15: Number of variables and constraints for real-world benchmarks
Dataset
Variables
Constraints
IP
1,083
195
SC
200
40
CA
300
40
FC
200
60
L ADDITIONAL UNFOLDABLE EXPERIMENTS.
Although this paper primarily conducts experiments on foldable MILP instances, we also perform representation experiments unfoldable instances. Similarly, compare method with graph- based networks. In this part, the max window size chosen as and the MILPnet embedding size is chosen 32, the graph-based methods (We select GCN) embedding size is 8.
Spwrl (6) Fessbll ERGORaamber
(4) Feualble MSE
(c) Optinal Solution MSE
Figure 15: In the representation experiments on unfold20. MILPnet still outperforms graph based methods with a smaller approximation error. Additionally, maintains a smaller parameter size while achieving an estimation error lower than that of the graph-based method.
38


Reviews:
[{"review_id": "juM3Us2Y5o", "reviewer": "Reviewer_vnbQ", "summary": "This paper proposes MILPnet, a multi-scale hybrid attention model that represents MILP problems as geometric sequences instead of graphs to address GNN limitations on \"Foldable MILPs.\" It encodes constraints, objectives, and variables into sequences, uses shifted-window attention for local-global feature capture, and proves approximation capabilities for feasibility, optimal values, and solutions.", "questions": "None", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper shifts from graph-based to sequence-based representations for MILPs, addressing GNN limits in expressiveness for Foldable instances. It provides proofs of universal approximation results. The numerical results report improvements in accuracy and efficiency on Foldable instances.", "weaknesses": "1. In Theorem 2-4, only a finite dataset $D$ is considered. Since the dataset is finite, I do not think the notation $P$ is needed and the $\\epsilon$ in equations (9) and (10) is not needed. The authors are basically borrowing proof ideas from the work of Chen et al but the proof presentation and notation have much room for improvement.\n2. The sequential representation breaks the permutation-invariance property of MILP.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KEdaCZJbI7", "reviewer": "Reviewer_qd9C", "summary": "This paper proposes MILPnet, a multi-scale hybrid attention framework that departs from traditional graph-based models to address the Mixed Integer Linear Programming problem. Instead of modeling MILPs as bipartite graphs, MILPnet represents each MILP instance as a geometric feature sequence. The paper proves that MILPnet can approximate MILP feasibility, optimal objective value, and optimal solution mappings with arbitrarily small error in a measurable topological space, and empirical results demonstrate that it outperforms graph-based methods by multiple orders of magnitude in feasibility prediction accuracy and convergence speed on Foldable MILPs.", "questions": "Please see the Weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1) By modeling MILPs as geometric feature sequences, it overcomes the Weisfeiler-Lehman test’s expressive limit, enabling effective distinction of Foldable MILP instances that graph-based models fail to differentiate. \n\n2) It provides strict mathematical proofs that guarantee arbitrary-precision approximation of MILP feasibility, optimal objective value, and optimal solution mappings in a measurable topological space, ensuring reliability. \n\n3) Empirically, it outperforms graph-based baselines by multiple orders of magnitude in feasibility prediction accuracy and convergence speed on Foldable MILPs.", "weaknesses": "1. The maximum window size in its multi-scale attention mechanism does not follow a monotonic pattern—smaller windows accelerate convergence on simple instances (e.g., FOLD(20,6)) but degrade performance on complex ones (e.g., FOLD(100,20)), requiring careful tuning for different problem scales. \n\n2) The paper provides limited empirical validation on extremely large-scale MILP instances, leaving uncertainty about its scalability in industrial-grade scenarios with massive constraints/variables. \n\n3) Although padding ensures topological equivalence, the framework’s performance on MILP instances with drastically varying constraint counts lacks in-depth analysis, raising questions about its stability across highly heterogeneous problem sizes.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2vpzmAHgCM", "reviewer": "Reviewer_DtRt", "summary": "The paper proposes MILPnet, a multi-scale hybrid-attention model that represents MILP instances as sequences of geometric features (constraints/objective), rather than graphs. The motivation is that message-passing GNNs are limited by 1-WL expressivity and can fail on families of “Foldable MILPs” that are indistinguishable under WL-type symmetries. MILPnet builds multi-scale views of these sequences and applies cross-scale attention to capture local and global structure. The authors prove universal approximation properties (feasibility, optimal value, solution mapping) on a measurable topological space and report large empirical gains over graph-based methods on Foldable MILPs, with fewer parameters and better convergence; they also show generalization across scales and integration into a solver pipeline.", "questions": "1. Where exactly in a production solver would you place MILPnet (branching score predictor, primal heuristic, node ranking, cut selection)? Please quantify wall-clock speedups and gap closure.\n2. Is MILPnet permutation-invariant to constraint/variable orderings? If not, what data augmentation or canonicalization is used?\n3. Can the theory be tightened to show sample complexity or stability under small perturbations of coefficients?\n4. How sensitive is performance to scale hyper-parameters (window sizes, strides) and to sequence length (very large models)?\n5. Could you hybridize with recent high-capacity GNNs (e.g., higher-order/WL-beyond) to reap graph-topology benefits while preserving sequence expressivity?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. Compelling motivation: explicit WL-type failure mode for GNNs on MILPs; sequence modeling avoids that bottleneck. \n2. Multi-scale design: principled way to capture local/global geometric structure in constraint/objective sequences. \n3. Theoretical guarantees beyond typical empirical claims. \n4. Practical angle: improved convergence and smaller models; integration into solver workflows.", "weaknesses": "1. The comparison is not comprehensive. Please compare to modern ML-augmented B&B (e.g., branching/cuts/node-selection learned policies) and clarify where MILPnet plugs into the B&B stack (pre-solve heuristic? branching score? feasibility classifier?). \n2. While the paper compares against standard GNN-based MILP representation methods, to convincingly establish state-of-the-art performance, it should also include or discuss recent baselines such as RL-based MILP solvers (e.g., RL-MILP Solver 2024), partial-assignment reduction frameworks (e.g., ConPaS 2023/24), solver-configuration learning methods (e.g., Separator-Learning 2023), and model-reduction methods (e.g., Learning Model Reduction 2024). Including these would strengthen the empirical claim of the universality of your sequence-based representation.\n3. More ablation studies are required. Provide ablations on (i) feature choices per token, (ii) number and granularity of scales, (iii) attention depth vs. performance, and (iv) effect on different MILP families (set-covering, capacitated facility location, routing, scheduling).\n4. Generalization tests are needed to demonstrate the practical potential for the work. Beyond Foldable MILPs, evaluate on large real-world sets and report time-to-first-feasible and optimality gap closed vs. SCIP/CPLEX.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)