RoME: Domain-Robust Mixture-of-Experts for MILP Solution Prediction across Domains
Authors: Tianle Pu
Zijie Geng
Haoyang Liu
Shixuan Liu
Jie Wang
Li Zeng
Chao Chen
Changjun Fan
Venue: NeurIPS 2025 poster ()
Keywords: Learning to Optimize
Machine Learning for Combinatorial Optimization
Mixed-Integer Linear Programming
Primary area: machine_learning_for_sciences
TL;DR: A domain-robust mixture-of-experts for MILP solution prediction across domains

Abstract: Mixed-Integer Linear Programming (MILP) is a fundamental and powerful framework for modeling complex optimization problems across diverse domains.
Recently, learning-based methods have shown great promise in accelerating MILP solvers by predicting high-quality solutions.
However, most existing approaches are developed and evaluated in single-domain settings, limiting their ability to generalize to unseen problem distributions.
This limitation poses a major obstacle to building scalable and general-purpose learning-augmented solvers.
To address this challenge, we introduce RoME, a domain-Robust Mixture-of-Experts (MoE) framework for predicting MILP solutions across domains.
RoME dynamically routes problem instances to specialized experts based on learned task embeddings.
The model is trained using a two-level distributionally robust optimization strategy: inter-domain to mitigate global shifts across domains, and intra-domain to enhance local robustness by introducing perturbations on task embeddings.
We reveal that cross-domain training not only enhances the model's generalization capability to unseen domains but also improves performancewithin each individual domain by encouraging the model to capture more general intrinsic combinatorial patterns.
Specifically, a single RoME model trained on three domains achieves an average improvement of $67.7\%$ then evaluated on five diverse domains.
We further test the pretrained model on MIPLIB in a zero-shot setting, demonstrating its ability to deliver measurable performance gains on challenging real-world instances where existing learning-based approaches often struggle to generalize.

RoME: Domain-Robust Mixture-of-Experts for MILP Solution Prediction across Domains
Tianle Pu', Zijie Geng", Haoyang Liu', Shixuan Liu Jie Wang', Li Zeng', Chao Chen', '- Changjun Fan
Laboratory for Big Data and Decision, College of Systems Engincering, National University of Defense Technology
MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University Science and Technology of China
College of Computer Science and Technology, National University of Defense Technology {put122, liushixuan, zengli24, chenc1997, fanchangjun unn @nudt. @dn.dteedu... edu. cn fzijiegeng, dgyoung) ustc. edu. en, jiewangxdustc. cn
Abstract
Mixed-Integer Linear Programming (MILP) is a fundamental and powerful frame- work for modeling complex optimization problems across diverse domains. Re- cently, learning- -based methods have shown great promise in accelerating MILP solvers by predicting high-quality solutions. However, most existing approaches are developed and evaluated in single-domain settings, limiting their ability to generalize to unseen problem distributions. This limitation poses major obstacle to building scalable and general-purpose learning- based solvers. To address this challenge, introduce ROME, domain Robust Mixture-of oxxperss framework for predicting MILP solutions across domains. RoME dynamically routes problem instances specialized experts based learned task embeddings. The model is trained using a two-level distributionally robust optimization strategy: inter- domain mitigate global shifts across domains, and intra-domain enhance local robustness by introducing perturbations on task embeddings. We reveal that cross-domain training only enhances the model's generalization capability unseen domains but also improves performance within each individual domain by encouraging the model capture more general intrinsic combinatorial patterns. Specifically, single RoME model trained on three domains achieves an average improvement of 67. 37.% then evaluated on five diverse domains. We further test the pretrained model on MIPLIB in a zero-shot setting, demonstrating its ability to deliver measurable performance gains on challenging real-world instances where existing learning-based approaches often struggle to generalize.
domain-Robust
Introduction
Linear Programming (MILP) is fundamental and expressive framework for modeling wide range of real-world optimization problems, including logistics optimization [1] network design (2). scheduling ( and industrial planning 4 Due to strong modeling capacity, MILP has become a cornerstone of operation research and combinatorial optimization 5 However, MILPs are well known to be NP-hard, and solving large-scale instances with thousands of variables and constraints remains computationally intensive. Even state-of-the-art solvers such as Gurobi [8] and SCIP [ which integrate advanced techniques like branch-and-boume cutting planes, and heuristics, struggle to deliver timely and scalable solutions problem complexity grows.
*Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Domain (LS)
Domain 2 (IP)
Domain (SC)
Unseen Domain (MIPLIB)
Number of Experss
2.64
2:99
25
1.9
1.31
BMX 122 Mi
min
min
20
1,1 51 (PACy
max 3x, de,
Big 4.5 (KPS)
A
(COM)
One Single
Model
(PAR) 25,1%6 (KP5)
4 4,,45
(PAC)
1.8
1.57
1:24
Bh 400,
Number of Doanins
(a) Cross- Pross-Dommin Training and Generalization
(h) Objective gap (%)
Figure I: Illustration of cross-domain training. (a) A single model is trained on collection of MILP domains with varying constraint types [TO] such as PAC (set packing), KPS (knapsack), COV (set covering), and PAR (set partitioning), and evaluated on unseen domains such MIPLIB. (b) We conduct experiments across five distinct domains, reporting the average objective gap in percentage relative to the best-known solution while varying the number of experts and training domains. Each value in the heatmap denotes the average percentage gap to the best-known solution. The results show that training on more diverse domains and selecting an appropriate number of experts can effectively reduce this objective gap.
To alleviate these limitations, recent research has explored the use of machine learning to accelerate MILP solving by identifying data-driven patterns in problem instances (11444) These approaches fall broadly into two categories. The first integrates neural networks into solver internals by replacing
key components such as variable selection [TT), cutting plane selection [12, and neighborhood destruction strategies in large neighborhood search [15] [6]. The second line of research, which
we focus on in this paper, aims to directly predict initial feasible solutions [7-19, which are then refined by solvers to improve solution quality. These methods have shown promising results on large- scale MILPs and are gaining increasing attention. However, while this solution- -entiioo-rreitioon paradigm has shown strong empirical performance, most existing methods are trained and evaluated within single-domain settings, which presents two major challenges. First, the learned models often overfit to comain-specific patterns rather than capturing generalizable combinatorial structures, leading to limited generalization to unseen domains. Second, deploying such models practice often requires collecting new training data for each specific scenario, limiting scalability and broader applications.
To build scalable learning-based based solvers that generalize across domains, we advocate for a paradigm shift from single-domain training to frameworks that explicitly embrace cross-domain structural diversity. An illustration of the cross-domain training task is in Figure work is that training on multiple MILP domains not only improves generalization under distribution shift but also enhances solution quality within each individual domain (see Section (42). Rather than treating distributional differences as noise, we view them as valuable signals that help the model capture transferable combinatorial patterns. In this light, cross-domain training naturally acts as form of domain-level regularization, steering the model away from memorization and towards learning general principles that transfer across tasks.
central insight of this
In this paper, we propose ROME, a Domain- Robust Mixture-of -ffteet-of-Experts framework for MILP so- lution prediction. ROME aims to generalize effectively across various problem distributions while maintaining strong performance on specific tasks. The framework integrates Mixture-of-Exper (MoE) architecture with a two-level Distributionally Robust Optimization (DRO) objective into a unified trainable framework. The MoE consists of multiple expert networks, each specializing in distinct MILP distributions. For each instance, the MoE dynamically routes it to specialized experts based on its graph embedding, allowing the model to adapt both its internal representations and final outputs to the unique structural characteristics of each instance. To further enhance robustness, ROME is trained using a two-level distributionally robust optimization strategy. (1) Inter-domain robustness is achieved through group-DRO, which minimizes the worst-case loss across domains, thereby reducing the impact of global distributional shifts. (2) Intra-domain robustness is ensured by applying isotropic perturbations to the task embeddings, which stabilizes expert selection and predictions.
To demonstrate the effectiveness of ROME, we conduct comprehensive experiments demonstrate the effectiveness of RoME. Specifically, we train a single RoME model on three domains and evaluate across five diverse domains, where it achieves an average improvement of 67.7% over strong baselines. We further test the pretrained model on MIPLIB in zero-shot setting, demonstrating significant performance gain on challenging instances that existing learning- approaches often struggle to generalize. Finally, gain deeper understanding of ROME, we conduct interpretability

---

analyses that find expert activation patterns and task embeddings, shedding light on how the model captures and transfers structural knowledge across domains.
Related Work
2.1 Machine Learning for MILPs
Machine learning has been widely applied to the field of MILPs 220 23 based MILP solvers can be roughly classified into two categories. The first line of research integrates machine learning to enhance the solving efficiency of conventional solvers. Researchers utilize a learned model to replace some key modules in the solvers, including variable selection [1T
Recent work on learning-
24 13)
node selection (25) cutting plane selection 12 [88 and large neighborhood search [5 16 29
31. Another line of research focuses on leveraging learning-based models to predict an initial solution for MILPs. Nair et al. [17] introduces the first solution-predictioo framework for MILPs. Building on this foundation, Han al.
[8] proposes the Perdict-and-eearde (PS) framework, which incorporates trust-region search to improve feasibility and solution quality. To further enhance prediction accuracy, Huang et al. [19 employs contrastive learning to train the solution paretioonreeiiioon network. Liu et al. [32] adopts prediction-cor correctio paradigm to obtain higher- iteratively refining initially mis- mis-predicted variable assignments. Notably, Geng al. [33] is the first to propose an unsupervised method that incorporates gradient information into the prediction model for MILPs, thereby significantly reducing the cost of collecting high-quality training data.
solutions by
Cross-Domain Training Techniques
Mixture of Experts (MoE) 34 and Distributionally Robust Optimization (DRO) 33 36 are two core methodologies for cross-domain training, where MoE gives the model with the ability to perceive and adapt to different domains, and DRO enhances robustness by balancing training across domains. Early studies (37 focus on designing dedicated experts for individual domains, while recent research [38 [39] has shifted toward sparse architectural variants. The MoE paradigm has been successfully applied to graph-based combinatorial optimization problems. For instance, MVMoE [40] tackles a suite of vehicle-routing variants, and MoE-style encoder-decoder frameworks, e.g., MAB- architecture. In contrast, DRO guarantees robustness to potential domain shifts by minimizing the worst-case risk over an uncertainty set. Sagawa et al. [44] first introduces Group DRO, which balances the losses across different groups. CCD-DG [45] builds class-conditioned Wasserstein ball and automatically tunes its radius to protect against conditional shifts, while Moderately-DRe [46] and its stochastic variant [47] improves exploration of unseen domains and offeres tighter generalization bounds with lower sampling complexity.
MTL ET, GCNCO 42), GOAL 43], consistently adopt a
3 Methodology
In this section, we present ROME, a -obust Mixture-of- -f--erertssssseeeeeeo framework for cross-domain MILP prediction. This section is organized as follows. Section 3.1 introddues the problem formulation, the predict-and--sarch (PS) framework, and the cross-domain learning settings. Section 3.2 presents the Mixture-of-Expert. (MoE) architecture for instance-adaptive structure-awareness across different domains. Section 3.3 presents a robust training objective that effectively improves the robustness of the MoE model training. Finally, Section 3.4 introduces a group- Distributionally Robust Optimization (DRO) scheme to handle the cross-domand training process.
3.1 Problem Formulation
We focus on the task of learning to predict high-quality solutions for Mixed-Integer Linear Program- ming (MILP) problems. MILP instance I is defined as:
min RE
{c x Ax < b, 10.550 bbls
where æ E R denotes the decision variables, with the first p entries being integer and the remaining p continuous. The vector c 1. denotes the coefficients of the linear objective, the constraints
3

---

are defined by the matrix A E R and the hand side vector b E R",, and the variable bounds are given by 1 E (R u - and u = (R U (+o0])". Without loss of generality, we focus on binary integer variables, and general integer variables can be handled via standard preprocessing techniques [7].
We encode each MILP instance as a bipartite graph 9 = (WUV,,) E),, where W and V denote the sets of constraint and variable nodes, respectively, and the edge set corresponds to non-zero entries in A. Each node and edge is associated with a set of features derived from problem coefficients and structural attributes. Such bipartite graph can completely describe MILP instance, enabling graph neural networks (GNNs) to process the instances and predict their solutions.
We adopt the predict-and-Searde (PS) paradigm to approximate the solution distribution of given MILP. Specifically, the distribution is defined via an energy function that assigns lower energy to high-quality feasible solutions and infinite energy infeasible ones:
exp( Ex I))
c. if a is feasible, +00, otherwise.
p(
I) Ene Exx Z))
where
E(x
I)
(2)
Our goal learning distribution pa (x:[T) for given instance I. To make learning tractable, we as- sume fully factorized solution distribution over the binary variables, i.e., v= [Z) -II-- (m;
where po (I; (Z) denotes the predicted marginal probability for variable Ty. To do so, we use
GNN model to output a p-dimension vector ã fe( = (é),... .p)) =p) E [0, 1J", where
2 (a( 1[Z). To train the model, we use a weighted set of feasible solutions {x(") 11=1
as supervised signals, where each solution is assigned a weight w, x exp(-c" x We let (() (x) 1|2). Then, the training loss function is binary cross-entropy loss defined as:
N
(B) =
log i, + (1 2/)0) log(1 - 2;) 2
i=1
1=1
At inference time, the GNN model outputs a predicted marginals ē E 1]". A standard MILP solver, e.g., Gurobi or SCIP is then used to search for a feasible solution in local neighborhood around à by solving the following trust region problem:
min DE
{er x
Ax < b, I < x
S u. 11:p E B(2 A)},
(4)
where the trust region B( A) == fo ER. r.i.p - l11 S A constrains the solver to remain
close to the predicted binary configuration..
It's evident that the quality of the final solution depends heavily on the accuracy of the predicted marginals. Poor predictions can mislead the solver and limit optimization performance. Therefore, we focus on training a predictive model fe that can accurately predict the marginal probabilities.
3.2 Mixture-of-Expert with Structure-Aware Routing
We consider the setting where training samples are drawn from K different domains (D1,.... D& each corresponding to a structurally distinct MILP family. Our goal is to learn a single unified model that can robustly generalize not only across these diverse domains, but also those domains unseen during training. However, due to the significant combinatorial differences between domains, capturing all structural variations within single monolithic model is challenging.
To address this challenge, we adopt a Sixture-of-Exxerts (MoE) framework that comprises three key components: a shared graph encoder, multiple expert networks, and task decoder. The shared graph encoder learns common structural representations across tasks, while the expert networks capture task task-specific decision patterns under the guidance of dynamic gating mechanism that adaptively assigns expert weights based on task-level embeddings. Finally, the task decoder integrates the aggregated expert outputs to generate the final prediction. This design enables structure-aware representation learning and output specialization.
Shared Graph Encoder Given a MIIP instance I, represented as bipartite graph G. we apply
a graph neural network (GNN) to extract node embeddings (hi,..., h2) c IRd, where is the dimension of the hidden space. We then compute global graph embedding he E Rd by mean pooling over variable nodes, i.e., hg Ei 1 hj. In this work, we follow the GNN encoder design used in [1].

---

MILP Problem
Bipartite Graph
Group-Level Domain Robustness
91
min 2
Domain
Loss
s.L.Az &
Domain
Loss2
92
1 S
# # R
Loss3 9x
Domain
Expert Ouiput a
Expert
Head
Encoder
GNN
Expert
C2
Head
Aggregated Representation
Predicted Logits
Ground- truth
Variable
Expert
Head
Embedding
Gruph Pooling
MSE hg. Kg)
R
L
LACE
Task
hug
Embedding
ax
Expent Output
Perturb
Perturb
Router
Expert
Embedding
Expert Output
Expert
fig
a
Perturb Representation
hg
hg
Expert
1
Figure 2: The overview of RoME. RoME employ a MoE architecture with structure- aware represen- tation learning. For cross-domain training, RoME proposes a robust training object containing binary cross-entropy loss LBCE, expert diversity regularization LiV and fuzzy membership consistency loss CRObusa To further enhance generalization across structurally diverse MILP families, RoME uses group-level domain robust scheme to construct DRO loss.
Multiple Expert Network To capture structural diversity, define set of M expert networks (f() ..... {( ), where each expert (mm) R - Rdd maps the variable embeddings into an
expert-specific representation space:
(m)
(hz).
(5)
Each expert can be interpreted modeling different mode of structural regularity, allowing the model to specialize across domains. To route instances to experts, we use gating network god Rdd + R that maps the graph-level embedding hg into a soft weighting over all experts:
Softmax (g (ha)/T).
where is temperature parameter controlling the sharpness of the routing distribution. The vector a E [0, 1 can be interpreted as softly indicating which experts are most relevant for the current instance. The final representation for each variable is computed by aggregating the expert outputs:
M
f ((m) (hz)
am
M==
Task Decoder To further enhance specialization, we adopt a multi-head decoder that enables model to learn and combine multiple task-specifie representations. Specifically, the decoder set (d(1) , .... {(H) with H decoders, each mapping zj to a scalar logit:
(h)
dg (N) (zz).
(8)
Another gating network which shares a similar architecture but different parameters with the encoder gate- producess a soft weighting vector B E [0, 1] over the decoders:
f Softmax (g dec (hg)/r')
(9)
5

---

where r' is temperature parameter controlling the sharpness of the routing distribution. The final logit is obtained by aggregating the outputs from all decoder heads:
H
Bp
(h)
(10)
h=1
and the predicted marginal is given by
f Sigmoid(a,).
(11)
This flexible MoE design allows the model to dynamically adjust both internal representations and final outputs to match the structural characteristics of each instance. The expert network captures variable-level diversity, while the task decoder adds output-level specialization to further enhance generalization across domains.
3.3 Intra-Domain Robust Training Objective
To enable intra-domain robustness across structurally diverse instances in a domain, we introduce a composite loss function consisting of three components: a binary cross-entropy loss to supervise the training and a regularization term to promote output stability under domain uncertainty. Formally, for given training instance I, the total objective is given by:
[)) LBCE + ÀDiv LDi + ARabust LRobust:
(12)
where 'Di and ARobust are hyperparameters balancing these terms.
Binary Cross-Entropy Loss LBCE. As introduced in Section 3.1 we train the model by minimizing
the weighted binary cross-entropy (3) between the predicted marginals po(n, = 1 I) and weighted
set of feasible solutions sampled from the MILP instance. This loss serves as the primary signal for learning meaningful solution distributions.
Expert Diversity Regularization LDVV To encourage each expert to learn distinct behaviors and avoid mode collapse, regularize the similarity among expert outputs. Recall that zf() denote
the output embedding of expert m for variable j after the expert network layer. Let ((m) denote (m)
the concatenation of the representation of all variables, i.e., (mm) Concat(z(") 1... 2p ((m)). The diversity loss is then computed as:
(m) (mm')
DiV
Σ
(13)
M(M
(mm)]]
mymmm
This term encourages orthogonal expert representations, promoting more diverse perspectives across MILP structures.
Fuzzy Membership Consistency Loss LRobust- To improve the model's robustness to domain uncertainty, we adopt a fuzzy membership view of the expert routing mechanism. The routing vector go (hg) E [0, 1j derived from the task embedding hg. can be interpreted as soft domain membership, indicating the degree to which each expert should be activated. However, for unseen or ambiguous MILP instances, the true domain identity may be uncertain poorly represented in the training data. In such cases, small variations in the task embedding may lead to unstable routing decisions and inconsistent predictions. To address this, encourage the model to maintain output consistency under slight changes in the domain assignment. Concretely, we apply an isotropic perturbation to the task embedding:
hgg hi + r. 10112' ~~~ ~00,0
(14)
where r a learnable scalar controlling the perturbation magnitude. This perturbed task embedding induces a new routing vector = (hg),, which results a different mixture of expert representa- tions žj. We then enforce consistency between the original and perturbed representations using the following loss:
CRobust
Σ |
(15)
P
j=1

---

This regularization encourages the model to produce stable representations even under uncertainty in domain assignment, thus improving its robustness to both domain shift and embedding noise. Beyond its regularization role, this loss is also theoretically motivated from two complementary perspectives. First, from computationallcomplee implying that different MILPs can, in principle, be reduced to one another, a fact confirmed by recent advances in graph-based combinatorial optimization [5] 48). Building on this insight, we exploit structural similarities among domains so that related problems route to overlapping expert subsets, enhancing cross-domain generalization. Perturbing task embeddings partially simulates such cross- domain shifts, enforcing the same expert outputs for perturbed and original embeddings strengthens this alignment and encourages domain- domain-invariam invariant reasoning. Second, from an adversarial -traninng viewpoint, embedding perturbations stabilize the training, helping RoME capture high-level pattemss that persist across domains.
viewpoint, most MILP classes are NP-complete,
3.4 Inter-Domain Group-Level Robustness
To further enhance generalization across structurally diverse MILP families, we introduce a domain- level training strategy based on distributionally robust optimization (DRO). Rather than minimizing a uniform average of per-domain losses, we adopt min-max objective that optimizes performance under the worst-case distribution over training domains. Formally, let Cs denote the expected loss on domain Dr, estimated empirically via mini batch sampling:
(0) == EI-DA [c(O]Z)),
(16)
where [(O is the instance-level training loss defined in Section 3.3 We introduce proba- bility vector q E Ak. where Ax denotes the probability simplex over the K domains, satisfying K Kk=1 qh and gk O. We aim to optimize the following objective:
K min sup Σ Th- ()),
(17)
qEAR k=1
which seeks minimize the worst-case expected loss under all convex combinations of domains- specific losses. To achieve this, we alternatively update 0 and q. Intuitively, we maintain distribution q over groups, with high masses on high-loss groups, and update on each example proportionally to the mass on its group. In each iteration, we first estimate Ls for a randomly chosen domain using a miníbatch. Then, we update the domain weights q using exponential gradient ascent so that 9& x exp(1) Lx), where 17 > O is hyperparameter controlling the sensitivity to high-loss domains. The resulting weights q are then normalized to form a valid probability vector over domains. Therefore, domains with higher empirical losses are thus assigned larger weights, allowing the model to focus learning capacity on harder or underrepresented domains. By coupling this domain-level reweighting with the instance-level regularization objectives introduced in Section 3.3 our training pipeline enables scalable and principled optimization for robust multi-domain MII Pearning.
Experiments
We conduct comprehensive experiments to evaluate the effectiveness of RoME in learning a unified model for solving MILP problems from several different domains, including those in-distribution and out-of-distribution datasets, and zero-shot generalization to real-world instances from MIPLIB. We then conduct experiments to analyze the contributions of different components, as well as the featured patterns of our learned model. All experiments are conducted on a single machine with NVIDIA GeForce RTX 3090 GPUs and 24-core AMD EPYC 7402 CPUs (2.80 GHz), Code is available at in Appendix A
http://aw..cmm/aaappppps//////ee More implementation details can be found
4.1 Experimental Setup
Datasets We conduct experiments on five MILP problem families widely studied in the literature: Independent Set (IS), Set Covering (SC), Item Placement (IP), Combinatorial Auctions (CA), and Workload Appointment (WA). The instances in the dataset have diverse kinds of combinatorial structures and constraints, with difffernt levels of constraint coefficient sparsity, variable numbers, and objective complexity. For IS SC and CA, we follow the conventional instances generation process in existing works [2 19 The and WA datasets are from NeurIPS 2021 ML4CO competition 49 Please see Appendix for more detailed information on the benchmarks.
7

---

Table Performance comparison across various MILP domains, under a 1, 000s time limit. We train RoME on IS, IP and SC, while the performance of RoME on WA and CA is zero-shot performance. "p" indicates that higher is better, and 'L" indicates that lower is better. We mark the best values in bold. We also report the improvement of our method over the traditional solvers in terms of gaPabs
IS (BKS 685.00)
IP (BKS 11.18
SC (BKS 126.64)
WA (BKS 70305)
CA (BKS 97524.37)
Obj
Time
Obj
gaPabs
Obj
gapats
Obj 4
gapats
Obj
gapabs
Gurobi
685.00
$7.35
11.43
0.27
125.21
0.57
703.47
0.42
97308.93 93
215. 45.44
PS+Gurobi
685.00
14.67
11,40
0.24
125.17
0.53
703.47
0.42
97280.70
243.67
ConPS+Gurobi
685.00
20.13
11.36
0.20
125.18
0.54
703.47
0.42
97395.83
128.54
RoME+ Gurobi
685.00
2.08
11.22
0.06
124.69
0.05
703.11
0.06
97489.28
35,09
Improvement
27.5 x
77.8%
91.2%
85.7%
83.7%
0.10
0.230
WA
0:003
CA
Ave Panu
boees
eie
0.913
00001
010
0.9
107
11
Tixck
Cureti
CoapS
Gard
Figure 3: The average primal gap of different methods over 100 instances as the solving process proceeds. We use Gurobi for implementation and set the time limit to be 1,000 seconds.
Baselines We mainly consider two baselines, Predict-and-Search (PS) 18] and Contrastive Predict- and-Search (ConPS) [9]]. Contrastive Predict-and-Searde (ConPS) is stronger baseline, leveraging contrastive learning to enhance the performance of PS. For ConPS, we set the ratio of positive to negative samples ten, using low-quality solutions as negative samples. Our method and baselines can be integrated with conventional solvers such as SCIP [9] and Gurobi [8). Therefore, we also include SCIP and Gurobi baselines for a comprehensive comparison. Following Han al. [8] Gurobi and SCIP are set to focus on finding better primal solutions.
Metrics We compare the performance of our method and baselines using the best objective value OBJ within 1,000 seconds. Following the setting in Han et al. [88, we run a single-thread Gurobi for 3,600 seconds and record the best objective value as the best-known solution (BKS) to approximate the optimal value. We calculate the absolute primal gap as the difference between the best objective found by the solvers and the BKS, i..., gapate [BJ- BKS|. Within the same solving time, lower absolute primal gap indicates stronger performance. For IS, where most test cases can be solved within the 1,000s, additionally report the time cost to obtaín the optimal value, with lower time cost indicating better performance.
Training and Evaluation Throughout all the performance experiments, we use one unified model for ROME. To train this model, we use a training dataset composed of 720 instances, where the dataset contains 240 IS instances, 240 IP instances and 240 SC instances. The validation set contains 240 instances with 80 IS, TP and SC instances, respectively. The performance of RoME on WA, CA and MIPLIB is the performance of zero-shot generalization. The baselines are trained individually on each dataset using 240 instances for training and 80 instances for validation. For evaluation, we run all the methods on 100 testing instances for each dataset.
4.2 Main Results Cross-Domain Generalization
We first evaluate the effectiveness of RoME trained on multiple domains on various test domains. In this experiment, RoME is trained on IS, IP. and SC. We train just one unified model on these three datasets, and then test it on all five domains, where WA and CA serve as unseen zero-shot testing datasets. The baseline methods are trained on each domain individually and evaluated on the corresponding testing datasets. Table [ presents the average objective values and gapabs achieved by each method using the Gurobi solver across all five combinatorial problem families. Corresponding results for SCIP are provided in Appendix D.2 For the IS dataset, we also report the average solver time in seconds, as all the methods can find the optimal solution within the time limit. The results in Table I show that RoME consistently outperforms all baselines across domains. (1) For IS, IP and
8

---

SC, RoME is able to achieve the best objective values and absolute primal gap than the baselines trained on the corresponding dataset. This implies that cross-domain training can benefit from the performance of each single domain. (2) Notably, even though not trained on WA and CA, ROME achieves the best performance on these unseen domains, demonstrating its zero-shot generalization capability.
We also report the curves that the average primal gap (defined as gap := |OBJ BKS| [BKS)
changes as the solving process proceeds in Figure 3 A rapid decrease in the curves indicates superior solving performance and better convergence. The results in Figure 3 show that RoME exhibits rapid decline in primal gap and ultimately achieves the lowest primal gap.
Additionally, conduct comprehensive ablation studies investigate the critical components of ROME, evaluating the effects key elements including DRO, the MoE architecture, embedding perturbation strategies, and expert diversity. Detailed analyses can be found in Appendix DD.5
4.3 Zero-Shot Generalization to MIPLIB Problems
To further evaluate the performance RoME in the real-world complex dataset, we evaluate it on a curated subset of the MIPLIB benchmark [10]. The MIPLIB dataset is a challenging MILP dataset, where the instances are drawn from diverse industrial applications with heterogeneity modeling structures. In this experiment, the baselines are trained and evaluated splits within MIPLIB. In contrast, RoME is trained on synthetic datasets as used in the main evaluation and directly applied to MIPLIB in zero-shot fashion. This setting simulates practical scenario where the solver is deployed in unseen domains without any retraining.
We select two groups of MIPLIB testing instances following the selection criterion in Appendix C.3 The first subset follows Liu al. [32] and uses the IIS dataset selected from MILPLIB. IIS contains six instances for baselines and five testing instances, where we draw the solving curve in Figure 4 And another subset contains fifteen challenging MIPLIB instances in total, with Figure showcassing solving curves for representative subset of five selected instances. We find RoME consistently outperforms all the baselines across all the instances, highlighting its strong generalization ability on real-world complex instances. Average results of objective values and gapabs across both two selected MIPLIB datasets are presented in Appendix D. with each instance result also provided in the same appendix.
0.2
gl
02
fos0007
Acp Serip
Avrage Wenk Oap
300
0.0
117 Thnok)
Gumbs
P Gardsl
Con'5 à Surthl
RoPE
CUroSI
Figure 4: The primal gap of different methods on five easier instances from IIS dataset as the solving process proceeds. We use Gurobi for implementation and set the time limit to be 1,000 seconds.
0U004
heh2
3.05
00004
00000
60
00000
$000
0:00
Tineto
Tiness
TiaatO
Taxto
Gurbi
P5 Durstl
Cant5 Curchi
1:PE Sumo
Figure 5: The primal gap of different methods on five harder instances from MIPLIB the solving process proceeds. We use Gurobi for implementation and set the time limit to 1,000s.
4.4 Interpretability of RoME
Current works typically adopt GNN-based models, which are inherently treated as black-boxes. However, with the recent progress in large language models (20.52), an increasing number of inter- pretability techniques have emerged, particularly for MoE architectures, offering new opportunities

---

to better understand model behavior. Following this, to gain deeper understanding of RoME, we visualise its expert routing and embedding behaviour. Using the model trained IS, IP and SC following the main evaluation. we record which expert the gate selects when it solves instances from the five benchmark domains and from five selected challenging instances from MIPLIB, as summarised in Fig (a). We then obtain the task vectors of these instance and plot their distribution through T-SNE, as shown in 6 (b). These interpretability analyses reveal that RoME develops clear and structured specialization among its experts. Each problem with unique constraint type consistently activates a distinct expert, demonstrating that the mixture model avoids collapse and that individual experts capture unique structural regularities across tasks. The alignment between task embeddings and expert activations shows that RoME learns coherent, task-aware representations: tasks with similar underlying formulations are embedded closely in the latent space and routed to overlapping expert subsets. Furthermore, mixed-constraint MIPLIB instances are positioned near and assigned the domains they most resemble, explaining RoME's strong generalization and zero-shot adaptability to previously unseen problem families.
Actinaiinn Pulknn of Evaat Nevells
Shil Projection of Iiaheddings
WA
Expen
CA
IS
Drpon2
Apper
now 185
now Ma36
Figure 6: Model interpretability. (a) Expert network activation patterns across different MILP instances. Red indicates activated experts, while blue denotes inactive ones. (b) The distribution of instances in the task embedding space.
Conclusion
In this paper, we present RoME, a cross-domaie MILP solver that pairs a mixture-of-experts architec- ture with distributionally robust training objective. The mixture-of-experts architecture enhance representation ability across different domains, while distributionally robust optimization shields against distribution shifts, and together they boost generalization across MILP problem families. Experiments demonstrate substantial improvements over existing learning-base based baselines and strong zero-shot performance on challenging MIPLIB instances.
Acknowledgments and Disclosure of Funding
The authors would like to thank all the anonymous reviewers for their valuable suggestions. This work was supported by the National Natural Science Foundation of China (NSFC, 7257 1284, 72421002, 62206303, 62273352), the Human Provincial Fund for Distinguished Young Scholars (20251J20073), the Science and Technology Innovation Program of Hunan Province (2023R33009), the Foundation Fund Program of National University of Defense Technology (JS24-05), the Major Science and Technology Projects in Changsha, China (kq2301008), the Key Research and Development Program of Hunan Province (2025JK2143), and the Hunan Provincial Innovation Foundation for Postgraduate (XJQY2025044).
10

---

References
[1] Reinhard Baller, Pirmin Fontaine, Stefan Minner, and Zhen Lai. Optimizing automotive inbound logistics: A mixed-integer linear programming approach. Transportation Research Part E: Logistics and Transportation Review, 163: 2022. ISSN 1366-5545, doi: https:
//bbi.org/00.1016/j.....222 102734. URL
comp:///ceeeeeeeeceo/ooooo/eeeee sttp//sceence,.
att.ccee/pi1/$1116645222000
[2]
Mahdi Sharifzadeh, Marti Cortada Garcia, and Nilay Shah. Supply chain network design and op- eration: Systematic decision-make making for centralized, distributed, and mobile biofuel production using mixed integer linear programming (milp) under uncertainty. Biomass and Bioenergy, 81: 401-414, 2015. ISSN 0961-9534 doi: https::/doi.org//In bioooooo .026. URL
htttp:///ww. sci scceeneedieett.. htt.c///ciiieeeearrrraee////////
[3] David M Ryan and Brian A Foster. An integer programming approach to scheduling. Computer scheduling of public transport urban passenger vehicle and crew scheduling, pages 269-280, 1981.
[4] Belil, S. Kemmoé- Tchomméé and N. Tchernev. Milp-based approach to mid-term production
planning of batch manufacturing environment producing bulk products. IFAC- PPAC-PuppssOOLLine (1(11); 1889-1669,, 2018. ISSN 2405-8963, doi: https://doi.org//00 http://///....oo.////// 08 213. 831334X
URL buttps://waxe sci encedireett.. com /tc//iiennii
16th IFAC Symposium on Information Control Problems in Manufacturing INCOM 2018.
[5] Tianle Pu, Chao Chen, Li Zeng, Shixuan Liu, Rui Sun, and Changjun Fan. Solving combinatorial optimization problem over graph through qubo transformation and deep reinforcement learning. In 2024 IEEE International Conference on Data Mining (ICDM), pages 390-399. IEEE, 2024.
[6] Tianle Pu, Changjun Fan, Mutian Shen, Yizhou Lu, Li Zeng. Zohar Nussinov, Chao Chen, and Zhong Liu. Exploratory combinatorial optimization problem solving via gauge transformation. In 2024 IEEE International Conference Data Mining (ICDM), pages 821-826. IEEE, 2024.
[7] Haopeng Duan, Kaiming Xiao, Lihua Liu, Haiwen Chen, and Hongbin Huang. Reinforce with bound-guided gradient estimator for the traveling salesman problem toward scale generalization. Engineering Applications of Artificial Intelligence, page 109819, 2025.
[8] LLC Gurobi Optimization. Gurobi optimizer. URL http://www. gurobi. com, 2021.
[9] Tobias Achterberg. Scip: solving constraint integer programs. Mathematical Programming Computation, 1:1-41, 2009.
[10] Ambros Gleixner, Gregor Hendel, Gerald Gamrath, Tobias Achterberg. Michael Bastubbe, Timo Berthold, Philipp Christophel, Kati Jarck, Thorsten Koch, Jeff Linderoth, et al. Miplib 2017: data-driven compilation of the 6th mixed-integer programming library. Mathernatical Programming Computation, 13(3):443-490 2021.
[11] Maxime Gasse, Didier Chételat, Nicola Ferroni, Laurent Charlin, and Andrea Lodi. Exact combinatorial optimization with graph convolutional neural networks. Advances in neural information processing systems, 32, 2019.
[12] Zhihai Wang, Xijun Li. Jie Wang, Yufei Kuang, Mingxuan Yuan, Jia Zeng, Yongdong Zhang, and Feng Wu. Learning cut selection for mixed-integer linear programming via hierarchical sequence model. In The Eleventh International Conference on Learning Representations, 2023.
[13] Yufei Kuang, Jie Wang, Haoyang Liu, Fangzhou Zhu, Xijun Li, Jia Zeng. HAO Jianye, Bin Li, and Feng Wu. Rethinking branching on exact combinatorial optimization solver: The first deep symbolic discovery framework. In The Twelfth International Conference on Learning Representations, 2024.
[14] Hongyu Liu, Haoyang Liu, Yufei Kuang, Jie Wang, and Bin Li. Deep symbolic optimization for combinatorial optimization: Accelerating node selection by discovering potential heuristics. In Proceedings of the Genetic and Evolutionary Computation Conference Companion, pages 2067-2075, 2024.
11

---

[15] Nicolas Sonnerat, Pengming Wang, Ira Ktena, Sergey Bartunov, and Vinod Nair. Learning large neighborhood search algorithm for mixed integer programs. ArXiv, abs/2107.10201. 2021.
[16] Taoan Huang, Aaron Ferber, Yuandong Tian, Bistra N. Dilkina, and Benoit Steiner. Searching large neighborhoods for integer linear programs with contrastive learning. In International Conference on Machine Learning, 2023.
[17] Vinod Nair, Sergey Bartunov, Felix Gimeno, Ingrid Von Glehn, Pawel Lichocki, Ivan Lobov, Brendan O' o'Donoghue, Nicolas Sonnerat, Christian Tjandraatmadja, Pengming Wang, et al. Solving mixed integer programs using neural networks. arXiv preprint arXiv:2012.133493 2020.
[18] Qingyu Han, Linxin Yang, Qian Chen, Xiang Zhou, Dong Zhang, Akang Wang, Ruoyu Sun, and Xiaodong Luo. A gnn-guided predict-and-search framework for mixed-integer linear programming. In The Eleventh International Conference on Learning Representations, 2023.
[19] Taoan Huang, Aaron M Ferber, Arman Zharmagambetov, Yuandong Tian, and Bistra Dilkina. Contrastive predict-and-sentt for mixed integer linear programs. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors, Proceedings of the 41st International Conference on Machine Learming, volume 235 of Proceedings of Machine Learning Research, pages 19757-19771. PMLR, 21-27 Jul 2024. mlr. prwss/ /vaess//22355 hw12223/huuggg2444 html
URL httpp:://prcceeeeee http:///ww.mlll.ees//e/es////
[20] Elias B Khalil, Christopher Morris, and Andrea Lodi. Mip-gnn: A data-driven framework for guiding combinatorial solvers. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 10219-10227, 2022.
[21] Zijie Geng, Xijun Li, Jie Wang, Xiao Li, Yongdong Zhang, and Feng Wu. A deep instance generative framework for milp solvers under limited data availability. In Advances in Neural Information Processing Systems, 2023.
[22] Haoyang Liu, Yufei Kuang, Jie Wang, Xijun Li, Yongdong Zhang, and Feng Wu. Promoting generalization for exact solvers via adversarial instance augmentation, 2023. URL https
//arxiv.. www.g//aaa////aa 14161
[23] Haoyang Liu, Jie Wang, Wanbo Zhang, Zijie Geng, Yufei Kuang, Xijun Li, Bin Li, Yongdong Zhang, and Feng Wu. MILP--stdiie MILP instance generation via block structure decomposi- tion. In The eiirtt- Annual Conference on Neural Information Processing Systems, 2024.
[RL.htttp://ooeereeviee hta.tp/:/gppaaeeeeeeeeeeee//aaaa .ww.://orrum..d-4333310000
[24] Qingyu Qu. Xijun Li, Yunfan Zhou, Jia Zeng, Mingxuan Yuan, Jie Wang, Jinhu Lv, Kexin Liu, and Kun Mao. An improved reinforcement learning algorithm for learning to branch. arXiv preprint arXiv:2201.062/1 2022.
[25] He He, Hal Daume III, and Jason M Eisner. Learning to search in branch and bound algorithms. Advances in neural information processing systems, 27, 2014.
[26] Abdel Ghani Labassi, Didier Chetelat, and Andrea Lodi. Learning to compare nodes in branch and bound with graph neural networks. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Infor- mation Processing Systems, volume 35, pages 32000-333 32010. Curran Associates, Inc.. 2022. URL https: https::/pproceedingg http://roccediin....eeeurrr.pppp neurips. cc/ httprrippssppppppeeeeeseee/eeeee wtt.ff///paaerr/22//iii..
htto://877Aeeccaaaaeeeeeee1111/0 Coneeeeee pdf
[27] Sijia Zhang, Shuli Zeng, Shaoang Li, Feng Wu, and Xiangyang Li. Learning select nodes in branch and bound with sufficient tree representation. In The Thirteenth International Conference on Learning Representations, 2025. httttps://opereeieee id= id=gyvYKLEmst
[28] Sirui Li, Wenbin Ouyang, Max B Paulus, and Cathy Wu. Learning to configure separators in branch-and-cut. In sevenhh Conference Neural Information Processing Systems, 2023,
12

---

[29] Jialin Song, ravi lanka, Yisong Yue, and Bistra Dilkina. A general large neigh- borhood search framework for solving integer linear programs. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Infor- mation Processing Systems, volume 33, pages 20012-20023. Curran Associates, Inc., 2020. URL https: pttppe://proceeedingg iew.reeeddddnnnn.nnee...e...p.p cc/ http/pppr__fiiie www.fffllesspppeeeee httee////ppr/222///22eee 27990055tt22006666b0001141444444
saaperpptt
[30] Yaoxin Wu, Wen Song, Zhiguang Cao, and Jie Zhang, Learning large neigh- borhood search policy for integer programming. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Infor- mation Processing Systems, volume 34, pages 30075-30083
Curran Associates, Inc.. wwwpccpppppp.ppeeee hatppppe_ffilee. filesfiiees//ee wwt.fffflesseeee/eeeee patls///ppprr/2///fiieee
2021. URL httpss://proceeddnii neurips. cc.
.df
[31] Huigen Ye, Hua Xu, Hongyan Wang, Chengming Wang, and Jiang, GNN&GBDT-guide fast optimizing framework for large-scale integer programming. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 19864-39878 PMLR, 23-29 Jul 2023. URI. https //proceedings.co mlr. www.semw......m.e.ee..memmmmeeee sitess/v02//yeee html
[32] Haoyang Liu, Jie Wang, Zijie Geng, Xijun Li, Yuxuan Zong, Fangzhou Zhu, Jianye HAO, and Feng Wu. Apollo-MILP: An alternating prediction prediction-corrrctionn neural solving framework for mixed-integer linear programming. In The Thirteenth International Conference on Learning Representations, 2025. URL hattps://openreviiee net/ htt/:/oormmddddddmm PO
[33] Zijie Geng, Jie Wang, Xijun Li, Fangzhou Zhu, Jianye HAO, Bin Li, and Feng Wu. Differen- tiable integer linear programming. In The Thirteenth International Conference on Learning Representations, 2025. URL httpps://oeeneeviez http::///ppppeeeeeeeeeeeoooooooo
[34] Saeed Masoudnia and Reza Ebrahimpour. Mixture of experts: literature survey. Artificial Intelligence Review, 42:275-299 2014.
[35] Rui Gao, Xi Chen, and Anton J Kleywegt. Wasserstein distributionally robust optimization and variation regularization. Operations Research, 7273):1177-1191, 2024.
[36] Daniel Kuhn, Peyman Mohajerin Esfahani, Viet Anh Nguyen, and Soroosh Shafieezadeh- Abadeh. Wasserstein distributionally robust optimization: Theory and applications machine learning. In Operations research & management science in the age of analytics, pages 33-1666. Informs, 2019.
[37] Jiang Guo, Darsh J Shah, and Regina Barzilay. Multi- domain adaptation with mixture of experts. arXiv preprint arXiv: (a::::009000000 2018
[38] William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research, 23 (120):1-39, 2022.
[39] Bo Li, Yifei Shen, Jingkang Yang, Yezhen Wang, Jiawei Ren, Tong Che, Jun Zhang, and Ziwei Liu. Sparse mixture-offexpprts are domain generalizable learners. arXiv preprint wrXiv22066.44046 2022.
[40] Jianan Zhou, Zhiguang Cao, Yaoxin Wu, Wen Song. Yining Ma, Jie Zhang, and Chi Xu. Mvmoe: Multi-task vehicle routing solver with mixture-of-exppert, 2024. URL htt.tttt///aaxtttit org/ has/2405.0100
[41] Fei Liu, Xi Lin. Zhenkun Wang, Qingfu Zhang, Xialiang Tong, and Mingxuan Yuan. Multi- task learning for routing problem with cross-problem zero-shot generalization, 2024. URL
https://arxiv org/ absp//02.... 16891
[42] Yang Li, Jiaxi Liu, Qitian Wu, Xiaohan Qin, and Junchi Yan. Toward learning generalized cross-problem solving strategies for combinatorial optimization, 2025.
13

---

[43] Darko Drakulic, Sofia Michel, and Jean-Mare Andreoli. GOAL: A generalist combinatorial opti- mization agent learner. In The Thirteenth International Conference on Learning Representations, 2025.
[44] Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint 1xXiv:191.0888
2019.
[45] Jingge Wang, Yang Li, Liyan Xie, and Yao Xie. wasserstein distributional robust optimization. arXiv preprint urXiv22109036670 2021.
domain generalization via
[46] Rui Dai, Yonggang Zhang, Zhen Fang, Bo Han, and Xinmei Tian. Moderately distributional exploration for domain generalization. arXiv preprint arXiv:2304 2023.
[47] Lijun Zhang, Peng Zhao, Zhen- Zhen-Hua Zhuang. Tianbao Yang, and Zhi-Hua Zhou, Stochastic approximation approaches to group distributionally robust optimization. Advances in Newral Information Processing Systems, 36:52490-525555 2023.
[48] Wenzheng Pan, Hao Xiong, Jiale Ma, Wentao Zhao, Yang Li, and Junchi Yan. Unico: On unified combinatorial optimization via problem reduction to matrix-encoded general tsp. In The Thirteenth International Conference on Learning Representations, 2025.
[49] Maxime Gasse, Simon Bowly, Quentin Cappart, Jonas Charfreitag, Laurent Charlin, Didier Chételat, Antonia Chmiela, Justin Dumouchelle, Ambros Gleixner, Aleksandr M. Kazachkov, Elias Khalil, Pawel Lichocki, Andrea Lodi, Miles Lubin, Chris J. Maddison, Morris Christopher, Dimitri J. Papagcorgiou. Augustin Parjadis, Sebastian Pokutta, Antoine Prouvost, Lara Scavuzzo, Giulia Zarpellon, Linxin Yang, Sha Lai, Akang Wang, Xiaodong Luo. Xiang Zhou, Haohan Huang, Shengeheng Shao, Yuanming Zhu, Dong Zhang, Tao Quan, Zixuan Cao, Yang Xu, Zhewei Huang, Shuchang Zhou, Chen Binbin, He Minggui, Hao Hao, Zhang Zhiyu, An Zhiwu, and Mao Kun. The machine learning for combinatorial optimization competition (ml4co): Results and insights. In Douwe Kiela, Marco Ciccone, and Barbara Caputo, editors, Proceedings of the NeurlPS 2021 Competitions and Demonstrations Track, volume 176 of Proceedings of Machine Learning Research, pages 220-231. PMLR, 06-14 Dec 2022. URL https:// proceedings mlr. press/ wwwppsewese../e//e//ee/pp gasse22a html
[50] Jincai Huang, Yongjun Xu, Qi Wang, Qi Cheems Wang, Xingxing Liang, Fei Wang, Zhao Zhang, Wei Wei, Boxuan Zhang, Libo Huang, et al. Foundation models and intelligent decision decision-making: Progress, challenges, and perspectives. The Innovation, 2025.
[51] Rui Wang, Shixuan Liu, Changjun Fan, Guozheng Li, Jincai Huang, Zhong Liu, and Gang Zhou. Towards bridging the gap between machine intelligence and machine wisdom: dilemmas and conjectures. The Innovation, page 100834, 2025.
[52] Liu Haoyang, Wang Jie, Cai Yuyang, Han Xiongwei, Kuang Yufei, and Hao Jianye. Optitree: Hierarchical thoughts generation with tree search for LLM optimization modeling. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025.
[53] Changjun Fan, Mutian Shen, Zohar Nussinov, Zhong Liu, Yizhou Sun, and Yang- Yu Liu. Search- ing for spin glass ground states through deep reinforcement learning. Nature communications, 14(1):725, 2023.
[54] Changjun Fan, Li Zeng, Yuhui Ding, Muhao Chen, Yizhou Sun, and Zhong Liu. Learning to identify high betweenness centrality nodes from scratch: A novel graph neural network approach. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pages 559-568, 2019.
[55] Changjun Fan, Li Zeng, Yizhou Sun, and Yang- Yu Liu. Finding key players in complex networks through deep reinforcement learning. Nature machine intelligence, 266)331 2020.
[56] Shixuan Liu, Changjun Fan, Kewei Cheng, Yunfei Wang, Peng Cui, Yizhou Sun, and Zhong Liu. Inductive meta-path learning for schema-complex heterogeneous information networks.
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
14

---

[57] Cunchao Zhu, Muhao Chen, Changjun Fan, Guangquan Cheng, and Yan Zhang. Learning from history: Modeling temporal knowledge graphs with sequential copy-generation networks, In Proceedings of the AAAI conference artificial intelligence, volume 35, pages 4732-4740, 2021.
[58] Hong Wang, Jie Wang, Minghao Ma, Haoran Shao, and Haoyang Liu. Symmap: Improving computational efficiency in linear solvers through symbolic preconditioning In The Thirty- ninth Anmual Conference on Neural Information Processing Systems, 2025. URL https /'opnrevvew.net//ooeee wttp:/eoooonmn/tonneeerrrr
[59] Hong Wang, Haoyang Liu, Jian Luo, Jie Wang, al. Accelerating pde data generation via differential operator action in solution space. In Forty-first International Conference on Machine Learning.
[60] Shixuan Liu, Yue He, Haotian Wang, Wenjing Yang. Yunfei Wang, Peng Cui, and Zhong Liu. Environment inference for learning generalizable dynamical systems. In Advances in Neural Information Processing Systems, volume 39, 2025.
[61] Chang Liu, Yunfan Ye, Fan Zhang, Qingyang Zhou, Yuchuan Luo, and Zhiping Cai. Humansam: Classifying human-centric forgery videos in human spatial, appearance, and motion anomaly, arXiv preprint ArXiv 19924, 2025.
[62] Li Liu, Songyang Lao, Paul W. Fieguth, Yulan Guo, Xiaogang Wang, and Matti Pietikäinen. Median robust extended local binary pattern for texture classification. IEEE Transactions Image Processing, 25(3): 136--181,, 2016.
[63] Li Liu, Shuzhou Sun, Shuaifeng Zhi, Fan Shi, Zhen Liu, Janne Heikkilä, and Yongxiang Liu. A causal adjustment module for debiasing scene graph generation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(5):4024 4043, 2025.
[64] Weijie Li, Wei Yang, Yuenan Hou, Li Liu, Yongxiang Liu, and Xiang Li. Saratr-x: Toward building foundation model for sar target recognition. IEEE Transactions on Image Processing, ((()::86-884, 2025. doi:
[65] Bingxu Zhang, Changjun Fan, Shixuan Liu, Kuihua Huang. Xiang Zhao, Jincai Huang, and Zhong Liu. The expressive power graph neural networks: survey. IEEE Transactions on Knowledge & Data Engineering, 101):1-20, 2024.
[66] Kevin Leyton-Brown, Mark Pearson, and Yoav Shoham. Towards universal test suite for combinatorial auction algorithms. In Proceedings the Commerce, pages 66-76, 2000.
the 2nd ACM Conference on Electronic
[67] Egon Balas and Andrew Ho. Set covering algorithms using cutting planes, heuristics, and subgralient optimization: compulational study. Springer, 1980.
[68] Maxime Gasse, Simon Bowly, Quentin Cappart, Jonas Charfreitag. Laurent Charlin, Didier Chételat, Antonia Chmiela, Justin Dumouchelle, Ambros Gleixner, Aleksandr Kazachkov, al. The machine learning for combinatorial optimization competition (ml4co): Results and insights. In NeurIPS 2021 competitions and demonstrations track, pages 2.1-231. PMLR, 2022.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims can reflect the paper's contribution and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Please see Appendix E.1
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
16

---

Justification:
Section 3.3 and 3.4 introduce our DRO method in detail.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We provide the implementation details in Appendix A
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
17

---

Answer:
[No]
Justification:
Code is available at
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https Cooeeeoodeesssseeeeee 7/nips.com /ttc://lllllliiiiee//e/eeeaaaass swiniioiicypyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: We specify the training and test details in Section 4.1 the hyperparameters in Appendix and benchmark details in Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification:
We analyze the experiment results Section
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
18

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We report the machine we used in Appendix A.3
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question:
Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http:////aauuoscccpppu/ac/cccaaa
Answer:[Yes [Ys]
Justification:
We obey the NeurlPS Code of Ethics in the research.
Guidelines:
The answer NA means that the authors have reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss in Appendix E.3
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
19

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer:[Yes)
Justification:
We cite the dataset and code used in Section 4.1 and Appendix A.3
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
20

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
21

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA
that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
22

---

A Implementation Details
A.1 Bipartite Graph Representation
Following previous works [18 32]. we represent each MILP instance as a weighted bipartite graph = (V (VU U/W,& E), where and W denote the sets of variables and constraints, respectively. The
features attached to the bipartite graph are similar to the previous works 18 32 description of these features can be found in Table 2
The detailed
Table 2: Description of the variable, Constraint and edge features used our bipartite graph representation.
Index
Variable Feature Name
Description
0
Objective Variable coefficient Variable degree
Normalized objective coefficient
Average variable coefficient all constraints
2
Degree of the variable node in the bipartite graph representation
3
Maximum variable coefficient
Maximum variable coefficient in all constraints Minimum variable coefficient in all constraints Whether the variable is integer variable or not)
4
Minimum variable coefficient
5
Variable type
Index
Constraint Feature Name
Description
Constraint coefficient Constraint degree
Average of all coefficients in the constraint
Degree of constraint nodes
Bias
Normalized right-hand-side of the constraint
3
Sense
The sense of the constraint
Index
Constraint Feature Name
Description
Coefficient
Constraint coefficient
A.2
Implementation Details of the GNN Encoder
In this work, we use a graph neural network (GNN) parameterized by 0 to extract diverse structural representations for MILP instances. For given instance I, we initialize features for variables E IR5, constraints wy E RR. and edges eij R then embed them through the MLP layers:
h ( MLPo(1:) A = MLPo(w;). her
MLPo(egj)
(18)
Subsequently, we apply two graph convolution layers, each consisting of two interleaved half- convolution operations II]:
h (++1) + MLP wy
Σ
MLP of (h
(k), Berg (1(1))
jeeiEEE
(19)
([+1) <- MLP
(b)
Σ
MLP (k++)) Bess h())))
ite 33
Unlike the previous works that employ extra MLPs for logit prediction, GNN encoder directly propagates the refined node embeddings to the following modules.
A.3
Implementation Details of the Baselines
Optimization has broad applications in the real world, including statistical physics 53
complex
networks [4 55 knowledge graph [56. 57]. and partial differential equations (58-60 Recent
breakthroughs in computer vision 6 64 graph models (65), and reinforcement learning have
23

---

propelled the adoption of machine learning for faster combinatorial optimization solving. In our work, we select two state-of the methods, PS [8] and ConPS [9]. as baseline approaches, each exemplifying a prominent direction in recent research, For PS, we utilize the official implementation available at/https: //g: thub. www.so/sioooooi. www.m///rrbdnn/eeeeiii...o
-nd--aaddddeeee and we incorporate the same graph neural network (GNN) encoder from PS into our model architecture. Since ConPS does not have publicly available source code, we meticulously reimplemented its framework based on the original descriptions and performed extensive hyperparameter tuning to replicate its reported performance. All experiments are conducted on a single machine equipped with NVIDIA GeForce RTX 3090 GPUs and AMD EPYC 7402 24-core CPUs running 2.80GHz. We use Gurobi version 11.0.3 and SCIP version 8.1.0 in all experiments.
For each baseline, we collect 300 instances to serve as training and validation data. Each instance solved using Gurobi with single thread for 3600 seconds, during which we record the best 50 solutions. We split the data into 80% for training and 20% for validation, and designate an additional 100 instances as the test set. During training, we set the initial learning rate to 0.0005 and train the model for 10,000 epochs, employing an early stopping mechanism to prevent overfitting. In the testing phase, the parameters (Ko, k1, A) determine which variables are fixed to 0 or I and define the scope of the subsequent search process. These parameters significantly influence the performance of the methods. These parameters used in this work are detailed in Table
Table 3: The partial solution size parameter (ko, k1) and neighborhood parameter A.
Benchmark
IS
CA
SC
IP
WA
PS+Gurobi
(300, 300, 20)
(600,0.1)
(2000,0,10)
(400,5,10)
(0,500,10)
ConPs+
(1200, 600, 10)
(900,0,50)
(1000,0,200)
(400,5.3)
(0,500,10)
RoME+Gurobi
(250, 200, 15)
(350, 0, 55)
(1000, 0, 200)
35, 55)
(20, 200, 100)
PS+SCIP
(300,300, 15)
(400,0,10)
(2000.0,
(400,5,1)
(0,600,5)
ConPS+SCIP
(1200, 600, 10)
(900,0,50)
(1000,0,200)
(400,5,3)
(0,400,50)
RoME+SCIP
(250, 200, 15)
(350, O, 55)
(1000, 0, 200)
(60, 35, 55)
(20, 200, 100)
B Hyperparameters
The key training parameters are summarized in Table
Table 4: Hyperparameters used in our experiments.
Name
Value
Description
embed_size
128
The embedding size of the GNN encoder. Number of experts.
num_ _um_experts
3
num heads
Number of heads.
batch_ _ize
4
Number of MILP instances in each training batch.
num _en_ee
2000
Number of max running epochs.
Ir
0.0005
Learning rate for training.
perturbation ratio
0.1
Perturbation rate on the task embedding space.
sipert_diversity_rati
0.2
Expert diversity ratio for training.
C Details on the Benchmarks
C.1 Benchmarks in Main Evaluation
The CA, SC, and IS benchmark instances are generated following the procedure outlined in [ Specifically, the CA instances are created using the algorithm described [66), while the SC instances are produced based on the method presented in [67] The IP and WA instances are sourced from the
NeurIPS ACACO 2021 competition [68]. Table provides detailed statistical information for all the instances.
24

---

Table 5: Statistical information of the benchmarks we used in this paper.
CA
SC
IP
WA
IS
Constraint Number
2593
3000
195
64306
5943
Variable Number
1500
5000
1083
61000
1500
Number of Binary Variables
1500
5000
1050
1000
1500
Number of Continuous Variables
0
33
60000
0
Number of Integer Variables
0
0
C.2 Benchmarks used for Generalization
To evaluate the generalization capabilities of our methods, we construct larger CA and SC instances using the data generation code from Gasse al. [1]. The newly generated CA instances contain approximately 2,596 constraints and 4,000 variables on average, while the SC instances comprise 6,000 constraints and 10,000 variables. These problem sizes substantially larger than those encountered during training, providing a more rigorous evaluation of the models' performance on unseen, large-scale instances.
C.3 Subset of MIPLIB
Table 6: Statistical information of the instances in the constructed MIPLIB dataset.
Instance Name
Constraint Number
Variable Number
Nonzero Coefficient Number
ex1010-pi
1468
25200
102114
fast0507
507
63009
409349
ramos3
2187
2187
32805
scpj4scip
1000
99947
999893
scpk4
2000
100000
1000000
scpl4
2000
200000
2000000
dws008-03
16344
32280
165168
dws008-01
6064
11096
56400
neos2
1103
2101
7326
bab2
17245
147912
2027726
bab5
4964
21600
155520
bab6
29904
114240
1283181
neos-35559044turame
146493
37461
793605
neos-3656078-kume
17656
14870
59292
supportcase17
2108
1381
5253
fastxgemm-n22774411
6972
904
22584
neos-3754480-nidd:
203
253
1488
bg512142
1307
792
3953
tr12-30
750
1080
2508
s100
14733
364417
1777917
s55
9892
78141
317902
To evaluate the solvers' performance on challenging real-world instances, we construct two groups of MIPLIB [10] instances. The selection is based on instance similarity, measured using 100 human- designed features as described in [10 Group follows Liu et al (32), utilizing the IIS dataset from MIPLIB. This dataset comprises 11 instances, divided into a training set of six instances glass-sc, iis-glass-cov, 5375, 214, 56133, iis-hc-cov, seymour, and v 1150330-2han 150300-2hopeds hhwpo--hhopss-anddddn and test set of five instances: ex1010-pi, fast0507, ramos3, sepj4scip, and scpk4. We note that baseline methods are trained on this split, whereas our method is evaluated in zero-shot setting. To further evaluate generalization, we construct Group by selecting five challenging instances from MIPLIBB's reported hard instances, dws008-03, bab2, neos -oso0--65778-kuuu
$754480-nidda and 100, and then
25

---

identifying another 10 similar instances based on feature similarity to form the new group. Detailed information on the MIPLIB dataset is provided in T
D
More Experiment Results
D.1 Results on MIPLIB Dataset
To further evaluate the generalization ability of RoME MIPLIB, conduct group experiments. For the ISS subset, the baselines are trained on the designated training instances and then evaluated, whereas RoME evaluated in zero-shot setting. And for the remaining instances, every method is run in zero-shot mode. Since the instance sizes vary on different instances, we fix variables by proportion, using (kg. k1, A) (0.7, O, 1000). We report two selected MIPLIB dataset from Section 4.3, each containing five selected instances, with the average performance across both datasets for each method presented in Table The detailed MIPLIB results are presented in Table g
Table 7: The results in the IIS and the more complicated MIPLIB Instance. We build the ML approaches on Gurobi and set solving time limit to 3,600s.
IIS (BKS
Hard Instance (BKS
Obj
gapats
Obj 4
gaPabs
Gurobi
211.00
15.00
-58663.30
325,40
PS+Gurobi
210.60
14.60
-5727.770
1710.99
ConPS+Gurobi
210.60
14.20
-58050.10 10
938.58
RoME+Gurobi
206.80
10.80
-58988.70
0.00
Table 8: The best objectives found by the approaches on each test instance in MIPLIB. BKS represents the best objectives from the website of MIPLIB https://mipll: zib. de/ index html
BKS
Gurobi
PS+Gurobi
ConPS+Gurobi
RoME+Gurobi
ex1010-pi
233.00
239.00
241.00
239.00
237.00
fast0507
174.00
174.00
179,00
179.00
174.00
ramos3
186.00
233.00
225.00
225.00
224.00
sepj4scip
128.00
132.00
133.00
133.00
131.00
scpl4
259.00
277.00
275.00
275.00
273.00
dws008-03
62831.76
64452.67
71234.06
67473.85
62831.75
dws008-01
37412.60
37412.60
39043.26
38817.50
37415.68
ncos2
454.86
454.86
454.86
454.86
454.86
bab2
-357544.31
-357538.58
-357449.20
335554.31
-357542.75
bab5
-106411.88
-104111.8
-106411. 1.84
10641 1.84
106411.84
bab6
-284248.23
284248.23
-284224.56
288224.56
-284224.56
350435555944turmma
-34.7
-34.7
-34.7
-34.7
-34.7
neos-3655608-kkumee
-13172.2
-13171.3
-13114.0
13120..6
-13172.2
supportcase17
1330.00
1330.00
1330.00
1330.00
1330.00
fastxgemm-n277s44l
42.00
42.00
42.00
42.00
42.00
neos- 3754480-nidda
12939.80
12940.78
12940.50
12940.50
12939.80
b2142
184202.75
755
190193.00
190193.00
190018.00
tr1 2-30
130596.00
130596.00
130596.00
130596.00
130596.00
s100
-0.1697
-0.1642
-0,0198
-0.1643
-0.1696
s55
-2.115
-22.15
-22.15
-22.15
-22.15
D.2 SCIP results in Main Evaluation
Consistent with the experimental setup Section 4.2 the SCIP solver as shown in T
we reported the results of different methods on
26

---

Table Performance comparison across various MILP domains using SCIP solver, under 000s time limit. We also report the improvement of our method over the traditional solvers in terms of gbs
IS (BKS 685.00)
(BKS 11.16)
SC (BKS (2461)
WA (BKS 703.05)
(BKS 97524.37)
Obj f
Time +
Obj + gaPais
Obj 4
gaPax
Ohj
gaPata
Ohj 1
gaPata
SCIP
685 00
254.63
16.55
5.39
126.43
1.38
705.77
2.72
96423.90
1100.47
PS+SCIP
685.00
129.35
16.18
5.02
126.65
1.60
705.21
2.16
96426.46
1097.91
ConPS+SCIP
685.00
78.53
16.16
5.00
126.40
1.35
705.21
2.16
96428.83
1095.54
RoME+ 685.00
13.55
16.08
4.92
126.27
1.22
705.08
2.03
96439.82
1084.55
Improvement
18.7
8.7%
11.6%
25.3%
1.4%
D.3 Fine-grained Instance-level Results
To provide a lineggrined, instance-level view of RoME's performance, we quantify its performance drop relative Gurobi. Following standard practice in multi-task learning, introduce win count metric, the number of test instances, out of 100, on which each method achieves the best objective value. We first compare Gurobi, PS, and RoME on the CA and SC benchmarks. As shown in Table TO RoME consistently dominates both benchmarks.
Table 10: Total win counts. # denotes the number of instances on which each method achieves the best objective.
CA (wins)
SC (wins)
Gurobi
26/100
5/100
PS+Gurobi
19/100
10/100
RoME+Gurobi 55/100
85/100
We further separate the instances on which each learning-enhanced solver wins or loses against Gurobi and report the mean absolute objective gap (Gap, in Table The results show that RoME not only wins on a much larger portion of instances but also yields greater improvements when wins and smaller degradations when it loses, confirming its stability and overall superiority.
Table 11: Average win/lose gap against Gurobi.
CA
SC
PS+Gurobi win
178.94 (43/100)
0.18 (68/100)
PS+Gurobi lose
207.17 (57/100)
0.41 (32/100)
RoME+Gurobi win
295.28 (65/100)
0.37 (91/100)
RoME+GGrrbi lose
I 114.93 (35/100)
0.10 (9/100)
D.4
Generalization Results
To evaluate the generalization capabilities of our approach, we evaluate its performance on larger instances of the CA and SC problems. These instances, detailed in Appendix C.2 are significantly larger than those used during training. We utilize the model trained on the dataset described in Section 4.1 to perform zero-shot evaluations on these larger instances. The results, presented in Table indicate that our method, ROME, consistently outperforms baseline methods on these larger instances, demonstrating its strong generalization ability.
D.5 Ablation Studies
The Effect DRO and MoE clarify the contribution of each component in ROME, we compare the full model with three ablated variants: (i) an MoE-only version trained without the DRO, (ii) DRO-only version that replaces the MoE with a GCN proposed in the PS [18), and (iii) single-head variant that removes expert diversity within the MoE architecture. As reported in Table I omitting
27

---

Table 12: We evaluate the generalization performance on 100 larger instances with 1,000s time
limit.
SC (BKS 451
CA (BKS 115787.97)
Obj 4
gapats
Obj
gaPabs
Gurobi
102.29
0.84
114960.25
827.72
PS+Gurobi
102.27
0.82
115228.20
559.77
ConPS+Gurobi
102.18
0.73
115343.23
444.74
RoMME+Gurobi
102.03
0.58
115787.97
0.00
either DRO MoE substantially degrades performance, confirming that these modules play indispensable roles in RoME's cross-domain generalisation.
Table 13: Impact of DRO and MoE components under 1, 0000 time limit. We report the average best objective values and absolute primal gap.
IP (RKS 11.16)
SC (BKS 124.64)
WA (BKS 70305)
CA (BKS 97524.37)
Obj 4
gaPabs
Obj
gapabs
Obj
gapats
Obj 1
gaPabs
Gurobi
11.43
0.27
125.21
0.57
703.47
0.42
97308.93
215.44
MoF-only+GGuroht
11.27
0.11
124.65
708.33
5.18
95282.51
2241.86
DRO-only+Gurobi
11.27
0.11
124.64
0.00
704.13
1.08
97194.98
329,39
Single-head-Gurrbi
11.18
0.02
124.83
0.19
703.14
1.09
92248.18
5276.19
RoME+Gurobi
11.22
0.06
124.69
0.05
703.11
0.06
97489.28
35.09
The Effect of Task Embedding Perturbation Furthermore, evaluate how perturbation influ- ences cross-domain generalization. In ROME, the perturbation is applied in the task-embedding space. We compare this design with three variants: one without any perturbation, one that perturbs the variable- eeriadddeembddiingg space, and one that perturbs the raw feature space. Table [4 shows that perturbing task embeddings yields the most robust predictions. This aligns with our expectation: the task-embedding space offers higher-level representation of the entire instance, whereas perturbations in the ceiiableembbedding or feature spaces can distort the underlying problem structure, leading to performance degradation, particularly when transferring across domains.
Table 14: Comparison of different perturbation locations under 1, 0000 time limit. We report the average best objective values and absolute primal gap.
IP (Ks 11.16
SC (DKS 12464)
WA (BKS 70305)
CA (BKS 97524.37)
Obj
gupats
Obj 1
gapabs
Obj
gapass
Obj 1
g4Pabs
Gurobi
11.43
0.27
125.21
0.57
703.47
0.42
97308.93
215.45 44
RoME without Perturb Embedding
11.61
0.45
125.27
0.63
703.20
0.15
94898.67
2625.7
Perturb on Variable Embedding
11.16
0.00
124.84
0.20
703.24
0.19
94302.71
3221.66
Perturb on Variable Features
11.23
0.01
124.74
0.10
703.10
0.05
91472.19
6052.18
RoMEECiurobi
11.22
0.06
124.69
0.05
703.11
0.06
97489.28
35.09
We also explore different perturbation magnitudes. As shown in Table 0.1-0.15 yields the best trade-off.
15
perturbation ratio of
The Effect of Expert Diversity Finally, we study the diversity of experts during training. Table T6 shows that appropriate augmentation of expert diversity improves model performance.
28

---

Table 15: Effect of perturbation ratio on CA and SC. The ML approaches are implemented using
Gurobi, with a time limit set to 1,000s. "f" indicates that higher is better, and "J" indicates that lower better. We mark the best values bold.
SC (BKS 124.64)
CA (BKS 97524.37)
Perturbation Ratio
Obj 4
gaPabs
Obj 1
gaPabs
0.05
124.70
0.06
91362.30
6162.07
0.10
124.69
0.05
97489.28
35.09
0.15
124.66
0.02
97374.32
150.05
0.20
124.70
0.06
93362.68
4161.69
Table 16: Effect of expert diversity on CA and SC. The ML approaches are implemented using
Gurobi, with time limit set to 1,000s. "f" indicates that higher is better, and ' indicates that lower better. We mark the best values bold.
SC (BKS 124.64)
CA (BKS 97524.37)
Expert Diversity Ratio
Obj
gapahs |
Obj
gapahe
0.1
125.27 124.69
0.63
94898.67 97489.28
2625.70
0.2
0.05
35.09
E Discussions
E.1
Limitations
Higher training cost. Although cross-domain solvers demonstrate strong performance, their training requires substantial diverse data and incurs relatively high computational costs. Limited by GPU memory capacity, we selected 300 samples per cross-domain dataset. Our observations indicate that while model performance improves with larger training sets, this comes at the cost of significantly increased computational overhead.
Waiting for more sophisticated designs. Cross-domain MILP solving remains in its early stage and currently lacks sophisticated architectural designs. For instance, the field would benefit from more efficient sparse MoE architectures specifically optimized for large-scale MILP applications.
E.2
Future works
Better optimization algorithms. In this work, we employ DRO to balance training across different domains, while simulating cross-domain distribution shifts through perturbations in the task embed- ding space. We note that more sophisticated mathematical techniques could further enhance model robustness by explicitly incorporating uncertainty. Future research directions include developing advanced optimization algorithms to strengthen robustness and improve cross-domain generalization performance.
Better model architecture. Future research directions may explore more sophisticated MoE archi- tectures to enhance the current framework. such incorporating sparse MoE variants attention mechanisms. Additionally, replacing the current GNN encoder with more expressive graph neural network models could further improve representation learning capabilities.
E.3 Broader Impacts
This paper presents ROME, domain-robust Mixture-of- -f-eetstts framework that predicts high-quality initial solutions for diverse MILP problems without retraining. By plugging RoME into off-the-shed solvers, practitioners logistics, network design, and manufacturing can accelcrate solution times and lower computational costs. Because RoME learns structural patterns rather than proprictary coefficients, it can be safely deployed on sensitive industrial instances while easing data-collection burdens and expanding access learning- based optimization where labeled solutions scarce.
29


Reviews:
[{"review_id": "rTO2JJ2WwM", "reviewer": "Reviewer_SAaz", "summary": "- The paper introduces RoME, a domain-robust Mixture-of-Experts framework designed to predict solutions for MILP problems across various domains. Traditional MILP solvers struggle with generalization across different problem distributions, but RoME addresses this by dynamically routing problem instances to specialized experts based on learned task embeddings. \n\n- RoME utilizes a Mixture-of-Experts (MoE) architecture combined with a two-level Distributionally Robust Optimization (DRO) strategy. The MoE consists of multiple expert networks, each specializing in different MILP distributions. For each problem instance, RoME uses a graph neural network (GNN) to extract embeddings and dynamically routes the instance to the most suitable expert. The DRO strategy include inter-domain robustness, achieved by minimizing the worst-case loss across domains, and intra-domain robustness, which stabilizes expert selection through perturbations on task embeddings. This approach allows RoME to adapt to the unique structural characteristics of each instance and improve generalization across domains.\n\n- Experiments shows that RoME significantly improves generalization and performance across multiple domains, including unseen ones, achieving a 67.7% average improvement over strong baselines.", "questions": "Please see \"Strengths And Weaknesses\"", "limitations": "Please see \"Strengths And Weaknesses\"", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Overall, I think this is a good paper. It demonstrates the efficacy of ML-model-based methods to significantly boost the runtime of  traditional solvers.\n\nStrengths:\n- Experimental results are convincing. RoME demonstrates strong generalization capabilities across multiple domains, including unseen ones, which is a significant improvement over existing methods (e.g. ConPS).\n\n- The use of a two-level DRO strategy is quite neat and, according to the ablations in D.4, it looks like it indeed enhances the model's robustness to distributional shifts, improving its adaptability to new problem instances.\n\n- The MoE architecture allows for specialization among experts, leading to better performance on diverse MILP distributions.\n\nWeaknesses and Limitations:\n\n- The complexity of the model may pose challenges in terms of interpretability and understanding the decision-making process of the experts. Additionally, the model's performance heavily relies on the quality of the task embeddings and the effectiveness of the routing mechanism.\n\n- The need for a large and diverse training dataset to fully leverage the cross-domain capabilities of RoME could be a barrier in some applications.\n\n- The approach may not be easily applicable to domains with limited data availability or where the problem structure is not well-represented in the training set.\n\n- The method's effectiveness on domains with drastically different structures from the training set remains to be fully explored.", "was_revised": true, "final_justification": "Authors addressed my concerns during rebuttal.", "rebuttal": "Dear Reviewer SAaz,\n\nThank you for your kind support and valuable comments. We sincerely hope our rebuttal adequately address your concerns. If so, we would deeply appreciate it if you could consider raising your score. If not, please let us know your further concerns, and we will continue actively responding to your comments.\n\n**W1 Model interpretability & performance**\n\n> The complexity of the model pose challenges for interpretability.\n\nWe appreciate this concern. Model complexity is indeed a common challenge for most learning algorithms. Below we address your concerns in detail.\n\n**1.RoME is designed as a pretrained model for MILP solution prediction, not merely a task-specific predictor.**\n\nUnlike prior works such as PS [1] and ConPS [2], which are tailored to a single domain, **RoME is a pretrained model and targets multiple domains simultaneously**. Although this slightly increases model complexity, it also delivers **substantial performance gains across domains**. We analyze the reasons for the success of RoME below.\n\n- **Scalability is essential for a cross-domain MILP model.**  Inspired by progress in large language models, computer vision, and drug discovery, we adopt a MoE architecture. The **GNN encoder** is identical to earlier work [1-4] and remains the primary computational cost. We add multiple lightweight MLP-based experts and decoder heads, so these introduce minimal inference overhead. Here we provide an intuitive comparison between prior work and our approach. **RoME has 697.89K parameters (storage 2.66MB), compared to 449.05K parameters (storage 1.71MB) in PS** [1], showing only a slight increase in model size. Meanwhile, our approach achieves substantial gains in cross-domain generalization ability. In addition, as **Fig. 1(b) in the main text** shows, scaling the number of experts together with the number of training tasks **consistently improves performance**. \n- **RoME meets the three pillars of a foundation model.** In recent years, foundation models have emerged across a wide range of fields. Empirically, their success relies on three core components: a robust training approach, a scalable architecture, and access to large-scale data. (i) *Training approach*: we use a **two-level DRO objective** to balance inter- and intra-domain robustness. (ii) *Scalable architecture*: the **MoE** design enables the model to **perceive and differentiate** diverse domains. (iii) *Large training data*: RoME holds the ability to scale along with the large data. Our work represents the first step toward a foundation model in the MILP domain. While still in its early stages, we hope it lays the groundwork for future development in this direction.\n- **Lower overall cost compared with single-domain baselines.** Once **pre-trained**, RoME can be reused for any target domain without retraining, whereas previous methods must be **re-trained from scratch** for each new domain and degrade sharply outside it. In practice, this makes RoME ***more economical*** despite its larger architecture.\n\n**2.RoME learns a deeper understanding in the routing mechanism.**\n\nCurrent works typically adopt GNN-based models, which are inherently treated as black-boxes. However, with the recent progress in large language models, an increasing number of interpretability techniques have emerged, particularly for MoE architectures, offering new opportunities to better understand model behavior. Following this, we have presented an interpretability study in **Section 4.4** of the main text. Below we give a further analysis.\n\n- **Expert-activation patterns across tasks.** We examine which experts are activated for different tasks and find that **tasks with similar structure consistently select the same subset of experts**. This indicates that each expert has learned **intrinsic cross-domain patterns** rather than memorizing single tasks.\n- **Latent-space visualization of task embeddings.** We project the learned task embeddings into 2-D space (T-SNE). **Structurally similar tasks cluster tightly together**, confirming that RoME captures correct task representations. This, in turn, shows that the routing mechanism maps tasks to experts in a coherent, task-aware manner.\n\n> The model performance heavily depends on routing & task embedding quality.\n\nWe acknowledge that **routing precision and task embedding quality are the core targets of our intra-domain DRO strategy, which acts as the bridge between training and architecture.** By perturbing task embeddings during training, intra-DRO properly activates the routing gate, ensuring that only the appropriate experts are activated—even under small representation shifts. This is pivotal for RoME’s cross-domain generalization; its effectiveness stems from the joint design of our DRO training strategy and MoE architecture, as detailed in **Appendix D.4**.\n\n**W2 & W3.1 Data efficiency**\n\n> RoME requires more data for training.\n\nIn fact, in data-limited scenarios, our approach demonstrates stronger performance. This is because **RoME is a pretrained model** that can be trained on diverse synthetic datasets to learn transferable, cross-domain patterns. In contrast, prior methods are trained on task-specific data and can only be applied to the corresponding target domain. Unfortunately, collecting sufficient training data for each new domain is often impractical. Additionally, to further demonstrate the **scalability** of our approach, and in line with your suggestion, we have conducted the following analysis. We trained **RoME** and the **PS** baseline using 30 %, 50 %, and 80 % of the original training data, then evaluated both models on the widely adopted **Set Cover** benchmark. The results (see the Table 1) show that RoME consistently surpasses PS at every data budget, and the performance gap **widens as the data volume increases**.\n\nTable 1: Performance of each method with the limited data.\n\n|             | 30%        | 50%        | 80%        |\n| ----------- | ---------- | ---------- | ---------- |\n| PS+Gurobi   | 125.29     | 125.22     | 125.20     |\n| RoME+Gurobi | **125.21** | **125.14** | **125.11** |\n\nIn **Section 4.3 and Appendix D.1**, RoME takes a large step forward by delivering **zero-shot** performance on the challenging industrial benchmark **MIPLIB**, whereas previous methods first have to build and train on a dedicated MIPLIB dataset—often infeasible in practice. To mirror real-world constraints more closely, we follow [3]: (i) we test RoME in a data-scarce setting where only **80%** of the original IIS training data are available, and (ii) we fine-tune RoME on the training data constructed in [3]  while freezing the GNN encoder and finetuning only the expert and head networks. With **fewer than 10 steps**, RoME already surpasses training-specific baseline such as PS. The complete results are reported in Table 2.\n\nTable 2: Performance of finetuning on the IIS datasets. \n\n|           | BKS  | PS+Gurobi | RoME zero-shot+Gurobi | RoME finetune+Gurobi |\n| --------- | ---- | --------- | --------------------- | -------------------- |\n| ex1010-pi | 233  | 241       | 241                   | **239**              |\n| fast0507  | 174  | 179       | 179                   | **174**              |\n| ramos3    | 186  | 225       | 233                   | **222**              |\n| scpj4scip | 128  | 133       | 133                   | **132**              |\n| scpl4     | 259  | 275       | 279                   | **274**              |\n\n**W3.2 & W4 More complicated problem structure**\n\n> RoME should explore more complicated problem structure during training.\n\nWe sincerely thank you for your insightful suggestions! As illustrated in **Figure 1(a)** of the main text, the key difference between prior works and **RoME** lies in the **training domains**. Compared to previous methods [1–4], our approach takes a significant step forward by being trained on **diverse domains**, each with distinct problem structures, whereas prior works are trained on a **single fixed domain**. For example, in our main experiments, RoME is trained on three different MILP families, each representing a unique problem structure. During training, we randomly sample instances from these families, meaning that the **problem structure dynamically varies across training batches**. In contrast, prior methods are trained on data from a single domain, where the structural pattern remains constant. Moreover, to evaluate **cross-domain generalization**, we not only test RoME on **unseen synthetic domains**, but also evaluate it on the **real-world MIPLIB benchmark**, which includes a wide variety of industrial instances spanning domains such as **manufacturing, energy, and transportation**. To give a clearer picture of the structural diversity, we list the **constraint types** of a subset of MIPLIB instances used in our experiments, as shown in **Table 3**.\n\nTable 3: Constraints type of different benchmarks.\n\n| Benchmark          | Constraints type                                             |\n| ------------------ | ------------------------------------------------------------ |\n| IP                 | knapsack                                                     |\n| SC                 | set_covering                                                 |\n| bab2               | aggregations, set_partitioning, set_packing, set_covering, cardinality, invariant_knapsack, equation_knapsack, knapsack, mixed_binary |\n| neos-3754480-nidda | precedence, variable_bound, knapsack, mixed_binary           |\n\n[1] A gnn-guided predict-and-search framework for mixed-integer linear programming. ICLR 2023.\n\n[2] Contrastive Predict-and-Search for Mixed Integer Linear Programs. ICML 2024.\n\n[3] Apollo-MILP: An Alternating Prediction-Correction Neural Solving Framework for Mixed-Integer Linear Programming. ICLR 2025.\n\n[4] Differentiable Integer Linear Programming. ICLR 2025."}, {"review_id": "QoEieIn764", "reviewer": "Reviewer_SKRg", "summary": "This paper proposes a domain-Robust Mixture-of-Experts (RoME) framework for computing high-quality initial solutions to mixed-integer linear programs (MILPs) that can generalize to different types of MILPs. The framework leverages a two-stage MoE architecture (on both the encoder and decoder parts). In terms of the intra-domain generalization, the authors propose a fuzzy consistency loss that forces the routing results of the original and perturbed task embeddings to be the same, which can potentially make it generalize to unseen types of problems. For the inter-domain generalization, the authors use group-DRO that puts more focus on the hardest domain in the training mini-batch. \n\nThe experiment's results look promising. One RoME model combined with Gurobi can either largely reduce Gurobi's running time or improve the quality of the solutions.", "questions": "1. See cons above.\n2. In terms of the testing problem sizes, what's the gap between them and the real-world problem size?", "limitations": "N/A", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Pros:\n\n1. The entire RoME framework shows a good generalization ability to unseen types of problems. All the proposed gadgets contribute to the performance.\n2. The paper is well written and very easy to follow.\n\nCons:\n\n1. The reason why the fuzzy consistency loss works seems unclear. In the high-dimensional task embedding space, why can forcing the original and perturbed outputs to be the same help generalize? Do you have more insights into this part?", "was_revised": true, "final_justification": "All my questions are resolved, and I will keep my score unchanged.", "rebuttal": "Dear Reviewer SKRg,\n\nThank you for your insightful and valuable comments. We sincerely hope our rebuttal adequately address your concerns. If so, we would deeply appreciate it if you could consider raising your score. If not, please let us know your further concerns, and we will continue actively responding to your comments.\n\n**W1 & Q1 Rationale of fuzzy consistency loss remains unclear**\n\n> The reason why the fuzzy consistency loss works seems unclear. In the high-dimensional task embedding space, why can forcing the original and perturbed outputs to be the same help generalize? Do you have more insights into this part?\n\nThank you for raising this point! To clarify the effect of the fuzzy-consistency loss, we have conducted an ablation study in which we **removed** this loss—that is, we no longer require the outputs of the original and perturbed task embeddings to be the same. We evaluate this change on the **Set Cover** benchmark and on the MIPLIB instance **`bab2`** (a real-world instance which also contains the `set_covering` constraint type). The results are shown Table 1.\n\nTable 1: Ablation study of fuzzy consistency loss.\n\n|      | PS + Gurobi | RoME w/o $\\mathcal{L}_{robust}$ + Gurobi | RoME + Gurobi  |\n| ---- | ----------- | ---------------------------------------- | -------------- |\n| SC   | 125.17      | 125.17                                   | **124.69**     |\n| bab2 | -357449.20  | -357168.59                               | **-357542.78** |\n\nThe improvement confirms the effectiveness of the fuzzy-consistency loss. We attribute the gains to three factors:\n\n- **More accurate routing.** A task embedding represents the structural pattern of a MILP instance and is directly consumed by the MoE gate. The loss encourages ***similar*** embeddings to select the ***same*** expert subset. In Section 4.4 we observed that both SC and bab2 activate **Expert #3**. When the loss is removed, SC still selects **Expert #3**, but bab2 mistakenly switches to **Expert #2**, degrading performance.\n- **Alignment with cross-domain reducibility.** From a computational-complexity viewpoint, most MILP classes are NP-complete, implying that different MILPs can, in principle, be reduced to one another—a fact confirmed by recent advances in graph-based combinatorial optimization [1]. Building on this insight, we exploit structural similarities (e.g., the IS and CA domains both have the `set_packing` constraints) so that related domains route to the same expert subset, enhancing generalization. Perturbing task embeddings partially simulates cross-domain shifts; forcing the perturbed and original embeddings to yield identical outputs tightens this alignment and further boosts generalization.\n- **Enhanced robustness.** From an adversarial-training viewpoint, embedding perturbations stabilize the training of DRO, helping RoME capture high-level patterns that persist across domains.\n\n[1] UniCO: On Unified Combinatorial Optimization via Problem Reduction to Matrix-Encoded General TSP. ICLR 2025.\n\n**Q2 Gap between benchmark and real-world problem sizes**\n\n> In terms of the testing problem sizes, what's the gap between them and the real-world problem size?\n\nThank you for your feedback! **Appendix C** lists the sizes of all test instances, covering both our synthetic benchmarks and the real-world **MIPLIB** benchmarks. MIPLIB instances originate from **industrial production, transportation, and other operational settings**. And their variable and constraint counts can reach **hundreds of thousands**. Here we select some of them which are reported in Table 2, and the complete information is detailed in **Appendix C**.\n\nTable 2: Statistical information of some partial benchmarks.\n\n| Benchmark           | #Constraint | #Variable |\n| ------------------- | ----------- | --------- |\n| CA                  | 2593        | 1500      |\n| SC                  | 3000        | 5000      |\n| WA                  | 64306       | 61000     |\n| scpl4               | 2000        | 200000    |\n| bab6                | 29904       | 114240    |\n| neos-3555904-turama | 146493      | 37461     |\n| s100                | 14733       | 364417    |"}, {"review_id": "fV9TveR2W0", "reviewer": "Reviewer_4Hsn", "summary": "The paper addresses the problem of domain generalization for MILP solution prediction using machine learning. The author proposes a general-purpose framework that can perform well across different MILP problem distributions,which leverages a Mixture-of-Experts model that dynamically selects experts based on learned task embeddings. Training is guided by a novel two-level DRO objective that mitigates domain shifts and promotes robustness. Results show that a single RoME model trained on multiple domains generalizes better than single-domain models.", "questions": "1. Although the model performs well on average, for some problem, a performance drop can be observed compared to Gurobi. This performance drop should be quantified and reported.\n2. The effect of the number of experts in the MoE architecture is not explored, which is essential to understand the scalability and architectural sensitivity of the proposed approach.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "#Strengths\n- This paper is well-written and easy to understand\n- This paper is technically sound\n- The experiments are extensive\n\n#Weakness\n- Although the model performs well on average, for some problem, a performance drop can be observed compared to Gurobi. This performance drop should be quantified and reported.\n- The effect of the number of experts in the MoE architecture is not explored, which is essential to understand the scalability and architectural sensitivity of the proposed approach.\n- The paper includes minor inconsistencies and typos (e.g., “T-SNE” should be “t-SNE” with lowercase “t”).", "was_revised": true, "final_justification": "I've read the rebuttal. The rebuttal addresses my concerns. I will maintain the current scores.", "rebuttal": "Dear Reviewer 4Hsn,\n\nThank you for your insightful and valuable comments. We sincerely hope our rebuttal adequately address your concerns. If so, we would deeply appreciate it if you could consider raising your score. If not, please let us know your further concerns, and we will continue actively responding to your comments.\n\n**W1 & Q1 Need to quantify performance drop vs. Gurobi**\n\n> Although the model performs well on average, for some problem, a performance drop can be observed compared to Gurobi. This performance drop should be quantified and reported.\n\nThank you for highlighting the importance of quantifying the performance drop against Gurobi. Following standard practice in multi-task learning, we introduce a “**win**” metric—i.e., the number of test instances (out of 100) on which each method attains the best objective. First, we compare Gurobi , PS, and RoME together on the CA and SC benchmarks, as shown in Table 1.\n\nTable 1: The total win counts of each method.\n\n|                 | CA (wins)  | SC (wins)  |\n| --------------- | ---------- | ---------- |\n| Gurobi          | 26/100     | 5/100      |\n| PS+Gurobi       | 19/100     | 10/100     |\n| RoME+Gurobi | **55/100** | **85/100** |\n\nRoME dominates on both benchmarks. We then separate the instances on which each learning-enhanced solver **wins** or **loses** against Gurobi and report the mean absolute objective gap ($\\text{Gap}_{\\text{abs}}$) in Table 2.\n\nTable 2: The win/lose gap against Gurobi of each method. \n\n|                      | CA                  | SC                |\n| -------------------- | ------------------- | ----------------- |\n| PS+Gurobi win        | 178.94 (43/100)     | 0.18 (68/100)     |\n| PS+Gurobi lose       | 207.17 (57/100)     | 0.41 (32/100)     |\n| RoME+Gurobi win  | **295.28 (65/100)** | **0.37 (91/100)** |\n| RoME+Gurobi lose | **114.93 (35/100)** | **0.10 (9/100)**  |\n\nRoME not only wins far more often but also gains more when it wins and loses less when it falls short. For a concrete view, the Table 3 lists the objective values on the first 20 CA instances:\n\nTable 3: The objective of partial instances obtained by each method.\n\n|      | Gurobi       | PS+Gurobi | RoME+Gurobi  |\n| ---- | ------------ | --------- | ------------ |\n| 1    | 97078.81     | 97064.76  | **97337.83** |\n| 2    | 98538.49     | 99177.49  | **99484.99** |\n| 3    | 98290.87     | 97450.07  | **97590.69** |\n| 4    | **95981.26** | 95614.13  | 95966.05     |\n| 5    | **98260.07** | 98528.65  | 97705.04     |\n| 6    | 97524.37     | 96509.32  | **97524.37** |\n| 7    | **99969.14** | 99753.87  | 99887.29     |\n| 8    | **98248.98** | 98212.79  | 97823.80     |\n| 9    | **97644.47** | 97640.97  | 97553.09     |\n| 10   | **99367.75** | 99028.51  | 98837.78     |\n| 11   | 99538.28     | 99147.39  | **99608.86** |\n| 12   | **93831.97** | 93668.45  | 93551.63     |\n| 13   | 95768.25     | 95984.20  | **96180.39** |\n| 14   | **98510.61** | 97720.41  | 97598.75     |\n| 15   | 99458.79     | 98479.86  | **99860.52** |\n| 16   | 95993.82     | 96851.43  | **96851.43** |\n| 17   | 98138.46     | 97219.51  | **98735.76** |\n| 18   | 97137.85     | 96635.77  | **97469.91** |\n| 19   | **96691.47** | 96087.46  | 96480.80     |\n| 20   | **97592.81** | 97062.06  | 97125.03     |\n\n**W2 & Q2 Effect of expert count on scalability unexplored**\n\n> The effect of the number of experts in the MoE architecture is not explored, which is essential to understand the scalability and architectural sensitivity of the proposed approach.\n\nThank you for stressing this point. As **Fig. 1(b) in the main text** shows, scaling the number of experts together with the number of training tasks **consistently improves performance**. We further evaluate scalability by varying the expert count on four benchmarks (For each benchmark, we select 20 instances) in Table 4:\n\nTable 4: Effect of expert count on RoME performance.\n\n| # Experts | IP BKS: 13.78 | WA BKS: 724.19 | SC BKS: 121.60 | CA BKS: 97428.41 |\n| --------- | ------------- | -------------- | -------------- | ---------------- |\n| 1         | 14.54         | 724.69         | 121.99         | 90769.83         |\n| 2         | 14.81         | 730.69         | 122.39         | 94101.89         |\n| 3     | **13.85**     | **724.25**     | **121.60**     | **97463.46**     |\n\n**W3 Typos**\n\n> The paper includes minor inconsistencies and typos (e.g., “T-SNE” should be “t-SNE” with lowercase “t”).\n\nThank you. We have corrected these typos in the manuscript."}, {"review_id": "eEXOISpqzS", "reviewer": "Reviewer_i5gC", "summary": "The paper introduces a machine learning model designed to predict solutions for mixed-integer linear programs (MILPs) to serve as warm starts. Specifically, the model is a variant of the mixture-of-experts framework. The predicted solution is then used to initialize the solver, which subsequently solves the original MILP constrained to a neighborhood around the warm-start solution.", "questions": "- Question 1: What is the denominator in equation (2) summation over (x')? My concern is the feasible region of a MIP can have uncountably infinite solution.\n\n- Question 2: The paper seems to claim its method works on general MIP at first, but starting from line 123, it started to assume that MIP only contains binary variables. Can you clarify this point?\n\n- Question 3: What is your intuition for the \"fully factorized solution\" assumption in line 124? Besides the part that it makes it easy to design a loss function, is there any mathematical justification for that? Ignoring the part the definition of p(x|I) is not yet well-defined, it is doubtful that this assumption can be true, as given an instance I, the value of x_1 can change the feasible region of x_2. In addition, using this assumption, are we ignoring the continuous part of the solution? it seems like even if the continuous part give bad objective, we still assign high probability.\n\n- Question 4: The definition of x^(i)_j in line 129 is a bit confusing. Why there is no j in the right-hand side?\n\n- Question 5: In Append A.3 , It is said the the selection of k_0, k_1 and Delta have great influences. Given that in Table 3, different methods use different configuration of these values. Would you be able to provide tables/plots showing the impact of these values when varying?\n\n- Question 6: I do not understand why, in Figure 3, the lines plot do not start second 0, but rather at the 10th second (4 out of 5)? Is it because the solver takes a while to find its first feasible solution or something else?\n\n- Question 7: I am curious about the quality of the binary solution returned by RoME. I think it give us more insight if you can show the objective of the returned solution of RoME or how many of it was feasible (for all instances in your experiments). For the non-pure binary case, we can assign value for the continuous variables by the solving the LP when fixed all binary variables at the predicted solution.", "limitations": "Yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strength**\n\n- The paper provides an interesting framework (RoME) for the the prediction of MIP.\n\n**Weakness**\n\n- The theoretical foundation for the choice of the loss function used in training seems weak (see Questions).\n\n- As the main contribution of the paper is the RoME model, there should also be numerical study for the solution returned by the model, not just a combined effect of the model and solving a constrained MILP (see Questions)\n\nI am willing to re-evaluate my rating if my concerns are answered.", "was_revised": true, "final_justification": "I have given my response in the comment sections. (Sorry I forgot to raise my rating)", "rebuttal": "Dear Reviewer i5gC,\n\nThank you for your insightful and valuable comments. We sincerely hope our rebuttal adequately address your concerns. If so, we would deeply appreciate it if you could consider raising your score. If not, please let us know your further concerns, and we will continue actively responding to your comments.\n\n**Q1: The denominator in equation (2).**\n\n> What is the denominator in equation (2) summation over (x')? My concern is the feasible region of a MIP can have uncountably infinite solution.\n\nThank you for your feedback! We would like to clarify Equation (2) as follows.\n\n- First, it's important to note that the denominator in Equation (2) does not represent an infinite sum of solutions. In our implementation, we utilize Gurobi to run training instances for 3,600 seconds, resulting in the **50 solutions with the highest objective values**. The denominator, therefore, is a summation over these top 50 solutions.\n- More specifically, following the previous works [2-3], RoME also aims to **approximate high-quality solution distributions**. This means we focus solely on solutions with favorable objective values during training. Consequently, we use the top 50 solutions to estimate these high-quality distributions. As you noted, the total solution space can be infinite, making a summation over the entire space impractical.\n\n**Q2: The clarification of our methods on general MILPs.**\n\n> The paper seems to claim its method works on general MIP at first, but starting from line 123, it started to assume that MIP only contains binary variables. Can you clarify this point?\n\nThank you for your insightful comments! To address your concerns, we have conducted experiments on general MILPs that include integer, binary, and continuous variables.\n\n- **Our method is applicable to general MILPs**. In Section 6.1.3 of [1], the authors indicate that the model can be generalized to non-binary variables with certain modifications. Following the guidance in [1], we have adapted our model to accommodate general MILPs.\n- We performed experiments on four challenging instances from MIPLIB, which are general MILP problems featuring integer variables. The results, presented in Table 1, demonstrate our method's superior performance compared to the baseline approaches. As in the main-text experiments, we first select a small set of similar instances (based on the instance-similarity metrics) and use them to train the PS model, which is then evaluated on the testing instances. By contrast, our method is applied to the same testing instances **in zero-shot mode**, requiring no additional training.\n\nTable 1: Experiments on the general MILP instances. \n\n|                    | #Integer | #Binary | #Continuous | BKS         | PS+Gurobi   | RoME+Gurobi |\n| ------------------ | -------- | ------- | ----------- | ----------- | ----------- | ----------- |\n| neos-3656078-kumeu | 4455     | 9755    | 660         | -13172.2    | -13114.0    | **-13172.2**    |\n| buildingenergy     | 26287    | 0       | 128691      | 33283.85    | 33284.03    | **33283.85**    |\n| germanrr           | 5251     | 5323    | 239         | 47095869.64 | 47146028.22 | **47135773.34** |\n| adult-max5features | 41       | 32597   | 36          | 5642.12     | 5679.12     | **5642.12**     |\n\n**Q3: Explanation on the \"fully factorized solution\" assumption.**\n\n> What is your intuition for the \"fully factorized solution\" assumption in line 124? Besides the part that it makes it easy to design a loss function, is there any mathematical justification for that? Ignoring the part the definition of p(x|I) is not yet well-defined, it is doubtful that this assumption can be true, as given an instance I, the value of x_1 can change the feasible region of x_2. In addition, using this assumption, are we ignoring the continuous part of the solution? it seems like even if the continuous part give bad objective, we still assign high probability.\n\nThanks for your question!  We would like to address your points regarding the definition of our probability distribution and the justification for the independence assumption.\n\n- **Clarification of the definition of the probability distribution $p(x|\\mathcal{I})$**. This is not a theoretical distribution over all possible solutions, but rather an **empirical probability** derived from a collected set of high-quality feasible solutions for a given instance $\\mathcal{I}$. To generate the training data for an instance, we first solve it to collect a set of top-performing solutions (e.g., the best 50 solutions found within a time limit). The probability for a single binary variable, $p_\\theta(x_i=1|\\mathcal{I})$, is then computed as the frequency of that variable taking the value 1 across this set of elite solutions. In essence, it is the average value of that variable in the collected solution set, weighted by solution quality.\n- **The Assumption is widely adopted in the previous works**. We acknowledge that variable independence is a strong assumption that may not hold true in practice. However, this is a widely adopted simplification in the field for reasons of computational tractability, as seen in related works [1-3]. The core intuition is that if a variable takes on a consistent value (e.g., 1) across the vast majority of top-tier solutions, then its marginal probability will be high.\n- **The variable dependencies are captured to a degree by the GNN encoder**. The GNN processes the entire MILP instance as a single graph, and through its multi-layer message-passing mechanism, the final embedding for each variable is informed by the global problem structure, including its relationships with other variables and constraints. Therefore, the complex relationships between variables are considered during the feature extraction stage, even if they are not explicitly modeled in the final probabilistic output.\n- **Explanation on the continuous variables**. The computational complexity of solving MILPs stems primarily from the combinatorial nature of the **integer** and **binary variables**. By predicting the values for a significant portion of the integer and binary variables, we effectively prune the search tree and guide the solver to a much smaller, more promising region of the solution space. Finding the optimal values for the remaining variables and continuous variables becomes significantly more tractable for the solver.\n\n**Q4 Typos**\n\n> The definition of x^(i)_j in line 129 is a bit confusing. Why there is no j in the right-hand side?\n\nThank you for pointing this typo.  We have corrected it to $x_j^{(i)} = p_{\\theta}(x_j^{(i)}=1|\\mathcal{I})$.\n\n**Q5 Sensitivity of $k_0$, $k_1$ and $\\Delta$**\n\n> In Append A.3 , It is said the the selection of k_0, k_1 and Delta have great influences. Given that in Table 3, different methods use different configuration of these values. Would you be able to provide tables/plots showing the impact of these values when varying?\n\nThank you for the insightful comments. To address them, we have conducted an additional sensitivity study on the CA benchmark. The results are summarized in Table 2. Please note that PS is trained directly on the CA dataset, whereas RoME has never seen CA during training. Despite this disadvantage, RoME still outperforms PS across all parameter settings.\n\nTable 2: Sensitivity of hyperparameters.\n\n| $k_0$ | $k_1$ | $\\Delta$ | PS+Gurobi | RoME+Gurobi |\n| ----- | ----- | -------- | --------- | ----------- |\n| 400   | 0     | 60       | 97280.70  | **97489.28**    |\n| 400   | 10    | 60       | 97235.69  | **97273.60**    |\n| 400   | 20    | 60       | 97224.88  | **97272.06**    |\n| 450   | 0     | 40       | 97231.42  | **97265.50**    |\n| 450   | 20    | 60       | 97221.65  | **97228.26**    |\n| 500   | 0     | 60       | 97194.68  | **97228.06**    |\n| 500   | 10    | 40       | 97188.40  | **97227.41**    |\n| 500   | 10    | 60       | 97195.65  | **97220.95**    |\n\n**Q6 Explanation on the Figure 3**\n\n> I do not understand why, in Figure 3, the lines plot do not start second 0, but rather at the 10th second (4 out of 5)? Is it because the solver takes a while to find its first feasible solution or something else?\n\nThanks for your feedback! We observed that our approach usually finds the first feasible solution at around 10th second—except on the IS benchmark, which is relatively easy, so all methods locate a feasible point almost instantly. This observation matches prior works [2-4], which also plot results starting from the 10th second.\n\n**Q7 Quality of RoME**\n\n> I am curious about the quality of the binary solution returned by RoME. I think it give us more insight if you can show the objective of the returned solution of RoME or how many of it was feasible (for all instances in your experiments). For the non-pure binary case, we can assign value for the continuous variables by the solving the LP when fixed all binary variables at the predicted solution.\n\nThank you for the comments! Methods in the **Predict-and-Search** family [2-3] learn a distribution of high-quality solutions and then obtain a **partial** assignment of variables. Following your suggestion, we fixed these predicted variables, solved the remaining problem, and reported the resulting objectives for PS and RoME on the SC and WA benchmarks in Table 3.\n\nTable 3: Results of directly fixing variables.\n\n|             | SC BKS: 124.64 | WA BKS: 703.05 |\n| ----------- | -------------- | -------------- |\n| PS+Gurobi   | 125.41         | 703.80         |\n| RoME+Gurobi | **125.08**         | **703.65**         |\n\n[1] Solving Mixed Integer Programs Using Neural Networks.\n\n[2] A GNN-Guided Predict-and-Search Framework for Mixed-Integer Linear Programming. ICLR 2023.\n\n[3] Contrastive Predict-and-Search for Mixed Integer Linear Programs. ICML 2024.\n\n[4] Differentiable Integer Linear Programming. ICLR 2025."}]

Author rebuttal:

Meta-review: (Partial) solution prediction is one of the paradigms in speeding up MILP solving. However, prior methods typically focused on learning on particular domains of MILPs, limiting the general applicability of the learnt models. In this paper, the authors propose to use a mixture-of-experts architecture to learn multiple domains simultaneously, with a routing mechanism based on learnt task embeddings. Beyond being able to handle the learnt domains, the resulting model also demonstrates generalization to other domains not in the training set.

Reviewers and I generally find the result relevant to the NeurIPS community, and the experiments show promise against prior methods. The rebuttal/discussion also resolved reviewer concerns. I am happy to recommend acceptance.
Decision: Accept (poster)