RouteFinder: Towards Foundation Models for Vehicle Routing Problems
Authors: Federico Berto
Chuanbo Hua
Nayeli Gast Zepeda
André Hottung
Niels Wouda
Leon Lan
Junyoung Park
Kevin Tierney
Jinkyoo Park
Venue: Accepted by TMLR ()

Abstract: This paper introduces RouteFinder, a comprehensive foundation model framework to tackle different Vehicle Routing Problem (VRP) variants. Our core idea is that a foundation model for VRPs should be able to represent variants by treating each as a subset of a generalized problem equipped with different attributes. We propose a unified VRP environment capable of efficiently handling any combination of these attributes. The RouteFinder model leverages a modern transformer-based encoder and global attribute embeddings to improve task representation. Additionally, we introduce two reinforcement learning techniques to enhance multi-task performance: mixed batch training, which enables training on different variants at once, and multi-variant reward normalization to balance different reward scales. Finally, we propose efficient adapter layers that enable fine-tuning for new variants with unseen attributes. Extensive experiments on 48 VRP variants show RouteFinder outperforms recent state-of-the-art learning methods. Our code is publicly available at https://github.com/ai4co/routefinder.

Published in Transactions on Machine Learning Research (09/2025)
RouteFinder: Towards Foundation Models for Vehicle Routing Problems
Federico Berto*!.5, Chuanbo Hua"!.s, Nayeli Gast Zepeda",, André Hottung', Niels A. Wouda Leon Lan", Junyoung Park', Kevin Tierney', Jinkyoo Park
KAIST Bielefeld University *Rotterdam School of Management
VU Amsterdam Omelet A14C0
Reviewed OpenReview: http:://pppaaerrsss...r////////d
Abstract
This paper introduces ROUTEFINDER, comprchensive foundation model framework to lackle different Vehicle Routing Problem (VRP) variants. Our core idea is that a foundation model for VRPs should be able to represent variants by treating each as a subset of a generalized problem equipped with different attributes. We propose unified VRP environment capable of efficiently handling any combination of these attributes. The ROUTEFINDER model leverages modern transformer-based encoder and global attribute embeddings to improve task representation. Additionally, introduce two reinforcement learning techniques to enhance multi-task performance: mixed batch training, which enables training on different variants at once, and different reward scales. Finally, we propose efficient adapter layers that enable fine-tuning for new variants with unseen attributes. Extensive experiments on 48 VRP variants show ROUTEFINDER outperforms recent state-of- the-art learning methods. Our code is publicly available at https://gi thub. cwwccc///////c/eeccccecccccc.o.e httta//ooooooefiiaeieeaaaaa inder.
reward normalization to balance
1 Introduction
Vehicle Routing Problems (VRPs) are an important class of Combinatorial Optimization (CO) problems that have received much attention in Operations Research (OR) and Computer Science. Since the VRP is an NP-hard problem, finding an optimal solution by exhaustively exploring the solution space is not possible for large instances. Instead, heuristic methods that quickly generate good (but possibly suboptimal) solutions are commonly used. The OR community has developed many hcuristics over the years, including the well-known -e-Kernigghan-Helggun (LKH) heuristic (Helsgaun, 2017), Fast Iterated Local Optimization (FILO) (Accorsi & Vigo, 2021; 2024) and Hybrid Genetic Search (HGS) (Vidal, 2022; Wouda et al., 2024). While these algorithms deliver state-of-tho-art results for certain VRP variants, they often require expert knowledge and careful adaptation be effectively applied in practice. Recently, Neural Combinatorial Optimization (NCO) approaches have been developed to solve CO problems. By leveraging deep learning, these approaches seek to learn from data, potentially providing more flexible and scalable solutions (Kool et al., 2019; Hottung & Tierney, 2020; Kwon et al., 2020; Kim et al., 2022; Berto et al., 2025; Hottung et al., 2025a).
Similar to how the developments in natural language processing have resulted in Large Langnage Models (LLMs), research efforts solving CO problems through machine learning are also trending toward foundation models (Liu et al., 2024c; Ye al., 2024a; Liu al., 2024a; Zhou al., 2024). However, despite the recent progress made in learning VRP variants, there lack of unified approach that provides a platform for effectively finetuning unseen variants (Lin et al., 2024). Such foundation model for VRPs would have
"Equal contributions. Authors are members the AT4CO open research community.

---

Published in Transactions on Machine Learning Research (09/2025)
important implications for real-world applications it can be casily adapted to new business requirements (constraints) outside of the training distribution.
In this work, we introduce ROUTEFINDER, a comprehensive foundation model framework for solving VRPs. We summarize our key contributions as follows:
We introduce a general framework to solve different VRP variants via unified VRP environment that can handle any number of attributes.
We propose a modern Transformer-uueed encoder and introduce Global Attribute Embeddings to enable the model to better understand and differentiate between VRPs.
We introduce two novel reinforcement learning techniques, Mixed Batch Training and Multi- Voriant Reword Normalization, to ensure stable and effective training across multiple VRP variants.
We present Efficient Adapter Layers, a lightweight yet powerful mechanism for finctuning pre-trained ROUTEFINDER models to tackle new variants with previously unseen attributes.
We evaluate ROUTEFINDER through extensive experiments on 48 VRP variants, i.e., three times as many as previous works, assessing the impact of each novel component on performance. ROUTEFINDER significantly outperforms recent multi-task learning models by reducing optimality gaps by more than 10% across all variants.
2 Related Works
Neural combinatorial optimization for VRPs NCO has emerged pivotal solution approach for VRPs, leveraging advancements in machine learning and neural network architectures (Bengio et al., 2021; Peng et al., 2021; Mazyavkina et al., 2021; Bogyrbayeva et al., 2024). While several works have explored enhancing exact solvers for other CO problems with deep learning (Gasse et al., 2019; Prouvost et al., 2020), these are not generally scalable for real-time complex VRPs (Wu et al., 2024; Kim et al., 2024), which instead usually employ hcuristics in practice. In this work, we aim to learn hcuristics for fast solution generation following the seminal work of Vinyals et al. (2015) who paved the way in applying NCO to VRPs, further developed by Bello et al. (2016) and Nazari et al. (2018). Subsequent works, including the transformer-based encoder with sclf-attention of Kool et al. (2019) and the training methods of POMO (Kwon et al., 2020) and Sym-NCO (Kim et al., 2022), have significantly enhanced solution quality. These advancements have been complemented by novel training algorithms, including learning with (partial) problem re-encoding at each step (Bdeir et al., 2022; Drakulic et al., 2024; Luo et al., 2024; 2025) and population based approaches (Grinsztajn et al., 2024; Chalumeau al., 2024; Hottung et al., 2025a). Despite this progress, challenges remain in the form of requiring manual tuning for inductive bias and the need for problem-speeifi models which impact deployment and generalizability (Liu et al., 2023; Thyssens et al., 2023). The field has also explored non-autoregreesive construction methods that predict promising edges (Joshi et al., 2022; Fu et al., 2021; Kool et al., 2022; Sun & Yang, 2024), improvement methods that iteratively refine solutions through local adjustments (Hottung & Tierney, 2020; Ma et al., 2021; 2022; 2024; Hottung et al., 2025b), and test-time adaptation methods (Hottung et al., 2022; Choo et al., 2022) which allow for solution improvement given larger time budgets. Recent works additionally explore alternative ways of solving VRPs, such as learning heuristics for Ant Colony Optimization (Ye al., 2024b; Kim et al., 2025) and sividd-andd-onneee (Kim et al., 2021; Li al., 2021; Hou et al., 2022; Ye et al., 2024c; Chen et al., 2024; Zheng et al., 2024).
Multi-task learning for VRPs In this work, we develop unifidd VRP solver to solve multiple tasks that can be efficiently fine-tuned to new ones. Due to its promise, multi-task learning for VRPs has garnered much attention recently. Wang & Yu (2025) introduces a multi-armee bandit method that solves several VRP variants with limited training budget. Lin et al. (2024) proposes training a backbone model (i.e., deep layers) for VRPs that can then be adapted via low-dimensional layers, such as lincar projections, fino-tune different problems efficiently. Drakulic al. (2025) propose a multi-task model for CO problems trained via supervised learning, similar to Large Language Models (LLMs). Jiang et al. (2024a) introduce method to
2

---

Published in Transactions on Machine Learning Research (09/2025)
transfer different problems to the embedding space via textual description through an LLM. Most related to this work are the works of Liu et al. (2024a) and Zhou et al. (2024), which use attribute composition (Ruis et al., 2021) to achieve (zero-shot) generalization on several VRP variants. Liu et al. (2024a) builds on the Cettorrceent---aannnne based POMO (Kwon et al., 2020), on top which Zhou et al. (2024) employ a mixlure-of-exeers model lo improve generalization.
3 Preliminaries
3.1 Vehicle Routing Problems
We formulate the classic VRP, the foundation for more complex variants, on graph G (N, where m + n - 1) represents the nodes, with Na = (O...., m - 1} denoting the m depots
= (O..... m - 1, (m for the classic VRP) and N₂ {m..... m + n - denoting the n customers. The edges E connect. pairs of nodes, and each edge (i.j) E has travel cost Cý (e.g., distance or travel duration). Vehicles depart from the depot to serve each customer exactly once and then return to the depot(s), while minimizing the total travel cost. Following Vidal et al. (2014), consider collection of VRP voriants that extend
Linehaul demands (C) Backhaul demands (B) Time windows (TW)
Multi-depots (MD)
Open (0)
Duration limits (L)
Mixed backhaul (MB)
0.4
s1:8.5
68.0.7
0.2
0.3
0.1
0.1
B
8.4
0.5
0.2
adoo.5
8.3
Depot
Customer
Linchaul
Backhaul
Feasible route
Customer window
Service duration
Figure 1: VRP attributes. Linehaul demands (C), backhaul demands (B), time windows (TW), and multi-depot (MD) are node attribates, whereas open routes (O), duration limits (L.), and mixed backhaul (MB) mode are global attributes. Attribute combinations define new VRP variants.
the classic VRP by one more attributes (see Figure 1). This results in rich set of routing problems with practical relevance. Fach of these variants offers unique generalization task for ROUTEFINDER. We describe the attributes in the following, separating them into node attributes and global attributes. Table in Appendix A provides list of all 48 VRP variants considered in this work.
Node attributes
Demand and Vehicle Capacity (C) [g E [0, Q]]: Every customer i E N₂ has linehaul demand 4 and is serviced by vehicles with fixed capacily Q The total customer demand per vehicle cannot exceed its capacity at any point.
Backhauls (B) [p E [0, Q]]: Backhauls generalize demand to also account for return shipments. Customers are either linehaul or backhaul customers. Linehaul customers require delivery of a demand di that needs to transported from the depot to customer i (as in the CVRP), whereas backhaul customers need a pickup of an amount P; that is transported from the client back the depot. Vehicles can serve a combination of linehaul and backhaul eustomers on single route, but any linehaul customers must precede the backhaul customers the route. An application with returnable bottles is presented in Ropke & Pisinger (2006).
Time Windows (TW) [e. s, E [0, ]] Every customer i e N₂ has a time window [e,, 4] during which service must begin. Service takes s; time. The depot has time window [co:4] = [0, 7] and service duration of Su = O. Vehicles must reach node before 4, but any early arrivals must wait at node i until e; before service may start.
3

---

Published in Transactions on Machine Learning Research (09/2025)
Global attributes
Open Routes (0) [o E {0, 1H}: Vehicles are not required to return to the depot after serving all customers. Open routes can be found in applications with third-party drivers, who are often only compensated until they have completed their last delivery (Li et al., 2007).
Duration Limits (L) |l [0, LJ]: Imposes limit on the total travel duration (or length) of each route, balancing the workload across vehicles, and, e.g., limiting the duration of the work day.
Mixed Backhauls (MB) [p {0, 1}}: Relaxes the strict precedence constraint of linehaul customers preceding backhaul customers: with mixed backhauls, linehaul and backhaul customers may be mixed along route. The vehicle's capacity must still be respected at any point along the route. Since both the current linehaul and backhaul demands must be tracked per vehicle, this varianl requires careful planning.
Multi-depot (MD) (m > 1]: Generalizes single-depot (m = variants to multiple depot nodes m from which vehicles can start their tour. Each vehicle must return its starting depot. This variant requires decisions about depot-to-customer assignments, making the problem more realistic for organizations operating from multiple facilities (Karakatič & Podgorelec, 2015).
3.2 Learning Neural Solvers for VRPs
Solving VRPs using Autoregressive Sequence Generation Autoregressive (AR) methods address CO problems by constructing solutions sequentially. The process begins with encoding the problem instance a (e.g., node and global attributes) using a trainable encoder fo that maps to an embedding h = fo()). The solution a is then decoded based on h through series of actions, where each action determines the next step in the solution based on the current partial sequence. This is achioved using a decoder go- The encoding and decoding process are formalized as:
(a(a)
II go ag, h),
(Ia)
where a = (ai,..,, uT) represents a feasible solution to the CO problem, T the steps in solution construction, and my the stochastic solver mapping problem instance æ to a.
Training VRP Solvers via Reinforcement Learning The solver me be trained using either supervised learning (SL) or reinforcement learning (RL). This paper focuses on RL due to its ability to train solvers independently of the availability of optimal solutions. Under the RL framework, the training objective for neural combinatorial optimization solvers is defined as:
9" argmax Em= (Ra
where P(x) is the distribution of problem instances, and R(a, represents the reward (i.c., the negative cost), associated with the solution a for the given æ. The above training problem can be tackled using various RL. algorithms such as REINFORCE and its modern variants (Kool et al., 2019; Kwon et al., 2020).
4 The RouteFinder Recipe
ROUTEFINDER leverages attribute composition from Liu et al. (2024a); Zhou et al. (2024) to solve multiple VRP variants. We treat different variants of the VRP as combinations of fundamental attributes (Section 3.1) and use common network to learn their representations. We go further than previous works and consider different combinations of attributes within training batches (see Section 4.3.1). Figure provides an overview of ROUTEFINDER'S architecture.
4

---

Published in Transactions on Machine Learning Research (09/2025)
Policy
Solutions
Encoder
Decoder
done
CVRPB
Global Embedding
Layers
Hidden States
Context Embedding
Decoder Layers
OVRRPRM
EAL
Nx
EAL
OVRPENL TW
DVRRBMLTV
CURUP
OVRPLTW
Node Embedding
Context
CURPLITS
FAL
EAL
OVRPOMLTW
B
CVRPB
Rewards
Global
Node
OVRPENLTY
Attrihutes Attributes
Multi-task
Roward Normalization
Mixed Batch Training
reset()
Unified VRP
StepC)
Environment
CVRPS
DVRPLTT
DVRPBMLTM
OVRPENLTH TM
Artions
Artion Probahilities
Normalized Rewards
Figure/ 2: ROUTEFINDER overview. The unified VRP environment is used for data generation and solution construction (Section 4.1). Our Transformer-based encoder (Section 4.2.1) processes node and global embeddings (Section 4.2.2) of problem instances. During training, we sample multiple variants the same batch (Section 4.3.1) whose multi-task reward then normalizod (Section 4.3.2). Efficient Adapter Layers (EAL) are employed for efficient fine-tuning to new variants (Section 4.4).
4.1 Unified VRP Environment
In previous works proposing multi-task learning for VRPs, like MTPOMO (Liu et al., 2024a) and MVMoE (Zhou et al., 2024), instance variants (CVRP, VRPTW, etc.) are sampled out of the set of available variants during training. Every instance within that batch is then of the same problem category. This can bias the optimization at each gradient step toward a specific task, potentially hindering stable and effective training for foundation model. We thus propose learn across problems throughout training and include instances of various variants within each training batch.
We define environment capable of modeling all of the previously discussed VRP attributes (see Section 3.1) simultaneously. Essentially, we build an MDOVRPMBL.TW environment: a multi-depot open route vehicle routing problem with linehauls, (mixed) backhauls, distance limit, and time windows. The environment supports subsets of the MDOVRPMBLTW by turning attributes "on" or "off". For example, if an instance does not have time window constraints, the time windows attribute of each customer is set to [0, oo], rendering them irrelevant during solution construction. This modular attribute composition allows us to model up to 48 different problem types with one single environment. This approach can be easily extended, e.g., by including different location sampling mechanisms and new constraints, allowing for even more future problem variants to be modeled within the same environment.
4.2 Model
Attention Model
RouteFinder Transformer
4.2.1
Transformer-based Architecture
Norm
The ROUTEFINDER transformer encoder architecture shown in Figure 3 introduces key enhancements to the Attention Model (AM) from Kool et al. (2019), the de- facto standard in recent works (Liu et al., 2024a; Zhou et al., 2024). Firstly, the ROUTEFINDER transformer encoder employs Root Mean Square (RMS) normaliza- tion (Zhang& & Sennrich, 2019), improving stability and training speed. Secondly, we transition from post-norm to pre-norm in the transformer layers, applying normalization before the residual connections, which enhances gradient flow and promotes faster convergence (Jiang et al., 2024b). Thirdly, ROUTEFINDER uses Feed Forward SwiGLU, (Shazeer, 2020), extension of the Gated Linear Unit
Feed Forward SwiGLU
Feed Forward
RMS Norm
Norm
Multi-Head Attention
Multi-Head Attention
RMS Norm
Nx
Nx
Figure 3: Attention model structure ROUTEFINDER transformer structure.
5

---

Published in Transactions on Machine Learning Research (09/2025)
(GLU) (Dauphin et al., 2017), instead of the AM's ReLU-based foed-forward network. This enhances the model's capacity to capture complex relationships in the data. Finally, we employ Fltentio (Dao et al., 2022; Dao, 2024) in the Multi-Head Attention layer of all models to enhance overall performance. These improvements build on recent advances in foundation models in areas such as language modeling and biology (Dubey et al., 2024; Nguyen el al., 2024). By building on modern architectures, we create a robust foundation model for VRPs. Further details on architecture and modcling are provided in Appendix B.
4.2.2 Global Attribute Embeddings
Global attributes as outlined in Section 3.1 are essential for modeling VRPs. For instance, given an open (0) attribute, the solver may find optimal routes that do not necessarily loop back to the starting depot. Previous multi-task learning models for VRPs (Liu et al., 2024a; Zhou et al., 2024) project such features on the shallow decoder as dynamic features. However, such a design can be suboptimal, since the deep transformer layers carry out most of the learning and can enable effective attribute mixing, which is essential for understanding a (uew) problem. We therefore design Global Attribute Embeddings for effective problem representation, which incorporate problem variants and help the deep layers understand which problem is being faced. Global attributes do, ; Ōb are projected via a projection layer:
- so.
fo R + Rd
(3)
into d-dimensional space. Given our unified VRP representation, some attributes, such as the duration limit for unconstrained VRPs, might be co. These attributes are padded as Os before being processed by the deep transformer layers. We highlight the significance of Global Attribute Embeddings in Appendix D.6, where an analysis of the t-SNE latent space (Van der Maaten & Hinton, 2008) provides insights into their interpretability and importance.
4.3 Training
4.3.1
Mixed Batch Training
Optimizing a neural solver for tackling multiple tasks requires careful consideration of its training scheme, which needs to be robust against different variant distributions. We introduce flexible approach which we call Mixed Batch Training (MBT) to efficiently reuse a single dataset to generate multiple problem variants. This optimizes data storage and processing.
Let X be dataset of MDOVRPMBITW problem instances and V be the set of attributes, where each attribute v E V is associated with a sampling probability Pv- For each instance E X we can write =((11)vev) to conveniently express using indicator functions 11 for each attribute v E V that the instance x has altribute v. The sampling procedure of MBT can be defined as follows:
X subsampled = == (p
where rand(0, 1) draws an independent sample from U00 1]. To sample uniformly across all problem variants, set P. 2. Vv V.
Note that the MDOVRPMBLTW problem variant is the most general problem variant we study in this paper and can be used to generate any of the other variants by selectively removing any combination of the global attributes (MD), (O), (MB) and (L), and node attributes (B) and (TW). We first consider OVRPBLTW as the most general problem variant from which we sample other variants. Then, for zero-shot generalization and few-shot learning, we additionally sample with multiple depots (MD) and mixed backhauls (MB). This increases the number of distinet variants from 16 variants that can be generated from OVRPBLTW to 48 variants that can be generated from MDOVRPMBLTW.
MBT is a flexible and scalable approach, capable of adapting to any problem where different constraints or features might be selectively activated or deactivated. Figure 4 illustrates the stabilizing effect of MBT during training. Specifically, MBT significantly reduces the variance of the training loss, leading to faster and more stable convergence, as demonstrated in our experimental results in Section 5.2.

---

Published in Transactions on Machine Learning Research (09/2025)
Without Mixed Batch Training
With Mixed Batch Training
0.06
MBT
OVRP
0.05
No MBT
CVRP
CVRP VRPTW
A Optimal
x Optimal
0.03
0.02
OVRPB
VRPTW
0.00
20000 40000
G0000
5000
100000
Step
Figure 4: [Let]] Training without MBT leads to instability, since at each step the optimization is biased toward single task. [Middle] Training ROUTEFINDER with MBT allows for stable training. [Right] Effect of MBT on the loss during training.
4.3.2 Multi-task Reward Normalization
As explained in Section 3.2, the objective for RT-based NCO solvers is to maximize the expected reward. However, in multi-task learning settings different problems can yield rewards on different scales. To counteract potential biases during learning, we propose to apply reward normalization per problem variant. We implement four simple normalization techniques to calculate the normalized rewards norm,, for all problem variants & & {l,..., K} at training steps 2 1: a) subtraction of the simple mean reward, b) division through the simple mean reward, e) subtraction of the exponentially smoothed mean, and d) division through the exponentially smoothed mean. We calculate the average reward f((²) up to training step using the average batch reward (k) at training step (see Appendix C.1). The simple mean reward at step t is calculated as:
(k)
{(()) = ((t ((-- 1) - (1) + + + /t, t 121.
(4)
For the exponential moving average we set p(()) = =r and calculate the values for >> based on Hunter (1986) using a smoothing factor a:
(k)
P! (1 (1- pp( + a .t. (k)
0 0<a<l, t>L.
5
Normalized rewards d) can be calculated from original rewards (( with (k)
nnrrm, = (k)
r P/ and
(.()) - = -r(h ) for subtraction and division variants, respectively. Let E(a, = rnoem (a, x) be a function calculating the normalized reward for instance æ that additionally maps instance x to variant The multi-task newari--orrmaizzed gradient becomes:
N Σ 5(a), .r) N
1 N
1
VoJ(O)
E(a',x)
Vo log (og
(6)
ie., we employ the REINFORCE loss function with the POMO (Kwon et al., 2020) shared mean bascline (right side of the parenthesis) to improve convergence, where both the reward and the shared baseline are normalized by g.
4.4 Efficient Adapter Layers: Finetuning to Unseen Attributes
Previous multi-task learning works (Liu et al., 2024a; Zhou et al., 2024) train in environments of single- attribute VRP variants and, using compositionality (Ruis et al., 2021), achieve promising results on zero-shot generalization to VRP variants combining these individual altribules. In ROUTEFINDER, we go a step further and investigate how to efficiently generalize our pre-trained foundation model to variants with wnseen attributes. Lin et al. (2024) propose pretraining a backbone model, on top of which specific Adapter Layers (AL) can be applied for finetuning to new problens. The rationale that the backbone (i.e., the encoder layers) may capture transferable knowledge. However, doing excludes previous information accumulated in
7

---

Published in Transactions on Machine Learning Research (09/2025)
the projection layers from the raw attribute features the hidden space, complicating optimization. For example, if the first two out of & dimensions encoded the Fuclidean locations of nodes as (2, v), re-initialixing new adapter layer from scratch will eliminate such transferable knowledge. Therefore, we propose Efficient Adapter Layers (EAL), an effective approach to few-shot learning for VRP foundation models.
Consider a linear projection layer W Rkxd as the original weight matrix for the projection from the raw attribute to latent space, where & is the number of attributes and d is the hidden dimension. For simplicity, consider unbiased linear projections to the latent space. This can be readily extended to general affine projections using a bias term. To accommodate 1 new attributes, EAL augments W with zeros. The new
matrix W = [[ 0] can be writlen as:
W0o
Wol
W = d
:
Wdd
W.d
where 0 E lR is matrix of zeros. The augmented matrix W retains the original attributes and adds new attributes, which are initialized to zcro. Doing so does not affect the model for seen attributes like AL does. Instead, these I dimensions have no effect until fine-tuning on new variants occurs. This allows for new attributes to be included in any part of the model via EAL, as shown Figure 2.
5 Experiments
In this scction, we empirically demonstrate the state-of-tho-art performance of ROUTEFINDER extensive experiments on 48 VRP variants. Our code is publicly available". We address the following research questions:
(RQ1) Does ROUTEFINDER outperform state-of-the-ard foundation models on different VRP variants?
(RQ2) How do the individual novel components of ROUTEFINDER contribute to its performance? (RQ3) Is the proposed FAL, effective in finetuning ROUTEFINDER models to unseen VRP variants?
Hardware All training runs are conducted on NVIDIA A100 GPUs and take between 9 to 24 hours per model. Evaluation is conducted on an AMD Ryzen Threadripper 3960X 24-core CPU with a single RTX 3090 GPU,
Baselines Traditional solvers: We use VRP (Wouda al., 2024), an open-source, stale-of-the-ard heuristic VRP solver built on top of HGS-CVRP (Vidal, 2022), and the popular Google OR-Tools (Perron & Furnon, 2023). Both solve each instance instances with 50 and 100 nodes, respectively. We parallelize (raditional solvers across 16 CPU cores in Zhou et al. (2024). Neural solvers: We consider recent multi-task learning baselines for the VRP, including the recent MТPOMO (Liu et al., 2024a), which is based on РOMO (Kwon et al., 2020), and MVMoE (Zhou et al., 2024), which introduces coxture--ffexpeeree (Fedus et al., 2022) to improve the model performance. Our models: We consider three versions of ROUTEFINDER denoted as RF in the tables: one considering the (MT)POMO encoder (RF-POMO), one with the MVMož model with four experts and hierarchical gating (RF-MoE), and one with our modern Transformer-basee Encoder (RF-TE). We use Reward Normalization with division through the exponentially smoothed mean with 0.25. Further details are available in Appendix B.
single CPU core with time limit of 10 and 20 scconds for
Data Generation To train and evaluate RouteFinder across a diverse set of VRP variants, we employ a unified data generation process detailed Appendix A.1. Problem instances are generated using our modular environment that supports all combinations of seven core VRP attributes: capacity (C), open routes (O),
httpe://gthhuu.cm/aalee/rrrreree
hhttpe://github.com wwt.://ww...omai/i///ooaae http://tww.ccmmaaeeeeeoottttntae
8

---

Published in Transactions on Machine Learning Research (09/2025)
Table 1: Performance 1000 test VRP instances. The lower, the better (1). "* represents the hest-known solut ions.
ROUTEFINDER (RF) models outperform ttate-of-the-art neural baselines in all settings.
Salver
50
100
Solver
100
Obj-
Cap
Time
Ohj.
Gap
Time
Obj
Gap
Time
Ohj.
Cap
Time
IIGS-PyVRP
10.372
10.4m
15.628
20.Stn
IICS-PyVRP
16.031
10.4m
25.423
20.8mn
OR-Tools
10.372
1.907%
10.4m
16,250
4.178%
20.5mn
OR-Tooll
16.089
0.347%
10:4m
25.814
1.506%
20.8an
CVRP
MTPOМO
10.518
1.411%
15:934
1,988%
MТPOМM
16.410
2.3619
1s
26/112
3.873%
MVMoE
10.501
1.242%
2s
15.888
1.694%
96
MVMoFE
16.404
2.3/21%
2s
26.380
3.788%
9k.
RE-POMO
10.508
1.31495
26
15.908
1.826%
7s
RE-POMO
16.367
2.0945
1s
26.336
3.575%
7s
RF-MOE
10.499
1.226%
15.876
1,62%
RE-MoE
16.389
2:2319%
2s
26.322
8.515%
RF-TE
10.504
1,274%%
2s
15.857
1.506%
7%
RF-TE
16.364
2.077%
26,235 14
3.178%
74
HGS-PyyRPP
6.507
10.1m
9.725
20.8un
HGS-PyVRP
10.587
10:1m
15.
20. Son.
OR-Tools
6.553
0.680%
10.4m
9. 9.9996
2.7322%
20.8m
OR-Tools
10.570
2.343%
10.4m
16.466
5.302%%
200 .8.8
OVRP
MTPOMO
6.718
3.209%
10.210
4.965%
6c
10.775
1.734%
1s
16.149
2.404%
MVMoE
6.702
2.965%
25
10.177
1.621%
St
MVM&E
10.751
1.505%
28
16.099
2.115%
96
RF-POMO
6,608
2.904%
18
10:180
4. 6.600%
RF-POMO
10.751
1.623%%
14
16.107
2.174%
RE-MoE
6.607
2.886%
10.139
4.229%
RE-MoE
10.737
1.388%
2s
16.070
1.941%
RE-TE
6.684
2. 6.687%
1s
10.121
4.055%
Go
RF-TE
10.749
1.502%
16.051 1s
1.827%
66
HGS-PyVVR
9.687
10.4m
11:377
20.5mn
HGS-PyVRP
10.510
10:4m
16.926
20. San
OR-Tools
9,802
1.130%%
10.4m
14:933
3 3.858%
20.8m
OR-Tools
10.519
0.078%
10:4m
17:027
0.583%
20..8m
VRPD
МТPOMO
10.033
3.564%
1s
15.082
4.922%
6x
MTPOMO
10.668
1.479%
1s
17.420
2
7s
MVMoE
10.005
3,2703%
26
15.023
4.508%
MVMO
10.669
1.492%
2s
17.116
2.872%
10s
RF-POMO
9,996
3,174%
15.016
4.468%
RF-POMO
10.637
1.378%
1s
17.399
2. 2.720%
RF-MoF
9,580
3.0115%
28
14.973
4.164%
RF-MoE:
10.674
1.530%%
2s
17.387
2.697%
10s
RF-TE
9.977
2.059%
1s
14.942
3.952%
RF-TE 10.652
1.326%
1s
17.327
2.346%
7s
HGS--PyVRP
10.186
10.4m
14:779
20.8m
HCS-PyVRP
18.361
10.4m
29.026
20.88m
OR-Tools
10.331
1.3909%
10.4m
15:426
4.338%
20.5m
OR-Tools
18.422
0.3325%
10:4m
29.830
2.770%
20,8an
VRPEL
MTPOMO
10.672
1.697%
18
15.712
6. 6.2511%
To
MTPOMO
18.990
2.128%
1s
30.896
3.621%
7e
MVMoE
10.637
4.354%
24
15.640
5.708%
MVMoR
18.995
2.100%
24
30.892
3.608%
10-
RF-POMO
10.590
3.942%
15:628
5.695%
6c
RF-POMO
18.937
1.851%
1s
30.796
3.284%
RF-MoE
10.575
7.5%
26
15.511
5.121%
9e
RF-MoE
18.957
1.960%
28
30,808
3.9233
10s
RF-TE
10.378
3.803%
18
15.528
5.039%
RF-TE
18:941
1.877%
1s
30.688
2.9/3%%
HGI--PVVR
18:202
10.4m
29-467
HGS-PyVRP
16.356
10:4m
25.707
20
OR-Tools
18.366
0.383%
10.4m
29:945
L.577%
20.8en
OR-Tools
16.441
0.499%
10.4m
26.259
1880%%
20.8mm
VRPEET
MTPOMO
18.639
1.878%
1s
30/137
3, 3,285%
7s
МТРОМО
16.821
2.623%
1s
26.891
1.368%
76
MVMoE
18.640
1.883%
30-436
3.2811%
MVMoR
16.811
2.730%
24
26.868
4.277%
RF-POMO
18.001
1.670%
18
30.341
2.961%
7%
RF-POMO
16.750
2.382%
1s
26. 78.7
3.948%
75
RF-MoE
18.618
1.757%
26
30.341
2.95.%
96
RF-MoE
16.777
2.530%
2s
26.774
3.912%
96
RF-TE
18.600
1.676%
30.241
2.619%
RF-TE
16.762
2.451%
26.689
3.579%
HGS-PyVRP
6,898
10.4m
10.335
20.5m
HGS-PyVRP
6.599
10.4m
10.333
20. San
OR-Tools
6,9288
0.112%
10.1m
10.577
2.315%
20.8un
OR-Tools
6.927
0.386%
10:4m
10.582
2.363%
20. 8cn
OVRPB
MTPOMO
7.108
3.005%%
10.878
5.224%
MTРPМM
7.112
3.011%
14
10.884
5.270%
MVMoE
7,089
2.741%
2s
10.840
4.861%
96
MVMGE
7.098
2.848%
2s
10.847
4.928%
RF-POMO
7,086
2.688%
1s
10.836
1.821%
Te
RF-POMO
7.087
2.693%
1s
10.537
1.890%
6e
RF-MoE
7,080
2.613%
28
10.806
4.522%
96
RF-MoE
7,083
2.633%
24
10.806
4. 4.534%
RF-TE
7.071
2.479%
10.772
4.208%
RF-TE
7.074
2.508%
1s
10.778
4.262%
JIGS-PyVRP
11.668
10.4m
19.156
20.8tn
IIGS-PyVRP
11.669
10.4m
19.156
20. 8mm
OVRPBLTW
OR-Tools
11.681
0.106%
10.4m
19.305
0.767%
20.5an
OR-Tools
11.682
0.109%
10:4m
19.303
0.757%
20. Sa.
MTPOMO
11.817
1.2900%
19:637
2.496%%
MTPOМM
11,814
1.220%
1s
19.633
2-486%
MVMoE
11.822
1.3015%
2s
19:641
2.518%
10s
MVMoE
11.819
1.271%%
2s
19.638
2.503%
10s
RF-POMO
11.805
1.157%
1s
19.009
2.341%
&
RE-POMIO
11.801
1.148%
1s
19.607
2.339%
76
RF-MOE
11.824
1.312%
19:607
2.334%
10s
RF-MoE
1.304%
2s
19.606
2. 2.328%
RF-TE
11,805
1.150%
18
19.551
2.048%
7s
RF-TE
11,805
1.151%
1s
19.550
2.042%
7s
HGS-Py-VRR VRP
6.507
10.4m
9.724
20.8u
HGP-PyVRP
10.510
10:4m
16:926
20. 8c..
OR-Tools
6.552
0.608%
10.4m
10.001
2.791 1%
20.8m
OR-Tools
10.497
0.114%
10.4m
17.023
0.728%
20.88m
OVRPI.
MTPOMO
6.719
3.227%
10.214
5.002%
G6
MTPOMO
10.670
1.5001%
1s
17.420
2.889%
MVMoE
6.707
3.090%
25
10.18-1
1.696%
9h
MVMsE
10.671
1.511%
2s
17.119
2.885%
10s
RF-POMO
6.701
2.949%
10.180
4.600%
RF-POMO
10.657
1.371%
14
17.3553
2.731%
RF-
6.606
2.864%
10.140
4.249%
RF-MOE
10.673
1.532%
2s
17.366
2.603%
10s
RF-TE 6.656
2.721
1s
10.120
4.052%
Ge
RF-TE
10.653
1.3411
1s
17.327
2.347%
7a
backhauls (B), duration limits (L), time windows (TW), mixed backhauls (MB), and multi-depots (MD). Node locations are uniformly sampled in [0, 112, while vehicle capacity is set relative to the number of nodes, i.e., C = 40 for n = 50 and C = 50 for n = 100. Linehaul and backhaul demands are integers uniformly sampled from {I....,9) 9). with each customer having either a linehaul or backhaul demand with probability 0.8 and 0.2, respectively. Time windows are generated using a randomized offset procodure ensuring fcasibility with lengths sampled uniformly from [0.18, 0.2], and service times are sampled uniformly from [0.15, 0.18). Duration limits are drawn from the distribution described in Appendix A, ie., ensuring reachability while
9

---

Published in Transactions on Machine Learning Research (09/2025)
introducing meaningful constraints. For multi-depot problems, three depots used by default. Attribute combinations are sampled uniformly using mixed batch training with with = 0.5, Vu E V to cover all variants equally.
Training Each model is trained for 300 epochs on 100,000 VRP instances that are generated on the fly and include all attributes except for (MB) and (MD) which are only used finetuning, We use the Adam optimizer (Kingma & Ba, 2015) with a Icarning rate of 3 10 and batch size of 256. At epochs 270 and 295, the learning rate is multiplied by 0.1. Note that setup differs from the in Liu et al. (2024a) and Zhou et al. (2024) in that do not artificially restrict the variants with single attributes (such as only (B) (TW)) but train on all available variants, similarly to how LLMs are trained on all available data. This is readily available through our unified VRP environment (more details in Appendix A).
We evaluate all approaches on 1,000 instances of held-out test data for each size n of cach variant. We roll out greedy solutions for all NCO approaches using multi-staris and 8x symmetric augmentations resulting in solutions per instance from which the best selected (Liu al., 2024a; Zhou al., 2024).
Evaluation
5.1 (RQ1) Main Results
In-distribution Table shows the in-distribution testing results for 50 and 100 nodes n. ROUTEFINDER models consistently outperform neural baselines across all variants by more than 10%. While changing the encoder to the MVMoE's structure (RF-MoE) may slightly improve the performance in limited seltings with higher inference cost due the more complex structure of mixture--ofcoxpers the proposed Transformer Encoder (RF-TE) outperforms the other models in almost all settings, particularly for n 100.
Out-of-distribution For CVRP, the training distribution with 100 nodes considers a vehicle capacity C 50. We study generalization over different capacities C' E {30, 50, 70, 90, 110, 130, 150, 200} and show the results Table 2. POMO trained on CVRP performs best for capacities close to the training distribution, but ROUTEFINDER demonstrates superior generalization and finds the best solutions for larger capacities.
Table Comparison of model with single-task POMO on out-of-distribution CVRP instances.
Vehiele Capacity
30
50
70
90
110
130
150
200
Ob).
Gap
Ob).
Gap
Obj
Gap
Obj
Gap
Obj.
Gap
Obj.
Gap
Obj.
Gap
Obj.
Gap
POMO CVRP
22.95
15.72
12.91
11.48
10.64
10.04
9.75
9.24
MTPOMO
23.29
1.50%
15.87
0.94%
13.07
1.24%
11.69
1.77%
10.88
2.30%
10.34
2.90%
10.04
2.97%
9.59
3.77%
MVMoE
23.34
0.43%
15.83
0.67%
12.99
0.61%
11.34
0.49%
10.67
1.33%
10.06
0.12%
9.74
-0.09%
9.21
-0.28%
RE-OOMO
1.07%
1.12%
9.38
1.51%
RF-MoE
23.13
0.80%
15.81
0.58%
13.00
0.74%
11.59
0.89%
10.74
0.92%
10.11
0.96%
9.82
0.69%
9.31
0.75%
RF-TE
22.96
0.06%
15.79
0.44%
12.95
0.29%
11.47
-0.07%
10.56
-0.71%
9.92
-1.22%
9.59
-1.67%
9.02
-2.36%
We further evaluate out-of-distribution val-
ues for additional attributes (Appendix D.1)
Average Gap (%)
4.0
and large-scale CVRPLIB instances (Ap- pendix D.2), and find that ROUTEFINDER can
3.8
3.6
consistently generalize more robustly in real- world settings than neural baselines, including single-variant POMO (Kwon et al., 2020).
3.2
MTPOMO
RouteFinder (full)
No Trannformet Encoder
No Global Banbeddings
No Mixed Batch Training
No Roward Normalization
5.2 (RQ2) Ablation Studies
Contribution of components We conduct ablation studies to evaluate the impact of our individual contributions.
Figure 5: Ablation study on ROUTEFINDER components.
Figure compares the performance of ROUTEFINDER (RF-TE) against its variants with ablated components, using the results for MTPOMO baseline. All components contribute to the performance of ROUTEFINDER.
10

---

Published in Transactions on Machine Learning Research (09/2025)
Effect of MBT on training loss and convergence speed We compare two ROUTEFINDER models trained with identical hyperparameters on = 50, one with and one without MBT. By keeping the overall sampling distribution the same but mixing variants in the same batch, MBT allows for a more stable gradient across the different tasks, resulting in a substantially more stable loss compared to training without it.
In Figure we show the validation gaps on held-out instances exemplarily for two variants, CVRP and OVRPBLTW. In addition to stabilizing the training loss, MBT also speeds up convergence. Figure 11 in Appendix D.5.1 shows that this holds for all 16 variants that can be generated from OVRPBLTW.
CVRP
OVRPBLTW
2.-
MBT
2.0
No MBST
244
22-
(5)
2.0
1.0
Gap
Gop
1.4
1.6
1.0
1.2
50
100
150
200
250
0
30
100
150
200
250
Egods
Epoche
Figure Mixed Batch Training (MBT) improves convergence during training, shown here for the CVRP and OVRPBLIW.
Further ablation studies We study the ablation of single Transformer Encoder components and find that the combination of all components provides the best performance (Appendix D.3). We further study the effect of different Reward Normalization techniques (Appendix D.4), and the importance of MBT for convergence speed (Appendix D.5.1) and its effect on imbalanced variant distributions (Appendix D.5.2). Finally, we visualize the rich latent space of ROUTEFINDER via t-SNE (Van der Maaten & Hinton, 2008) and compare it to those from MTPOMO and MVMoE, where we find that ROUTEFINDER generates more and better defined clusters, indicating a better-learned representation due the Global Attribute Embeddings (Appendix D.6).
5.3 (RQ3) Finetuning with EAL
We finally evaluate ROUTEFINDER (RF-TE) in few-shot learning, finetuning with our proposed EAL to 32 unseen variants, introducing the multi-depot (MD) and mixed backhauls (MB) attributes. Most of these variants are modeled with a learning approach for the first time in this work. We compare 1) zcro-shot performance of ROUTEFINDER, 2) training a new model from scratch, 3) AL from Lin et al. (2024), which adds new adapter layers while keeping the pre-trained backbone, and 4) our proposed EAL. We train a model from scratch and finetune ROUTEFINDES INDER with AL and EAL similarly to the main experiments, but for 10 epochs and 10,000 instances sampled for each. Table 3 shows that EAL consistently outperforms baselines in few-shot learning, including a 20% relative improvement over AL. We also compare AL. and FAA. at "step 0", i.e, after replacing the new adapter layers. Notably, while AL with the untrained new layers greatly degrades the performance unless further training is performed, EAL maintains the zero-shot performance without additional training, providing a much better starting point.
We conduct additional experiments on zero-shot generalization and finetuning across three different settings of unseen variants in order of difficulty: (a) mixed backhaul, (b) multi-depot, and (c) mixed backhaul & multi- depot. We again train for 10 epochs with 10,000 instances sampled for each epoch, and use ROUTEF INDER models with Transformer Encoder (RF-TE). We show the validation gap trends for these three settings in Figure 7, comparing training from scratch, finetuning with AL, and with EAL. For AL and EAL we continue training from the checkpoints that result from the main experiments in Section 5.1. We can see that EAL, having a much better starting point than AL and training from scratch, clearly dominates the other two methods. This dominance becomes more pronounced with increasing difficulty of the finetuning task from MB to MD, indicating it is suitable method for efficient finetuning to new tasks.
11

---

Published in Transactions on Machine Learning Research (09/2025)
Table 3: Finetuning performance 1,000 with multi-depot (MD) and mixed backhaul (MB) variants. ROUTEFINDER'S EAL maintains the zero-shot performance and performs significantly better than AL (Lin al., 2024).
MDVRPMB
MIXORPMMB
MDVRPMBL
MEVRPMIBTW
MDOVRPMBL
MDOVEPMBIW
MIVRPMBETW
MIDVRPMBETT
Method
Obi
Gop
Obj
Gop
Oljj
Gap
Obj.
Gap
Obj
Gap
Oloj
Gap
Gop
Objj.
Gag
HGS-PyVRR
10.68
7.06
10.71
19.29
7.66
12.96
19.31
12.96
OR Tools
12.22
14.37%
8.88
15.83%
12.25
14.29%
22.39
16.12%
8.87
15.73%
14.19
11.79%
22.43
16.10%
11.49
11.79%%
Zero-diot
14.99
40.80%
10.77
40.67%
15.28
13.275%
28.43
47.83%
10.75
60.6236
18.49
43.14%
28.50
19.625 19.63%
18. 33
13.1796
Train (seratch)
13.12
22.58 82.888
9.57
22.32%
13.24
23. 7355
22.85
18.06%
9.38
22.4456
15.13
16.79%
22.50
18.65%
15.11
16.6055
AL (ste ()
34.12
223 14%
26 35
245.53%
27.41
88%
48.91
155,28%
24.11
236.013%
31.53
144,809
46.50
143.89%
30.38
18%
AL
13.10
22 70%
9.36
22, 14%
13.20
3675
22.00
18.70%
9.38
22.4656
15.28
17.91%
23.02
19.20%
15.
18.7756
(stop 0)
14.99
40.907%
40.677
43.2775
28.43
47.93%
10.78
40.82%
18:49
43.147
49.637%
18.
43.1775
EAL
12.70
18.98%
8.53
11.35%
12.68
18.56%
21.41
11.05%
8.54
11.49%
13.93
7.41%
21.44
11.0955
13.91
7.32%
35
Avenge Gap Acress (N)
BAL
Ilaks (46
EAL.
Al
Al:
Thakk.
30
25-
30>
20-
15-
Averge
20
0
8
10
10
ppochs
(8) Mixed backhaul
(b) Multi-depot
Mixed backhaul multi-depot
Figure Validation gaps averaged across new tasks including unseen features (a) mixed backhaul (MB), (b) multi-depot (MD), and (c) their combination (MR&MD) for retraining from scratch, AT. and EAT. finetuning.
In Appendix D.7 we provide more detailed results on finetuning for unseen variants as well as additional results finetuning with FAL for single-variant models. These results show that finetuning ROUTEFINDER to unseen attributes achieves better results than finetuning singlc-variant models based on POMO. This is strong argument for foundation models in routing, the ability to quickly adapt to new tasks is critical in world routing problems.
6 Conclusion
In this work, we presented ROUTEFINDER, comprehensive framework to develop foundation models for VRPs. Extensive evaluations on 48 VRP variants showed that ROUTEFINDER outperforms state-of-the-an neural basclines. ROUTEFINDER represents an carly attompt to learn a foundation model across VRP variants. While demonstrating strong generalization, it does so at a slight expense in solution quality for in-distribution results compared models trained on single variants. For future work, we plan to extend ROUTEFINDER support further variants in the vast VRP literature, We also intend improve the model with exciting research directions, including decomposition methods (Ye et al., 2024c; Zheng et al., 2024) and end-to-end construction and improvement. (Kong et al., 2024).
Acknowledgements
We are deeply grateful to the members of the AI4CO open research community for their invaluable contributions to ROUTEFINDER and related projects, including RL4CO. Our thanks also extend to OMELET for providing additional computing resources. This work was supported by the Institute of Information & Commuiccations Technology Planning & Evaluation (ITPP) grant, funded by the Korean government. (MSIT) [Grant No. 2022-0-01032, Development of Collective Collaboration Intelligence Framework for Internet of Autonomous Things]. Nayeli Gast Zepeda and André Hottung received support from the Deutsche Forschungsgomeinschafff (DFG, German Research Foundation) under Grant No. 521243122. We also gratefully acknowledge the Paderborn Center for Parallel Computing (PC²) for providing valuable computing time for this project.
12

---

Published in Transactions on Machine Learning Research (09/2025)
References
Josh Abramson, Jonas Adler, Jack Dunger, Richard Evans, Tim Green, Alexander Pritzel, Olaf Ronneberger, Lindsay Willmore, Andrew J Ballard, Joshua Bambrick, et al. Accurate structure prediction of biomolecular interactions with AlphaFold 3. Nature, pp. 1-3, 2024.
Luca Accorsi and Daniele Vigo. A fast and scalable heuristic for the solution of large-scale capacitated vehicle routing problems. Transportation Science, 55(4):832-885 856, 2021.
Luca Accorsi and Daniele Vigo. Routing one million customers in a handful of minutes. Computers & Operations Research, 164:106562, 2024.
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, lgge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint urXiv:2303.0877 2023.
The Jin Ai and Voratas Kachitvichyamukul. A particle swarm optimization for the vehicle routing problem with simultaneous pickup and delivery. Computers & Operations Research, 36(5):1693 1702, 2009.
Sanjeev Arora, Wei Hu, and Pravesh K Kothari. An analysis of the t-sne algorithm for data visualization. In Conference on learning theory, pp. 155--462. PMLR, 2018.
Mustafa Avci and Seyda Topaloglu. An adaptive local search algorithm for vehicle routing problem with simultaneous and mixed pickups and deliveries. Compuueers & Industrial Engineering, 83:15-29 2015.
Ahmad Bdeir, Jonas K Falkner, and Lars Schmidt-Thieme Attention, filling in the gaps for generalization routing problems. Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 505-520. Springer, 2022.
Irwan Bello, Hieu Pham, Quoe V Le, Mohammad Norouzi, and Samy Bengio. Neural combinatorial optimization with reinforcement learning. arXiv preprint Niw:1611.09940 2016.
Yoshua Bengio, Andrea Lodi, and Antoine Prouvost. Machine learning for combinatorial optimization: a methodological tour d'horizon. European Journal of Operational Research, 290(2):405-421, 2021.
Federico Berto, Chuanbo Hua, Junyoung Park, Laurin Luttmann, Yining Ma, Fanchen Bu, Jiarui Wang, Haoran Ye, Minsu Kim, Sanghyook Choi, Nayeli Gast Zepeda, André Hottung, Jianan Zhou, Jicyi Bi, Yu Hu, Fei Liu, Hyeonah Kim, Jiwoo Son, Haeyeon Kim, Davide Angioni, Wouter Kool, Zhiguang Cao, Jie Zhang, Kijung Shin, Cathy Wu, Sungsoo Ahn, Guojie Song, Changhyun Kwon, Lin Xie, and Jinkyoo Park. RIACO: an Extensive Reinforcement Learning for Combinatorial Optimization Benchmark. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2025.
Jieyi Bi, Yining Ma, Jiahai Wang, Zhiguang Cao, Jinbiao Chen, Yuan Sun, and Yeow Meng Chee. Learning generalizable models for vehicle routing problems via knowledge distillation. Advonces in Neural Information Processing Systems, 35:31226 31238, 2022.
Aigerim Bogyrbayeva, Meraryslan Meraliyev, Taukekhan Mustakhov, and Bissenbay Dauletbayev. Learning to solve vehicle routing problems: A survey. IEEE Transactions on Intelligent Trunsportation Systems, 2024.
Léo Boisvert, Hélène Verhaeghe, and Quentin Cappart. Towards a Generic Representation of Combinatorial Problems for Learning-Basern Approaches. In International Conference on the Integration of Constraint Programming, Artificial Intelligence, and Operations Research, pp. 99-108. Springer, 2024.
Felix Chalumeau, Shikha Surana, Clément Bonnet, Nathan Grinsztajn, Arnu Pretorius, Alexandre Laterre, and Tom Barrett. Combinatorial optimization with policy adaptation using latent space scarch. Advances in Neural Information Processing Systems, 36, 2024.
Xinwei Chen, Yurui Li, Yifan Yang, Li Zhang, Shijian Li, and Gang Pan. Extnco: A Fine-Graincl Divide- and-Congun Conquer Approach for Extending Nco to Solve Large-Seale Traveling Salesman Problem. Available at SSRN 4679497, 2024.
13

---

Published in Transactions on Machine Learning Research (09/2025)
Jinho Choo, Yeong-Dae Kwon, Jihoon Kim, Jeongwoo Jac, André Hottung, Kevin Tierney, and Youngjune Gwon. Simulation-guided beam search for neural combinatorial optimization. Advances in Neural Informa- tion Processing Systems, 35:8760 8772, 2022.
Tri Dao. Fltettott::: Faster Attention with Better Parallelism and Work Partitioning. International Conference on Learning Representations, 2024.
Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory-efficiet exact attention with io-awareness. Advances in Neural Information Processing Systems, 15:16344-16355 2022.
Yann N Dauphin, Angela Fan, Michael Auli, and David Grangier. Language modeling with gated convolutional networks. In International conference on machine learning, pp. 933 941. PMLR, 2017.
Darko Drakulic, Sofia Michel, Florian Mai, Arnaud Sors, and Mean-Mare Andrcoli. BQ-NCO: Bisimulation Quotienting for Efficient Neural Combinatorial Optimization. Advances in Neural Information Processing Systems, 36, 2024.
Darko Drakulic, Sofia Michel, and Jean-Mare Andreoli. GOAL: A Generalist Combinatorial Optimization Agent Learning. In International Conference Learning Representations, 2025.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprinl arXiv:2407.21733 2024.
Stefan Flfwing, Fiji Uchibe, and Kenji Doya. Sigmoid-weighted linear units for neural network function approximation in reinforcement learning. Newral networks, 107:3-11, 2018
William Fedus, Jeff Dean, and Barret Zoph. A review of sparse expert models in deep learning, arXiv preprint 0rXiv:2209.01660 2022.
Zhang- Hua Fu, Kai-Bin Qiu, and Hongyuan Zha. Generalize a small pre-trained model to arbitrarily large tsp
instances. Tn Proceedings of the AAAT conference on artificial indelligence, volume 35, pp. 7474 7474-7482, 2021.
Chengrui Gao, Haopu Shang, Ke Xue, Dong Li, and Chao Qian. Towards generalizable neural solvers for vehicle routing problems via ensemble with transferrable local policy. IJCAI, 2024.
Maxime Gasse, Didier Chételat, Nicola Ferroni, Laurent Charlin, and Andrea Lodi. Exact combinatorial optimization with graph convolutional neural networks. Advonces in neural information processing systems, 32, 2019.
Mare Goetschalekx and Charlotte Jacobs- Blecha. The vehicle routing problem with backhauls. Europeon Journal of Operational Research, 42(1):33 51, 1989. ISSN 0377-2217 doi: 10.1018/0377-22117))))0000
Nathan Grinsztajn, Daniel Furelos- Furelos-Blanco, Shikha Surana, Clément Bonnet, and Tom Barrett. Winner Takes It All: Training Performant RL Populations for Combinatorial Optimization. Advances in Neural Information Processing Systems, 36, 2024.
Keld Helsgaun. An extension of the Keaaenggggaannnnlaaa TSP solver for constrained traveling salesman and vehicle routing problems. Roskilde: Roskilde University, 12:966-980, 2017.
André Hottung and Kevin Tierney. Neural large neighborhood search for the capacitated vehicle routing problem. European Conference Artificial Intelligence, 2020.
André Hottung, Yeong-Dae Kwon, and Kevin Tierney. Efficient active search for combinatorial optimization problems. International Conference on Learning Representations, 2022.
André Hottung, Mridul Mahajan, and Kevin Tierney. PolyNet: Learning Diverse Solution Strategies for
Neural Combinatorial Optimization. International Conference Learning Representations, 2025a.
14

---

Published in Transactions on Machine Learning Research (09/2025)
André Hottung, Paula Coung, and Kevin Tierney. Neural Deconstruction Search for Vehicle Routing Problems. Transactions on Machine Learning Research, 2025b.
Qingchun Hou, Jingwei Yang, Yiqiang Su, Xiaoqing Wang, and Yuming Deng. Generalize learned heuristics to solve large-scale vehicle routing problems in real-time. In The Eleventh International Conference Learning Representations, 2022.
J. Stuart Hunter. The Exponentially Weighted Moving Average, Journal of Quality Technology, 18(4):203 210, 1986. doi: 00000000022240.1186.111101
Xia Jiang, Yaoxin Wu, Yuan Wang, and Yingqian Zhang. UNCO: Towards unifying neural combinatorial optimization through large language model. arxiv preprint arXiv::408. 12214, 2024a.
Zixuan Jiang, Jiaqi Gu, Hanqing Zhu, and David Pan. Pre-RMSNorm and CRMSSNrr transformers: equivalent and efficient Pre-LN transformers. Advances in Neural Information Processing Systems, 36, 2024b.
Chaitanya K Joshi, Quentin Cappart, Louis-Martin Rousseau, and Thomas Laurent. Learning the travelling salesperson problem requires rethinking generalization. Constrainds, 27(1):70-99) 98, 2022.
Saão Karakatië and Vili Podgorelec. A survey of genetic algorithms for solving multi depot vehicle routing problem. Applied Soft Computing, 27:519-532 532, 2015.
Hyeonah Kim, Jinkyoo Park, and Changhyun Kwon. A neural separation algorithm for the rounded capacity inequalities. INFORMS Journal on Computing, 36(4):987- 1005, 2024.
Minsu Kim, Jinkyoo Park, et al. Learning collaborative policies to solve NP-hard routing problems. Advances in Neural Information Processing Systems, 34:10418-10430, 2021.
Minsu Kim, Junyoung Park, and Jinkyoo Park. Sym-NCO: Leveraging symmetricity for neural combinatorial optimization. Advances in Newral Information Processing Systems, 35:1936-1949, 2022.
Minsu Kim, Sanghyeok Choi, Hyeonah Kim, Jiwoo Son, Jinkyoo Park, and Yoshua Bengio. Ant colony sampling with gflownets for combinatorial optimization. In AISTATS, 2025.
Diederik Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization. In International Conference on Learning Representations (ICLR), San Diega, CA, USA, 2015.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVE International Conference on Compuler Vision, pp. 4015 4026, 2023.
Çagn Koç, Gilbert Laporte, and ilkmur Tükenmez. A review of vehicle routing with simultaneous pickup and delivery. Computers & Operations Research, 122:104987, 2020.
Çagri Koç and Gilbert Laporte. Vehicle routing with backhauls: Review and research perspectives. Compulers 8 Operations Research, 91:79 91, 2018. ISSN 0305-0548. doi: 10. 10.1016/j..or.2017.. 2017..1......
Detian Kong, Yining Ma, Zhiguang Cao, Tianshu Yu,, and Jianhua Xiao. Efficient Neural Collaborative Search for Pickup and Delivery Problems. IEFE Transactions on Pattern Analysis and Machine Intelligence, 2024.
Wouter Kool, Herke Van Hoof, and Max Welling. Attention, learn to solve routing problems! International Conference on Learning Representations, 2019.
Wouter Kool, Herke van Hoof, Joaquim Gromicho, and Max Welling. Deep policy dynamic programming for vehicle routing problems. In International conference on integration of constraint programming, artificial intelligence, and operations research, pp. 190-213. Springer, 2022.
Jerome Ku, Eric Nguyen, David W Romero, Garyk Brixi, Brandon Yang, Anton Vorontsov, Ali Taghibakhshi, Amy X Lu, Dave P Burke, Greg Brockman, et al. Systems and Algorithms for Convolutional Multi-Hybria Language Models at Scale. arXiv preprint ar Xri:22003.00 01868, 2025.
15

---

Published in Transactions on Machine Learning Research (09/2025)
Yeong-Dae Kwon, Jinho Choo, Byoungjip Kim, Iijoo Yoon, Youngjune Gwon, and Seungjai Min. POMO: Policy optimization with multiple optima for reinforcement learning. Advances in Neural Information Processing Systems, 33:21188 21198, 2020.
Feiyue Li, Bruce Golden, and Edward Wasil. The open vehicle routing problem: Algorithms, large-scale test. problems, and computational results. Computers & Operations Research, 34(10):2918 2930, 2007. ISSN 0305-0548. doi: https://doi.org 10.1016/j.cor. 200....0000.........
Sirui Li, Zhongxia Yan, and Cathy Wu. Learning to delegate for large-scale vehicle routing. Advonces in Neural Information Processing Systems, 34:26198-2663 2021.
Ivan Lima, Eduardo Uchoa, Oliveira, and E Queiroga. CVRPLIB: Capacitated vehicle routing problem library. Date accessed, 8(02):2022, 2014.
Zhuoyi Lin, Yaoxin Wi, Bangjian Zhou, Zhiguang Cao, Wen Song, Yingqian Zhang, and Senthilnath Jayavelu. Proos-Proolem Learning for Solving Vehicle Routing Problems. IJCAI, 2024.
Fei Liu, Xi Lin, Qingfu Zhang, Xialiang Tong, and Mingxuan Yuan. Multi-Task Learning for Routing Problem with Cross-Problem Zero-Shot Generalization. Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2024a.
Fei Liu, Xialiang Tong, Mingxuan Yuan, Xi Lin, Fu Luo, Zhenkun Wang, Zhichao Lu,, and Qingfu Zhang, Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Model. In International Conference on Machine Learning, 2024b.
Fei Liu, Xialiang Tong, Mingxuan Yuan, Xi Lin, Fu Luo, Zhenkun Wang, Zhichao Lu, and Qingfu Zhang, Evolution of Heuristics: Towards Efficient, Automatic A Algorithm Design Using Large Language Mode. In ICML, 2024c.
Shengeai Liu, Yu Zhang, Ke Tang, and Xin Yao. How good is neural combinatorial optimization? A systematic evaluation on the traveling salesman problem. IEEE Compudational Intelligence Magazine, 18(3):14 28, 2023.
Fu Luo, Xi Lin, Fei Liu, Qingfu Zhang, and Zhenkun Wang. Neural combinatorial optimization with heavy decoder: Toward large scale generalization. Advances in Neural Information Processing Systems, 36, 2024.
Fu Luo, Xi Lin, Zhenkun Wang, Tong Xialiang, Mingxuan Yuan, and Qingfu Zhang. Boosting Neural Combinatorial Optimization for Large-Scale Vehicle Routing Problems. International Conference on Learning Representations, 2025.
Yining Ma, Jingwen Li, Zhiguang Cao, Wen Song, Le Zhang, Zhenghua Chen, and Jing Tang. Learning to iteratively solve routing problems with dual-aspect collaborative transformer. Advances in Neural Information Processing Systems, 34:11096-11107 2021.
Yining Ma, Jingwen Li, Zhiguang Cao, Wen Song, Hongliang Guo, Yuejiao Gong, and Yeow Meng Chee. Efficient Neural Neighborhood Search for Pickup and Delivery Problems. International Joint Conference on Artificial Intelligence, 2022.
Yining Ma, Zhiguang Cao, and Yeow Meng Chee. Learning to search feasible and infeasible regions of routing problems with flexible neural k-opt. Advances in Neural Information Processing Systems, 36, 2024.
Nina Mazyavkina, Sergey Sviridov, Sergei Ivanov, and Evgeny Burnaev. Reinforcement learning for combina- torial optimization: A survey. Computers & Operations Research, 134:105400, 2021.
Mohammadreza Nazari, Afshin Oroojlooy, Lawrence Snyder, and Martin Takác. Reinforcement learning for solving the vehicle routing problem. Aduances in neural information processing systems, 31, 2018.
Eric Nguyen, Michael Poli, Matthew G Durrant, Armin W Thomas, Brian Kang, Jeremy Sullivan, Madelena Y Ng, Ashley Lewis, Aman Patel, Aaron Lou, et al. Sequence modeling and design from molecular to genome scale with Evo. bio RioRi pp. 2024-00 02, 2024.
16

---

Published in Transactions on Machine Learning Research (09/2025)
Yun Peng, Byron Choi, and Jianliang Xu. Graph learning for combinatorial optimization: survey of state-of-the-art. Dala Science and Engineering, 6(2): 2021.
Laurent Perron and Frédéric Didier. CP-SAT, 2024.
Laurent Perron and Vincent Furnon. OR-Tools. Google, 2023.
Antoine Prouvost, Justin Dumouchelle, Lara Scavuzzo, Maxime Gasse, Didier Chételat, and Andrea Lodi. Ecole: Gym-like Library for Machine Learning in Combinatorial Optimization Solvers. In Learning
Meets Combinatorial Algorithmss at NeurIPS2020, 2020.
Bernardino Romera-Paredes, Mohammadamin Barckatain, Alexander Novikov, Matej Balog, M Pawan Kumar, Emilien Dupont, Francisco JR Ruiz, Jordan S Ellenberg, Pengming Wang, Omar Fawzi, al. Mathematical discoveries from program search with large language models. Nature, (25(7995):468-444
2024.
Stefan Ropke and David Pisinger. A unified heuristic for large class of Vehicle Routing Problems with Backhauls. European Journal of Operational Research, 171(3))750-775 2006. doi: 010016//.cjor22004.9.00000
Frank Ruis, Gertjan Burghouts, and Doina Bucur. Independent protolype propagation for zero-shot composi- tionality. Advances in Neural Information Processing Systems, 34:10641-1065x 2021.
Noam Shazeer. Glu variants improve transformer. arXiv preprint urXiv:2002.05202 2020.
Marius M Solomon. Algorithms for the vehicle routing and scheduling problems with time window constraints. Operations reseorch, 35(2):254-200 265, 1987.
Zhiqing Sun and Yiming Yang. Difusco: Graph-based diffusion solvers for combinatorial optimization. Advances in Neural Information Processing Systems, 36, 2024.
Daniela Thyssens, Tim Dernedde, Jonas K Falkner, and Lars Schmidt-Thieme. Routing Arena: A Benchmark Suite for Neural Routing Solvers. arXiv preprint ar axXiv22310004140, 2023.
Laurens Van der Maaten and GeofTrey Hinton. Visualizing data using i-SNE. Journal of machine learning research, 9(11), 2008.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Thibaut Vidal. Hybrid genetic search for the CVRP: Open-source implementation and SWAP* neighborhood.
Computers & Operations Research, 140:105643, 2022.
Thibaut Vidal, Teodor Gabriel Crainic, Michel Gendreau, and Christian Prins. A unified solution framework for multi-attribute vehicle routing problems. European Journal of Operational Research, 234(3):658-677 673, 2014. ISSN 0377-2217. doi:
Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. Pointer networks. Advances in neural information processing systems, 28, 2015.
Chenguang Wang and Tianshu Yu. Efficient training of multi-task neural solver with multi-armed bandits. Transactions on Machine Learning Research, 2025.
Niels A Wouda and Leon Lan. ALNS: A Python implementation the adaptive large neighbourhood search metaheuristic. Journal of Open Source Software, 8(81):5028, 2023.
Niels A Wouda, Leon Lan, and Wouter Kool. PyVRP: A pafo-ncee VRP solver package. INFORMS Journal Computing, 36(4):943 955, 2024. doi: 10. 2.1287/jjoc.0023.005.
17

---

Published in Transactions on Machine Learning Research (09/2025)
Xuan Wu, Di Wang, Lijie Wen, Yubin Xiao, Chunguo Wu, Yuesong Wu, Chaoyu Yu, Douglas L Maskell, and You Zhou. Neural combinatorial optimization algorithms for solving vehicle routing problems: A comprehensive survey with perspectives. arXiv preprint (Xx::446.00415, 2024.
Haoran Ye, Jiarui Wang, Zhiguang Cao, Federico Berto, Chuanbo Hua, Haeyeon Kim, Jinkyoo Park, and Guojie Song. ReEvo: Large Language Models Hyper-Heuristies with Reflective Evolution. In Advances in Newral Information Processing Systems, 2024a.
Haoran Ye, Jiarui Wang, Zhiguang Cao, Helan Liang, and Yong Li. DeepACO: Neural-cnhanced Ant Systems for Combinatorial Optimization. Advances in Newral Information Processing Systems, 36, 2024b.
Haoran Ye, Jiarui Wang, Helan Liang, Zhiguang Cao, Yong Li, and Fanzhang Li. GLOP: Learning global partition and local construction for solving large-scale routing problems in real-time. AAAI 2024. 2024c.
Biao Zhang and Rico Sennrich. Root mean square layer normalization. Advances in Newral Information Processing Systems, 32, 2019.
Zhi Zheng, Changliang Zhou, Tong Xialiang, Mingxuan Yuan, and Zhenkum Wang. UDC: A Unified Neural Dividc-and-Conquee Framework for Large-Scale Combinatorial Optimization Problems. Advances in Neural Information Processing Systems, 2024.
Jianan Zhou, Yaoxin Wu, Wen Song, Zhiguang Cao, and Jie Zhang. Towards ommi-generalizable neural methods for vehicle routing problems. In International Conference on Machine Learning, pp. 42769-42789.
PMLR, 2023.
Jianan Zhou, Zhiguang Cao, Yaoxin Wu, Wen Song, Yining Ma, Jie Zhang, and Chi Xu. MVMoE: Multi-Tass Vehicle Routing Solver with Mixture-of Experts. In International Conference on Machine Learning, 2024.
18

---

Published in Transactions on Machine Learning Research (09/2025)
A Unified VRP Environment Details
We consider the seven attributes from Section 3.1 for instance generation through our environment definition explained Section 4.1, Leveraging our environment's modular structure, we build the VRP variants as used in MVMoE (Zhou al., 2024), but by differentiating between traditional (B) and mized (MB) backhauls, defined in Avei & Topaloglu (2015), we extend that number to 24. By considering multi-depot problems, we further increase that number to 48 variants that can be solved with ROUTEFINDER (see Table 4).
Table 4: The 48 VRP variants consider. All variants include the base Capacity (C). The k 5 features O, B, L, TW, and MD be combined into any subset, including the empty and itself (i.e., power set) with 2 possible combinations. variants.
The Mixed (M) global feature creates Mixed Backhaul (MB) variants generalization studies, adding
Capacity (C)
Open Route
Backhaul (B)
Mixed (M)
Duration Limit (L)
Time Windows (TW)
Multi-depot
VRP Variont
(0)
(MD)
CVRP
OVRP
VRPB
VRPL
VRPTW
OVRPTW
OVRPB
OVRPL
VRPBI.
VRPBTW
OVRPBL
OVRPBTW
OVRPLTW
VRPBLTW
OVRPBLTW
VRPMB
OVRPXB
VRPMBL
VRPMBTW
OVRPMBL
OVRPMETW
VRPMBLTW W
OVRPMBLTW
MDCVRP
MDOVRP
MDVRPB
MDVRPL
MDVRPTW
"
MDOVRPTW
MDOVRPB
MDOVRPL,
MDVRPBL
MDVRPBTW
MDVRPLIW
MDOVRPBL
MIDORPBTWW
MDOVRPLTW
MDVRPBLTW
MDOVRPELTW
MDVRPMB
MDOVRPMB
MDVRPMBL
MDVRPMETW
MIDORPMMEL
MDOVRP' MBTW
MDVRPMBIIT
MDOVRPMBLTW
We describe additional details the Unified VRP environment, including data generation in Appendix A.1 and environment logic in Appendix A.2. For better understanding, we invite the reader look the source code, which tried our best to comment. for clarity, at https://githut. http://ttb..com//acc////oreeeee
19

---

Published in Transactions on Machine Learning Research (09/2025)
A.1 Data generation
We now explain the individual steps in the data generation process use for our modular VRP environment, including the node attributes and global attributes.
While throughout the main part of this paper, we have focused on routing problems with a single depot, our unified environment can actually handle problems with multiple depots, where we define m as the number of depots. For comparability to the neural baselines, the main experiments were run on single-depot problems, but we report results for multi-depot problems (Appendix D.7).
Locations We generate m + n locations randomly with 2; and y/ ~ (0, 1), (0, m + ---nn11 11,, where (s,, Vil, (0.-.., }} denote the depols and [dl E {m, m 1}, the n customer nodes. Note that this setting can be expanded to consider more realistic distributions as in (Bi et al., 2022; Zhou et al., 2023; Gao et al., 2024), and our implementation is already set up in such way to allow for different distributions in the future via the get_sampler method.
Multiple depots (MD) Depot nodes, in principle, have the samne node attributes customer nodes. The location, however, is the only attribute that is generated in the same way, which is explained below. For all other attributes, the values are fixed and identical for all depots. Linehaul and backhaul demands, as well as service durations, are set to zcro, while the time windows of all depots in an instance are set to [ei, = [0, m E (O..., m - 1}, where lmaz denoles the system end time and M the number of depots. For problems without time windows, max set to co. In the unseen variants experiments of Appendix D.7, employ m depots for the MD finctuning variants.
Vehicle capacity (C) The vehicle capacity C is a fixed value applied to all vehicles and calculated according to:
30
+
1000
n n-1000 33.3
if 1000 < n if 20 < n S 1000 othwewise
5
C
30
+
n
30
which is commonly used in NCO for VRP approaches (Kool et al., 2019; Kwon al., 2020).
Linehaul and backhaul demands (C) / (B) (MB) We generale demands according to the following scheme:
1. Generate linehaul demands q for all customers i E N₂ by sampling uniformly from the set of integers (1,2...., 9}.
Generate backhaul demands p; for all customers E N₂ by sampling uniformly from the set of integers {1, 11.,,...,9) 9}.
For each customer i E Ne, generate a temporary decision variable 2; E (0, with probabilities P(z: 0) 0.8 and PP(z; = 1) 0.2.
Tf 2; = 0, keep the linehaul demand q and set the backhaul demand pt = 0.
If 24 1, set the linehaul demand q: = 0 and keep the backhaul demand p;-
This demand generation scheme ensures that each customer has either linehaul demand backhaul demand, but not both. With probability of 0.8, customer will have only linchaul demand, and their backhaul demand will be set to O. Conversely, with a probability of 0.2, a customer will have only a backhaul demand, and their linehaul demand will be set to O. It is important to note that not all customers are typically backhaul customers, even a backhaul setting. Therefore, this scheme allows for the consideration of both linchaul and backhaul demands in backhaul problem settings while ensuring that each customer has only one type of demand.
20

---

Published in Transactions on Machine Learning Research (09/2025)
We note that this can be easily extended to the case of VRP with simultaneous pickup and delivery (VRPSPD), in which a customer can have both linehaul and backhaul demand (Ai & et al., 2020). In such a case, we could duplicate the customer node into two nodes with the same attributes, such as locations, but different values for linchaul (pickup) and backhaul (delivery) in the current VRP environment or allow for both linehaul and backhaul to be present at the same time a single node with small modifications in the action masking-
2009; Kog
Backhaul class (B) / (MB) For testing the few-shot setting described in Section 5.3, generate instances with mized backhauls. The instances themselves are actually identical to instances with the traditional backhaul, and we use a global attribute in the instance to differentiate between them. For this purpose, we allow either setting a fixed value & (1,2} or sampling from [1,2} for every customer with equal probabilities p(1) = p(2)= = 0.5, allowing for different backhaul settings within one batch, if needed (see the batching procedure described in Scction 4.3.1). Note that we sample from {1,2] instcad of boolean sampling because we plan to exlend the number of backhaul settings in the future.
Open routes (0) For open routes, generate a boolean vector with all True values. During sampling (see Section 4.3.1), the actual ratio of open route instances is defined, not at the initial instance generation (iee,, we temporarily change the True value to False for every batch element with a certain probability).
Time Windows (TW) We generate the time windows [ea, L] and service times 8, in several steps for all customers E Ne:
1. Generate service times E [0.15, 0.18).
2. Generate time window lengths t, [0.18, 0.2].
3. Calculate the maximum distance from any of the depots E (0, 1} to customer i: dmaz= maxy(dy)).
4. Calculate upper bounds for time window start times h; fmee- 81
dmax
Calculate time window start times as e; = (1 + (h: - 1) uz) dmao with us (0,
Calculate time window end times as 4 e, + t4.
When calculating the action mask, we have the constraint that the expected arrival time should be earlier than the end time of nodes; if the problem is a closed problem, we should also consider the time back to the depot, ie., max(teur + ez) + sj + dmoz < lo. We note that for simplicity, we set the vehicle speed to 1.0 in equations and normalize time windows accordingly so that travel time from two nodes is the same numerically as the distance between them. This can be easily modified in the code.
We mention an alternative TW generation procedure the one from the Solomon benchmark (Solomon, 1987; Li et al., 2021), which may perform better in that benchmark, done in Zhou et al. (2024).
Distance limit (L) The distance limit is sampled from uniform distribution to ensure meaningful and feasible constraints. Specifically, we sample L from U(2 max(doi), {mxx)), where dai is the distance from the depot to customer i, and Imax = 3.0 is a predefined upper bound. This approach ensures that L is always greater than the round trip to the farthest customer (2 max max()):)), making all customers reachable, while also allowing for variation in the constraint tightness. For the multi-depot case we replace max(do) with (max;(() E {m....,
n}.j E [0....,,, i.e., first get the maximum distance from any customer node to each of the depots and then take the minimum out of those distances. By taking the maximum in the first step we ensure that all customers are reachable, and by taking the minimum across depots, we make the problem more challenging, because even though all nodes can in principle be serviced, some may only be serviced by one (or a subset) of the available depots. This sampling method produces more variation than previous works Liu al. (2024a); Zhou et al. (2024) (where there was virtually no difference in solutions of (L) and non- (L) variants), as it guarantees feasible instances while still providing a range of challenging scenarios.
21

---

Published in Transactions on Machine Learning Research (09/2025)
Attribute Normalization and Scaling All demands, both linchauls and backhauls, are scaled to lie in [0,1] through division by the vehicle capacity. 1/ q: ///C,14 = Pt/C. All other features are already sampled from a normalized range. Note that during loading instances from e.g. CVRPLib, we normalize features before passing them to the policy for instance, locations are normalized between 0 and 1.
A.2 Environment Logic
To determine available actions for the Unified VRP environment formulation, the constraints for the individual problems have to be combined in the action mask (action mask in the code following RL4CO, where True means that the action is feasible (Berto et al., 2025)). We build a logical test structure, essentially soparating the checks in the action mask according to the individual VRP problem types and then bringing them all together again. The individual action_mast checks are the following:
a) Can reach in time: depending on the current time and the travel distance to every node not yet visited, we reach that node before its service time window ends? teurr + dij < <bj, where teurr is the current time.
b) Does not exceed dislance limit: depending on the current length the route, we travel to any available node, will we exceed the total distance limit for the route? leurr + diy < L, where leurr is the current length.
c) Can reach depot: there are two types of constraints from time windows (TW) and distance limit (L):
If we need to ensure we can reach the depot in time, i.e, the current time plus traveling time to the depot must be smaller than the system end time: max(feur + diy, + 8, + dyo < tmas.
If need to ensure
reach the depot without excceding the distance limit, i.c., the current distance plus the traveling distance to the depol must be smaller than the distance limit: leurr + dej + dio < L.
For the multi-depot case we replace dyn in both these constraints with djs, where k E {0, m 1} indexes the depot the current route started from. For open routes, this will always be set to True, ie., this constraint does not apply.
d) Demand constraints for backhaul problems:
Checks for all backhauls problems:
Docs the linchaul demand excood vehicle capacity if we add node's demand to the current vehicle? Ceurr + Qý C, where Ceur is the used capacity.
Does the backhaul demand exceed vehicle capacity we add a node's demand to the current vehicle? Ceurr + Pj < C, where Ceurr is the used capacity.
Chocks for traditional backhaul settings:
Carrying backhaul: if we are already picking up backhaul demands, we cannot service any linehaul demands on this route anymore.
If we are not carrying backhaul demands yel, are there any unserved linehaul demands left? If there are no linchaul demands left or we are already carrying backhauls, are there still unserved backhaul demands?
Checks for mized backhaul settings:
Cannot service linehaul demands: depending on the backhaul demands currently loaded in the vehicle, do we have space left for further linehaul demands?
We additionally remark that our definition of backhauls follows the generally accepted definition in the OR community, originally due to Goetschalekx & Jacobs-Blecha (1989). This definition differs from the routing problems with backhaul considered in several recent papers in the machine learning (e.g., Liu et al. (2024a); Zhou et al. (2024)), who define backhaul customers as having a negative demand of the same commodity used for linchaul, and do not consider the precedence constraint that all linchaul must be completed before backhaul may start on the route. The problem setting with a single commodity is not commonly studied in the OR literature since it implies pickups may be used
22

---

Published in Transactions on Machine Learning Research (09/2025)
for deliveries at later customers, while the relaxation of the precedence constraint is more properly referred to a mized backhaul problem (Kog & Taporte, 2018).
e) Already visited: every customer node needs to be visited exactly once.
We bring together checks a) to e) and introduce additional check for the depot: if we are currently in the depot and there are still unserved customers, we cannot select the depot as the next action to ensure the model cannot get stuck during decoding.
For the multi-depot case we further extend this check. If we are currently in a depot and there are unserved customers, we cannot visit any depot. no further customers can be servicod, all depots are available actions again. However, we are currently in a depot and no customers can be served from this depot, we mask it out so as to service the remaining customers from the remaining depots that actually service them.
Combining these checks in this way allows to meticulously check for individual VRP settings while at the same time maintaining the necessary flexibility the unified environment formulation requires.
B RouteFinder Model Details
ROUTEFINDER follows the encoder-decoder architecture from the Attention Model (Kool et al., 2019), transformer-like architecture based on the attention mechanism (Vaswani et al., 2017). We additionally improve the encoder architecture in RF-TE as explained Section 4.2. We focus the explanation on modeling all attributes possible with the MDOVRPMBLTW, noting that in the main training runs, we do so without. considering attributes from multi-depots and mixed backhaul, whose additional paramcters are added upon EAL finetuning.
B.1 Multi-Head Attention
At the core of ROUTEFINDER lies the Multi-Head Attention (MHA) mechanismm proposed by Vaswani el al. (2017). MHA concurrently attends information from various representation subspaces, facilitating the capture of diverse relationships between input elements. Notably, MHA is capable of handling variable nunber of elements.
The MHA operation starts by linearly projecting the input sequences of queries Q, keys K, and values V to distinct subspaces using Iearned projection matrices W.O, W.K, and WV, respectivcly, where H denotes the number of attention heads: Q: Qw?, K, KW.K V, = VWY for i = 1..... H. Subsequently, the attention weights for each head are computed by performing a scaled dot product between the projected queries and keys, followed by softmax operation:
QQ.TT Vdz + M
A₃ = Softmax
where dz represents the dimension of the keys, acting as a scaling factor to prevent the dot products from exp(x;) and M is an optional attention mask that can be used to growing too large, Softmax(z;) Σ exp(ay)
prevent attending to certain positions (c.g., infcasible actions), which can be done by setting elements to -oc. The output of each atlention head is then calculated as a weighted sum of the projected values, using the attention weights: Z, = A.V.
Lastly, the outputs from all attention heads are concatenated and linearly projected using learned matrix o to yield the final output of the MHA operation:
MHA(Q, K,V) = Concat(Z,..... Zz WO
(8)
While the MHA grows quadratically, i.c., with sequence length (i.c., number of nodes) N, grows as O(N2), several efficient implementations have been proposed over the years, and we use FlashAttention (Dao et al., 2022; Dao, 2024) to speed up the model.
23

---

Published in Transactions on Machine Learning Research (09/2025)
B.2 Encoder
The Encoder transforms input instance æ into hidden embedding h. The Encoder architecture consists of the following main components: 1) Global Embedding, 2) Node Embedding, and 3) a series of Encoder Layers. We consider a VRP instance of n locations as having n + nodes, where node 0 is the depot and nodes [I,....n} are n customers. For problems with multiple depots, we define m as the number of depots, 1} are the depot nodes, and m,...,,.. + n 1 are the n customer nodes.
i.e., nodes (O...., -1
Global Embedding Since Global Attributes contain single value for all the + problem nodes, we embed them in depot nodes, in a similar fashion to how traditional solvers as PyVRP encode information about the global problem structure on depot nodes.. Global Embeddings include global attributes Open Routes o E {0. 1), Duration Limits E [0, L), and Mixed Backhauls flag E {0, 1}, as well as the locations of the depot node(s) [zv, yd E IR, e {O...., m - 1} and the system end time max (i.e., the depot(s) time window). In practice, for the multi-depot case with m > 1, the global attributes are projected on the depot nodes. In ROUTEFINDER, the global embedding f is a linear projection layer W, Rxx where k = 6 features and d 128 is the hidden dimension. The initial projected global hidden embedding per depot gi can be written as h Wy Wa yi, /max 0, pA]_.
Node Embedding The node embeddings, the other hand, capture customer-specife attributes and are projected onto the remaining nodes. These attributes include for nodes i E {m,... m + n - 1}: Linehaul demands q E [0, Q), Time Windows paramcters e,, 8;,4, E T1 where e and denote the time window's start and end and is the service time, the Backhaul demands pa E [0, Q), and finally the node locations [r, vi] E IR? In ROUTEFINES FIDER this a linear projection layer W,, E IRd where k = 7 features and d = 128 is the hiddon dimension. The initial projected node hidden embodding can be written for cach node n, as (0) [ Vs. dr C, 8,, P.p.
Raw Features to Hidden States The projected global embedding and node embeddings are concatenated to obtain the initial hidden representation h0))) gim+tjjxd, where m the total number of nodes (m depots customers) and d is the hidden dimension:
(9)
bmh (0) = Concat(h! (
91 ...., h 00 (0) (0)
(10)
The initial hidden representation h (9) is then passed through a series of Encoder Layers to refine and enrich the representation. Each Encoder Layer consists of a Multi-Head Attention (MHA) layer and Multi-Layer
Perceptron (MLP) layer, as described in Equation (12) and Equation (13), respectively.
The Encoder can be represented as:
h = EncoderBlocks((t (0)
(11)
Each EncoderBlock consists of two sub-layers: a Multi- Head Attention (MHA) layer and a Multi-Layer Perceptron (MLP) layer (or SwiGLU as we propose). The MHA layer allows the model to capture dependencies between different. positions in the input sequence, while the MLP layer applies non-linear transformations to the features at each position. The input to each EncoderBlock is first passed through the MHA layer, which computes the self-attention using the input as queries, keys, and values:
(1-1) h Norm (e-1 MH 12, 1e (,C--1,,(0-- (())
(12)
where (f-1) represents the input to the f-th EncoderBlock, and Norm denotes normalization operation, in ROUTEFINDER employ Instance Normalization (IN). The output of the MHA layer, h, is then passed through the MLP layer, which applics a series of linear transformations with non-lincar activations:
( Norm hh + MLP(A)
(13)
The pointwise MLP layer consists of two linear layers with non-linear activation function as ReLU, between them.
24

---

Published in Transactions on Machine Learning Research (09/2025)
Transformer-based Encoder We further explicit proposed Transformer-basedd encoder. Each En- coderBlock consists of two sub-layers: a Multi-Head Attention (MHA) layer and Feed Forward SwiGLU layer (Shazeer, 2020). The MHA layer captures dependencies between different positions in the input sequence, while the SwiGLU layer applics non-linear transformations to the features. We employ RMS normalization (Zhang & Sennrich, 2019) and pre-norm architecture for improved stability and faster convergence:
h (6-1) + -MEA(BMSNor((t(-))) RMSNom (h (t())), (...NNo((hh
(14)
h = h SwiGI.J(((MSNNm(()))
(15)
where h(-11) represents the inpul to the E-th EncoderBlock. The SwiGLU MLP (Shazeer, 2020) is defined as:
SwiGLU(z) = Wa (S(S) @ (W2x)
(16)
where @ denotes the Hadamard product, SiLU is the activation function (Elfwing et al., 2018), and W1, W2 and Wg are learnable weight matrices". We use FlashAttention (Dao et al., 2022; Dao, 2024) in the MHA layer for enhanced performance.
B.3 Decoder
The Decoder autoregressively constructs the solution based on the Encoder output h and the state at the current step t.
Context Embedding The context embodding is used to modify the query embedding of the problem node of the current partial solution. It consists of a linear layer that projects the concatenated current node embedding and state embedding to the embedding space. The state embedding is computed by projecting the following: the current node embedding h, and a set of dynamic features from state 8t, í.e. the available load C, current time 4,, current distance traveled de; the available backhaul load b, i.e. the difference between the vehicle capacity Q and the used backhaul expacily, which is necessary because we pick up items, the deliverable quantity must exceed the remaining capacity after pick up for mixed backhauls (MB) as well the location of the origin depot In FOUTEFINNER the context embedding W. (d+((++)) is linear projection matrix, d = 128 is the hidden dimension, and k = 6 is the number of state features. The context embedding at step is thus computed as h W.Connat((ha. (he [et, le, de, be, 2t, vfl))
have to return to at step t: [ag vfl for the multi-depot variants (MD).
Attention and Pointer Mechanism The query qe obtained directly from the context embedding
a: (1) and then passed into a masked MHA layer and final single-head attention to obtain logits z:
hg MHA(q, KP.V8, M,),
(17)
VPhgg
(18)
ddL.
where M, is the set of feasible actions (i.e., the action sasiion_mssk,.. and projections K7, V2, V? Wgh, Wyh, With are precomputed once as cache. We note that Equation (18) is usually referred to the pointer mechanism (Vinyals et al., 2015).
Logits processing Finally, logits z are transformed into a probability distribution:
p = Softmax (C tanh(z))
(19)
where logits for infeasible actions can be masked, and C is the tanh clipping that serves improving the exploration, which we set to 10 according to Bello et al. (2016).
2In our code this the ParallelGatedNi inspired by the StripedHyena architectures (Ku et al., 2025).
25

---

Published in Transactions on Machine Learning Research (09/2025)
Action selection During training, use the POMO multistart sampling.
For the multi-depot case we force the first action to start from all depots in the instance. For the single-depot case we force the first action to start with every customer node to maximize diversity. Note that if num starts is not divisible by the number of depots m, the resulting tensor will not have an equal mumber indices for each depot, i.e., the number of starts will not be distributed evenly across the depots, as we use the modulo operator for the assignment.
During testing, we also employ multistart but with greedy selection (i.e., selecting the maximum probability). Prior to the selection, dihedral augmentation is also performed prior to encoding instance æ in the encoder, which enables exploring 8x as many solutions with 4 rotations x 2 flips. We note that additional augmentations and techniques can be performed during inference, which can further boost evaluation performance (Kim et al., 2022; Ma et al., 2022; Choo et al., 2022; Luo et al., 2024). For fairness of comparison, we do not employ additional augmentations but assume that this could further boost the performance of ROUTEFINDER.
B.4 EAL Modeling
We describe in more detail the procedure for Efficient Adapter Layers (EAL) modeling. Our initial model trained from Section 5.1 has linear projections layers as referenced in full detail in Appendix B.2 and Appendix B.3 without additional parameters for mixed backhaul and multi-depots.
B.4.1 EAL for mixed backhauls
This adds, as explained in Section 4.4, a single (1 = 1) parameter row W for the mixed backhaul flag μ to the global embedding. Moreover, we add / 1 rows for the context embedding resulting W for the available backhaul load at step i.e. the difference between the vehicle capacity and the used backhaul capacity.
B.4.2 EAL for multi-depor
In this case, we do not modify the global embedding but directly project multiple times global altributes and depot locations at each depot node as explained in Appendix B.2. However, we modify the context. embedding W by adding = 2 rows keep track of the location of the origin depot we have to return at step [xf.v[].
B.4.3 EAL for multi-depots & mixed backhauls
Here we combine the EAL implementations of the previous two paragraphs. We add the = parameter row W for the mixed backhaul flag μ to the global embedding and project the global embedding m according to the number of depots and modify the context embedding W by adding I = 3 rows to keep track of the available backhaul load by and the location of the origin depot [f,
C Additional Material
C.1 Details for Average Batch Reward for Multi-task Reward Normalization
each training step I,...,T we train on a batch of b = I...., problem instances, each of which belongs to one of the K problem variants covered by ROUTEFINDER. Let 1s, E (0, 13 be an indicator function such that:
[1 if instance b is of type k
1
0 otherwise
(k) which is efficiently calculated in our unified VRP environment based on vectorized checks, The reward rir for instance b of variant & training step & can then be expressed as Tbt Fbt 1 The average batch
(k)
26

---

Published in Transactions on Machine Learning Research (09/2025)
reward F(()) for variant at training step i over all instances of type in batch can then be expressed as:
(k)
Σ b=
(k)
E
16,,k
Ak E K.
=== 16,k
b==
16,
This average batch reward Ft is the basis for the reward normalization explained in Section 4.3.2.
C.2 Hyperparameter Details
We report in Table 5 the hyperparameter details common across the main experiments. ROUTEFINDER variants additionally employ the proposed contributions as outlined in the main experiments of Section 5.1.
Table 5: Experiment hyperparameters. Values with */" indicate different choices depending on the model, Lee, on the right are values the Transformer Baseed encoder.
Hyperparameter
Value
Model
Embedding dimension
128
Number attention heads
Number encoder layers
Use Pre-norm
False True
Normalization
Instance / RMSNorm
Feedforward hidden dimension
512
Feedforward structure
MLP Gated MLP ReLU / SwiGLU
Feedforward activation
Tanh clipping
10.0
Mask logits
Truc
Training
Train decode type
multistart sampling
Val & Test decode type Augmentation function
multistart greedy
dihedral
Batch size
256
Train data per epoch
100,000
Reward normalization
Exponentially smoothed mean
Normalization
0.25
Optimization
Optimizer
Adam
Learning rate
3e-4
Weight decay
1e-6
LR scheduler
MultiStepLR
LR milestones
[270, 295]
LR gamma
0.1
Gradient clip value
1.0
Max epochs
300
C.3
Additional Discussion
Motivation Foundation models have been successful in several areas in recent years, including large language models (Achiam et al., 2023), computer vision (Kirillov et al., 2023) as well as other domains such as biology (Abramson et al., 2024; Nguyen et al., 2024). However, foundation models for discrete decision-making such CO and our target VRPs, are still under-explored an area one reason being the lack of large, high-quality open datasets that can effectively be employed to train such models which
27

---

Published in Transactions on Machine Learning Research (09/2025)
motivates use of RL. Such foundation models may not only obtain solutions faster than traditional OR counterparts but also avoid the requirement of possibly decades of research and resources to tackle a single task, while a foundation model may automatically learn heuristics without supervision.
Generalist, or specialized? Another open question is the idea generality behind the model. ROUTEFINDER, we argue that model might not need to be extremely complex and be specialized for a specific application (such as routing). One such reason that with larger model capabilities comes larger size and inference time, which is crucial for real-world deployment. An interesting future direction would be to attempt to generalize model as "foundation model for CO', for instance, based formulation (Boisvert et al., 2024), and see whether the additional training and inference costs are worth a (possible) boost in optimality gaps and generalization ability. Such model may be able to allain a better few-shot generalization to totally unseen attributes, either with adapter layers (Lin et al., 2024) or with our proposed EAL. However, we believe that tailored, specialized foundation models as POUTEFINDEES for VRPs may be more practical and efficient. We note that an orthogonal direction ours is the use of LLMs hyper-heuristics (Romera-Paredes et al., 2024; Liu et al., 2024b; Ye et al., 2024a), which starts from generalist LLM agent to generate algorithms that can be used to improve the optimization of CO problems as VRPs. However, such models are not used at inference time due to the inefficiency of using billions of parameters that are not tailored for the problem at hand.
general
Going forward in specialized foundation models for VRPs, there are several challenges yet to be addressed. One such challenge is the still sub-par performance compared to state-of-the-art solvers (Wouda Tan, 2023; Wouda et al., 2024), which may be offset on larger scale by several means, including decompositions. Another way attain better performance would be to integrate with local search (Ye et al., 2024b; Kim et al., 2025) and hybridize ruetive (the current policy paradigm) with improvement mot (Ma et al., 2021; 2024) to guarantec monotonic improvements given larger time budgets. Finally, given the robust cross-task performance even compared to single-task models, we believe expanding to more VRP variants (and their attribute distributions) may further improve overall performance.
C.4 Licenses for used assets
Table lists the used assets and their licenses. Our code is licensed under the MIT License.
Table Used asscts and their licenses.
Type
Asset
License
Usage
POMO (Kwon et al., 2020)
MIT License
Fvaluation
MTPOMO (Liu et al. 2024a)
MIT License
Evaluation
MVMoE (Zhou et al., 2024)
MIT License
Evaluation
Code
RL4CO (Berto et al., 2025)
MIT License
Evaluation
AL (Lin et al., 2024)
MIT License
Evaluation
ORTools (Perron & Didier, 2024)
Apache-2.0
Evaluation
VRP (Wouda et al., 2024)
MIT Lioense
Evaluation
Dataset
CVRPLIB (Lima et al., 2014)
Available for any non-commercial use
Testing
D
Additional Empirical Results
This Section supplements the main paper with several experiments evaluating various aspects of ROUTEFINDER:
Appendix D.1: here we motivate our ROUTEFINDER foundation model for VRPs when compared to single-variant models in 1) finetuning performance and 2) out-of-distribution generalization.
Appendix D.2: we evaluation large-scale and real-world distributions in CVRPLIB.
Appendix D.3: study the effect and interactions of Transformer Encoder components.
28

---

Published in Transactions on Machine Learning Research (09/2025)
Appendix D.4: here, we show the effect of different reward normalization techniques.
Appendix D.5: here we study Mixed Batch Training and its effect on 1) training stability and 2) imbalanced variant distributions.
Appendix D.6: we study the latent learning representation ability of different models via t-SNE across 1) encoding layers 2) effect of different attributes on the latent embeddings.
Appendix D.7: this section adds additional experiments for zero-shot and finetuning performances with EAL on three unseen new attribute setups: 1) with mixed backhauls 2) with multi-depots and 3) with both mixed backhauls and multi-depots as well as comparison to finetuning with single-variant models.
D.1 Out-of-Distribution Attribute Generalization
In this section, we study our foundation model and ask the following question: how does ROUTEFINDER perform when compared to models trained specifically compare ROUTEFINDER and other multi-task learning methods with POMO trained on single variants, including CVRP, VRPL. VRPTW, OVRP, and VRPB. For fairness of comparison, we train the POMO models with the same hyperparameters as the other models (from Table 5), including the same batch size, learning rate, and training epochs on n = 100 nodes.
single variant? To answer this question, we
We also study out-of-listribution generalization for unseen attribute values of time windows (C) and duration limits (L), for multi-task learning models and single-variant POMO ones. We compare cost values and gaps (the lower, the better) to the results of POMO training specifically for that single variant, similarly to Liu et al. (2024a, Appendix D). All experiments are performed on 1000 variants for each setting with n = 100.
In VRPTW, we consider different values of the time interval, i.e., the minimum and maximum values from which service times 8; and time window lengths t, are sampled (points 1 and 2 for time window generation of Appendix A.1). In distribution, these values are sampled from [0.15, 0.20]. In the out-of-distributie we consider them as ([0.05, 0.1], 10.15, 0.20). [0.85, 0.9), [0.85, 1.0)). The results in Table demonstrate again that for values differing from the in-training distribution, our model obtains better results than POMO trained solely VRPTW.
settings,
Table 7: Comparison of our model with single-task POMO on out-of-distrrbbttion VRPTW instances.
Time Interval
[1,0, 0.1of
E:15, 0.20
[0:25,
0,20]
[0:35,
0,40]
J0,45, 0.20
11.55, 0.00]
[0,65
0,70)
1o, 0.A0)
B.M) 0.85
[0550 1.00]
Olj
Gap
Clbj
Gap
Obh
Grip
Olij
GGog
Onj
Glpp
(lk)
Cap
Clij
Gag
Ohj:
Gap
Dbs
Gap
[lkj.
Gap
POMO VRPTW
25.30
26.27
28.11
31.95
35.26
39.66
44.43
68.17
52.60
55:24
MIPOМO
25.51
0.8455
26.59
1.20%
28:07
0.575
$1.26
-0.3655
$5.06
-0.56%
39.51
-0.35%
11.51
-0.31%
16:25
0.17%
52.85
0.17%
55.07
0.7855
MIVMuE
25.47
0.603
26.07
1.16%
28.26
0.525
31.19
-0.5455
$4.97
-0.70%
39.34
-0.52%
41.15
-0.6955
65.00
-0.25%
$2.68
0.14%
[5.6)
0.6855
RF-POMO
25:45
0.385
0.85%
23:25
0.8456
31.37
-0.11%
-0.18%
-0.22%%
41.81
-0005%
08:20
0.00%%
39:11
0.00%
55:22
-0.05%
RF-M--
-0.0%
RE-TE
25.33
0.1055
26.80
0.50%
28.14
0.11%
81.17
-0.615%
$4.91
-0.95%
39.90
-0.03%
41.06
-0.80%
-0.05%
$2.40
-0.38%
65.16
-0.14%
For VRPL. we consider different distance limit values During training, we sample feasible instances with lmax 3.0 as described in Appendix A.1. For out-of-distribution settings, we test distances for values 3.4, 3.5). 3.3, Interestingly, shown in Table 8, our model already outperforms POMO_VRPI VRPL in distribution, and the trend is maintained for larger values of l.
of 1 (2.9, 3.0, 3.1,
Finally, Appendix D.2 reports the results for large-scale CVRPLIB, which demonstrate ROUTEFINDER better generalize across sizes and real-world distributions than other multi-lask models and single-variant ones. Overall, we can see that ROUTEFINDER is robust, and its advantage more pronounced the further away from the training distribution we go. This motivates future work foundation models for VRPs, where we believe that exploring diverse solutions and variants will significantly advance the field.
D.2 CVRPLIB Evaluation
We report in Table 9 the results for largo-scale CVRPLIB (Lima et al., 2014) with sizes greater than 500 as done in MVMoF (Zhou et al., 2024). We report the original POMO (Kwon et al., 2020) alongside versions
29

---

Published in Transactions on Machine Learning Research (09/2025)
Table &: Comparison of our model with single-task POMO on out-of-distribution VRPI, instances.
Distance Limit
2.9
3.0
3.1
3.2
3.3
3.1
3.5
Obj.
Gap
Obj.
Gap
Obj.
Gap
Obj.
Gap
Obj.
Gap
Obj
Gap
Obj.
Gap
VOMO_VRPI.
VRPL.
15.84
16.00
16.04
15.52
16.02
15.74
15.85
MTPOMO
15.92
0.49%
16.08
0.53%
16.12
0.54%
15.59
0.47%
16.11
0.00%
15.61
0.48%
15,92
0.43%
MVMoE
15.88
0.22%
16.03
0.22%
16.08
0.27%
15.54
0.11%
16.04
0.13%
15.78
0.25%
15.88
0.20%
RF-POMO
15.91
0.41%
16.04
0.27%
16.09
0.33%
15.56
0.28%
16.06
0.29%
15.78
0.29%
15.87
0.13%
RF-MoR
15.86
0.12%
16.03
0.21%
16.05
0.08%
15.53
0.09%
16.04
0.13%
15.77
0.215%
15.87
0.12%
RF-TE
15.82
-0.17%
15.96
-0.21%
16.02
-0.10%
15.50
-0.10%
16.00
-0.11%
15.72
-0.11%
15.82
-0.16%
Table 9: Results on large-scale CVRPLIB instances from the sct. All models are only trained on the uniformly distributed data with the size 100 and evaluated via greedy rollouts. Results for methods with f are drawn from Zhou et al. (2024), models trained with single features excluding feature compositions (except for OVRPTW). Trriiing on multiple variants enhances generalization across models.
Set-X
POMO!
MTPOMO
MVMoB!
MVMoE-L/
MTPOMO
MVMoB
RF-TE
Instance
Ope.
Obj.
Glap
Dbj.
Gap
Obj.
Gap
Okj.
Gap
Obj.
Ohj.
Gap
Obj.
Glap
X-e502-k39
69226
73617
9.252%
77284
11.600%
75533
6,222%
74420
7.5166%
69226
9.410%
76338
10.274%
71791
3.705%
X-n513-1e21
24201
30618
102%
28510
17,805%
52102
32.6417%
31231
29.048%
21201
42.511%
34.866%
28465
17.610%
X-s24-kiss
154353
201877
30.580%
192249
24.358%
186540
20.005%
182202
17.982%
1543933
14.771%
170999
10.612%
174881
12.800
Xe538-
91846
106073
11.857%
106314
12.302%
109581
15.530%
108543
14.441%
96866
16.109%
105847
11.099%
100272
8.8845
X-65184300
86700
103083
18.908%
94262
9.068%
05894
10.60455
95817
10.931%
86700
27.851% 16
104289
20 20.257%
100866
16.443%
X-666-1422
42717
40370
15.575%
47846
12.00756
56008
31.11455
61810
21.257%
42717
30.770%
63383
24.000%
49454
15.771%
X-3573-330
50673
63545
64.871%
60913
20,20850
59473
17.366%
57042
12.569%
50673
20.210%
61524
21.414%
55852
10.418%
X-5586-k159
190316
220887
20. 79.%
205803
9.7615
215868
13.921%
214577
12.748%
190316
10.125%
212151
11.473%
206575
8.018%
X-utoo-lo2
108451
160072
38.837%
1200333
10.9665
128949
18.901%
126279
15.537%
108451
21. 21.0888%
126678
16.714%
116500
7.477%
X-661s002
500030
68461
14.976%
67V64
14.19256
82586
38.718%
74946
25.4844%
506395
30.6229
73456
67207
12:987%
X-6627-443
62161
64434
35.825%
73060
17.528%
14.100%
70806
14.061%
62164
23.103%
70414
13:271%
67672
8.700%
Xa0t1-k0 les
63682
73373
18.672%
72643
14.071%
75329
18.289%
72655
14.0900%
63882
30.321%
71975
13.028%
70881
11.226%
X k131
127211
19.134%
116988
9.560%
117878
10.200%
118475
10.952%
106780
12.731%
119057
11.497%
1122002
5,078%
X-607041300
146332
208079
42. 197%
100118
29.92295
1977605
35,100%
183447
25.364%
146332
24,809%
168226
14.062%
168959
15.400%
X-68s-A775
68205
79482
16.534%
80802
18.00136
97388
42.7875
89441
31.136%%
68205
36.650%
82269
20.0200%
77847
14.137%
X-n701-ke4
81923
97843
19.433%
92075
12. 12,3925
98:69
20.197%
94924
15.870%
81923
13.319%
80180
10.090%
69632
9.776%
X-a716-k35
43373
51381
18.463%
52709
21. 11.525%
56773
30 806%
52305
20.303%
49973
37. 657%
52250
20.467%
19689
14.516%
X-17s3-k199
136187
169088
16.825%
161961
18.92656
178322
30.30095
167477
22.570%
1966187
28.910%
106NN7
14.833%
148463
90014%
X-a799-0s
77229
87796
13.611%
50082
17.229%
100038
29.9865
96497
22.2240
77289
32 18.18
92147
19:2530%
85171
30.227%
X-6796-471
114417
135464
18.396%
144041
25.891
152352
33.156%
136256
19.099%
114417
16.602%
130505
14.061%
X-6783-445
72396
90280
24.733%
83160
14.897%
100983
38.677%
92960
28 28.423%
72396
50.140%
96336
33.067%
83186
14.919%
X-a801-kd0 k:00
73305
124278
69.536%
85077
16.0595
24.900%
87662
19.583%
73305
2
87118
18.843%
86164
17.542%
k171
158121
193451
22.344%
177157
12.039%
183599
16.113%
1858322
17. 17.523%
158121
22.148%
179596
13.5819
174441
10.32155
X-4857-4142
193737
237884
22.787%
214207
10.00076
229526
18.4785
221286
14.227%
1937737
19:42006
2303622
18.04%
206528
7.435%
X-astoloos
88983
162628
71.447%
1017744
14.39876
99129
11.420%
106816
20.003%
889065
38.1099%
106801
18.024%
98299
10.4833%
X-6876-330
99299
119764
20.00 600600%
116617
17.400%
119610
20.460%
114399
15. 140%
90299
15.240%
114016
14.801%
107416
8.174%
X-8805-137
53890
70245
30.421%
65587
21.773%
70018
46.710%
64310
19.402%%
53990
96.818%%
69000
28. 294%
64871
20.444%
Xa296-ka27
329179
399372
21.324%
361719
9.886%
383681
16.567%
370016
13.021%
3291799
18.134%
373600
13.499%%
3521688
7.236%
1-10064k1511
132715
237625
79.040%
186202
40 40.347%
66.466%
190407
43.471%
1.92715
50.654%
161343
21.571%
163162
22.942%
X-19574687
85485
130860
53. 13.104%
98196
14. 14.898%
113982
33.250%
106629
23.5003%
854655
48.1279
123633
44.650%
102689
20.15
X-a979 ks8
118976
147687
138092
16.0675
1983c7
23.005%
139682
17.404%
118976
16.711%
131754
10.740%
129952
9.225%
X-n10001-k43
72355
100300
38. 739%
87660
21.153%
116668
58.176%
94734
30.90 10.9999
72355
82.677%
68060
22.962%%
83829
18. 18.760%
Aug. Cap
29.638%
16.794%
26.408%
19.007%
30..022%
18.796%
12.308%
of MTPOMO and MVMoE that were initially trained on mixtures of only CVRP, OVRP, VRPL, VRPB, VRPTW, and OVRPTW for more than 3x longer than our setting with all variants. Interestingly, training all variants improves the generalization performance of MVMoF compared to the original setting, while
30

---

Published in Transactions on Machine Learning Research (09/2025)
it decreases the MTPOMO one (possibly due to the fact several more CVRP instances were sampled in MVMoFFss setting). Notably, ROUTEFINDER vastly outperforms other SOTA single and multi-task RL baselines.
D.3 Effect of Transformer Encoder Components
We study the effect of the proposed Transformer Encoder by ablating its components, in particular:
ROUTEFINDER: uses the full proposed Transformer Encoder as described in Section 4.2.1.
ROUTEFINDER (No RMSNorm): removes the RMSNorm in pre-norm, but keeps the SwiGLU MLP.
3. ROUTEFINDER (No SwiGLU): removes the SwiGLU MLP, but leaves the RMSNorm
4. ROUTEFINDER (No SwiGLU, No RMSNorm): removes all components and is equivalent to the commonly used Attention Model-style encoder (Kool et al., 2019).
Avg Cap % (all vncients, 50)
2.20
2.15
2.10
2.05
RouteFinder (No RMSNorm)
RouteFinder
RouteFinder (No SwiGLU)
RouteFinder (No SwiGLU, No RMSNorm)
Figure 8: Effect of encuder components.
We show in Figure the effect of each component on the test gaps for n 50 nodes, averaged across the 16 variants of Table The full ROUTEFINDER provides the best performance. We additionally study the behavior of each single component Appendix D.3. Interestingly, shown in Appendix D.3, while the final performance for the variant with no RMSNorm outperforms the bascline due to its enhanced capability in represcntation lcarning, its convergence is slower in the beginning. However, the full Transformer Encoder containing both RMSNorm and SwiGLU not only performs the best, but also converges the fastest, indicating the importance of each single component.
validation data during the training epochs across different. variants in
FlashAttentinn speedup FlashAttention (Dao et al., 2022; Dao, 2024) is recent exact attention algorithm that can be used to significantly speed up computations with mixed precision. This can be applied any model with an attention based mechanism, so we apply it by default all neural networks compared this work. Overall, improve training and inference speed by up to 20%+ with performance degradation.
31

---

Published in Transactions on Machine Learning Research (09/2025)
Dap (CNTery 10
50
100
150 Epochi
210
550
50
100
L5n
200
230
Rppodss
Gap (OVIE)
Gop (V4222)
50
100
1.50
200
200
50
100
150
200
250
Epochi
Epodaa
Cap (VIP))
Clap (UNAPPN)))
-2-
Epodl
Cle (VRPPL..
Oop WRRPOUU
6
200
Epodas
Cop (VICDTT)))
Gkip (VWPLEE
Cop (0N71P9)
50
150 Epucis
Epochs
Cip (OVRPaLT
2.5
(OVIOPOTT)
2.5
2.0
1.0
100
150
200
250
50
100
150 Egreitx
200
250
Fprchs
Cipy (OVR01))
Cep CONMPLET
50
100
150
203
250
1000
150
2000
250
Eyosdss
Eguect
RonttPPrder
BirteFlrvlee (No (MSNNor:)
RontoPiner (No SwIGUU)
RonteFinder (No SuIGUU, No Norm)
Figure Ablation study on proposed encoder components training.
32

---

Published in Transactions on Machine Learning Research (09/2025)
D.4 Effect of Reward Normalization Techniques
In this section, we study the effect of different reward normalization techniques, i.e., 1) 1)-4)
4) from Section 4.3.2.
Average Gap (%)
9.905
9.900
9.895
9.890
9.885
No Normalization
Cumul (Div)
Cuunu (Suh)
Exp (Sub, 0.1)
Exp (Sub, 0.25)
Exp (Div, 0.1)
Exp (Div, 0.25)
Figure 10: Effect of reward normalization.
Figure 10 shows that reward normalization successfully improves performance across variants. We also show different values of a for the exponential moving averages and find that the division through the exponentially smoothed mean with a 0.25 works best. Future normalization research may further improve performance.
D.5 Studies Mixed Batch Training
D.5.1 Effect on convergence speed
In addition to the effect of MBT on the training loss shown in Section 5.2, we also show the validation gaps held-out instances in Figure 11, where MBT speeds up convergence across all variants.
33

---

Published in Transactions on Machine Learning Research (09/2025)
4.0
(CVMP)
3.5
30
1000
150 Ezochh
200
250
30
100
150 Epocha
200)
250
Cxp (0VIL))
-2.5
2.0
1.5
2.5
Gip (VRPS)
2.0
Dap
3.5-
50
100
150
300
50
100
200
Epochs
Epochs
(PPU)L)))))
35.
Cip (VR770))
2.0
(NRPBWW)
Gop (VRPUST)))
2.3
2.0
50
100
100
2000
250
30
100
150
200
250
Ebochh
Epocha
Cop (OVTU11))
Gop (ONBPPL)
4-
Gap (OVAPOLTW)
2.00
1.50
2.25
Goup
1.25
1.00
50
100
150
300
50
1550 Epochs
300
Epochs
Gip (OVWPLWW)
2.5
2.0
1.5
200
200
MRT
No MET
Figure 11: Mixed Batch Training (MBT) allows for better convergence all variants.
34

---

Published in Transactions on Machine Learning Research (09/2025)
D.5.2 Effect on imbalanced variant distributions
As explained in Section 4.3.1, we can sample variants uniformly by setting the probability of sampling base attributes V as Рμ, = 0.5. We study the behavior of MBT in imbalanced attribute distributions. We train ROUTEFINDER models from scratch with the same sotting as the main experiments for 50 epochs with 10, 000 instances of size 50 sampled per epoch, with and without MBT, and at different values of the sampling probability for time window attributes prw as 0.5, 0.25, and 0.10. Figure 12 shows the validation gaps over the training.
0.50
0.25
Prw 0.10
CVRI
Grp (95)
1.-
15
13-
10
2000
3000
30:00
2000
Scace
Steps
Steps
25
Prw
20
Prw
55
Pinw
30-
20-
Gop (80
5-
5-
1000)
2000
3200
1000
20000
3000
2000
20001
Stepe
Steps
MBTT
No MRT
Figure 12: Effect of Mixed Batch Training (MBT) imbalanced variant distributions with varying probability Prw of sampling time windows (TW). MBT stabilizes training not only the downsampled variants such as VRPTW also improves the performance for variants with more samples as CVRP.
Decreasing prw (vowards the right of the plot) results in fewer time window attributes; thus, the convergence is slower for variants such VRPTW. On the other hand, variants like the CVRP will be sampled with higher probability, which results in slightly faster convergence. MBT plays important role in stabilizing the training for all cases. Interestingly, while its effect is more moderate for the majority samples (CVRP), this effect is higher on minority samples as VRPTW, where it results in a stable training curve, yielding fast. convergence.
D.6 T-SNE Visualizations
For interpretability, we study the representations learned from the model across different variants. Given their high dimensionality, we employ (-SNE (Van der Maaten & Hinton, 2008) to project them in 2D space. We employ the implementation from scikit-learn with the default perplexity of 30 and use 100 instances of size 100 for cach of the 16 variants of the main experiments from Scction 5.1.
D.6.1 Layer-wise visualization
We study ROUTEFINDER'S Transformer Encoder layers. As shown in Figure 13, distinct clusters emerge at different model layers, indicating that the model progressively separates the problem variants with increasing depth. Early layers (Layer 1) exhibit high overlap between different variants, suggesting shared feature extraction. However, as we procoed to deeper layers (Layer 6), the clusters become more distinct, particularly for more complex variants such as OVRPB, VRPBLTW, and VRPBTW, signifying the model's capacity capture and differentiate intricate problem structures.
35

---

Published in Transactions on Machine Learning Research (09/2025)
Layer
Layer 2
Layer
40
40-
20
20-
200
20
-00
-40
Layer
Layer 5
Layer
40
20
0
-20
-00
-00
-00
-20
20
40
-40
-20
0
20
40
-40
-20
20
40
CVRP
VRPTW
VRPAL
OVRPETW
OVRP
OVRPTW
VRPRTW
OVRPLTW
VRPB
OVRPB
VRPLTW
VRPELTW
VRPL
OVRPL
OVRPBL
OVRPBLIW
Figure 13: Visualization of ROUTEEFINDER' Transformer Encoder lateut space via t-SNE analysis by layer. Problem patterns become visible with deeper layers, generating distinct clusters.
D.6.2 Comparison across models and VRP variants
We also compare t-SNE analyses across the models, in particular, MTPOМO and MVMOE, compared to our ROUTEFINDER with Transformer Encoder layers, with embeddings taken in the last encoder layer for all models. In particular, we aim to analyze the differences in latent representation problem variants across the four attributes: open routes (O), distance limits (L.), backhauls (B), and time windows (TW). Figure 14 shows that ROUTEFINDER generates more and defined clusters, indicating better-learned representation (Arora et al., 2018). For open routes, ROUTEFINDER has more defined clusters than the baselines. In distance limits, our model generates double the clusters, which indicates different relations between attributes; for instance, the model clearly separates backhaul variants VRPB and VRPBL (green and grey, respcctively), while other models do not clearly do this. This also holds in the backhaul altribute clusters, where ROUTEFINDER more clearly separates different types of time windows as well as distance limits. Finally, for time windows clusters, we notice the most striking difference while MTPOMO and MVMoE fail to distinguish between time window variants, resulting in single and sparse cluster, ROUTEFINDER separates time window variants with and without the open (O) attribute into two separate clusters thanks to the Global Attribute Embeddings.
36

---

Published in Transactions on Machine Learning Research (09/2025)
Open Routes (0)
Distance Limits (L.
Backhauls (B)
Time Windows (TW)
40
40
20
20
MTPOMO
-10
-20
-20
-20
-30
-50
90
30-
20
20
MVMoE
10
-10
-10
-20
-30
30
-2
40-
ROUTEEFINDER
20
20
001
-00
20
-5
25
60
-25
25
50
-30
-20
-10
VRPTW
OVRPTW
OVRPL
OVRPB
OVRPIW
OVRPB
VRIBL
VRPBL
VIPBTW
OVRPI.
VRPLTW
VRPHTW
VRPEIW
OVRPEL
OVRPBL
OVRPBL
OVRPBTW
OVRPRTW
OVRPLIW
OVRPBTW
OVRPLTW
OVRPLTW
VRPBLTW
VRFBLIW
VREBLTW
OVRPBLTW
OVRPBLTW
OVRPBLTW
OVRPBLTW
Figure 14: Analysis of the t-SNE latent space for the last encoder layer for different attributes. ROUTEFINDER yields well-defined, tightly grouped, and distinct clusters on all variants strong indicator of its capahility to generalize and specialize effectively in solving diverse VRP variants. For example, unlike the baselines, ROUTEFINDER distinctly separates time window variants into two clusters with and without routes (bottom-right image) thanks to the Global Attribute Embeddings.
D.7
Finetuning with EAL
D.7.1
Finetuning to Unseen Variants
We conduct additional experiments zero-shot generalization of various models and finctuning across three different settings of unseen variants in order of difficulty:
Mixed backhauls (MB): this the setting from Section 5.3. We report the results full in Table and trends over epochs in Figure 7a.
Multi-depot (MD): we add more attributes for finetuning approaches as per Appendix B.4 with data generated in Appendix A.1. Results reported in Table 11 and trends epochs in Figure 7b.
37

---

Published in Transactions on Machine Learning Research (09/2025)
3. Mixed backhauls & multi-depot (MB&MD): this is the hardest setting, which considers as finetuning variants only the ones containing both the unseen MB and MD attributes at the same time from Table 4. Full results are in Table 12 with trends over epochs in Figure 7c.
We keep the same methodology as outlined in Section 5.3, ie.. 10 epochs with 10k instances sampled for each epoch. We use ROUTEFINDER models with Transformer Encoder (RF-TE), untrained for the scratch training and pretrained from the same checkpoints as the main experiments in Section 5.1 for AL and EAL finetuning. Additional details on EAL modeling are available in Appendix B.4.
Tabll 10: Zero-shot, retraining, and fine-tuning performance on unseen mixed backhaul (MB) variants. "2" are models and fine-tuning methods evalusted in zero-shot settings. EAL maintains the zero-shot performance and performs best overall.
VRPMB
OVRPMB
VRPMBL
VRPMBTW
OVRPMBL
OVRPMBTW
VRPMBLTW
OVRPMBETW
Method
Obj-
Gap
Obj.
Gap
Obj.
Gap
Obj.
Gapp
Obj.
Gopp
Obj.
Gap
Obj.
Gap
Olaj.
Gap
HGS HGS-PyVRP VRP
13.54
9.01
13.78
25.51
9:01
16.97
25.85
16.97
OR- Tools
14:90
10.27%
10.59
17.54%
15.42
11.90%
29.97
17.48%
10.50
17.54%
19.31
13.78%
30.44
17.78%
19.31
13.78%
MTPOMO*
15.04
11.322%
10.87
20.663%
15.41
11.97%
28.31
11.0656
10.85
20.433%
18.51
9.08%
28. 73
11.275%
18.51
1.12%
MVMoE
14.99
10.94%
12%
15.33
11.37%
11.105
20. 14%
18:5
9.35%
28.
11.16%
18.55
9.30%
RF-POMO"
14.98
10.50%
10.81
20.31%
15.29
11.12%
28.53
11.91%
10.81
20.32%
18.02
9.72%
28.89
11.89%
18.62
9.71%
RF-MoE
14.90
10.49%
19.49%
15.21
10.47%
28.20
10.63%
10.76
19.40%
8.74%
28.55
10.57%
18.45
6.72%
RF-TF"
14.88
10.13%
10.72
19.02%
15. 18
10.322%
28.20
10.8776
10.72
19.011%
18:45
8.68%
28.65
10.9255
18.45
8.00
Train (scratch)
15.18
12.13%
10.40
15.38%
15.48
12.37%
28.11
10.17%
10.46
16. 16.055%
18.85
11.00%
28.
10.95%
18.86
11.199
AL"
13.15
221 25%
37.98
3291.25%
92.81
199.849%
39.17
133.
29.15
221.3%%
39.03
131.09%
66.62
158.219%
10.92
141.51%
AL
14.91
10.10%
10.14
12.53%
15.
9.73%%
27.79
5.92%
10.18
12.95%
18.52
9.13%
28.33
9.50%
18.51
9.05%
EAL.
14.89
10.13%
10.72 72
19.02%
15.
10.32%
28.29
10.877
10.72
19.01%
18:45
8.68%
28.65
10.925
18.45
8-60%
EAL
14.59
7.89%
9.66
7.18%
14.78
7.39%
26.69
4.613
9.65
7.18%
17.60
3.70%
27.13
4.90%
17.59
3.653
Table 11: Zero-shot, retraining, and fine-tuning performance unseen multi-depot (MD) variants. "Ø" denotes models and fine-tuning methods evaluated in zero-shot settings. EAL maintains the zero-shot performance and performs best overall.
MDCVRP
MDOVRP
MDVRPB
MDVRPL
MDVRPTW
MDOVRPTW
MDOVRPB
MDOVRPL
Method
Obj-
Gup
Obj.
Gap
Olij
Gap
Obj.
Gap
Olo].
Gap
Olj..
Glpp
Ol).
Gap
Obj.
Gap
HGS-PyVRP
11.89
7.97
11.64
11.90
19.33
13.00
8:69
7.97
OR-Toob
12.32
5.27%
2.38%
12.
5.01%
12.52
5.245%
19.62
1.35%
0.74%
8.87
2.15%
2.33%
MTPOMO
16.07
35.74%
10.28
29.06%
15.18
30.66%
16.30
37.58%
26.68
38.56%
17.57
35.6775
10.91
28.08%
10.28
29.07%
MVMol
16.02
35.35%
10.24
28.50%%
15.12
30.13%
16.25
37.17%
26.67
38. 51.5515
17.57
35. 63.6895
10.89
25.56%
10.24
60%
RF-POMG
16.03
35.46%
10.23
28. 52.5
15.11
30. 10.6
16.25
37. 1976
266
38. 1656
17.54
35. 35.4355
10.88
25.48%
10.23
28.55%
RF-MoE"
16.01
35,24%
10.20
28. 0666
15.06
29: 6936
16.21
36. 8936
26
38. 1156
17.64
35-4455
10.84
25.022%
10,20
28 069
RF-TE"
15.98
35.01%
10.18
27.82%
15.05
29.53%
16.20
36.76%
26.51
37.615
17.18
31.96%
10.82
21.71%
10.18
27.81%
Toin (scrwcch)
14.44
21 5.%
9.88
23.87%
14.86
27.75%
14.50
21 9003
233.35
20. 825
13:48
19.1655
10..5
233.84%
9.89
24.07%
AL"
33.91
188.705
215.
33.56
189. 189.58%
161.78%
49.08
155.57%
141.42%
26.30
203.63%
21.12
73%
AL
14.23
19.84%
9.67
21.28%
14.84
27.57%
14.33
20.51%
22.64
17.18%
15.05
15.81%
10.69
23.112%
9.68
21.45%
EAL*
15.98
33.02%
10.18
27.82%
15:05
29.5336
16.20
36.765
26.51
37.645
17.48
34.9655
10.82
24.74%
10.18
27.84%
EAL
12.96
9..44%
8.64
8.37%
13.05
12.15%
12.99
9.31%
21.14
9..43%
13.81
6.24%
91.46
8.88%
8.64
8.333%
MDVRPBI
MIVRPBIW
MIVRPPTTW W
MDOV
MIOVRPPIWW
MDOVRPLTW
MIVRPBEETW
MDDVRPBLIW
Method
Obj.
Gap
Obj.
Gap
Obj.
Gap
Obj.
Gapp
Obj.
Gap
Obj.
Gap
Obj.
Gap
Obj.
Gap
HGS--Py PyVRP
11.68
19.35
14.369
1/3000
22.06
14.37
OR-Touls
12.22
4.66%
22.10
1.69%%
19.66
1.58%
8.87
2.18%
14.49
0.87%
13:09
0.70%
22:43
1.70%
14.89
0. 0.86%
XTPOMO"
15.50
95.54%
39.23%%
27.13
40.71%
10.91
26.11%
19.09
37.62%
17.58
35.70%
31.09
41.52%
19.09
37.64%
MVM&E
15.73
34.95%%
30.55
39.22%%
27.12
40.67%
10:90
25.06%
19.69
37.625%
17.58
35.745
31.06
41.3995
19.89
37.6611 61.6
RF-POMO
15.71
34.807%
30.46
38.80%
27.04
40.22%%
10.80
40%
19.66
37.3355
17:54
35.4935
30.97
41.0055
19.80
37.37%
RF-
15.65
34.25%%
30.47
34.87%
27.03
40.18%
10.84
25.03%
19.66
37. 40%%
17.55
35.4555
30.98
41.06%5
19.96
47%4
RF-TES
15.62
33.98%
30.36
38.30%
26.93
30.607%
10.82
78%
19.59
36.9155
17.48
34.9556
30.96
40.4955
19.80
36 96%
Train (suzalch)
15.05
28. 91%
26.43
20.08%
23.41
21.08%
10.77
24.02%
16.86
17.41%
13.50
19.285%
26.52
20.301%
16.58
17.64%
AL"
32,08
175.25%
51.70
136.019%
47.65
147.90%
25.00
186. 83%
32.45
127.08%
29.91
131.91%
50.14
128.59%
30.93
116.41%
AL
14.95
28. 03.%
25.81
17.19%%
22.70
17.33%
10.70
14%
18.47
14.68%
15:07
15.99%5
25.84
17.16%
16.48
14.71%
EAL"
15.62
33. 98%
30.36
38.36%
26.93
39. 99.699%
10.82
21. 78%
19.59
36.95%
17.48
31.95%
86
40.
19.90
36.969
EAL
13.16
12.70%
23.88
8.42%
21.18
9.47%
9.46
8.85%
15.18
5.61%
13.81
6.24%
23.94
8.544%
15.17
5.60%
ROUTEFINDER models perform the best in zero-shot generalization across all experiments; moreover, EAL finetuning achieves the same zero-shot performance as the backbone ROUTEFINDER model RF-TE thanks to the zero-padded initialization, while AL does not due to the introduction of untrained embedding layers. Notably, experiments with multi-depots are much harder than mixed backhaul variants since they require the model to understand multiple starting (and returning) point locations and to schedule vehicle assignments to their respective depots efficiently. EAL performs the best across all variants in finetuming performance. Remarkably, FAL's performance compared to AL and retraining model from scratch is more prominent.
38

---

Published in Transactions on Machine Learning Research (09/2025)
Table 12: Zero-shot, retraining, and fine-tuning performance unseen variants with combined multi-depots (MD) and mixed backhauls (MB). "e² denotes models and fine-tuning methods evaluated zero-shot settings, EAL finetuning maintains the zero-shot performance and performs best overall.
MDVRPMD
MDOVRPMB
MDVREMBL
MDVRPMBTW
MDOVRPMDL
MDOVEPMBTWW
MDVRPMDLTW
MDOVRFMELIWW
Method
Obj.
Obj.
Cop
Obj.
Obj.
Cap
Obj.
Cop
Obj.
Gap
Obj.
Clap
Obj.
HGS-PyYRR
10.68
7.36
10.71
19.29
7.05
12.96
19:31
12.96
OR- Tools
12.22
14.5 14.37%
8.88
15.83%
12.25
14.2855
22.30
16.125
8.87
15-7956
14.40
11-70%
22.43
16.10%
14.40
11.7995
*TPOМO
15.14
42.22%
10:91
42.5776
45.2375
48.0155
10.90
42.435
18.56
43.637%
30.337
43.6555
MV M
15.08
41.67%
10.90
42.41%
15
44.3755
46
48. 48.1256
10.88
42.1556
15.61
44.04%%
28.89 89
50.10%
18. 13
43.965%
RF-POMO"
15.00
41.18%
42.41%
44.0575
08
49.2756
10.90
42.3776
18.69
44.717%
44.0275
RF MoE
15.02
41.98%
10 82
41.40%%
15.29
13.34%
25.35
47.67%6
10.82
41.30%
18.50
43.19%%
26.77
49 19.58%
18. 13.50
13.22%%
RF-TE"
14.99
40.907 60.90%
40.677
43.2775
47.935
10.75
40.6255
18.49
43.14%
28.50
18.
43.1775
Train (sczaschs
13.12
22.88%
9.57
22.32%
13.24
20. 72%
22.85
18.50%
9.38
22.44%
15.13
10.791%
22.50
18.65%
15.11
16,0075
225.14%
245.53%
27.41
68%
155.28%
21:11
31.53
30.08
08%6
AL
13.10
22.70%
9.36
22.14%
13.20
20.30 30.30%
22.90
18.76%
9.38
22.00%
15.28
17.91%
23.02
19.20%
15.39
18.7798
EAL?
80%
13.279
47.93%
00.6250
18.09
43.14%
EAL
12.70
18.98%
8.53
11.35%
12.68
18.56%
21.41
11.05%
8.54
11.48%
13.93
7.41%
21.44
11.0055
13.91
7.32%
with the incrcasing difficulty of the finctuning task from MB to MB+MD, indicating it is a suitable method for efficient deployment in finetuning to new lasks.
D.7.2 Finetuning with EAL for Single- Variant Models
We finetune all POMO models with the same sctting as the experiment with unseen mixed backhaul and multi-depots (MB&MD) from Appendix D.7.1 with EAL.
Table 13: Fine-buning performance on unseen variants of single-variant POMO models and ROUTEFINDER. foundation model for VRPs is crucial for fast adaptation to downstream tasks.
Finetuning a
MDVRPMB
MDOVRUMB
MDVRIMBBL
MDVRPMBTW
MDOVRPMBL
MIOVRPMBBW
MEVEPMBLIW
MDOVRPMBBIW
Method
Obj.
Gap
Gap
Obsj
Gap
Olt).
Gap
Obj
Gap
Obj.
Gap
Obj.
Gap
Ot.
Gap
BIG-PPVRP
10.68
7.66
10.71
19.29
7:05
12.96
19:31
12.96
TR-Ta
8.88
15.6316
16.23%
22.39
10.125
15.7355
14.49
11.79%
22.
POMO CVRP
13.31
20.97%
9.66
26.015
13:43
25.50%
25.19
30.845%
9:66
25.975
25.14
30.399
17.66
36.50%
17.65
36. 43%
POMO VRPL
25.1456
9.88
28.0755
24.90%
03-4336
28.70%
45.5890
20.74
53%
POMO OVRP
13.31
24.62%
9.54
24.45%6
13.35
24.77%
26.03
35.2756
9.55
24.6376
25.03
35.07%
18.65
44.21%
18.06
44. 14.30%
POMO VRPTW
30.27%
77%
30.72%
28.1356
33.405
4339
16.74
20.32%
16.80
77%
POMO VRPB
13:00
21.697
9.25
20.6355
13.06
22.077
22.50
16.6655
9:23
20.4456
22
16.64%
14.96
15.30%
14.97
15.54%
ROUTEFINDER
18.98%
8.53
11.30%
12.68
18.567%
11.05%
11.43%
13.93
7.41%
21.44
11.09%
13.91
7.32%
Table 13 shows that fine-tuning our ROUTEFINDER foundation model achieves the best results, even when comparing variants that include only unseen features for both. For instance, POMO trained only VRP with backhauls (POMO_VRE) VRPB in the table) was trained by sampling many more (classical) backhaul features, but ROUTEFINDER can still fine-tune better on MDVRPMB. Models trained on similar features as the target ones (e.g., POMO VRPTW being trained on time windows) can overall fine-tune better to variants that include these features (e.g., time windows) than other models. This is expected, but our foundation model performs even better. This is a strong motivation for practitioners and rescarchers: developing foundation models for VRPs is crucial for fast adaptation to new tasks that may arise in real-world scenarios, such adding new constraints attributes.
E Visualizations
We provide qualitative visualizations for the solutions of randomly picked instances from the instances of the main results of Table for different models: MTPOMO, MVMoE, and ROUTEFINDER model variants RF-POMO, RF-MoF, RF-TE.
39

---

Published in Transactions on Machine Learning Research (09/202))
MITOMO (14.888)
MVMCC (14.820)
RE-POMO (14.735)
RF-MDE (14.809)
RF-TE (1 (14.806)
CVRP
ATTPOM (10.885)
MVMME (10. 805)
RF-POMO (10.872)
RF-MOE (10.789)
RF-TE (10.748)
OVRP
NIPPМM (1699
MVMME 366)
RF-POMO 351)
RF-MDE (16 296)
RF-TE (16.26))
VIP
MTPOMO (17.359)
MVMOE (17.350)
RF-POMO (17.368)
RF-MDE (17.245)
RF-TE (17,208)
VAPL
NTPPOM (28.076).
MVMOE (28. 0944
RF-POMO 511)
RF-MOE (28 292)
RF-TE (28.195)
VRPP TW
MTPOMO (16.875)
MVMGE (16. (16.706)
RF-POMO (16. 654)
RF-MDE (16.784)
RF-TE (16.652)
CWRPPTWW
(9.904)
OVYPP
MTPOMO (9.967)
MVMDE E (9.983)
RF-POMO (10. 021)
RF-MOE (9.978)
RF-TE (9.918)
CWRPP
Figure 15: Solutions for CVRP, OVRP, VRPB, VRPL, VRPTW, OVRPTW, OVRPB, OVRPL.
40

---

Published in Transactions on Machine Learning Research (09/202))
NITOMO (15.996)
MVMCE (16.078)
RF-POMO (15.900)
RI-MDE (15.747)
RF-TE (15.879)
VROPUL
ATPOMO (34.826)
MVMME (35, 204)
RF-POMO (34. 369)
RF-MDE (34 (34,595)
RF-TE (34.252)
Vee9 TW
NIPPМM (31368)
MVMME (30. 742)
RF-POMO 140)
RF-MOE (30.911)
RF-TE (30.167)
MTPOMO (11.674)
MVMoE (11 (11.499)
RF-POMO (11. 505)
RF-MDE (11 562)
RF-TE (11.507)
OVARUL
NTPOMO (21.495)
(21.
RF-POMO 520)
RF-MOE (21, 572)
RF-TE (21.542)
DWPPPBTI
MTPOMO (14.763)
MVMDE (14.775)
RF-POMO 6233)
RF-MDE (14.697)
RF-TE (14.569)
CURPLLD
(35.74 730)
MTPOMO (19.734)
MVMGE (19.733)
RE-POMO (19.894)
RF-MOE (19.588)
RF-TE (19.510)
OVAPULTW
Figure 16: Solutions for VRPBL, VRPBTW, VRPLTW, OVRPBL, OVRPETW, OVRPLTW, VRPBLTW, OVRPBLTW.
41


Reviews:
[{"review_id": "WDuMhenZY4", "reviewer": "Reviewer_hdwG", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n1.\tDifferent from the previous works, which incorporated global attributes into the model on the shallow decoder dynamically, this work embeds these features along with node attributes, which helps the model to better understand the problem.\n2.\tThe mix batch training technique contributes to a more stable training process and faster convergence.\n3.\tThe efficient adapter layers allow the model to generate to variants with unseen attributes, enabling the model to solve new problems while reusing learned knowledge.\n4.\tThe RouteFinder outperforms the existing state-of-the-art foundation models for VRPs.\n\nWeaknesses and questions: \n1.\tThe mix batch training sample VRP attributes uniformly. But the relationship between different attributes can be complex. Some attributes may lead to conflicts, while some attributes may share the same nature. Have you considered the problem (attribute) characteristics and leverage them to sample instances for training more efficiently?\n2.\tAlthough the RouteFinder outperforms existing NCO models, there are still certain gaps with the best-known solutions. What do you think about the possible reasons? For example, the incapability of the backbone model, the intrinsic interference between different problems, etc.\n3.\tHave you considered finetuning the whole model? How about its training efficiency and testing performance?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes a foundational NCO model named RouteFinder for various VRPs. Each variant is represented by a combination of predefined attributes. The model can adapt to unseen variants with new attributes through adapter layers. To train the model on different tasks more efficiently, two training techniques, mix batch training and multi-variant reward normalization, are introduced. Experiments on 48 VRP variants show an outperforming performance.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Additional discussion on finetuning the whole model to show how the adapter layers can save fine-tuning resources and retain performance. This would strengthen the work.\n2. Additional discussion on the sampling method in mix batch training. This would strengthen the work.", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "4y7rYFtO3A", "reviewer": "Reviewer_agDC", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n\n1. This paper is well-written and easy to understand.\n    \n2. The greatest advantage of this study lies in the versatility of its framework. The proposed ROUTEFINDER is able to handle up to 48 different problem variants, far beyond the scope of previous studies, and provides a scalable platform for solving complex and changeable routing problems in the real world.\n    \n3. The core technologies proposed in the paper have proven their effectiveness through experiments.\n\n## Weaknesses\n1. As a general base model, ROUTEFINDER has a \"slight sacrifice\" in the quality of its solution when dealing with specific tasks within its training distribution compared to models trained specifically for that single task. This is a typical trade-off between \"generalists\" and \"specialists\". The authors have also discussed this situation.\n    \n2. Some proposed techniques may lack a little bit of novelty, such as mixed batch training (MBT).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces a framework called ROUTEFINDER, which aims to build a foundation model to solve multiple VRP variants. The authors propose a series of innovative and effective methods, including a unified VRP environment, a modern Transformer-based encoder, and introduce Global Attribute Embeddings, mixed batch training (MBT), multi-variant reward normalization, and efficient adapter layers (EAL) for efficient fine-tuning. The experimental results demonstrate the promising problem-solving ability of the proposed ROUTEFINDER, which outperforms existing SOTA learning methods on a large number of VRP variants.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Overall, I think this work does a great job and no further requested changes are needed, but I have some questions that, if discussed further, may strengthen the work in my view.\n\n1. The unified environment is able to simulate 48 variants by switching attributes, which is a very powerful design. However, when constraints of different attributes exist at the same time, is there potential redundancy or conflict? For example, in an instance with a strict time window (TW), a very loose duration constraint (L) may be non-binding. How does the model learn to ignore this redundant information, or does this information act as noise and affect learning efficiency?\n    \n2. EAL performs well when dealing with new constraints that are similar to existing attribute types (such as multiple depots and mixed return pickups). However, does its effectiveness generalize to \"unseen\" constraints that are more fundamentally different in structure?", "broader_impact_concerns": "N/A.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "TjNHHc5R9d", "reviewer": "Reviewer_tvLi", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1) The paper proposes a detailed framework for the generic VRP, and show that its variants can be solved within this framework\n2) The paper introduces embeddings of the problem attributes (global and local) related to capacities, time windows etc\n3) The paper proposes mixed-batch training and mixed-variant reward functions to enable the stable solution of all variants of the problem in the same framework\n4) The paper describes adequate experiments on simulated data and demonstrate superior performance of their model. \n5) The paper also validates the importance of different components of their architecture, and even show its adaptability to out-of-distribution tasks\n\nWeakness:\n1) Visualizations of the problem setting, results by the proposed method and competitors (including pure RL methods) could have been better\n2) Comparisons presented are mainly with respect to a couple of recent neural models like MTPOMO and MVMoE with favorable results, but there is not much discussion on reasons for improved performance. A more thorough discussion of existing approaches with comparsions would have been appreciated", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This article considers the Vehicle Routing Problem (VRP), along with its different variants related to vehicle capacity, linehaul/backhaul demands, time-windows, multiple depots etc. The basic idea of the paper is that a foundation model can be developed on the full setting of the problem with a set of attributes, and that model can be adapted to any of the variants by enabling the relevant attributes. These attributes are represented by embeddings as inputs to the neural model. The problem is solved under a Reinforcement Learning Framework where the policy computation is done through a Encoder-Decoder based architecture.  They use suitably defined reward function for which gradient can be defined to optimize the model, using the proposed strategy of mixed-task reward normalization. Extensive experiments with ablation studies demonstrate the superior performance of the proposed approach.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1) The data generation part is described entirely in the Appendix. It should be partially discussed in the main paper\n2) It would be good to have some visualizations of the solutions found by the different models - routefinder vs MTPOMO or MVMoE on specific instances of the problem\n3) The experiments are performed entirely on synthetic data. Is there some benchmark dataset on which the comparisons can be done? It is fine if not all variants are available. \n4) Even for synthetic data, the results of Table 1 suggest degrading performance as we increase n from 50 to 100. In general, how does the performance of the proposed model vary with 'n' and other parameters of the problem?", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)