Virne: A Comprehensive Benchmark for RL-based Network Resource Allocation in NFV
Authors: Tianfu Wang
Liwei Deng
Xi Chen
Junyang Wang
Huiguo He
Zhengyu Hu
Wei Wu
Leilei Ding
Qilin Fan
Hui Xiong
Venue: ICLR 2026 Poster (2026)
Keywords: Network Resource Allocation
Deep Reinforcement Learning
Library
Benchmark
Network Simulation
Primary area: datasets and benchmarks
TL;DR: We introduce Virne, a benchmarking framework for NFV-RA that offers comprehensive simulations, implementations, and evaluations, along with insightful findings to advance research in ML for networking.

Abstract: Resource allocation (RA) is critical to efficient service deployment in Network Function Virtualization (NFV), a transformative networking paradigm. This task is termed NFV-RA. Recently, deep Reinforcement Learning (RL)-based methods have been showing promising potential to address this combinatorial complexity of constrained cross-graph mapping. However, RL-driven NFV-RA research lacks a systematic benchmark for comprehensive simulation and rigorous evaluation. This gap hinders in-depth performance analysis and slows algorithm development for emerging networks, resulting in fragmented assessments. In this paper, we introduce Virne, a comprehensive benchmarking framework designed to accelerate the research and application of deep RL for NFV-RA. Virne provides customizable simulations for diverse network scenarios, including cloud, edge, and 5G environments. It features a modular and extensible implementation pipeline that integrates over 30 methods of various types. Virne also establishes a rigorous evaluation protocol that extends beyond online effectiveness to include practical perspectives such as solvability, generalizability, and scalability. Furthermore, we conduct in-depth analysis through extensive experiments to provide valuable insights into performance trade-offs for efficient implementation and offer actionable guidance for future research directions. Overall, with its capabilities of diverse simulations, rich implementations, and thorough evaluation, Virne could serve as a comprehensive benchmark for advancing NFV-RA methods and deep RL applications. The code and resources are available at https://github.com/GeminiLight/virne.

Under review as a conference paper at ICLR 2026
000
VIRNE: A COMPREHENSIVE BENCHMARK FOR RL- BASED NETWORK RESOURCE ALLOCATION IN NFV
001
002
003
004
005
Anonymous authors
Paper under double-blind review
006
007
008
009
ABSTRACT
010
011
Resource allocation (RA) is critical to efficient service deployment in Network Function Virtualization (NFV), a transformative networking paradigm. This task is termed NFV-RA. Recently, deep Reinforcement Learning (RL)-based methods have been showing promising potential to address this combinatorial complexity of constrained cross-graph mapping. However, RL-drriee NFV-RA research lacks a systematic benchmark for comprehensive simulation and rigorous evaluation. This gap hinders in-depth performance analysis and slows algorithm development for emerging networks, resulting in fragmented assessments. In this paper, we intro- duce Vire,, comprehensive benchmarking framework designed to accelerate the research and application of deep RL for NFV-RA. Virne provides customizable sim- ulations for diverse network scenarios, including cloud, edge, and 5G environments. It features a modular and extensible implementation pipeline that integrates over 30 methods of various types. Virne also establishes rigorous evaluation protocol that extends beyond online effectiveness to include practical perspectives such as solv- ability, generalizability. and scalability. Furthermore, we conduct in-depth analysis through extensive experiments to provide valuable insights into performance trade- offs for efficient implementation and offer actionable guidance for future research directions. Overall, with its capabilities of diverse simulations, rich implementa- tions, and thorough evaluation, Virne could serve as a comprehensive benchmark for advancing NFV-RA methods and deep RL applications. The code and resources are available at https::/anonyyoous.. wwwp:ouuu.oooooooonoooooo wopp...ccoccc////oooooossssss-ss
012
013
NEW
014
015
FIX
016
FIX
017
018
019
020
021
022
023
024
025
026
027
028
029
030
031
032
INTRODUCTION
033
034
Network Function Virtualization (NFV) has emerged as an essential enabler for modern networks,
035
such as cloud data centers, edge computing and 5G, due to its remarkable flexibility and scalability (Yi et al. 2018). By transforming traditional hardware-bound network services into flexible software modules, NFV enables the deployment of Virtual Network Functions (VNFs)
036
037
reneral-purpose
038
servers (Zhuang et al., 2020). A central challenge in NFV is Resource Allocation (NFV-RA) that is essential for effective resource management and service quality (Yang et al., 2021). As illustrated in
039
040
Figure L NFV-RA involves mapping service requests (modeled as virtual networks of interconnected VNFs) onto shared physical infrastructure while satisfying constraints. It is an NP-hard combinatorial optimization problem (Rost& & Schmid 2020), highlighting the need for efficient solutions.
041
042
043
Traditional approaches NFV-RA, such as exact solvers (Chowdhury al., 2009 Shahriir et al. 2018) that seek optimal solutions or heuristics (Gong et al., [014]] Fan et al., (0223) dependent on manual design, suffer from neither excessive time consumption nor suboptimal performance. Recently, Reinforcement I Learning (RL) has shown promise in solving NFV-RA, which enables the autonomous learning of efficient heuristics by interacting with simulated network environments (Haeri & Trajković| 2017; Yan et al.) 2020 Wang al. [2024b] Wu et al. 2024b). This training paradigm operates without requiring high-quality labeled datasets, which are difficult to obtain due to the computational complexity and privacy concerns, making it particularly well-suited for NFV-RA.
044
045
046
047
048
049
050
051
However, the advancement of RL for NFV-RA is significantly hampered by the lack of standardized, comprehensive benchmarking. On the one hand, existing benchmarks, summarized in Table I are limited to specific scenarios (e.g., cloud) and a narrow range of non-RL methods. On the other hand, the increasing complexity of modern networks leads to fragmented problem definitions and
052
053

---

Under review as a conference paper at ICLR 2026
054
055
Table 1: Comparison of NFV-RA benchmarks.
056
Supported
Implemented
RI. & Gym Support
Evaluntion Perspectives
Last Update
Simulation
Algorithms
057
VNE-Sim (Hacri & Trajkovid 2016
Cloud
heuristies
X
Efoccivennss
2014
058
ALEVIN Beek ct 2014
Cloud
heuristics
X
Effectivencss
2016
059
ALib [Rost al 2019
Cloud
X
Effectiveness
2019
SFCSim
2022
Cloud
leuristicss
X
Elfectiveness
2022
060
10ye Tomascck 2021
Cloud
heuristics
X
Effoctiveness
2024
061
Cloud, Edge.
30+ algorithms
Effectiveness
062
Vime (Ours)
SG Slicing, etc.
(10+ non- non-RL.
& Practicality
2025
063
inconsistent simulations, making fair comparisons and robust evaluations difficult. We summarize the current state of NFV-RA research in Table 4 to highlight these issues. Developing a framework to address these issues poses significant technical challenges, including unifying diverse models, standardizing numerous algorithms, and designing comprehensive evaluations. A unified, accessible framework for reproducible research and standardized evaluation is urgently needed.
064
065
066
067
068
In this paper, we introduce Virne, a comprehensive benchmarking framework for NFV-RA that offers diverse simulations, unified implementations and thorough evaluations. Firstly, Virne is designed to serve as a unified and readily accessible framework for researchers from both the machine learning and networking communities. It provides a highly customizable simulation environment capable of accurately modeling a wide array of NFV scenarios, from cloud data centers to edge and 5G networks, allowing for the exploration of various resource types, constraints, and service requirements. Secondly. Vime features modular architecture that simplifies the implementation of various NFV-RA algorithms, which facilitates both efficient utilization and new algorithm development. It includes over 30 NFV-RA algorithms, covering both exact, heuristic and advanced Slaming-bbsed methods. Thirdly, beyond standard performance metrics, Virne enables in-depth analysis through practical evaluation perspectives such solution feasibility, generalization across varying network conditions, and scalability with increasing problem size. Finally, through extensive empirical studies conducted within Virne, offer valuable insights into the effectiveness and characteristies of various algorithms, providing data-driven guidance for future research directions and practical deployments.
069
070
071
072
073
074
075
076
077
078
079
080
081
082
Our main contributions, aimed at accelerating data-centric ML research in network optimization,
083
084
Comprehensive Simulations. Virne is the most comprehensive benchmark for NFV-RA to date, along with gym-style environments supporting highly customizable simulations.
085
086
Streamlined Implementations. To facilitate community use, we design modular and easily expandable implementation pipeline and integrate a wide range of NFV-RA algorithms.
087
0B8
Emerging Evalutions. Beyond online effectiveness alone, Viree enables practical evalua- tions of solution feasibility, generalization across diverse network conditions, and scalability.
089
090
Insighful Findings. Extensive results reveal the impact of different modules and nuanced performance comparisons, providing actionable insights for future applied ML research.
091
092
093
094
2
NFV-RA PROBLEM DEFINITIONS
095
096
Due to distinct considerations of network scenarios, existing studies formulate NFV-RA varying in system models, constraints, and objectives. To enhance clarity and consistency, we provide a unified definition that combines a basic cost optimization model with extensions for emerging scenarios.
097
098
099
100
2.1
BASIC DEFINITION FOR CoST OPTIMIZATION
101
102
System Model. In NFV, user network services and the physical infrastructure are virtualized as virtual networks (VNs) and physical network (PN), respectively. As illustrated in Figure [ in online network systems, service requests represented VNs are continuously arriving at PN, seeking the physical resource with specific service requirements. Each arrived VN and the corresponding snapshot of PN consist of an instance I (G., Gp), where the PN Gp and VN G, modeled as undirected graphs. G, = (Np, C,) and G, = (N,, C,, w. w), respectively. Here, N₂ and C, denote the sets of physical nodes and links, indicating servers and their interconnections; Nr, and L. denote
103
104
105
106
107
2

---

Under review as a conference paper at ICLR 2026
108
109
Network Topology Syrthetic (W0m vark Futtinee. Pos Topulagia
Resounce Type Mode: CPU) Ghd Stragn Thrwaghou
Extemobre Awarenena Energy Seurty Polocilty
Networkk Scenanis
110
Stmulation Configuration
Cyper Edge Hylrnd Aiteh Duschaliss
Auxilory Uditty
111
Infrastructure Controller
112
User Service Bequesta (V166)
InstATiON
Algoritii Implementation
113
Mi-booud Methods
Tead tronal Marhods
Carainant Checker
Bantanament
114
115
System
infrestructure
vettic
hourletis
Solution Counter
116
(PN)
bosedd nartind
Performacce Recorder
117
118
Eraluation
Rostune Kilittton
Practial
Gereraiation
In-repth
Algoritin Gerigy
Data
119
Evaluation Criteela
Metrics
Perspotivcs
Indiht
Viezaliatioon
120
121
Figure 2: The architecture of Vime benchmark. Vime offers a streamlined workflow for supporting comprehensive experimentation of NFV-RA algorithms. The process begins with customizing simulation configurations that define network scenarios and conditions. Then, the network system is instantiated, triggering a series of service request events process. At each event, the selected NFV-RA algorithm interacts with the system to resolve the instance. For each simulation, both the processing and final results are automatically recorded for subsequent analysis.
122
123
124
125
126
127
the sets of virtual nodes and links, representing services and their relationships: w and w denote the arrival time and lifetime of VN. We denote C(n,) as available computing resources for physical node n. E Np and (l,p) available bandwidth resources of physical link 1p Lz. Besides, ((n)) and (4.) denote the demands for computing resource by a virtual node ny E Ny and bandwidth resource by a virtual link L E Lu. Over specifie period, we collect all instances into set I.
128
129
130
131
132
Embedding Constraints. Mapping a VN G. onto the sub-PN 9p' is formulated as
133
User Requests
Continuous
BB Video Service
Finance Service
Game Service
134
mapping function fo G. + 9p'. It con-
135
sists of two sub-mapping processes, i.e., node mapping fAr and link mapping Sc- Node mapping fAr aims find suitable physical node n2 = (n.) ) to place each virtual node n, while adhering to the one- to-one mapping and resource availability constraints. Concretely, virtual nodes in the same VN must be placed in different phys- ical nodes and each physical node can host at most one virtual node. And the phys- ical node np must have enough available resources to place the corresponding virtual node n., i.e., (m,)) < (nn,), Vn, E NN, Link mapping aims find connected Figure 1: A brief illustration of the NFV-RA problem. physical path Pa = fc(()) to route each virtual link 4, while satisfying the path connectivity and resource capacity constraints. Concretely, the physical path Pp should connect physical nodes hosting the two endpoints of virtual link 4. And each physical link I E Pp of physical path Pp should have enough bandwidth to route the corresponding virtual link L,, i.e., B(I.) S B(I,), vI, E L.,, Ip E Pp-
Virtual
VNr
136
Network
Vinul Link
(VN)
137
Node Mapping
138
Resource Allocation
139
Likk Mapping
140
Physical Network (PN)
141
Server
142
Physkcal
Lrk
143
144
Infresturcture Prouider
Computing
Storage
Bandwiitt
Server
145
146
147
148
149
150
151
Optimization Objective. Considering the randomness of online service requests, NFV-RA mainly aims to maximize the resource utilization of each instance T, which facilitates long-term resource profit and request acceptance (He et al. 2023) Zhang et al., 2023). To assess the solution quality = fI) of instance 1, the cevenue-to-cost ratio (R2C) is a widely used metric, defined as follows: max R2C (S) (x REV (S)) (S).
152
153
154
155
156
(I)
157
Here, x is binary variable indicating the fcasibility of the solution S: 0 if S violates some constraints, and x otherwise. REV(S) and COST(S) denote the generated revenue and incurred cost by embedding VN Go. If x 1, (EV(S) denotes the revenue from the VN, calculated as EmEEN, C(n.) + Elsec. B(l.) and COST(S) denotes the resource consumption of PN, calculated as En C(n,)) + E1, EC, (l.)((.)) ) x B(L.)). Here, (1..) quantifies the length of the physical path Pp routing the virtual link l,.. See Appendix A. for the detailed formulation.
158
159
160
161
3

---

Under review as a conference paper at ICLR 2026
162
163
NPV Environment
RL Apent
164
Orlinefevel
Instancoffeeel
Festure Constructor
Networks
Section
Rollout Memory
165
Krning Vil
Deploying VN
Artion
Vii Row
166
Teatures
Adtion Frobucility
167
Roward
FN Raw Features
Decodlr
Dearmttion
168
bocodit Scuttgy
i Learning Algorithms
ooli 00nG ASC IP00
169
Othara
Globel Feonures
Oventy Boars
170
Snapphot of
171
Figure A unified pipeline of gym-style environment and RL-based NFV-RA methods in Virne.
172
173
2.2
EXTENSIONS IN EMERGING NETWORKS
174
The application of NFV is pivotal in emerging network scenarios, such as heterogeneous resourcing networks, latency-aware edge networks, and energy-efficient green networks. These scenarios require additional considcration of their unique challenges. To better align NFV-RA with practical require- ments, discuss several extended definitions of NFV-RA in popular scenarios in Appendix A.2
175
176
177
178
179
180
3
VIRNE: A COMPREHENSIVE NFV-RA BENCHMARK
181
182
To provide high-quality simulation and implementation, introduce three design principles for Virne, following established software engineering practices (Van Vliet et al. 2008): (a) Versatile customization allows the platform to meet diverse simulation needs varying network scenarios and conditions. (b) Scalable modularity built platforms with modular architecture to support flexible configurations and easy extensibility. (c) Intuitive usability prioritizes user-friendly interface, enabling users to focus on experiment outcomes rather than implementation complexities. Guided by these principles, as illustrated Figure 2
183
184
185
186
187
we implement Viree with five key modules as follows.
188
189
3.1
SIMULATION CONFIGURATION
190
Modern network systems are diverse and intricate, associated with various realistic factors such as resource availability and service requirements. To support high customization for simulating different network scenarios and conditions, Virne abstracts both the PN and VN graphs, with customizable attributes for nodes, links, and the overall network. Specifically, Virne provides the following key customizable elements: (a) Network Topologies. Users can select from various topological synthesis methods or real-world physical infrastructure topologies, such as those from SDNLib (b) Resource Availability. Virne enables users to define multiple resource types (e.g.. CPU, GPU, bandwidth) and their availability across different levels of the network (i.e., nodes, links and graph), allowing the reflection of the specific resource characteristics of different network environments (c) Service Requirements. Users can specify additional service requirements, such as latency, energy efficiency, and reliability, to ensure that the simulation reflects the needs of different network applications. By adjusting these parameters, Virne provides the flexibility to simulate a wide array of network scenarios, including cloud-based infrastructures, edge computing and 5G networks. This customization enables comprehensive testing, accommodating fluctuating resource demands and evolving network conditions. We provide configuration examples in Figure and Appendix C.
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
3.2
NETWORK SYSTEM
206
With the above-mentioned configurations, Virne automatically creates network system that functions as an event-driven simulator, which mainly consists of a PN as infrastructure and a series of sequential arrived VN requests. Vime treats each VN request arrival as discrete event that triggers the corresponding resource scheduling and allocation procedures. An event represents the interaction between the VN request G. and snapshot of the PN Gp at the time of the request's arrival, denoted as an instance The specific NFV-RA algorithm then solves the corresponding NFV-RA problem for that instance to obtain a solution, S fo(!). Next, the system evaluates the feasibility of the solution based on the resource availability constraints and service requirements satisfaction. If feasible, the VN request is accepted; otherwise, it is rejected. This wentt-drieen mechanism is designed to simulate realistic, complex network environments while enabling efficient handling of diverse network scenarios and fluctuating conditions.
207
208
209
210
211
212
213
214
215
4

---

Under review as a conference paper at ICLR 2026
216
3.3 ALGORITHM IMPLEMENTATION
217
To facilitate direct utilization and further extensions, Virne integrates diverse NFV-RA algorithms, covering both learning- based and traditional methods (see Appendix C.3). These algorithms are
218
219
systematically organized to streamline the implementation process. Here, we highlight our unified pipeline of gym-style environments and RL-baase NFV-RA methods, as illustrated in Figure 3
220
221
3.3.1
NFV-RA AS A MARKOV DECISION PROCESS
222
To address the randomness of online networks, most existing works model NFV-RA solution con- struction as Markov Decision Process (MDP), which sequentially selects a physical node to place a virtual node at each decision step t, until all virtual nodes are placed any constraints are violated. Decision sequence defaults to VN node index, with support for customizable node ranking in Virne.
223
224
225
NEW
226
Formally, we define this process as tuple (S, A, P. R. X). Concretely, S is the state space, where 8, E represents the embedding status of VN and PN. A is the action space, where o referring to set of physical nodes. P : S x A x S 00, 1] is the state transition function, indicating the conditional transition probabilities between states. During one transition, the system will attempt to place the selected action, i.e., a physical node a Mp route the to-be-decided virtual node nu. If node placement succeeds, link routing will be conducted to route the virtual links connecting ny and its prready-placed neighbor nodes. The shortest- path algorithms are used to identify the shortest and mmnstraint-satisfiee physical paths for these virtual links. Only when both node placement and link routing are successful, the physical network's s available resources are updated with the VNR's requirements. Otherwise, the VN is rejected. R S x A > TE is designed reward function to guided optimization. E [0, 1) is the discount factor.
227
228
229
230
231
232
233
234
235
236
237
At each decision timestep i, observing the state 8, of the environment, the agent takes an action a₂ ~ (-1ss) according to a policy тe parameterized by 0. Then, the environment will transit to a new state 8t+1 P(-[st; dz) and feedback a reward R[s;, a,) and. During interactions, trajectory memory collects these step information as experiences. Using these experiences, the objective of the agent is to learn optimal policy for resource allocation that maximizes the expected sum of
238
239
240
241
242
discounted reward, i.e., the solution quality of NFV- T& = arg max ExD
Σ 7 R(s:, a:)
243
Note that the above MDP introduced is the most adopted version. As a holistic framework, Virne also supports the emerging MDP version in recent studies. Due to the presentation clarity and page limit, we offer other MDP versions of NFV-RA in Appendix(C.2) such as Constrained MDP (Wang et al. 2025) for constraint handling and Multi-task MDP (Wang et al. 2024b) for generalization.
244
245
246
247
248
3.3.2
UNIFTED PIPELINE FOR EFFICIENT IMPLEMENTATIONS
249
Considering this widely-used MDP model, we generally unify existing NFV-RA RL-based algorithms into three key components: MDP modeling, policy architecture, and training methods. These methods model the NFV-RA solution construction process as above-introduced MDPs with specific reward design and feature engineering. Subsequently, they use various neural networks to build policy architectures, such as Convolutional Neural Networks (CNN) (Krizhessky et al. 2012) and Graph Convolutional Network (GCN) (Kipf & Welling 2017). These policies are trained with selected RI. method, such as Asynchronous Advantage Actor-critic (A3C) (Mnih et al. 2016) and Proximal Policy Optimization (PPO) (Schulman et al., 2017).
250
251
252
253
254
255
256
257
Following this insight, Vire implements NFV-RA algorithms through a modular and extensible pipeline comprising several core components illustrated in Figure Specifically, an instance- level environment enable interactions with the agent for sequentially constructing solutions, where a customized reward function provides feedback each step. The RL agent consists of a feature construcior, which extracts relevant features as inputs for the neural networks, and neural policy, implemented with various architectures. To support RI. training, an experience memory module continuously collects interaction data, while designated training method optimizes the policy. This modular design is central to Virne's 's ability to integrate diverse approaches seamlessly, due to the high customization of each module. By standardizing the implementation process, Virne ensures consistency, reusability, and reduced complexity of implementation, which accelerates the development of new algorithms by providing a unified pipeline.
258
259
260
261
262
263
264
265
266
267
3.3.3
IMPLEMENTED RL- RL-BASED NFV-RA ALGORITTMM
268
As summarized in Table in Appendix B.2 various studies on NFV-RA often share similar core methodologies in their core RL framework design but differ primarily in terms of the specific network
269
5

---

Under review as a conference paper at ICLR 2026
270
scenarios considered or specialized implementation techniques employed. Vime offers compre-
271
hensive suite of state-of-the- art reinforcement learning methods for NFV resource allocation. The framework flexibly combines various RL training approaches (including MCTS, PG, A3C, and PPO) with different neural network architectures (such as MIP, CNN, S2S, GCN, GAT, BiGCN, BiGAT, and HeteroGAT)). It also incorporates various implementation enhancements like custom reward functions, feature engineering combinations, and action masking mechanisms. See Appendix C33.1 for descriptions of these RL methods, neutral policies, and implementation techniques.
272
273
274
275
276
277
To provide clarity in both implementation and evaluation, we categorize these works under common names that reflect RL algorithms and their neural policy architectures. For example, the PPO- PPO-GCN method uses graph convolutional networks as feature encoders while employing Proximal Policy Optimization for efficient training. In the subsequent experiments, we will first investigate the impact of distinct implementation techniques, such as reward function and feature engineering, and identify the most efficient configurations for general implementation. Then, we evaluate their effectiveness in the cloud and other popular scenarios.
278
279
280
281
282
283
284
3.4
AUXILIARY UTILITY
285
To enhance usability and streamline analysis, Vime includes several key auxiliary utilities, as shown in Figure 2 These are designed to simplify the process of managing simulations and interpreting results, particularly including: (a) A system controller manages the simulation of physical and virtual networks, providing method choices for customization; (b) A solution monitor tracks solution feasibility and performance during execution, helping users assess whether solutions meet their defined criteria; (c) Visualization tools offer interactive and visual representations of simulation results, providing users with an intuitive way to analyze network system behaviors. These utilities support more advanced customizations for algorithm design and result analysis.
286
287
288
289
290
291
292
293
294
3.5
EVALUATION CRITERIA
295
To offer a systematic evaluation, Vime provides a suite of metrics and multiple practical perspectives.
296
297
Performance Metrics. Virne includes the critical performance metries of NFV-RA algorithms (Fis cher et al. 2013) Wu et al.] (2024a). including request acceptance rate (RAC), long- term revenue-to- cost (LRC), long-term average revenue (LAR) and average solving time (AST). See Appendix C.5
298
299
300
for their definitions.
301
Practicality Perspectives. To comprehensively assess the practicality of NFV-RA algorithms, we develop multiple emerging evaluation protocols that extend beyond mere online effectiveness. In particular, we consider three aspects of NFV-RA algorithms: (a) Solvability denotes the ability to find feasible solutions; (b) Generalization indicates reliable performance across various network conditions; (c) Scalability measures how effectively it accommodates increases in network size and complexity. Gaining these insights with our pre-provided evaluation interfaces, Virne helps users understand the practical viability of an algorithm and guides further development and application.
302
303
304
305
306
307
308
309
4
EMPIRICAL ANALYSIS
310
4.1
EXPERIMENTAL SETUP
311
312
We consider the most general scenarios main network systems, i.c., cloud computing, and describe default settings for simulation and implementation. Subsequently, some parameters may be changed to simulate diverse network scenarios and conditions.
313
314
315
Simulation Settings. We adopt the widely-used topologies as physical networks: synthetic WX100 (Waxman) 1988), and real-world GEANT and BRAIN (Orlowski et al.] 2010). which cover various network scales and densities. See Appendix 1.2 for their detailed descriptions. Computing resources of physical nodes and bandwidth resources of physical links uniformly distributed within the same range of units, i.c.. &e ~ (50, 100) and XBG,, (500 100). In default settings, for each simulation run, we create 1000 VN with varying sizes following uniform distribution, Mg.. ~ (2, 10). The computing resource demands of the nodes and the bandwidth requirements of the links within each VN are uniformly distributed, i.e., Xe(m..) ~ U(0, 20) and Nna.)) ~ (0, 50), respectively. The virtual nodes in each VN are randomly interconnected with a probability of 50%. The lifetime of each VN follows an exponential distribution with average of
316
317
318
319
320
321
322
323

---

Under review as a conference paper at ICLR 2026
324
325 Table 2: Impact of key implementation techniques. Reward function is specified by its type (fixed or adaptive) and the intermediate reward value (0. 0.01, 0.1, 0.2: where 0 indicates no intermediate reward). Features indicate whether node Status (S) and/or Topological (T) metrics were used (v for
326
327
used, X for not used). Action Mask indicates whether action masking was applied (v) or not (X).
328
329
Reward Function
Features
Action Mask
PPO-MLP
PPO-ATT
PPO-DuaIGAT
330
Type
Value
s
T
RAC
LRC
LAR
RAC
LRC
LAR
RAC
LRC
LAR
331
Fixed
0.1
0.666
0.670
11745.0
0.667
0.644
12174.0
0.648
0.750
12430.1
332
Fixed
0.1
/
x
0.678
0.675
11159.0
0.660
0.667
11134.3
0.766
0.754
15054.7
333
Fixed
0.1
x
v
0,703
0.654
12693.8
0.579
0.575
9323.5
0.733
0.685
13333.1
334
Adaptive
0.705
0.619
12139.6
0.702
0.643
12368.6
0.772
0.744
14216.6
335
Fixed
0.2
0.616
0.643
11869.4
0,706
0.675
12420.7
0.769
0.731
15045.5
336
Fixed
0.01
<
0,709
0.645
12719_1
0.517
0,601
8339.7
0.766
0.748
14516.6
Fixed
0
0,560
0.596
8259.5
0.716
0.658
12629.0
0.741
0.753
14047.6
337
338
Fixed
0.1
0.719
0.647
12944.4
0.712
0.661
12657.7
0.781
0.738
15138.6
339
500 time units. The arrival of these VNs follows a Poisson process with an average rate 17, where denotes the average arrived VN count per unit of time. Due to the varying physical resource supply in these topologies caused by distinct scale and density, we use different i.e., 60.16, 0.016, 0.004) for (WX100, GEANT, BRAIN), to accommodate the reasonable request demand. In the subsequent experiments, we manipulate the distribution settings of VNs and change the PN topologies to simulate various network systems.
340
341
342
343
344
345
Algorithm Implementation. In Appendix D we provide the details of parameter setting, experi- mental methods on training and testing, and descriptions of computing resources.
346
347
348
4.2
PERFORMANCE COMPARISON
349
350
4.2.1
EXPLORATION ON IMPLEMENTATION TECHNIQUES
351
352
Due to its training efficiency and strong empirical performance Wang et al. 2022). we adopt PPO as the default RL algorithm in subsequent experiments. This choice is further supported by the study of Impact on RL Training Methods in Appendix [ and learning curves in Figure 9 Furthermore, there are several core implementation choices, such reward design, feature engineering, and action masking, that substantially affect the performance of RL-based NFV-RA algorithms. To quantify these effects, we systematically evaluate three representative policy architectures (PPO-MLP, PPO- ATT, PPO-DualGAT) on WX100, with results summarized in Table 2 Here, we summarize the key findings of each impact study and place their detailed analysis in Appendix D.2
353
354
355
356
357
358
359
Impact on Reward Function Design in Appendix D.2.2 suggests that while integrating adaptive- ness to network seale is conceptually appealing. a fixed and moderate reward signal is more effective for guiding policy learning in the NFV-RA context. Thus, we adopt a fixed reward signal of 0.1.
360
361
362
363
Impact on Feature Engineering Combination in Appendix D.2.3 reveals that @ topological metrics serve as valuable augmentation, capturing global network context and node importance, even for graph neural networks. Thus, we augment features with both node status and topological metrics.
364
365
366
Impact on Action Mask Strategy in Appendix D.2.4 demonstrates that explicit constraint enforcement is vital for robust RL-based NFV-RA solutions, since the constraints of NFV-RA are intricate and hard. Thus, always use the action mask mechanism to enhance performance.
367
368
369
This study reveals that the most effective implementation techniques of the RL-based NFV-RA method are achieved by combining moderate intermediate rewards, comprehensive feature engineering, and action masking. In subsequent experiments, we consider them as the default implementation.
370
371
372
373
4.2.2
EFFECTIVENESS IN ONLINE ENVIRONMENTS
374
375 To evaluate the effectiveness of these NFV-RA algorithms, we conduct online simulations across three distinct network topologies: WX100, GEANT, and BRAIN, each with different traffic rates (7)) as specified. Table Breports experimental results of the implemented RL-based and traditional algorithms on key metrics. While traditional heuristics exceptionally fast, they fall short in
376
377
7

---

Under review as a conference paper at ICLR 2026
378
379
Table 3: Performance comparison of implemented RL.-based and traditional NFV-RA algorithms.
380
Solver
WX100 (1)
0.14)
GEANT
0.016)
BRAIN
0.004)
381
RAC
LRC
LAR
AST
RACI
LRC
LAR"
AST 1
RACI
LRC1
LAR
AST 1
382
PPO-MLP
71.90
0.65
12964.40
0.13
55,80
0.67
645,04
0.03
$1.30
0.69
155.10
0.14
PPO-CNN
71.70
0.65
12964.87
0.13
54,80
0.65
643,83
0.09
51.10
0.69
151.51
0.13
383
PPO-ATT
71.20
0.66
12657.69 69
0.14
54.50
0.65
707.01
0.10
$1.00
0.68
156.40
0.15
384
PPO-GON
66.80
0.64
11462.65
0.14
58,70
0.72
763.68
0.12
49.50
0.71
125.63
0.09
PPO-GAT
71.90
0.70
13178.13
0.15
58.40
0.70
724.31
0.07
44.60
0.51
95.32
0.09
385
PPO GCN&S2S
65.80
0.63
11501.94
0.13
58.50
0.72
718.76
0.06
44.40
0.59
99.83
0.18
386
PPO-GAT&S2S
67.90
0.67
12445.03 03
0.16
$7,30
0.69
754.61
0.15
51.80
0.68
136.67
0.19
PPO- DualGCN
70.20
0.71
13467.57
0.17
60.40
0.75
791.75
0.22
54.80
0.78
176.15
0.23
387
PPO-DualGAT
78.10
0.74
14938.60
0.18
59.10
0.72
739.27
0.10
58.90
0.75
180.78
0.13
388
PPO-HeteroGAT
72.50
0.66
12691.00 03
0.27
53,30
0.66
621.47
0.30
30
0.66
133.52
0.38
MCTS
74,30
0.44
12642.27
3,38
48.20
0.45
494.64
2,96
40.80
0.47
83,91
3.59
389
SA-Meta
65.50
0.63
10467.60
38.60
0.62
396.49
0.49
75.50
390
GA-Meta
71.70
0.59
11977.41
3.22
49.90
0.58
517.63
2.34
42.50
0.57
85,45
3.75
391
PSO-Meta
69.10
0.52
10706.48
4.29
45.30
0.46
457.93
3.68
41.50
0.46
80.67
4,20
TS-Meta
65,70
0.66
11141.91
1,35
40,90
0.69
402.04
0.62
37.10
0.64
68.41
1.11
392
NRM-Rank
60.70
0.52
9826. 94
0.07
37,90
0.51
394.29
0.03
48 30
0.64
142,99
0.04
393
RW-Rank
60.10
0.56
9396.32
0.04
38.70
0.52
418.14
0.01
50.20
147.64
0.05
Bank
58,90
0.56
9269.03
0.04
36.70
0.47
353,21
0.01
48.40
0.64
144.55
0.03
394
PL-Rank
68.10
0.67
11570.27
0.32
55,30
0.66
661.93
0.04
0.70
136.79
0.36
395
Baakkaak
64.00
0.66
10837.51
0.83
47.20
0.62
$43.18
0.11
53.90
0.70
148.70
1.46
BFS
40.00
0.57
7771.38
0.05
56.70
0.64
736.28
0.01
140.38
396
397
solution quality. Similarly, meta-heuristics, despite achieving competitive acceptance rates, are hindered by significant computational overhead, making them impractical for real-time environments. This highlights that @ RL-based methods often offer a favorable balance between solution quality and efficiency, making them a compelling choice for NFV-RA. Advanced RL agents, specifically those with dual graph neural network architectures like PPO-DuaIGAT and PPO-DuaIGCN, consistently deliver superior performance. This suggests that S dual-GNN models' ability to concurrently process and relate features from both the virtual and physical network graphs provides significant edge in finding high-quality embeddings. Furthermore, it is notable that O the topological characteristics of PN, such as scale and density, impact the relative performance of NFV-RA algorithmss, particularly GNN architectures. For instance, on the smaller-scale GEANT and BRAIN, GCN-based methods achieve more competitive and even better performance. This reveals that GATs with an adaptive aggregation mechanism may be more advantageous in denser topologies for prioritizing critical connections, but provide less improvement sparse topologies.
398
399
400
401
402
403
404
405
406
407
408
409
410
411
4.3
EVALUATION FROM PRACTICALITY PERSPECTIVES
412
Solvability via Offline Evaluation. Tra- ditional online testing of NFV-RA algo- rithms, while reflecting real-world scenar- ios, complicates direct comparisons of algo- rithmic solvability. The continuous evolu- tion of the PN state, coupled with dynamic VN arrivals and lifetimes, makes it diffi- cult to isolate performance. Specifically, it is challenging to determine if solution failures arise from an algorithm's inher- ent limitations from transient unsolvable network states, thereby hindering fair com- parisons on solvability. To address this, Vire provides a controlled offline evalu- ation environment. This features a series of static instances, each comprising a PN and a VN, evaluating algorithms' abilities to find high-quality, feasible solutions un- der reproducible conditions. Through such evaluation, we assess the fundamental soly- ability of each method, not only across an entire dataset but also for distinct VN scales. This evaluation protocol offers a granular view of algorithmic solvability as VN complexity grows
413
7i-Rak-
0124
61111
8.097
1.622
8.504
A,529
0,50
414
415
KBA Aac
IKC
6.1%
8
8.5K1
0,593
416
03. Mim
4109
6.125
8
8.99
3.90
C
417
NPV/LA Alpectes
750-06ca
X123
0677
6,22
8:500
8.700
8,544
CATS
418
MTTS
X:44
F:44
611
8.885
8.9
4.50
6.415
419
420
0.84
00122
02:1
0.753
8
0.3+
0
421
PRO.
8.40
6:541
422
7P0 Heeewon
8:0%
6.59
423
-EA
424
425
426
-05
427
428
Figure 4: Results the solvability study. For each size of VN, we highlight the worst porst-performng solver in green and the best-performing solver in red.
429
430
431
8

---

Under review as a conference paper at ICLR 2026
432
Concretely, we evaluate the solution quality, i..., R2C, achieved by different algorithms for VNs of varying sizes. We conduct experiments on the representative PN, WX100. The results are shown in Figure[4] which presents average solution quality for VN sizes ranging from 2 to 10. There is generally decreasing trend as the size of the VN increases due to increased complexity. We observe that RI. methods with advanced graph representations exhibit superior solvability across most VN sizes. For instance,
433
434
435
436
437
consistently achieves the highest R2C ratios, particularly for larger VNs. Conversely, simpler RL policies like PPO-MLP perform well on small VNs but degrade noticeably on larger ones due to limited representation ability. This highlights that @ more powerful foundation policies excel in the vast search space of large VNs, dynamically prioritizing the most critical nodes and links for embedding. Additionally, it is obvious that truditional heuristics and meta-heuristics show mixed results and generally underperform compared to the top RI. methods, especially for larger VNs. Particularly, NEA-Rank initially achieves excellent performance for 2-node VNs, but as the VN size increases, it declines and then falls behind the top RL methods.
438
439
440
441
442
443
444
445
Generalization on Network Conditions. Network conditions are inherently complex and subject to continuous changes, such as fluctuations in request frequencies and varying resource demands. As such, evaluating the generalization of these trained NFV-RA policies is critical to ensure they can adapt effectively to different, evolving network environments. To address this, we conduct the experiments in various network conditions to study the generalization conditions of pretrained models, including (a) Evaluation on Varying Traffic Rates and (b) Evaluation on Fluctuating Demand Distribution. See Appendix D.3 for detailed experimental setup and result analysis.
446
447
448
449
450
451
Scalability on Network Sizes. Network systems are growing in size as the physical infrastructure extends and service requests increase. Thus, assessing the scalability of NFV-RA policies is essential to ensure they can maintain their effectiveness as the network size and complexity increase. To assess this, Vime supports scalability evaluations through two primary perspectives: (a) Performance on Large-scaled Networks and (b) analysis of Solving Time Scale as problem size grows. Detailed experimental setups and results for these evaluations are placed in Appendix D.4
452
453
454
455
456
457
458
4.4
VALIDATION ON EMERGING NETWORKS
459
NFV is widely applied across diverse modern networks, where specific scenarios often present unique characteristics. To further validate the adaptability of NFV-RA algorithms in emerging networks, we conduct evaluations in two key environments: (a) heterogeneous resourcing networks, which involve varied resource types, and (b) latency-awan edge networks, where delay constraints should be satisfied. See Appendix D.5 for experimental analysis.
460
461
462
463
464
4.5
DISCUSSION ON FUTURE DIRECTION
465
While RL-based methods for NFV-RA show promise, there are still practical challenges, as observed through our empirical observation. To advance deep RI. for NFV-RA, we identify several future directions that also pose significant challenges to existing MI. methods. They are included but not limited to (a) developing sophisticated representation learning for cross-graph statuses and attributed constraints, (b) achieving generalizable policies adaptable to varying network scales and dynamic conditions, (c) creating robust learning frameworks to enable learning in conflicting operational constraints, and (d) engineering highly scalable algorithms for extremely large-scall scale physical infrastructures. We also provide experimental analysis of emerging solutions along these directions, such as safe RL and meta Rl-bbased methods. See Appendix and Figure 13 for details.
466
467
468
469
470
471
472
473
474
475
5
CONCLUSION
476
In this paper, we introduce Virne, a comprehensive and unified benchmarking framework specifically designed for evaluating deep RL-based algorithms for NFV-RA problem. Virne supports diverse net- work scenarios, including cloud, edge, and 5G environments, facilitated by customizable simulations. In addition, we provide a modular and extensible pipeline that integrates over 30 diverse algorithms, particularly deep RL-baased methods, enabling extensible implementation. Beyond traditional effec- tiveness metrics, Virne offers practical evaluation perspectives such as solvability, generalization, and scalability. Furthermore, we conduct extensive experiments to evaluate the deep RL-based NFV-RA method from comprehensive perspectives. We provide crucial insights on the impact of implementation techniques and algorithm performance trade-offs. Our extensive empirical analysis also reveals valuable findings on potential challenges of RL-based NFV-RA methods, guiding future research the interaction of data-driven networking optimization and applied machine learning.
477
478
479
480
481
482
483
484
485

---

Under review as a conference paper at ICLR 2026
486
ETHICS STATEMENT
487
488
This work introduces Viree, a benchmarking framework designed to accelerate scientific research in ML for network optimization. This study does not involve human subjects or sensitive data, All datasets are either synthetically generated or based on publicly available network topologies from SNDlib. As such, we identify no direct negative societal impacts from this research.
489
490
491
492
493
REPRODUCIBILITY STATEMENT
494
495
To ensure reproducibility, we have made our benchmark and all necessary resources publicly available. The complete source code for the Virne framework, including all algorithm implementations and simulation environments, provided in the anonymous codebase link. We provide a detailed description of the experimental setup in Section 4.1 and Appendix D. These details consist of implementation details, network topologies, and data generation. Overall, these resources ensure the full reproducibility of our work and empower the community build upon our findings.
496
497
498
499
500
501
502
REFERENCES
503
504
Jacob Beck, Risto Vuorio, Evan Zheran Liu, Zheng Xiong, Luisa Zintgraf, Chelsea Finn, and Shimon Whiteson. A survey of rentoreeiffrr learning. arXiv preprint arXiv:2301.08028 2023.
505
506
Michael Till Beck, Claudia Linnhoff-Pore Pooiee,, Andreas Fischer, Fabian Kokot, and Hermann de Meer. A simulation framework for virtual network embedding algorithms. In 2014 16th International Telecommunications Network Strutegy and Planning Symposium (Networks), pp. 1-6, 2014.
507
508
509
510
Yoshua Bengio, Andrea Lodi, and Antoine Prouvost. Machine learning for combinatorial optimization: methodological tour d'horizon. European Journal of Operational Research, 290(2):405 421, 2021.
511
512
513
Andreas Blenk, Patrick Kalmbach, Patrick der Smagt, and Wolfgang Kellerer. Boost online virtual network embedding: Using neural networks for admission control. In 2016 12th International Conference on Network and Service Management (CNSM). pp. 10-18. 2016.
514
515
516
517
Xiang Cheng, Sen Su, Zhongbao Zhang, Hanchi Wang, Fangchun Yang. Yan Luo, and Jie Wang. Virtual network embedding through topology-awar node ranking. ACM SIGCOMM Computer Communication Review, 41():388-47 2011.
518
519
520
Mosharaf Chowdhury, Muntasir Raihan Rahman, and Raouf Boutaba. Vineyard: Virtual network embedding algorithms with coordinated node and link mapping. IEEE/ACM Transactions on Networking, 20(1):206-21 229,,,22922222 2012.
521
522
523
524 N. M. M. K. Chowdhury, M. R. Rahman, and R. Boutaba. Virtual network embedding with coordi- nated node and link mapping. In IEEE International Conference on Computer Communications, pp. 783-791, 2009.
525
526
527
Chinmaya Kumar Dehury and Prasan Kumar Sahoo. DYVINE: Fitness-basee dynamic virtual network embedding in cloud computing. IEEE Journal on Selected Areas in Communications, 37 (5):1029-1045, 2019.
528
529
530
531
Mahdi Dolati, Seyedeh Bahereh Hassanpour, Majid Ghaderi, and Ahmad Khonsari. Deep ViNE: Virtual network embedding with deep reinforcement learning. In IEEE International Conference on Computer Communications Workshops, pp. 879-885, 2019.
532
533
534
Ilhem Fajjari, Nadjib Aitsaadi, Guy Pujolle, and Hubert Zimmermann. VNE-AC: Virtual network embedding algorithm based ant colony metaheuristic. In IEEE International Conference on Communications, pp. 1-6, 2011.
535
536
537
538
Weibei Fan, Fu Xiao, Xiaobai Chen, Lei Cui, and Shui Yu. Efficient virtual network embedding of cloud-based data center networks into optical networks. IEEE Transactions on Parallel and Distributed Systems, (2(11):77932808 2021.
539
10

---

Under review as a conference paper at ICLR 2026
540
Weibei Fan, Fu Xiao, Mengjie Lv, Lei Han, Junchang Wang, and Xin He. Node essentiality assessment and distributed collaborative virtual network embedding in datacenters. IEEE Transactions on Parallel and Distributed Systems, 34(4): 1265)):222211111 1265-12800, 2023.
541
542
543
Xincai Fei, Fangming Liu, Hong Xu, and Hai Jin. Adaptive vnf scaling and flow routing with proactive demand prediction. In JEEE INFOCOM 2018 IEEE Conference on Computer Communications,
544
545
pp. 486-494, 2018.
546
547
Andreas Fischer, Juan Felipe Botero, Michael Till Beck, Hermann de Meer, and Xavier Hesselbach. Virtual network embedding: A survey. IEEE Communications Surveys Tutorials, (144):888-1196 2013.
548
549
550
Xiaoyuan Fu, F. Richard Yu, Jingyu Wang. Qi Qi, and Jianxin Liao. Dynamic service function chain embedding for nfv-enabled iot: A deep reinforcement learning approach. IEEE Transactions on Wireless Communications, 19(1)500-551, 2020.
551
552
553
554
Haoyu Geng, Runzhong Wang. Fei Wu, and Junchi Yan. Gal-vne: Solving the vne problem with global reinforcement learning and local one-shot neural prediction. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 531-543, 2023.
555
556
557
Long Gong, Yonggang Wen, Zuqing Zhu, and Tony Lee. Toward profit-seeking virtual network embedding algorithm via global resource capacity. In JEEE International Conference on Computer Communications, pp. 1-9, 2014.
558
559
560
Shangding Gu, Long Yang, Yali Du, Guang Chen, Florian Walter, Jun Wang, Yaodong Yang, and Alois Knoll. A review of safe reinforcement learning: Methods, theory and applications. arXiv preprint arXiv:2205. 10330, 2022.
561
562
563
564
Qihang Guo, Xibei Yang, Weiping Ding, and Yuhua Qian. Cross-graph interaction networks. IEEE Trans, on Knowl, and Data Eng., 37(5):2341-2355, February 2025, ISSN 1041-4347.
565
566
Shaoyong Guo, Yuanyuan Qi. Yi Jin, Wenjing Li, Xuesong Qiu, and Luoming Meng. Endogenous trusted dri-based service function chain orchestration for iot. IEEE Transactions on Computers, 71 (2):397-406, 2022.
567
568
569
570
Soroush Haeri and Ljiljana Trajković. Vne- sim: A virtual network embedding simulator. In Proceed- ings of 9th EAI International Conference on Simulation Tools and Techniques, SIMUTOOLS' 16, pp. 112-117, 2016.
571
572
573
Soroush Haeri and Ljiljana Trajković. Virtual network embedding via Monte Carlo tree search. IEEE Transactions on Cybernetics, 48(2):510-52) 521, 2017.
574
575
Nan He, Song Yang, Fan Li, Stojan Trajanovski, Liehuang Zhu, Yu Wang, and Xiaoming Fu. Leveraging deep reinforcement learning with attention mechanism for virtual network function placement and routing. IEEE Transactions on Parallel and Distributed Systems, 3464):1186-1201, 2023.
576
577
578
579
580
Haojun Huang. Cheng Zeng, Yangmin Zhao, Geyong Min, Yingying Zhu, Wang Miao, and Jia Hu. Scalable orchestration of service function chains in nfv-enabled networks: A federated reinforcement learning approach. IEEE Journal on Selected Areas in Communications, 39(8):
581
582
2558-2571, 2021.
583
584
Dedy Irawan, AAN Ananda Kusuma, Mohammad Hamdani, Rifqy Fajar Giyana, Arfan Ridwan Hartawan, Toto Bahctiar Palokoto. Tahar Agastani, Widrianto Sih Pinastiko, and Sakinah Puspa Anggracni. Network slicing modeling and simulation on 5g networks using vimee In 2023 International Conference on Computer, Control, Informatics and Its Applications (IC3INA), pp- 114-119. IEEE, 2023.
585
586
587
588
589
Chunxiao Jiang and Peiying Zhang. VNE-HPSO Virtual Nenwork Embedding Algorithm Based on Hybrid Particle Swarm Optimization, pp. 129-152. Springer Singapore, Singapore, 2021.
590
591
592
Panpan Jin, Xincai Fei, Qixia Zhang, Fangming Liu, and Bo Li. Lateney-aware vnf chain deployment with efficient resource reuse at network edge. In IEEE INFOCOM 2020 IEEE Conference on Computer Communications, pp. 267-276, 2020.
593

---

Under review as a conference paper at ICLR 2026
594
Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In Interaational Conference on Learning Representations, 2017.
595
596
597
Wouter Kool, Herke van Hoof, and Max Welling. Attention, learn to solve routing problems! In International Conference on Learning Representations, 2018.
598
599
Alex Krizhevsky, llya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolu- tional neural networks. Advances in neural information processing systems, 25, 2012.
600
601
602
Aris Leivadeas, Chrysa Papagianni, and Symeon Papavassiliou. Efficient resource mapping framework over networked clouds via iterated local search- based request partitioning. IEEE Transactions on Parallel and Distributed Systems, 24: 24:1077-1086, 2013.
603
604
605
Yongshuai Liu, Jiaxin Ding, and Xin Liu. A constrained reinforcement learning based approach for network slicing. In 2020 IEEE 28th International Conference on Network Protocols (ICNP), pp- 1-6, 2020.
606
607
608
609
Sihan Ma, Haipeng Yao, Tianle Mai, Jingkai Yang. Wenji He, Kaipeng Xue, and Mohsen Guizani. Graph convolutional network aided virtual network embedding for internet of thing. IEEE Trans- actions on Network Science and Engineering, 10(1):265-274, 2023.
610
611
612
Horaa Maity, Thang X. Vu, and Symeon Chatzinotas. Revine: Reinforcement bearning-based virtual network embedding satelliteteerestrial networks. IEEE Transactions on Communications, 72 (10):6316-6327, 2024.
613
614
615
616
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning, arXiv preprint arXiv:1312.5602, 2013.
617
618
619
Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim
620
Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforcement learning. In International Conference on Machine Learning, pp. 1928-1937, 2016.
621
622
623 S. Orlowski, R. Wessäly, M. Pióro, and A. Tomaszewski. Sndlib 1.0- survivabbe network design library. Netw., 55(3):276-28) 286, may 2010. ISSN 0028-3045.
624
625
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. volume 32, pp. 8026-8037, 2019.
626
627
628
629
Matthias
Rost and Stefan Schmid. On the hardness and inapproximability of virtual network embeddings. IEEE/ACM Transactions on Networking, 28(2):791-803, 2020.
630
631
632
Matthias Rost, Elias Döhne, and Stefan Schmid. Parametrized complexity of virtual network embeddings: dynamic linear programming approximations. SIGCOMM Comput. Commun. Rev., 49(1):3-10, February 2019.
633
634
635
Rania Sahraoui, Omar Houidi, and Fetia Bannour. Energy-aware vnf-fg placement with transformer- based deep reinforcement learning. In 2024 IEEE/IFIP Network Operations and Management Symposium (NOMS 2024), 2024.
636
637
638
639
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.063477 2017.
640
641
Nashid Shahriar, Shihabur Rahman Chowdhury, Reaz Ahmed, Aimal Khan, Siavash Fathi, Raouf Boutaba, Jeebak Mitra, and Liu Liu, Virtual network survivability through joint spare capacity allocation and embedding. IEEE Journal Selected Areas in Communications, 36(3):502- 508-558.. 2018
642
643
644
645
646
Ruben Solozabal, Josu Ceberio, Aitor Sanchoyerto, Luis Zabala, Bego Blanco, and Fidel Liberal. Virtual network function placement optimization with deep reinforcement learning. JEEE Journal on Selected Areas in Communications, 38(2):292-303, 2020.
647
12

---

Under review as a conference paper at ICLR 2026
648
Sen Su, Zhongbao Zhang. Alex X. Liu, Xiang Cheng, Yiwen Wang, and Xinchao Zhao. Energy-aware virtual network embedding. JEEE/ACM Transactions on Networking. 2265):1607-1620, 2014.
649
650
651
Ilya Sutskever, Oriol Vinyals, and Quoe V Le. Sequence to sequence learning with neural networks. Advances in neural information processing systems, 27, 2014.
652
653
Richard S. Sutton, David McAllester, Satinder Singh, and Yishay Mansour. Policy gradient meth- ods for reinforcement learning with function approximation. Advances in Neural Information Processing Systems, 12, 1999.
654
655
656
Yawen Tan, Jiajia Liu, and Jiadai Wang- 5g end-to-end slice embedding based on heterogeneous graph neural network and reinforcement learning. IEEE Transactions on Cognitive Communications and Networking, 2024.
657
658
659
660
Hiren Kumar Thakkar, Chinmaya Kumar Dehury, and Prasan Kumar Sahoo. Muvine: Multi-stage virtual network embedding in cloud data centers using reinforcement learning-based predictions. 2020.
661
662
IEEE Journal on Selected Areas in Communications, 38(6):
663
Aleteng Tian, Bohao Feng, Yunxue Huang, Huachun Zhou, Shui Yu, and Hongke Zhang. Drl-based two-stage sfc deployment approach under latency constraints. In IEEE INFOCOM 2024 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), pp. 1-6, 2024.
664
665
666
667
Stefan Tomaszek. Modellbasierte einbettung von virtuellen netzwerken in rechenzentren. 2021.
668
Hans Van Vliet, Hans Van Vliet, and JC Van Vliet. Software engineering: principles and practice, volume 13. John Wiley & Sons Hoboken, NJ, 2008.
669
670
671
Runzhong Wang, Yang Li, Junchi Yan, and Xiaokang Yang. Learning to solve combinatorial optimization under positive linear constraints via non-autoregressive neural networks. arXiv preprint arXiv:2409.044: 2024a.
672
673
674
Tianfu Wang, Qilin Fan, Xiuhua Li, Xu Zhang, Qingyu Xiong, Shu Fu, and Min Gao. DRL- SFCP: Adaptive service function chains placement with deep reinforcement learning. In IEEE International Conference on Communications, pp. 1 1-6, 2021.
675
676
677
678
Tianfu Wang, Li Shen, Qilin Fan, Tong Xu, Tongliang Liu, and Hui Xiong. and resource allocation of virtual network embedding via hierarchical deep reinforcement learning. IEEE Transactions Services Computing, pp. 1-14, 2023.
Joint admission control
679
680
681
Tianfu Wang, Qilin Fan, Chao Wang, Long Yang, Leilei Ding, Nicholas Jing Yuan, and Hui Xiong. Flagvne: A flexible and generalizable reinforcement learning framework for network resource allocation. In International Joint Conference on Artificial Intelligence, 2024b.
682
683
684
Tianfu Wang, Long Yang. Chao Wang, Chuan Qin, Liwei Deng, Li Shen, and Hui Xiong. Towards constraint- learning for resource allocation in nfv-enabled networks. In ICML 2025 Workshop on MI44Wrreen 2025.
685
686
687
688
Wenbo Wang. Xiaolin Chang, Jiqiang Liu, and Bin Wang. Simulated annealing based resource allocation for cloud data centers. In Proceedings of the 15th Anmual Conference Companion on Genetic and Evolutionary Computation, GECCO '13 Companion, pp. 81-82, 2013.
689
690
691
Wenzhe Wang. Peilin Hong, Defang Lee, Jianing Pei, and Lei Bo. Virtual network forwarding graph embedding based on tabu search. In 2017 9th International Conference on Wireless Communica- tions and Signal Processing ( (WCSP), pp. 1-6, 2017.
692
693
694
695
Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S Yu. Heterogeneous graph attention network. In The world wide web conference, pp. 2022-2032, 2019.
696
697 Xu Wang, Sen Wang, Xingxing Liang, Dawei Zhao, Jincai Huang, Xin Xu, Bin Dai, and Qiguang Miao. Deep reinforcement learning: A survey. IEEE Transactions on Neural Networks and Learning Systems, 35(4):5064-5078 2022.
698
699
700
701
B.M. Waxman. Routing of multipoint connections. JEEE Journal on Selected Areas in Communica- fions, 611:161771 1988.
13

---

Under review as a conference paper at ICLR 2026
702
Sheng Wu, Ning Chen, Ailing Xiao, Peiying Zhang, Chunxiao Jiang, and Wei Zhang. Ai-empowered virtual network embedding: A comprehensive survey. IEEE Communications Surveys Tutorials,
703
704
pp. 1-1, 2024a.
705
706
Sheng Wu, Ning Chen, Ailing Xiao, Peiying Zhang, Chunxiao Jiang, and Wei Zhang. Ai-empowered virtual network embedding:a comprehensive survey. IEEE Communications Surveys & Tutorials, 2024b.
707
708
709
Xiancui Xiao. Dvne-dri: dynamic virtual network embedding algorithm based deep reinforcement learning. Scientific Reports, 13(1):19789, 2023.
710
711
712
Yikai Xiao, Qixia Zhang, Fangming Liu, Jia Wang, Miao Zhao, Zhongxing Zhang, and Jiaxing Zhang. Nfvdeep: Adaptive online service function chain deployment with deep reinforcement learning. In IEEF/ACM International Symposium on Quality of Service, 2019.
713
714
715
Lingyi Xu, Hefei Hu, and Yuanan Liu. Sfesim: a network function virtualization resource allocation simulation platform. Cluster Computing, 26(1):423 436, July 2022. ISSN 1386-7857.
716
717
718
Zhongxia Yan, Jingguo Ge, Yulei Wu, Liangxiong Li, and Tong Li. Automatic virtual network embedding: A deep reinforcement learning approach with graph convolutional networks. IEEE Journal on Selected Areas in Communications, 3866))1040-100 2020.
719
720
721
Song Yang. Fan Li, Stojan Trajanovski, Ramin Yahyapour, and Xiaoming Fu. Recent advances of resource allocation in network function virtualization. IEEE Transactions on Parallel and Distributed Systems, 32(2):295-314, 2021.
722
723
724
725
Haipeng Yao, Sihan Ma, Jingjing Wang, Peiying Zhang, Chunxiao Jiang, and Song Guo. A continuous- decision virtual network embedding scheme relying on reinforcement learning. IEEE Transactions on Network and Service Management, 17(2):864 875, 2020.
726
727
728
Bo Yi, Xingwei Wang, Keqin Li, Min Huang, al. A comprehensive survey of network function virtualization. Computer Networks, 133:212-282 262, 2018.
729
730
731
Peiying Zhang, Haipeng Yao, and Yunjie Liu. Virtual network embedding based on computing, network, and storage resource constraints. IEEE Internet Things Journal, 5(5):3298-333 3304, 2018.
732
733
Peiying Zhang, Haipeng Yao, Maozhen Li, and Yunjie Liu. Virtual network embedding based on modified genetic algorithm. Peer-to-Peer Networking and Applications, 12(2):481-49) 2019.
734
735
736
Peiying Zhang, Chao Wang, Chunxiao Jiang, and Abderrahim Benslimane. Security-aware virtual network embedding algorithm based on reinforcement learning. IEEE Transactions on Network Science and Engineering, 8: 100,55-1155, 202
737
738
739
740
Peiying Zhang, Yu Su, Jingjing Wang, Chunxiao Jiang, Ching Hsien Hsu, and Shigen Shen. Reinforce- ment learning assisted bandwidth aware virtual network resource allocation. IEEE Transactions on Network and Service Management, 19(4):411 111 111-1121,,,1 2022.
741
742
743
Peiying Zhang, Yi Zhang, Neeraj Kumar, Mohsen Guizani, Ahmed Barnawi, and Wei Zhang. Energy- aware positioning service provisioning for Gooud-edge-vehiche collaborative network based on drl and service function chain. IEEE Trunsuctions on Mobile Computing, 23(4):3424-3435, 2024a.
744
745
746
747
Qixia Zhang, Fangming Liu, and Chaobing Zeng. Online adaptive interference-awaree vnf deployment and migration for 5g network slice. IEEE/ACM Transactions on Networking, 2155)22 2021b.
748
749
750
Xiaoquan Zhang, Lin Cui, Fung Po Tso, Zhetao Li, and Weijia Jia. Dapper: Deploying service function chains in the programmable data plane via deep reinforcement learning. IEEE Transactions on Services Computing. pp. 1-14, 2023.
751
752
753
754 Yi Zhang, Peiying Zhang, Chunxiao Jiang. Shangguang Wang, Hongxia Zhang, and Chumming Rong. Qos aware virtual network embedding in space-air-ee aoieee-ii--rouune ground-ocean integrated network. IEEE Transactions Services Computing, 1714):11121122) 1723, 2024b.
755
14

---

Under review as a conference paper at ICLR 2026
756
Hang Zhao, Qijin She, Chenyang Zhu, Yin Yang, and Kai Xu. Online 3d bin packing with constrained deep reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 741-749, 2021.
757
758
759
Jianan Zhou, Yaoxin Wu, Wen Song. Zhiguang Cao, and Jie Zhang. Towards omni-generalizable neural methods for vehicle routing problems. In International Conference on Machine Learning, PMLR, 2023.
760
761
762
pp.
763
Weihua Zhuang, Qiang Ye, Feng Lyu, Nan Cheng, and Ju Ren. MANINFV-empowerss future loV with enhanced communication, computing, and caching. Proceedings of the IEEE, 108(2)277--211, 2020.
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
15

---

Under review as a conference paper at ICLR 2026
810
APPENDIX
811
812
813
A NFV-RA Problem Definitions
18
814
815
Basic Formulation of NFV-RA
18
816
A.I.I Constraint Conditions
18
817
818
A.1.2 I.2 Optimization Objective
18
A.2
Extensions in Emerging Networks
19
820
821
Related Work
19
822
823
B.
Traditional NEV-RA Algorithms
19
824
B.2
RL-based NFV-RA Algorithms
20
825
B.3 Existing NFV-RA Benchmarks
20
827
B.4 RL for Combinatorial Optimization
828
829
Benchmark Details
21
830
831
Simulation Configuration
832
MDP Variation for NFV-RA
22
833
834
C221
Multi-Task MDP for Meta RL Methods
22
835
22,2
Constrained MDP for Safe RL Methods
23
836
837
C,3
Implemented NFV-RA Algorithms
23
838
C.3.1
RL-based Algorithm Implementations
23
839
C33.2
Traditional Algorithm Implementation
25
840
841
C.4
Modular and Extensible Implementation Pipeline
................................
26
842
C.4.1
Modularity Description of Pipeline
26
843
844
C.4.2
Extensibility Practice
27
845
C,5
Evaluation Metrie Definitions
27
846
847
Evaluation Protocol Definitions
28
C.7
Benchmark Documentation
29
849
850
851
D Experiment Details
29
852
D.1
Experimental Setup
29
853
854
D.I.I Implementation Settings
................................
29
D.1.2 Evaluated Physical Topologies
30
856
D.2
Exploration on Implementation Ie..............................
857
858
D.2.1
Impact on RL. Training Methods
31
859
D.2.2
Impact on Reward Function Design
860
861
D.2.3
Impact Feature Engineering Combination
31
862
D.2.4
Impact on Action Mask Strategy
32
863
D.3
Generalization on Network Conditions
32
16

---

Under review as a conference paper at ICLR 2026
864
D.3.1
Evaluation on Varying Traffic Rates
32
865
D.3.2
Evaluation on Fluctuating Demand Distribution
33
867
D.4
Scalability on Network Sizes
35
................................
868
D.4.1
Performance on Large-scale Networks
................................
35
869
870
D.4.2
Analysis of Solving Time Scale
................................
36
871
D.5
Validation on Hmerging Network
37
872
D.3.1
Heterogeneous Resourcing Networks
37
874
D.5.2 Latency-aware Edge Networks
38
875
D.6 Evaluation on Real- Real-World Scenario Datasets
38
876
877
878
Discussion on Future Directions
39
879
E.
Representation Learning for Cross-graph Status
40
881
Robust Learning for Complex and Hard Constraints
40
882
E,3
Scalability in Extremely Large-scale Networks
40
883
Generalizable Policy Cross Varying Scenarios
884
885
886
Limitations Sim-to-Real Gap
41
887
G The Use of Large Language Models Statement
41
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
17

---

Under review as a conference paper at ICLR 2026
918
A
NFV-RA PROBLEM DEFINITIONS
919
920
A.
BASIC FORMULATION OF NFV-RA
921
922
Here, we provide the mathematical formulation of the basic cost minimization model, including constraints and the objective.
923
924
925
A.1.1
CONSTRAINT CONDITIONS
926
During the embedding process of a VN G. onto the PN Gp. we need to decide two types of boolean
927 variables: (1) x:" 1 if virtual node n is placed in physical node nig and O otherwise; (2) Vi.j 12.50 =
928 if virtual link ( = (n"", m²)) traverses physical link l- (mg. nz)) and 0 otherwise. Here, m and w are identifiers for virtual nodes, while i and are identifiers for physical nodes. VN request is successfully embedded if feasible mapping solution is found, satisfying the following constraints:
929
930
931
932
Σ BEEN,
1, Vn E Noo
933
934
935
Σ n EN.
x S $1, Vng E NN:
(3)
936
937
1 Teth C(n")) S C(n)), Vny" EN E Np,
(4)
938
939
940
Σ
Vku
m.w
Σ
Dikk
m, = 2x Fh Vi Los ny E ENp
941
(5)
n E)E((())
942
943
Vi.j m.s + Vj
m,,
< 1, VI
mm E Ly E Exp
(6)
944
(wig + V3 B(mm..) S (((L)))) E Exp.
m.w
m.
945
946
IN EC
947
Here, (nn5) denotes the neighbors of ng. Constraint 2 ensures that every virtual node is mapped to one and only one physical node. Conversely, constraint Dlimits each physical node to hosting at most one virtual node, thus enforcing a unique mapping relationship. This one-to-one mapping constraint enforces intra- intra-request node disjointness, standard requirement in virtual network embedding to ensure survivability and prevent single points of failure (Fischer et al. 2013). Note that this does not preclude inter-request multi-tenancy; virtual nodes from different VNs can share the same physical node, provided resources allow. Constraint 4 verifies that virtual nodes are allocated to physical nodes with adequate resources. Following the principle of flow conservation, constraint (5) guarantees that each virtual link (n n²²) is routed along a physical path from ni (the physical node where n.
948
949
950
951
952
953
FIX
954
955
956 is placed) to ni. (the physical node where n is placed). Constraint (6) eliminates the possibility of routing loops, thereby ensuring that virtual links are routed acyclcally. Lastly, constraint (7) ensures that the bandwidth usage on each physical link remains within its available capacity. Overall, constraints 4 enforce the one-to-one placement and computing resource availability required in the node mapping process. And constraints (5 ensure the path connectivity and bandwidth resource availability required in the link mapping process.
957
958
959
960
961
962
A.1.2
OPTIMIZATION OBJECTIVE
963
964
Revenue-to-Consumpiic (R2C) ratio is widely used metric to measure the quality of solution S = [T). The objective function is maximize the resource utilization as follows:
965
966
max R2C(S)
(REV (S) /COST (5)),
967
968
where x is a binary variable indicating the solution's 's feasibility; x = 1 for a feasible solution and 0 otherwise. When the solution is feasible, REV(S) represents the revenue from the VN, calculated as EnSEN, C(n.) + Eiyc,, B(la). If x = 1, COST(S) denotes the resource consumption of PN, calculated as EN,EN, ((n,) + Ei.ec (lfc((.)) x B(l.)). Here, [fc(4.) quantifies the length of the physical path Pp routing the virtual link I,..
969
970
971
18

---

Under review as a conference paper at ICLR 2026
972
A.2
EXTENSIONS IN EMERGING NETWORKS
973
974
The application of NFV is pivotal in emerging network scenarios, which require additional considera- tions of their unique challenges. Particularly, we discuss several extended definitions of NFV-RA in popular scenarios to better align with their practical requirements.
975
976
977
978
Resource Heterogeneity In modern data center networks, computing resources are often hetero- geneous, meaning physical nodes may have varying capacities in terms of CPU, GPU, memory, To account for this, we extend the basic model by incorporating heterogeneous resources, where both virtual and physical nodes are associated with set of computing resourees C. Thus, a physical node np for placing virtual node ny must have enough resources across all types, i.c., C(n.) S C(np), Vn, N,, C np fx(())-
979 etc.
980
981
982
983
984
Latency Requirement In sime-sensitive networks (e.g., edge computing and 5G), satisfying latency requirements is crucial. We consider the latency requirement of virtual link ly as ((c,) and the incurred latency of physical link pv: as D(pp). The latency of physical path Pp that routes l, should not exceed such specified threshold, i.e., D(pp) S D(I,), where D(pp) = =i,Ep D(4p).
985
986
987
988
989
Energy Efficiency Energy consumption is a significant concern in green data centers due to high sustainability and economic efficiency. Energy-efficient NFV-RA also considers the energy consumption minimization of physical infrastructure. We denote the energy consumed by the physical node n as E(Rp), associated with its status (idle or active) and workload. The objective function is to optimize both resource utilization and energy consumption, formulated as: max -Waa En,EN, E(pp) + wa R2C(S), where Wa and W/ are weights of different objectives.
990
991
992
993
994
995
Note that these definitions extend the basic concepts of modeling, constraints, and objectives. Varia- tions of NFV-RA for other scenarios can be easily derived using the approaches discussed above.
996
997
998
B
RELATED WORK
999
1000
1001 As network scenarios become increasingly diverse and complex (e.g., cloud, edge, 5G), NFV has emerged as a critical paradigm for flexible service provisioning. Efficient resource management in NFV spans the entire service lifecycle, encompassing tasks such as efficient allocation QJi et al. 2020), adaptive scaling (Fei et al. 2018), and dynamic migration (Zhang et al. 20216). Among these, Resource Allocation in NFV (NFV-RA) forms the foundation. It is typically formulated as
1002
1003
1004
1005
an embedding or placement problem, with the central challenge of mapping virtual requirements onto physical infrastructure. While many studies investigate NFV-RA in emerging networks while addressing additional factors (e.g., latency, energy, heterogeneity), their formulations often follow similar methodological backbone. This shared structure enables a unified perspective on how NFV-RA techniques have evolved over time. In this section, we focus on the methodological aspects and review the development of key approuches in NFV-RA. Then, we describe existing benchmarks
1006
NEW
1007
1008
1009
NEW
1010
1011
to emphasize the gap between Virne Finally, we compare NFV-RA with other COPs, highlighting the need for specific designs.
1012
FIX
1013
1014
B.
TRADITIONAL NFV-RA ALGORITTMM
1015
Earlier studies for the NFV-RA problem employed exact solvers, such integer linear programming (Shahrhh et al. 2018) and mixed integer programming (MhP))Chowdhury et al., 2009), to find optimal solutions. However, their high computational complexity makes them impractical for real-world scenarios, especially in larger networks where dynamic service requests require rapid solutions. To address these limitations, heuristic-based methods emerged as alternatives, offering suboptimal yet computationally efficient solutions. Among these, node-ranking strategies stand out as a prominent approach (Gong et al.) 2014 Zhang et al. 2018 Fan et al., 2021 2023). These strategies construct solutions by prioritizing the virtual and physical nodes based specific metrics to guide
1016
1017
1018
1019
1020
1021
1022
1023
the node mapping process, followed by link mapping. For instance, Node Resource Management (NRM) (Zhang et al. 2018) ranks nodes by weighting both node and link resources, while the Node Essentiality Assessment (NEA) (Fan et al. 2023) incorporates topology connectivity into decision-
1024
1025
making. Although reducing computational overhead, they rely heavily on manual design with
19

---

Under review as a conference paper at ICLR 2026
1026 in complex NFV-RA requirements. Furthermore, meta-heerisste have been adopted to solve NFV-RA
domain-specific knowledge and lack adaptation to diverse scenarios, which can lead to inferior results
1027
1028 by modeling the solution search as evolutionary process (Dehury & Sahoo 2019 Leivadeas et al.
1029 2013] Fajjari et al. 2011). These algorithms, such as Genetic Algorithms (GA) (Zhang et al. 2019
1030
Dehury & Sahoo 2019) and Particle Swarm Optimization (PSO) (Su et al. 2014;Jiang & Zhang
1031
2021). explore the solution space by iteratively evolving a population of candidate solutions. However,
1032
they are computationally expensive and byperparameeeer-senttiiiee limiting their practicality.
1033
1034
1035
B.2
RL-BASED NFV-RA ALGORITHMS
1036
1037
In recent years, machine learning bMLed methods have gained prominence in solving NFV-RA problems due to their superior performance and adaptability dynamic network conditions (Wu et al.,
1038
1039 2024a). Several works (Blenk et al. 2016: Thakkar et al.) 2020) have explored predictive models
trained on high-quality datasets forecast the future service requests. However, acquiring labeled, high-quality datasets for large-scale and unseen network scenarios remains impractical, limiting
1040
1041 their applicability. More dominantly, RL has demonstrated its promise for NFV-RA tasks (Haeri
1042 & Trajkovid 2017 Xiao et al.) 2019 Yao et al., 2020: Wang et al. 2021 Yan et al. [2020] Zhang
1043
al., 2022; Dolati et al. [201 Wang et al., 2023; Geng al. 2023] Wang al. (2024b). mainly
1044
due to its label-free nature and adaptation to handle dynamics of network. RL-based methods model
1045
NFV-RA as a Markov Decision Process (MDP), allowing an agent to learn optimal policies through
1046
iterative interactions with the environment. In general, we unified existing RL-bbased methods for
1047
NFV-RA under a framework with three core components: MDP modeling, policy architecture, and training methods. These methods conceptualize the node mapping allocation process as a sequential decision-making task, where physical nodes are incrementally selected to host VNFs. To execute these decisions, various policy network architectures
1048
1049
designed to represent the network state and generate corresponding actions. Then, they leverage a specific RL method to optimize the policy network with collected experience during interactions. For example, (Xiao al., 2019) utilized a multilayer perceptron (MLP) and trained it using the policy gradient (PG) algorithm, while the work (Yan et al., 2020) combined MLPs with graph convolutional networks (GCNs) and employed the asynchronous ad advantage actor-critic (A3C) algorithm. While RL-based methods offer notable advantages in terms of performance and adaptability, there is lack of in-depth analysis of the impact of specific model design choices and consistent performance comparison. Moreover, systematically identifying the remaining significant challenges in RL-based NFV-RA is critical to open opportunities for future exploration and innovation.
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
B.3
EXISTING NFV-RA BENCHMARKS
1062
1063
For the NFV-RA problem, several benchmarks have been developed for simulation and evaluation. As
1064
summarized in Table I the most notable existing benchmarks include: VNE-Sim Haari & Traiković
2016). ALEVIN Beck et al., 2014), ALib" (Beck al. (2014). SFCSim (Xu et al., (022). and
1065
lfl Tomaszek 2021). For instance, VNESim(Haeri & Trajković 2016) supports three heuristic
1066
algorithms and focuses exclusively on cloud-based simulations, evaluating solutions based on their
1067
effectiveness. Similarly, SFCSim (Xu et al.] 2022). incorporates three additional heuristic methods,
1068
continuing the trend of focusing on cloud-based scenarios However, most of these benchmarks are limited to cloud-based simulations and lack the flexibility to accommodate more modern network scenarios, such as edge computing or 5G environments. Moreover, these benchmarks generally implement only few exact and heuristic algorithms and focus mainly effectiveness evaluation.
1069
1070
1071
1072
These limitations highlight the need for more comprehensive benchmarks that can assess algorithms across a broader range of network environments and evaluation perspectives.
1073
1074
1075
1076
hhtps://tehreemmf..ccm wwxste..wwwwwiiieewe.ece commvne-si/vvveeooeeeee
1077
http:://ounneeraggeeeeeeedlliiie httpl/ewwwwwww///onn/eeeee wiik/hhom/// '
1078
1079
hhtps://gthuh.comm/
20

---

Under review as a conference paper at ICLR 2026
1080
B.4
RL FOR COMBINATORIAL OPTIMIZATION
1081
1082 RL has gained sigmificant attention for solving Combinatorial Optimization Problems (COPs), where the goal is to learn high-quality solutions (Bengio al. 202I). These problems span various domains, including the Traveling Salesman Problem (TSP) Kool et al. 2018), Vehicle Routing Problem (VRP) (Zhou et al.) 2023). and bin packing (Zhao et al. (021). Ri.-based approaches to solving these problems generally fall into two categories: construction methods that build solution incrementally
1083
1084
1085
1086
from scratch and improvement methods that start with an initial solution and iteratively refine
1087
Compared with these classic COPs, the NFV-RA problem presents unique challenges, highlighting
1088
the need for specialization of RL approach design. First, NFV-RA requires real-time decision- meet strict network service requirements. NFV-RA algorithms must execute decisions almost immediately, making construction methods more suitable to incrementally build solutions within time constraints. Second, NFV-RA operates in highly dynamic environments where both service demands and network resources fluctuate in real time. This variability requires RL methods that can
decision-making making
1089
1090
1091
1092
1093
adapt changes in network topologies, resource availability, and demand patterns. Lastly, the state space of NFV-RA is highly complex, involving intricate interactions such as cross-graph mapping and cowssidtte-eottriieeedee routing. These factors require RL methods to navigate a multidimensional decision space, accounting for diverse resource types and connectivity requirements.
1094
1095
1096
1097
1098
C BENCHMARK DETAILS
1099
1100
C.I SIMULATION CONFIGURATION
1101
1102
Vire offers a highly customizable simulation framework designed to accommodate the diverse and complex nature of modern network environments. As illustrated in Figure users can define the net- work scenarios and conditions within the simulator through configuration files. These configurations
1103
1104
1105
cover two key components: PN settings and VN requests, providing users the flexibility to simulate variety of network scenarios.
1106
1107
1108
Network Topologies Virne provides a range of customizable network topologies to simulate differ-
1109
ent network conditions. Users can select between: (a) synthetic topologies that are generated using algorithms such as Waxman and FatTree, which provide structured and predictable network designs for controlled testing; and (b) realistic topologies that are sourced from SDNLib and the Topology Zoo, offering more realistic and complex network structures that mimic real-workd environments. Users can specify both the PN and VN topologies through their configuration files, allowing the reflection of various network scales and topological characteristics.
1110
1111
1112
1113
1114
1115
Resource Availability Virne supports the customization of resource availability across multiple levels of the network, including nodes, links, and graph. This flexibility enables simulations that represent differrnnt resource conditions, such as: (a) resource types: Users can define various types of resources, including computing resources (e.g., CPU, GPU, memory) and network resources (e.g., bandwidth). This allows for the modeling of diverse resource requirements in both PN and VN settings; (b) availability distribution: The resources can be distributed across the network using different statistical models, such as uniform or exponential distributions, which reflect real-world variations in resource supply and demand. This level of customization provides users with the ability to model networks with differing levels of resource availability.
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
Service Requirements Vire allows users to define specific service requirements, which add complexity and realism to the simulations. These requirements can be tailored to address wide range of network use cases, reflecting varying demands and constraints. As illustrated in Figure we introduce three key types of service awareness that can be customized:
1126
1127
1128
1129
Heterogeneous resources. To configure this, users can add two new types of node resources (e.g., CPU, GPU) into the node_attrs cattrs_settiine section of the PN and VN configuration files.
1130
1131
1132
Latency constraints. By introducing the latency into the link attrs setting section of the PN and VN configuration files, users can specify maximum latency thresholds for virtual links and the physical paths used for routing.
1133
21

---

Under review as a conference paper at ICLR 2026
1134
Energy efficiency. By including energy consumption parameters into the graph aaph_aatrs attrs setting section of the PN configuration files, users can simulate green data centers and optimize resource allocation with sustainability in mind.
1135
1136
1137
1138
Through these service requirements, Virne enables the simulation of networks with diverse operational demands. Moreover, Virne supports adaptable constraint modeling to accommodate specific scenarios. For instance, users can enable VNF aggregation strategies (e.g., for SFC) by relaxing node disjointness constraints via the reusable parameter. This provides users with the flexibility to model complex
1139
1140
1141
1142
1143
Table 4: Summary of existing studies on RL-based NFV-RA algorithms. (a) Existing studies investigate RL-based NFV-RA algorithms various network scenarios, such as cloud computing and latency-aware edge computing. (b) These studies employ different RL methods, such as DQN and PPO, to optimize customized neural policies, including CNN- or GNN-based architectures. (c) They also vary implementation techniques, including whether they incorporate intermediate rewards in the reward function, use action masking mechanisms to prevent selecting nodes with insufficient resources, leverage topological features augmented inputs. (d) Additionally, we summarize the benchmarks used in these studies, noting whether they rely on existing benchmarks or use custom ones that are not publicly available.
1144
1145
1146
1147
1148
1149
1150
1151
1152
Network Secmario
Care Rl. Framework
Implementation Techniques
Code Rase
Network
Additional Consiivntions
Training
Niuno!
Intermediate Rewards
Action
Zepologica! Features
Used Benchoark
1153
Synew
Method
Policy
Masking
1154
el
2023
Edge Eddge
A2C
LSTM. Attention
X
X
Not available
Zhing
Energy, Lateney
CNN
Not availbble
1155
2014
56
Socurity
Hatroggseous GEN
Not available
1156
Ja m
2000
5C Slicing
Larency
PPO
ML.P
Not masitable
3G
Lueecyy
GEN
X
Virne
1157
Ln 21 m
1024
Internet Things
Larency
PPO
GNN
Not available
1158
Larency
X
available
Guo el 3
2022
Internet Things
A3C
MLP
Not available
1159
Maity d
Satellite
Lutency
DQN
X
1160
Zhang ct
Zhancet 0133
2024h
Spuce-Air Grounudd
Data Planc
I Larency
Latency
A3C
GCN
CNN
X
X
Not available
Not available
1161
Hopri Tralovid
7
Cloud
MCTS
X
VNE
1162
Joba el 2019
Cloud
PG
CNN
Not available
1163
Xino 2099
Cloud
Laency
Self impllmmmeeeedd
Selozabol m k020
Cloud
Energy
PG
LSTM: Attation
Sclf-iiplemmnted
1164
Cloud
RNN
Not available
Yan al. 3020)
Cloud
A3C
GEN
VNE--Si
1165
201
Cloud
VNE--Sim
1166
Wang et 2021
Cloud
A3C
GCN, GRU
Virne
1167
Zhang
Ma <t 2033)
Cloud
Cloud
Secunity
PG
GEN
CNN
Not available
1168
Cloud
GCN,
VNE
Ciog 204
Cloud
PPO
GNN
Virne
1169
Xaa 1013
Cloud
1170
Wing ct 2023
Cloud
PPO
Edge-aware GAT
Virne
Shnavaa et
2024
Cloud
Energy
A2C
Anention
Virne
1171
Wang 2024h
Cloud
Heteragencity
Clos--UUN
Virne
1172
Wang el 2033
Cload, Fdge
Larency
PPO
Hetamgencous GAT
Virne
1173
1174
and dynamic network systems beyond standard definitions.
NEW NEW
1175
1176
C.2
MDP VARIATION FOR NFV-RA
1177
Apart from the MDP formulations introduced in Section 3.3.1 several extended MDPs for NFV-RA are also proposed to handle the specific aspects of NFV-RA problems.
1178
1179
1180
C.2.1
MULTI-TASK MDP FOR META RL METHOPS
1181
1182
In practical network systems, VNRs often exhibit significant diversity in their characteristics, such as varying sizes, topologies, and resource demands. standard single-policy approach struggles to generalize across this wide spectrum. To address this, multi-task MDP formulation can be employed, framing the embedding of different categories of VNRs as distinct tasks. Specifically, VNRs are grouped into tasks M, drawn from a distribution p(M). The objective is to train meta-policy To that captures common, transferable strategic knowledge across all tasks. This meta-policy can then be rapidly adapted with minimal data derive specialized sub-policies ma for each task. The
1183
1184
1185
1186
1187
22

---

Under review as a conference paper at ICLR 2026
1188
meta-oppimizztion objective is to find meta- meta- parameters that maximize the expected performance over the task distribution after adaptation:
1189
1190
max (MMp)( [J(O)] = EM.rp(M) (F(((D)))
(9)
1191
1192
where (0,) is the expected return for the policy тe, task M,, and 0; = (D,) represents the parameters of the sub-policy adapted from the meta-policy & using a small amount of task-specific experience D. The meta-policy is then updated by aggregating gradients from the adapted policies, for instance, using a second-order meta- veta-gradient over the mask-specific losses.
1193
1194
1195
1196
1197
C.2.2
CONSTRAINED MDP FOR SAFE RL METHODS
1198
The NFV-RA problem is fundamentally defined by its hard resource constraints, where any violation renders solution infeasible. Standard MDPs, which typically integrate constraint violations into the reward signal via simple penalties or doing nothing, often fail to guarantee zero-violation solutions.
1199
1200
1201
more rigorous approach is to formulate the problem as Constrained CMDP. This method explicitly separates the primary reward from costs associated with constraint violations and is formally expressed tuple (S, A, P. R. H, C,y). Here, in addition standard MDP components, H is violation function that measures the degree of constraint violation at each state, and is a cost function that
1202
1203
1204
1205
maps violations non-negative cost, i.e., C(s) max(H (s))H(s), 0). The objective is to learn policy that maximizes the expected cumulative reward (T) while ensuring the expected cumulative cost (z) remains below a predefined threshold d (e.g., g.. 0):
1206
1207
1208
1209
max J,(TT) = EYon
rd R(ss an)
1210
t=
(10)
1211
1212
s.t. Jo(T) = ETNn
Σ (C(ss)
<d
1213
1214
To enforce stricter state-wiss safety, this model can be enhanced with reachability analysis. This extension aims to ensure the policy remains within feasible region where future constraint violations are avoidable, often by optimizing an objective that guarantees the worst-case long-term violation, or feasible value function VíA" (s), remains non-positive.
1215
1216
1217
1218
1219
C.3
IMPLEMENTED NFV-RA ALGORITHMS
1220
1221
C.3.1
RL-BASED ALGORITHM IMPLEMENTATIONS
1222
Our RL-based implementations are structured around core components: RL training algorithms that define the learning paradigm, neural policy architectures that parameterize the agent's decision- making function, and additional techniques that can enhance learning or address specific challenges NFV-RA. This modularity allows researchers to easily experiment with different combinations and contribute new components.
1223
1224
1225
1226
1227
1228
RL training Methods Virne supports several foundational and advanced RL. training algorithms, which are used to guide the learning process of the neural policies.
1229
1230
Monte Carlo Tree Search (MCTS) (Haeri & Trajković 2017 is a planning algorithm that explores the decision space by building search tree, balancing exploration and exploitation to find action sequences.
1231
1232
1233
Policy Gradient (PG or REINFORCE) Sutton el al. 1999) policy-based RL method that learns parameterized policy maximize expected returns, i.e., solution quality.
1234
1235
1236
Asynchronous Advantage Actor-Critic (A3C) (Mnih et al. 2016) is an actor-critic-based RL algorithm that uses multiple parallel actors, each interacting with its own copy of the environment. It learns both a policy (actor) and value function (critic) to estimate the advantage of taking certain actions, leading to more efficient learning than pure PG methods.
1237
1238
1239
1240
Proximal Policy Optinization (PPO) (Schulman al. (017) is a popular RL method known for its stability and strong empirical performance across a wide range of tasks. achieves policy optimization stability by using clipped surrogate objective.
1241
23

---

Under review as a conference paper at ICLR 2026
1242
Deep Q-Network (DQN) and variants (Muih et al. 2013) are value-based RL methods that aim to learn the state-action value space. However, NFV-RA often involves large, structured action spaces not directly amenable to these methods.
1243
1244
1245
1246
Neural Policy Architectures The neural policy architecture defines how the RL agent perceives the environment (state representation) and decides on actions. Virne implements a range of architectures to capture attribute and structural information in both the PN and VN.
1247
1248
1249
1250
Muyri-Layer Perceptron (MLP)-based Policy (Liu et al. 2020 Maity al. 2024 Xiao 2023
Xiao et al. 2019). It concatenates the features of the current to-be-placed virtual node with every physical node's s features. Then, it fed them into multiple fully connected layers and outputs probabilities for selecting each physical node for placement.
1251
1252
1253
1254
Convolutional Neural Network ( (CNN)-based Policy (Zhang et al., 2024a Fu et al. 2020 Zhang et al. 2024b 2021a Dolati et al. 2019). They use CNN to process graph-structured data by treating node features and adjacency information as grid-like inputs. Similar to the MLP, virtual link demands are integrated as node features.
1255
1256
1257
1258
Attention-based Policy(He et al. 2023 Solozabal et al. 2020). It first embeds the features of the current to-be-placed virtual node to form a query. Then, the features of each physical node are similarly embedded form keys and values. The attention mechanism then computes weighted sum of physical node values based on their compatibility with the virtual node query.
1259
1260
1261
1262
1263
Graph Convolutional Network (GCN)-based policy (Zhang et al. 2023 Ma et al. 2023
1264
Yan et al. 2020 It uses GCN to learn node representations by aggregating information from their local neighborhoods. In Virn,, GCNs are used to process the PN topology, often with virtual node demands incorporated as additional node features on the PN nodes.
1265
1266
1267
Graph Convolutional Network (GCN)-based) policy. It uses GCN to learn node representa- tions by aggregating information from their local neighborhoods. GCNs is mainly used to process the PN topology, often with virtual node demands incorporated as additional node features on the PN nodes.
1268
1269
1270
1271
GCN Seceenccett-SSquenee (GCN&S2S) Policy (Wang et al.) 2027). This hybrid architec- ture combines GCNs for graph representation learning with a Sequence-to-Sequemee (S2S) model (Sutskever et al. 20144, an RNN-based encoder-decoder with attention, to generate ordered sequence of actions.
1272
1273
1274
1275
GAT&S2S Policy Wang et al. 2023). Similar to GCN&S2S, but replaces the GCN encoder(s) with GAT(s).
1276
1277
Graph Attention Network bbaee- policy (Wang al. 2023). GATs extend GCNs by incorporating attention mechanisms into the neighborhood aggregation process. Particularly, it also incorporates edge features (e.g., link bandwidth, latency) into the attention calculation.
1278
1279
1280
Dual GCN (DualGGN)-basee policy Wang et al. 2024b). This architecture uses two separate GCNs, i.e., one to embed the topology and features of the VN, and another for the PN. The embeddings from both graphs are then combined into an MLP to make placement.
1281
1282
1283
Dual GAT' (DualGAT)-basee Policy (Wang et al. GATs for both VN and PN embedding.
2024b): Similar to DualGCN, but it utilizes
1284
1285
Heterogeneous Graph Attention Network (HeteroGAT)-basse Policy Tan et al. 2024 Wang et al. 2025). Recent studies model NFV-RA instances as a heterogeneous graph. where virtual nodes, physical nodes, virtual links, and physical links are different types of nodes/edges with distinct features. Additionally, HeteroGAT introduces the cross-graph connection links to represent the historical mapping status, i.e, if virtual node is placed onto a physical node, a heterogeneous link will be created.
1286
1287
1288
1289
1290
1291
1292
Additional Implementation Techniques
1293
1294
Reward Function Design. NFV-RA is characterized by its combinatorial space and complex constraints for NFV-RA, which may present a challenging sparse reward problem. Virne allows exploration of different reward structures.
1295
24

---

Under review as a conference paper at ICLR 2026
1296
1297
Mhapi MELUE Mil velast www..ienniale
Extension Energy effidinn Networks
1298
Dawendon & Resuraa-bteeoppnuu Netwerks
1299
Loy
1300
prntian
pracction tra
Pope: Fost
1301
1302
1303
Ca
1304
1305
High
1306
Lyprr
Opt.
1307
20ke: Ba
Extension B: Lutency--eriitieee Networks
1308
oiner: Tine
Diner: 134
Lear
1309
Eyper
duccen 290.
aiclare
1310
1311
1312
Bosic Configuratoon Viis
1313
Figure 5: An example of basic configurations on both PN and VNs, along with their extensions. By adding specific settings on the levels of node, link, or graph, Virne can be easily extended to support emerging networks with additional awareness.
1314
1315
1316
1317
No Intermediate Reward (NoIR). The agent only receives a terminal reward based on the final R2C(S) if the entire VN is successfully embedded, and 0 otherwise.
1318
1319
Fixed Intermediate Reward (FIR). The agent receives a small positive fixed reward (ImR_valuee value) for each successful virtual node placement and link routing step. A negative reward is given for failed steps. The final R2C(S) is added to the sum of intermediate rewards if the overall embedding is successful.
1320
1321
1322
1323
Adaptive Intermediate Reward (AIR). A specialized version of FIR considers [W.] as an intermediate reward value, where |Noll is the number of virtual nodes in the VN. This normalizes the scale of intermediate rewards based on the complexity of the VN.
1324
1325
1326
Feature Engineering Combinations. Raw node/link attributes and basic topology might not be sufficient for optimal RL performance. Some studies use engineered features to augment the input to neural policies. These include:
1327
1328
1329
(a) Node Embedding Status. Binary flags indicating whether a physical node is currently hosting a virtual node or whether a virtual node has already been placed.
1330
1331
(b) Topological Features. Standard graph centrality measures computed for both PN nodes and VN nodes, i.e., degree, closeness, betweenness, and eigenvector.
1332
1333
Action Masking Mechanism. During training, not all actions (c.g.. placing a virtual node on a physical node) are valid at every step due to resource constraints or additional service requirements. Action masking a technique where invalid actions are explicitly disallowed by placing the selection probability with zero
1334
1335
1336
1337
1338
C.3.2
TRADITIONAL ALGORITHM IMPLEMENTATION
1339
Virne implements 10+ traditional algorithms, which can be broadly categorized into three primary types: exact and rounding methods, node ranking- based approaches, and meta-heuristic methods. We summarize these key algorithms and their core strategies in Table 5 Next, we elaborate on each category and introduce implemented algorithms as follows.
1340
1341
1342
1343
1344
Exact and rounding methods for NFV-RA either guarantee optimal solutions or provide approxi- mate solutions through rounding techniques, mainly based on mathematical solvers.
1345
1346
MIP (Chowdhury al. 2009):
Mixed-Integer Programming (MIP) is an exact method that provides optimal solutions by solving a system of linear equations with integer constraints.
1347
1348
1349 R- Rounding (Chowdhury et al. 2012): A rounding method that applies random rounding to generate approximate solutions for NFV-RA.
25

---

Under review as a conference paper at ICLR 2026
1350
D Rounding (Chowdhury et al. 2012): This method uses deterministic rounding map continuous variables to discrete values while providing an approximation to the optimal solution.
1351
1352
1353
1354
Node ranking-based methods for NFV-RA that first prioritize nodes using different strategies for node mapping, then exccute the link mapping stage with the shortest path algorithm.
1355
1356
NRM-Rank (Zhang et al.) 2018: heuristic method based on Node Resource Management (NRM) metric. This approach ranks virtual and physical nodes before mapping with a greedy selection method.
1357
1358
1359
GRC-Rand (Gong al. 2014): This method uses a Global Resource Control (GRC) strategy based on random walks for node ranking. After ranking, the nodes are mapped accordingly.
1360
1361
1362
201 y: A node ranking approach based on random walks to estimate NEA-Rank (Fan et al.
2023]: It ranks nodes using the Node Essentiality Assessment (NEA).
1363
RW-Rank (Cheng et al.
1364
node priorities in the embedding process.
1365
PL-Rank (Fan et al., (202T): This algorithm considers node proximity and evaluates paths comprehensively, ranking nodes based on their location priorities and the overall efficiency of the physical infrastructure.
1366
1367
1368
1369
Meta-heuristic methods for NFV-RA employ nature-inspired processes to iteratively improve solutions. Viree implements the following meta-heuristies for NFV-RA:
1370
1371
GA-Meta (Zhang et al. 2019) A meta-heuristic method based on genetic algorithms.
1372
models each node mapping solution chromosome and iteratively explores the solution space by simulating the process of genetic selection and evolution.
1373
1374
1375
PSO-Meta (Su et al. 2014): A meta-heuristic using Particle Swarm Optimization (PSO), where partieles explore the NFV-RA solution space by adjusting their positions based on their own and neighbors' experiences.
1376
1377
1378
ACO-Meia (Fajaari et al. 2011): A meta-heuristic based on Ant Colony Optimization (ACO) that simulates ant foraging behavior explore the NEV-RA solution space and update pheromone trails to guide the search.
1379
1380
1381
SA-Meta (Wang et al. 2013): A meta-heuristie using Simulated Annealing (SA) to approx- imate the global optimum, exploring the NFV-RA solution space with both upward and downward transitions to escape local minima.
1382
1383
1384
TS-Meta (Wang et al. 2017): A meta-heuristic based on Tabu Search (TS), which uses a memory structure to store visited solutions, promoting diversity and avoiding local optima.
1385
1386
1387
C.4 MODULAR AND EXTENSIBLE IMPLEMENTATION PIPELINE
1388
1389
To facilitate rapid implementation and community-drive development, Virne is with a highly modular and extensible implementation pipeline for RL-based methods. This design is guided by the principle of separation of concerns, where distinct functional components of the RL workflow can be independently developed, configured, and interchanged. This section outlines the architecture of this
1390
1391
1392
pipeline and a customized configuration example, as illustrated in Figures 6 and 7
1393
1394
C.4.1
MODULARITY DESCRIPTION OF PIPELINE
1395
1396
As shown in Figure 6 our pipeline consists of the following high-level, interoperable modules.
1397
Instance Environment simulates the NFV system and handles the agent's interactions. It is designed
1398 with customizable components, including the Action Controller, which defines the step-wise logic for state transition in gym-style environment; the Reward Calculator, which allows for different reward schemes to be easily plugged in.
1399
1400
1401
1402
Feature Constructor is responsible for processing the raw state information from the environment
1403
(PN and VN) and extracting meaningful features. It supports various feature engineering strategies, such including basic node/edge attributes or augmenting them with advanced topological metrics.
26

---

Under review as a conference paper at ICLR 2026
1404
1405
Table 5: Implemented Traditional NFV-RA Algorithms
1406
Core Strategy
1407
MIP (Chowdhury al., 2009)
Mixed-Integer Programming (MIP)
1408
R- R-Rounding Chowdhury et al.
2012
Random Rounding
1409
D D-Rounding Chowdhury al.
2012)
Deterministic Rounding
1410
RW-Rank Cheng
al.
201
Random Walk (RW)
1411
GRC-Rank Gong ct al.
2014
Global Resource Control (GRC)
1412
NRM-Rank Zhang ct al.
2018
Node Resource Management (NRM)
1413
NEA-Rank (Fan et al. 202.3
Node Essentiality Assessment (NEA)
1414
PL-Rank (Fan et al.
2021
Priority of Location (RL)
1415
GA-Meta Zhang et al. 2019)
Genetic Algorithm (GA)
1416
PSO-Meta (hang Zhang
2021
Particle Swarm Optimization (PSO)
1417
ACO-Meta (Fajaaii et al. 201
Ant Colony Optimization (ACO)
1418
SA-Meta Wang et al.
2013
Simulated Annealing (SA)
1419
TS-Meta (Wang al.
2017
Tabu Search (TS)
1420
1421
1422
Neural Policy represents the core of the agent's learned strategy. Our pipeline supports wide array of Neural Networks, from simple MLPs to more complex structures like CNNs, RNNs, and various GNNs. The user also customize the Decoding Strategy that translates the probabilistic output of the neural policy into a concrete action.
1423
1424
1425
1426
RL Trainer manages the learning process, which includes the rollout buffer that stores the interaction experiences (state, action, reward, etc.) used for training; and Learning Algorithms:: that is used to update the agent's 's policy. Virne integrates comprehensive suite of solvers, such PG, A2C, PPO, and more advanced options like SafeRL. and MetaRL. solvers.
1427
1428
1429
1430
1431
C.4.2
EXTENSIBILITY IN PRACTICE
1432
1433
This modular design enables
develop algorithms flexibly, shown in Figure
1434
Configuration-driven Customization: Users can easily assemble different pipelines by modifying configuration files. As shown in the top part of Figure parameters like the reward calculator or the types of features to use can be set with simple string or boolean flags. This enables extensive experimentation without writing any new code.
1435
1436
1437
1438
For more advanced customization of flexible components, as shown in the bottom part of Figure 7 users can create custom environments or solvers (c.g., CustomEnv, CustomSolver) by inheriting from base classes. New components, such as novel policy architecture or a custom RL solver, can be injected as callable functions or classes during instantiation. This powerful design pattern significantly lowers the barrier for implementing and testing new research ideas within standardized framework.
Class-based Extension:
1439
1440
1441
1442
1443
1444
1445
C.5
EVALUATION METRIC DEFINITIONS
1446
1447
We provide the definitions of key evaluation metrics commonly used to evaluate the NFV-RA algorithms as follows.
1448
1449
1450
Request Acceptance Rate (RAC) measures the proportion of VN requests that the system successfully accepts. Formally, it is given by
1451
1452
RAC Σ7 I(t) ΣV [(t)]
1453
x 100,
(11)
1454
1455
where T denotes the total operational time of the network system. I(t) is the set all VN requests arriving at time slot t, and (()) is the subset of those requests that are accepted. The notation denotes the cardinality of set.
1456
1457
27

---

Under review as a conference paper at ICLR 2026
1458
1459
Instance Environment
Feature Constructor
Neural Policy
AL Trainer
1460
Envmonmnnt
Festuree
Netonntis
caleered
Buffer
farura
1461
Request $1 Lifetine
MLP Policy
General Yamsory
CMN, Py
Selective Memory
1462
RMK_Policy
baware Hanory
1463
Action Controler
VN Feature Constructor
OMK Pollocy
1464
PlaceRoute_Ste
Selecied
Action
Mode_ Festures
Neterr Better__aa__ae Policy
Dusl_6W(_Polscy
Algorthmm
1465
KodePair Step
Edge Festures
P6 Solver
1466
Solution Step
copological_ Features
Aetion Probooi ity Suintuttrr
A2C Solven
A3C_
1467
Feature Constructor
Deanding Strelegy
PPO Solver
1468
ZeroPenalty, Alwaadd
Sreach_Searchh Sampl Ing, Search
Tollyy
SafeRL Solven
DON Solvve
1469
FixedPenaliy: Reward
Features
NotaRLSolven
1470
Adaptive Reward
Topological Features
1471
1472
1473
Figure 6: The modular architecture of the RL-based implementation pipeline in Virne The pipeline decoupled into three core, interchangeable modules: Instance Environment, RL RL. Agent, and RL Trainer. Each module contains several sub-components (e.g., feature constructor, neural policy, learning algorithm) that can be individually customized and combined.
1474
1475
1476
1477
1478
Long-term Revenue-to-cost Ratio (LRC) evaluates the economic efficiency of the system by comparing revenue to resource consumption. It is formulated as
1479
1480
En EiEio, REV(S) x X 100,
1481
LRC
(12)
1482
EL EiEt() COST(S) X
1483
1484
where S = (T before, and REV(S) and COST(S) denote the revenue obtained and resources consumed by the embedding, respectively.
1485
1486
Long-term Average Revenue (LAR) quantifies the total revenue generated over a period, reflecting the economic benefits of processing VN requests. It is defined as
1487
1488
1489
LAR Σ
Σ REV(S) x x
(13)
1490
reZ(!)
1491
1492
where E fc(() represents the embedded VN request, and w is the lifetime of the corresponding VN.
1493
1494
Average Solving Time (AST) indicates the efficiency of the NFV-RA algorithm by measur- ing the average consumed time (in seconds) for solving one simulation run.
1495
1496
1497
C.6
EVALUATION PROTOCOL. DEFINITIONS
1498
1499
To provide a rigorous basis for comparing algorithmic performance across diverse NFV scenarios, formally define the three core evaluation perspectives follows:
1500
1501
1502
Solvability (S): This metric quantifies an algorithm's ability to satisfy the hard constraints of the NFV problem (e.g., CPU, bandwidth, and latency limits) in the offline scenario. It is formally defined the acceptance ratio:
1503
NEW
1504
1505
S Nouczess
(14)
1506
Niotal
1507
where N Nsuccess is the number of successfully embedded VNRs and Niatt is the total number of arrived requests.
1508
NEW
1509
1510
Generalization (G): We define generalization conceptually the robustness of policy ne. trained on a source topology distribution Dimin when applied directly to target distribution Dres (where Deran * Diea) without parameter updates. In experiments, this is observed
1511
28

---

Under review as a conference paper at ICLR 2026
1512
1513
Modular settings action step. env env.cls: cls: BaseEnv config. awaard_calculater_naee str
Enviroment
1514
c.9., DeinteRinstanceRLEn adaptive
1515
1516
# Wodular settings of Feature constructor config.if_use. ww..oo_osseooollooloeeoooooo wopological..ntriic... bool
1517
config.18 if. use_node node status flags: bosl
1518
customEnv(action. sttpcciionstpeeee.cllaa
1519
def -nnit__eell punet, v_net, controller, recordor, counter, logger, config, **wwargs): suspt(CustonEnv, self)._iniss in_n_(__net,
controller, recorder, counter, Togger, config, *Awarrgs)
1520
1521
1522
Modular settings Neural Policy tensor obseration_ func: callable paltccyburller:: Callable.
Tensorconvertor.ok www.weerrrrrro...ee... as_tensor___t for___n e-g., Policyaul (eerruulddebbuu grtiiddggnnnlppppl
1523
1524
Modular Setting Trainer rl_solver_cls,
1525
e.g., PGGenM]pSolver
1526
class vuttansolver
rl_solver_cls):
1527
def init (self, controller, recorder, counter, Togger, config, **wargs): Instancekgent.. -int -nniiintttlll (self, custonEnwv)
1528
rl_solver_cls___ (self, controller, policy_builder, tensor.obseratioo_fun. v*kwargs)
counter, logger, config,
1529
1530
1531
Figure 7: A code example of implementing new NFV-RA algorithm via module customization.
1532
1533
1534
by evaluating the degradation or retention of performance metrics (specifically RAC, LRC, and LAR) when the agent is transferred to unseen network topologies with varying scales and structures.
1535
1536
NEW
1537
Scalability (S]): We define scalability as the algorithm's capability to maintain efficient decision-making latency the problem size increases. Formally, we analyze the growth rate of the inference time relative to the network size V (number of nodes):
1538
1539
1540
ØT Ø
1541
S)
x
(15)
1542
1543
A scalable algorithm exhibits linear or near-linear growth rather than exponential growth, ensuring feasibility in large-scale, real-world systems with high-frequency request arrivals.
1544
1545
NEW
1546
1547
C.7 BENCHMARK DOCUMENTATION
1548
1549
To ensure usability and facilitate community adoption, we provide comprehensive and well-structured documentation website for Virne project. This documentation includes detailed guides installation, simulation configuration, API usage, and tutorials for developing new algorithms. For the anonymous requirement of the double-bling review, include static version of the documentation in PDF dwwp:mmtttddddmmd.....d
1550
1551
1552
format at
1553
1554
1555
D EXPERIMENT DETAILS
NEW
1556
1557
D.1 EXPERIMENTAL SETUP
1558
1559
We summarize the key simulation parameters in Table including the settings of PN and VNs.
1560
1561
D.1.1
IMPLEMENTATION SETTINGS
1562
We provide the details on algorithm implementation (e.g., neural networks and RL training), experi- mental methods training and testing, and computing resources.
1563
1564
Algorithm Implementation. The implementation of the RL-based NFV-RA algorithm is based on
1565
py Torchh (Paszke et al. 2019). Regarding neural policies (e.g., MLP, CNN, GNN, etc), we generally
29

---

Under review as a conference paper at ICLR 2026
1566
1567
Table 6: Key simulation parameters and their default values.
1568
Symbol
Description
Default Distribution
1569
XN,
Physical Node Resources
4(50, 100) 4(50, 100)
1570
Nc
Physical Link Bandwidth
1571
1572
Ng.
VN Request Size
U(2, 10)
Xy.
Virtual Node Resources
(10,20)
1573
Xc.
Virtual Link Bandwidth
(0)
1574
VN Arrival Rate
Poisson(1)
1575
1576
1577
1578
set their layers to and the hidden dimension of neural networks 128. For RL optimization, the reward discount factor À to 0.99 and use the Adam optimizer with learning rate of 0.001 and batch size of 128. The action section during training and testing follows a sampling strategy and greedy strategy, respectively. Additionally, we employ the k-shortest paths algorithm with k = 10
1579
1580
1581
for link routing, selecting the shortest physical path that meets the bandwidth requirements for each virtual link.
1582
1583
Training and Testing. For RL-based methods, we train policies for each average arrival rate 7,, with random initialization of seeds in every simulation. The number of training epochs is set to 50, which sufficient for all algorithms converge. Typically, training deep RL--based NFV-RA method completes within 5 hours. During testing, evaluate the performance of all algorithms by repeating each test with 10 different random seeds (i.e., O, 1111, 2222, ..., 9999) for each average arrival rate 77, thereby ensuring statistical significance.
1584
1585
1586
1587
1588
1589
Computing Resources. We conducted all experiments on a Linux server running Ubuntu 22.04.2
1590
LTS. This server is equipped with 8 x NVIDIA H20 GPU, 128 AMD EPYC 9K84 96-Core Processor, and 1.2 TB of memory.
1591
1592
1593
1594
D.1.2
EVALUATED PHYSICAL TOPOLOGIES
1595
To comprchensively evaluate the performance of NFV-RA algorithms, conducted experiments using both simulated and real-world network topologies from SNDlit widely adopted in existing studies. These topologies cover wide range of network scales and densities, ensuring that the evaluation reflects diverse operational scenarios. Below, we describe the characteristics of the evaluated PN topologies.
1596
1597
1598
1599
1600
WX100 a medium-scale network generated by the
1601
Waxman method (Waxman [988). It consists of
1602
100 nodes connected by approximately 500 links,
1603
resulting in density of 0.05.
1604
1605
WX500 is larger variant of the Waxman topology, extending the scale to simulate large-scale network systems. With 500 nodes and around 1300 links, it exhibits a higher density of 1040.
1606
1607
1608
1609
GEANT is a well-known academic research network, designed for high-speed data transfer across Europe. It consists of 40 nodes interconnected by 64 edges, density of 0.0821.
1610
Node
1611
Min
1612
Link Resouree
1613
BRAIN is high-speed data network for scientific and cultural institutions in Berlin. consists of 161 nodes and 166 edges, with a density of 0.0129, which the largest topology in SDNLib.
1614
Min
Mas
1615
Figure 8: GEANT topology visualization.
1616
1617
1618
SSNDlib is well-known open-source library for telecommunication network design that offers collection of realistic network system topologies. We use network topologies like GEANT and BRAIN from SNDlib. You access this resource hhttps///sddihh.uuuoooma though the specific licensing terms aren't clearly stated.
1619
30

---

Under review as a conference paper at ICLR 2026
1620
1621
Table 7: Key Information Evaluated Network Topologies
1622
Network
Node Count
Link Count
Network Density
1623
WX100
100
500
0.05
1624
WX500
500
13,000
0.1042
1625
GEANT
40
64
0.0821
1626
BRAIN
161
166
0.0129
1627
1628
0.9
1629
Ang. Rouur Over Obe Senulation
1630
0.8
1631
0.7
PPO-DualGAT and its variations
1632
PPO-DuaIGAT (Reeard-Fixed 0.1, Features-S-T, Mask)
1633
PPO-DuiIGAT (Roward-Fixed 0.01, FeaturceSST. Mask)
1634
PPO IPO-DuiGAAT (Reward-Adsptvve, Features-S Mask)
1635
0.5
PPO Doo-DulGAT (Reward-Fixed 0.1, Features-S, Mask)
PPO IPO-DuaIAAT (Reward-Fixed 0.1, Features-T, Mask))
1636
0.4
A3C-DuaGGAT (Roward-Fixed 0.1, Featurcs--TT Mask)
1637
PG-DualGAT (Reward-Fixed 0.1, Features-ST, Mask)
1638
0.3
16
1639
Training Simulations
1640
Figure 9: Training curves of PPO-DualGAT and its variations. Each point represents the R2C ratio all VN Requests within single training simulation round.
1641
1642
1643
We summarize the key characteristics of these topologies Table Furthermore, we provide a visualization of the topology of GEANT in Figure 8 better understand the topological information.
1644
1645
1646
D.2
EXPLORATION ON IMPLEMENTATION TECHNIQUES
1647
1648
D.2.1
IMPACT ON RL TRAINING METHODS
1649
1650 We evaluate the impact of different RL training algorithms by comparing the performance of PG, A3C, and PPO. All three methods are paired with the same PPO-DuaIGAT policy architecture. As PPO demonstrates significantly superior performance. It converges rapidly to the highest average return, stabilizing at high level of solution quality early the training process. contrast, A3C shows slower convergence and reaches a lower performance plateau. PG struggles the most, exhibiting the slowest learning rate and the lowest final performance among the three. These findings highlight PPO's s sample efficiency and robust performance, justifying its selection the default RI method for subsequent experiments.
1651
1652
illustrated by the learning curves in Figure 9
1653
1654
1655
1656
1657
1658
D.2.2
IMPACT ON REWARD FUNCTION DESIGN
1659
1660
The design and magnitude of intermediate rewards strongly affect training and final performance. Across all solvers, a moderate fixed intermediate reward (fixed, 0.1) consistently yields the best or near-best results. contrast, very small drops. highlighting the importance of sufficient reward signals for effective exploration. Interestingly. the adaptive intermediate reward, which intuitively normalizes the total reward for different VN sizes, does not achieve optimal performance in practice. As illustrated in Figure g we observe that well-tuned fixed reward of 0.1 has higher convergence than the adaptive reward, which validates our initial performance observations. The adaptive reward is also outperformed by smaller fixed reward of 0.01, which further demonstrates that it may introduce a noisy signal for the optimization process, leading to sub-optimality. This suggests that, I while normalization is conceptually appealing, fixed and moderate reward signal is more effective for guiding policy learning in the NFV-RA context.
1661
1662
intermediate rewards result clear performance
1663
1664
1665
1666
1667
1668
1669
1670
1671
D.2.3
IMPACT ON FEATURE ENGINEERING COMBINATION
1672
We compare the use Status (S), Topological (T), and their combination in the feature constructor. The results show that combining both Status and Topological features (V.v) generally leads to the
1673
31

---

Under review as a conference paper at ICLR 2026
1674
best or second-best outcomes for acceptance rate and revenue, regardless of the solver architecture. For example, PPO-ATT+ achieves the best acceptance rate (0.712) and LRC (0.661) with both features, and marked drop when using only one. As illustrated in Figure | leveraging these advanced features also contributes to faster and higher convergence. This demonstrates that e topological metrics serve a valuable augmentation, capturing global network context and node
1675
1676
1677
1678
1679
importance, even for graph neural networks such as GATs.
1680
1681
D.2.4
IMPACT ON ACTION MASK STRATEGY
1682
1683
Impact on Action Mask Strategy Applying action masking (V), which prevents the agent from se- lecting infeasible actions (e.g., resource availability satisfaction), consistently improves performance across all methods. For instance, in PPO-MLP4 and PPO-DuaIGAT+, enabling action masking increases acceptance rate by up 0.053 compared otherwise identical configurations without masking. This demonstrates that @ explicit constraint enforcement is vital for robust RL-based NFV-RA solutions, since the constraints of NFV-RA are intricate and hard.
1684
1685
1686
1687
1688
1689
1690
D.3
GENERALIZATION ON NETWORK CONDITIONS
1691
1692
Network conditions in real-world deployments are characterized by their inherent complexity and continuous evolution, including fluctuations in request frequencies and varying resource demands. Therefore, evaluating the generalization capabilities of trained NFV-RA policies is critical to ensure their adaptability and robust performance in diverse and dynamic network environments. Viree facilitates this by allowing pre-trained models to be tested under conditions different from their initial training setup. In this section, we investigate the generalization of various NFV-RA algorithms to two key aspects, i.e., varying traffic rates and fluctuating demand distributions. The algorithms are pre-trained under the default settings described in the main paper (Section 4.1 on Experimental Setup), and then evaluated under these new conditions. In the following experiments, to ensure figure clarity, we selected NFV-RA methods that demonstrated strong performance across various NFV-RA types in the main paper (Section 4.2.2 on Effectiveness in Online Environments).
1693
1694
1695
1696
1697
1698
1699
1700
1701
1702
1703
D.3.1
EVALUATION ON VARYING TRAFFIC RATES
1704
1705
To assess how well different NFV-RA algorithms adapt changes in network load, we evaluate their performance under a range of VN request arrival rates (17). The pre-trained policies, originally trained with specific 1) values for each topology (WX100 with 17 = 0.16, GEANT with rj = 0.016, BRAIN with 1) = 0.004), are tested across spectrum of values. For WX100, this range is from 0.04 to 0.28; for GEANT, from 0.004 to 0.028; and for BRAIN, from 0.001 to 0.007. This setup simulates scenarios from low to high network congestion, while ensuring the comparability of results. The
1706
1707
1708
1709
1710
results across WX100, GEANT, and BRAIN topologies are presented in Figure TO
1711
1712 As illustrated in Figure TO (a), (d), and (g), there is a general downward trend in the RAC for all algorithms as the average arrival rate 17 increases across all three topologies. This is an expected outcome, as higher traffic intensity leads to increased competition for finite physical network resources, inevitably resulting in more VN request rejections. As shown in Figure TO (b), (e), and (h), many RL-based algorithms, particularly PPO-Dua and PPO-DualGCN, demonstrate remarkable stability in LRC across varying traffic rates. The results on LAR are depicted in Figure 10 (c), (f), and (I). Generally increasing with n, most algorithms process more VN requests. Algorithms like PPO-DuaIGAT and PPO-DuaIGCN consistently achieve higher LAR across the range of values, showcasing their superior capability maximizing revenue even as conditions change. These experiments highlight that @ while some algorithums (often sophisticated RL policies like PPO- DualGAT) exhibit graceful performance degradation and maintain high efficiency, others are more sensitive to load changes. Additionally, from the perspective of network density, it is notable that ! the performance difference is more significant in sparser topologies, such as BRAIN, mainly due to limited feasible action space. The denser topologies, like WX100, often have more path diversity for routing virtual links. This richer connectivity makes the embedding problem slightly less constrained, and most algorithms, including heuristics like PL-Rank. perform relatively well. However, in sparser PNs, each placement is critical because fewer alternate routes mean a poor choice can drain a key bridge's is resources and trigger subsequent failures.
1713
1714
1715
1716
1717
1718
1719
1720
1721
1722
1723
1724
1725
1726
1727
32

---

Under review as a conference paper at ICLR 2026
1728
1729
Table 8: [Note: Same as 2ar readability]. Impact of key implementation techniques. Reward
1730 0.2; where 0 indicates no intermediate reward). Features indicate whether node Status (S) and/or
function is specified by its type (fixed or adaptive) and the intermediate reward value (0, 0.01, 0.1,
1731
Topological (T) metrics were used (V/ for used, X for not used). Action Mask indicates whether action
1732
masking was applied (V) or not (x).
1733
1734
Reward Function
Features
Action
PPO-MIP
PPO-ATT
PPO-DualGAT
1735
Type
Value
RAC
LRC
LAR
RAC
LRC
LAR
RAC
LAR
1736
Fixed
0.1
/
/
0,666
0.670
11745.0
0.667
0,644
12174.0
0,648
0.750
12430.1
1737
Fixed
0.1
/
0.678
0.675
11159.0
0.660
0.667
11134.3
0.766
0.754
15054.7
1738
Fixed
0.1
0.703
0.654
12693.8
0.579
0.575
9323.5
0.733
0.685
13333.1
1739
Adaptive
v
0,705
0.619
12139.6
0,702
0,643
12368.6
0.772
0.744
14216.6
1740
Fixed
0.2
0.616
0.643
11869.4
0.706
0.675
12420.7
0.769
0.731
15045.5
1741
Fixed
0.01
0,709
0.645
12719.1
0.517
0.601
8339.7
0.766
0.748
14516.6
Fixed
0
0,560
0.596
8259.5
0.716
0.658
12629.0
0,741
0.753
14047.6
1742
1743
Fixed
0.1
0.647
12944.4
0.712
0.661
12657.7
0.781
0.738
15138.6
1744
1745
1746
Overall, this study allows researchers and practitioners to select algorithms that are well-suited to the anticipated traffic dynamics of their target environments. For instance, for networks expecting highly variable loads, algorithms demonstrating flatter RAC and LRC curves across 17 would be preferable.
1747
1748
1749
1750
D.3.2
EVALUATION ON FLUCCTUAIIGG DEMAND DISTRIBUTION
1751
1752
In practical NFV environments, the characteristics of VN requests, such as the number of virtual nodes (VN size) or their resource requirements, can change over time due to evolving service demands. To simulate such dynamic conditions and evaluate algorithmic adaptability, we subject pre-trained policies to a sequence of VN requests where the underlying distribution of these requests' characteristics changes. Figure [1] illustrates the performance trends during a simulation these changing demands. The simulation comprises 1000 VN requests, divided into four distinct sub-groups (250 requests each). Compared to the default simulation settings, we modified one parameter related to the distribution of VN resource demand or VN node size for each subgroup:
1753
1754
1755
with
1756
1757
1758
1759
For the first group. the VN node and link resouree distributions were changed [0. 30] and [O. 75]. respectively. For the second group, the VN node and link resource distributions were adjusted to [0, 40) and [O, 100], respectively. For the third group, the VN node size distribution was changed to [2,
1760
1761
1762
15]. For the fourth group, the VN node size distribution was altered to [2, 20].
1763
As illustrated in Figure II (a), (d), and (g) for RAC, most algorithms exhibit an initial sharp decrease, particularly as the simulation transitions through phases with increasing VN resource demands. This phase quickly consumes available physical resources, leading to lower acceptance rates. As the simulation progresses into phases where VN sizes increase, the RAC for many algorithms continues to decline or stabilizes at a lower level. This is because larger and more complex VNs are inherently more challenging embed. The results on R2C shown in Figure IT (c), and (h) show dip when the demand characteristics change, but PPO-DualGCN keeps a relatively higher LRC. For 1X100 records LRC of 0.703, whereas PPO-MLP has an LRC of 0.643. This suggests they are better at finding resource-efficiem embeddings even when faced with unfamiliar or more challenging VN request types. The total revenue (i.e., cumulative LAR), depicted in Figure I1 (c), (f), and (i), shows clear differentiation in the algorithms' ability to accumulate revenue under these dynamic conditions. Algorithms that adapt well and maintain higher RAC and LRC will naturally accrue more total revenue. Notably, ad advanced RL methods like PPO-
1764
1765
1766
1767
1768
1769
1770
(n) 0.04) under changeable demands,
1771
1772
1773
1774
1775
DualGAT and PPO-DuaIGCN tend to maintain a higher performance compared to others throughout these fluctuations, indicating better adaptation. From the perspective of stage evolution, in stages I and 2 with increased resource demands, P the superior performance of dual-GNN models here suggests that their learned policies latently identify action space with sufficient and rich resources within the PN to leave sufficient resources for future. Regarding stages and related to increased topological complexity, this mainly estimates the ability solve more complex graph mapping problem. The great performance of GNN-based methods further demonstrates the importance of the representation power of neural network architectures.
1776
1777
1778
1779
1780
1781
33

---

Under review as a conference paper at ICLR 2026
1782
1783
1784
1785
1786
1787
1788
1789
1790
1791
1792
FL-Rank
GA-Meta
MCTS PPO-MLP
PPO-ATT
PPO-DuaKIAT
1793
NEA-Rank
PSO Meta
PPO-HEenTUAAT
PPO- UPO-DuaIGCN
1794
1795
1796
0.9
1797
L1 KEV CDVVO100 (410%)
1798
0.8
L.T R32
1799
0.5
1800
1801
0.5
1802
1803
0.5
0.04
0.12
0.20
0.4
0.04
0.12
0.20
0.28
0.5
0.04
0.12
0.20
1804
Average Amival Rate
Ariiaa Rate
Average Arrical Rate
RAC WX100
LRC WX000
(c) LAR WX100
1805
0.70
0.8
12.0
1806
0.65
1807
0.7
REV (X105)
1808
11 ROC
1809
0.55
0.5
1810
0.50
4.0
1811
1812
0.45
1813
0.40
1814
9.01
DU02
0.03
Average Arrival Rate fl
Average Arrival Rate
Average Arrival Rate f
1815
(d) RAC on GEANT
(+) LRC on GEANT
(i LAR on GEANT
1816
0.8
1817
1818
4.0
1819
VN ACR
K2C
1820
0.5
ASV IT
1821
1822
0.5
1823
0.
1824
0.4
0.0
0.00
0.00
0.01
0.01
0.00
0.00
3.01
0.01
0.00
0.00
0.01
0.01
1825
Antiaal
Average Anival
1826
(g)
(h)
LAR BRAIN
1827
Figure 10: Results the generalization study varying traffic rates.
1828
1829
1830
1831
1832
1833
1834
1835
34

---

Under review as a conference paper at ICLR 2026
1836
PL-Rank NEA-Rank
GA-Meta
MCTS PPO-MLP
PPO-ATT PPO-HecerGGAT
PPO-DuaIGAT
1837
PSO-Meta
PPO-DuaIGCN
1838
1dk
1839
1.2
1840
1841
0,9
0.9
1.0
1842
0.8
0.8
1843
0.7
0.6
1844
0.6
0.4
1845
1846
0.5
0.0
1847
5000
20000
25000
1848
Simulation Time
Simobstin Time
Sinulation Time
1849
(4) RAC on WX100
(b) LRC WX100
le7
(c) Total REV on WX WX100 100
1850
0.9
1851
1852
0.8
0.5
1853
0.7
1854
0.5
1855
1856
1857
1858
S00n0 100000
500nn 200000 250000
500A0
100000
1Sh00
200000 250000
56000
100000
50000 200000 250000
Simulation Time
Simulation
Sinulstin
1859
107
GEANT
1860
1.0
1861
3,0
1862
2.5
1863
0.6
0.6-
Tool MEV
2,0
1864
1.5
0.4
1865
1866
0.5
1867
6.0
0.9
0.0
1868
1869
Simulation Time
le6
Stmolation Time (b) LRC on BRAIN
Simulatin Time
1870
(8) RAC on DRAIN
(1) Total REV on BRAIN
1871
Figure 11: Results on the generalization study on fluctuating demand distribution.
1872
Overall, these results highlight that ④ policies trained on a specific VN distribution may not generalize well others, underscoring the need to generalize not only across load levels but also across demand types. For practical systems where service characteristics can evolve, choosing algorithms that demonstrate robustness such fluctuating scenarios, as identifiable through Virne, is important.
1873
1874
1875
1876
1877
D.4
SCALABILITY ON NETWORK SIZES
1878
1879
The ability of an NFV-RA algorithm to scale effectively with increasing network size and complexity paramount for its practical deployment in real-world, large-scale infrastructures. Virne facilitates a thorough assessment of scalability from primary perspectives, i.e., (1) performance quality
1880
1881
large-scale network topologies, and (2) the growth trend of average solving time as both PN and VN sizes increase.
1882
1883
1884
D.4.1
PERFORMANCE ON LARGE-SCALL NETWORKS
1885
1886
To evaluate how algorithms perform when the underlying physical infrastructure is extensive, utilize the WX500 topology within Vime. WX500, with 500 nodes and approximately 13,000 links, represents significantly larger and denser environment compared to WX100 (100 nodes, 500 links) or other real-world topologies like GEANT (40 nodes) and BRAIN (161 nodes) used in earlier evaluations. The VN arrival rate (n) for WX500 is appropriately adjusted reflect its increased
1887
1888
1889
35

---

Under review as a conference paper at ICLR 2026
1890
1891
Table 9: Results the large-scale network, WX500.
1892
RACT
LRC1
LART
AST
1893
PPO-MLP
69.50
0.646
924217.96
4.98
1894
PPO-CNN
69.70
0.654
826532.68
5.16
1895
PPO-ATT
68.10
0.647
923953.53
5.64
1896
PPO-GCN
59.50
0.610
660552.71
3.80
1897
PPO-GAT
68.80
0.698
920725.67
5.04
1898
NO-GGC&&S2S
61.70
0.541
682516.68
4.89
1899
PPO-GAT&SSS
58.10
0.540
699216.61
4.74
1900
PPO-DuaIGGN
69.90
0.682
909759.61
5.10
1901
PPO-DualGAT
69.80
0.715
917515.34
5.08
PPO-HeteroGAT
61.20
0.546
774076.11
7.42
1902
MCTS
62.19
0.543
77138.32
18.13
1903
1904
SA-Meta
61.40
0.591
691030.70
15.09
1905
GA-Meta
64.10
0.568
72314.48
16.49
1906
PSO-Meta
68.90
0.566
750871.95
6.31
TS-Meta
62.10
0.614
678597.92
14.34
1907
1908
NRM-Rank
55.10
0.553
597325.51
0.22
1909
RW-Rank
55.50
0.561
592502.79
0.24
1910
GRC-Rank
58.50
0.559
656379.96
0.22
1911
PL--Rank
64.17
0.632
79374.48
18.83
1912
NEA-Rank
66.70
0.641
837297.93
12.97
RW-BFS
5.30
0.592
38341.41
0.27
1913
1914
1915
capacity, ensuring a comparable level of resource contention. The performance of various algorithms on WX500 detailed Table
1916
1917
1918 We observe that PPO-DuaIGAT and PPO-DualGCN continue to exhibit strong performance on the large-scale WX500 network. This demonstrates the good scalability of their learned policies to more complex physical infrastructures. While other RL methods with simpler extractors, like PPO-MLP, exhibit limited performance, this suggests that ① the capability to effectively capture and process complex topological information becomes increasingly critical in larger networks. Additionally, while MCTS and meta-heuristics (GA-Meta, PSO-Meta, SA-Meta, TS-Meta) can sometimes achieve competitive solution quality, their AST becomes significant bottleneck larger networks. Furthermore, it interesting that S there is also a trade-off between maximizing immediate revenue and ensuring long-term network efficiency. Simpler models like PPO-MLP exemplify the first approach, achieving the highest total revenue (LAR) by aiming to maximize acceptance volume. In contrast, sophisticated models like POO-DuaGATT exemplify the second, securing the highest resource efficiency (I (LRC) with strategic policy that prioritizes high-quality placements to preserve resource utilization efficiency.
1919
1920
1921
1922
1923
1924
1925
1926
1927
1928
1929
1930
1931
D.4.2
ANALYSIS OF SOLVING TIME SCALE
1932
Beyond solution quality, the computational efficiency, measured by the Average Solving Time (AST), critical factor for scalability. Virne allows for detailed profiling of how AST evolves with increasing problem scales. We investigate this from two angles:
1933
1934
1935
1936
The impact of increasing VN size (number of virtual nodes from 5 to 30) on AST, typically keeping the PN fixed (i.e., WX100).
1937
1938
The impact increasing PN size (number of physical nodes from 200 1000) on AST, for a consistent distribution of VN requests.
1939
1940
1941
Figure 12 illustrates these relationships for various algorithms. We observe that rRL-based methods generally maintain much flatter and lower AST curve, while meta- hecaahherissics exhibit a substantial increase. Their inference time is primarily dictated by the forward pass through the trained neural network, which scales less dramatically with VN size compared to exhaustive search numerous
1942
1943
36

---

Under review as a conference paper at ICLR 2026
1944
1945
PL-Raakk NEA-Rank
GA-Meta PSO-Meta
MCTS PPO-MLP
PPO-ATT
PPO-DuiKAT
PPO-HeteoGAT
PPO-DuaNGCN
1946
1947
30
1948
25
1949
80
1950
Average Solving Time
Average Solving Time
1951
60
1952
15
1953
40
10
1954
1955
5-
20
1956
1957
0-
1958
5
10
15
20
25
30
200
800
1000
1959
Size of Virtual Network
Size of Physical Network
1960
Figure 12: Results Inference Time Scale with both VN and PN.
1961
1962
iterations. For other heuristics, they are typically the fastest, exhibiting very low and stable ASTs irrespective of VN size. While MCTS and certain meta-heuristic might achieve competitive solution quality in some scenarios, their substantial computational overhead, especially with increasing problem dimensions, limits their applicability in dynamic, large-scale NFV environments. This further highlights that @ as the problem space grows exponentially, explicit search strategies at decision time become computationally prohibitive, rendering them infeasible for dynamic environments that require near-insee haar-intantanneus decisions. Additionally, ① RL-based methods, particularly those utilizing GNNs, strike a more favorable balance. Simpler RL methods and traditional heuristics offer the highest computational speed but may compromise optimality.
1963
1964
1965
1966
1967
1968
1969
1970
Overall, through systematic AST profiling analysis, Virne starkly highlights the trade-off between solution quality and computational scalability.
1971
1972
1973
D.5
VALIDATION ON EMERGING NETWORK
1974
1975
NFV is a cornerstone technology for variety of modern and emerging network paradigms, each presenting unique characteristics and operational requirements. The Virne benchmark is designed to validate the adaptability and effectiveness of NFV-RA algorithms in such specialized environments. This section focuses two prominent emerging scenarios, i.e., (1) networks with heterogeneous computing resources, and (2) latency-aware edge networks where delay constraints are critical.
1976
1977
1978
1979
1980
Performance in these scenarios is detailed in Table I0
1981
1982
D.5.1
HETEROGENEOUS RESOURCING NETWORKS
1983
Modern data centers and network infrastructures often deploy servers with diverse computing ca- pabilities (e.g., varying CPU cores, GPU availability, memory capacities). Handling such resource heterogeneity is crucial for efficient VNF placement. Vime simulates this by configuring both PN nodes and VN nodes with multiple, distinct types of computing resources. For this evaluation, the WX100 topology is used with a VN arrival rate π 0.16. An embedding is successful only if physical node can satisfy all specified resource demands (i.e.. CPU, GPU, and memory) of virtual node simultaneously, adding layer of complexity the node mapping process.
1984
1985
1986
1987
1988
1989
1990 As illustrated in Table I0 we observe that PPO-DuaIGAT and PPO-GAT&S2S demonstrate superior performance in heterogeneous resourcing environments, outperforming PPO-DuaIGCN. hetero- geneous resource scenarios, GAT-based models (e.g., PPO-DuaIGAT and PPO-GAT&S2S) excel because they can dynamically weigh the importance of different resource types, such as CPU versus scarcer GPU, which is more nuanced approach than the uniform feature treatment in GCNs.
1991
1992
1993
1994
1995
This highlights that ① attention mechanisms over features and neighbors are highly effective for multi- dintiddiinnnnnlll node-level constraints. Simpler RL architectures like PPO-MLP and PPO-GCN, while competent, tend to show a slight dip in performance compared to scenarios with homogeneous resources. This reveals that Effectively managing and balancing multiple, diverse resource require-
1996
1997
37

---

Under review as a conference paper at ICLR 2026
1998
ments simultaneously poses greater challenge for architectures that may not explicitly differentiate
1999
or dynamically weigh these varied resource dimensions.
2000
2001
D.5.2
LATENCY-AWARE EDGE NETWORKS
2002
2003
Edge computing networks bring computational resources closer to end-users, making them ideal for latency- latency-sensitive applications. In such scenarios, NFV-RA algorithms must not only allocate resources
2004
2005
efficiently but also strictly adhere to the delay requirements of services. This evaluation uses the WX 100 topology with a VN arrival rate 1) = 0.08, which could represent typical edge deployment with specific traffic characteristics, The critical factor introduced here is latency constraint for each virtual link. When a virtual link is mapped to a physical path, the cumulative propagation delay
2006
2007
2008
of the physical links constituting that path must not exceed the virtual link's specified maximum tolerable latency, i.e., 100ms. This necessitates that the algorithms should possess the ability to
2009
2010
manage constraint management. We additionally consider the latency attributes as the input features of neural networks for all deep RL--based methods.
2011
2012
2013
As illustrated in Table 10 the introduction of latency constraints generally leads to lower RAC and LRC values across all algorithms compared to scenarios without such strict path requirements. However, PPO-DualGAT again shows the most robust performance, achieving the highest RAC (68.20%). LRC (0.714) 14), and LAR (12056.58). In contrast, traditional heuristics and MCTS struggle to satisfy resource and latency simultaneously, suffering larger drops. This reveals that @ The fundamental weakness of traditional heuristics is exposed in scenarios with path-level constraints.
2014
2015
2016
2017
2018
Their decoupled "rank nodes, then find paths" approach is brittle, as an optimal node choice based on local metrics can make it impossible to subsequently find valid low-latency path, leading to cascading failures.
2019
2020
2021
2022
D.6
EVALUATION ON WORLD SCENARIO DATASETS
2023
2024
To further validate Virne's adaptability to the complex, non-uniform dynamics inherent in real- world NFV systems, we conducted an additional experiment using realistic datasets and trace-driven simulations. This experiment moves beyond standard synthetic distributions (e.g., Uniform) to demonstrate the framework's ' capability to handle heavy-tailed resource distributions and realistic traffic patterns. We adopted the GEANT topology for the physical network. Unlike the uniform distribution used in default settings, the computing and bandwidth resources in this experiment follow a heavy-tailed Pareto (Power-Law) distribution with a shape parameter a = 1.2 and scale parameter Fm = 50. This models a realistic environment where resource capacity is highly skewed, ie., approximately 20% of the nodes possess 80% of the total system capacity. For the Virtual Networks (VNs), rather than using random graph generation, we utilized traffic-driven topologies
2025
2026
2027
2028
2029
2030
2031
2032
2033
generated via CloudSin P based on actual GEANT traffic patterns. Furthermore, service demands were derived from real-world traces: node CPU requests were sampled from Alibaba Cloud Traces 9 and link bandwidth requirements were derived from GEANT traffic matrices. All demand values were normalized to the range [0, 50).
2034
2035
2036
2037
2038
The performance results under these real-world conditions are summarized in Table IT We observe that the performance gap between RL-based methods and traditional heuristics is more pronounced in this realistic scenario. Advanced RL methods, such as PPO-DuaIGCN and PPO-DualGAT, achieve significantly higher RAC of 67.90% and 66.50% respectively. Additionally, traditional heuristics (e.g., Rank, RW-Rank), which often rely on average-based metrics or simple greedy strategies, struggle to exploit the heavy-tailed Pareto resource distribution effectively, resulting in lower acceptance rates. In contrast, RL policies demonstrate a superior ability to identify and utilize the high-capacity nodes ("heavy tails") within the network. Among RL approaches, graph-based architectures (DualGCN, DualGAT) continue to outperform simpler architectures (MLP, CNN) and meta-heuristics, confirming that capturing topological structure is crucial even when processing realistic, trace-driven workloads.
2039
2040
2041
2042
2043
2044
2045
2046
2047
NEW NEW
2048
2049
2050
2051
38

---

Under review as a conference paper at ICLR 2026
2052
2053
Table 10: Results emerging networks with heterogeneous resources and latency-awareness.
2054
Heterogenous Resourcing WX10 (1) = 0.16)
Latency-aware Edge WX100 (1) = 0.08)
2055
RAC1
LRC²
LAR
AST
RAC1
LRC1
LAR
AST
2056
PPO-MLP
71.10
0.665
12694.51
0.14
56.70
0.609
9792.61
0.20
2057
PPO-ATT
69.30
0.655
12477.79
0.16
57.70
0.605
9460.70
0.21
2058
PPO-GCN
56,30
0.623
10019.26
54.50
0.653
9045.22
0.20
PPO-GAT
73.40
0.683
13214.29
0.16
53.30
0.635
9570.29
0.26
2059
PFO-GCN&S2S
54.20
0.619
9770.58
52.10
0.652
9144.30
0.22
2060
PPO-GAT&S25
75.00
0.688
13857.09
63.30
0.663
10703.74
0.22
2061
PPO-DuaIGCN
66.10
0.715
12823.80
0.690
11301.84
0.33
2062
PPO-DualGAT
75.10
0.735
0.714
12056.58
0.32
PPO-HeteroGiA
71.20
0.663
13225.30
64.70
0.673
11563.99
0.40
2063
MCTS
73.60
0.448
12847.17
1.72
52.70
0.480
8920.53
2.42
2064
SA-Meta
62.10
0.617
10397.84
0.72
44.80
0.557
7789.51
0.80
2065
GA-Meta
72.30
0.589
12245.59
1.16
56.10
0.542
9193.42
1.60
2066
PSO-Meta
67.80
0.519
10837.89
1.37
52.50
0.488
8371.48
1.46
2067
TS-Meta
65.50
0.650
11574.57
0.68
45.90
0.579
7970.68
0.77
2068
NRM-Rank
60.30
0.525
9713.59
0.03
46.30
0.528
7385.33
0.05
2069
RW-Rank
60.20
0.543
8947.27
0.02
39.40
0.554
7042.06
0.05
GRC-Ran
57.50
0,550
9008.80
0.01
42.50
0.556
7313.99
0.04
2070
PL-Rank
69.70
0.659
12485.17
0.14
51.60
0.617
8546.75
0.23
2071
NEA-Rank
64.60
0.671
10304.28
0.23
45.80
0.624
7367.66
0.34
2072
RW-BFS
33.70
0.572
6472.31
0.02
36.70
0.594
5944.21
0.05
2073
2074
E DISCUSSION ON FUTURE DIRECTIONS
2075
2076
The empirical analysis enabled by the Virne benchmark illuminates several promising avenues for advancing deep RL-based NFV-RA methods. Although state-of-the-art algorithms show substantial capability, significant challenges remain that hinder more robust, efficient, and practical solutions. In this section, we outline several key future directions, and provide empirical study of an emerging algorithm addressing some of these directions Figure
2077
2078
2079
2080
2081
2082
2083
Table 11: Performance comparison on Real-World Scenario Datasets.
2084
Solver
RAC1 t
LRC +
LAR 1
AST
2085
PPO-MLP
58.40
0.62
802.89
0.10
2086
PPO-CNN
58.90
0.65
805.19
0.11
2087
PPO-ATT
60.20
0.64
835.59
0.11
2088
PPO-GCN
63.50
0.71
915.82
0.13
2089
PPO-GAT
64.10
0.70
928.44
0.14
2090
PPO-GCN&S2S
62.80
0.69
894.10
0.12
2091
PPO-GAT&S2S
63.40
0.68
908.25
0.15
2092
PPO-DuaIGCN
67.90
0.74
987.30
0.20
2093
PPO-DuaIGAT
66.50
0.73
982.15
0.18
2094
PPO-HeteroGAT
61.80
0.67
875.60
0.35
MCTS
56.40
0.51
650.20
3.12
2095
2096
SA-Meta
45.20
0.61
512.85
1.15
2097
GA-Meta
54.70
0.59
680.40
2.85
2098
PSO-Meta
51.10
0.53
615.90
4.02
2099
TS-Meta
47.50
0.67
550.12
1.20
2100
NRM-Rank
44.60
0.53
498.20
0.06
2101
RW-Rank
43.20
0.54
485.75
0.02
2102
GRC-Rank
41.80
0.49
460.30
0.02
2103
PL-Rank
59.10
0.68
808.50
0.08
2104
NEA-Rank
55.30
0.64
710.15
0.15
2105
RW-BFS
35.40
0.58
380.60
0.03
39

---

Under review as a conference paper at ICLR 2026
2106
PPO-DulGATT (Basic RL)
PPO-DuaIGAT (Meta-RL)
PPO-DuaGAT (Safa RL)
2107
85
0.80
2108
2.0
2109
0.78
1.5
2110
0.76
2111
0.74
LAR
2112
2113
60
0.72
2114
15
WXX100
GEANT
0.70
WX100
0.0
2115
Network Topology
Network Topology
Network Topology
(a) RAC
(b) LRC t
(c) LAR 1
2116
2117
Figure 13: Performance comparison of AAT and its advanced variations that are augmented by the safe RL method proposed in Wang et al. (2025) and the meta-RL proposed in Wang al. (2024b) training methods. Please see their MDP formulation in Appendix C.2 The results show that both the safe RL and meta-RL. variants consistently outperform the basic algorithm, with the safe RI. approach achieving higher improvements. This underscores the importance of robust constraint management and generalization in solving complex resource allocation problems, which are highlighted in this section for future algorithmic innovation.
2118
2119
2120
2121
2122
2123
2124
E. REPRESENTATION LEARNING FOR CROSS-GRAPH STATUS
2125
2126
Future research should focus on developing more sophisticated representation learning techniques to capture the intricate, dynamic interplay between VN requirements and PN states, including the evolving mapping status itself. While Virne demonstrates the strength of dual-GNN architectures in processing VN and PN features separately, there is significant potential in exploring emerging architectures such as Transformers and diffusion models. These advanced models could enable the learning of even richer cross-graph relational embeddings and path-level attribute awareness (Wang) ct al. 2019; Guo et al. 2025). Specifically, they may offer superior capabilities in capturing long- range dependencies across multiple resource types and handling complex constraints, potentially leading to more nuanced and context-aware embedding decisions. Techniques that explicitly model the partially embedded state of VNs and the resultant constraints on future placements are crucial, and Vime's s extensible architecture provides ready platform for benchmarking these novel designs.
2127
2128
2129
2130
2131
2132
2133
2134
2135
2136
NEW
2137
2138
2139
E.2
ROBUST LEARNING FOR COMPLEX AND HARD CONSTRAINTS
2140
2141
The management of multifaceted and often conflicting constraints (e.g., resource capacity, latency, reliability, energy consumption) remains a central challenge in NFV-RA, which requires guaran- teeing zero-violation. Thus, it significant to explore constraint-awar RL frameworks that more explicitly model and navigate these hard constraints. This could involve advancing constrained policy optimization methods and developing novel reward structures that better reflect constraint satisfaction (Gu et al.] 2022 Wang et al. 2025). The goal is train safe policies that are not only effective in optimizing primary objectives but are also inherently robust in satisfying diverse operational constraints.
2142
2143
2144
2145
2146
2147
2148
2149
2150
E.3 SCALABILITY IN EXTREMELY LARGE-SCALE NETWORKS
2151
2152
Virne's scalability studies demonstrate that while current RL methods, particularly GNN-based ones, scale better than many traditional approaches, their computational and memory demands can still grow significantly with network size. Addressing NFV-RA in truly massive, carrier-grade networks requires further breakthroughs in algorithmic scalability. Future directions could include exploring hierarchical RL where policies operate at different levels of abstraction, building non-autoregrressiv solution modeling method (Bengio et al. 2021), ) or designing GNN architectures that are less sensitive
2153
2154
2155
2156
2157
2158 to the global network scale (Wang et al. (024a). Methods that can learn transferable knowledge
2159
localizable policies that effectively stitch together solutions for very large infrastructures are highly
desirable (Geng et al. 2023
40

---

Under review as a conference paper at ICLR 2026
2160
E.4
GENERALIZABLE POLICY CROSS VARYING SCENARIOS
2161
2162
Achieving policies that generalize effectively not only to unseen network topologies but also across varying scales (both PN and VN sizes) and highly dynamic operational conditions (e.g., non-stationary demand patterns, unexpected resource outages) is a critical frontier. To facilitate research in this direction, Virne explicitly incorporates Multi-Task MDP framework (see Appendix C.2.1), allowing users to define diverse tasks encompassing different physical topologies, resource distributions, and traffic patterns. As demonstrated by the meta-learning results in Figure 13, this foundation enables the training of agents capable of adapting to new environments. Future research should leverage this flexible task definition to further investigate advanced techniques such as meta-RL (Beck et al.) 2023), curriculum learning to incrementally expose agents to more complex scenarios, and domain randomization to enhance robustness against a wider array of unseen conditions (Zhou et al. 2023 Wang et al., (2024b). The ultimate aim is to develop omni-genete onni-generalizable NFV-RA solutions that require miniaal retraining for deployment in diverse and evolving network environments.
2163
2164
2165
2166
2167
NEW
2168
2169
2170
2171
2172
2173
2174
F LIMITATIONS ON SIM-TO-REAL GAP
2175
2176
As a simulation-bast based benchmark, Virne makes specific abstractions to ensure the computational efficiency required for training Deep RL agents. There are several primary gaps between default simulation and physical deployment:
2177
2178
NEW
2179
2180
Simulation Granularity (Flow vs. Packet). Virne functions as a flow-level orchestration simulator. To maintain high training throughput, it abstracts packet-level dynamics (e.g.. microsecond queuing delays, jitter) into statistical link attributes. While this simplifies latency modeling, it allows for rapid algorithmic iteration.
2181
2182
2183
NEW
2184
Infrastructure Observability (Global vs. Partial). The default experimental settings assume a centralized SDN controller with global state visibility. In contrast, real-world large-scale networks may only offer partial observability. Virne mitigates this via modular architecture, allowing users to apply feature masking to simulate POMDP scenarios without altering the core engine.
2185
2186
2187
2188
NEW
2189
Resource Dynamies (Tdealized Noisy). The default environment assumes deterministic resource allocation. Physical systems, however, often experience fluctuations due to back- ground processes or hardware overhead ("noisy neighbors"). Virne supports extensibility in its transition function, allowing users to inject stochastic noise to model these environmental perturbations.
2190
2191
2192
2193
NEW
2194
2195
G THE USE OF LARGE LANGUAGE MODELS STATEMENT
2196
2197
The authors use Large Language Models (LLMs) as an assistive tool in the preparation of this manuscript, in accordance with the ICLR 2026 policy. The core roles of LLMs are described as follows. First, we utilize the I.L.-powered search and summarization tools to survey related works and understand the current state of the field. Second, we use LLMM-based coding assistants to assist in coding and suggest optimizations, thereby accelerating the development of the Virne codebase. Third, while preparing the manuscript, we use LLMs to proofread, check grammar, and refine the language in the manuscript for improved clarity and readability.
2198
2199
2200
2201
2202
2203
2204
2205
2206
2207
2208
2209
2210
2211
2212
2213
41


Reviews:
[{"review_id": "D9wpcZgQ73", "reviewer": "Reviewer_1Brs", "summary": "This paper introduces Virne, a benchmark designed to accelerate research and application of deep RL for network resource allocation in NFV. The authors address a critical gap in the field by providing a unified, modular, and extensible framework that supports diverse network scenarios (e.g., cloud, edge, and 5G), integrates over 30 algorithms (including traditional and RL-based methods), and evaluates them with rigorous metrics such as solvability, generalizability, and scalability. Through experiments, the authors provide insights for future research directions.", "questions": "a) The authors use a fixed 50% random probability for interconnections between virtual nodes. A fixed interconnection probability may not accurately reflect real-world VN topologies, which often exhibit more structured (sequential or hierarchical) connectivity patterns. To better align with real-world NFV scenarios, the authors should consider adopting more dynamic and realistic VN topology generation methods (e.g., using real-world datasets).\n\nb) The computing and bandwidth resource configurations in physical and virtual networks are modeled using uniform distributions, which may oversimplify real-world resource dynamics. Actual NFV systems often have more complex, non-uniform resource distributions influenced by hardware constraints and workload patterns.\n\nc) The abstract lacks clarity and precision. The abstract includes terms and phrases that are not clearly defined, which may confuse readers. For example, the abstract mentions \"this complexity\" without explicitly explaining what aspect of complexity it refers to. Similarly, \"the field\" does not specify whether it refers to NFV or RL research. Also, key terms such as \"NFV-RA\" are not clearly defined in the abstract.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "a) The Virne framework supports a wide variety of network scenarios and resource types, including heterogeneous resources and latency-sensitive environments.\n\nb) The authors conduct experiments across diverse topologies (e.g., Waxman, GEANT, BRAIN) and network scales, providing valuable insights into the strengths and weaknesses of different algorithms.", "weaknesses": "a) The authors use a fixed 50% random probability for interconnections between virtual nodes. A fixed interconnection probability may not accurately reflect real-world VN topologies, which often exhibit more structured (sequential or hierarchical) connectivity patterns. To better align with real-world NFV scenarios, the authors should consider adopting more dynamic and realistic VN topology generation methods (e.g., using real-world datasets).\n\nb) The computing and bandwidth resource configurations in physical and virtual networks are modeled using uniform distributions, which may oversimplify real-world resource dynamics. Actual NFV systems often have more complex, non-uniform resource distributions influenced by hardware constraints and workload patterns.\n\nc) The abstract lacks clarity and precision. The abstract includes terms and phrases that are not clearly defined, which may confuse readers. For example, the abstract mentions \"this complexity\" without explicitly explaining what aspect of complexity it refers to. Similarly, \"the field\" does not specify whether it refers to NFV or RL research. Also, key terms such as \"NFV-RA\" are not clearly defined in the abstract.\n\nd) Given the rapid development of NFV, some relevant literatures are necessary to be compared in the context of related work, e.g., NFVdeep: Adaptive Online Service Function Chain Deployment with Deep Reinforcement Learning, iwqos’19; Adaptive VNF Scaling and Flow Routing with Proactive Demand Prediction, infocom’18, etc.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6Sm8mzLRLO", "reviewer": "Reviewer_JndN", "summary": "This paper presents a comprehensive benchmarking framework for resource allocation (RA) in Network Function Virtualization (NFV). It enables a modular, standardized, and customizable pipeline for the implementation of reinforcement learning (RL)-based NFV-RA methods, and offers an in-depth analysis of the design principles behind different RL components. Extensive experiments are conducted to investigate the performance, solvability, scalability, and generalization of these methods.", "questions": "see weaknesses", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The paper decomposes the implementation of RL into three parts, the RL algorithm, the neural network architecture, and the implementation techniques, and provides multiple design options for each component to investigate their underlying design principles, offering valuable insights into the application of RL in NFV-RA.\n\nThe benchmarking framework proposed in the paper provides a comprehensive platform that includes diverse NFV application scenarios, implementations of over 30 RA algorithms, and supports the evaluation of a wide range of research metrics.", "weaknesses": "Considering that this is a unified benchmarking framework, assuming a one-to-one mapping from virtual networks to physical networks in the system model seems unreasonable. In general, multiple virtual nodes should be allowed to map to the same physical node to enable flexible deployment.\n\nThere is a lack of research on emerging network architectures, such as Transformers and diffusion models.\n\nThe paper repeatedly selects virtual nodes from the virtual network and maps them onto the physical network using a RA policy, then applies a shortest-path algorithm to determine the deployment of virtual links and identify feasible embedding solutions. However, the paper lacks an explanation of the selection order of virtual nodes.\n\nThe mathematical formulation of the system model in Appendix A has some issues, for example, \n\tn_p  may be N_p in Constraint 1;\n\tIf the directionality of path mapping is not considered, the left-hand side of Constraint 5 will always be zero.\nPlease check it carefully.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "S0Y42fQSy3", "reviewer": "Reviewer_4Vws", "summary": "This paper presents VIRNE, a comprehensive benchmarking framework for reinforcement learning-based network function virtualization resource allocation (NFV-RA). The framework unifies diverse simulation environments (cloud, edge, 5G), integrates over 30 algorithms (including RL, heuristic, and exact solvers), and introduces new evaluation perspectives such as solvability, generalization, and scalability. Extensive experiments demonstrate that advanced RL agents (especially dual GNN-based PPO variants) outperform traditional baselines. The benchmark is open-sourced to promote reproducibility and standardization in the NFV-RA community. Overall, the work provides strong engineering value and community impact but offers limited methodological novelty.", "questions": "How are the “solvability,” “generalization,” and “scalability” metrics formally defined and measured? Are they normalized or directly comparable across different network topologies?\n\nAre the results averaged over multiple runs or random seeds? If not, could the authors report standard deviations or confidence intervals to support claims of superiority?\n\nCan the proposed benchmark be easily extended to other RL-based network optimization problems (e.g., SDN routing, SFC placement)?\n\nWhat are the main limitations of VIRNE when applied to real-world NFV systems — for example, partial observability, dynamic resource noise, or latency constraints?\n\nBeyond engineering value, is there a potential direction for new algorithmic or theoretical insights enabled by this benchmark (e.g., constrained or meta-RL for NFV-RA)?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "+ VIRNE represents the most systematic and extensive benchmark for NFV-RA to date. It consolidates numerous algorithms, diverse simulation environments, and multiple evaluation perspectives into a unified, open, and reproducible framework. This kind of infrastructure contribution fills a long-standing need in the NFV and RL communities.\n+ The modular design, well-documented implementation, and thorough experimental setup reflect a high level of technical maturity. The open-source release with detailed appendices ensures reproducibility, aligning with ICLR’s best practices.\n+ The experiments cover diverse network scenarios, real-world topologies, and multiple metrics beyond traditional performance, offering a valuable empirical reference for future work.\n+ The paper is well-structured and logically consistent. Figures and tables are well-presented, helping readers understand the system architecture and experimental findings.", "weaknesses": "- The paper’s contribution is primarily infrastructural rather than algorithmic. The RL formulations (e.g., MDP setup, PPO training, GNN encoders) follow standard designs without introducing new theoretical insights or model innovations.\n- While experiments are comprehensive, most results are reported as single averages without statistical significance tests or error margins. This weakens the strength of empirical claims.\n- Several sections (notably Sections 3–4) emphasize implementation details at the expense of high-level conceptual insights. The paper reads more like a system report than a research contribution.\n- Terms like solvability, generalization, and scalability are described qualitatively but lack formal or quantitative definitions, limiting their interpretability and comparability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Wx4Kvu7Ca5", "reviewer": "Reviewer_eJHY", "summary": "The paper introduces Virne, a comprehensive benchmarking framework designed to evaluate RL-based Network Function Virtualization (NFV) Resource Allocation (RA) algorithms. Virne aims to bridge the gap in current NFV-RA evaluations, which lack comprehensive, standardized benchmarking methods. It provides highly customizable simulations for diverse network environments (e.g., cloud, edge, 5G), incorporating over 30 algorithms, including both traditional and RL-based approaches.", "questions": "Please refer to the weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The Virne framework is a significant contribution to the field, addressing the lack of comprehensive benchmarks for RL-based NFV-RA solutions.\n2. Virne is highly customizable, allowing users to simulate different network topologies, resource availability, and service requirements. This versatility enables testing across various real-world conditions, including energy-efficient, latency-sensitive, and resource-heterogeneous networks.\n3. The paper provides extensive experimental results, evaluating over 30 different NFV-RA algorithms across multiple topologies and network conditions.", "weaknesses": "1. More emphasis on real-world network scenarios (such as live 5G networks or dynamic, large-scale production environments) could further strengthen the validity and practical applicability of the framework.\n2. The NFV-RA problem often involves strict resource constraints (e.g., computing power, bandwidth, latency, etc.). RL agents may face challenges when dealing with these constraints, especially as the network scale increases, causing the solution space to grow rapidly and leading to inefficiencies in the agent's learning of optimal strategies. It is recommended to introduce a detailed discussion on handling constraints: further analysis and discussion on how to manage these complex resource constraints during the RL training process.\n3. It is worth discussing how to enhance the agent's transferability across multiple NFV-RA tasks through a multi-task MDP framework or meta-learning. For example, does the Virne framework support training a general strategy across different network environments and evaluate its generalization ability in new environments?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)