# Pointer-Based Deep Reinforcement Learning Agent for Network Slice Placement in Large Scale Networks

Abdel Kader Chabi Sika Boni, Eya Ben Henda, Hassan Hassan, Khalil Drira

► To cite this version:

Abdel Kader Chabi Sika Boni, Eya Ben Henda, Hassan Hassan, Khalil Drira. Pointer-Based Deep Reinforcement Learning Agent for Network Slice Placement in Large Scale Networks. The 14th IFIP/IEEE International Conference on Performance Evaluation and Modeling in Wired and Wireless Networks, Nov 2025, Paris, France. hal-05430998

HAL Id: hal-05430998

https://laas.hal.science/hal-05430998v1

Submitted on 24 Dec 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Pointer-Based Deep Reinforcement Learning Agent for Network Slice Placement in Large Scale Networks

Abdel Kader Chabi Sika Boni¹, Eya Ben Henda¹,², Hassan Hassan¹, and Khalil Drira¹

¹LAAS-CNRS, Université de Toulouse, CNRS, UPS, Toulouse, France
{akchabisik, ebenhenda, hhassan, khalil}@laas.fr

²Ecole Polytechnique de Tunisie

*Abstract*—Deep Reinforcement Learning (DRL) has proven effective across numerous fields, making it a promising approach for optimizing network slice placement in 5G mobile networks. Many studies have highlighted the advantages of DRL agents in autonomously and efficiently placing network slices, aligning well with the vision of major operators and organizations for next-generation networks. However, the challenge of stable convergence in DRL agents, particularly when their action space becomes large, has limited their application to small- and medium-sized infrastructures. In this paper, we address this limitation by introducing P-DDQN, a novel pointer-based DRL algorithm that uses a fixed set of navigation actions to enable spatial exploration of the infrastructure. P-DDQN is capable of exploring very large-scale networks while maintaining a constant number of actions. We evaluated it on small, medium, and large infrastructures, and simulation results demonstrate that it outperforms traditional DRL agents, such as Double Deep Q-Network (DDQN) and Deep Q-Network (DQN).

*Index Terms*—Network slicing, Deep Reinforcement Learning, Scalability.

## I. INTRODUCTION

Among the many innovations introduced with fifth-generation (5G) networks is the concept of Network Slicing. This refers to the segmentation of a shared physical infrastructure into logical networks, known as network slices, each designed to deliver specific Quality of Service (QoS) levels based on distinct network capabilities and characteristics. In theory, this concept can be leveraged to provide tailored QoS for Internet of Things (IoT) applications even when these applications have conflicting requirements. This is achieved by allocating the necessary resources to a particular network slice, which in turn hosts an application whose performance depends on the availability of those resources.

In the context of Zero-Touch Networks and autonomous IoT systems, several studies have explored the use of Deep Reinforcement Learning (DRL) agents for slice placement. These agents interact directly with the physical infrastructure, applying an observation-action-reward loop to continuously optimize their performance. While such agents have shown strong capabilities in self-optimizing slice placement, scalability remains a major unresolved challenge particularly when the slicing infrastructure includes more than a hundred physical

nodes. In fact, most existing approaches use DRL agents with neural networks whose final layer includes as many neurons as there are physical nodes in the infrastructure. However, experimental results have shown that when the DRL agent's action space exceeds a hundred options, convergence becomes problematic [15] [14]. As a result, these agents perform well on small infrastructures but suffer from poor performance as the infrastructure size increases. This work specifically addresses the scalability issue of DRL agents for network slicing. We propose a Pointer-based DDQN (P-DDQN) agent performing a pointer-based network slicing, where physical nodes are organized into a grid. P-DDQN navigates this grid using a fixed set of five possible actions (UP, RIGHT, DOWN, LEFT, and PLACE) regardless of the size of the infrastructure. Simulation results using this method were promising enough to warrant a comparison with two baseline DRL agents that follow conventional designs, where the number of possible actions equals the number of physical nodes. The simulations show that P-DDQN not only performs competitively with traditional agents on small infrastructures but, more importantly, demonstrates high scalability, maintaining strong performance on large-scale infrastructures. Throughout the remaining of this paper, we use P-DDQN and CNN+DDQN interchangeably.

The contributions of this work are as follows:

* We introduce a pointer-based network slice placement approach called P-DDQN that allows a DRL agent to reduce and maintain its placement decisions to just five actions, regardless of the size of the network slicing infrastructure.

* We fairly evaluate the performance of P-DDQN against two alternatives built on top of the Deep Q-Network (DQN) and Double Deep Q-Network (DDQN) algorithms.

* Performance comparisons were conducted across three infrastructure sizes: 3x3, 6x6, and 15x15. These experiments progressively demonstrate the effect of infrastructure size on the behavior and performance of each agent.

The rest of this paper is organized as follows. In Section
II. we reviewed previous works that have also addressed the problem of placing slices or virtual networks on physical infrastructure. The network slicing problem is formally defined in Section III, along with illustrations to help the reader fully grasp the key challenges involved. Our proposed DRL agent is presented in Section IV, and its performance is compared to two alternatives in Section V. Finally, we conclude the paper in Section VI and outline possible directions for future extensions of this work.

## II. RELATED WORK

Existing approaches to Virtual Network Function (VNF) chain placement span exact optimization methods, heuristic algorithms, meta-heuristics, and learning-based techniques, each offering different trade-offs between optimality and computational efficiency while facing fundamental scalability limitations as network size increases. Mathematical optimization through Integer Linear Programming (ILP) and Mixed Integer Linear Programming (MILP) formulations provide optimal solutions with binary decision variables for VNF-to-node mappings subject to resource, latency, and ordering constraints [3], with notable contributions including De Domenico's hybrid cloud deployment achieving 43% resource reduction [3], Luu's Branch-and-Bound approach with A* search improving slice acceptance rates by 10-20% [7], and Liu's dynamic Linear Programming (LP) relaxation framework reducing runtime by 90% while maintaining solutions within 5-10% of optimal for 1000-node networks [6]. However, most ILP/MILP approaches fail to scale beyond hundreds of nodes due to exponential computational complexity growth. Heuristic and metaheuristic methods provide computationally efficient alternatives through rule-based strategies and evolutionary algorithms, including Manzanares-López's Modified Priority-based Greedy (ModPG) heuristic for edge computing [8], Botez's enhanced A* algorithm achieving order-of-magnitude processing improvements for URLLC in 5G networks [2], and evolutionary approaches such as Particle Swarm Optimization [1] and Genetic Algorithms [11] demonstrating superior multi-objective performance, though these require extensive manual parameter tuning and cannot adapt dynamically to changing conditions. Deep Reinforcement Learning has emerged as a promising paradigm enabling agents to learn optimal strategies through environmental interaction, with pioneering work by Yan applying Graph Convolutional Networks with Asynchronous Advantage Actor-Critic (A3C) algorithms achieving 39.6% improvement in acceptance ratio [18], Pei's DDQN-VNFPA achieving 92.5% accuracy in Software-Defined Networking/Network Functions Virtualization (SDN/NFV) networks [10], and Dolati's DeepViNE introducing spatial intelligence through directional movements achieving 22% reduction in blocking probability [4]. Recognition of DRL limitations has motivated hybrid approaches combining reinforcement learning with complementary techniques, including Esteves's HA-DRL achieving faster convergence (50 versus 400 training phases) [5], Xiao's DNFS-VNE combining Deep Neuro-Fuzzy Systems with DRL [17], and Onsu et al.'s VAE-assisted

frameworks achieving 100% success rates for certain Service Function Chain (SFC) types [9], though these introduce significant computational complexity while remaining constrained by traditional action space designs. Despite innovations such as parallel training, hierarchical decomposition, and distributed orchestration, a fundamental challenge remains that cuts across these methodological differences: the scalability crisis driven by the explosion of the action space as network infrastructure grows. As network size increases, the corresponding expansion of variable action spaces introduces computational bottlenecks that scale exponentially, hindering the practical deployment of orchestration systems in industry-scale networks, where thousands of network functions must be managed across vast infrastructures. To overcome this limitation, we propose a pointer-based approach that enables spatial navigation with a constant action space size, independent of the network topology.

<figure><img src="image_2.png" /><figcaption>Fig. 1: Illustration of a network slice request composed of 3 VNFs.</figcaption></figure>

## III. PROBLEM STATEMENT

The problem of network slicing involves two main elements: the slice requests that need to be placed, and the physical infrastructure on which they must be deployed.

### A. Network slice request

A slice request is assumed to originate from an IoT service provider. It consists of Virtual Network Functions, each requiring a specific amount of Central Processing Unit (CPU) resources. Two successive VNFs are connected by a Virtual Link (VL) that has its own bandwidth requirements, as illustrated in Figure 1. In this work, we consider the QoS demanded by the IoT service provider to be satisfied only if all the resource requirements of both the VNFs and the VLs in the slice request are fully met. In practical terms, satisfying the QoS means that the slice request is successfully deployed on the physical infrastructure. Throughout the rest of this paper, we'll use the terms slice request and NSPR (Network Slice Placement Request) interchangeably.

### B. Physical infrastructure

The physical infrastructure consists of NFV-compatible computing nodes capable of running the VNFs included in a slice request, as shown in Figure 2. Each computing node has limited CPU resources. All computing nodes are interconnected through a network core composed of SDN-compatible switches and routers that dynamically route traffic between active VNFs. The physical links in this core network have limited bandwidth capacities.
<figure><img src="image_3.png" /><figcaption>Fig. 2: Illustration of a physical infrastructure with 9 computing nodes capable of running VNFs while being limited in CPU resources. Physical links are also limited in available bandwidth resources.</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Fig. 3: Illustration of a network slice request placement in a physical infrastructure.</figcaption></figure>

### C. Network slice placement/deployment

Deploying a slice request on the physical infrastructure therefore involves selecting, for each VNF, a computing node on which to place it, such that the remaining CPU resources are non-negative. From the second VNF onward, a placement is considered successful only if a physical path can also be found in the core network to connect the hosting node of the previous VNF to that of the current one. This path must have at least the minimum bandwidth required by the virtual link connecting the two VNFs. The challenge of network slicing thus lies in making anticipatory decisions that increase the chances of successfully placing the VNFs as well as successfully mapping their virtual links. Figure 3 provides an example of deploying the slice request from Figure 1 onto the physical infrastructure shown in Figure 2. The overall goal in this work is to maximize the acceptance ratio of placed network slices on large infrastructures as that means many QoS demands from IoT service providers were satisfied.

## IV. POINTER-BASED DRL AGENT FOR PLACING NSPR

P-DDQN interacts with its environment through an observation-action-reward mechanism, as illustrated in Figure 4. The physical infrastructure, along with the NSPRs to be processed, are considered part of the environment. The remaining resources on the infrastructure nodes, as well as the resource requirements of the VNF currently being processed, are described in an observation sent to the agent. This observation is fed into the agent's neural network, which then generates an action. The design of this action is the key aspect that distinguishes our DRL agent from those found in the literature. In existing approaches, the number of possible actions (i.e., the size of the action space) is equal to the number of physical nodes in the infrastructure. In contrast, our approach fixes the number of possible actions to five, regardless of how the

<figure><img src="image_5.png" /><figcaption>Fig. 4: Our agent (P-DDQN) interacting with its environment through a observation-action-reward mechanism.</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Fig. 5: Illustration of a physical infrastructure represented as a grid in order to enable pointer-based directional movements.</figcaption></figure>

number of physical nodes varies. Finally, a reward function is used to quantify the optimality of the actions taken by P-DDQN.

### A. Actions

The actions correspond to directional movements (UP, RIGHT, DOWN, LEFT, and PLACE) that P-DDQN performs to move a virtual pointer as it interacts with the environment. To enable this mechanism, we transform the physical infrastructure by reorganizing its physical nodes into a grid, as shown in Figure 5. In fact, any planar graph can be transformed into a grid in linear time [13], which ensures that our approach generalizes to arbitrary network topologies commonly found in real-world deployments, while preserving the spatial reasoning benefits inherent to grid-based processing. When P-DDQN determines that the physical node currently indicated by the pointer is the most suitable for placing the ongoing VNF, it triggers the PLACE action to actually deploy the VNF on that node. The key advantage of this design lies in its high scalability and increased stability. Indeed, it has been shown that with the conventional approach where the number of actions equals the number of physical nodes, DRL agents face significant convergence issues once the number of nodes exceeds a few hundred [15] [14].

### B. Observation

To accurately represent the grid-based organization of physical nodes within the infrastructure, we provide P-DDQN with a grid-like observation, as shown in Figure 6. For each physical node, five features are extracted: available CPU, CPU usage, pointer position, embedded VNFs, visitation flag, and visit count. For each link, we record the available bandwidth. The resulting grid-like observation is extended with the requirements of the ongoing VNF, specifically its required CPU and the bandwidth of the virtual link connecting it to the previous VNF. All empty areas in the infrastructure are filled with zero-padding in the observation. To preserve the spatial structure of this information during processing, we use Convolutional Neural Network (CNN) layers for feature extraction, as illustrated in Figure 4.
<figure><img src="image_7.png" /><figcaption>Fig. 6: Provided grid-like observation to agent.</figcaption></figure>

### C. Reward

The reward function guides the agent toward efficient placement strategies through carefully designed incentives and penalties that balance resource utilization, physical paths efficiency and exploration behavior.

1) Successful VNF placement reward: When a VNF is successfully placed, the agent receives a composite reward encouraging resource and physical path efficiency.

$$R_{\text{placement}} = 100.0 \times \rho_{\text{cpu}} \times \rho_{\text{bw}} \quad (1)$$

where:

* $\rho_{\text{cpu}} = (\text{CPU}_{\text{available}} / \text{CPU}_{\text{total}})^2$ encourages placement on less loaded nodes through quadratic scaling

* $\rho_{\text{bw}} = \frac{1}{\text{path_length}-1}$ for inter-node placements, favoring shorter communication paths

* $\rho_{\text{bw}} = 1.0$ for same-node placement, providing maximum bandwidth efficiency bonus

2) Penalty structure: The system applies graduated penalties to discourage inefficient behaviors and constrain violations as follows: -100 points (constraint violation) for placement attempts on nodes lacking sufficient resources, providing strong negative feedback; -50 points (invalid movement) for attempting to move outside grid boundaries, teaching spatial awareness; -1 point (valid movement) per movement to encourage efficient navigation and prevent excessive wandering and additional -5 points (repeated placement) for attempting placement at previously failed locations within the same VNF placement.

## V. EVALUATION

### A. Methodology

We built our P-DDQN agent on top of the DDQN algorithm [16], which leverages a Prioritized Replay Memory [12] to store experiences. An experience is defined as a tuple consisting of (observation, action, reward, next observation). The interactions between the agent and the environment are grouped into episodes, each of which begins by initializing the resource state of the slicing infrastructure and setting the ranges for the resource requirements of each NSPR. An episode ends when the agent fails to successfully place a VNF. We evaluate our agent by comparing it to two alternatives: the DDQN and DQN baselines where the observation is flattened and z-score normalized and the number of actions equals the number of computing nodes. Each agent is tuned to achieve its best performance by optimizing three key parameters: the

number of steps (the number of consecutive tuples that form an experience), the replay start size (the number of stored experiences before the agent begins sampling for training), and the batch size (the number of experiences sampled simultaneously during training). The fine-tuning phase was conducted using a 3x3 infrastructure. Additional comparisons were then carried out using 6x6 (36 computing nodes) and 15x15 (225 computing nodes) infrastructures to assess scalability performance. All performance metrics are evaluated based on the number of successfully placed NSPRs and expressed as an acceptance ratio, defined as the number of placed NSPRs divided by the maximum number of NSPRs placed.

### B. Fine-tuning all agents

1) Our agent: Figures 7-a to 7-c present the fine-tuning results of our agent P-DDQN (CNN+DDQN) within the 3x3 infrastructure. Figure 7-a shows that setting the number of steps to 1 yields the highest acceptance ratio. This indicates that treating each movement of the pointer as a separate experience is more effective than considering the entire path taken by the pointer as a single experience. Looking at Figure 7-b, we observe that the improvement in acceptance ratio begins at different episodes depending on whether the replay start size is set to 10000, 25000, or 50000. This is expected, as collecting 10000 experiences before training the neural networks naturally takes less time (and therefore fewer episodes) than collecting 25000 or 50000. However, once training begins, the version of P-DDQN with a replay start size of 25000 consistently achieves a significantly better average acceptance ratio. Finally, Figure 7-c shows that once the replay start size is reached, sampling 128 experiences simultaneously as the minibatch size during each backpropagation step is more effective than using smaller minibatch sizes of 32 or 64.

2) Baseline DDQN: We repeated the same fine-tuning simulations using the baseline DDQN. Figures 8-a to 8-c show the results obtained. Overall, the best-performing parameters remained similar: setting the number of steps to 1 resulted in the highest acceptance ratio (Figure 8-a), a replay start size of 25000 provided the best balance between training efficiency and performance (Figure 8-b), and a minibatch size of 128 led to up to 10% more NSPRs successfully placed compared to DDQN versions using minibatch sizes of 32 or 64 (Figure 8-c).

3) Baseline DQN: The fine-tuning of the baseline DQN further confirmed the selection of these hyperparameter values, as shown in Figures 9-a to 9-c. Therefore, for the baseline DQN, we kept the number of steps set to 1 (Figure 9-a), the replay start size at 25000 (Figure 9-b), and the minibatch size at 128 (Figure 9-c).

### C. Performance comparison

We compared the performance of the three agents on three grid-based infrastructures of sizes 3x3, 6x6, and 15x15. For all agents, we used the same reward function, the same initial resources in the network slicing infrastructures, and the same NSPRs to be placed.
<figure><img src="image_8.png" /><figcaption>Fig. 7: Fine-tuning our agent P-DDQN (CNN+DDQN) on 3x3 infrastructure.</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>Fig. 8: Fine-tuning of baseline DDQN on 3x3 infrastructure.</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>Fig. 9: Fine-tuning of baseline DQN on 3x3 infrastructure.</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>Fig. 10: Performance comparison : P-DDQN (CNN+DDQN) vs baseline DDQN vs baseline DQN.</figcaption></figure>

Figure 10-a shows the acceptance ratios on the 3x3 in-
frastructure. It can be observed that P-DDQN (referred to as
CNN+DDQN) reaches the minimum required 25000 experi-
ences for training around episode 550. In contrast, the baseline
DDQN and DQN agents reach this threshold around episode
930. From these points onward, the agents begin training
their neural networks, refining their VNF placement policies.
Around episode 1600, the baseline DDQN outperforms both
P-DDQN and DQN by approximately 12% in terms of slices
successfully placed. This trend continues for most of the re-
maining episodes, although the performance briefly converges

around episode 2500. These results confirm that our fine-
tuning of the baseline agents was effective, and also show that
using CNN layers for feature extraction does not necessarily
guarantee a performance boost. However, it is noteworthy
that the pointer-based VNF placement approach used by P-
DDQN did not hinder its ability to place slices or to learn an
effective placement policy, managing to stay within 12% of
the performance of the DDQN and DQN baselines.

Figure 9-b shows the performance of the three agents
on the 6x6 infrastructure. What immediately stands out is
the performance gap between our P-DDQN agent and the
DDQN and DQN baselines. Yet, all agents used the same
previously fine-tuned hyperparameters. Conceptually, on this infrastructure, the DDQN and DQN baselines operate with an action space of size 36 (one for each of the 6x6 nodes), while P-DDQN keeps its action space limited to just 5. This already highlights the impact that action space size can have on the performance of DRL agents in network slicing tasks. On average, P-DDQN placed about 70% more slices than the two baselines across all episodes.

To further validate our previous conclusions, we conducted a performance comparison of the three agents on an infrastructure with 225 computing nodes (15x15). The results, shown in Figure 9-c, are conclusive. The DDQN and DQN baselines exhibit near-zero performance. While surprising at first glance, this reinforces the well-known issue that DRL agents struggle to converge when their action space exceeds a hundred possible choices. Meanwhile, our P-DDQN agent maintains an average performance of around 50%, relative to the highest number of slices placed throughout the entire simulation (which was also achieved by P-DDQN itself). The slight drop in its performance can be attributed to the increase in the size of the observations the agent receives. This suggests that, although action space size has the most significant impact on agent performance, the size of the observation space also plays a non-negligible role. Overall, these results position P-DDQN as one of the few DRL-based network slicing agents capable of effectively placing slices on infrastructures with over a hundred computing nodes, without a collapse in performance.

# VI. CONCLUSION

In this work, we proposed a pointer-based Deep Reinforcement Learning agent named P-DDQN capable of placing the Virtual Network Functions of network slices using only five directional actions. P-DDQN navigates a grid-organized set of computing nodes by moving a virtual pointer in four directions (UP, RIGHT, DOWN, LEFT). When it determines that the currently pointed computing node is the most suitable for placing the VNF under consideration, it triggers the 5th action (PLACE) to deploy the VNF on that node. We compared this approach with two baselines, DDQN and DQN, which follow the conventional method in the literature where the number of possible actions equals the number of computing nodes in the slicing infrastructure. After fine-tuning all agents and evaluating their performance across three infrastructures of sizes 3x3, 6x6, and 15x15, we observed that our P-DDQN agent can place up to 60% more slices on large-scale infrastructures. This result positions P-DDQN as one of the few capable of maintaining strong performance across both small and large infrastructures.

In future work, we intend to consider more resource types per computing node, such as Random Access Memory (RAM) and storage capacity. Additionally, we plan to extend our approach to a multi-domain setting. Currently, we assume a single-domain scenario in which the infrastructure manager (typically a network operator) has full control over the entire physical network slicing infrastructure. Supporting multiple

domains will require a collaboration between many tenants thus designing a distributed version of our P-DDQN agent.

# VII. ACKNOWLEDGMENTS

This work was supported by the ANR under the France 2030 program, grant "NF-NAI: ANR-22-PEFT-0003".

# REFERENCES

[1] Samane Asgari, Mohammad Mahdavi, et al. Performance-aware placement and chaining scheme for virtualized network functions: A particle swarm optimization approach. *Computer Networks*, 185, 2020.

[2] Diana Botez, Naveen Kumar, Dan Popescu, and Mariana Mocanu. Efficient network slicing with sdn and heuristic algorithm for low latency services in 5g/b5g networks. *Electronics*, 12(15), 2023.

[3] Antonio De Domenico, Ya-Feng Liu, and Wei Yu. Optimal virtual network function deployment for 5g network slicing in a hybrid cloud infrastructure. *IEEE Transactions on Wireless Communications*, 19(12), 2020.

[4] Mahdi Dolati, Seyedeh Bahereh Hassanpour, Majid Ghaderi, and Ahmad Khonsari. Deepvine: Virtual network embedding with deep reinforcement learning. *IEEE INFOCOM 2019 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS)*, 2019.

[5] Jose Jurandir Alves Esteves, Amina Boubendir, Fabrice Guillemin, and Pierre Sens. A heuristically assisted deep reinforcement learning approach for network slice placement. *arXiv preprint arXiv:2105.06741*, 2021.

[6] Kecheng Liu, Yuling Chen, Zhiyong Zhang, and Tao Huang. Towards efficient large-scale network slicing: An lp dynamic rounding-and-refinement approach. *IEEE Transactions on Services Computing*, 16(3), 2023.

[7] Quang-Trung Luu, Minh-Thanh Nguyen, Tuan-Anh Do, Michel Kieffer, Van-Dinh Nguyen, Tai-Hung Nguyen, and Huu-Thanh Nguyen. Network slicing with flexible vnf order: A branch-and-bound approach. *arXiv preprint arXiv:2412.05993*, 2024. Accessed: January 2025.

[8] Pilar Manzanares-López, Juan Pedro Muñoz-Gea, and Josemaria Malgosa-Sanahuja. Vnf placement for service function chains with strong low-delay restrictions in edge computing networks. *Applied Sciences*, 10(18), 2020.

[9] Murat Arda Onsu, Poonam Lohan, Burak Kantarci, and Emil Janulewicz. Genai assistance for deep reinforcement learning-based vnf placement and sfc provisioning in 5g cores. *arXiv preprint arXiv:2404.08845*, 2024.

[10] Jianing Pei, Peilin Hong, Miao Pan, Jianqing Liu, and Jingsong Zhou. Optimal vnf placement via deep reinforcement learning in sdn/nfv-enabled networks. *IEEE Journal on Selected Areas in Communications*, 38(2), 2020.

[11] Thien Phan, Fan Wu, Mahmoud Bekhit, Ahmed Fathalla, and Ahmad Salah. Optimizing placement and scheduling for vnf by a multi-objective optimization genetic algorithm. *International Journal of Computational Intelligence Systems*, 17, 03 2024.

[12] Tom Schaul, John Quan, Ioannis Antonoglou, and David Silver. Prioritized experience replay, 2016.

[13] Walter Schnyder. Embedding planar graphs on the grid. In *Proceedings of the First Annual ACM-SIAM Symposium on Discrete Algorithms (SODA)*, Philadelphia, PA, USA, 1990. Society for Industrial and Applied Mathematics.

[14] Khuong Tran, Maxwell Standen, Junae Kim, David Bowman, Toby Richer, Ashlesha Akella, and Chin-Teng Lin. Cascaded reinforcement learning agents for large action spaces in autonomous penetration testing. *Applied Sciences*, 12(21), 2022.

[15] Tom Van de Wiele, David Warde-Farley, Andriy Mnih, and Volodymyr Mnih. Q-learning in enormous action spaces via amortized approximate maximization, 2020.

[16] Hado van Hasselt, Arthur Guez, and David Silver. Deep reinforcement learning with double q-learning, 2015.

[17] Ailing Xiao, Ning Chen, Sheng Wu, Peiying Zhang, Linling Kuang, and Chunxiao Jiang. Dnfs-vne: Deep neuro fuzzy system driven virtual network embedding. *IEEE Transactions on Network and Service Management*, 20(3), 2023.

[18] Zhongxia Yan, Jingguo Ge, Yulei Wu, Liangxiong Li, and Tong Li. Automatic virtual network embedding: A deep reinforcement learning approach with graph convolutional networks. *IEEE Journal on Selected Areas in Communications*, 38(6), 2020.