# Deep Reinforcement Learning for Resource Management on Network Slicing: A Survey
## 1 Introduction  
A major goal of 5G and 6G networks, from now on called 5G/6G, is to deliver a wide variety of services with distinct performance requirements under a (physical/virtual) shared infrastructure [1]. All 5G/6G networks must offer high-speed connections, very high reliability, and extremely low latency for empowering different verticals and enabling new business models [2, 3]. These networks promote realizing novel use cases, including ultrareliable low-latency communication (uRLLC), massive machine-type communication (mMTC), enhanced mobile broadband (eMBB), strengthened enhanced mobile broadband (sEMBB), ultramassive machine-type communications (umMTC), massive ultrareliable low-latency communications (mURLLC), mobile broadband reliable low-latency communications (MBR-LLC), and extremely reliable and low-latency communications (ERLLC) [1, 4, 5, 6]. Network slicing (NSL) and deep reinforcement learning (DRL) are two key enabling technologies of 5G/6G [7]. A 5G/6G network can comprise one or more network slices belonging to single or multiple tenants. A slice is a customized and isolated logical network conceived to support strict quality of service (QoS) and quality of experience (QoE) requirements [8, 9] such as those demanded by, for instance, remote surgeries and immersive media. Network providers need to perform intelligent and efficient resource management to realize slices that meet the requirements of 5G and 6G use cases. Resource management is far from being a straightforward task since it requires mechanisms for its constitutive phases: admission control (accept/reject multitenant network slice requests--NSLRs) [10], resource allocation (assign resources to admitted NSLRs) [11], resource scheduling (program the timely use of allocated resources) [12], and resource orchestration (instantiateand manage the life cycle of slices) [13]. In each phase, the mechanisms must meet diverse performance requirements (e.g., reliability, throughput, latency, packet loss), while increasing provider profits, improving network utilization, and guaranteeing resource provisioning (or re-provisioning) dynamically [14]. Furthermore, intelligent and effective resource management involves predicting the demand coming from many tenants (each tenant with multiple NSLRs) and achieving autonomous behavior of slices. Although many studies have proposed solutions to manage resources in NSL using, for instance, different heuristics [15; 16; 17] and genetic algorithms [18], this paper surveys network slicing resource management approaches based on reinforcement learning (RL) and DRL techniques. RL and DRL will play a critical role in turning 5G/6G into a reality. Remarkably, they allow evolving resource management in NSL from techniques based on models to those without models, which learn deeply by interacting with the environment to satisfy experience level agreements (XLA) related to QoE and service level agreements (SLA) associated with QoS. In RL, an agent makes decisions considering the environment's states (e.g., set of computing and networking resources available to attend slices). Decisions are made to select actions (e.g., allocate and instantiate a node for an accepted NSLR) to apply in an environment. A RL agent monitors the result (expressed as a reward, for example, optimized network utilization) of its interaction with the environment (e.g., a 5G/6G physical network) to adjust its strategy to achieve an optimal policy automatically (e.g., optimize the action selection to support requirements of MBRLLC slices) [19; 20]. RL approaches slowly converge to the optimal policy when exploring and acquiring knowledge in large-state action sets, making it difficult to use in large-scale 5G/6G deployments. Deep learning (DL) has been used to face RL limitations, leading to DRL [21; 22]. From a high-abstraction level, DRL uses RL to train deep neural networks (DNNs), such as feed-forward neural networks (FNNs) [23] and recurrent neural networks (RNNs) [24], to quickly learn accurate optimal policies.