# Deep Reinforcement Learning for Resource Management on Network Slicing: A Survey
## 4 Resource Allocation
### Allocation Based on DRL  
The following paragraphs review modern investigations using DRL to allocate resources in NSL or related technologies. Refs. [58; 59] presented a framework that allocates computing, storage, and radio resources to manufacturing services for maximizing the providers' long-term incomes. The framework used DDQN and dueling DQN agents trained by the stochastic gradient descent (SGD) algorithm [150] and modeled their states considering requested resources, available computing resources, and connectivity capabilities of data centers responsible for storing virtualized radio resources. The agentsoperated with actions defined as the resources to assign per request. Using TensorFlow, the environment was emulated by creating three slice classes (i.e., utilities, automotive, and manufacturing) under different parameter settings. The slice requests followed a Poisson distribution. Ref. [60] proposed a DQN-based strategy that allocates the radio and backhaul resources in a virtualized RAN to balance the QoS satisfaction and resource utilization of slices. The strategy considered the space of states as a vector involving the satisfaction ratio and the resources allocated to a slice. The DQN agent operated with actions modeled as percentages representing optimal resource provisioning. The evaluation involved an emulated mobile network with four classes of slices (enhanced-user equipment broadband, ultralow-latency communications, massive Internet of Things, and high-definition TV) following an exponential distribution. Ref. [61] proposed a resource allocation method based on a proprietary RL algorithm and a DNN to optimize the positioning of functions forming service function chains in metro-core optical networks. Four hidden layers formed the DNN with 100 nodes per layer. The method modeled the space of states regarding three layers (optical, IP/MPLS, and service slicing) and the space of actions as a decision state that indicates if one or more service function chains need reconfiguration. The emulation environment and algorithm were created using openAI gym; the proposed algorithm was trained using a mobile traffic dataset of the Milan urban area [62]. Ref. [66] introduced a resource allocation framework based on constrained DQN formed by a DNN (composed of two fully connected layers with 64 and 32 nodes) trained with various RL algorithms, and devised to meet performance requirements of video, VoLTE, and uRLLC slices in 5G-RAN. The framework modeled the space of states as the number of active users per service. The space of actions defined the bandwidth to allocate for each service. Ref. [64] introduced a resource allocation approach based on powered DDQN to meet SLAs as well as maximize resource utilization and provider revenue according to the dynamics of service requests on 5G-RAN. The approach used two generative adversarial networks (GANs) trained by the gradient descent (GD) algorithm [151] to minimize the difference between the estimated action-value distribution and the target action value distribution. Dueling GAN-DDQN represented the space of states as the service demands within a specific time window and modeled the space of actions as the bandwidth to assign to each slice. Refs. [67; 68; 69] presented an approach based on DQN that flexibly allocated resources on 5G-RAN to maximize network slice requirement satisfaction and improve resource blocks usage ratio. The approach modeled the space of states as the available radio resource blocks and the space of actions as allocation or not of resource blocks. Furthermore, the approach used the Ape-X method [152] to accelerate the learning by processing multiple DQN agents. Ref. [70] introduced a two-tier resource allocation approach aimed to meet QoE requirements and achieve efficient utilization on 5G end-to-end slicing. The first tier proposed a dynamic resource optimization problem for allocating the radio resources under constraints on rate, power, and interference.