# Deep Reinforcement Learning for Resource Management on Network Slicing: A Survey
## 4 Resource Allocation
### Allocation Based on DRL
The second tier employed DQN enhanced with two FNNs to allocate radio, edge, and cloud resources considering the slices' arrival requests. The approach modeled the space of states representing available radio units and QoE satisfaction. The space of actions clustered remote radio heads to form access units intended to reduce intercell interference. Ref. [71] presented a DQN-based mechanism to allocate bandwidth and 5G-RAN resources to slices serving mobile, videos, and vehicle communications for increasing the long-term resource utilization and the revenue of virtual network providers. The DNN used ReLU as an activation function. The mechanism modeled the space of states as the requested bandwidth (arriving randomly) and the consumed energy. The space of actions was represented as the slice selected by the mobile virtual network operator to maximize profit. Ref. [72] proposed a DQN-based approach to allocate bandwidth and virtual machines to services queued in a time window or during their arrival seeking optimizing delays and resource usage costs. The approach used a DNN activated by ReLu and designed the space of states as resource request arrivals and queueing levels from the last assignment. The actions represented binarily the operation of allocating bandwidth successfully. Ref. [74] introduced a resource allocation approach based on DQN and DDQN, focused on elastic and real-time slices, and conceived to maximize spectral efficiency utilization while reducing costs in 5G-RAN with many intelligent devices. The DNNs of DQN and DDQN used two hidden layers trained by the GD algorithm [153]. The approach modeled the space of states as the carrier power traffic assigned to each slice and the actions as a binary representation of bandwidth allocation per slice. Refs. [75; 76] proposed a DQN-based and dynamic framework that reserves and assigns unused bandwidth resources to virtualized RAN to maximize QoS satisfaction and resource utilization. The framework used a FNN with 4 neurons in the input layer, 2 hidden layers, and 20 neurons in the output layer. The space of states was defined considering the percentage of allocated virtual resources and the average resource utilization of each slice. The space of actions was modeled as the percentage for decreasing or increasing assigned resources. Ref. [77] presented a DQN-based resource management approach to reserve and allocate cache resources at the edge network for maximizing QoS satisfaction and network utilization to mobile virtual network operators. The DQN agent used a FNN with 4 and 11 neurons in the input and output layers and 2 hidden layers. The approach conceived the space of states regarding resource utilization, QoS satisfaction, reserved resources per slice, and allocated cache resource. The space of actions was defined to increase or decrease resources to cache slices. Ref. [65] introduced a bandwidth allocation strategy that uses a LSTM-based advantage actor-critic (A2C) [154] algorithm (i.e., it combines policy-based and value-based RL techniques) to maximize spectral efficiency, SLA satisfaction ratio, and profit in RAN. The LSTM-based A2C agent used Softmax [155] as activation function in the output layer. The strategy modeled the space of states as the number of slice arrival requests within a specific time window and the space of actions as the bandwidth to allocate to each slice. Ref. [63] presented a DQN-based solution for allocating Internet vehicular and smart city applications. The DQN model included a fully connected DNN with an input layer, two hidden layers activated through ReLU, and an output layer activated through a linear activation function. The DQN agent operated with a space of states based on the number of resource blocks used at time \(t\). The actions employed by the DQN agent allow for deciding if the user requests with heterogeneous latency demands and diverse computing loads must run at the cloudified RAN or edge network. Ref.