# Deep Reinforcement Learning for Resource Management on Network Slicing: A Survey
## 4 Resource Allocation
### Allocation Based on RL  
The following paragraphs review modern investigations using RL to allocate resources in NSL or related technologies. Ref. [48] presented a radio resource allocation approach based on Q-learning and centered on 5G haptic communications for maximizing the utilization of scarce radio resources according to dynamism and the requirements of vertical haptic applications. The approach represented the space of states considering the allocated resources, application performance requirements, and resource utilization in each haptic vertical slice. The Q-learning agent modeled the space of actions in a binary way to denote the allocation or nonallocation of slices. The performance evaluation was performed in a nonspecified tool and conceived by a seven-cell hexagonal grid layout model with two vertical slices: a radio slice for the connectivity service to haptic communications and the other for human-to-human communications. The haptic devices were randomly distributed throughout the radio slice, while the users requesting the vertical applications followed a Poisson distribution. Ref. [49] presented a resource allocation framework based on various RL algorithms, such as Q-learning, SARSA, expected SARSA, and Monte Carlo, and devised to maximize the efficient utilization of resources in 5G Fog-RAN while guaranteeing the low-latency requirements of IoT applications. The framework modeled the space of states as resource (computing and processing) blocks occupied in fog nodes and IoT applications' characteristics (latency, throughput, and channel capacity). The actions decided the appropriate layer (fog or cloud) to assign resources to provide IoT applications. The performance evaluation involved an IoT environment, including 10 applications from diverse domains (smart farming, smart retail, smart home, wearables, entertainment, smart grid, smart city, industrial Internet, autonomous vehicles, and connected health) with different latency requirements and profit features. Figure 4: Resource allocation architecture using RL/DRL. [PERSON]. [50] introduced a Q-learning-based resource allocation method to minimize end-to-end latency and improve computing resource utilization in 5G Fog-RAN. The method modeled the space of states regarding the user requests, request arrival rate, percentage of allocated resources, percentage of unused allocated resources, minimum allocation requirements, and the maximum delay allowed. The Q-learning agent operated a space of actions defined to allocate or not compute resources. The method was evaluated in an open-source 5G K network simulator based on NS3 while openAI gym served to implement the Q-learning algorithm. In low-orbit satellite networks, [PERSON]. [51] proposed a dynamic resource allocation approach based on Q-learning for maximizing the provider's revenue (also known as system utility) and improving the users' QoS satisfaction. The approach modeled the space of states considering the allocated and utilized resources per slice in a specific time \(t\). The Q-learning agent used the actions to indicate whether the resource unit must be allocated (or not) for a particular user. The evaluation included two low-orbit satellite slices containing fixed radio resource pools. Mininet emulated the abovementioned slices. [PERSON]. [53; 54] introduced an efficient resource allocation scheme based on Q-learning, focused on eMBB and vehicle-to-everything (V2X) services on 5G-RAN, and devised to maximize the overall resource utilization taking into account the services' performance requirements and traffic dynamism. The scheme considered a space of states modeled as the number of resource blocks of cell bandwidth in the uplinks and downlinks. The actions were represented regarding allocation ratios of eMBB and V2X slices. [PERSON]. [55] proposed an approach based on Monte Carlo [149] and Q-learning in an edge-computing and multitenant environment seeking to provide social welfare, meet QoS requirements, and maximize power resource allocation per network slice.