## 2 Related Work  
Radio resource management for improving the performance of next-generation wireless networks, such as those involving B5G, is a complex problem to solve using traditional mathematical methods. Therefore, considering related challenges, in the last decade, many research projects involving DRL, D2D, and NS have been developed, achieving promising results. In [6], the authors explored the sparse channel estimation (CE) problem of frequency-selective multipath channels in a filter bank multicarrier with offset quadrature amplitude modulation (FBMC/OQAM) industrial Internet of Things (IIoT) communication systems from the perspective of a sparse Bayesian learning (SBL) framework. The authors proposed the Block SBL (BSBL) algorithm to estimate the channel performance by exploiting the block-sparse structure of the sparse multipath channel model. Computer simulation results demonstrated the robustness of the BSBL CE approach in FBMC/OQAM systems, which can achieve lower mean square error (MSE) and bit error rate (BER) than the traditional least squares (LS) method and classical compressive sensing method. An approach to allocating radio resources in networks that consider network slicing and D2D communication was presented in [2]. The authors presented an algorithm based on traditional mathematical methods that solved the NP-hard optimization problem to maximize the average spectral efficiency and throughput of the 5G heterogeneous cellular network based on D2D with NS. In other approaches that did not consider network slicing, the authors also formulated the resource allocation problem through a complex optimization problem. They used traditional algorithms to solve the problem, such as those described in the papers [3; 7; 8]. However, these approaches are not adaptive to the stochastic changes in the environment that consider network slicing and D2D communication. In this work, we propose using a DRL-based algorithm that adapts according to the environment variations and performs the adaptive joint allocation of power and bandwidth efficiently. Among the approaches that consider network slicing and D2D simultaneously, those that use reinforcement learning are the ones that have shown the best results [9]. Due to this reason, we focused on comparing the proposed DDPG-KRP algorithm with an approach using DQN that is similar to the one used in [4; 9]. In [4], the authors investigated the application of DQN to solve some typical resource management problems in NS scenarios and demonstrated the advantages of this technique over some resource allocation schemes. The DRL agent allocated a bandwidth between slices, but a pre-training delay was found to satisfy the SLAs of each slice. In [1], similar to [4], the same probability distributions were used to represent the network traffic demand in each slice. However, in [1], the resource allocation problem was formulated as a constrained Markov decision process. To solve the problem, the authors proposed a DRL-based algorithm using interior-point policy optimization. The simulations, however, showed that the agent proximal policy optimization (PPO) acting without restriction can achieve good rewards. In this paper, the same network traffic demand was used for each slice and D2D communications were inserted for spectral efficiency optimization. In [3], the authors formulated the resource virtualization problem in a wireless network with D2D communication underlay. This combination generated a nonlinear integer programming problem divided into two smaller linear integer problems. Both possible solutions were combined, and the problem was solved using two different schedulers. In [9], the authors proposed a resource allocation scheme between slices in a D2D-based virtualized communication network. A DRL agent was used for resource adjustment between slices. The authors formulated the resource allocation problem as a convex optimization problem and solved it with an alternating direction method.