###### Abstract  
Next-generation mobile networks, such as those beyond the 5 th generation (B5G) and 6 th generation (6G), have diverse network resource demands. Network slicing (NS) and device-to-device (D2D) communication have emerged as promising solutions for network operators. NS is a candidate technology for this scenario, where a single network infrastructure is divided into multiple (virtual) slices to meet different service requirements. Combining D2D and NS can improve spectrum utilization, providing better performance and scalability. This paper addresses the challenging problem of dynamic resource allocation with wireless network slices and D2D communications using deep reinforcement learning (DRL) techniques. More specifically, we propose an approach named DDPG-KRP based on deep deterministic policy gradient (DDPG) with K-nearest neighbors (KNNs) and reward penalization (RP) for undesirable action elimination to determine the resource allocation policy maximizing long-term rewards. The simulation results show that the DDPG-KRP is an efficient solution for resource allocation in wireless networks with slicing, outperforming other considered DRL algorithms. deep reinforcement learning; network slicing; device-to-device; resource allocation 2024246079. [[https://doi.org/10.3390/s2418607910](https://doi.org/10.3390/s2418607910)]([https://doi.org/10.3390/s2418607910](https://doi.org/10.3390/s2418607910))  
utilizes a different band. For the underlay subcategory, cellular and D2D communications share the cellular spectrum, while in the overlay subcategory, each D2D pair uses dedicated resources within the cellular spectrum [3]. Recently, the idea of combining D2D communication with NS in wireless network models has been on the rise [2]. Radio resource allocation effectively and efficiently in light of these paradigms is a complicated problem to solve; existing traditional approaches fail to provide good solutions due to the following characteristics [1]:  
* Mathematical modeling: taking into account the diversity of network slice services combined with D2D communications and the constraints of the physical environment, it becomes complex to obtain a mathematical expression for the allocation of radio resources that meets the requirements of each service (e.g., latency, service-level agreement (SLA) and throughput). * Traditional approaches do not adapt to episodic uncertainty: traditional optimization approaches require precise mathematical models with known parameters to adequately perform, which is often difficult to achieve in practice. Due to the lack of ability to explore and learn from the environment, traditional approaches frequently do not allocate resources efficiently. In this context, deep reinforcement learning (DRL) algorithms, a branch of artificial intelligence (AI) that uses deep neural networks (DNNs), are attractive for exploring the environment and learning how to allocate radio resources. Algorithms using DRL have been considered excellent solutions for allocating radio resources in next-generation wireless networks that consider the NS paradigm [1; 4; 5]. This work proposes an approach named (DDPG-KRP) based on the deep deterministic policy gradient (DDPG) algorithm with K-nearest neighbor (KNN) clustering and reward penalization (RP). The KNN algorithm discretizes action at the output of the actor network that is part of the DDPG; the proposed reward penalization strategy consists of assigning a null reward if the action generated does not meet the SLA requirements of each slice. The proposed approach is based on improving agent learning and providing SLAs in each slice. To apply and evaluate the performance of the proposed approach through computational simulations, a mobile network scenario with D2D communication and network slices based on two service categories (ultra-reliable and low-latency communication (URLLC) and video) was considered.