## 2 Related Work  
The state-of-the-art researchers propose different methods to manage interference in D2D-enabled heterogeneous cellular networks. Depending on interference behavior, interference management is mainly categorized as interference avoidance, interference cancellation, and interference reduction. Interference avoidance techniques are used in [10] where DUEs coverage area is defined based on pre-defined SINR and the specific transmission power of UEs, and the resource allocation problem is solved using graph coloring techniques. To improve spectral efficiency, system throughput, and link outage probability, in [11] authors proposed an interference-aware power allocation (IAPA) solution. To minimize interference for a 3-tier network, in [12] the authors initially derived the outage probabilities of all links and then defined the lower distance boundaries from DUEs to base stations and vice-versa. Finally, the authors proposed an optimal small cell deployment density algorithm to ensure the minimum QoS for both D2D and cellular communications [1]. Using DRL methods in [13], the authors proposed a knowledge plane (KP)-based management and orchestration system for network slicing (NS) in 5G and beyond. Finally, the authors compared the outcomes of the proposed solutions in terms of energy consumption, CPU utilization, and time efficiency. Reinforcement learning is becoming popular for solving resource allocation problems in 5G networks. In [14], the authors proposed a Q-learning-based resource allocation technique for solving power allocation and spectrum problems for a single-tier D2D network. Here, Q-learning is used to assign the proper channel and appropriate transmit power for a D2D user. In [15], [PERSON] et al. propose anew solution for their earlier resource allocation problem in a D2D multi-tier HetNet by using an ML algorithm. To maximize system throughput by minimizing the interference of cellular users, an ML-based distributed resource allocation scheme is proposed where D2D users interact with the environment and autonomously select the proper cellular resources. Whereas in [16], the authors propose a resource allocation scheme based on Q-learning where D2D users and small cell users select resource block (RB) cooperatively to meet their QoS requirements. To maximize the overall system throughput for a 2-tier network, the authors formulate a mixed integer non-linear programming resource optimization problem in [17]. Initially, they assign resources to the cellular users using an orthogonal technique, and for D2D users, they adopted a low complexity-based ML algorithm where D2D pairs reuse different sub-carriers to further enhance the system throughput without affecting the CUE's performance. A coalition game-based resource allocation for D2D communication in underlying cellular HetNets with mm-wave communication is proposed in [18]. Here, the authors first formulate a resource allocation scheme for D2D pairs using both mm-wave and cellular spectrum and, based on their characteristics, propose a coalition game theory to maximize the overall sum rate. The joint resource allocation and power control scheme for D2D communication in cellular HetNets is proposed in [19]. For achieving an optimum solution with higher overall system throughput and lower computational complexity, the mixed-integer non-linear optimization problem is subdivided into smaller pieces. In [20], the authors proposed ML and distributed Q-learning decision tree PC algorithms to manage interference in D2D-enabled cellular networks and achieved better throughput and energy efficiency with lower computational complexity. In [21], the authors proposed a hybrid resource allocation scheme where deep neural network (DNN) and heuristic-based PC and resource allocation approaches are used for D2D communication in D2D-enabled underlay cellular networks for two different QoS requirements. To achieve optimal resource allocation, a DRL-based distributed resource matching scheme is proposed in [22].