## 2 Related Work
The simulation findings demonstrated that the suggested strategy highly improved the network coverage, the data rates of users, and the SE of the system. In [20], a multi-agent Q-learning approach is presented to enhance the throughput of D2D communications underlying cellular networks. First, the multi-agent Q-learning technique-based channel resource allocation is implemented. Furthermore, to tackle the issue of slow convergence in the Q-learning algorithm for the D2D communications system, the authors included a Fuzzy C-Means algorithm into a multi-agent Q-learning framework. This integration aimed to enhance the utilization of power management by employing a Q-learning approach. The results showed that the implementation of the multi-agent Q-learning approach enhances the system's throughput. In [21], the authors proposed an auction technique based on a D2D relay selection algorithm underlaying cellular networks. The cooperative willingness of user relay devices was evaluated from a social level. As social relationships become stronger, the willingness to cooperate increases and the transmission power of the relay also increases. Subsequently, they determined a relationship between the outage probability and the transmission power. Next, an auction approach was employed to stimulate relays to enhance the transmitted power via monetary incentives. The results demonstrated the technique's superiority over previous relay methods, as it not only enhanced the system's data rate but also decreased the probability of a communications outage. Additionally, the researchers in [22] suggested a graph coloring approach to address interference issues in D2D communications-based cellular networks. The primary objective is to exploit the weighted prioritization of spectrum resources, allowing several D2D pairs to use the same resources as cellular users inside the network. Once the spectrum allocation has been achieved, the power management process is employed to reduce the transmission power of D2D pairs which leads to minimizing the interferences and enhancing the energy consumption of the entire cell. The simulation outcomes illustrated that the suggested method successfully mitigated co-channel interference, enhanced system throughput, and minimized power consumption in comparison to the traditional techniques. The authors in [23] suggested a method based on the D2D multicast clusters approach along with a Q-Learning-aided approach to tackle the issue of joint sub-channel assignment and power management for D2D communication under cellular networks. An agglomerative hierarchical clustering approach using unsupervised ML was suggested to establish clusters of D2D pairs, considering user preferences and ensuring reliable D2D multicast communications across D2D users. The numerical simulation outcomes indicated that the suggested approach provided substantial benefits for the throughput and EE of the introduced system compared to existing techniques. In [24], the paper examined the issues of joint channel assignment and power management in D2D communications underlaying cellular networks based on the concept of the NOMA technique. The suggested approach divided the issue into two sub-issues: channel assignment and power management. The assignment of channels to the pairs is shown using a matching game theory, providing a stable solution while considering the interferences imposed by users in the network. The power allocation problem is addressed in both phases utilizing difference-of-convex programming that is optimized iteratively by the Frank-Wolfe method. The performance investigations have shown that the suggested method enhanced SE, fairness, and network connectivity. In [25], the authors examined the issue of resource allocation for D2D communications in cellular networks. A one-to-many matching game was implemented to enhance spectrum utilization and tackle frequency interference problems.