Device-to-device (D2D) communication has been recognized as a promising technique to improve spectrum efficiency. However, D2D transmission as an underlay causes severe interference, which imposes a technical challenge to spectrum allocation. Existing centralized schemes require global information, which can cause serious signaling overhead. While existing distributed solution requires frequent information exchange between users and cannot achieve global optimization. In this paper, a distributed spectrum allocation framework based on multi-agent deep reinforcement learning is proposed, named Neighbor-Agent Actor Critic (NAAC). NAAC uses neighbor users' historical information for centralized training but is executed distributedly without that information, which not only has no signal interaction during execution, but also utilizes cooperation between users to further optimize system performance. The simulation results show that the proposed framework can effectively reduce the outage probability of cellular links, improve the sum rate of D2D links and have good convergence.
Reference REF has recourse to the deep reinforcement learning and a distributed spectrum allocation framework is devised, which requires only the neighbor users' information.
A Multi-Agent Deep Reinforcement Learning Based Spectrum Allocation Framework for D2D Communications
2019 IEEE Global Communications Conference (GLOBECOM)
2019 IEEE Global Communications Conference (GLOBECOM)
Computer Science