  With the incoming 5G network, the ubiquitous Internet of Things (IoT) devices
can benefit our daily life, such as smart cameras, drones, etc. With the
introduction of the millimeter-wave band and the thriving number of IoT
devices, it is critical to design new dynamic spectrum access (DSA) system to
coordinate the spectrum allocation across massive devices in 5G. In this paper,
we present Hermes, the first decentralized DSA system for massive devices
deployment. Specifically, we propose an efficient multi-agent reinforcement
learning algorithm and introduce a novel shuffle mechanism, addressing the
drawbacks of collision and fairness in existing decentralized systems. We
implement Hermes in 5G network via simulations. Extensive evaluations show that
Hermes significantly reduces collisions and improves fairness compared to the
state-of-the-art decentralized methods. Furthermore, Hermes is able to adapt
the environmental changes within 0.5 seconds, showing its deployment
practicability in dynamic environment of 5G.
