# An Intelligent Access Channel Algorithm Based on Distributed Double Q Learning
## 1 Introduction  
With the explosive growth of wireless communication devices, wireless communication presents networked development [1; 2]. Channel access is one of the key problems to be solved in wireless communication networks in order to prevent interference caused by multiple users using the same channel at the same time [3]. In addition, due to the openness of electromagnetic wave propagation space [4], wireless communication networks are susceptible to external malicious jamming [5; 6]; as an important threat to the security of wireless communication, it mainly affects and destroys the detection of useful signals by electromagnetic radiation signals, which seriously degrades the system performance. Existing communication anti-jamming measures are mainly extended spectrum communication technology [7], which can effectively deal with conventional jamming such as multi-tone and blocking. With the application of artificial intelligence [8] to the field of communication jamming, the jamming modes are characterized by diversity, dynamics and intelligence [9; 10]. The conventional extended spectrum communication can no longer ensure the reliability of communication. In the face of a more complex and harsher electromagnetic environment, how to use more \"clever\" algorithms to avoid mutual interference and external malicious jamming, to ensure reliable communication between users, is a wireless communication network problem to be solved urgently. In order to solve the problem of mutual interference between different users, the traditional approach is to formulate access criteria for users through access protocol [11]. However, the current protocols do not consider the existence of external jamming. Considering future more complex wireless network scenarios, an artificial intelligence algorithm is a worthwhile idea. Machine learning is an important kind of artificial intelligence algorithm, which can be divided into supervised learning, unsupervised learning, semi-supervised learning and reinforcement learning according to learning methods [12; 13]. The paper mainly studies how to use reinforcement learning [14] to solve the communication anti-jamming problem. The main idea of reinforcement learning is that in the face of an unknown environment, the agent can take actions, conduct trial and error experiments, interact with the environment, constantlylearn from the environment feedback and update and improve its action strategy, so as to learn the optimal strategy and make the return value of the agent reach the optimal value. Multi-agent reinforcement learning algorithms can be divided into centralized Q-learning [15] and distributed Q-learning [16] according to whether they depend on the central controller. Centralized Q-learning is aimed at cooperative reinforcement learning scenarios with a central controller, which is the learning subject. Distributed Q-learning is aimed at the non-centralized cooperative reinforcement learning scenario, where all communication nodes are learning subjects. Reference [17] proposes a multi-user cooperative anti-jamming algorithm, but the convergence effect is slow, which is difficult to cope with the real-time demand for decision making under the battlefield. Reference [18] designs a Markov game framework and proposes a multi-user joint anti-jamming algorithm. The algorithm can shorten convergence time, but only for a few users. Many users will lead to a dimensional disaster problem, resulting in difficult convergence. Reference [19] considers the influence of distance on mutual disturbance and proposes a multi-agent Q-learning algorithm based on distance metrics. References [17; 18; 19] are all centralized Q-learning. Centralized Q-learning requires centralized learning and distributed execution. It requires the central controller to perform all the operations. When there are many users, the response speed will slow down.