# An Intelligent Access Channel Algorithm Based on Distributed Double Q Learning
## 4 Simulation Results and Discussion
### Analysis of Simulation  
In this paper, simulation verification is carried out on the MATLAB platform. In order to highlight the superiority of the proposed algorithm, we compare it with the other two algorithms. * Channel selection algorithm: based on Sense and random selection. Before selecting a channel, the user can sense whether there is interference in the channel. If no interference is detected in the current transmission channel, the user still transmits in the same channel; otherwise, the user randomly switches to other channels in the next action slot. Here, no information is exchanged between users. \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameters** & **Value** \\ \hline Communication users \(N\) & 10–150 \\ Available channels \(M\) & 5 \\ Subframes \(K\) & 50 \\ Slots \(B\) & 100 \\ Packet to be transmitted \(L\) & 100 \\ Number of blocked channels \(Z\) & 3 \\ Learning \(\alpha\) & 0.9 \\ Discount factor \(\gamma\) & 0 \\ Greedy index \(\varepsilon\) & \(1/\sqrt{i}\) \\ Number of iterations \(T\) & 300 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Settings of Model related parameter. Figure 2: (a) Linear frequency sweep jamming. (b) Multi-channel intelligent blocking jamming. * Binary independent Q learning algorithm (ILQ): each user executes the Q-learning algorithm independently only through the local learning results to make transmission decisions and never considers the decision of other users. For multi-channel intelligent blocking jamming, we consider the impact of the number of users in the network on the normalized throughput of the network. We calculate the maximum access load \(\rho{=}(M-Z)*K=100\) that can be borne by a frame. The maximum number of users can be allowed to access in a frame. We control the number of users within the range of 10-150, as shown in Figure 3. It can be seen that within the maximum access load of the system, the normalized throughput increases with the increase in the number of users. When the maximum access load exceeds; that is, the number of users exceeds 100, the normalized throughput decreases significantly, because the number of users increases. The interference between users due to the frequency conflict leads to the decrease in the overall network throughput. To investigate the effect of learning factors \(a\) on the convergence effect, we set the learning factors of the proposed algorithm DDQL and the independent Q-learning algorithm IQL to \(\alpha=0.2\) and \(\alpha=0.1\).As shown in Figure 4, in both algorithm DDQL and algorithm IQL, the learning factor \(\alpha=0.2\) converges faster than \(\alpha=0.1\). For the proposed algorithm DDQL, \(\alpha=0.2\), it converges after about 50 iterations, and for \(\alpha=0.1\), it converges after about 80 iterations. The independent Q-learning algorithm IQL, for \(\alpha=0.2\), converges after approximately 170 iterations, and for \(\alpha=0.1\), the algorithm converges after approximately 200 iterations. We verify the convergence of the proposed algorithm through simulation. Figure 5 shows that the normalized throughput of the proposed DDQL can converge to the optimal  
Figure 4: Relationship between normalized throughput and learning factor. Figure 3: Normalized throughput versus number of users. value of 1 at the 50 th iteration, which indicates that users can still communicate normally in the environment of multi-channel intelligent blocking jamming. Independent of binary Q-learning (IQL) at about 160 times convergence, convergence to 0.9, compared with the proposed DDQL gap in the convergence and reliability of the algorithm, occurs mainly because the classical Q-learning algorithm can interact with the environment, and trial and error to effectively avoid multi-channel jamming, but because each user is an independent subject, it does not interact with other users, so there is no way to avoid interference between users; the algorithm is based on perception and random channel effect is worse.