## 4 Proposed Intelligent Identification Method
### Summary of the Method  
This subsection summarizes the proposed physical layer identity authentication (DQN-PIA) method based on the deep Q-network in Algorithm 1. Among them, \(\varepsilon,\varepsilon_{decay}\), and \(\varepsilon_{\min}\) represent the initial value, attenuation value, and minimum value of the \(\varepsilon\)-greedy algorithm, respectively. The Equation (13) in Step 12 of Algorithm 1 is similar to Equation (8), and it can be represented as follows:  
\[L\big{(}\mathbf{\theta},\mathbf{\theta}^{\prime}\big{)}=\mathrm{E}\Big{[}\big{(}r(s,a )+\theta\mathrm{max}_{a+1}Q\big{(}s+1,a+1\mid\mathbf{\theta}^{\prime}\big{)}-Q(s,a \mid\mathbf{\theta})\big{)}^{2}\Big{]}, \tag{13}\]  
where \(\mathbf{\theta}\) is the weight of the target network. It is worth emphasizing that the method stores the learning experience of the network by using the history playback library \(\mathbf{\Gamma}\), and it obtains small batches of data by referring to a certain priority for network training so as to avoid the dilemma of overfitting in the training process. For the training of the network, the method also sets an activation threshold: that is, when the threshold for the number of actions is reached, it can avoid frequent learning operations and allow the intelligent agent to obtain a sufficient exploration experience. Meanwhile, the method sets the frequency threshold \(\phi\) for the target network to update the weights of the main network, and it does this mainly to prevent the main network from entering the overfitting state during the training process, thus ensuring the performance of the method and increasing the convergence speed of the training process.  
```
1:Input: State space \(\mathbf{S}\), action space \(\mathbf{A}\), discount rate \(\vartheta\), learning rate \(\xi\).
2:Output: Weight matrix of the main DNN \(\mathbf{\theta}\), terminal category determination (i.e., model actions).
3:Initialize:\(\mathbf{\theta}=\mathbf{\theta}^{\prime}\), replay memory library \(\mathbf{\Gamma}\), \(\varepsilon,\varepsilon_{\min},\varepsilon_{\mathrm{decay}}\), threshold for loop execution \(I\), threshold for model training \(L_{t}\), update threshold of the target DNN \(F\), and \(w=0\).
4:While\(\varepsilon>\varepsilon_{\min}\)Do
5:\(\varepsilon\gets\varepsilon\times\varepsilon_{\mathrm{decay}}\)
6:For\(i\gets 1,2,\ldots,I\)Do
7: Generate a random value among 0 to 1, and perform Equation (10);
8: Choose action \(a\), obtain the reward, and store quad \((s,a,r_{w},s^{\prime})\) into \(\mathbf{\Gamma}\);
9:If\(i>L_{t}\)Do
10:\(w=w+1\);
11: Randomly select a batch of experience samples from \(\mathbf{\Gamma}\);
12: Obtain \(L(\mathbf{\theta},\mathbf{\theta}^{\prime})\) through calculating Equation (13);
13:EndIf
14:If\(w\)mod\(F\)=0 Do
15:\(w=w+1\);
16:\(s\gets s^{\prime},\mathbf{\theta}^{\prime}\leftarrow\mathbf{\theta}\);
17:EndIf
18:EndFor
19:EndWhile
```  
**Algorithm 1**The proposed DQN-PIA method  
The proposed DQN-PIA method in this paper is consistent with typical deep learning methods, and both are offline decision methods that require pre-model training. The characteristic of this type of method is that it relies on a large amount of sample data for model parameter calculation during the training process, while, in testing or application, it only involves simple forward linear and nonlinear mapping processing. It can be considered that the complexity of the model after deployment is relatively low. It should be added that the method proposed in this paper significantly reduced the volume of space and reduced the complexity of model training and deployment by over 80% in the process of state setting. In order to obtain the decision model of the method proposed in this paper, the gateway device can generally be required to train or update the parameters of the model through offline or online parallel methods, which is combined with its perceptual recorded historical channel state sample data, without affecting the normal working process.