# Intelligent Reception of Frequency Hopping Signals Based on CVDP
## 2 CVDP Network Design
### Network Structure
#### 2.2.2 Transformer Encoder Module
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Index** & **Layers** & **Structure** &
\begin{tabular}{c} **Structural Parameters** \\ **(Kernel Size, Stride)** \\ \end{tabular} & **Output Dimension** \\ \hline
1 & Input & - & - & \(3\times 256\times 128\) \\ \hline
2 & Conv1 & Conv-BN-ReLU & \(7\times 7\times 64.2\) & \(64\times 128\times 64\) \\ \hline
3 & M & Maxpool & \(3\times 3\times 64.2\) & \(64\times 64\times 32\) \\ \hline
4 & Res1 & Conv-BN-ReLU & \(3\times 3\times 64.0\) & \(64\times 64\times 32\) \\ \hline
5 & Res2 & Conv-BN-ReLU & \(3\times 3\times 128.0\) & \(128\times 32\times 16\) \\ \hline
6 & Res3 & Conv-BN-ReLU & \(3\times 3\times 256.0\) & \(256\times 16\times 8\) \\ \hline \hline \end{tabular} Note: Conv stands for Convolutional layer (Conv); BN stands for Batch Normalization (BN); ReLU stands for Activation function ReLU; Res stands for Residual blocks.  
\end{table} 
Table 1: CNN network structure. \[p(\mathbf{k}_{j}|\mathbf{q}_{i})=\frac{c(\mathbf{q}_{i},\mathbf{k}_{j})}{\sum_{l}c (\mathbf{q}_{i},\mathbf{k}_{l})}\mathbf{V}, \tag{4}\]  
\[c(\mathbf{q}_{i},\mathbf{k}_{j})=\exp(\frac{\mathbf{q}_{i}\mathbf{k}_{j}^{T}}{ \sqrt{d}}), \tag{5}\]  
\[\beta(\mathbf{k}_{j}|\mathbf{q}_{i})=\frac{1}{L}. \tag{6}\]  
The difference between the \(p(\mathbf{k}_{j}|\mathbf{q}_{i})\) and the uniform distribution is measured using the Kullback-Leibler divergence, as shown in Equation (7). \[ \begin{split}& KL(\beta||p)=-\sum_{j=1}^{L}\beta(\mathbf{k}_{j}| \mathbf{q}_{j})\ln\left(\frac{p(\mathbf{k}_{j}|\mathbf{q}_{j})}{\beta(\mathbf{ k}_{j}|\mathbf{q}_{i})}\right)\\ &=\sum_{j=1}^{L}\beta(\mathbf{k}_{j}|\mathbf{q}_{i})\ln(\beta( \mathbf{k}_{j}|\mathbf{q}_{i}))-\sum_{j=1}^{L}\beta(\mathbf{k}_{j}|\mathbf{q }_{i})\ln(p(\mathbf{k}_{j}|\mathbf{q}_{i}))\end{split} . \tag{7}\]  
Substituting Equation (4) and Equation (6), we get  
\[KL(\beta||p)=\ln\sum_{l=1}^{L}\exp(\frac{\mathbf{q}_{i}\mathbf{k}_{l}^{T}}{ \sqrt{d}})-\frac{1}{L}\sum_{j=1}^{L}\frac{\mathbf{q}_{i}\mathbf{k}_{j}^{T}}{ \sqrt{d}}-\ln L. \tag{8}\]  
After removing the constant term \(\ln L\), the sparsity measure is obtained, as shown in Equation (9). \[M(\mathbf{q}_{i},\mathbf{K})=\ln\sum_{j=1}^{L}\exp(\frac{\mathbf{q}_{i} \mathbf{k}_{j}^{T}}{\sqrt{d}})-\frac{1}{L}\sum_{j=1}^{L}\frac{\mathbf{q}_{i} \mathbf{k}_{j}^{T}}{\sqrt{d}}. \tag{9}\]  
where the first term is an LSE (Log-Sum-Exp) calculation and the second term is an arithmetic mean. However, in calculation \(M(\mathbf{q}_{i},\mathbf{K})\), it is still necessary to calculate the dot product of each \(\mathbf{q}_{i}\) for all \(\mathbf{k}_{j}\), whose computation complexity is \(O\left(L^{2}\right)\), and the LSE calculation suffers from numerical instability, so the approximation calculation method of the sparsity measure as shown in Equation (10) is used, and \(\hat{M}(\mathbf{q}_{i},\mathbf{K})\geqslant M(\mathbf{q}_{i},\mathbf{K})\) is proved in the [29]. \[\hat{M}(\mathbf{q}_{i},\mathbf{K})=\textit{max}\left\{\frac{\mathbf{q}_{i} \mathbf{k}_{j}^{T}}{\sqrt{d}}\right\}-\frac{1}{L}\sum_{j=1}^{L}\frac{\mathbf{ q}_{i}\mathbf{k}_{j}^{T}}{\sqrt{d}}. \tag{10}\]  
With the above equations, the original LSE operation is replaced by the maximum operation, which makes the value more stable. In the calculation of the \(\hat{M}(\mathbf{q}_{i},\mathbf{K})\), there are only sample dot-product pairs with \(U=L\text{ln}L\), other pairs filling with zero. The final _Probsparse_ self-attention is calculated as shown in Equation (11). \[A(\mathbf{Q},\mathbf{K},\mathbf{V})=softmax\left(\frac{\hat{\mathbf{Q}} \mathbf{K}^{T}}{\sqrt{d}}\right)\mathbf{V}. \tag{11}\]  
where \(\hat{\mathbf{Q}}\in\mathbb{R}^{u\times d}\) is a sparse matrix and it only contains the Top-\(u\)\(\mathbf{q}\) under the sparsity measurement \(\hat{M}(\mathbf{q},\mathbf{K})\), \(u=\text{cln}L\), \(c\) is the sampling coefficient.