# Intelligent Reception of Frequency Hopping Signals Based on CVDP
## 2 CVDP Network Design
### Network Structure
#### 2.2.2 Transformer Encoder Module  
The structure of the Transformer Encoder module consists of parallel dual Multi-head _Prob sparse_ Self-attention (MPS) layers and the Multi-layer Perceptron block (MLP), which applies Residual Connection (Residual Connection) and Layer Normalization (Layer Norm) before each layer. To learn more about different subtle features, the Transformer Encoder in this paper uses parallel dual multi-headed self-attention modules with different sizes. \(h\) denotes the number of heads in the multi-headed self-attention layer. Let the total dimension of each multi-headed self-attention layer be 256, one multi-head self-attention layer was set to \(h1\), and the other multi-head self-attention layer was set to \(h2\). Specifically, the time-frequency features are extracted in the low dimensional subspaces with the \(256/h1\)-dimension and \(256/h2\)-dimension, respectively, to improve the network performance. The self-attention mechanism is to extract the feature with attention as shown in Equation (3). \[A(\mathbf{Q},\mathbf{K},\mathbf{V})=softmax\bigg{(}\frac{\mathbf{Q}\mathbf{K} ^{T}}{\sqrt{d}}\bigg{)}\mathbf{V}. \tag{3}\]  
where \(\mathbf{K}\), \(\mathbf{Q}\) and \(\mathbf{V}\) are the key matrices, query matrices, and value matrices of the self-attention module, which are obtained from the input sample sequence by spatial transformation, and \(d\) is the dimension of the input. Let the input sample sequence be \(\mathbf{x}=\mathbb{R}^{L\times d}\) and \(L\) be the length of the sequence, then define \(\mathbf{K}=\mathbf{x}\cdot\mathbf{W}^{K}\in\mathbb{R}^{L\times d}\), \(\mathbf{Q}=\mathbf{x}\cdot\mathbf{W}^{Q}\in\mathbb{R}^{L\times d}\), \(\mathbf{V}=\mathbf{x}\cdot\mathbf{W}^{V}\in\mathbb{R}^{L\times d}\), \(\mathbf{W}^{K}\), \(\mathbf{W}^{Q}\) and \(\mathbf{W}^{V}\) are the fully connected layer learnable parameter matrices. Let \(\mathbf{q}_{i}\), \(\mathbf{k}_{i}\), \(\mathbf{v}_{i}\) stand for the \(i\)-th row in \(\mathbf{K}\), \(\mathbf{Q}\), \(\mathbf{V}\) respectively. Since the traditional self-attention mechanism requires dot product operations on all \(\mathbf{K}\) and \(\mathbf{Q}\) values to calculate the correlation, a dual multi-headed self-attention layer will increase the computational amount, so the _Prob sparse_ self-attention idea is introduced to reduce the computational amount. The distribution of self-attention probability has potential sparsity, only part of the dot product contributes to the main attention, and the other dot products can be ignored. The attention weight of the \(i\)-th component \(\mathbf{q}_{i}\) of \(\mathbf{Q}\) to all \(\mathbf{K}\) can be viewed as a probability distribution \(p(\mathbf{k}_{j}|\mathbf{q}_{i})\) shown in Equation (4), where \(c(\mathbf{q}_{i},\mathbf{k}_{j})\) is the correlation between \(\mathbf{q}_{i}\) and \(\mathbf{k}_{j}\) as shown in Equation (5), and let \(\beta(\mathbf{k}_{j}|\mathbf{q}_{i})\) be uniformly distributed as shown in Equation (6). If a \(\mathbf{q}_{i}\) contributes significantly to attention, its \(p(\mathbf{k}_{j}|\mathbf{q}_{i})\) will deviate from the uniform distribution; conversely if it does not contribute to attention, its \(p(\mathbf{k}_{j}|\mathbf{q}_{i})\) will approach the uniform distribution.