## 3 Proposed Method
### Frequency Domain Similarity Gated Selection Vision Transformer
#### 3.1.1 Fab  
We start with an input feature \(X\), which undergoes a three-branch process involving \(1\times 1\) convolutions and \(3\times 3\) Depthwise Separable (DW) convolutions to obtain \(Q\), \(K\), and \(V\). Within the self-attention mechanism, our initial step involves applying Discrete Cosine Transform (DCT) to \(Q\) and \(K\), thereby transforming them into the frequency domain. This transformation results in frequency representations, namely \(Q_{det}\) and \(K_{det}\).  
\[K_{det}/Q_{det}=D\{ConV_{3\times 3}[ConV_{1\times 1}(X)]\} \tag{3}\]  
where \(D\) represents Discrete Cosine Transform (DCT) in the frequency domain conversion.  
Next, we compute the correlation scores between \(Q_{det}\) and \(K_{det}\). Let us assume we use a matrix \(M\) to represent the correlation matrix, where \(M(i,j)\) indicates the correlation score between query position \(i\) and key position \(j\). We achieve this by element-wise multiplication of these correlation scores.  
\[M=K_{det}\otimes\overline{Q_{det}} \tag{4}\]  
Next, we utilize the Gather function to collect or extract the relevant correlation scores from the frequency-domain representation \(M\) obtained after the DCT transformation. Firstly, we need to set a threshold value \(T\) for filtering attention weights. The threshold value \(T\) can be a fixed number or a parameter dynamically adjusted based on the characteristics of tasks and data. With the threshold value \(T\), we compute an index array \(ijdx\), which contains the indices of all positions exceeding the threshold \(T\). The specific calculation is as follows:  
\[ijdx=[i,j\text{ for }i,j\text{ in range}(N)\text{ if }M[i,j]>T] \tag{5}\]  
For each index \(ijdx\), the Gather function selects the element at position \([i,j]\) from the input tensor \(M\) and keeps it unchanged. For the elements that were not selected, a Gather function is used to replace their probabilities with 0 at the given indices. Additionally, a  
Figure 2: The overall architecture of the proposed FSGS-ViT in this paper is depicted. It primarily comprises the FAB and the GSB. The detailed structure of the ConV Block within GSB is illustrated adjacent to the overall architecture.  
learnable parameter matrix is introduced to appropriately process or adjust the collected correlation scores \(M_{gather}\).  
\[M_{gather}[i,j]=\left\{\begin{array}{cc}M[i,j]&M[i,j]>T\\ 0&otherwise\end{array}\right. \tag{6}\]  
\[M_{adjusted}=M_{gather}\otimes W_{randn} \tag{7}\]  
where \(W\) represents the learnable parameter matrix.  
Utilizing the adjusted correlation scores, \(M_{gather}\), the final attention weights \(A\) are computed by applying the inverse DCT and the softmax function.  
\[A=\sigma\Big{[}D^{-1}\Big{(}M_{adjusted}\Big{)}\Big{]} \tag{8}\]  
where \(\sigma\) represents the softmax function, and \(D^{-1}\) represents the inverse DCT frequency domain transformation. Normalizing the correlation scores into attention weights ensures they lie within the range of \((0,1)\) and sum up to \(1\). Finally, applying the attention weights \(A\) to \(V\) yields the ultimate self-attention output. By implementing correlation computation and adaptive adjustment of attention weights, the model's perception of crucial features and details within the image is enhanced, thereby boosting the performance and effectiveness of image processing tasks. The overall output expression of FAB can be represented as:  
\[\begin{array}{ll}FAB\_OUT=&MLP(LN(X+(Conv_{1\times 1}(V\otimes\\ &\sigma(D^{-1}(W_{randn}(Gather(D(Q)\otimes\\ &D(K)))))))))))))\end{array} \tag{9}\]  
where \(LN\) represents Layer Normalization.