## 2 Dual-Channel Transformer Model
### Transformer Model Architecture  
As shown in Figure 4, the transformer consists of an encoder and decoder and consists of several network blocks. It includes positional encoding (PE), self-attention (SA), multi-head attention (MHA), feed-forward network (FFN), and residual connection and layer normalization (LN) blocks (Add & Norm), in which SA is the basic block of MHA. Transformer uses MHA to associate global features, FFN to improve model learning ability, and Add & Norm to enhance model fitting ability. 1. **PE:** The main function of the positional encoding is to retain the spatial position information between the input image blocks. The positional encoding of features is  
\[\left\{ \begin{aligned} PE_{(pos,2i)}=\sin\!\left(pos/10000^{2i/d} \right)\\ PE_{(pos,2i+1)}=\cos\!\left(pos/10000^{2i/d}\right)\end{aligned} ,\right. \tag{2}\]  
where \(PE\) is a two-dimensional matrix, the variable \(\sin\) is placed in the even term of the two-dimensional matrix, and the variable \(\cos\) is placed in the odd term of the \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Stage \(i\)** & **Operator** & **Resolution** & **Channels** & **Layers** \\ \hline
1 & Conv3 \(\times\) 3, stride = 2 & 224 \(\times\) 224 & 32 & 1 \\
2 & MBConv1, k3 \(\times\) 3, stride = 1 & \(112\times 112\) & 16 & 1 \\
3 & MBConv6, k3 \(\times\) 3, stride = 2 & \(112\times 112\) & 24 & 2 \\
4 & MBConv6, k5 \(\times\) 5, stride = 2 & \(56\times 56\) & 40 & 2 \\
5 & MBConv6, k3 \(\times\) 3, stride = 2 & \(28\times 28\) & 80 & 3 \\
6 & MBConv6, k5 \(\times\) 5, stride = 1 & \(14\times 14\) & 112 & 3 \\
7 & MBConv6, k5 \(\times\) 5, stride = 2 & \(14\times 14\) & 192 & 4 \\
8 & MBConv6, k3 \(\times\) 3, stride = 1 & \(7\times 7\) & 320 & 1 \\
9 & Conv1 \(\times\) 1 \& Pooling \& FC & \(7\times 7\) & 1280 & 1 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Efficientnet-b0 network parameters. Figure 4: Schematic diagram of the transformer structure. two-dimensional matrix. The variable \(\sin\) and the variable \(\cos\) are used to form the two-dimensional matrix, and \(Z^{l-1}\) is encoded in positional. 2. **SA:** Self-attention is a core component of transformer. It mimics the characteristics of biological observation targets and extracts features of some key areas by focusing attention through a mathematical mechanism. The advantages of the self-attention mechanism lie in distance learning, improved local attention, and parallel computing. As is shown in Figure 5a, the self-attention mechanism is mainly implemented using scaled dot-product attention, \[Attention(Q,K,V)=softmax(\frac{QK^{T}}{\sqrt{d}})V,\] (3) where \(Q\), \(K\), and \(V\) represent the query matrix, key matrix, and value matrix, respectively, which are obtained through the multiplication of feature matrix and three random weight matrices, and \(d\) is the dimension of the input feature. 3. **MHA:** MHA is used to establish different projection information in multiple different projection spaces. As is shown in Figure 5b, the input matrix is projected in different ways, and the output matrix is spliced together. For each projection result, MHA executes SA in parallel; \[head_{i}=Attention(QW_{i}^{Q},KW_{i}^{K},VW_{i}^{V}),\] (4) where \(W_{i}^{Q},W_{i}^{K}\in\mathbb{R}^{d_{\text{model}}\times d_{k}}\), \(W_{i}^{V}\in\mathbb{R}^{d_{\text{model}}\times d_{v}}\), and \(d_{\text{model}}\) represents the length of the input feature. \(d_{k}=d_{v}=d_{\text{model}}/h\), and \(h\) indicates the number of heads. Concatenated the projection calculation results of multiple heads, \[MHA(Q,K,V)=Concat(head_{1},head_{2},\cdots,head_{h})W^{o},\] (5) where \(W^{O}\in\mathbb{R}^{hd_{v}\times d_{\text{model}}}\). Multi-head can extract the features of different heads on a more detailed level. When the total computation amount is the same as that of a single head, the feature extraction effect is better. 4. **FFN:** FFN maps features to the high-dimensional space and then to the low-dimensional space.