## 2 Materials and Methods
### Deep Learning Networks
#### 2.3.4 Attentional Mechanisms  
When we process an image, we want the convolutional neural network to pay attention to the areas of the image that can positively affect the predicted outcome, rather than paying attention to everything. The attention mechanism allows the convolutional neural network to adaptively adjust what to pay attention to. Attention mechanisms have been used in computer vision and natural language processing. They have been widely used in sequential models with recurrent neural networks and long short-term memory (LSTM) [34]. Among the attention mechanisms, they are generally divided into spatial attention mechanisms and channel attention mechanisms. This paper used a combination of both CBAM models. The CBAM module is a simple and effective attention module for feedforward convolutional neural networks [35], and the structure of the model is shown in Figure 4. \begin{table}
\begin{tabular}{c c c c} \hline
**Network Layer** & **Kernel Size** & **Stride** & **Output** \\ \hline Input & - & - & (a,3,224,224) \\ Conv2d & 7 & 2 & (a,64,112,112) \\ Dense Block (1) & - & - & (a,256,112,112) \\ Transition (1) & - & - & (a,128,56,56) \\ Dense Block (2) & - & - & (a,512,56,56) \\ Transition (2) & - & - & (a,256,28,28) \\ Dense Block (3) & - & - & (a,1024,28,28) \\ Transition (3) & - & - & (a,512,14,14) \\ Dense Block (4) & - & - & (a,1024,14,14) \\ AvgPool & 1 & 1 & (a,1024,11) \\ Classification & - & - & (1024,class\_num) \\ \hline \end{tabular}  
* \({}^{a}\) represents the number of 3D tensors.  
\end{table} 
Table 2: DenseNet-121 network structure. Figure 3: Structure of DenseNet for achieving maximum information flow during image recognition. Firstly, an intermediate feature map of the process is provided; then, different attention weights are assigned sequentially along two different dimensions of the spatial attention module and the channel attention module. Then, the original feature map is multiplied by the inferred attention weights to obtain the adaptive adjustment. The two sub-modules of the CBAM module, the spatial attention module and the channel attention module are shown in Figure 5.  
The Channel Attention Mechanism Module  
The channel attention mechanism first takes the input feature map \(F\in\mathbb{R}^{C\times H\times W}\) and passes it through global maximum pooling and global average pooling to obtain two \((C\times 1\times 1)\) feature maps \(M_{S}\in\mathbb{R}^{C\times 1\times 1}\). Then, the resulting feature maps pass through a shared network consisting of a two-layer multilayer perceptron (MLP) with the number of neurons in the first hidden activation layer \(C/r\), where \(r\) is the shrinkage rate, and the number of neurons in the second layer is \(C\). Afterwards, the output features of the shared network are summed element-wise, and the final complete feature vector is output by the channel attention mechanism \(M_{C}(F)\). The formula of the channel attention mechanism is shown in Equation (1). \[ \begin{split} M_{C}(F)&=\sigma(MLP(AvgPool(F))+MLP( MaxPool(F)))\\ &=\sigma\Big{(}W_{1}\Big{(}W_{0}\Big{(}F_{avg}^{C}\Big{)}+W_{1} \big{(}W_{0}\big{(}F_{\max}^{C}\big{)}\big{)}\Big{)}\end{split}  \tag{1}\]  
where \(\sigma\) denotes the sigmoid function; \(W_{0}\in\mathbb{R}^{C/r\times C}\), \(W_{1}\in\mathbb{R}^{C/r\times C}\), and the MLP weights \(W_{0}\) and \(W_{1}\) are shared for both inputs; and \(W_{0}\) is after the ReLU activation function. Figure 4: Structure of the CBAM module for implementing both spatial and channel attention mechanisms in feedforward convolutional neural networks. Figure 5: Structure of the spatial attention module and the channel attention module in the CBAM module for adaptive adjustment of feature maps. The spatial attention mechanism first takes the feature vectors obtained from the previous channel attention mechanism module as the input feature vectors for this module.