# Multi-Granularity Dilated Transformer for Lung Nodule Classification via Local Focus Scheme
## 3 Methods
### Local Focus Scheme
#### 3.1.1 Regularized Channel-Wise Grouped Attention  
Generally, CNNs transform high-level semantic information into presentative feature maps. Each channel of feature maps indicates multiple signals to different regions of CT images, channels that have more high correlations focus on more salient features of lung nodules. Therefore, understanding channel-wise topology is essential to learn fine-grained spatial relations. To fully mine discriminative channel-wise dependencies in the feature maps, as shown in Figure 2, we propose a Regularized Channel-wise Grouped Attention to obtain the fine-grained local feature representation. Specifically, we first split the feature map into \(G\) groups along the channel dimension to prepare for calculating grouped channel-wise attention, since each group channel is indeed corresponding to a part in the pulmonary computed tomography (CT) images. However, due to similar patterns and inherent noise in medical images, it is difficult to learn a precise channel-wise feature distribution. With the goal to learn more well-distributed and discriminative channel-wise relations, we decide to leverage information from each group channel to enhance feature modeling in key regions and obtain the group-wise channel attention by calculating the correlation of global statistical information and high-level semantic features \(\mathbf{\chi}_{in}^{C\times H\times W}\) extracted by the backbone. That is, we divided \(\mathbf{\chi}_{in}\) into \(\{x_{hw}^{1}\), \(x_{hw}^{2}\),..., \(x_{hw}^{g}\}\) along the channel dimension, and the adaptive global average and max pooling are performed on \(\{x_{hw}^{1}\), \(x_{hw}^{2}\),..., \(x_{hw}^{g}\}\), respectively, to obtain the statistical features in the whole CT image \(\mathcal{R}_{hw}\), which can be formulated as:  
\[\mathbf{\chi}^{m}=\sum_{g=1}^{G}(\underset{(h,w)\in\mathcal{R}_{hw}}{Max}x_{hw}^{ g}+\frac{1}{|\mathcal{R}_{hw}|}\sum_{(h,w)\in\mathcal{R}_{hw}}x_{hw}^{g}) \tag{1}\]  
In Equation (1), \(C\) denotes the channel dimension, and \(H\) and \(W\) are height and width of the image, respectively, where \(x_{hw}^{g}\) is the pixel-wise feature of each group channel. As shown in Equation (2), we then leverage the inner product \(M\) to calculate the correlation between the high-level semantic features \(\mathbf{\chi}_{in}\) and statistical features \(\mathbf{\chi}^{m}\). Ideally, the ones that are closer to \(\mathbf{\chi}^{m}\) have larger initial weight to prevent features of the whole space from dominating by unrelated information:  
\[\mathbf{\chi}^{r}=M(\mathbf{\chi}^{m},\mathbf{\chi}_{in}) \tag{2}\]  
Figure 2: Illustration of the proposed Local Focus Scheme. The Local Focus Scheme consists of two key parts: the regularized channel-wise grouped attention and KeyMask.  
Next, we use a group normalization [34] to improve _Internal Covariate Shift_ between multiple grouped channels for accelerating the convergence, and the corresponding mean and variance are denoted in Equation (3):  
\[\begin{split}\mu&=\frac{1}{(C/G)HW}\sum_{c=kC/G}^{(k+1 )C/G}\sum_{i=1}^{H}\sum_{i=1}^{W}x^{cij}\\ \sigma=&\sqrt{\frac{1}{(C/G)HW}\sum_{c=kC/G}^{(k+1 )C/G}\sum_{i=1}^{H}\sum_{i=1}^{W}(x^{cij}-\mu)^{2}}\end{split} \tag{3}\]  
where \(G\) is the number of groups and \(k\in[0,G-1)\), \(x^{cij}\) is the pixel-wise feature of each grouped channel. The normalized feature representation is:  
\[\mathbf{X}^{\prime}=\frac{\mathbf{X}^{\prime}-\mu}{\sigma+\epsilon} \tag{4}\]  
In Equation (4), \(\epsilon\) is the hyperparameter for numerical stability. Finally, we combine the regularized channel-wise attention \(\mathbf{X}^{\prime}\) and the high-level semantic features \(\mathbf{X}_{in}\) to focus on more discriminative local relations:  
\[\mathbf{X}^{\prime}=\delta(W_{1}\mathbf{X}^{\prime})*\mathbf{X}_{in} \tag{5}\]  
In Equation (5), \(\delta\) is the sigmoid function and \({}^{*}\) is the element-wise multiplication. \(W_{1}\) is the weight matrix of a fully-connected layer.