# Coal-Rock Data Recognition Method Based on Spectral Dimension Transform and CBAM-VIT
## 2 Materials and Methods
### Cbam-Vit
#### 2.2.1 Channel Attention Mechanism  
During the process of determining whether a particular pixel belongs to coal or gangue, it is observed that each channel of the feature map carries different levels of importance. Utilizing the feature information from each channel equally may lead to the inefficient use of computing resources. To address this issue, the channel attention mechanism is introduced to capture the interrelationship between channels in the feature map. This allows for differential treatment of the feature information from different channels.  
In Figure 3, the input feature map is represented as \(\text{F}\in\text{R}\subset\text{R}\times\text{H}\times\text{W}\), where H and W denote the height and width of the input feature map, respectively, and C represents the number of channels in the input feature map. The feature map captures global spatial information by applying both global maximum pooling (Max Pool) and global average pooling (Avg Pool) operations, resulting in two feature maps with dimensions of \(\text{C}\times 1\times 1\). To fully utilize the compressed feature information, these feature maps are subsequently fed into a multilayer perceptron (MLP) to obtain two one-dimensional feature maps. The MLP consists of two fully connected layers activated by the ReLU function. By summing the two one-dimensional feature maps along the channel dimension and normalizing the result using the Sigmoid function, the output of the channel attention mechanism is obtained. This output is then multiplied element-wise with the original image to restore its size to \(\text{C}\times\text{H}\times\text{W}\). The entire process can be mathematically expressed by Formula (6):  
\[\begin{array}{ll}\text{M}_{\text{c}}(\text{F})&=\sigma(\text{MLP}(\text{Avgpool }(\text{F}))+\text{MLP}(\text{Maxpool}(\text{F})))\\ &=\sigma\Big{(}\text{W}_{1}\Big{(}\text{W}_{0}\Big{(}\text{F}_{\text{cvg}}^{ \text{c}}\ \Big{)}\Big{)}+\text{W}_{1}\Big{(}\text{W}_{0}\Big{(}\text{F}_{\text{avg}}^{ \text{c}}\ \Big{)}\Big{)}\Big{)}\end{array} \tag{6}\]  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Layer** & **Output Volume** & **Description** \\ \hline \multirow{2}{*}{Dropout-6} & \multicolumn{2}{c}{Encoder Block 1} & \\  & \([-1,149,768]\) & Gaussian dropout: 0.3 \\   LayerNorm-7 & \([-1,149,768]\) & Layer normalization \\   Linear-8 & \([-1,149,2304]\) & 1:3 fully connection \\   Dropout-9 & \([-1,12,149,149]\) & Gaussian dropout: 0.3 \\   Linear-10 & \([-1,149,768]\) & 3:1 fully connection \\   Dropout-11 & \([-1,149,768]\) & Gaussian dropout: 0.3 \\   Attention-12 & \([-1,149,768]\) & Multi-attention \\   LayerNorm-13 & \([-1,149,768]\) & Layer normalization \\   Linear-14 & \([-1,149,3072]\) & 1:4 fully connection \\   GELU-15 & \([-1,149,3072]\) & Activation \\   Dropout-16 & \([-1,149,3072]\) & Gaussian dropout: 0.3 \\   Linear-17 & \([-1,149,768]\) & 4:1 fully connection \\   Dropout-18 & \([-1,149,768]\) & Gaussian dropout: 0.3 \\   MLP-19 & \([-1,149,768]\) & Fully connection. Activation: GELU. \\  & \multicolumn{2}{c}{Encoder Block 2} & \\  & \multicolumn{2}{c}{\(\vdots\)} & \multicolumn{2}{c}{\(\vdots\)} \\   Linear-120 & \([-1,2]\) & 1538 \\ \hline \hline \end{tabular}
\end{table}
Table 1: _Cont._  
Figure 3: Channel attention module diagram.