# Research on Coal and Gangue Recognition Model Based on CAM-Hardswish with EfficientNetV2
## 2 Coal and Gangue Recognition Network Model
### EffcientNetV2 Network Based on CAM-Hardswish  
The role of the attention mechanism module is to allow the model to learn which parts of the content are more important by adding weights to allow the model to learn the more important content, amplifying the important features and improving the accuracy of the model. The SE attention mechanism module is used in both EfficientNetV1 and V2. In coal and gangue recognition research, solely relying on a single pooling layer can result in the loss of essential information. To improve the extraction of critical information from coal and gangue images by EfficientNetV2, we devised a new block structure comprising EfficientNetV2-S as the basic feature network backbone and utilizing the CAM attention mechanism module. This module differs from the SE module by introducing a parallel pooling layer, allowing for a more comprehensive and richer extraction of high-level features. Compared with a single pooling operation, the parallel pooling of AvgPool and MaxPool can minimize information loss and improve efficiency. The CAM attention mechanism is similar to the SE attention mechanism, and both are classified as channel attention mechanisms. The structure of the CAM module is shown in Figure 4. The CAM module consists of two parallel pooling layers (max pooling and global average pooling) and a common MLP module.  
The feature map matrix input to the module first passes through two parallel pooling layers (the max pooling layer and average pooling layer). The pooling operation does not change the number of channels of the feature map, but only its size, changing the feature map matrix from C \(\times\) H \(\times\) W to a size of C \(\times\) 1 \(\times\) 1, where C, H, and W are the number of channels, height, and width of the feature map, respectively. It then passes through the shared MLP module, where it first compresses the number of channels to a multiple of 1/R (rate of reduction) and then expands to the original number of channels, and after this, the ReLu activation function obtains the two activated results. Finally, the two output results are added together, element by element, through a sigmoid activation function to obtain the final output result, and then multiplied by the original image to return to the C*H*W size.  
Figure 4: CAM structure.  
Figure 3: Fused-MBConv structure.  
Figure 2: SE structure.  
After replacing the SE module in the EfficientNetV2 structure with the CAM module, a new MBConv module and a Fused-MBConv module are formed. The module structure is shown in Figure 5.  
The activation function in the original CAM attention mechanism module is the ReLu activation function. The advantage of this activation function is that it can improve the training effect of the neural network and has a faster convergence speed. However, the ReLu activation function has the problem of dead neurons and disappearing gradients. The input value is too small (less than zero), causing the neuron to die and stop updating the parameters. If the input value is too large, the gradient disappears and the neural network is ineffective. As a result, the neural network does not learn useful features effectively, which has an impact on the performance and accuracy of the model. To further improve the performance of the neural network, the Hardswish activation function is introduced, which has the advantages of good numerical stability and fast computation speed. The Hardswish activation function combines the ReLu6 and H-sigmoid activation functions to provide faster calculation speed, solve complex calculation and derivation problems, compensate for the shortcomings of the ReLu activation function, and improve the effectiveness of neural network training. The function is expressed as follows:  
\[h-swish[x]=x\frac{\text{ReLU6}(x+3)}{6} \tag{1}\]  
where \(x\) represents the input pixel value.  
Figure 5: Modified MBConv (**a**) and Fused-MBConv (**b**) modules.