# Small-Scale Foreign Object Debris Detection Using Deep Learning and Dual Light Modes
## 2 Methods
### Improvement Method
#### 2.3.2 Attention Module  
The attention mechanism is widely used in computer vision tasks, with its efficacy in enhancing model performance well established. Currently, the mainstream attention mechanisms can be divided into channel, spatial, and self-attention. The purpose of channel attention is to capture the relationship between different channels (feature maps). The weight of each feature channel is automatically obtained through network learning. Then, different weight coefficients are assigned to each channel to reinforce important features and suppress non-important ones. Channel attention mechanisms, such as SE (Squeeze and Excitation) [30], have been shown to be effective in lightweight network research. Spatial attention is intended to enhance the expression of features in critical areas. Its essence is to transform the spatial information in the original picture to another space and retain the critical information through the spatial conversion module. It generates a weight mask for each position and outputs it with weighting, enhancing the specific target area of interest while weakening irrelevant background areas. Attention mechanisms can help models focus on important features better, improving model performance. The CBAM (Convolutional Block Attention Module) proposed by [PERSON] et al. [31] combines attention mechanisms from two dimensions: feature channel and feature space. This paper proposes a multi-attention module to enhance the feature extraction of small-scale targets, reduce background interference, and overcome the lack of visible light images in dim environments. This module was added to the backbone to improve the detection performance of the model in complex environments. Figure 4 shows the structure of the module. It consists of two channel attention modules and one spatial attention module. The features extracted from the previous layer are fed into the first channel attention module, which outputs a channel-wise weighted feature map. The spatial attention module then takes this feature map as input and outputs a spatially weighted feature map. The second channel attention module further refines the feature map by applying another channel-wise weighting. The final output is the refined feature map. Figure 5 illustrates the detailed structure of each sub-module. Figure 3: Network structure. The loss of information is first reduced by using parallel pooling, which consists of a maximum pooling and an average pooling to represent the maximum pooled features and average pooled features, respectively. Maximum pooling is a technique that selects the maximum value within a local region of a feature map. It can preserve the texture features, which are the most important information for image recognition. The maximum pooling unit is sensitive to the local maximum value, which means it can enhance the contrast and reduce the noise in the feature map. The input mixed feature map \(F\in R^{C\times H\times W}\) includes visible and infrared feature maps, and different descriptors, \(F_{MaxPool}\in R^{C\times 1\times 1}\) and \(F_{AvgPool}\in R^{C\times 1\times 1}\), are generated using maximum pooling and average pooling. Then, these two descriptors are fed into a multi-layer perceptron (MLP) for learning, concatenating the output results of the MLP. Then, the sigmoid function is used for mapping to obtain the channel attention map \(C_{C1}\in R^{C\times 1\times 1}\), \(C_{C1}\) and \(F\) is input to obtain the output feature map \(F^{\prime}\in R^{C\times H\times W}\) through replication. The calculation of the channel attention module is shown in Equation (1). \[ \begin{array}{c}C_{C1}(F)=\mathit{Sig}(MLP(MaxPool(F))+MLP(AvgPool(F)))\\ F^{\prime}=C_{C1}\otimes F\end{array}  \tag{1}\]  
The location information of the target cannot be ignored, so we use a spatial attention module right after the first channel attention module. Firstly, the output of the first channel attention module is taken as the input of the spatial attention module.