# Research on Rapid Recognition of Moving Small Targets by Robotic Arms Based on Attention Mechanisms
## 2 Related Work
### Introduction to the Attention Mechanism
[29] presented a model based on dual attention mechanisms. The self-attention mechanism effectively extracted dependencies between local and global features in images, while the channel attention mechanism automatically determined the importance of each feature channel, leading to more efficient resource allocation. Experimental results demonstrated that this method outperformed others on the CelebA dataset, producing higher-quality images. [PERSON] et al. [30] proposed a novel ASPP structure with global attention information to provide distant detailed information more effectively for semantic segmentation models. Subsequently, a new SAM (selective attention module) was introduced in the decoder stage to provide different attention weight information for different spatial positions. Results demonstrated that both the ASPP with global attention and the decoder with a selective attention mechanism significantly improved accuracy. The proposed attention mechanisms mainly exist in three types: channel attention, spatial attention, and those incorporating both [31]. The squeeze-and-excitation (SE) model serves as a representative of channel attention, explicitly modeling interdependencies across different channels to extract channel-wise attention. The convolutional block at tention module (CBAM) represents a simplistic yet effective feed-forward convolution neural network attention module [32]. However, CBAM can only extract local positional attention information, and lacks the ability to extract long-distance relationships. The coordinate attention (CA) mechanism is a new, efficient attention mechanism that takes both the channel and direction-related position information into account. The CA attention mechanism can provide significant benefits for downstream tasks based on lightweight networks. It allows attention blocks to capture long-distance relationships in one direction while retaining spatial information in the other. In this way, the positional information can be preserved in the generated attention maps which focus on areas of interest, helping the network to more effectively and accurately localize the targets [33]. The computational formula for the attention mechanism is as follows [34]:  
\[Attention(Q,K,V)=Softmax\left(\frac{QK^{T}}{\sqrt{d_{k}}}\right)V \tag{1}\]  
In Formula (1), \(Q\) represents Query, \(K\) represents Key, and \(V\) represents Value. \(Q\), \(K\), and \(V\) are linear transformations derived from the same input matrix \(X\)[35]. The attention function transforms each query into a weighted sum of values, where the weight is calculated by normalizing the dot product between queries and their corresponding keys. The fundamental computational formulas for \(Q\), \(K\), and \(V\) are as follows:  
\[Q=XW^{Q}K=XW^{K}V=XW^{V} \tag{2}\]  
In the above, \(W^{Q}\), \(W^{K}\), and \(W^{V}\) represent three trainable parameter matrices. The input matrix \(X\), when multiplied with \(W^{Q}\), \(W^{K}\), and \(W^{V}\), generates \(Q\), \(K\), and \(V\), effectively undergoing a linear transformation [36]. The attention mechanism does not directly use \(X\), but rather the three matrices generated through matrix multiplication, because utilizing three trainable parameter matrices can enhance the model's fitting capacity. \(Q\) and \(K^{T}\) undergo a matrix multiplication (MatMul) operation to yield a similarity matrix. Each element of the similarity matrix is then divided by \(\sqrt{d_{k}}\), where \(d\) is the size of \(K\)'s dimension. This ensures a more stable gradient update during training. Following normalization via the Softmax function, each value is a weight coefficient greater than 0 and less than 1, with their sums equal to 0. This result can be understood as a weight matrix. Finally, the weight matrix just obtained is multiplied with \(V\) to calculate the weighted sum. The calculation process is illustrated in Figure 3.