# Object Tracking Algorithm Based on Multi-Layer Feature Fusion and Semantic Enhancement
## 2 Method
### Construction of Object Semantic Enhancement Model
#### (1) Cross-attention learning  
There has always been an inherent correlation between template images and search images, which can be explicitly learned through cross-attention to improve object recognition ability. ResNet50 is used to extract the second and third layer features of template image and search image and obtain template features [Layer2: \(f_{Z_{2}}\in R^{C_{2}\times H_{Z_{2}}\times W_{Z_{2}}}\); Layer3: \(f_{Z_{3}}\in R^{C_{3}\times H_{Z_{3}}\times W_{Z_{3}}}\)] and search features [Layer2: \(f_{X_{2}}\in R^{C_{2}\times H_{X_{2}}\times W_{X_{2}}}\); Layer3: \(f_{X_{3}}\in R^{C_{3}\times H_{X_{3}}\times W_{X_{3}}}\)], where C, H, and W represent the number of channels, height, and width of the feature map, respectively, and \(C_{2}=512\) and \(C_{3}=1024\). Z represents a template image, and X represents a search image. The position of the query Q and key K of the extracted features is encoded, and the specific position of each sequence in the feature vector is marked by sine-cosine coding. The feature mapping of the two layers is expanded on the plane dimension. Firstly, cross-attention is applied to the Layer2 features of the template image and search image to learn the details and shape matching between the template and search images. As shown in Equations (1) and (2), \(X_{2}^{\prime}\) and \(Z_{2}^{\prime}\), respectively, represent the search feature and template feature after Layer2 features are applied with cross-attention; \(P_{X_{2}}\) and \(P_{Z_{2}}\), respectively, represent the position coding of Layer2's search feature and template feature; and _MultiHead_ represents the 'Cross-Attention' part in Figure 3. Layer2 features  
Figure 3: Network framework of attention mechanism fused with multi-layer deep features. are processed through cross-attention, and the feature representation in the search image highlights more detailed features similar to the template image, such as edges, textures, etc. This change enables the model to better match the object and background at the level of detail. When facing clutter from similar objects, it can reduce the model's distraction from non-similar objects and improve tracking accuracy. \[X_{2}^{\prime}=f_{X_{2}}+MultiHead\big{(}f_{X_{2}}+P_{X_{2}},f_{Z_{2}}+P_{Z_{2}},f_{Z_{2}}\big{)} \tag{1}\]  
\[Z_{2}^{\prime}=f_{Z_{2}}+MultiHead(f_{Z_{2}}+P_{Z_{2}},f_{X_{2}}+P_{X_{2}},f_{ X_{2}}) \tag{2}\]  
At the same time, cross-attention is applied to the features at the Layer3 level to enhance the understanding of the high-level semantic information of the object. As shown in Equations (3) and (4), \(X_{3}^{\prime}\) and \(Z_{3}^{\prime}\), respectively, represent the search feature and template feature after Layer3 features are applied with cross-attention; \(P_{X_{3}}\) and \(P_{Z_{3}}\), respectively, represent the position coding of Layer3's search feature and template feature; and _MultiHead_ represents the 'Cross-Attention' part in Figure 3. After applying cross-attention in Layer3, the feature representation of the search image is more focused on the global and advanced semantic information of the object, such as the overall shape and object category features. This focus on high-level semantic variations enhances the model's overall visual grasp of the object, allowing the model to distinguish based on global features rather than just local or low-level features when encountering objects with similar appearances to the object, in order to maintain stable tracking. \[X_{3}^{\prime}=f_{X_{3}}+MultiHead\big{(}f_{X_{3}}+P_{X_{3}},f_{Z_{3}}+P_{Z_{3} },f_{Z_{3}}\big{)} \tag{3}\]  
\[Z_{3}^{\prime}=f_{Z_{3}}+MultiHead\big{(}f_{Z_{3}}+P_{Z_{3}},f_{X_{3}}+P_{X_{3} },f_{X_{3}}\big{)} \tag{4}\]  
To integrate attention information across multiple levels and boost the model's capacity to represent features, the quantity of stacked cross-attention modules in the model is aligned with that in TransT, specifically set at N = 4.