# Visual Tracking with FPN Based on Transformer and Response Map Enhancement
## 2 Related Work  
In recent years, a large number of excellent trackers have proposed many advanced suggestions. The trackers based on correlation filter use multi-level features, template updating and deeper feature extraction to further improve MOSSE and KCF [9; 10; 11; 12; 13; 14], which effectively improve the accuracy and robustness of target tracking.  
Based on SiamFC, SiamRPN adopts anchors for region proposal to make use of the deep feature maps and avoid repeated computation, which has great speed performance. Subsequent studies have improved SiamRPN from the aspects of feature extraction, proportion of positive and negative samples, feature enhancement and so on [15; 16]. However, the anchor-based trackers are very sensitive to relative hyper-parameters such as size, number and aspect ratio of anchor boxes. Moreover, anchor-based trackers possess limited performance in processing objects with significant appearance change due to their fixed hyper-parameters. Inspired by FCOS, SiamCAR [17] uses an anchor-free framework to reduce the number of hyper-parameters and maintain an excellent performance, both in accuracy and efficiency. In this paper, the anchor-free framework is adopted and SiamCAR is used as our baseline algorithm.  
The feature fusion network is a common strategy in visual tracking [18]. To solve the scale variation problem in tracking, the fusion of different deep features are used to exploit complementary spatial and semantic information in SiamFPN [19; 20]. SiamHOG fuses the HOG features and deep features of the image [21]. These feature fusion methods, to some extent, improve the tracker's performance when dealing with the target scale variation and small object tracking. However, limited by the receptive field, the above fusion method simply fuses the information of each location with the surroundings, and fails to make effective use of the global context information.  
Transformer can overcome receptive field limitations and focus on global information [22]. Inspired by Transformer, HIFT combines Transformer with the feature fusion network [23], which directly fuses different levels of the feature map, but the Transformer encoder and decoder structure is still complex and limited by its adoption of the AlexNet feature extraction network [24]; the tracking accuracy and efficiency still need to be improved. Based on DETR [25], our proposed method designs efficient multi-head attention  
Figure 1: This is the schematic diagram of TR-Siam. The \(\ddagger\) stands for cross-correlation operation. Template contains target of interest. Search area is centered on the template position and three times the size of the template in the detection frame.  
layers that are embedded into the feature pyramid through the cross attention between different feature layers to enhance the object features. This then improves its performance when facing challenges such as severe variations in target appearance, similar object and occlusion. In this way, we improve the robustness of the object tracking.  
After the backbone models, we can obtain two feature maps of the target template and search region. Siamese-based trackers perform cross-correlation on the feature map of search region with the feature map of target exemplar as the kernel. In this way, we can obtain the response map and send it to the classification and regression branch for visual tracking. However, for the tracker, the template branch and the search branch are always independent, so the template information in the response map always comes from the ground truth of the first frame. So, the tracking may fail when the appearance of an object changes. In this paper, the response map enhancement module is added to enhance the response map by calculating the statistical information of the response map. Specifically, we obtain the context information of the response map to adjust itself. In this way, the useful information between the target template and search area is strengthened and implicitly updates the template to some extent.