# Target-Aware Feature Bottleneck for Real-Time Visual Tracking
## 2 Related Work  
There has been an ongoing trend in the visual tracking literature to formulate the visual tracking task as a tracking-by-detection problem [30], which is to localize the target object inside a designated search area by performing a binary classification task. Trackers are formulated as classifier models where the model can classify the region of the target as a positive class and the region outside the target area as a negative class. At test time, the region score with the largest positive value is chosen as the predicted output region, where additional bounding box regression and refinement is possible as in the two-stage object detection literature [7]. **CNN-based Trackers:** Along with powerful feature representations brought by CNNs trained from classification tasks [6; 31; 32], visual tracking algorithms also make use of these representations. Notable examples include MDNet [33] and RT-MDNet [34] which uses a VGG [31] network with a binary multi-domain classifier on top. Employing correlation filters on pretrained representations has also gained attention where the aim of the filters is to produce a 2D Gaussian score map with the maximum score at the center position of the target region. Starting from the most notable KCF [35], further extensions have been proposed, such as using a hierarchical structure [36], joint group features [37], continuous correlation operators [38], and many more [39; 40; 41]. **Siamese Network-based Trackers:** Inspired by the simplicity and fast speed of these convolutional correlation filter-based formulations, SiamFC [9] aims to use Siamese networks to further make use of deep representations. Different from [42; 43], SiamFC [9] has gained popularity owing to its fully convolutional architecture and simpler formulation compared to correlation filters. Given two image patches, where the target image patch is cropped from the initial frame using the initial target state, and the search image patch  
Figure 1: Motivation for the target-aware feature bottleneck. is cropped around the candidate region in a given video frame, target and search feature maps are extracted using a common backbone CNN feature extracting model. Afterward, a cross-correlation operation is performed between the feature maps to obtain a similarity value map where the location with the highest similarity value is picked as the predicted position. Many works improving the original SiamFC have been proposed, where deeper and wider backbones are employed [10; 11], region proposal networks [7] are used [13; 14], an additional branch for estimating the target center is added [15; 16], and anchor-free formulation is used [44; 45]. **Transformer-based Trackers:** With the recent advances in the transformer architecture [46] for the natural language processing (NLP) field, the application of attention-based modeling to computer vision research has gained popularity [8; 47]. Starting with TransT [17] which substitutes the feature cross-correlation layer of the Siamese tracker with their proposed multi-head attention (MHA)-based feature fusion block, most transformer-based tracking algorithms have proposed modifications to the Siamese network architecture to replace the cross-correlation operation between target and search feature maps. Other approaches include STARK [18], CTT [48], and DTT [49] which employ encoder-decoder architecture. Recently, inspired by vision transformers (ViT) [8], joint feature extraction and correlation modeling with transformers have been proposed. Representative works include OSTrack [50], which uses one-stream framework using only MHA blocks, and SwinTrack [12], which uses the efficient Swin Transformer [51] architecture for feature extraction and fusion. **Lightweight Trackers:** Due to the importance of a fast, real-time speed of visual tracking algorithms for practical applications, approaches aimed for designing lightweight and efficient networks for visual tracking have also gained attention.