# FETrack: Feature-Enhanced Transformer Network for Visual Object Tracking
## 3 Proposed Method
### Network Architecture  
The structure of the encoder and decoder modules is presented in Figure 2. Before describing the flows of the encoder and decoder, we first introduce their input representations. **Input Representation. ** The inputs to FETrack include two pairs of template images (the initial template \(it_{img}\in\mathbb{R}^{3\times H_{r}\times W_{t}}\) and dynamic template \(dt_{img}\in\mathbb{R}^{3\times H_{r}\times W_{t}}\)) and a search image \(s_{img}\in\mathbb{R}^{3\times H_{s}\times W_{s}}\). Similar to ViT [13], we split and flatten the input images into sequences of patches, i.e., \(it_{p}\in\mathbb{R}^{N_{l}\times 3\cdot P^{2}}\), \(dt_{p}\in\mathbb{R}^{N_{l}\times 3\cdot P^{2}}\), and \(s_{p}\in\mathbb{R}^{N_{s}\times 3\cdot P^{2}}\). Where \(P\times P\) represents the resolution of each patch and \(N_{t}=H_{t}W_{t}/P^{2}\) and \(N_{s}=H_{s}W_{s}/P^{2}\) denote the number of patches of the template and the search region, respectively. Then, these patch sequences are fed into a linear projection layer to obtain the D-dimensional patch embedding. Subsequently, after introducing the learnable position embedding, the patch embeddings are fed into the encoder. The inputs to the decoder comprise visual features of the search region extracted by the encoder and the discretized bounding box tokens. For the discretized representation of the bounding box values, similarly to [35], we evenly discretize the bounding box values corresponding to the target position and scale into a vocabulary \(V\) between [1, 4000]. In this way, the input sequence to the decoder can be expressed as [start, x, y, w, h], and the special token (start) also corresponds to a learnable embedding. Then, these corresponding embeddings are fed into the decoder. **Encoder. ** The modeling scheme of the one-stream pipeline allows for the deep bidirectional information interaction between the template and the search region, which dynamically acquires the discriminative target-oriented features by mutual guidance. However, such extensive and unrestricted information interaction potentially corrupts the quality of the template features, leading to tracking drift or even failure. As shown in Figure 3, for the one-stream pipeline, the template features are progressively contaminated during the bidirectional information interaction due to interference from the search region. In contrast, the modeling scheme of the two-stream pipeline effectively avoids this contamination by processing the template and the search region separately, thus ensuring that the template features are more focused on the target. Nevertheless, the template features extracted by the two-stream pipeline possess low discrimination, which struggles to handle challenges such as appearance change and occlusion. To address the inherent defects of these two modeling schemes as well as to leverage their strengths, we propose a novel encoder modeling scheme that employs an independent template stream and a bidirectionally interacting template-search stream to acquire visual features in parallel. In addition, we take Vanilla ViT [13] as the backbone and employ the proposed encoder to replace the traditional ViT-based encoder. We also remove the class token and add a linear projection layer at the last layer of the encoder to align the feature scales of the encoder and decoder. As shown in Figure 1, the input of the encoder is composed of the initial template \(it\in\mathbb{R}^{N_{l}\times D}\), dynamic template \(dt\in\mathbb{R}^{N_{l}\times D}\), and search region \(s\in\mathbb{R}^{N_{l}\times D}\). Firstly, the in  
Figure 3: Comparison of template features between two-stream (T-S) and one-stream (O-S) pipelines. -dependent template stream is intended to preserve the purity of the template features and minimize the potential interference from irrelevant search regions. The structure of the independent template stream is illustrated in Figure 2. Both the initial and dynamic template tokens are utilized as queries for attention enhancement.