# Target-Aware Feature Bottleneck for Real-Time Visual Tracking
## 3 Proposed Method
### Overview of the Baseline Tracking Algorithm  
The first stage of the tracking algorithm involves feature extraction from input video frames. For input, a pair of RGB frame images \(I_{z},I_{x}\in\mathbb{R}^{H\times W\times 3}\) is given, where \(I_{z}\) is the initial (query) frame image which includes the target object along with its initial state \(b_{0}\in\mathbb{R}^{4}\) as the bounding box coordinates, and \(I_{z}\) is the current frame image in which we aim to find the target object. The dimensions of the input images are identical, with spatial sizes of height \(H\) and width \(W\) with RGB channels. Feature representations are extracted by utilizing the backbone feature extraction net \(\phi(\cdot)\), and feature representations for input images \(I_{z}\) and \(I_{x}\) are obtained as \(F_{z}=\phi(I_{z}),F_{x}=\phi(I_{x})\in\mathbb{R}^{h\times w\times c}\), respectively. The output feature maps hold spatial sizes of width \(w\) and height \(h\), with \(c\) channels, where the pooling layer and kernel stride reduce the spatial size from the original \(H\times W\)The backbone feature extractor \(\phi(\cdot)\) can be any deep neural network pretrained with an image classification task, where the linear classifier layer is removed. For efficient training purposes, some of the last layers are chosen for finetuning, whereas other layers remain frozen. Additional technical details on the architecture, dimensions, hyperparameters, and training method are explained in Section 3.4. After extracting the target and search feature maps \(F_{z},F_{x}\in\mathbb{R}^{h\times w\times c}\), the region proposal stage is conducted to search candidate areas that are similar to the target appearance, and these areas can be further classified and refined at the subsequent region classification stage. From the target feature map \(F_{z}\), the initial bounding box coordinates \(b_{0}\) can be used with the ROIAlign [60] operation to obtain spatially pooled target representation \(z\in\mathbb{R}^{s\times s\times c}\). The \(\texttt{ROIAlign}(\cdot)\) operation receives two inputs, which are the feature map and the bounding box coordinates. Then, the feature map is cropped based on the given bounding box coordinates and resized using linear interpolation to obtain a spatially pooled feature representation. We obtain the target representation as in \(z=\texttt{ROIAlign}(F_{z},b_{0})\), and \(s\) is predefined as the width and height dimension of the pooled target representation. Subsequently, a cross-correlation operation is performed in a depth-wise manner on the search feature map \(F_{x}\) using target feature representation \(z\) as a kernel by  
\[\hat{x}=F_{x}*z, \tag{1}\]  
in which \(*\) represents the cross-correlation operation with a stride size of 1 and 0-padding size of \(\lfloor\frac{s}{2}\rfloor\) in order to keep the spatial dimensions consistent. The output correlation feature map \(\hat{x}\in\mathbb{R}^{h\times w\times c}\) is passed onto the prediction heads for the region proposal net (RPN) operation, to produce two output prediction maps for the region proposal. Similar to the formulation in FCOS [61], the point-wise class label map \(y_{C}\in\mathbb{R}^{h\times w\times 2}\) with the point-wise box estimation map \(y_{R}\in\mathbb{R}^{h\times w\times 4}\) are predicted as below,  
\[y_{C}=f_{C}^{p}(\hat{x}),\ y_{R}=f_{R}^{p}(\hat{x}), \tag{2}\]  
Figure 2: Overview of the overall tracking process. where \(f_{C}^{P},f_{R}^{P}\) are RPN branches which consist of convolution, ReLU, and normalization [62] layers. Both output maps represent the point-wise class labels and bounding box coordinates, where the predicted point-wise class label map \(y_{C}\) contains the binary logit values \(y_{C}^{(i,j)}\in\mathbb{R}^{2}\) for a given row and column indices \((i,j)\), where the softmax probability of this point being inside or outside the target box region can be calculated for each position.