# MPVT: An Efficient Multi-Modal Prompt Vision Tracker for Visual Target Tracking
## 2 Materials and Methods
### Decoupled Input Embedding Enhancement Module  
For the input template images \(\mathbf{Z}_{RGB},\mathbf{Z}_{AUX}\in\mathbb{R}^{H_{x}\times W_{z}\times 3}\) and search regions \(\mathbf{X}_{RGB}\), \(\mathbf{X}_{AUX}\in\mathbb{R}^{H_{x}\times W_{z}\times 3}\), two image embedding layers are used to map them to token sequences. Since the backbone has been pre-trained, only the auxiliary modal embedding layer needs to be fine-tuned, while the RGB embedding layer is frozen. The embedding process can be written as  
\[\tilde{\mathbf{X}}_{RGB}^{0}=\mathcal{E}_{RGB}(\mathbf{X}_{RGB}) \tag{1}\] \[\tilde{\mathbf{Z}}_{RGB}^{0}=\mathcal{E}_{RGB}(\mathbf{Z}_{RGB})\] \[\tilde{\mathbf{X}}_{AUX}^{0}=\mathcal{E}_{AUX}(\mathbf{X}_{AUX})\] \[\tilde{\mathbf{Z}}_{AUX}^{0}=\mathcal{E}_{AUX}(\mathbf{Z}_{AUX})\]  
where \(\tilde{\mathbf{X}}_{RGB}^{0},\tilde{\mathbf{X}}_{AUX}^{0}\in\mathbb{R}^{L_{ x}\times d}\), \(\tilde{\mathbf{Z}}_{RGB}^{0},\tilde{\mathbf{Z}}_{AUX}^{0}\in\mathbb{R}^{L_{ x}\times d}\), \(L_{x}=\frac{H_{x}}{s}\times\frac{W_{x}}{s}\), and \(L_{z}=\frac{H_{z}}{s}\times\frac{W_{z}}{s}\), with the patch size being \((s,s)\). After obtaining the embedding token sequences for each image, this module enhances the input token using two procedures: resolution-decoupled positional embedding enhancement and type embedding enhancement. Traditional multi-modal tracking models independently embed the positions of the template and searching regions by utilizing one-dimensional absolute embedding. If the resolution of the template region is significantly lower than that of the searching region, this will disrupt the input distribution of the pre-trained ViT model, resulting in the failure of fine-tuning. To solve this problem, unified position embedding through a resolution-decoupled position embedding method is presented. Specifically, let the one-dimensional absolute embedding be \(\mathbf{R}\); the corresponding two-dimensional form is \(\mathbf{R}_{2D}\), as shown in Equation (2). For a searching region of the size \(h_{x}\times w_{x}\), if \(h_{x}=handw_{x}=w\), \(\mathbf{R}_{2D}\) can be directly used as the position embedding of the search region, i.e., \(\mathbf{R}_{x}=\mathbf{R}_{2D}\). \[\mathbf{R}_{2D}=\left( \begin{array}{ccc}r_{11}&\cdots&r_{1w}\\ \vdots&\ddots&\vdots\\ r_{h1}&\cdots&r_{hw}\end{array} \right), \tag{2}\]  
A sub-matrix of \(\mathbf{R}_{x}\) is used as the position embedding of a template region of the size \(h_{z}\times w_{z}\) due to its resolution being smaller than that of the searching region. Specifically, starting from the top-left corner of \(\mathbf{R}_{x}\), we select a region whose size equals the size of the template region, which can be written as  
\[\mathbf{R}_{z}=\left( \begin{array}{ccc}r_{11}&\cdots&r_{1w_{z}}\\ \vdots&\ddots&\vdots\\ r_{hz1}&\cdots&r_{hzw_{z}}\end{array} \right). \tag{3}\]  
Since the position encoding of the template region \(\mathbf{R}_{z}\) and that of the searching region \(\mathbf{R}_{x}\) are derived from the same position encoding \(\mathbf{R}_{2D}\), which is consistent with the pre-trained ViT model, the uniformity of position encoding can be ensured. Additionally, two learnable embeddings, \(\mathbf{E}_{z}^{type}\in\mathbb{R}^{d}\) and \(\mathbf{E}_{x}^{type}\in\mathbb{R}^{d}\), are designed to distinguish between the template and searching region embedding and between the targets in the template region and background embedding. \(\mathbf{E}_{z}^{type}\) is further divided into the target region embedding \(\mathbf{E}_{z_{o}}^{type}\in\mathbb{R}^{d}\) and the background region embedding \(\mathbf{E}_{z_{o}}^{type}\in\mathbb{R}^{d}\). Through resolution-decoupled position and type embedding enhancement, the template region embedding \(\mathbf{E}_{z}\) and the searching region embedding \(\mathbf{E}_{x}\) can be represented as  
\[\mathbf{E}_{x}^{(i,j)}=\mathbf{R}_{x}^{(i,j)}+\mathbf{E}_{x}^{type}, \tag{4}\]  
and  
\[\mathbf{E}_{z}^{(i,j)}=\left\{