# Rethink Motion Information for Occluded Person Re-Identification
## 3 Methodology
### Visual Encoder and Visual Adapter  
As shown in Figure 2b, the visual encoder module serves as a backbone to extract visual features. As traditional convolutional-based neural networks cannot extract robust features of the target person under different background regions with diverse characteristics very well [7], we adopt a pre-trained ViT-B/16 as our default visual encoder.  
During the training process, the images in each batch will be paired, and there will not be a situation where there is only one image with one ID. This collection strategy is prepared for calculating motion consistency loss in Section 3.2.  
Similarly, in the ViT part, the visual encoder module reshapes the T input frames of 2D images \(X\in\mathbb{R}^{T\times M\times W\times C}\), into a sequence of flattened image patches \(X_{p}\in\mathbb{R}^{T\times N\times(P^{2}\cdot C)}\). Here, \((H,W,C)\) represent the height, width, and channel of the original image, respectively. The sequence contains a total of \(N=H\cdot W/P^{2}\) image patches, and the dimension size of each image patch is \((P^{2}\cdot C)\). To keep the constant latent vector size \(D\) consistent through all layers in this module, we also apply the linear projection to transform these patches from \((P^{2}\cdot C)\) dimensions to \(D\) dimensions. Then, we add the embedding vector of positional information to obtain a sequence of extracted visual features \(\mathbf{F}_{N}\in\mathbb{R}^{T\times N\times D}\).  
The next module, named the visual adapter module, is in charge of integrating these extracted visual features \(\mathbf{F}_{N}\) into visual tokens \(\mathbf{V}\mathbf{T}_{L}\in\mathbb{R}^{L\times D}\). Note that \(L\) denotes the number of output visual tokens. Typically, we set \(L\) to be smaller than \(N\) to further reduce the subsequent computational complexity. For previous ReID methods for video, the input features are \(f\in\mathbb{R}^{T\times N\times D}\) and its computational complexity is \(\mathcal{O}(T\times N\times D)\). When we utilize the visual adapter, its computational complexity becomes \(\mathcal{O}(L\times D)\), where \(L<<(T\times N)\), and thus the complexity of our method could be significantly reduced.  
The key point in the visual adapter module is its capability to process both video and image inputs. By changing the input frame \(T\), we can adapt to different types of input data (i.e., image, video, and hybrid). As shown in Figure 2b, the visual adapter module associates each input frame with the yellow \"position\" embeddings, according to its frame sequence order. Then, the resulting sequence of embedding vectors serves as input keys and values of the following multi-head self-attention block. Finally, the adapter module generates a fixed number of \(L\) visual tokens, and \(L\) adapter queries are trained to learn about how to integrate visual features together [28; 29].  
Figure 2: An overview of the proposed motion-aware fusion network for person re-identification. In (**a**), the overall network consists of two branches, one for extracting visual features from video and image inputs and the other for mining implicit motion information. In (**b**), we show the detail of our visual adapter. In (**c**), we show the detail of the motion-aware transformer (MAT) module.  
As several images \((T>1)\) of the same person contain more rich spatial-temporal information when compared to a single image input [30], this integration process also helps retaining adequate spatial-temporal information and improves the robustness of the training process [24].