# PMG--Pyramidal Multi-Granular Matching for Text-Based Person Re-Identification
## 3 Proposed Method
### Pyramidal Multi-Granular Feature Extraction
#### 3.1.1 Visual Feature Extraction  
In order to extract global and local visual features from a given image \(I\), a ResNet-50 backbone that has been pretrained on the ImageNet dataset is first employed to generate the shared visual feature map \(\psi(I)\in\mathbb{R}^{w\times h\times c}\) with width \(w\), height \(h\) and channels \(c\). Considering that horizontally partitioning \(\psi(I)\) into a fixed number of non-overlapping stripes may suffer from the misaligned bounding box problem as discussed in the Introduction section, we adopt a coarse-to-fine pyramidal matching method to obtain visual representations with various pyramidal scales. Specifically, PMG introduces four different pyramidal scales by horizontally partitioning \(\psi(I)\) into 1, 2, 4, or 6 spatial bins, respectively. Accordingly, the feature map is denoted as \(\psi_{K}(I)\in\mathbb{R}^{w\times h_{K}\times c}\), where \(K=1,2,4,6\) and \(h_{K}=h/K\) means \(\psi(I)\) is horizontally portioned by \(K\) uniform parts. For each \(\psi_{K}(I)\), the partitioning operation is carried out by an average pooling module  
\[AvgPooling_{K}=\frac{1}{h_{K}\times w}\sum_{i=1}^{h_{K}}\sum_{j=1}^{w}\psi_{K} ^{[1:K]}(I)_{i,j,1:c}, \tag{1}\]  
and a maximum pooling module  
\[MaxPooling_{K}=\max_{1\leq i\leq h_{K}\leq j\leq w}\psi_{K}^{[1:K]}(I)_{i,j,1:c}, \tag{2}\]  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Method**} & **Network** & **Attention** & **Aiding** & **Local Feature** \\  & **Architecture** & **Mechanism** & **Information** & **Alignments** \\ \hline SUM [7] & ✓ & & & - \\ PMA [8] & & ✓ & ✓ & \(V_{K}^{6}\)-\(T_{L}^{6}\) \\ CAIRC [9] & ✓ & & ✓ & \(V_{L}^{6}\)-\(T_{L}^{6}\) \\ GNA-RNN [10] & ✓ & & & \(V_{L}^{6}\)-\(T_{L}^{6}\) \\ PWM-ATH [11] & & ✓ & & - \\ MIA [12] & ✓ & ✓ & ✓ & \(V_{L}^{6}\)-\(T_{L}^{6}\) \\ SSAN [13] & & ✓ & & \(V_{L}^{6}\)-\(T_{L}^{6}\) \\ A-GANet [20] & ✓ & ✓ & & \(V_{L}^{6}\)-\(T_{L}^{6}\) \\ TIMAM [21] & ✓ & & & \(V_{L}^{6}\)-\(T_{  
hence the visual pyramidal module is defined as follows:  
\[\Phi_{K}(I)=AvgPooling_{K}(I)+MaxPooling_{K}(I), \tag{3}\]  
where \(K\in\{1,2,4,6\}\) denotes the pyramidal level and \(\Phi_{K}(I)\in\mathbb{R}^{K\times 1\times c}\). This fused pooling module takes advantage of the down-sampling characteristic of the average pooling layer to reduce computation complexity, as well as the capability for capturing discriminative signals of the maximum pooling layer, which may be weakened when they are surrounded by signals relatively poor (e.g., background). Figure 3: The algorithm framework of PMG. Based on the pretrained ResNet-50 and Bi-GRU backbones, PMG extracts global and local features, respectively, from visual and textual modalities. For each modality, multiple parallel branches are utilized for extracting multi-granular representations. For simplicity, only the global branch and the \(K\)-strip branch are shown. A Word Attention Module is adopted to establish semantic correlations between the \(K\)-strip and relative noun phrases. Then, the four different pyramidal scales are separately processed to obtain the final visual representations. To extract the global visual representation \(V_{g}\in\mathbb{R}^{P}\), we reshape \(\Phi_{1}(I)\in\mathbb{R}^{1\times 1\times c}\) to a \(c\)-dimensional vector and then pass it through a batch normalization (BN) layer followed by a \(1\times 1\) convolutional layer consisting of \(P\) kernels. As for local visual representations of \(V_{I2}\), \(V_{I4}\) and \(V_{I6}\), we first create three different \(K\times 1\times c\) (\(K=2,4,6\)) processing structures, in which a \(1\times 1\) convolutional layer is, respectively, adopted to reduce the dimensionality from \(c\) to \(c/2\).