# PMG--Pyramidal Multi-Granular Matching for Text-Based Person Re-Identification
## 1 Introduction
[8] incorporated pose information to aid in localizing the discriminative body parts for effective local visual feature extraction. However, utilizing prior knowledge like poses can suffer from the inaccuracy of human pose estimation and great computation burden. In the study conducted by [PERSON] et al. [12], a Multi-granularity Image-text Alignments (MIA) model was proposed, by which a feature map is horizontally cropped into a group of non-overlapping image stripes for local visual feature extraction. Following this work, [PERSON] et al. [13] proposed a Word Attention Module (WAM) to acquire correlations between words and image stripes, thus extracting semantically aligned part-level features. Briefly, existing works typically utilize partitioning methods to divide pedestrian images into multiple strips to capture more discriminative local visual features, and employ attention mechanisms for better matching of image strips with relevant textual descriptions. Although existing methods have achieved significant progress in cross-modal retrieval, there still exist some limitations. Taking the SOTA method SSAN [13] as an example, it is difficult to determine how many stripes an image should be partitioned into. In fact, the coarser the image slicing, the more difficult it is to capture detailed information, while the finer the slicing, the more likely it is to compromise the integrity of local features. Intuitively, diverse partitioning scales are preferred, ensuring that for each body part of a pedestrian, there always exists an appropriate scale that allows an effective extraction of local features from a certain image strip and accurate alignment with relevant words/phrases at this scale. Figure 1 illustrates this concept, at scale 4, the weakened signal of \"glasses\" is greatly strengthened by minimizing the surrounding interference of excessive background while the integrity is well-preserved, which facilitates extracting discriminative local features and aligning with relevant words/phrases highlighted by green rectangles. Inspired by this idea, we propose a Pyramidal Multi-Granular matching network (PMG) with an overall architecture illustrated in Figure 2. PMG extracts eight representations with different granularities, including a global visual representation (\(V_{g}\)), a global textual representation (\(T_{g}\)), three local visual representations (\(V_{I2}\), \(V_{I4}\), \(V_{I6}\)), and three local textual representations (\(T_{I2}\), \(T_{I4}\), \(T_{I6}\)), and then four different levels of cross-modal similarities between visual and textual features, i.e., \(V_{g}\)-\(T_{g}\), \(V_{I2}\)-\(T_{I2}\), \(V_{I4}\)-\(T_{I4}\) and \(V_{I6}\)-\(T_{I6}\) are calculated to better match visual and textual information. To be more specific, PMG horizontally partitions feature maps of the visual backbone into multiple non-overlapping stripes with various pyramidal scales and employs a Word Attention Module to acquire correlations between words and image stripes, then extracts modality features with different scales independently, which can incorporate the gradual transition process between the coarsest global information and the finest local information. After training the combination of multi-granular visual and textual representations, PMG obtains the final cross-modal similarity by fusing the multi-granular cross-modal similarities. In contrast with existing methods which extract local features with a single granularity for matching, our work employs a pyramidal method for the extraction and matching of features with four different granularities. Obviously, multiple granularities can better adapt to the extraction and matching of different features, even for pedestrian images under intensive changes, such as pose, viewpoint, etc., thus suppressing the variances of feature extraction and improving the robustness of feature matching.