# Parts Semantic Segmentation Aware Representation Learning for Person Re-Identification
## 3 The Proposed Method  
### Overview of the Proposed Method  
Given a probe person image, person re-identification targets the most similar persons from gallery sets according to the distance between appearance representations. Our object is to learn the generic and robust feature representations of person. Figure 2 illustrates the architecture of the proposed parts aware person re-identification network, consisting of four CNN branches which learn person appearance and three body parts feature maps. The four feature maps are fused to an image descriptor. Three local patches, including head patch, torso patch and lower-body patch, are inferred from a semantic segmentation map. Four image patches, including whole person image and three image patches, are resized to the fixed size and then input into the proposed four-branch network. Each branch learns the representation of one part and finally is fused by a concatenation layer and a fully connected layer. A softmax layer is used to classify person ID. **INPUT**: Given an image \(\mathbf{I}\in R^{M\times N}\) and its semantic segmentation map \(S\in\{0,1,2,3\}^{M\times N}\), where semantic labels 0, 1, 2 and 3 represent background, head, torso and lower-body pixels of person image respectively; \(M\) and \(N\) are the height and width of person image, respectively. **NETWORK**: The bounding boxes \(\{\mathbf{B}B_{i}\}_{i=1,2,3}\) of the three local parts are fixed by the minimum enclosing rectangles of pixels with the same semantic label. The corresponding image patches are denoted as \(\{\mathbf{P}_{i}\}_{i=1,2,3}\) (\(\mathbf{P}_{i}\) is a null matrix if its corresponding part is missing). The person image \(\mathbf{I}\) and three local parts patches \(\{\mathbf{P}_{1}\), \(\mathbf{P}_{2}\), \(\mathbf{P}_{3}\}\) go through four network branches \(\{\text{CNN}_{i}\}_{i=1,2,3,4}\), each image passes through one branch. The feature vectors of four network branches are \(\text{CNN}_{1}(\mathbf{I})\), \(\text{CNN}_{2}(\mathbf{P}_{1})\), \(\text{CNN}_{3}(\mathbf{P}_{2})\) and \(\text{CNN}_{4}(\mathbf{P}_{3})\) respectively, \(\text{CNN}_{i}(\cdot)\in R^{d_{i}x1}\). This paper uses a 3-dimensional vector to represent the absence of all 3 parts:  
\[PA=[\alpha_{1},\,\alpha_{2},\,\alpha_{3}] \tag{1}\]  
Figure 2: The architecture of proposed parts aware person re-identification network. The network consists of four convolutional neural network branches which learn person appearance and three body parts feature maps respectively, and then fuses four feature maps to an image descriptor. Its input images include a whole person image, head part patch, torso part patch and lower-body part patch. Each branch learns the representation of the whole person image or part patch and finally is fused by a concatenation layer and a fully connected layer. where \(\alpha_{i}=\left\{ \begin{array}{l}0,\;\text{size}(\text{P}_{i})=0\\ 1,\;\text{size}(\text{P}_{i})>0\end{array} \right..\)  
The proposed DropParts method (detailed in Section 3.2) maps parts absence vector \(PA\) to another 3-dimensional vector \(\widetilde{\text{PA}}\):  
\[\widetilde{\text{PA}}=\;\left[\alpha_{1}^{\prime},\alpha_{2}^{\prime},\; \alpha_{3}^{\prime}\right] \tag{2}\]  
Scale the part feature vectors and concatenate them with the whole image feature vector, get a fusion vector:  
\[\widetilde{\text{F}}_{\text{concate}}(\mathbf{I},\mathbf{P}_{1},\mathbf{P}_{2},\mathbf{P}_{3} )=\;\left[ \begin{array}{c}Normalize(\text{CNN}_{1}(\mathbf{I}))\\ \alpha_{1}^{\prime}Normalize(\text{CNN}_{2}(\text{P}_{1}))\\ \alpha_{2}^{\prime}Normalize(\text{CNN}_{3}(\mathbf{P}_{2}))\\ \alpha_{3}^{\prime}Normalize(\text{CNN}_{4}(\text{P}_{3}))\end{array} \right] \tag{3}\]  
where \(Normalize(\cdot)\) is a normalized operator. This paper uses batch normalization method [27] to normalize features of each part branch.