# Multi-Modal Long-Term Person Re-Identification Using Physical Soft Bio-Metrics and Body Figure
## 3 Methodology
### Proposed Architecture  
Most of the previous state-of-the-art models often used for person re-identification receive the entire body image as input [10; 30; 31; 32]. A multi-modal human recognition is proposed, rather than feeding the entire image of the body to a CNN. The facial region is extracted using the previously described face detector (Faceboxes) and separated from the body figure. The body figure is the second modality. Figure 3 provides visualization for the entire architecture. For the first modality, The proposed two-stream Siamese Network illustrated in Figure 4. Siamese neural networks (sometimes called a twin neural network) are two-stream neural networks used to learn local features and find similarities between pairs of images. Since the two branches of the network are identical (weights are shared), meaning  
Figure 3: Visualization of the proposed architecture. Figure 2: (**a**) A sample from PRCC dataset (117 \(\times\) 361) (**b**) A sample from PRCC dataset after resizing to (351 \(\times\) 1083) and detecting face using Faceboxes detector (**c**) A sample from PRCC dataset after extending the bounding box around the detected face to include hair, neck, and part of the chest. the network uses the same weights while working on two different input images so that the resulting output feature vectors are comparable. The similarity between feature vectors is often determined using a distance metric such as Euclidean distance, the Mahalanobis distance, Jaccard distance, etc. [33] The proposed model, each branch starts with a pre-trained model for feature extraction to extract high-level features. After that, the distance between the two feature vectors produced is determined using the Euclidean Distance [34]. A pre-trained FaceNet model [35] was used as a feature extractor. Using a pre-trained model allowed us to utilize the transfer learning technique; transfer learning exploits the knowledge gained by a network from one task to complete another task. Network optimization requires collecting a considerable amount of data which increases the amount of training time and computational resources. Transfer learning reduces network training time dramatically by using a pre-trained network such as FaceNet to train the new model on a smaller dataset by exploiting its fine-tuned parameters and pre-trained weights. The FaceNet model accepts images of size 160 \(\times\) 160 and creates 128-dimensional vectors. The pre-processing applied to the Faces of subjects does not compromise the accuracy of FaceNet, because it was trained on the Labeled Faces in the Wild (LFW) benchmark dataset. After examining this dataset It was found that images contain faces and other features (e.g., hair, neck, a portion of the chest). Each stream of the Siamese network is a FaceNet model that produces a 128-dimensional vector to represent an image. After that, we experimented with several distance metrics such as Hamming Distance, Euclidean Distance, Manhattan Distance (Taxicab or City Block), Minkowski Distance, Cosine Similarity to calculate the distance between the two images. After conducting several experiments, it was confirmed that Cosine Similarity and Euclidean Distance achieved the best performance. Briefly, the Cosine similarity between two feature vectors (\(u_{i},v_{i}\)) is the cosine angle between two N-dimensional spaces [36]. It is calculated using the dot product of the two vectors divided by the product of the two vectors' lengths or magnitude, the exact formula is described in Equation (3). \[similarity(u_{i},v_{i})=\frac{u.v}{||u||*||v||}=\frac{\sum_{i=1}^{N}u_{i}*v_{ i}}{\sqrt{\sum_{i=1}^{N}u_{i}^{2}*\sqrt{\sum_{i=1}^{N}v_{i}^{2}}}} \tag{3}\]  
Before calculating the Euclidean distance, it is often a good practice to normalize them, which means scaling values until the length/magnitude of vectors is one/unit length. L2 normalization was used to standardize the numerical values across all columns. The normalized vectors are passed to the distance metric; the Euclidean Distance.