# Multi-Modal Long-Term Person Re-Identification Using Physical Soft Bio-Metrics and Body Figure
## 2 Related Work
### Multi-Scale Deep Learning Architectures for Person Re-Identification  
[PERSON], [PERSON] et al. [3] proposed an advanced multi-scale deep learning model (MuDeep) that can re-identify a person by extracting discriminative features at multiple scales with automatically determined scale weighting. The idea behind multi-scaling is to mimic how a human brain re-identifies people from coarse to fine. For example, some people are distinguishable by their global features (e.g., gender, body, etc.). On the other hand, some people are distinguishable by their local features (e.g., shoes, bags, etc.). They based their model on the fact that people are distinguishable at correct spatial locations and scales. One might identify a person from an obvious detail like glasses. Sometimes a closer look at the fine details (e.g., eye colour) is needed to identify a person. The architecture of the network is a Siamese one [15]. The main contributions presented in this work are:  
* Tied Convolution for pre-processing the input
* Multi-scale stream layers: Analyses the data stream with several receptive field sizes. * Saliency-based learning fusion layer: To fuse the output streams from the previous layers. * Sub-nets for person re-identification: Used for verification so the problem can be transformed into a binary classification one, stating whether a pair of images represent the same person or not. [PERSON] et al. [16] proposed Omni-Scale Feature Learning for Person Re-Identification. They utilized Omni-scale features in the person re-identification task. The idea behind Omni-scale feature learning is combining features of multiple scales, OSNet is a network that specializes in that domain. Unlike [PERSON] et al. [3] Omni-scale features include heterogeneous and homogeneous features as well as features at multiple scales. In other words, homogeneous features are global features (young man, white shirt, grey shorts, etc.) and local features (trainers or sandals). Heterogeneous features take a closer look at the complicated and richer details like a specific logo on the white shirt. The key to Omni-scale feature is to combine both homogeneous and heterogeneous features. A white shirt on its own or a specific logo is not discriminative or enough to identify a person. OSNet starts by decomposing the convolutional layer and then introduces a full-size residual block and a unified aggregation gate. The basic building block of the OSNet is made of multiple convolutional streams each with a different receptive field (different scales). The basic building block of OSNet is the lit 3 \(\times\) 3 convolutional layer, which is given by Equation (1). The block receives input \(x\) and learns \(x^{\prime}\). \[x^{\prime}=F(x) \tag{1}\]  
Inspired by the Inception model architecture [17] to achieve multi-scale feature learning, several lit 3 \(\times\) 3 convolutional blocks are stacked together to achieve a lightweight network. As shown in the Equation (2) to stack several 3 \(\times\) 3 blocks function \(F\) is extended by introducing a new dimension, exponent \(t\), which represents the scale of the feature. For \(t>\) 1, stacking \(t\) Lite 3 \(\times\) 3 layers, results in a receptive field of size \((2t+1)\times(2t+1)\). \[x^{\prime}=\sum_{t=1}^{T}F^{t}(x),\,s.t.\,T\geq 1 \tag{2}\]  
The resulting multi-scale feature maps from the previously mentioned convolutional stream are dynamically fused through unified aggregation gates. The aggregation gates are subnets that share parameters across all streams to generate weights for each channel in the convolutional stream. Training those parameters improves the weight given to each channel. In other words, if during the training process back-propagation of the error shows that the stream resulting from a receptive field of three contributes with more valuable information to the model's performance, the aggregation gates will assign a dominant weight to this stream and less weight to the other streams.