# MFCNet: Mining Features Context Network for RGB-IR Person Re-Identification
## 3 Our Method
### The Proposed MFCNet
For the input feature set \(F\in\mathbb{R}^{\mathbb{C}\times H\times W}\), 1 x 1 convolution and the softmax function are first used to obtain attention weights; then, the original input features are fused to obtain global context features. We use \(F_{1}\in\mathbb{R}^{\mathbb{C}\times  
the sigmoid function with weights, and the original input \(F\in\mathbb{R}^{\mathbb{C}\times H\times W}\) is multiplied by the weights to obtain the weighted feature matrix. Finally, the result is added to the original input set \(F\in\mathbb{R}^{\mathbb{C}\times H\times W}\) to improve the representation ability of the features. The corresponding calculation is based on formula (1). **GeM Pooling. ** Unlike the commonly used maximum pooling and average pooling methods, GeM pooling focuses on areas of different scales to capture the characteristics of specific modalities. The corresponding formula is as follows:  
\[\mathbf{f}=[f_{1}...f_{k}...f_{k}]^{T},f_{k}=(\frac{1}{|\chi_{k}|}\sum_{x\in\chi_{ k}}x^{p_{k}})^{\frac{1}{p_{k}}} \tag{6}\]  
where \(\chi_{k}\) represents each feature map and \(p_{k}\) is a hyperparameter. When \(p_{k}\rightarrow\infty\), formula (6) becomes the expression for maximum pooling. Correspondingly, when \(p_{k}\to 0\), formula (6) becomes the expression for average pooling. **Loss Function. ** To learn the feature representations, two types of loss functions are utilized to supervise the training of our network. One is identity loss, employed for identity information learning, and the other is triplet loss, used for similarity learning. However, contrastive loss and triplet loss are both common methods for similarity learning. The effect of triplet loss is better than that of contrastive loss, because it considers the distance between the positive and negative samples and the anchor point. Therefore, the combination of triplet loss and identity loss is one of the most popular solutions for deep person Re-ID model learning. _Identity Loss._ The identity loss aims to learn the discriminative feature representations by using modality-specific information, which could distinguish different persons within each modality. It treats the training process of person Re-ID as an image classification problem, where each identity is a distinct class. Given an input image \(v_{i}\) with the identity label \(y_{i}\), the probability of \(v_{i}\) being recognized as class \(y_{i}\) encoded with a softmax function, represented by \(p(y_{i}|v_{i})\). The identity loss is then computed by the cross-entropy  
\[L_{id}=-\frac{1}{n}\sum_{i=1}^{n}log(p(y_{i}|v_{i})) \tag{7}\]  
where \(n\) represents the number of training samples within each batch. Similarly, we could get the identity loss for the thermal image \(t_{i}\). _Weighted Regularization Triplet (WRT) Loss._ The triplet loss is used for similarity learning. It tries to reduce the feature distances between images of the same person and expand the distances between images of different people.The triplet loss with a margin parameter is represented by  
Figure 2.0  
where \(d(. )\) measures the Euclidean distance between two samples. \(\rho\) is the pre-defined margin. \(ij\) is a positive sample pair, and \(ik\) is a negative sample pair. The weighted regularized triple loss function is an improvement of the triple loss function. It not only inherits the advantages of the original method but also avoids the pre-defined parameter. Figure 2.1  
Figure 2.2  
Notably, \(i\), \(j\), and \(k\) are the sample indexes. Additionally, \(p\) represents the positive sample data set in each training batch, and \(n\) represents the negative sample data set. \(ij\) is a positive sample pair, and \(ik\) is a negative sample pair. \(d\) represents the Euclidean distance between two samples. In this paper, weighted regularized triple loss function \(L_{urt}\) and identity loss function \(L_{id}\) are combined for network training to learn discriminative features [12; 15; 24; 25; 26; 27; 28; 29]. Therefore, the loss function is defined as:  
Figure 2.3