A sample‐proxy dual triplet loss function for object re‐identification

IET Image ProcessingVolume 16, Issue 14 p. 3781-3789 ORIGINAL RESEARCHOpen Access A sample-proxy dual triplet loss function for object re-identification Hanxiao Wu, Hanxiao Wu orcid.org/0000-0001-7710-2177 College of Information Science and Engineering, Huaqiao University, Xiamen, ChinaSearch for more papers by this authorFei Shen, Fei Shen College of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, ChinaSearch for more papers by this authorJianqing Zhu, Corresponding Author Jianqing Zhu [email protected] College of Engineering, Huaqiao University, Quanzhou, China Xiamen Yealink Network Technology Company Limited, No. 666, Hu'an Road, High-tech Park, Huli District, Xiamen, Fujian, China Correspondence Jianqing Zhu, College of Engineering, Huaqiao University, Quanzhou 362021, China; Xiamen Yealink Network Technology Company Limited, Xiamen 361015, China. Email: [email protected]Search for more papers by this authorHuanqiang Zeng, Huanqiang Zeng College of Engineering, Huaqiao University, Quanzhou, ChinaSearch for more papers by this authorXiaobin Zhu, Xiaobin Zhu School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing, ChinaSearch for more papers by this authorZhen Lei, Zhen Lei National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, ChinaSearch for more papers by this author Hanxiao Wu, Hanxiao Wu orcid.org/0000-0001-7710-2177 College of Information Science and Engineering, Huaqiao University, Xiamen, ChinaSearch for more papers by this authorFei Shen, Fei Shen College of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, ChinaSearch for more papers by this authorJianqing Zhu, Corresponding Author Jianqing Zhu [email protected] College of Engineering, Huaqiao University, Quanzhou, China Xiamen Yealink Network Technology Company Limited, No. 666, Hu'an Road, High-tech Park, Huli District, Xiamen, Fujian, China Correspondence Jianqing Zhu, College of Engineering, Huaqiao University, Quanzhou 362021, China; Xiamen Yealink Network Technology Company Limited, Xiamen 361015, China. Email: [email protected]Search for more papers by this authorHuanqiang Zeng, Huanqiang Zeng College of Engineering, Huaqiao University, Quanzhou, ChinaSearch for more papers by this authorXiaobin Zhu, Xiaobin Zhu School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing, ChinaSearch for more papers by this authorZhen Lei, Zhen Lei National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, ChinaSearch for more papers by this author First published: 25 July 2022 https://doi.org/10.1049/ipr2.12593AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract Object re-identification, such as vehicle re-identification or pedestrian re-identification, plays a significant role in intelligent video surveillance systems for public security. Due to viewpoint variations and appearance changes, both pedestrians and vehicles usually have complex intra-class variations. However, most existing object re-identification methods often use a sample-level triplet loss function cooperating with a single-proxy softmax loss function, which could not handle complex intra-class variations well. In this paper, a sample-proxy dual triplet (SPDT) loss function is proposed, which works with a multi-proxy softmax (MPS) loss function. The MPS loss function is in charge of learning multiple proxies to represent a class. The SPDT loss function is responsible for enlarging inter-class distances as well as shrinking intra-class distances on both sample and proxy levels. Therefore, the method not only handles multi-proxy intra-class variations but also fully learns discrimination on samples and proxies. Experiments on two large datasets, that is, VeRi776 and DukeMTMC-reID, demonstrate that the method is superior to state-of-the-art object re-identification approaches. 1 INTRODUCTION Object re-identification [1-5] is important for intelligent video surveillance [6, 7], which can retrieve a suspicious target (e.g. vehicle or pedestrian) from a large-scale image set. However, due to a series of unfavorable factors, such as camera viewpoint variations, illumination changes, pose changes, and low resolutions, object re-identification is challenging in real surveillance scenarios. Therefore, object re-identification receives much attention from both academia and industry communities. With the explosive development of deep learning, many object re-identification methods are dedicated to design feature learning architectures [2, 3, 5, 8-15] and loss functions [4, 7, 16-19]. Regarding feature learning architectures, there are many impressive works. Liu et al. [8] proposed a graph convolutional network to learn context information for pedestrian re-identification. Zhong et al. [9] proposed a novel generation adversarial network (GAN) to augment training data for regularizing the object re-identification model. Kalayeh et al. [10] designed a segmentation network to exploit semantic information for improving pedestrian re-identification. Meng et al. [11] developed a parsing network to decompose a vehicle from four different views and then aligned the features by mask average pooling. In order to resist the adverse effect of viewpoint variations, Zhu et al. [1, 5] proposed multiple directional feature learning architectures to extract features from vehicle images. In addition, attention-based networks [2, 3], part regularization models [12, 13], pose aware models [14, 15] can learn discriminative local features for object re-identification. Regarding loss functions, the most popular approach [19, 20] for object re-identification is to use the combination of the triplet loss [21] function and the softmax loss [22, 23] function. Most works focus on improving triplet loss functions to improve the re-identification performance further. For example, Li et al. [7] proposed an adaptively weighted triplet loss function, which uses softmax weight distributions to weight triplet samples. The similar adaptively weighted strategy also can be found in ref.[4]. Chu et al. [16] proposed cross-space and within-space constraints to help the triplet loss function learn view-aware metrics. He et al. [18] designed an enhanced triplet loss (ETL) function that additionally optimizes the margin to obtain an efficient training process. Organisciak [17] designed a unification loss function to separate hard negative samples and gather hard positive samples. In our previous work [24], we designed a spatial structural similarity (S3) triplet loss to reserve the spatial discrimination for improving vehicle re-identification. Although these improved triplet loss functions have obtained excellent re-identification performance, there is still room for improvements for two reasons. (1) The softmax loss function working with the triplet loss function is to learn a single proxy to represent a class, which is difficult to describe complex intra-class variations of a subject. For example, as shown in Figure 1, a person with a red coat and gray trousers is likely to have multiple proxies. (2) These improved triplet loss functions are sample-level discrimination optimizations, that is, enlarging inter-class distances meanwhile shortening intra-class distances of samples, which ignores the discrimination existing among proxies. FIGURE 1Open in figure viewerPowerPoint The illustration of complex intra-class variations on DukeMTMC-reID [25] This paper proposes a sample-proxy dual triplet (SPDT) loss function for object re-identification, which cooperates with a multi-proxy softmax (MPS) loss function. Specifically, the MPS loss function learns multiple proxies for each class. The SPDT loss function enlarges inter-class distances meanwhile shortens intra-class distances on both sample and proxy levels. The novelty of this paper is that we make the first attempt to fully explore discrimination on both sample and proxy levels. In addition, experiments on two large-scale datasets, that is, VeRi776 and DukeMTMC-reID, show that our method outperforms many state-of-the-art methods using the same backbone. 2 RELATED WORK Benefit from the development of deep learning, the methods of improving object re-identification methods can be classified in two categories: (1) loss function designs [4, 7, 16-19, 26], and (2) feature learning architectures [2, 3, 5, 8-15, 27-29]. Loss function designs In object re-identification, the single-proxy softmax loss [23] directly optimize a posterior probability is a fundamental loss function. Another common loss function is the triplet loss function [30], which pursues discrimination via cooperating with a hard sample mining strategy to push the closest negative sample pairs as far as possible and pull the farthest positive sample pairs as close as possible. In addition to hard sample mining, there are many improved triplet loss functions. For example, Ye et al. [4] proposed a weighted regularization triplet (WRT) loss function by using softmax weight distributions to weight triplet samples, which makes triplet samples are more robust to outliers. The similar weighted regularization is also presented in [7, 17]. Chu et al. [16] jointly constrained similar-view triplets, different-view triplets and corss-view triplets to deal with various viewpoints. However, these improved triplet loss functions just pay attention to the sample-level optimization, ignoring the optimization on the proxy-level. Although Softtriple [31] also applies multi-proxy to replace hard mining, it only considers to reduce intra-class variations. Feature learning architectures Currently, the most popular feature learning architecture for object re-identification [3, 7, 16-18] is ResNet50 [32] followed global pooling layer and batch normalization (BN) layers. In addition to the ResNet50 backbone, Meng et al. [11] applied SEResNeXt50 [33] as a backbone to design a parsing-based view-aware embedding network (PVEN). Kalayeh et al. [10] adopted the Inception-V3 [34] backbone for semantic parsing re-identification (SPReID). Zhu et al. [5] applied four shortly and densely connected convolutional neural networks (SDC-CNN) [35] to learn improve quadruple directional features for object re-identification. There are also some specific architecture designs for object re-identification, such attention models [2, 3, 36], the multi-scale feature learning model [27], the camera style generative modal [9]. In summary, with the rapid development of deep learning, a lot of feature learning architectures are designed to promote object re-identification greatly. 3 PROPOSED METHOD Overall framework The overall framework of the proposed object re-identification model includes two aspects: (1) the feature learning architecture and (2) the loss function design. As shown in Figure 2, the ResNet50 [32] is applied as the backbone for feature learning as done in many existing works [3, 7-9, 12, 13, 16, 18, 19]. Furthermore, following the ResNet50 backbone, a global max pooling (GMP) and a 512-dimensional fully connected (FC) layer are applied to compress features. FIGURE 2Open in figure viewerPowerPoint The overall framework of the proposed object re-identification model. The GMP, FC and BN represent global max pooling, fully connected layer, and batch normalization layer, respectively; U and V represent the class of number of instance and the number of samples in each class, respectively; C and K denote the number of classes and the proxy number of each class, respectively Regarding the loss function design, as formulated in Equation (1), the loss function for object re-identification consists of two items: (1) the multi-proxy softmax (MPS) loss function and (2) the sample-proxy dual triplet (SPDT) loss function. The MPS loss function is applied to learn multiple proxies to represent a class, and the SPDT loss function is designed for enlarging inter-class distances as well as shrinking intra-class distances on both sample and proxy levels. Although the MPS loss function and the SPDT loss function are added to formulated the final loss function, they are assigned at different locations, as shown in Figure 2. To be more specific, a batch normalization (BN) layer is embedded after the fully connected (FC) layer to balance the SPDT loss and MPS loss functions. The MPS loss function is assigned to supervise the BN layer, and the STRI loss function is assigned to supervise the FC layer, which is the so-called batch normalization neck (BNNeck) [19]. Different from [19], the proxies of the MPS loss function are also supervised by the proxy-level triplet (PTRI) loss function. More details about MPS and SPDT loss functions are described in Sections 3.2 and 3.3, respectively. L = L M P S + L S P D T . \begin{equation} L = L_{MPS} + L_{SPDT}. \end{equation} (1) Multi-proxy softmax loss The traditional single-proxy softmax (SPS) loss [22, 23] function is formulated as follows: L S P S = − 1 M ∑ i = 1 M log p ( y i | x i ) = − 1 M ∑ i = 1 M e w y i ⊤ x i ∑ j = 1 C e w j ⊤ x i , \begin{equation} {L_{SPS}} = \frac{-1}{M} \sum \limits _{i = 1}^M {\log p({y_i}|{x_i}) = } \frac{-1}{M} \sum \limits _{i = 1}^M {\frac{{{e^{w_{{y_i}}^ \top {x_i}}}}}{{\sum \limits _{j = 1}^C {{e^{w_j^ \top {x_i}}}} }},} \end{equation} (2)where x i ∈ R d $x_i \in R^{d}$ and y i ∈ [ 1 , 2 , 3 , … , C ] $y_i \in [1,2,3,\ldots,C]$ denote the d-dimensional feature vector and class label of the i-th sample, respectively; C and M, respectively, represent the number of classes and the number of samples, respectively; w j ∈ R d $w_j \in R^{d}$ is a learn-able proxy representing the j-th class. From Equation (2), it can be found that each class's data are represented with just one proxy, which could not deal with complex intra-class variations well. Following [31], the traditional SPS loss function can be extended to a multi-proxy softmax (MPS) loss function, as follows: L M P S = − 1 M ∑ i = 1 M log e θ y i ⊤ x i ∑ j = 1 C e θ j ⊤ x i , \begin{equation} {L_{MPS}} = \frac{-1}{M} \sum \limits _{i = 1}^M {\log } \frac{{{e^{\theta _{{y_i}}^ \top {x_i}}}}}{{\sum \limits _{j = 1}^C {{e^{\theta _j^ \top {x_i}}}} }}, \end{equation} (3)where θ j $\theta _{j}$ denotes the j-th class's composite proxy calculated from K proxies, as follows: θ j = ∑ k = 1 K e w j k ⊤ x i ∑ n = 1 K e w j n ⊤ x i w j k , \begin{equation} {\theta _j} = \sum \limits _{k = 1}^K {\frac{{{e^{w_{jk}^ \top {x_i}}}}}{{\sum \limits _{n = 1}^K {{e^{w_{jn}^ \top {x_i}}}} }}} {w_{jk}}, \end{equation} (4)where w j k $w_{jk}$ represents the k-th proxy of the j-th class. From Equation (4), it can be seen that the correlation between x i $x_i$ and w j k $w_{jk}$ is applied to re-weight w j k $w_{jk}$ , and then all re-weighted proxies are summarized to form the composite proxy θ j $\theta _{j}$ . Sample-proxy dual triplet loss function The commonly-used sample-level triplet (STRI) loss [19, 30] function is defined as: L S T R I = 1 U × V ∑ i = 1 U × V max x p ∈ P x i ∥ x i − x p ∥ 2 − min x n ∈ N x i x i − x n 2 + α + , \fontsize{8}{10} \begin{equation} {L_{STRI}} = \frac{1}{{U \times V}}\sum \limits _{i = 1}^{U\times V} {{{{\left[ {\mathop {\max }\limits _{x_p \in {P_{{x_i}}}} {{{\Vert {{x_i} - x_p} \Vert} }_2} - \mathop {\min }\limits _{x_n \in {N_{{x_i}}}} {{{\left\Vert {{x_i} - x_n} \right\Vert} }_2} + \alpha } \right]}}_ + }}, \end{equation} (5)where U and V, respectively, denote the number of classes and the number of instances in each class in a mini-batch; [ · ] + $[\cdot ]_+$ equals to max ( · , 0 ) $\mathop {\max }(\cdot ,0)$ ; α > 0 $\alpha > 0$ is a margin, which is set to be 1.2 in this work; P x i $P_{x_i}$ and N x i $N_{x_i}$ , respectively, denote positive set and negative set of the i-th sample x i $x_i$ . That is, P x i $P_{x_i}$ consists of samples of the same class label to x i $x_i$ , and the N x i $N_{x_i}$ consists of samples of the different class labels to x i $x_i$ . Considering that the multi-proxy softmax (MPS) loss function can learn multiple proxies for each class, we propose a proxy-level triplet (PTRI) loss function to learn discrimination among these proxies, as follows: L P T R I = 1 C × K ∑ i = 1 C × K max w p ∈ P w i ∥ w i − w p ∥ 2 − min w n ∈ N w i w i − w n 2 + β + , \fontsize{8}{10} \begin{equation} {L_{PTRI}} = \frac{1}{{C \times K}}\sum \limits _{i = 1}^{C \times K} {{{{\left[ {\mathop {\max }\limits _{w_p \in {P_{{w_i}}}} {{{\Vert {{w_i} - w_p} \Vert} }_2} - \mathop {\min }\limits _{w_n \in {N_{{w_i}}}} {{{\left\Vert {{w_i} - w_n} \right\Vert} }_2} + \beta } \right]}}_ + }}, \end{equation} (6)where C and K, respectively, denote the number of classes and the number of proxies in each class; β > 0 $\beta > 0$ is a margin, which is also set to be 1.2; P w i $P_{w_i}$ and N w i $N_{w_i}$ , respectively, denote positive set and negative set of the i-th proxy w i $w_i$ . Combining STRI loss and PTRI loss functions, the sample-proxy dual triplet (SPDT) loss function is formulated as follows: L S P D T = λ 1 l S T R I + λ 2 l P T R I , \begin{equation} L_{SPDT} = \lambda _{1}l_{STRI} + \lambda _{2}l_{PTRI}, \end{equation} (7)where λ 1 > 0 $\lambda _{1}>0$ and λ 2 > 0 $\lambda _{2}>0$ are manually setting parameters, and they will be experimentally analyzed in the following section. Comparing the single-proxy softmax loss (SPS) loss function defined in Equation (2) and the multi-proxy softmax loss (MPS) loss function defined in Equation (3), one can see that the MPS loss function applies multi-proxies to represent a class, which could be more powerful than SPS loss function in describing complex intra-class variations. Furthermore, the sample-proxy dual triplet (SPDT) loss function simultaneously learns discrimination (i.e. enlarging inter-class distances meanwhile shrinking intra-class distances) on both sample and proxy levels, which homogenizes the proxy-level discrimination to the sample-level discrimination. As a result, a complete discrimination learning effect is acquired to improve object re-identification. 4 EXPERIMENTS AND ANALYSIS In this section, we evaluate our method on VeRi776 [37] and DukeMTMC-reID [25]. The mean average precision (mAP) [38], the cumulative match curve (CMC) [39, 40] and floating-point of operations (FLOPs) are applied as performance metrics. Rank1, Rank5 and, Rank10 denote the rank-1, rank-5, and rank-10 identification rates on a CMC, respectively. During the evaluation, batch-normalized features resulted from the batch normalization (BN) layer (see Figure 2) are used for evaluating the object re-identification performance. Setup In our experiments, the software tools are Pytorch 1.4 [41], CUDA 9.2, and python 3.6. The hardware device is one NVIDIA Tesla P40 GPU. All images are resized to 256 × 256 pixels for training and testing. The mini-batch stochastic gradient descent (SGD) optimizer [42] is used for training. Each mini-batch contains 32 subjects, and each subject consists of 4 images. The weight decays are set to be 5 × 10 − 4 $5\times 10^{-4}$ , and the momentums are set to be 0.9. There are 150 epochs for the training process. The learning rates are initialized to 3 × 10 − 4 $3\times 10^{-4}$ and linearly warmed up to 3 × 10 − 2 $3\times 10^{-2}$ in the first 10 epochs. After warming up, the learning rates are maintained at 3 × 10 − 2 $3\times 10^{-2}$ from 11st to 40th epochs. Then, the learning rates are reduced to 3 × 10 − 3 $3\times 10^{-3}$ between 41st and 70th epochs and further reduced to 3 × 10 − 4 $3\times 10^{-4}$ between 71st and 120th epochs. Finally, the learning rates are dropped to 3 × 10 − 5 $3\times 10^{-5}$ between 121st and 150th epochs. Besides, the default value for λ1 in Equation (7) is set to be 0.1. Databases VeRi776 [37] is a large-scale vehicle dataset captured under a real traffic scenario with 20 cameras. Following the evaluation protocol [37], the training subset contains 37,781 images of 576 subjects. The testing subset includes 3,257 images of the other 200 subjects. The testing subset is further divided into a probe subset of 1,678 images and a gallery subset of 11,579 images. DukeMTMC-reID [25] is a large-scale pedestrian dataset captured with 8 cameras. It contains 36,411 images of 1,812 subjects. The training subset consists of 16,522 images of 702 subjects. The testing subset includes a probe subset containing 2,228 probe images of 702 subjects and a gallery subset consisting of 17,661 gallery images of 1,110 subjects. For making a realistic evaluation, the gallery subset additionally includes 408 distracted subjects that do not appear in the probe subset. Comparison with state-of-the-art methods 4.3.1 Comparisons on VeRi776 From Table 1, it can be found that our proposed method (i.e. SPDT + MPS) is effective for vehicle re-identification task. Specifically, on VeRi776, our method acquires an 80.0% mAP and a 97.3% Rank1, which are the best results among all compared methods. Furthermore, it can be found that the proposed method outperforms state-of-the-art method who use the ResNet50 as backbone. For example, our proposed method defeats the second level method (i.e. SAVER [3]) 0.4% on mAP and 0.9% on Rank1. In addition, compared with those who use complex backbone, our proposed method still performs superior results. For example, compared with JQD3Ns [5] who uses Quadruple SDC-CNN [35] as backbone, our proposed method outperforms JQD3Ns by 18.7% in terms of mAP and 7.6% in terms of Rank1. TABLE 1. The performance (%) comparison of our method and multiple state-of-the-art approaches on VeRi776 Methods Backbone mAP Rank1 Reference SPDT + MPS ResNet50 80.0 97.3 ours SAVER [3] ResNet50 79.6 96.4 ECCV 2020 PVEN [11] SEResNeXt50 [33] 79.5 95.6 CVPR 2020 S3ANet [24] ResNet50-ibna [43] 78.5 96.6 SCIS 2021 Appearance + License Plate [18] ResNet50 78.1 95.4 ICIP 2019 Part Regularization [12] ResNet50 74.3 94.3 CVPR 2019 PAMTRI [14] DenseNet121 [44] 71.9 92.9 ICCV 2019 MidTriNet + UT [17] ResNet50 69.2 82.3 Access 2020 VANet [16] ResNet50 66.3 89.8 ICCV 2019 JQD3Ns [5] Quadruple SDC-CNN [35] 61.3 89.7 IoT 2020 JHV-DLF [1] SDC-CNN [35] 53.0 84.7 SCIS 2019 4.3.2 Comparisons on DukeMTMC-reID Table 2 shows the comparison result on DukeMTMC-reID. First of all, it demonstrates the effectiveness of the proposed method (i.e. SPDT + MPS) on pedestrian re-identification task. For example, our method also obtains the highest mAP (i.e. 77.0%) and Rank1 (i.e. 86.9%) on DukeMTMC-reID. Moreover, compared to methods those also improve the loss function, our proposed method achieve higher results. For example, the proposed method defeats AWTL[7] by 13.6% in term of mAP and 7.1% in term of Rank1. These results demonstrate the superiority of our proposed method. TABLE 2. The performance (%) comparison of our method and multiple state-of-the-art approaches on DukeMTMC-reID. Methods Backbone mAP Rank1 Reference SPDT + MPS ResNet50 77.0 86.9 ours BagTricks [19] ResNet50 76.4 86.4 CVPR 2019 PCB + PrGCN [8] ResNet50 74.6 84.7 Neurocomputing 2021 SPReID [10] Inception-V3 71.0 84.4 CVPR 2018 PCB [13] ResNet50 69.2 83.3 ECCV 2018 AWTL [7] ResNet50 63.4 79.8 CVPR 2018 JQD3Ns [5] Quadruple SDC-CNN [35] 60.8 79.4 IoT 2020 Camstyle [9] ResNet50 53.5 75.3 TIP 2019 Analysis 4.4.1 Comparison with existing loss functions As shown in Table 3, the compared loss functions include STRI [30] + MPS [31], STRI [30] + SPS [23], WRT [4] + SPS [23], BagTrick Loss [19], View-aware triplet [16], MPS [31], and SPS [23]. Among these loss functions, STRI [30] and WRT [4] are two common sample-level triplet loss functions in re-identification. Also, SPS [23] and Center loss [45] are frequently used loss functions not only in re-identification but also in classification/recognition tasks [23, 46-48]. Most important, the combination of STRI and SPS (i.e. STRI+SPS) is baseline and strong loss design in current re-identification research [18, 19]. The BagTrick Loss includes STRI [30], SPS [23] and Center Loss [45]. The View-aware Triplet [16] contains similar-view, different-view and cross-view triplet loss functions, but it still is a sample-level triplet loss functions. TABLE 3. The performance of comparison among our loss function and existing loss functions loss function VeRi776 [37] DukeMTMC-reID [25] mAP Rank1 Rank5 mAP Rank1 Rank5 Ours (SPDT + MPS) 80.0 97.3 98.7 77.0 86.9 94.9 STRI [30] + MPS [31] 79.8 96.8 98.6 74.7 86.2 94.2 STRI [30] + SPS [23] 79.5 96.5 98.3 73.8 85.5 93.4 WRT [4] + SPS [23] 79.6 96.0 98.2 73.8 85.6 93.6 BagTrick Loss [19] NA NA NA 76.4 86.4 NA View-aware Triplet [16] 66.3 89.9 96.0 NA NA NA MPS [31] 76.2 95.5 98.2 72.0 84.2 92.6 SPS [23] 79.2 96.0 98.0 71.0 83.5 92.6 From Table 3, there are three interesting observations. First, our proposed loss function (i.e. SPDT+MPS) obtains the best performance. For example, comparing with BagTrick Loss [19], our SPDT+MPS gets 0.6% higher mAP and 0.5% higher Rank1 on DukeMTMC-reID. Our proposed SPDT+MPS defeats View-aware Triplet [16] by 13.7% mAP, 7.4% Rank1 and 2.7% Rank5 on VeRi776. Comparing with WRT [4] + SPS [23], the proposed SPDT+MPS gets 0.4% higher mAP on VeRi776 and 3.2% higher mAP on DukeMTMC-reID, respectively. Second, both combining with the multi-proxy softmax (MPS) loss [31] function, our sample-proxy dual triplet (SPDT) loss function is superior to the traditional sample-level triplet (STRI) loss [30] function. Specifically, on both VeRi776 and DukeMTMC-reID, the SPDT loss function acquires the highest mAP, Rank1, and Rank5. For example, on DukeMTMC-reID, SPDT loss function outperforms STRI loss function by 2.3%, 0.7% and 0.7% in terms of mAP, Rank1 and Rank5, respectively. These results demonstrate that our SPDT loss function simultaneously learning discrimination on sample and proxy levels is more comprehensive than the traditional STRI loss function, resulting in better object re-identification performance. Third, the multi-proxy softmax (MPS) [31] loss function outperforms the single-proxy softmax (SPS) [23] loss function. Specifically, both cooperating with the STRI loss function, the MPS loss function acquires better performance than the SPS loss function on both VeRi776 and DukeMTMC-reID. For instance, on VeRi776, mAP, Rank1, and Rank5 of the MPS loss function are 79.8%, 96.8%, and 98.6%, respectively, while mAP, Rank1, and Rank5 of the SPS loss function are 79.5%, 96.5%, and 98.3%, respectively. In addition, when neither the MPS loss function nor the SPS loss function does not cooperate with the STRI loss function, the MPS loss function also is superior to the SPS loss function. These comparison results show that the MPS loss function considering complex intra-class variations is stronger than the SPS loss function simply assuming each class of one proxy. 4.4.2 Role of the proxy-level triplet loss We consider that the PTRI loss function is newly designed in this work, while the STRI loss function has been studied well in many works. Therefore, we construct a series of experiments to evaluate the role of the proxy-level triplet (PTRI) loss function. Specifically, we change the PTRI loss function's weight (i.e. λ2 in Equation 7), and fix the STRI loss function weight to be 0.1. Besides, the number of proxies of the MPS loss function is fixed to be 5. The results are shown in Table 4 and Figure 3. One can see that the proposed PTRI loss function achieves the best performance on VeRi776 at λ 2 = 0.1 $\lambda _{2}=0.1$ , and on DukeMTMC-reID at λ 2 = 1 $\lambda _{2}=1$ . It is clear that the best λ2 value on VeRi776 is smaller than the best λ2 value on DukeMTMC-reID. This is because the VeRi776 has more complex intra-class variation (e.g. different viewpoints), leading to more sensitivity of PTRI loss function on VeRi776. From Table 4, due to the change of λ2, there are some performance fluctuations on VeRi776 and DukeMTMC-reID. For instance, when adjusting λ2 from 0.1 to 1, Rank1 on VeRi776 decreases from 97.3% to 96.2%. Furthermore, from Figure 3, we can see that the fluctuation on VeRi776 is stronger than the fluctuation on DukeMTMC-reID. This is because the scale of VeRi776 is larger than DukeMTMC-reID, making re-identification on VeRi776 more complex. TABLE 4. The performance (%) of using different PTRI loss weights (i.e. λ2) in the total loss function of Equation (7) on VeRi776 and DukeMTMC-reID λ2 VeRi776 [37] DukeMTMC-reID [25] mAP Rank1 Rank5 Rank10 mAP Rank1 Rank5 Rank10 0 79.8 96.8 98.6 99.1