# Instance Hard Triplet Loss for In-video Person Re-identification
###### Abstract  
Traditional Person Re-identification (ReID) methods mainly focus on cross-camera scenarios, while identifying a person in the same video/camera from adjacent subsequent frames is also an important question, for example, in human tracking and pose tracking. We try to address this unexplored in-video ReID problem with a new large-scale video-based ReID dataset called PoseTrack-ReID with full images available and a new network structure called ReID-Head, which can extract multi-person features efficiently in real time and can be integrated with both one-stage and two-stage human or pose detectors. A new loss function is also required to solve this new in-video problem. Hence, a triplet-based loss function with an online hard example mining designed to distinguish persons in the same video/group is proposed, called instance hard triplet loss, which can be applied in both cross-camera ReID and in-video ReID. Compared with the widely-used batch hard triplet loss, our proposed loss achieves competitive performance and saves more than 30% of the training time. We also propose an automatic reciprocal identity association method, so we can train our model in an unsupervised way, which further extends the potential applications of in-video ReID. The PoseTrack-ReID dataset and code will be publicly released. person ReID; video; triplet loss; pose; unsupervised learning +
Footnote †: journal: Medical  
impossible to obtain. Furthermore, even if this cross-camera ReID dataset is available, we still cannot directly use those cross-camera ReID data for person tracking within the same video sequence, because person tracking within the same video needs to identify persons within the same video, which is different from traditional cross-camera ReID that identifies persons across multiple cameras. In this paper, we refer to person ReID within the same video as in-video ReID. As shown in Figure 1, the traditional cross-camera ReID task searches for the same person from images captured by different cameras at different times, while the in-video ReID task searches for the same person from subsequent frames of the same video. In this paper, we try to address the aforementioned in-video ReID problem that identifies a person in the same video sequence. We argue that traditional cross-camera ReID and in-video ReID are dissimilar in the following aspects:  
1. Single-camera view: Cross-camera ReID needs to identify persons appearing in multiple cameras; thus, it forces the network to extract features consistent in all camera views and drop the camera-specific features, leading to limited available features, while in-video ReID can fully utilize all features. 2. Short-term: Cross-camera ReID tries to construct a long-term association that inevitably discards transient clues. On the contrary, we will demonstrate that those transient clues are very critical for in-video ReID. 3. Background: As we mentioned above, temporary clues like the background always act as a distractor, and those features are often discarded in cross-camera ReID. For instance, MGCAM [9] learns to predict a foreground mask of the human body area to suppress background distraction. However, for in-video ReID, the background of a person usually does not change dramatically within a few subsequent frames and can help to distinguish people with a similar appearance. 4. Pose: The pose is also regarded as a distraction, and pose-unrelated features are preferred in cross-camera ReID. For example, the recent FD-GAN [10] use Generative Adversarial Nets (GAN) [11] to learn pose-unrelated representations. Figure 1: Illustration of (**a**) traditional cross-camera ReID and (**b**) the in-video ReID addressed in this paper. Cross-camera ReID retrieves images of the same person from different cameras, while in-video ReID finds the same person in the subsequent frames of the same video. The blue bounding boxes in each sub-figure indicate the same person. Similar looking persons often exist in the same video frame, engaged in the same activity, as shown in Figure 1b.