# Instance Hard Triplet Loss for In-video Person Re-identification
## 3 Our Approach
### Instance Hard Triplet Loss
Meanwhile, to distinguish all persons correctly, we should also force the representation of persons in the same image to be different. Based on the above observation, we come up with a new loss function with the following form:  
\[L_{IHT}=\sum_{p=1}^{p}\left[\max_{k=1..K}\max_{i=1..K}D(x_{p}^{k},x_{p}^{i})- \min_{j=1...K}\min_{ \begin{subarray}{c}n\in\mathbb{F}_{j}\\ n\
eq p\end{subarray} }D(x_{p}^{j},x_{n}^{j})+\alpha\right]_{+} \tag{3}\]  
where \(x_{p}^{k}\) is the feature of the \(p^{\text{th}}\) person in the \(k^{\text{th}}\) image, \(K\) is the number of images, \(P\) is the number of persons that appear in all \(K\) images, and \(\mathbb{P}_{j}\) is a set of all persons that show up in the \(j^{\text{th}}\) image. Recalling Figure 4, the proposed instance triplet hard loss in Equation (3) selects \(P\) persons appearing in all \(K\) images, and three constraints are required: (1) features of each person should be consistent across all images; (2) features of all persons in the same image should be distinguishable; and (3) even the hardest positive sample should still be more similar to the anchor than the closest negative sample in every image by a predefined margin \(\alpha\). The above motivation is clear, and the formulation is in line with our intuition. This loss functions also show much flexibility and can be generalized to an image-based cross-camera problem, where we can group a \(P\times K\) batch into \(K\) groups with \(P\) different persons in each group. The variant can be formulated as follows:  
Figure 4: The motivation of our instance hard triplet loss is to keep features consistent across frames and distinguishable with other persons in the same frame. \(I_{p}^{k}\) is the \(p^{\text{th}}\) person in the \(k^{\text{th}}\) image and \(x_{p}^{k}\) is the corresponding feature. Note that a variable number of persons shows up in every frame, because persons often appear and disappear in a video. \[L^{\prime}_{IHT}=\sum_{p=1}^{P}\left[\max_{k=1...K}\max_{i=1...K}D(x^{k}_{p},x^{i}_ {p})-\min_{j=1...K}\min_{ \begin{subarray}{c}n=1...P\\ n\
eq p\end{subarray} }D(x^{i}_{p},x^{i}_{n})+\alpha\right]_{+} \tag{4}\]  
A visualization is shown in Figure 3b, where we only compare negative samples in the same group. Different from batch hard triplet loss, instance hard triplet loss only compares the anchor person with persons in the same image rather than all negative samples. For \(K\) images with average \(P\) persons in each image, we compare the anchor sample with \(K\) positive samples and \(P\) negative samples, rather than \(P\times(K-1)\) negative sample in batch hard triplet loss. Thus, the proposed loss is more balanced between positive and negative pairs. On the other hand, we generate only one triplet for each person with comparison times of \(K^{2}\) and \(P\times K\) for positive and negative pairs, respectively. Therefore, for a batch with \(P\) person, the computational complexity is \(\mathcal{O}(PK(P+K))\). Since we exploit all instances of each person and find the hardest triplet, we call it instance hard triplet loss. In summary, compared with the widely-used triplet batch hard loss, our proposed loss is:  
1. more balanced between positive and negative pairs;
2. faster with a smaller computational complexity of \(\mathcal{O}(PK(P+K))\) compared to \(\mathcal{O}(P^{2}K^{2})\);
3. competitive in performance, which can be seen in the experimental part of this paper;
4. more general for both the cross-camera and in-video ReID problem, the image-based and video-based ReID problem, as well as training samples with an indefinite quantity persons.