# Instance Hard Triplet Loss for In-video Person Re-identification
## 3 Our Approach  
### ReID-Head Network  
To extract features of multiple persons in an image frame, the regular solution is first to use a human detector to detect the human body area and then feed those bounding box images into another ReID network to obtain the final representation. However, in this way, detection and ReID cannot be combined properly, and this two-stage paradigm does each job without reusing features.  
Efficiency is an important factor in a video-based real-time system, and the detection and ReID feature extracting can share the low-level features to speed up. An attempt of this idea was proposed in Person Search [40; 41], which adopts a Faster R-CNN-like [42] structure, first generating candidate bounding boxes from anchor, then feeding the ROI-pooling [43] feature to an identification net to get the ReID features and refined detection bounding boxes. Features are reused for both detection and ReID; however, there are many anchor boxes, and each will feed into an identification net separately without sharing features. As the detection area keeps advancing, this fixed anchor-based structure cannot be integrated with some newly proposed detection methods such as CornerNet [44] and YOLOv3 [45].  
In this paper, we propose a new standalone ReID module consisting of a convolutional ReID-Head and an ROIAlign [46] layer. It begins from the mid-level feature maps of a detection/pose model and then produces its final representation for the entire input image by a light-weight convolution network, as shown in Figure 2. The detection trunk network generates final bounding boxes, and an ROIAlign [46] is applied for each bounding box to obtain the final ReID features.  
In this paper, we adopt a Single Shot Detector (SSD) network [47] as a case study. As shown in Figure 2, SSD [47] uses a VGG network [48] as the backbone, and we insert our ReID-Head module in the output of conv4_3, where the first classifier of SSD connects. As for our ReID-Head, we use three ResNet [49] bottlenecks followed by a \(1\times 1\) convolution to reduce the channel number to 10, and the ROIAlign [46] size is \(5\times 5\), which makes the final feature vector for ReID have 250 dimensions.  
Because the ReID module is a standalone part, the backbone in Figure 2 can be replaced by the more complicated ResNet-50 [49], ResNet-101 [49] or ResNeXt-101 [50], as in [46], while the detection head can be replaced by other detection heads like Feature Pyramid Networks (FPN) [51]. More complicated backbones and detection heads can further improve the performance, but that is not the focus of this paper. We choose the VGG [48] backbone and SSD [47] head as a case study in this paper.  
There are several advantages of our proposed ReID-Head design:  
1. It can jointly get detection and ReID results in a single network, and the network can be trained end-to-end.
2. The ReID-Head module can be integrated into both one-stage detectors like YOLO [52] and CornerNet [44] and two-stage detectors like Faster R-CNN [42] and Mask R-CNN [46], as well as keypoint detectors like OpenPose [53; 54], benefiting from the progress in the detection area and improving.
3. It has high efficiency with feature reusing. Furthermore, the computational cost of ReID features will not linearly increase with more anchor boxes as in Person Search [40; 41].