# Joint Object Detection and Multi-Object Tracking Based on Hypergraph Matching
## 2 Related Works
### Joint Detection and Embedding  
In recent years, the Joint Detection and Embedding (JDE) frameworks have reduced the complexity of the DBE frameworks while also improving the accuracy of MOT [35; 36; 37]. Its strategy is to integrate some functional modules to a certain extent on the basis of DBE, reducing the algorithmic complexity brought by phased processing while increasing the coupling between functional modules, alleviating the problem of local optimization brought by manually connecting each module. In the detection algorithm, a head network is added to infer the whole picture information through an integrated network, then extract the corresponding pixel points of the object on the feature map as the object's appearance feature vector and output the object's detection box at the same time. JDE uses a YOLOv3 framework [38], adding an Re-ID branch parallel to the detection branch and extracting the feature vector at the center point of the positive anchor box on the feature map output by the branch as the object's appearance feature vector. However, as mentioned in FairMOT [11], the predicted anchor center may not fall within the object area, so the anchor box mechanism is not suitable for training appearance features; they built tracking algorithms on CenterNet [39] by using Deep Layer Aggregation (DLA) [40] in the backbone architecture and extracting feature vectors at the object center point to learn appearance features. One of the main challenges of the joint learning of detection and tracking comes from the conflict between these two tasks. The purpose of the detection task is to identify object categories from the background, such as pedestrians and vehicles, hoping to improve the similarity of appearance features belonging to the same category of objects, that is, to narrow the intra-class gap. While the purpose of the Re-ID embedding is to distinguish different objects, not categories, hoping to maximize the appearance feature differences between different objects, that is, to expand the intra-class gap. Therefore, some methods decouple the feature maps of different tasks. For instance, [PERSON] et al. [41] use a separate Re-ID head to learn appearance features in addition to the original detection branch. CSTrackv2 [12] learns the correlation and difference information between detection and re-identification through a channel mutual attention mechanism, and then use a multi-scale attention module to extract appearance features. SimpleTrack [42] is based on the FairMOT model and uses a feature fusion branch different from detection to extract appearance features. The design of these modules has increased the modeling of feature maps in the head network, enriched the differences between the respective task heads and the original feature maps, and improved the detection performance and recognition performance of the algorithm. Similarly, the appearance features extracted in a single frame image cannot obtain temporal information, so some researchers have used a multi-frame feature extraction method. For example, GSDT [43] is based on the FairMOT algorithm, using a graph convolutional model to model the local correlation between tracklet features and the current feature map, making the extracted appearance features contain historical information and improving the adaptability to object occlusion and deformation. However, since most of the association modules use the Hungarian matching data association algorithm, the errors generated in the data association module cannot be propagated back to the object detection module and the feature embedding module, and an end-to-end optimization cannot be formed. The feature prediction and detection phase of JDE are carried out simultaneously, through the addition of frame information in the detection network or multi-frame detection.