# Joint Object Detection and Multi-Object Tracking Based on Hypergraph Matching
## 2 Related Works  
### Detection-Based-Embedding  
Online MOT algorithms based on deep learning are divided into two frameworks: detection-based tracking and joint detection tracking, with the difference being the presence or absence of a tracking module integrated with the object detection network. MOT algorithms based on deep object detectors [15; 16; 17] incorporate high-performance deep learning object detectors in the detection part of the MOT process, such as Region-based Convolutional Neural Networks (R-CNNs) [18; 19], Single Shot Multibox Detector (SSD) [20], and You Only Look Once (YOLO) [21; 22], to enhance the performance of online MOT algorithms. The typical Simple Online and Real-time Tracking (SORT) algorithm [15] is based on the traditional Hungarian association algorithm and uses the Faster R-CNN object detection network to replace the original Aggregate Channel Feature (ACF), achieving a significant improvement in tracking accuracy and speed. Subsequent research [6] has also shown a high positive correlation between detection accuracy and online MOT performance. MOT algorithms based on deep object detectors generally take good object detection results as the pre-input and make improvements from the perspectives of appearance feature extraction, object feature prediction, and data association. Compared to some online trackers [15; 23; 24] that only use simple motion modeling, the appearance features of the object can more robustly track objects that are occluded for a long time. For example, DeepSORT [25] uses a pre-trained ResNet network [26] on a large-scale dataset to extract the appearance features of the object, and then incorporates the similarity measurement of appearance features into the association cost for association, reducing the problem of object identity switching during the tracking process. To better simulate temporal information, some algorithms, independent of the detector, use the encoded vectors of the object's geometric features (position, length, width, etc.) in historical framesand the appearance feature vectors and predict the change of the feature vectors through a temporal model for data association. For example, Exponential Moving Average (EMA) [10] aggregates the tracklet features to prevent occlusion and outlier feature points from affecting the match. [PERSON] et al. [27] connect the average appearance features with spatiotemporal features and then use Graph Convolutional Networks (GCNs) [28] to aggregate features to obtain long-term appearance features. In addition, for data association based on matching, a cost matrix is constructed through the similarity of tracklet features and detection features, and then the allocation matrix is solved, with the allocation matrix supervising the learning of motion features and appearance features. For instance, DeepMOT [29] proposes a deep Hungarian network, using a Bidirectional Recurrent Neural Network (BRNN) [30] to convey global information in the cost matrix, and its loss function is designed based on two differentiable evaluation indicators, making the allocation matrix output by the network more conducive to the improvement of the indicators. [PERSON] et al. [31] further integrate the object's position in the previous frames and appearance features to obtain better global spatiotemporal information, alleviating the problem of object feature variation over time. Each module of the detection-based tracking algorithm can be optimized separately, and with the improvement of the performance of each module the algorithm can achieve a higher tracking accuracy [32; 33; 34]. However, the coupling between modules is not high, and the independent training method of multiple modules may cause the model parameters to converge to the optimum of each module, rather than the global optimum of MOT.