# Joint Object Detection and Multi-Object Tracking Based on Hypergraph Matching
## 1 Introduction
However, as shown in Figure 1, due to the difference in features required by the object detection task and the Re-ID task, there is a competitive phenomenon between the detection task and the Re-ID task within the JDE paradigm; for the object detection task, the model aims to increase the similarity of appearance features of objects belonging to the same category, i.e., to narrow the intra-class distance, while for the Re-ID task, the model aims to maximize the differences in appearance features between different objects, i.e., to expand the inter-class distance. In addition, when the tracked objects are occluded by adjacent or cluttered objects, especially in severe occlusion, the reliability of the features detected will decrease, making it difficult to maintain the visual consistency of the occluded objects. Moreover, because most of the association modules in the above two paradigms use the Hungarian matching data association algorithm, the errors generated in the data association module cannot be propagated back to the object detection module and the feature embedding module, preventing the formation of an end-to-end optimization. Figure 1: (**a**) Input images for the MOT task. (**b**) Schematic representation of the object tracklets. Existing methods often fail when multiple objects with similar appearance or motion patterns or occlusion appear in proximity. In response to the above issues, we propose an MOT method (JDTHM) based on hypergraph matching association, which enhances object detection and feature extraction through the tracklet information formed by the data association module in an end-to-end tracking model to form positive feedback. Specifically, the end-to-end tracking model uses tracklet information to enhance the visual features of object detection, thereby reducing false positives and false negatives. At the same time, the enhanced visual features assist the feature extraction module in generating discriminative features, thereby reducing the impact of occlusion. Based on more robust detection results and aggregated features, the data association module can generate higher quality tracklets. The main contributions are as follows:  
* We introduce a feature extraction and detection module to initially achieve object localization and description of detection results. * We design a deep feature aggregation module that extracts temporal information from historical tracklet information, aggregates features from object detection and feature extraction, enhances the consistency area between the current frame features and tracklet features, and avoids the impact of identity swapping and tracklet breaks caused by object detection loss or distortion. * We construct a data association module based on the hypergraph neural network, integrating with object detection and feature extraction into a unified network. This integration transforms the data association problem into a hypergraph matching problem between the tracklet hypergraph and the detection hypergraph, thereby achieving end-to-end model optimization. * The hypergraph framework demonstrates highly competitive performance on three MOT datasets, confirming its effectiveness in enhancing the accuracy of change detection. The remainder of this paper is organized as follows: Section 2 reviews the related work; Section 3 introduces the relevant theories of hypergraph matching; Section 4 provides a detailed introduction to the proposed hypergraph architecture; Section 5 presents the experimental results and discussion; Section 6 concludes the paper.