# Multi-Vehicle Tracking Based on Monocular Camera in Driver View
###### Abstract  
Multi-vehicle tracking is used in advanced driver assistance systems to track obstacles, which is fundamental for high-level tasks. It requires real-time performance while dealing with object illumination variations and deformations. To this end, we propose a novel multi-vehicle tracking algorithm based on a monocular camera in driver view. It follows the tracking-by-detection paradigm and integrates detection and appearance descriptors into a single network. The one-stage detection approach consists of a backbone, a modified BiFPN as a neck layer, and three prediction heads. The data association consists of a two-step matching strategy together with a Kalman filter. Experimental results demonstrate that the proposed approach outperforms state-of-the-art algorithms. It is also able to solve the tracking problem in driving scenarios while maintaining 16 FPS on the test dataset. multi-vehicle tracking; object detection; data association; Kalman filter +
Footnote †: journal: _aciences_  
detection algorithms. The Faster-RCNN [8] is a representative work of a two-stage detector. On the contrary, the YOLOv3 [9] is a classic one-stage object detection approach, which ignores the region proposal stage and replaces it by a single network to directly output the predicted bounding boxes and classification scores in the image. Although two-stage detection algorithms are generally more accurate than the one-stage detection approaches when detecting small targets, the one-stage detectors are faster. In this way, a one-stage detector is useful for vehicles detection whose size is generally larger within a safe distance between the ego-vehicle and surrounding vehicles. Data association (DA) is the other part of the TBD approach. Its objective is to match detection results between adjacent frames of the video stream based on some metrics, such as the cosine distance and Intersection-over-Union (IoU) distance. To improve matching accuracy, the motion state prediction and linear assignment of the object are studied by some studies in the literature. Kalman filter [10] is an important method to solve the motion model, which includes the prediction and updating step. First, it predicts the object state based on the system state space model. Then, it corrects the prediction using the received measurements. Moreover, the optimal Hungarian algorithm [11] is normally employed to solve the linear assignment problem. Each object in the frame is assigned a Kalman filter. Thus, the number of Kalman filters and the dimension of the cost matrix which is fed into the Hungarian algorithm increase as the number of tracking objects increases. In this work, we focus on multiple vehicle tracking problems. The proposed method only uses the current and previous images that come from a monocular camera in driver view. It can track multiple vehicles in as real time as possible based on the single-shot TBD method. Here are our contributions:  
* Following the TBD paradigm, we design a detector, which is composed of a backbone block, a neck layer, and three prediction heads, to enhance the multi-vehicle tracking (MVT) performance;
* For data association, we derive the Kalman filter that is used to update the tracks, which decreases the identity switch to improve the HOTA and other relevant metrics;
* Quantitative and qualitative evaluations demonstrate that our method achieves good performance and can solve the multiple vehicle-tracking problems in the driver view with monocular cameras (e.g., illumination variants and deformations) while maintaining a good frame rate. The paper is structured as follows: Section 2 mainly introduces the existing related research work. Section 3 presents the general structure of the proposed algorithm and the derivation of Kalman filtering. Section 4 introduces the implementation details, including the acquisition of anchor setting, training and testing details of the algorithm. Section 5 shows the experimental results compared with state-of-the-art algorithms on public datasets.