## 2 Proposed Vehicle Detection and Tracking System
### Headlight Pairing and Taillight Pairing  
After performing the headlight and taillight tracking process, the set of vehicle light trackers is retrieved. As the vehicle lights of one vehicle are located in pairs and symmetrical, the light tracker set is used to determine the position of on-road vehicles by pairing the vehicle lights. Existing works [7; 19; 20] only use the spatial features within one frame, such as the area, width, height, vertical coordinate, and correlation, to check the similarity between two vehicle lights; these features are not sufficient for pairing lights. When the multiple vehicles in front are moving at the same distances from the host vehicle, incorrect light pairing may occur because the spatial features of the lights in these vehicles are similar. To reduce the false detection rate, a light pairing method using spatiotemporal information is proposed. The proposed light pairing method is separated into two stages: potential light pair generation and light pair verification. To generate the set of candidate light pairs, the pairing criteria are given as follows:  
1. Two vehicle lights are highly overlapped in vertical projections: \[O_{v}(TL_{i}^{t},TL_{j}^{t})>\tau_{vp}\] (9)
2. Two vehicle lights have similar heights: \[\frac{min\{H(TL_{i}^{t}),H(TL_{j}^{t})\}}{max\{H(TL_{i}^{t}),H(TL_{j}^{t})\}}> \tau_{h}\] (10)
3. The ratio of pair width to pair height must satisfy the following condition: \[\tau_{WH1}\leq\frac{max\{r(TL_{i}^{t}),r(TL_{j}^{t})\}-min\{l(TL_{i}^{t}),l(TL _{j}^{t})\}}{max\{b(TL_{i}^{t}),b(TL_{j}^{t})\}-min\{t(TL_{i}^{t}),t(TL_{j}^{t} )\}}\leq\tau_{WH2}\] (11)  
In the criteria \(\tau_{vp}\), \(\tau_{h}\), \(\tau_{WH1}\), and \(\tau_{WH2}\) are the thresholds that determine the characteristics of pairing vehicle lights. These values are statistically chosen as 0.7, 0.7, 2.0, and 14.0, respectively. The result of the first stage is the set of candidate light pairs, but this set may include some falsely detected pairs because one vehicle light can be shared in more than one pair. Therefore, light pair verification is performed to remove the false pairs. To reduce the false pairs, a _pairing score_ of the candidate light pair is used to perform a comparison between the conflicting pairs. Among the conflicting pairs, the pair with the highest _pairing score_ is retained to describe a vehicle, and the other pairs are removed from the candidate pair set. The _pairing score_\(S_{p}\) is a combination of spatiotemporal features of two vehicle lights (\(TL_{i}^{t}\) and \(TL_{j}^{t}\)) in a pair, including the number of tracked frames, displacement, size, and correlation. It is defined as follows:  
\[S_{p}=w_{0}\cdot r_{t}+w_{1}\cdot r_{d}+w_{2}\cdot r_{s}+w_{3} \cdot r_{corr}\] \[r_{t}=\frac{min\{|TL_{i}^{t}|,|TL_{j}^{t}|\}}{max\{|TL_{i}^{t}|,|TL _{j}^{t}|\}}\] \[ \begin{cases}r_{d}=\frac{min(t_{i}d_{j})}{max(d_{i}d_{j})}\\ d_{i}=\sum_{k=t-min(3,|TL_{i}^{t}|)}^{t}\sqrt{(vx_{i}^{k})^{2}+(vy_{i}^{k})^{2} }\\ d_{j}=\sum_{k=t-min(3,|TL_{i}^{t}|)}^{t}\sqrt{(vx_{i}^{k})^{2}+(vy_{i}^{k})^{2 }}\end{cases}  \tag{12}\] \[r_{s}=\frac{1}{2}\times\{\frac{min\{W(TL_{i}^{t}),W(TL_{j}^{t})\}}{max\{ W(TL_{i}^{t}),W(TL_{j}^{t})\}}+\frac{min\{H(TL_{i}^{t}),H(TL_{j}^{t})\}}{max\{H(TL_{i}^{t}),H( TL_{j}^{t})\}}\}\]  
where \(r_{t}\) is the ratio of the number of tracked frames of two vehicle lights; \(r_{d}\) is the ratio of the total displacements \(d\) of vehicle lights (\(TL_{i}^{t}\) and \(TL_{j}^{t}\)) in the four most recent frames. As both lights in a pair should move coherently in time, their motion vectors are used to calculate the displacements; \(r_{s}\) is the ratio of the sizes of two vehicle lights; \(r_{corr}\) is the correlation between histograms of vehicle lights \(TL_{i}^{t}\) and \(TL_{j}^{t}\). Herein, the Bhattacharyya coefficient [13] is used to compare the 3-D histograms of the left and right lights. In Equation (12), the coefficients \(w_{0},w_{1},w_{2}\), and \(w_{3}\) are experimentally chosen as 0.2, 0.2, 0.3, and 0.3, respectively.