# A Vision Detection Scheme Based on Deep Learning in a Waste Plastics Sorting System
## 3 Materials and Methods
### Multiple Object Tracking  
#### 3.3.1 DeepSORT  
For the input video sequence, information such as bounding box, confidence, and feature is output by the detector, which is filtered according to the confidence and input to the tracker for matching. The core of the DeepSORT algorithm lies in recursive Kalman filtering and frame-by-frame data association. Kalman filtering is an optimal estimation algorithm capable of generating a respective tracker for each object, which is used to estimate the position where each object will be located in the next frame. The Kalman filter predicts the trace of the object based on constant velocity motion and a linear observation model. It uses eight parameters to describe the state of the object at a certain time:  
\[X=\left(u,v,r,h,\dot{x},\dot{y},\dot{r},\dot{h}\right), \tag{7}\]  
where \((u,v)\) represents the 2D pixel location of the bounding box's center. \(r\) and \(h\) are the aspect ratio and height of the bounding box, respectively. The remaining variables represent the motion information of the object in the image coordinate system.  
DeepSORT adopts a certain rule to match and associate the detection results with the tracking results. The Kalman filter uses the detection results of the previous frame to output the prediction boxes of all objects in the current frame, denoted as the set \(U\). The object detection model outputs the prediction boxes of all objects in the current frame, denoted as the set \(V\). The process of establishing the matching relationship for prediction boxes in \(U\) and \(V\) can be regarded as a bipartite graph problem, which can be efficiently solved using the Hungarian algorithm with minimal matching cost. In terms of data association, both the motion information and the appearance information of the object are considered to improve the matching effect of the Hungarian algorithm and reduce the number of identity switches.  
The motion information is associated by using the Mahalanobis distance:  
\[d^{(1)}(i,j)=\left(d_{j}-y_{i}\right)^{T}\!S_{i}^{-1}\!\left(d_{j}-y_{i}\right), \tag{8}\]  
where \(d_{j}\) and \(y_{i}\) represent the \(j\)th bounding box detection and the \(i\)th track distribution, respectively. \(S_{i}\) is the covariance matrix.  
When motion uncertainty is high, using only the Mahalanobis distance is prone to association errors. Therefore, DeepSORT introduces appearance information for association, and measures the smallest cosine distance between the \(i\)th track and \(j\)th detection:  
\[d_{i,j}^{(2)}=\min\left[1-r_{j}^{T}r_{k}^{(i)}\right], \tag{9}\]  
where \(r_{j}\) denotes the appearance feature vector of the \(j\)th object, and \(r_{k}^{(i)}\) denotes the \(k\)th feature vector in the corresponding feature set of the prediction box of the \(i\)th track.  
When both the Mahalanobis and cosine distances meet their respective thresholds, \(\lambda\) is a hyperparameter, and the final metric is obtained by linear weighting:  
\[c_{i,j}=\lambda d^{(1)}(i,j)+(1-\lambda)d^{(2)}(i,j), \tag{10}\]  
In addition, DeepSORT introduces a matching cascade that gives priority to more frequently seen objects to ensure the continuity of tracking. The Kalman filter will update and output tracking results when the matching is successful, while the unmatched detection result and the unmatched tracking result will be matched by intersection over union.