# Multi-Object Detection for Inland Ship Situation Awareness Based on Few-Shot Learning
## 5 Multi-Object Tracking for Inland Ships  
Deep Sort [37] is an improvement based on Sort object tracking. The core of the Sort algorithm is Hungarian matching and Kalman filtering. In addition to the core part of the Sort module, DeepSORT also introduces a deep learning model ReID module. In the process of real-time object tracking, the appearance features of the target are extracted for nearest proximity matching. In the current frame, the minimum cosine distance between all Feature vectors of the \(i\)th object tracking and the detection of the \(j\)th object are calculated. The object tracking match can achieve a real-time tracking effect and reduce ID switching, which can be applied to industrial development. In this paper, to achieve a relatively continuous tracking effect for the input ship video data, we introduced the DeepSORT algorithm to combine it with the YOLOv5 model. The general process of the DeepSORT algorithm is as follows: First, the prediction video or continuous image is input, and the confidence after prediction and position information obtained from our improved YOLOv5 detection network are taken as input. The Kalman filter first determines whether the track exists. If a track exists, the prior probability prediction is made for its position information. After that, the prior probability prediction obtained is cascaded matching and IoU matching. Finally, the matching list is obtained. The specific process of the algorithm is shown in Figure 5.  
Figure 5: DeepSORT framework. The specific algorithm steps are divided into the following steps:  
First, the original video frame is input into our YOLOv5 detection network, from which we can obtain the location of the detection result and its confidential information. The Kalman filter determines whether the track exists. It makes a prior probability prediction for its position information if it exists and abandons it if it does not exist. Secondly, based on the mean value and variance of the previous moment, the forecast track predicts the mean value and variance information of the track at the next moment and updates the position and speed. The Kalman filter update formula is as follows:  
\[\hat{\chi}_{k}^{-}=A\hat{\chi}_{k-1}+B\mu_{k} \tag{2}\]  
\[P_{k}^{-}=AP_{k-1}A^{T}+Q \tag{3}\]  
where \(\hat{\chi}_{k}^{-}\) represents the state at time \(k\) predicted from time \(k-1\). Thirdly, after the prior prediction is obtained, the detection that predicted the location of YOLOv5 and the predicted location of the track are matched. Use the cascade matching policy. The matching policy divides the tracks into Confirmed, Tentative, and Deleted types. Cascade matching only matches the confirmed track. The cost matrix of cascade matching combines the cosine similarity distance and the Mahalanobis distance, and the measurement formula of the Mahalanobis distance matching is as follows:  
\[d^{(1)}(i,j)=\big{(}d_{j}-y_{i}\big{)}^{T}S_{i}^{-1}\big{(}d_{j}-y_{i}\big{)} \tag{4}\]  
where \(d_{j}\) represents the position of the \(j\)th detection box, \(y_{i}\) represents the predicted position of the target by the \(i\)th tracker, and \(S_{i}\) represents the covariance matrix between the detection position and the average tracking position. The Mahalanobis distance takes the uncertainty of state measurement into account by calculating the standard deviation between the detection and the mean tracking positions. Because the Mahalanobis distance association method will be invalid when the camera moves, this will lead to the rapid switching of the ID of the tracking target. To make the ID switching stable, cosine distance matching is also introduced, which combines the appearance features of the object detection frame with the features extracted from the REID network. The formula is as follows:  
\[d^{(2)}(i,j)=min\Big{\{}1\Big{|}-r_{j}^{T}r_{k}^{(1)}\Big{|}r_{k}^{(i)}\in R_{ i}\Big{\}} \tag{5}\]  
where \(R\) stands for appearance feature vector library and \(r\) stands for feature vector extracted for \(d\) detection blocks.