## 3 Methodologies
### Trajectory Generation
#### 3.2.2 Pedestrian Tracker  
In the previous section, we described how a set of bounding boxes can be obtained using a single-stage or two-stage detector for every image frame. CCTV video can be viewed as a sequence of image frames over time. In this section, we describe how a tracker can be used to associate pedestrians with the additional temporal dimension. We note that both SORT and DeepSORT trackers are publicly available for use [28; 29]. While further discussions of SORT and DeepSORT are available in these references, in this subsection, we describe the algorithms and the modifications that we made to deal with datasets from real-world scenarios that may have a low frame rate and lack annotations. To obtain the precise trajectory of a single object across multiple video frames, it is necessary to have a tracker algorithm that associates each bounding box to an identity. The tracking stage can be formulated as an assignment problem: At any time \(t\), the tracker must associate each bounding box in the set of all bounding boxes \(\mathcal{B}\) generated by the pedestrian detector to a pedestrian with identity \(i\), whose trajectory up to frame \(t\) is \(\mathcal{T}_{i}^{t}\). The trajectory \(\mathcal{T}_{i}^{t}\) is composed of a sequence of \(m_{i}\) bounding boxes \(\left\{\mathcal{B}_{i}^{1},\ \ldots,\ \mathcal{B}_{i}^{m_{i}}\right\}\). Thus, \(m_{i}\) denotes the total length of the trajectory \(\mathcal{T}_{i}^{t}\). In this work, we modify and implement two popular Kalman-filter-based tracking algorithms, SORT [28] and DeepSORT [29] to better accommodate our use cases, which presents new challenges for pedestrian tracking that are not present in modern high-quality public pedestrian tracking benchmarks. **SORT** The SORT algorithm first employs a Kalman filter predictor to estimate the state of a pedestrian based on observations from previous frames using a linear constant velocity model to represent each pedestrian's motion, observed as a sequence of bounding boxes. Given a set of \(|\mathcal{B}|\) bounding boxes from the detector as the input, and a set of \(|\mathcal{B}_{KF}|\) predicted bounding boxes from the Kalman filter algorithm, SORT uses the Hungarian algorithm [30] to find the best assignment of bounding boxes in \(\mathcal{B}\) (from the detector) to boxes in \(\mathcal{B}_{KF}\) (predicted by the Kalman filter) that minimizes the total cost. In SORT, the cost is computed using the intersection over union (IoU) of assigning a bounding box \(\mathcal{B}_{i}\in\mathcal{B}\) to a box \(\mathcal{B}_{KF,\ j}\in\mathcal{B}_{KF}\), given by:  
\[cost(i,j)=1-\text{IoU}\big{(}\mathcal{B}_{i},\mathcal{B}_{KF,\ j}\big{)}=1- \frac{\mathcal{B}_{i}\cap\mathcal{B}_{KF,\ j}}{\mathcal{B}_{i}\cup\mathcal{B }_{KF,\ j}} \tag{1}\]  
The resulting assignments are then used to update the existing trajectories and create new tracks for unmatched detections. **DeepSORT** The DeepSORT algorithm, similar to SORT, uses the Kalman filter for state estimation and solves the assignment problem with the Hungarian algorithm. As an extension to SORT, DeepSORT incorporates a reidentification (Re-ID) model, a CNN feature extractor, to improve tracking by using a matching cascade for the detection association process. The feature extractor is pre-trained on a large-scale person Re-ID dataset and is available publicly [31]. The matching cascade computes both the Mahalanobis distance and similarity in deep appearance features between predicted boxes and detections for the cost of association. The deep appearance similarity between a bounding box \(\mathcal{B}_{i}\in\mathcal{B}\) (from the detector) to a box \(\mathcal{B}_{KF,\ j}\in\mathcal{B}_{KF}\) (predicted by the Kalman filter) is computed as the cosine distance between the deep features of the pedestrian enclosed by \(\mathcal{B}_{i}\), and those of the pedestrian enclosed by \(B_{KF,j}\). Trajectories and detections that are unable to be matched by the matching cascade process are then associated using their IoU metric as in SORT.