## 3 Methodology
### Track Management and Zero-Shot ReID Integration
#### 3.3.1 Motion Prediction and Similarity Assessment  
**Motion Prediction Mechanism:** The Kalman filter is employed to predict the motion of detected objects across consecutive frames. Each detected pedestrian's state is represented by a state vector \(x_{k}=[x,y,\hat{x},\hat{y},w,h]^{T}\), where \((x,y)\) represents the center of the bbx, \((\hat{x},\hat{y})\) represents the velocity, and \(w\) and \(h\) are the width and height of the bbx. The state transition model \(\mathbf{F}\) is defined as given in Equation (12):  
\[x_{k+1}=Fx_{k}+w_{k} \tag{12}\]  
where \(F\) is the state transition matrix and \(w_{k}\) is the process noise, assumed to follow a Gaussian distribution with covariance \(Q\). The Kalman filter predicts the next state \(x_{k}+1\) and updates it with the observed measurements.  
**Distance Calculation:** The distance between predicted tracks and new detections is calculated using a combination of spatial and appearance-based metrics. For spatial distance, the Mahalanobis distance \(d_{M}\) is utilized to measure the difference between the predicted bounding box and the detected bounding box. The Mahalanobis distance is given by Equation (13):  
\[d_{M}(z_{k},Hx_{k})=\sqrt{(z_{k}-Hx_{k})^{T}S_{k}^{-1}(z_{k}-Hx_{k})} \tag{13}\]  
where \(z_{k}\) is the detected measurement, \(H\) is the observation model, and \(S_{k}\) is the innovation covariance.  
**Appearance Feature Descriptors and Similarity Score:** Feature locations obtained via XFeat within the bboxes of pedestrians detected by the fine-tuned YOLOv8 are fed into the Kalman filter. These feature locations are further used to compute appearance descriptors, enhancing the reliability of track association. The similarity score between the detected and tracked objects is calculated using the VLM that we also fine-tuned for this purpose in a similar manner as [58]. This score, \(S_{VLM}\), is integrated into the overall distance calculation, combining both spatial and appearance-based distances as in Equation (14):  
\[d_{total}=\alpha\cdot d_{M}+\beta\cdot(1-S_{VLM}) \tag{14}\]  
where \(\alpha\) and \(\beta\) are weights that balance the influence of spatial and appearance-based metrics, respectively. This integrated distance metric ensures robust track management by leveraging both motion prediction and appearance similarity.  
**Cost Matrix Construction:** To apply the Hungarian method, we first construct a cost matrix \(C\) that quantifies the dissimilarity between each detected object and each existing track. The elements of the cost matrix \(C_{ij}=d_{total}(z_{i},x_{j})\) are computed as a weighted combination of spatial and appearance-based distances as defined earlier in Equation (14), where \(d_{total}(z_{i},x_{j})\) appears to be the Mahalanobis distance between the \(i\)-th detection and the \(j\)-th track, and \(S_{VLM}(z_{i},x_{j})\) is the similarity score derived from the VLM.  
**Optimal Assignment:** The Hungarian method solves the assignment problem by minimizing the total cost of assigning detections to tracks. Given the cost matrix \(C\), the algorithm finds the optimal assignment AA, which minimizes the sum of the selected costs as given in Equation (15):  
\[A^{*}=\arg\min_{A}\sum_{i,j}A_{ij}C_{i,j} \tag{15}\]  
where \(A_{ij}\) is a binary variable that equals 1 if the \(i\)-th detection is assigned to the \(j\)-th track, and 0 otherwise. Solution \(A^{*}\) ensures that each detection is assigned to at most one track and each track is matched to at most one detection, thereby optimizing the association process.