## 3 Methodology
### Track Management and Zero-Shot ReID Integration  
Effective MOT requires robust track management and ReID mechanisms to address object occlusions, sudden appearances, and IDSws. ReTrackVLM incorporates a sophisticated  
Figure 5: The cross-modal embedding and fine-tuning generation process for the VLMs for which bbxes, keypoints, and descriptors are extracted from detected objects, yielding embeddings that align visual and textual modalities. Fine-tuning the VLMs ensures task-specific optimization for multi-object tracking, enabling robust association of objects across frames based on semantic similarity and appearance features.  
track management module with zero-shot ReID capabilities, ensuring seamless tracking across diverse scenarios. This module utilizes motion prediction, distance calculation, and data association strategies to maintain track consistency and ReID objects after interruptions, collectively enhancing tracking reliability and accuracy in challenging environments.  
The track management and zero-shot ReID integration of ReTrackVLM are demonstrated in Figure 6, which initially obtains the bbxes, keypoints, and descriptors for each detected pedestrian from the current frame. Motion estimation, implemented via a Kalman filter, predicts the future positions of tracks based on their previous states, aiding in association with new detections. For similarity evaluation, the cosine similarity metric is employed to calculate a distance matrix between the embedding vectors of existing tracks \(V_{t}\)s and new detections. This matrix quantifies the matching likelihood between tracks and detections. Following distance calculation, data association is performed using a bipartite matching algorithm to optimally pair the detections (Det1 to DetN) with tracks (Track1 to TrackM), taking into account both motion predictions and embedding similarities. After association, zero-shot ReID is applied by leveraging VLM embeddings, allowing for re-identification across frames without additional training. Confident tracks are stored in an array containing their associated embeddings, indexed by unique IDs such as ID1, ID2,..., IDk, along with confidence scores. This array serves as a gallery for ReID, facilitating identity retrieval for new detections and enhancing tracking consistency across occlusions or reappearances. The framework ensures seamless integration of motion, similarity assessment, and ReID for robust MOT.  
Tracks from the previous frame are predicted using a Kalman filter in this stage as shown in Algorithm 4. Detected objects in the current frame are associated with these predicted tracks based on IoU and appearance similarity scores, using the Hungarian algorithm. For unmatched tracks, zero-shot ReID compares them with previously stored  
Figure 6: Overview of the track management and zero-shot ReID modules, which includes motion estimation using a Kalman filter, cosine similarity-based distance calculation, optimal data association via bipartite matching, and zero-shot ReID integration utilizing VLM embeddings for maintaining consistent identities across frames. Confident tracks are stored as an array indexed by unique IDs, supporting robust tracking and ReID. Detailed depiction of the confident track storage mechanism highlights the array-based storage of tracks. The stored data facilitate efficient ReID by comparing gallery embeddings with query detections for identity retrieval.  
embeddings in the confident track storage, allowing for robust identity preservation without retraining.