## 3 Methodology  
The proposed MOT flow diagram in Figure 3 addresses challenges in real-world tracking scenarios. It begins with a robust object detection phase using a fine-tuned model to accurately identify and localize humans in images or video streams, generating precise bbox coordinates. Next, an intricate feature extraction process employs advanced techniques such as Xfeat and LightGlue to derive keypoints, descriptors, and confidence scores from detected bboxes. These features are input into a VLM that assesses whether a detected human matches an existing track, producing a Boolean response. The track management module then handles motion prediction and data association, ensuring accurate track continuity across frames despite challenges such as occlusions and sudden motion changes. To enhance reliability, the system incorporates confident track storage, archiving established tracks to reduce IDsws and improve long-term tracking fidelity. Finally, the ReID module uses stored information to reassociate newly detected humans with existing tracks, ensuring robust ID preservation and minimizing errors in dynamic environments. This integrated flow diagram not only overcomes the limitations of traditional MOT systems but also offers a scalable and efficient solution for diverse and challenging conditions, making it ideal for modern large-scale applications. ReTrackVLM is designed to overcome the persistent challenges encountered in real-world tracking scenarios, integrating VLMs and a novel embedding structure. As can be seen in Figure 4, the operation procedure of ReTrackVLM starts with fine-tuned YOLOv8 [53] detecting pedestrians in input images, followed by feature extraction using XFeat and LightGlue, which generate robust descriptors and keypoints for each bbox. These features are processed through an image encoder and text embedder, then fused using a fine-tuned transformer to predict motion and manage data association. The track management module ensures accurate tracking by predicting movements and refining appearance descriptors. High-confidence tracks are stored in a dedicated database for ReID, allowing the system to maintain track continuity even in challenging scenarios such as occlusions and complex motions. This combination of modules improves the accuracy of the tracking and reduces IDSws across various datasets. The ReTrackVLM pipeline, which is given in Algorithm 1, processes video frames for multi-object tracking through object detection, track prediction, and re-identification. It starts by initializing an empty track set and confident track storage (CTS) for managing tracks across frames. For each frame, objects are detected using a pre-trained YOLOv8, with keypoints, descriptors, and confidence scores forming embeddings. A Kalman filter predicts object locations based on previous tracks, and detected objects are associated with these predictions using IoU and appearance similarity scores. The Hungarian algorithm handles associations, updating matched tracks, initializing new ones, and maintaining unmatched tracks. Unmatched tracks are further processed using zero-shot ReID, querying the CTS for identity matches without retraining. High-confidence tracks are added to CTS for future use. After processing all frames, low-confidence tracks are removed, and trajectories are smoothed for output. The final tracks are saved in a format compatible with the MOT Challenge for evaluation. Figure 4: Overview of the proposed MOT framework. ReTrackVLM integrates VLMs and a new embedding structure to enhance MOT performance in challenging environments. Fine-tuned YOLOv8 detects pedestrians in input images, generating bbxes that are further processed by XFeat and LightGlue for feature extraction. These features, including descriptors, keypoints, and overlap ratios, are fed into an image encoder and text embedder. The resulting embeddings are fused through a fine-tuned transformer, aiding in motion prediction and data association within the Track Management module.