## 4 Experiments and Results
### Ablation Study  
In this ablation study, we analyzed the contributions of key components within the ReTrackVLM framework. By evaluating the impact of cross-modal embeddings, zero-shot ReID integration, track management optimizations, and fine-tuning, we identified which elements most significantly enhance tracking accuracy and robustness. This analysis provides insights into our modules' effectiveness and informs future improvements. In Table 2, we present the detection performance of YOLOv8 after fine-tuning on several datasets, optimizing it for person detection across diverse scenes. Since YOLOv8 processes the fixed-size input originally (\(640\times 640\)), we also followed this convention and, consequently, the fine-tuned YOLOv8 model achieved an average processing speed of 98 frames per second (fps) on our setup on average. Evaluation metrics include Box Precision, Box Recall, mAP50 (mean Average Precision at IoU threshold of 0.50), and mAP50-95 (mean Average Precision across IoU thresholds from 0.50 to 0.95). The results establish the ReTrackVLM's performance, showing that YOLOv8 fine-tuned on datasets like MOT15 and DanceTrack achieves high detection rates, with MOT15 achieving 94.8% Box Precision and 95.2% mAP50, while DanceTrack reaches 91.8% mAP50. In contrast, crowded datasets such as MOT20 exhibit lower precision (67.6%) and recall (69.4%), highlighting the challenges of densely populated scenes that require further refinement. When fine-tuned across all datasets, the model achieves a Box Precision of 71.9% and an mAP50 of 76.7%, offering a generalized performance that balances adaptability without overfitting, making it suitable for diverse MOT applications. For fine-tuning the models in our framework, the time required was closely tied to the complexity of the architecture and the dataset size. DETR2, being the least complex architecture among the tested models, required approximately 15 h for fine-tuning on the smallest dataset. In contrast, Florence-2, recognized as one of the most complex VLMs due to its multimodal architecture and large model size, required nearly 30 h for fine-tuning on the largest dataset. These time differences align with the relative complexities of the models: Florence-2, with its focus on integrating image-text pairs and object detection data, exhibits significantly higher computational demands than DETR2, a transformer-based object detection model that is less resource-intensive. In terms of runtime performance, our framework achieved a processing speed of 18 fps when using DETR2, unveiling its moderate complexity and suitability for real-time applications. Conversely, Florence-2 delivered a speed of 12 fps, reflecting the trade-off between its advanced capabilities and computational overhead. CLIP demonstrates superior tracking performance, as can be seen in Table 3, particularly in MOTA and MOTP, across most datasets, indicating better accuracy and track continuity. DETR2 also performs well, especially on challenging datasets like MOT16/17 and MOT20, but has a higher IDSw, suggesting challenges in identity preservation compared to CLIP. OpenCLIP generally shows lower performance across metrics, while Florence-2 strikes a balance between detection accuracy and identity handling, though it trails behind CLIP and DETR2. CLIP excels in challenging scenarios like WildTrack and DanceTrack, leading in MOTA, MOTP, and FM. \begin{table}
\begin{tabular}{l l l l l} \hline \hline
**Dataset** & **Box Precision (\%)** & **Box Recall (\%)** & **mAP50 (\%)** & **mAP50-95 (\%)** \\ \hline MOT15 & 94.8 & 90.1 & 95.2 & 70.4 \\ MOT16/17 & 84.3 & 80.9 & 86.6 & 64.7 \\ MOT20 & 67.6 & 69.4 & 69.5 & 44.5 \\ DanceTrack & 87.9 & 85.2 & 91.8 & 62.4 \\ WildTrack & 89.3 & 81.5 & 90.0 & 58.1 \\ CrowdHuman & 86.5 & 77.7 & 87.3 & 59.3 \\ All Datasets & 71.9 & 75.7 & 76.7 & 48.7 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Detection performance of fine-tuned YOLOv8 on various datasets. In Table 4, zero-shot ReID performance with confident tracks from a non-fine-tuned YOLOv8 detector is provided.