## 3 Methodology
### Data Preprocessing and Input Representation  
We utilize several well-known datasets in the MOT domain, including MOT15, MOT16/17 (which contains refined annotations), MOT20, DanceTrack, and WildTrack. Additionally, we incorporated CrowdHuman [54], which focuses on extremely crowded scenes with severe occlusions and dense pedestrian groupings, and is crucial for applications like public safety and surveillance. Each dataset offers a rich set of annotated frames focusing on pedestrian detection, which are preprocessed to ensure consistency and compatibility with the YOLOv8 framework and subsequent ReID processes. The preprocessing steps involve the following key operations:  
* **Resizing:** All input images are resized to a uniform size of 640 \(\times\) 640 pixels to match the input requirements of the YOLOv8 model. This resizing ensures that the aspect ratios of the pedestrians are preserved, minimizing distortion. * **Normalization:** Pixel values are normalized to the range [0, 1] to facilitate faster convergence during model training. This normalization is performed using the mean and standard deviation of the ImageNet dataset [55], which align with the pre-trained weights used in the model. * **Data Augmentation:** Data augmentation techniques such as horizontal flipping, random cropping, rotation, scaling, and color jittering are applied to enhance the model generalization capability. The augmentation parameters are carefully selected to maintain the integrity of the bbxes and the corresponding pedestrian identities. We used the training and validation sets provided by the datasets originally, with the training set used to optimize the model weights and the validation set used to monitor performance and prevent overfitting. Let \(B=b_{1},b_{2},b_{3},\ldots,b_{N}\) denote the set of bbxes in an image, where each bbx \(b_{i}\) is represented as \(b_{i}=[x_{i}^{min},y_{i}^{min},w,h]\). Here, \(x_{i}^{min}\) and \(y_{i}^{min}\) are the coordinates of the top-left corner of the bbx, and \(w\) and \(h\) are the width and height of the bbx, respectively. The content of the bbxes is further processed to extract features using the XFeat and LightGlue methods, which are critical for cross-modal embeddings and ReID tasks. The YOLOv8 model is fine-tuned on the preprocessed datasets to optimize pedestrian detection performance by following the fine-tuning procedure explained in [56]. The training process involves the following steps:  
* **[1] Model Initialization:** The YOLOv8 model is initialized with pre-trained weights from the COCO dataset [57]. The model is designed to detect pedestrians with high accuracy, utilizing the features extracted from the bbxes with Equation (1): \[L=L_{bbx}+L_{conf}+L_{class}\] (1) where \(L_{bbx}\) is the regression loss of the bbx, \(L_{conf}\) is the confidence loss, and \(L_{class}\) is the classification loss. * **[2] Training Configuration:** The model is trained using a custom configuration defined in a YAML file, specifying training parameters such as batch size, learning rate, augmentation settings, and the number of epochs. * **[3] Training Procedure:** The model is trained for 100 epochs with early stopping based on validation performance. The multi-scale training strategy is employed, where input images are randomly resized during training to improve the robustness of the model to varying scales. The fine-tuned YOLOv8 model serves as the backbone for the tracking pipeline, providing accurate detections that are fed into the cross-modal embeddings and track management modules. In the feature extraction stage using XFeat and LightGlue, we obtained the robust features from the bbxes. These features are employed as inputs to the VLMs and play an important role in track management and ReID. XFeat captures deep semantic features, while LightGlue focuses on point-based features that are useful for matching and association tasks.