## 3 Methods
### Detection  
Following background subtraction using the MOG2 algorithm, all detected moving entities are drosophilas, our subjects of interest. While MOG2 effectively identifies these moving targets against the static background, it lacks the precision required for accurate localization and differentiation, especially when drosophilas overlap or are partially occluded. To enhance the detection accuracy and ensure comprehensive coverage, we utilize a sliding-window technique that segments the video frame into overlapping windows of 128 \(\times\) 128 pixels, with a 50% overlap between adjacent windows. As a result, the step size for both horizontal and vertical movement is 64 pixels. This overlapping design guarantees that drosophilas near the edges of each window are not missed and are adequately captured in multiple windows, thereby improving the chances of their detection and accurate segmentation. The segmented images are then processed using the Mask-RCNN framework [14], renowned for its precision in instance segmentation. Mask-RCNN builds on the Faster R-CNN architecture by adding a branch for predicting segmentation masks on each Region of Interest (RoI) alongside the existing branches for object classification and bounding-box regression. This setup is particularly effective in handling partial occlusions and densely clustered subjects, as it allows for pixel-precise segmentation. Through the RoIAlign technique, Mask-RCNN refines the proposed bounding boxes, classifies the objects, and produces detailed segmentation masks that capture the essential features of partially occluded drosophilas. These masks provide crucial details that facilitate the precise determination of each insect's location and orientation. For the purpose of precise tracking, after each drosophila is detected and segmented in a sliding window, we calculate its global coordinates from its local coordinates within  
Figure 2: Visual comparison between an input frame and its subtracted frame. (**a**) Input frame; (**b**) after the subtraction using MOG2. the window. This is done using the following transformation, where \((x_{offset},y_{offset})\) are the coordinates of the top-left corner of the sliding window. The process is given by Equation (2):  
\[x_{global} =x_{offset}+x_{local} \tag{2}\] \[y_{global} =y_{offset}+y_{local}\]  
The offsets \((x_{offset},y_{offset})\) are calculated based on the indexed position of each sliding window as follows by Equation (3):  
\[x_{offset} =j\times S \tag{3}\] \[y_{offset} =i\times S\]  
Here, \(j\) and \(i\) are the indices of the sliding window in the horizontal and vertical directions, respectively, and \(S\) is the step size of 64 pixels. This method ensures the accurate translation of local coordinates detected within each window to the global coordinate system of the entire frame. The Mask-RCNN model, pre-trained on the COCO dataset [15] and fine-tuned with our annotated frames, outputs bounding boxes, class labels, and segmentation masks for each drosophila detected. These results lay a robust foundation for the particle filter tracking algorithm that follows, ensuring an effective and comprehensive analysis of the complex dynamics and interactions within drosophila swarms. A series of detected drosophila objects is shown in Figure 3, and each cut patch contains one object. While Mask-RCNN is a potent tool for object detection and segmentation, it faces challenges when tracking small, fast-moving targets like drosophilas, primarily due to the scale variability and its intensive computational demands. We adapted Mask-RCNN by integrating a sliding-window approach and biological modeling of drosophila features to enhance its performance for our specific application. These modifications focus the detection on localized areas and leverage drosophila-specific characteristics, significantly improving the detection accuracy.