# Occlusion-Robust Multi-Target Tracking and Segmentation Framework with Mask Enhancement
## 1 Introduction  
Multiple-object tracking (MOT) is one of the prominent tasks in the field of Computer Vision. The process of MOT involves associating multiple targets across different frames in a video sequence frame-by-frame and generating motion trajectories of the tracked targets. MOT serves as the foundation of action recognition and behavior analysis of targets [1]. The Tracking By Detection (TBD) framework is the mainstream approach in MOT methods. This framework divides MOT into two stages: detection and data association. Inthe detection stage, a pretrained detector is used to obtain targets' detection results in the video sequence. The detection process transforms the target tracking problem into a data association problem across consecutive frames. During the data association stage, targets in consecutive frames are associated so as to form complete trajectories. This stage outputs the positional information of different targets at different times. Recent years have witnessed various solutions proposed to optimize trajectory association in MOT tasks. They have addressed complex interactions such as inter-object occlusion and dynamic target counts in different scenarios [2; 3; 4; 5; 6; 7; 8]. Existing MOT trackers often perform well in simple scenarios. However, when applied to crowded scenes, their performance significantly deteriorates, since target occlusion interferes with the target detection and the data association processes. In practical applications, not all observations reflect target information correctly, since they suffer from background noise, motion blur, or occlusion [9; 10]. The lack of state evaluation mechanisms makes traditional approaches vulnerable to noise inputs and ineffective for long-term tracking in occluded scenarios. To address these challenges, [PERSON] et al. [1] integrated instance segmentation techniques with MOT algorithms. They constructed a multiple-object tracking and segmentation (MOTS) framework, which can achieve pixel-level tracking on multiple targets. Compared to traditional bounding box detections, instance segmentation provides pixel-level detection, assigning a unique mask to each target. Image instance segmentation aims to classify each pixel in an image, predicting both target categories and pixel-level segmentation masks to accomplish simultaneous object detection and semantic segmentation [11; 12]. Additionally, the non-overlapping nature of binary instance masks resolves the ambiguity involved in distinguishing target identities during occlusion. Approaches like Mask R-CNN [13] have demonstrated that pixel-level detection based on segmentation can effectively handle complex scenarios with occlusion by eliminating errors from traditional bounding boxes. Therefore, merging instance segmentation into MOT tasks can mitigate issues caused by traditional bounding box detections and enhance tracking performance in occluded scenarios. The main contributions of our paper can be summarized as follows:  
* We propose a mask-conditional feature fusion network that integrates a mask-guided attention mechanism and a spatial-temporal feature aggregation sub-network. The network improves tracking robustness under crowded scenes. The network leverages temporal information from short trajectories and learns to generate occlusion-robust spatial-temporal aggregation features under the guidance of masks. * We present an occlusion-aware mask propagation network. The network prevents the contamination of online tracking templates from noise inputs by perceiving the occlusion states of targets. It completes the association of missing trajectory segments through a constructed mask propagation network. * We propose a mask-enhanced multi-object tracking and segmentation framework. This framework combines mask-based methods with a mask-integrated multi-hypothesis tracking algorithm to achieve adaptability in occluded scenarios and enhance the robustness of the MOTS framework. The rest of this paper is organized as follows. Related work is discussed in Section 2.