# Mitigating Distractor Challenges in Video Object Segmentation through Shape and Motion Cues
## 1 Introduction  
Video object segmentation (VOS) is a fundamental task in video understanding, including many industrial information applications such as autonomous driving. In this paper, we will focus on video object segmentation under semi-supervised learning, i.e., inferring the object mask in the rest of the video frames given the first frame's object mask.  
During recent years, match-based methods [1; 2; 3; 4; 5; 6; 7; 8; 9; 10; 11] have become the mainstream. This approach makes full use of the segmented historical frames, encoding them into embedded features stored in a memory bank to assist in the segmentation of subsequent frames. The most recent matching-based VOS methods focus on addressing the memory cost problem [5; 6] and the distractor objects problem [7; 8; 9], both caused by the global matching mechanism. Though they have achieved good performance, some issues still exist in those methods: (1) The important shape information is often ignored, leading to a decrease in accuracy for the segmentation around object-edge regions. (2) For addressing distractor objects, pixel-level motion estimation is often involved along the video to restrict the global matching; this brings heavy computation costs and struggles against occlusion or fast/blurry motion.  
For the first problem, this paper introduces a lightweight shape extraction module aimed at obtaining object shape information. Specifically, a shape extraction head is added following the low-level features outputted by the network to focus on extracting shape-related features of the object. Simultaneously, high-level features from the network are also used to denoise the activations in the shape extraction module, eliminating irrelevantnoise from the low-level features. Thus, we use the extracted shape information to further optimize the segmentation results.  
For the second problem, we introduce a novel object-level motion prediction module beyond pixel level. To this end, we propose to use a motion memory bank to store the object-level representative motion features learned from the training stage, which can be used to predict the motion information during the inference stage. Specifically, a recurrent neural network [12] is used to extract object-level motion features during the training stage, then we retrieve the stored motion features for estimating the object motion at the inference stage in order to predict the position of the object in the next frame. This allows our method to achieve both high efficiency and strong robustness for handling distractor objects or fast/blurry motion.  
Figure 1 illustrates the improvement in segmentation accuracy using our method. In summary, the main contributions of this work are as follows:  
* We design a lightweight object shape extraction module that exploits both the high-level and low-level features to obtain object shape information. And the shape information is then used to further refine the predicted masks.
* We introduce a novel object-level motion prediction module that stores the representative motion features during the training stage, and predicts the object motion by retrieving them during the inference stage, by which our method achieves high efficiency and strong robustness.
* The proposed method performs well compared to recent state-of-the-art methods on benchmark datasets. An accuracy of 85.2 \(\mathcal{J}\)&\(\mathcal{F}\) was obtained on the DAVIS 2017 [13] validation set. An accuracy of 84.5 \(\mathcal{J}\)&\(\mathcal{F}\) was obtained on the YouTube-VOS [14] 2019 validation set.