# SKVOS: Sketch-Based Video Object Segmentation with a Large-Scale Benchmark +
## 6 Conclusions  
In this work, we explored the question of how to effectively segment objects in videos using user-provided sketches. To this end, we introduced three instance-level datasets specifically designed for sketch-based video object segmentation (VOS). These datasets provide a benchmark for evaluating the effectiveness of sketch-based methods and offer a novel perspective on leveraging sketches for video understanding. We extended a popular memory-based VOS method and investigated various fusion designs to better integrate sketch and visual features, demonstrating the potential of sketches as a powerful and intuitive modality for object segmentation in videos. To address the challenge of discontinuous segmentation, we proposed a Temporal Relation Module (TRM) that enhances the correlation of hierarchical features between adjacent frames, significantly improving segmentation consistency. Additionally, we introduced a Sketch-Anchored Temporal Contrastive Learning (STCL) approach, which explicitly leverages sketches as anchors to enable the model to learn temporal associations of objects. This approach enforces temporal coherence and substantially strengthens segmentation performance. Experimental results show that our method achieves state-of-the-art performance with overall scores of 77.3, 85.6, and 77.4 on the Sketch-YouTube-VOS, Sketch-DAVIS 2016, and Sketch-DAVIS 2017 validation sets, respectively.  
Despite these advancements, our method faces limitations with long videos, as a single sketch may fail to capture complex temporal variations, affecting segmentation performance. Future work will explore multi-frame sketches to better model object dynamics, improve accuracy, and enhance robustness, along with efficient fusion strategies for complex scenarios. This work advances video object segmentation by introducing sketches as an intuitive modality for user interaction. By offering a benchmark dataset and framework for sketch-based VOS, it aims to inspire further research and impact applications like video editing, autonomous systems, and human-computer interaction, where efficient segmentation is essential.  
Conceptualization, R.Y., D.L. and C.H.; methodology, R.Y., D.L., C.H. and H.Z.; software, R.Y.; validation, R.Y. and H.Z.; formal analysis, R.Y.; investigation, R.Y.; resources, R.Y.; data curation, R.Y.; writing--original draft preparation, R.Y.; writing--review and editing, R.Y., D.L., C.H. and H.Z.; visualization, R.Y.; supervision, D.L., C.H. and H.Z.; project administration, H.Z.; funding acquisition, H.Z. All authors have read and agreed to the published version of the manuscript.  
This research received no external funding.  
The raw data supporting the conclusions of this article will be made available by the authors on request.  
The authors declare no conflicts of interest.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**Sketch-DAVIS16\({}_{val}\)**} & \multicolumn{3}{c}{**Sketch-DAVIS17\({}_{val}\)**} \\  & \(\mathbf{\mathcal{J}\&\mathcal{F}}\) & \(\mathbf{\mathcal{J}}\) & \(\mathbf{\mathcal{F}}\) & \(\mathbf{\mathcal{J}\&\mathcal{F}}\) & \(\mathbf{\mathcal{J}}\) & \(\mathbf{\mathcal{F}}\) \\ \hline \#1 & 82.3 & 80.9 & 83.7 & 72.4 & 69.2 & 75.6 \\ \#2 & 85.6 & 84.8 & 86.3 & 77.4 & 72.6 & 82.1 \\ \hline \hline \end{tabular}
\end{table}
Table 10: Performance on Sketch-DAVIS16 and Sketch-DAVIS17 validation sets: #1 indicates testing the pre-trained model on YouTube-VOS directly; #2 means jointly training Sketch-YouTube-VOS and Sketch-DAVIS.