# SKVOS: Sketch-Based Video Object Segmentation with a Large-Scale Benchmark +
## 5 Experiments
### Sketch vs. Other References
#### 5.2.2 Fair Comparison Between Different References  
To assess the effectiveness of sketches, we conduct several comparative experiments under identical experimental conditions, comparing sketches with other types of references, including text, text+click, text+bounding box, cross, circle, contour, scribble, and mask. All experiments utilize STCN as the backbone and follow the same implementation settings described in Section 5.1. The fusion methods are tailored to accommodate the characteristics of each reference modality. Since the YouTube-VOS dataset does not include click, bounding box, cross, circle, contour, or scribble annotations, we adopt the following strategies:  
Figure 10: Visual comparison with ReferFormer [34] and SgMg [35] on the YouTube-VOS validation set. Best viewed in color. Red boxes indicate segmentation errors, while green boxes indicate correct segmentation.  
Figure 9: Qualitative results on the Sketch-YouTube-VOS validation set. Best viewed in color.  
Cross and click points are placed at the center of the ground-truth mask.
* Circles and bounding boxes are generated by fitting an outer circle or box to the ground-truth mask.
* Contours are extracted by computing the convex hull of the sketch.
* As for the scribbles, we first generate a skeleton tree according to the binary mask of the target object, and then select the longest path in the tree as the final scribble, following the method described in [77].
* For text references, we train and validate the model on standard Referring VOS benchmarks [14; 15].
* We directly utilize mask-based VOS datasets [15; 26; 27].  
Except for text and mask, all other references lack explicit semantic information and can be considered sparse masks. Therefore, we simply concatenate these references with the first frame before feeding them into STCN, following the approach in [10]. For language expressions, we encode first-frame expressions using the BERT [72] language encoder, initialized with its official pre-trained weights. Linguistic features are then fused with frame features using the Cross-KV module, as applied in language-based methods [34; 78]. For mask-based VOS, to ensure a fair comparison, we directly adopt the method used in STCN and do not pre-train the model on additional static image datasets.  
Table 4 provides a fair comparison of various reference types, including key metrics and the average annotation time for each. The results demonstrate that the sketch outperforms text-based references by more than 20 \(\mathcal{J}\)&\(\mathcal{F}\) points. Moreover, sketches offer greater benefits than simple indicators like cross, circle, box, or click, as these references only convey the object's location and lack semantic context or pose information. It is also evident that using only the contour performs worse than using the full sketch. This may be due to the fine-grained details in the sketch that help with object representation and segmentation, whereas relying solely on contours may cause ambiguity in later video frames. While scribbles also perform well, they still fall short of sketches, with an \(\mathcal{J}\)&\(\mathcal{F}\) margin of 8.0 points. Interestingly, without pre-training on large static image datasets, mask-based references only achieve 79.6 \(\mathcal{J}\)&\(\mathcal{F}\). Additionally, our method requires almost only one-third of the annotation time compared to mask-based methods, while achieving an inference efficiency of 18.8 FPS, processing 5.6 more frames per second on average than mask-based methods. These results suggest that sketches can effectively serve as a viable, cost-efficient alternative to other reference types for video object segmentation.