SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We repeat the experiment in Table 3 by choosing mutually exclusive subsets of 300 videos, 3 times. We report mean and standard deviation  $(\mu \pm \sigma)$  of experiments for each model in Table 6. These results are coherent with the reported result on the fixed subset, showing that segmentation performance peaks when all of our components are combined, i.e. model E. Similarly, removing our components  $\psi_{\mathrm{t - bind}}$  (model D) and  $\psi_{\mathrm{s - bind}}$  (model C) one at a time leads to a performance drop, as shown in Table 3. Removing both (model B), results in the worst performance, even falling behind its counterpart without slot merging (model A). Overall, the results of varying the evaluation set agree with the reported performance on the selected subset in the main paper. This shows that the performance of our model generalizes over different evaluation subsets.
Table 6: Varying evaluation splits. The effect of changing evaluation sets for models in the component ablation.
Model	mIoU ↑	FG-ARI ↑
A	37.72 ± 0.82	29.06 ± 2.24
B	37.20 ± 1.53	30.15 ± 1.90
C	42.76 ± 2.09	30.03 ± 1.67
D	42.98 ± 2.01	29.69 ± 2.02
E	43.28 ± 2.57	31.33 ± 1.51
Figure 8: Input resolution. The effect of input resolution, i.e. the number of tokens, on performance (mIoU).
E Additional Comparisons
Table 7: Comparison to supervised models. These results show the video multi-object evaluation results on the validation split of DAVIS17 between our model and supervised models.
Model	Unsupervised	mIoU ↑
UnOVOST [55]	X	66.4
Propose-Reduce [48]	X	67.0
Ours	✓	30.2
We compare the performance of our model to available supervised models, exploiting pre-trained object detectors on large data in a supervised manner, for unsupervised video object segmentation on DAVIS17 in Table 7. The results underscore a notable gap between supervised and unsupervised approaches.
F Failure Analysis
Although our model can detect in-the-wild objects in different scales, segmentation boundaries are not perfectly aligned with the object due to patch-wise segmentation, as has been pointed out in the Discussion (Section 5). In addition to these observations, we identify three types of commonly occurring failure cases: (i) The over-clustering issue that remains unresolved in some cases even with slot merging. (ii) The tendency to cluster nearby instances of the same class into a single slot. (iii) Failure to detect small objects, particularly when they are situated near large objects. We provide visual examples of these cases in Fig. 9.
(a) Failure cases due to over-clustering.
(b) Failure cases due to grouping nearby instances of the same class into one cluster.
(c) Failure cases due to missing relatively small objects.
Figure 9: Failure cases of our model with potential reasons, grouped into three.
We provide additional qualitative results in Fig. 10. Our model can recognize not only the most salient object in the middle but also small, subtle objects in the background. Furthermore, it can handle a varying number of objects in the scene with slot merging.
Feature Extractor: In Fig. 11, we provide visualizations of different feature extractors, corresponding to the quantitative evaluation in Table 5, including DINOv2 [62], DINO [7], and Supervised. Self-supervised models performs better than the supervised one, also qualitatively. In particular, DINOv2 stands out for its exceptional capability to learn object representations across a diverse range
Figure 10: Qualitative results of multi-object video segmentation on YTVIS19
of categories. It also effectively identifies and segments intricate details that are missed by other feature extractors such as tree branches on the left and the bag on the table on the right.
Components: In Fig. 12, we visually compare the segmentation results of the models corresponding to the component ablation study in the main paper (Table 3). First of all, model A, which corresponds to the temporally consistent DINOSAUR [67], struggles to cluster the instances as a whole and also fails to track all objects, for instance, the human on the right, due to discrepancies in mask index across three frames. With the help of slot merging, model B effectively addresses the over-clustering issue, as observed in the mask of the calf on the left. Integrating our binding modules  $\psi_{\mathrm{t - bind}}$  and  $\psi_{\mathrm{s - bind}}$ , resulting in model C and D, respectively, leads to a marked improvement in both segmentation and tracking quality. On the other hand, both models C and D have shortcomings in detecting the human in certain frames of the right video. Finally, combining them, our full model, i.e. model E, excels at segmenting and tracking not only labeled objects but also other objects, such as the car visible in the first frame of the second video.
Comparison: We provide a visual comparison between OCLR [83] and our model in Fig. 13 after matching the ground-truth and predictions. OCLR fails to detect static objects (top-left, middle-left, bottom-left) and deformable objects (middle-right) due to failures of optical flow in these cases. Moreover, it considers moving regions as different objects such as the water waves (top-right). On the other hand, SOLV can accurately detect all objects as a whole.
Figure 11: Qualitative comparison of different pretraining methods for visual encoder.
Figure 12: Qualitative results of different models in the component ablation study (Table 3).
Figure 13: Qualitative results of multi-object video segmentation on YTVIS19 after Hungarian Matching is applied. Segmentation results of OCLR [83] are provided in third row.