SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In this part, we first give some details about the experimented benchmarks in Sec. 4.1. Then, we report the main results in Sec. 4.2, which mainly demonstrates the superiority of the proposed approach over previous solutions. Furthermore, we give detailed ablation studies in Sec. 4.3 and Sec. 4.4 to validate the exquisite design of the whole framework and related proposed modules. Finally, we illustrate some qualitative visualizations in Sec. 4.5 to show the effectiveness.
We conduct extensive experimental evaluations on four popular and challenging benchmarks, including YouTube-VIS 2019 and 2021 [47], OVIS [35], and UVO [39]. Key statistics of these datasets are presented in the Appendix. YouTube-VIS 2019 [47] is the first large-scale dataset for video instance segmentation, with 2.9K videos averaging 4.61s in duration and 27.4 frames in validation videos. YouTube-VIS 2021 [47] includes more challenging longer videos with more complex trajectories, resulting in an average of 39.7 frames in validation videos. OVIS [35] dataset is another challenging VIS dataset, offering 25 object categories and focusing on complex scenes with significant occlusions between objects. While only containing 607 training videos, its videos are much longer, lasting
Table 2: Results comparison on the UVO and OVIS validation sets. We group the results by online or offline methods, and then with ResNet-50 or Swin-L backbone structures. TMT-VIS is the model built upon Mask2Former-VIS, while TMT-VIS† in 'Online' and 'Offline' are the model that we add our designs based on popular online and offline method, GenVIS and VITA. Our algorithm gets the highest AP performance compared with recent approaches.
Method	Backbone	OVIS	UVO
AP	\( AP_{50} \)	\( AP_{75} \)	\( AR_1 \)	\( AR_{10} \)	AP	\( AP_{50} \)	\( AP_{75} \)
Online	MaskTrack R-CNN [47]	ResNet-50	10.8	25.3	8.5	7.9	14.9	9.3	20.9	8.2
CrossVIS [48]	ResNet-50	14.9	32.7	12.1	10.3	19.8	-	-	-
MinVIS [18]	ResNet-50	25.0	45.5	24.0	13.9	29.7	-	-	-
IDOL [45]	ResNet-50	30.2	51.3	30.0	15.0	37.5	16.8	28.1	17.3
GenVIS [15]	ResNet-50	35.8	60.8	36.2	16.3	39.6	-	-	-
TMT-VIS†	ResNet-50	38.4	62.1	39.5	17.3	41.5	-	-	-
MinVIS [18]	Swin-L	39.4	61.5	41.3	18.1	43.3	-	-	-
IDOL [45]	Swin-L	42.6	65.7	45.2	17.9	49.6	-	-	-
GenVIS [15]	Swin-L	45.2	69.1	48.4	19.1	48.6	-	-	-
TMT-VIS†	Swin-L	46.9	71.0	48.9	18.8	52.0	-	-	-
Offline	IFC [19]	ResNet-50	13.1	27.8	11.6	9.4	23.9	-	-	-
Mask2Former-VIS [7]	ResNet-50	16.5	36.5	14.6	10.2	23.4	18.2	29.7	18.9
SeqFormer [44]	ResNet-50	15.1	31.9	13.8	10.4	27.1	-	-	-
VITA [16]	ResNet-50	19.6	41.2	17.4	11.7	26.0	18.9	30.6	19.8
TMT-VIS	ResNet-50	22.8	43.6	21.7	13.1	28.3	21.2	32.9	22.1
TMT-VIS†	ResNet-50	25.1	45.9	23.8	14.2	29.9	22.0	33.4	22.9
Mask2Former-VIS [7]	Swin-L	23.1	45.4	21.8	13.3	29.2	27.3	42.0	27.2
VITA [16]	Swin-L	27.7	51.9	24.9	14.9	33.0	-	-	-
TMT-VIS	Swin-L	28.0	50.9	27.6	14.4	34.1	29.0	42.8	29.5
TMT-VIS†	Swin-L	32.5	55.1	32.0	15.4	38.3	29.9	43.6	30.1
12.77s on average. UVO [39] dataset has 81 object categories (80 shared with COCO [27], plus an extra "other" category for non-COCO instances). UVO, a subset of 1.2K Kinetics-400 [20] videos, contains 503 videos densely annotated at 30fps, and provides segmentation masks exhaustively for all object instances.
We compare TMT-VIS with state-of-the-art approaches which are with ResNet-50 and Swin-L backbones on the YouTube-VIS 2019 and 2021 [47], OVIS [35], and UVO [39] benchmarks. The results are reported in Tables 1 and 2.
YouTube-VIS 2019. Table 1 shows the comparison on YouTube-VIS 2019 dataset. Our TMT-VIS sets new state-of-the-art results under all of the settings. Based on Mask2Former-VIS, TMT-VIS gets  $49.7\%$  AP and  $63.2\%$  with ResNet-50 and Swin-L backbones, respectively, outperforming the baseline by  $3.3\%$  and  $2.8\%$  absolutely. When the proposed designs are added onto VITA, TMT-VIS gets  $52.6\%$  AP and  $64.9\%$  AP with ResNet-50 and Swin-L backbones, respectively, outperforming the baseline by  $2.8\%$  and  $1.9\%$ . When GenVIS is added with our design, TMT-VIS gets  $53.9\%$  AP and  $65.4\%$  AP with ResNet-50 and Swin-L backbones, outperforming the baseline by  $2.6\%$  and  $1.6\%$ . We list the model parameters of Mask2Former (216M), SeqFormer (220M), VITA (247M), and our TMT-VIS (255M), which shows that TMT-VIS performs notably better with similar model parameters. Also, we compared our TMT-VIS with UNINEXT [25] on YouTube-VIS 2019 dataset. With our delicate design of TCM & TIM, our method achieves a higher performance of  $64.9\%$  AP with 4 VIS datasets on Swin-L backbone, and the total training time is approximately 1 day 12 hours. On the other hand, UNINEXT is trained on 8 video datasets and the total training time is
Table 3: Ablation study on training with multiple VIS datasets with Mask2Former-VIS (which is abbreviated as 'M2F') and TMT-VIS and their validation results on various VIS datasets. Experiments 1 - 7 report the results of training with M2F. Experiments 8 - 14 report the results of training with TMT-VIS.
ID	Method	YTVISTrain	OVISTrain	UVOtrain	YTVISval	OVISval	UVOval
1	M2F	✓	46.4	2.3	1.9
2	M2F	✓	5.2	16.5	3.6
3	M2F	✓	4.4	2.5	18.2
4	M2F	✓	✓	47.3	17.4	4.9
5	M2F	✓	✓	46.2	3.7	19.0
6	M2F	✓	✓	7.1	16.6	18.7
7	M2F	✓	✓	✓	7.1	16.6	18.7
8	TMT-VIS	✓	47.3	7.2	6.5
9	TMT-VIS	✓	10.5	17.8	8.0
10	TMT-VIS	✓	10.1	8.6	18.8
11	TMT-VIS	✓	✓	48.8	20.9	10.1
12	TMT-VIS	✓	✓	47.0	10.3	20.4
13	TMT-VIS	✓	✓	14.8	19.4	20.2
14	TMT-VIS	✓	✓	✓	49.7	22.8	21.2
Table 4: Ablation study on key component designs in TMT-VIS. The 'Multiple Datasets' setting refers to training on the three VIS datasets (Youtube-VIS 2019, OVIS, and UVO). Experiment I refers to the Mask2Former-VIS model. Note that all experiments are evaluated on Youtube-VIS 2019, abbreviated as 'YTVIS'.
ID	TCM&TIM	Taxonomy loss	YTVIS	Multiple Datasets
AP	\( AP_{50} \)	\( AP_{75} \)	AP	\( AP_{50} \)	\( AP_{75} \)
I	46.4	68.0	50.0	47.2	69.1	50.7
II	✓	47.0	69.1	50.3	49.2	72.7	53.4
III	✓	✓	47.3	68.9	50.8	49.7	73.4	53.9
approximately 3 days, but its performance is  $64.3\%$  AP. Our method is proved to be more effective and data-efficient.
YouTube-VIS 2021. Table 1 also compares the results on the YouTube-VIS 2021 dataset. Specifically, based on Mask2Former-VIS, TMT-VIS achieves  $44.9\%$  AP and  $56.4\%$  AP with ResNet-50 and Swin-L backbones, respectively. Compared to the baseline, TMT-VIS boosts performance by 4.3 and 3.8 points. When the exquisite designs are added to VITA, TMT-VIS achieves  $48.3\%$  AP and  $59.3\%$  AP respectively, boosting baseline performance by 2.6 and 1.8 points. When plugging to GenVIS, TMT-VIS achieves  $49.4\%$  AP and  $61.9\%$  AP respectively, boosting baseline performance by  $3.1\%$  AP and  $1.8\%$  AP.
OVIS and UVO. Table 2 illustrates the competitiveness of TMT-VIS on the challenging OVIS dataset. Particularly, TMT-VIS achieves  $25.1\%$  AP and  $32.5\%$  AP with ResNet-50 and Swin-L backbones with VITA as the base model, and harvests  $38.4\%$  AP and  $46.9\%$  AP with GenVIs as the base model on these backbones. Table 2 shows the strong performance of TMT-VIS among offline methods on the challenging UVO dataset. These appealing and encouraging results further prove the effectiveness and generality of the proposed approach on multiple dataset training.
In the following, we conduct extensive ablation studies to check the properties of each component. We first show that our algorithm can improve model performance by a large margin when trained on different combinations of datasets. Then we examine the detailed ablations of our design. Note that all of the TMT-VIS ablation experiments are conducted based on Mask2Former-VIS with the ResNet-50 backbone.
Multiple dataset joint-training problem. Even though mask precision increases with the data volume, due to the heterogeneity in category space, simply utilizing multiple datasets will dilute
Table 5: Ablation study on sampling ratio of multiple datasets (The order of the datasets is YTVIS:UVO:OVIS).
Sampling Ratio	AP	AP50	AP75
2:1:0.75	48.6	71.0	52.4
1:1:0.5	49.4	72.8	53.2
1:1:0.75	49.7	73.4	53.9
1:1:1	49.5	72.6	53.7
Table 6: Ablation study on the size of taxonomic embedding set in TCM.
Size	AP	Size	AP
1	47.1	15	49.2
2	47.5	20	48.5
5	49.3	50	47.4
10	49.7	100	47.2
Table 7: Ablation study on aggregation strategy in TIM.
Method	AP	\( AP_{50} \)	\( AP_{75} \)
Add	47.6	71.8	50.3
Concatenation	47.4	71.0	51.1
Cross-attention	49.7	73.4	53.9
the attention of models on different categories. In Table 3, we carefully examine the performance of multiple datasets joint training on various datasets. We find that the popular Mask2Former-VIS framework meets difficulties in dealing with datasets with different taxonomy spaces. As the number of datasets increases, the performance may even decrease. In contrast, as we increase the number of datasets, the performance of our TMT-VIS improves gradually.
Key component designs. Table 4 demonstrates the effect of our component designs when combined with the prevalent Mask2Former-VIS. By adopting our algorithm, Mask2Former-VIS achieves a huge gain of 2.5 points in AP performance when both models are trained with three VIS datasets. Our extra loss also demonstrates a gain of 0.5 AP.
Datasets sampling ratio. The sampling ratio of different VIS datasets is a crucial parameter. Diverse sampling ratios can cause model to focus on different categories, which can impact the attention and performance of the model. In this study, we arrange the sampling ratio according to the optimal number of iterations each dataset has trained individually. As shown in Table 5, '1:1:0.75' ratio yields the best results, which aligns with our previous training approaches on a single dataset. The performance gap between different ratios validates that selecting an appropriate sampling ratio is important for ensuring that the model learns the relevant category information effectively and achieves optimal performance during multi-dataset training.
4.4 TCM and TIM
In Tables 6 and 7, we investigate the effect of different sizes of taxonomic embedding set as well as the aggregation strategy used in TCM and TIM.
Size of taxonomic embedding set. Table 6 displays the performance of TMT-VIS with varying numbers of output taxonomic embedding  $N_{T}$  in the TCM from 1 to 100. The results indicate that the model performs best when  $N_{T} = 10$  embeddings are selected as input embeddings to TIM. When  $N_{T}$  is set to 1 or 2, which is apparently less than the overall categories in input video, topk taxonomic embeddings miss the groundtruth classes. As  $N_{T}$  decreases, the aggregated taxonomic guidance contained in embeddings becomes insufficient, and the selected embeddings may not encode taxonomic semantics for all instances in the video, resulting in a drop in performance. Conversely, when  $N_{T}$  gets larger than the optimum value, the redundant taxonomic embeddings dilute the category information, leading to performance degradation. When using all embeddings, the improvement is trivial since TCM can not provide filtration to irrelevant classes, and the TIM module is simply injecting the information of the whole category space to queries.
Aggregation strategy. Table 7 demonstrates the results of different aggregation strategies implemented in the TIM. There are multiple ways of aggregating taxonomic embeddings to queries in the transformer decoder: in the 'add' setting, taxonomic embeddings are expanded to match the dimensions of queries and addition is applied to the inflated embeddings; in the 'concatenation' setting, taxonomic embeddings are concatenated with queries to provide additional semantic queries; in the 'cross-attention' setting, taxonomic embeddings are fed to the cross-attention module as key and values to inject category semantics into queries. The results indicate that aggregating taxonomy guidance can improve multi-dataset training performance, and that cross-attention is found to be the most effective strategy for injecting taxonomic embeddings into instance queries. On the other hand, directly concatenating or adding taxonomic embeddings does not improve significantly. This suggests that straightforward modifications to instance queries don't infuse taxonomy information into queries, and such superficial modifications will be refined after passing the cascades of the decoder.
Figure 3: Visual comparison of our model with Mask2Former-VIS (abbreviated as 'M2F-VIS'). Our TMT-VIS shows better precision in segmenting small instances, e.g., the swimmers in the middle of the image, and classifying similar instances, e.g., truck and sedan are similar categories, their appearances are similar but different in sizes.
We also conduct qualitative evaluations and the visual comparison with Mask2Former-VIS. Results are illustrated in Fig. 4. Our TMT-VIS demonstrates better capacity in segmenting small instances and classifying instances with similar categories. The reason is that our TMT-VIS model is benefited from our taxonomy compilation module with multi-dataset training.
In this paper, we propose TMT-VIS to address the dilemma in multiple dataset joint training in VIS. By developing a two-stage module, Taxonomy Compilation Module (TCM) and Taxonomy Injection Module (TIM), our new algorithm is able to train and utilize multiple datasets effectively by incorporating taxonomy guidance into the DETR-based model. Our proposed TMT-VIS harvests great performance improvements over the baselines and sets new state-of-the-art records on multiple popular and challenging VIS datasets and benchmarks. We hope that our new approach can provide valuable insights and motivate future VIS research.
Limitations. The utilization and aggregation of taxonomic embeddings may need further investigations, and there might be more sophisticated ways of associating taxonomic embeddings with multi-level queries that may further improve the performance.
Broader impacts. TMT-VIS is designed for effectively training multiple VIS datasets which achieves promising performance. We hope that TMT-VIS can have a positive impact on many industrial areas where dataset biases are severe. We would like to note that VIS research should mind not violate personal privacy.
Acknowledgement. This work is partially supported by the National Natural Science Foundation of China (No. 62201484), National Key R&D Program of China (No. 2022ZD0160100), HKU Startup Fund, and HKU Seed Fund for Basic Research.
[43] Jialian Wu, Sudhir Yarram, Hui Liang, Tian Lan, Junsong Yuan, Jayan Eledath, and Gerard Medioni. Efficient video instance segmentation via tracklet query and proposal. In CVPR, 2022. 6
[44] Junfeng Wu, Yi Jiang, Song Bai, Wenqing Zhang, and Xiang Bai. Seqformer: Sequential transformer for video instance segmentation. In ECCV, 2022. 1, 6, 7
[45] Junfeng Wu, Qihao Liu, Yi Jiang, Song Bai, Alan Yuille, and Xiang Bai. In defense of online models for video instance segmentation. In ECCV, 2022. 3, 6, 7, 14
[46] Yuxin Wu, Alexander Kirillov, Francisco Massa, Wan-Yen Lo, and Ross Girshick. Detector2. https://github.com/facebookresearch/detectron2, 2019.5
[47] Linjie Yang, Yuchen Fan, and Ning Xu. Video instance segmentation. In ICCV, 2019. 1, 2, 3, 6, 7, 14, 15
[48] Shusheng Yang, Yuxin Fang, Xinggang Wang, Yu Li, Chen Fang, Ying Shan, Bin Feng, and Wenyu Liu. Crossover learning for fast online video instance segmentation. In ICCV, 2021. 3, 6, 7
[49] Shusheng Yang, Xinggang Wang, Yu Li, Yuxin Fang, Jiemin Fang, Wenyu Liu, Xun Zhao, and Ying Shan. Temporally efficient vision transformer for video instance segmentation. In CVPR, 2022. 1, 6
[50] Yifu Zhang, Chunyu Wang, Xinggang Wang, Wenjun Zeng, and Wenyu Liu. Fairmot: On the fairness of detection and re-identification in multiple object tracking. IJCV, 2021. 3
[51] Tiancheng Zhao, Peng Liu, Xiaopeng Lu, and Kyusong Lee. Omdet: Language-aware object detection with large-scale vision-language multi-dataset pre-training. arXiv:2209.05946, 2022. 3
[52] Xingyi Zhou, Vladlen Koltun, and Philipp Krahenbuhl. Simple multi-dataset detection. In CVPR, 2022. 3
[53] Xingyi Zhou, Tianwei Yin, Vladlen Koltun, and Philipp Krahenbuhl. Global tracking transformers. In CVPR, 2022. 3
[54] Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable transformers for end-to-end object detection. In ICLR, 2020. 3
This appendix provides more details about the proposed TMT-VIS, further details of VIS datasets, more qualitative visual comparisons.
Here, we provide a detailed overview of various VIS datasets in Table 8. Our extensive experimental evaluations are conducted on four challenging benchmarks, namely YouTube-VIS 2019 and 2021 [47], OVIS [35], and UVO [39]. YouTube-VIS 2019 [47] was the first large-scale dataset designed for video instance segmentation, comprising 2.9K videos averaging 4.61s in duration and 27.4 frames in validation videos. YouTube-VIS 2021 [47] poses a greater challenge with longer and more complex trajectory videos, averaging 39.7 frames in validation videos. The OVIS [35] dataset is another challenging VIS dataset with 25 object categories, focusing on complex scenes with significant object occlusions. Despite containing only 607 training videos, OVIS's videos last an average of 12.77s. Lastly, the UVO [39] dataset consists of 1.2K Kinetics-400 [20] videos, densely annotated at 30fps, featuring 81 object categories, including an extra "other" category for non-COCO instances. It provides exhaustive segmentation masks for all object instances in the 503 videos.
Among all categories in Youtube-VIS 2019, OVIS, and UVO, there are overlapping categories between each dataset, and there are also different categories that share similar semantics. The detailed overlapping categories are marked in Table 9. Overall, Youtube-VIS 2021 and OVIS share a more similar taxonomy space with Youtube-VIS 2019 than UVO, with a common category set of 34 out of 40 for Youtube-VIS 2021 and 22 out of 25 for OVIS. Typically, when the taxonomy spaces of datasets are similar, training them jointly will have smaller dataset biases, which leads to a better result in performance. The characteristics of these datasets align with the improvement in performance when validating the joint-training models on various datasets: the increase is more significant on Youtube-VIS 2021 and OVIS than on UVO. Further details of some specific categories can be found in Table 11.
Table 8: Key statistics of popular VIS datasets. Note that in UVO, the majority of the videos are for Video Object Segmentation, and only 503 videos are annotated for the VIS task. 'YTVIS' is the acronym of 'Youtube-VIS'.
YTVIS19	YTVIS21	OVIS	UVO
Videos	2883	3859	901	11228
Categories	40	40	25	81
Instances	4883	8171	5223	104898
Masks	131K	232K	296K	593K
Masks per Frame	1.7	2.0	4.7	12.3
Object per Video	1.6	2.1	5.8	9.3
B Additional Ablation Studies
In this section, we provide more experiments on our proposed methods, we will discuss the generality and zero-shot properties of our training approach, and we will provide further details of the performance change in different taxonomies.
Generality property. The proposed new taxonomy-aware training strategy is an effective and general strategy that can be adapted into various DETR-based approaches (both online & offline) and into various datasets. As in Table 10, when adding our TCM&TIM strategy to the popular VIS architecture Mask2Former-VIS [8], VITA [16], and IDOL [45], we harvest performance gains on all of the three challenging benchmarks. In OVIS, the increase can be up to  $6.3\%$  AP for Mask2Former-VIS. As for the popular IDOL, our strategy can also bring about an increase of  $2.8\%$  AP in performance. This demonstrates that the proposed taxonomy-aware module can be treated as a plug-and-play design that can be used in various DETR-based VIS methods (both online & offline) across different scenarios and all popular VIS benchmarks.
Per-category performance. In Table 11, we present the comparison in performance between Mask2Former-VIS and our TMT-VIS on several specific taxonomies. When datasets other than YTVIS have no such taxonomy, training Mask2Former-VIS on multiple datasets will end up decreasing the performance of such category, as shown in 'Duck' case from Table 11. When applying our approach, we can obtain a performance improvement due to that taxonomy information of 'Duck' is compiled and injected to the instance queries. In other taxonomies, such as 'Person' which appears across all VIS datasets, the improvements are also significant.
Zero-shot property. Further, our TMT-VIS can also perform well on zero-shot learning. We conducted the experiments on Youtube-VIS 2019 [47], OVIS [35], and UVO [39] benchmarks. As exhibited in Table 12, TMT-VIS can be utilized to transfer the knowledge of other VIS datasets to another dataset with a significant increase of  $4.5\%$  AP and  $3.8\%$  AP respectively. The extra taxonomy information provided by our newly designed TCM & TIM improve the model's performance when dealing with unfamiliar taxonomies.
In the visualization comparisons between Mask2Former-VIS and our model, we select some cases under different scenarios, which include setting with multiple similar instances, setting with fast-moving objects, and setting with different poses of instance.
In Fig. 4, we demonstrate videos with multiple similar instances, and TMT-VIS can both segment and track them more accurately than Mask2Former-VIS. The swimmers or the cyclists are all instances that belong to 'person' category, and TMT-VIS shows better segmentation and tracking. In Fig. 5, we present example videos which have quick movements in camera's perspective and have instances with different poses. In the top two rows, the sedan and the truck have similar appearances, and our model can distinguish and segment them with higher confidence (90% over 70%). In the last two rows, our model successfully segments the person in different poses, while Mask2Former-VIS fails to segment this person's arm in the first frame. However, the first two rows of Fig. 5 also show that our model still have the problem of segmenting instances with heavy occlusions. This suggests that simply combining taxonomy information is insufficient of solving severely occluded scenes, and that more information should be aggregated to instance queries to make the model more robust in segmenting instances.
Table 9: Overlapping categories of multiple VIS datasets with Youtube-VIS 2019 dataset. 'YTVIS' is the acronym of 'Youtube-VIS'. As demonstrated in the table, YTVIS2021 and OVIS have a more similar taxonomy space, with 34 and 22 overlapping categories respectively.
YTVIS19_40_categories	YTVIS21_40_categories	OVIS_25_categories	UVO_81_categories
Overlapping Categories	34	22	19
Person	✓	✓	✓
Giant_panda	✓	✓
Lizard	✓	✓
Parrot	✓	✓
Skateboard	✓	✓
Sedan	✓	✓
Ape
Dog	✓	✓	✓
Snake	✓
Monkey	✓	✓
Hand
Rabbit	✓	✓
Duck	✓
Cat	✓	✓	✓
Cow	✓	✓	✓
Fish	✓	✓
Train	✓	✓
Horse	✓	✓	✓
Turtle	✓	✓
Bear	✓	✓	✓
Motorbike	✓	✓	✓
Giraffe	✓	✓	✓
Leopard	✓
Fox	✓
Deer	✓
Owl
Surfboard	✓
Airplane	✓	✓	✓
Truck	✓	✓	✓
Zebra	✓	✓	✓
Tiger	✓	✓
Elephant	✓	✓	✓
Snowboard	✓	✓
Boat	✓	✓	✓
Shark	✓
Mouse	✓
Frog	✓
Eagle
Earless_seal	✓
Tennis_racket	✓	✓
Table 10: Ablation study on the generality property of TCM/TIM design with ResNet-50 backbone on multiple datasets.
Datasets	Method	AP	\( AP_{50} \)	\( AP_{75} \)
YouTube-VIS 2019	Mask2Former-VIS	46.4	68.0	50.0
+ TCM/TIM	49.7 (↑3.3)	73.4 (↑5.4)	53.9 (↑3.9)
VITA	49.8	72.6	54.5
+ TCM/TIM	52.6 (↑2.8)	74.4 (↑1.8)	57.6 (↑3.1)
IDOL	49.5	74.0	52.9
+ TCM/TIM	51.4 (↑1.9)	74.9 (↑0.9)	55.0 (↑2.1)
YouTube-VIS 2021	Mask2Former-VIS	40.6	60.9	41.8
+ TCM/TIM	44.9 (↑4.3)	66.1 (↑5.2)	48.5 (↑6.7)
VITA	45.7	67.4	49.5
+ TCM/TIM	48.3 (↑2.6)	69.8 (↑2.4)	50.8 (↑1.3)
IDOL	43.9	68.0	49.6
+ TCM/TIM	45.8 (↑1.9)	69.2 (↑1.2)	50.9 (↑1.3)
OVIS	Mask2Former-VIS	16.5	36.5	14.6
+ TCM/TIM	22.8 (↑6.3)	43.6 (↑7.1)	21.7 (↑7.1)
VITA	19.6	41.2	17.4
+ TCM/TIM	25.1 (↑5.5)	45.9 (↑4.7)	23.8 (↑6.4)
IDOL	30.2	51.3	30.0
+ TCM/TIM	33.0 (↑2.8)	55.7 (↑4.4)	33.2 (↑3.2)