SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In our dataset, videos are acquired by using two types of camera models. To study the domain gaps between the data collected from different devices, we focus on the binary segmentation task and adopt the best-performing model Mask2Former with the Swin transformer backbone. Specifically, we train the model on data collected from one device and test it on that of another device. The results are shown in Table 4. The performance of the models varies slightly across different devices. This suggests that the variations in the data collection process introduce some domain gaps.
C.3.3 Test across different clinics
To study model performance across different clinics, we focus on binary segmentation and SVP detection. Here, we denote the four clinics as P, Q, R, and S. We adopt the same model in Section C.3.1 and test it on each clinic test data. The results are reported in Table 5 and Table 6. In both vessel segmentation and SVP detection, the model performances across different clinics are different, indicating that domain gaps exist in different clinics.
C.3.4 Test on the third dataset
We further conduct experiments to evaluate the domain gaps between our RVD dataset and existing datasets. To be specific, we train segmentation models on RVD and DRIVE with the same amount of samples and then test them on a third dataset CHASEDB. As seen in Table 7, models trained on RVD have lower performance compared to those trained on DRIVE. This indicates our dataset exhibits a larger domain gap with the existing datasets.
Table 4: Results of models trained on one device and tested on another device.
Device	Metric	Device 1	Device 2
Device 1	mIoU	69.37	71.07
mAcc	77.07	80.45
mFscore	78.73	80.16
Device 2	mIoU	68.53	70.98
mAcc	74.2	77.99
mFscore	77.85	80.04
Table 5: Binary segmentation results of Mask2Former across different clinics.
Clinics	P	Q	R	S
mIoU	73.85	72.00	70.89	75.33
mAcc	79.63	77.72	75.63	81.78
mFscore	81.35	80.54	78.38	82.27
Table 6: Evaluation of the I3D model on SVP detection across various clinics.
Clinics	P	Q	R	S
Acc	64.58	50.00	60.00	58.33
AUROC	70.28	66.67	58.83	54.32
Recall	65.38	66.67	90.00	51.85
Figure 5: Visualization of domain gaps between different datasets. (a): Examples of binary segmentation datasets; (b): Visualization results of general Artery-Vein segmentation datasets.
C.4 Domain Gaps between RVD and Existing Datasets
In Section 4.2, we highlight the presence of domain gaps between existing datasets and our retinal vessel dataset (RVD). To further demonstrate this phenomenon, we show a set of visualization results, which can be found in Fig. 5. We consider two distinct scenarios:
(a) Initially, we present the visualization results of models trained on existing datasets and then applied to our dataset. The results reveal a concerning trend of presence of overgeneralization in the predictions, thereby overlooking finer details. This underscores the difficulty for models trained on existing datasets to generalize to our dataset.
Table 7: Evaluation of domain gaps between RVD and existing datasets. The models are trained on RVD and DRIVE and tested on CHASEDB.
Model-Backbone	RVD → CHASEDB	DRIVE → CHASEDB
mIoU	mAcc	mFscore	mIoU	mAcc	mFscore
DeepLabV3-R50	60.22	63.3	68.25	65.56	76.00	75.69
Mask2Former-R50	68.35	76.00	77.62	78.48	89.43	86.84
Mask2Former-Swin-L	63.47	67.60	73.04	79.88	90.97	87.87
(b) Conversely, we also show the visualization of models initially trained on our RVD, and then applied to existing datasets. Similar to the first case, the performance drop is observed after transferring. However, the transferred results reveal that more granular details, particularly of vessel structure, are preserved. Such a phenomenon suggests that models trained on our dataset exhibit much better generalizability and tend to adapt more efficiently to the existing datasets.
C.5 Computational Efficiency
We analyze the computational operations (i.e., GFLOPs) and inference time (i.e., ms) for trained models. Specifically, for models trained with spatial annotations, we analyze Mask2Former and DeepLabv3 with different backbones. The results are presented as follows in Table 8. The overall speed of different models is faster for processing our data. Models with larger backbones require more computational cost and more inference time, for example, from Swin-T to Swin-L, both GFLOPs and the inference time are increased a lot. However, we notice that the inference time may not linearly grow with GFLOPs across different models, since the inference time can be affected by many factors, e.g., CUDA optimization for different network structures.
For models trained with temporal annotations, we consider SVP detection with different backbones. We provide their GFLOPs and inference time in Table 9. The speed of models for SVP detection is still faster, e.g., LRCN processes each video in  $89~\mathrm{ms}$ . However, when compared with models in binary segmentation, their speed can be further improved. These results indicate that more sophisticated model structures are needed for our datasets.
Table 8: Computational efficiency of Mask2Former and DeepLabv3 with different backbones.
Model	Mask2Former	DeepLabv3
Backbone	ResNet50	ResNet101	Swin-T	Swin-S	Swin-B-1k	Swin-B-22k	Swin-L	ResNet50	ResNet101
GFLOPs (G)	40	59	28	36	86	86	160	270	347
Inference time (ms)	27.30	29.7	31.3	45.3	67.2	62.8	92.2	24.5	33.7
Table 9: Computational efficiency of models trained with temporal annotations.
Model	LRCN	I3D	X3D	TSN	VTN
GFLOPs	16	263	180	252	1374
Inference time (ms)	89.0	226.7	182.8	322.3	486.2
[1] MMSegmentation Contributors. MMSegmentation: Openmmlab semantic segmentation toolbox and benchmark. https://github.com/open-mmlab/mmsegmentation, 2020.
[2] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
[3] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition, 2015.
[4] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015: 18th
International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pages 234-241. Springer, 2015.
[5] Evan Shelhamer, Jonathan Long, and Trevor Darrell. Fully convolutional networks for semantic segmentation. IEEE transactions on pattern analysis and machine intelligence, 39(4):640-651, 2017.
[6] Robin Strudel, Ricardo Garcia, Ivan Laptev, and Cordelia Schmid. Segmenter: Transformer for semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7262-7272, 2021.
[7] Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia. Pyramid scene parsing network. In CVPR, 2017.