SlowGuess/NeurIPS_2023_OCR huggingface dataset:
and realizes that when  $x, y \in [0,1]^p$ , set operations can be relaxed with norm functions:
$$
| x | = \| x \| _ {1}, | y | = \| y \| _ {1}, | x \Delta y | = \| x - y \| _ {1}. \tag {21}
$$
Taking these together, the loss is defined as:
$$
\mathrm {J M L} = 1 - \frac {\| x + y \| _ {1} - \| x - y \| _ {1}}{\| x + y \| _ {1} + \| x - y \| _ {1}}. \tag {22}
$$
JML has the property that it is identical to the widely used soft Jaccard loss (SJL) [40, 41] when  $y \in \{0,1\}^p$ , and it is a metric on  $[0,1]^p$ . Therefore, it is compatible with soft labels, e.g.  $y = 0.5$  while SJL is not [50].
JML extends discrete IoU with a continuous surrogate. It was used to optimize mIoU $^{\text{D}}$ , but it is straightforward to adjust it to optimize fine-grained mIoUs by simply modifying how IoU values are aggregated during the loss computation.
G Results
In this section, we provide detailed tabular results, image-level histograms, and examples of the worst-case images for each dataset. In Table 13, we present the count of three most common worst-case images for each dataset.
Table 13: The count of three most common worst-case images for each dataset.
Dataset	#1	#2	#3
Cityscapes	9	6	0
Nighttime Driving	6	2	1
Dark Zurich	4	4	2
Mapillary Vistas	10	2	1
CamVid	5	3	2
ADE20K	3	3	2
COCO-Stuff	11	2	2
PASCAL VOC	4	4	3
PASCAL Context	3	3	2
DeepGlobe Land	13	2	0
DeepGlobe Road	9	3	1
DeepGlobe Building	11	2	2
G.1 Cityscapes
DeepLabV3+-ResNet101 min: 44.76, max: 94.66 mean: 76.21, std: 7.31 skew: -0.53, kurtosis: 0.65
DeepLabV3+-ResNet50 min: 43.73, max: 94.52 mean: 75.33, std: 7.44 skew: -0.50, kurtosis: 0.37
DeepLabV3+-ResNet34 min: 44.54, max: 93.57 mean: 75.24, std: 7.46 skew: -0.52, kurtosis: 0.50
DeepLabV3+-ResNet18 min: 48.19, max: 93.15 mean: 74.22, std: 7.60 skew: -0.45, kurtosis: 0.25
DeepLabV3+-EfficientNetB0 min: 44.92, max: 94.18 mean: 74.60, std: 7.48 skew: -0.43, kurtosis: 0.26
DeepLabV3+-MobileNetV2 min:44.92,max:93.93 mean:73.60,std:7.61 skew:-0.51,kurtosis:0.30
DeepLabV3+-MobileViTV2 min:44.05,max:94.31 mean:75.06,std:7.32 skew:-0.51,kurtosis:0.49
UPerNet-ResNet101 min: 45.71, max: 93.94 mean: 75.72, std: 7.49 skew: -0.52, kurtosis: 0.41
UPerNet-ConvNeXt min: 48.59, max: 95.03 mean: 77.13, std: 7.24 skew: -0.48, kurtosis: 0.28
UPerNet-MiTB4 min:52.55,max:94.65 mean:76.52,std:7.02 skew:-0.38,kurtosis:-0.10
SegFormer-MiTB4 min: 49.05, max: 93.95 mean: 76.78, std: 7.07 skew: -0.48, kurtosis: 0.19
SegFormer-MiTB2 min: 51.16, max: 94.06 mean: 75.66, std: 7.18 skew: -0.42, kurtosis: 0.09
DeepLabV3-ResNet101 min: 48.09, max: 94.62 mean: 75.26, std: 7.32 skew: -0.47, kurtosis: 0.23
PSPNet-ResNet101 min:47.54,max:94.16 mean:75.18,std:7.30 skew:-0.42,kurtosis:0.04
UNet-ResNet101 min:45.16,max:94.10 mean:75.36,std:7.34 skew:-0.46,kurtosis:0.38
Figure 6: Histograms: Cityscapes.
DeepLabV3+-ResNet101: lindau_000045_000019_leftImg8bit (IoU $_i^{\mathrm{I}}$  = 43.56).
DeepLabV3+-ResNet50: lindau_000045_000019_leftImg8bit (IoU_i^I = 41.71).
DeepLabV3+-ResNet34: lindau_000034_000019_leftImg8bit (IoU $_i^{\mathrm{I}}$  = 41.34).
DeepLabV3+-ResNet18: lindau_000034_000019_leftImg8bit (IoU_i^I = 43.90).
DeepLabV3+-EfficientNetB0: lindau_000045_000019_leftImg8bit (IoU $_i^{\mathrm{I}} = 42.24$ ).
Figure 7: Worst-case images: Cityscapes  $1/3$ . Left: image. Middle: label. Right: prediction.
DeepLabV3+-MobileNetV2: lindau_000034_000019_leftImg8bit (IoU $_i^{\mathrm{I}} = 41.51$ ).
DeepLabV3+-MobileViTV2: lindau_000045_000019_leftImg8bit (IoU_i^I = 42.11).
UPerNet-ResNet101: lindau_000045_000019_leftImg8bit (IoU $_i^{\mathrm{I}}$  = 44.28).
UPerNet-ConvNeXt: lindau_000034_000019_leftImg8bit (IoU $_i^{\mathrm{I}} = 44.72$ ).
Figure 8: Worst-case images: Cityscapes  $2/3$ . Left: image. Middle: label. Right: prediction.
UPerNet-MiTB4: lindau_000045_000019_leftImg8bit (IoU<sub>i</sub> = 44.25).
SegFormer-MiTB4: lindau_000045_000019_leftImg8bit (IoU_i^I = 45.04).
SegFormer-MiTB2: lindau_000034_000019_leftImg8bit (IoU_i^I = 46.96).
DeepLabV3-ResNet101: lindau_000034_000019_leftImg8bit (IoU $_i^{\mathrm{I}}$  = 44.04).
PSPNet-ResNet101: lindau_000045_000019_leftImg8bit (IoU $_i^{\mathrm{I}}$  = 45.87).
UNet-ResNet101: lindau_000045_000019_leftImg8bit (IoU $_i^{\mathrm{I}} = 42.14$ ).
Figure 9: Worst-case images: Cityscapes  $3 / 3$ . Left: image. Middle: label. Right: prediction.
G.2 Nighttime Driving
Table 15: Results: Nighttime Driving. Red: the best for each metric. Green: the worst for each metric.
Method	Backbone	mIoU1	mIoU1q	mIoU5	mIoU1
mIoUC	mIoUCq	mIoUC5	mIoUC1
mIoUD	mAcc	Acc
DeepLabV3+	ResNet101	43.02 ± 2.28	31.63 ± 2.07	13.68 ± 3.54	10.89 ± 4.59
33.15 ± 3.84	17.61 ± 2.88	5.08 ± 2.03	4.33 ± 1.70
25.93 ± 1.68	49.29 ± 2.43	69.50 ± 0.80
DeepLabV3+	ResNet50	38.46 ± 0.99	28.14 ± 1.77	14.04 ± 3.20	13.38 ± 3.21
29.18 ± 0.86	13.89 ± 0.53	2.48 ± 1.02	2.00 ± 1.41
24.28 ± 0.90	45.69 ± 2.02	58.49 ± 2.42
DeepLabV3+	ResNet34	28.85 ± 1.17	18.30 ± 0.30	4.97 ± 1.04	3.81 ± 1.03
20.49 ± 2.05	8.24 ± 0.98	1.02 ± 0.84	0.67 ± 0.94
14.35 ± 0.88	37.58 ± 3.42	53.31 ± 4.11
DeepLabV3+	ResNet18	29.66 ± 0.96	20.25 ± 0.23	7.22 ± 0.99	5.36 ± 0.38
19.27 ± 1.44	8.33 ± 0.56	0.82 ± 0.32	0.33 ± 0.47
14.20 ± 0.15	35.49 ± 2.45	57.58 ± 2.08
DeepLabV3+	EfficientNetB0	34.06 ± 2.34	23.63 ± 2.02	7.55 ± 2.75	5.71 ± 3.39
23.66 ± 1.42	10.04 ± 1.22	0.69 ± 0.40	0.00 ± 0.00
19.45 ± 1.79	40.71 ± 1.76	61.71 ± 4.52
DeepLabV3+	MobileNetV2	20.10 ± 1.19	10.23 ± 1.29	1.11 ± 0.71	0.68 ± 0.38
14.96 ± 1.44	4.78 ± 0.69	0.04 ± 0.05	0.00 ± 0.00
12.67 ± 1.48	31.86 ± 3.21	33.22 ± 3.82
DeepLabV3+	MobileViTV2	43.04 ± 1.12	32.93 ± 1.37	17.47 ± 2.35	17.00 ± 2.73
31.88 ± 1.38	16.49 ± 0.82	4.32 ± 1.03	3.67 ± 1.25
25.88 ± 0.81	43.99 ± 1.50	71.18 ± 1.90
UPerNet	ResNet101	39.39 ± 4.58	29.09 ± 4.26	14.49 ± 2.73	13.20 ± 2.86
28.96 ± 2.15	14.08 ± 1.63	1.99 ± 1.11	1.00 ± 0.82
24.24 ± 1.76	47.37 ± 1.62	63.55 ± 5.89
UPerNet	ConvNeXt	62.54 ± 1.21	53.40 ± 0.94	42.33 ± 0.68	41.86 ± 0.49
49.58 ± 0.99	33.40 ± 1.20	11.94 ± 1.42	11.33 ± 1.25
42.10 ± 0.21	63.38 ± 0.66	82.65 ± 0.33
UPerNet	MiTB4	61.27 ± 1.15	52.21 ± 1.16	39.71 ± 1.16	39.10 ± 1.17
49.89 ± 1.02	33.23 ± 0.83	13.09 ± 0.67	12.33 ± 0.94
41.59 ± 0.85	63.42 ± 1.22	83.34 ± 0.60
SegFormer	MiTB4	61.45 ± 1.53	51.21 ± 1.76	37.90 ± 3.07	36.07 ± 3.43
50.12 ± 2.94	33.41 ± 2.47	12.77 ± 1.48	12.00 ± 1.63
42.23 ± 1.66	62.04 ± 2.92	82.95 ± 1.14
SegFormer	MiTB2	50.89 ± 2.45	40.91 ± 2.29	24.63 ± 1.78	23.47 ± 1.18
41.33 ± 2.03	25.21 ± 1.32	9.61 ± 0.16	9.00 ± 0.00
31.40 ± 2.73	51.54 ± 2.54	79.19 ± 1.00
DeepLabV3	ResNet101	42.21 ± 3.33	31.12 ± 3.79	16.23 ± 3.87	15.68 ± 3.54
30.71 ± 2.58	14.88 ± 2.27	2.46 ± 1.22	1.67 ± 1.25
26.84 ± 2.30	46.24 ± 2.53	68.97 ± 6.15
PSPNet	ResNet101	40.48 ± 1.86	29.59 ± 1.97	13.87 ± 0.55	11.37 ± 1.37
30.22 ± 1.98	14.81 ± 2.10	2.57 ± 1.78	1.67 ± 1.70
23.65 ± 1.17	45.63 ± 2.74	61.03 ± 4.07
UNet	ResNet101	36.86 ± 2.73	25.72 ± 2.82	9.31 ± 3.44	8.20 ± 3.30
25.72 ± 2.38	12.31 ± 1.70	2.56 ± 1.36	1.67 ± 1.25
21.65 ± 1.17	45.33 ± 2.14	65.95 ± 3.74
DeepLabV3+-ResNet101  
min: 10.90, max: 80.01  
mean: 43.02, std: 14.41  
skew: 0.24, kurtosis: -0.18
DeepLabV3+-ResNet50 min: 13.68, max: 76.32 mean: 38.46, std: 13.00 skew: 0.53, kurtosis: 0.22
DeepLabV3+-ResNet34  
min: 6.60, max: 76.78  
mean: 28.85, std: 14.06  
skew: 0.96, kurtosis: 1.17
DeepLabV3+-ResNet18 min: 5.35, max: 70.27 mean: 29.66, std: 12.10 skew: 0.72, kurtosis: 1.12
DeepLabV3+-EfficientNetB0 min: 7.95, max: 66.73 mean: 34.06, std: 13.00 skew: 0.31, kurtosis: -0.33
DeepLabV3+-MobileNetV2 min: 0.71, max: 64.58 mean: 20.10, std: 13.82 skew: 1.02, kurtosis: 0.91
DeepLabV3+-MobileViTV2 min:18.36,max:74.31 mean:43.04,std:12.29 skew:0.44,kurtosis:-0.07
UPerNet-ResNet101 min: 16.50, max: 75.09 mean: 39.39, std: 12.86 skew: 0.50, kurtosis: -0.04
UPerNet-ConvNeXt min: 43.08, max: 87.03 mean: 62.54, std: 11.46 skew: 0.25, kurtosis: -0.66
UPerNet-MiTB4 min:39.69,max:83.31 mean:61.27,std:11.19 skew:0.11,kurtosis:-0.98
SegFormer-MiTB4 min: 39.44, max: 83.36 mean: 61.45, std: 12.34 skew: -0.06, kurtosis: -1.06
SegFormer-MiTB2 min: 26.66, max: 72.75 mean: 50.88, std: 11.89 skew: -0.18, kurtosis: -0.89
DeepLabV3-ResNet101 min: 16.55, max: 78.74 mean: 42.21, std: 14.04 skew: 0.46, kurtosis: -0.22
PSPNet-ResNet101 min: 14.53, max: 74.87 mean: 40.48, std: 13.78 skew: 0.28, kurtosis: -0.39
UNet-ResNet101 min:8.99,max:72.76 mean:36.86,std:14.05 skew:0.24,kurtosis:-0.20
Figure 10: Histograms: Nighttime Driving.
DeepLabV3+-ResNet101: 0_frame_1171_leftImg8bit (IoU_i^I = 6.57).
DeepLabV3+-ResNet50: 0_frame_1171_leftImg8bit (IoU<sub>i</sub> = 9.59).
DeepLabV3+-ResNet34: 0_frame_0287_leftImg8bit (IoU<sub>i</sub> = 2.42).
DeepLabV3+-ResNet18: 0_frame_1171_leftImg8bit  $(\mathrm{IoU}_i^{\mathrm{I}} = 4.82)$
DeepLabV3+-EfficientNetB0: 0_frame_1171_leftImg8bit (IoU $_i^{\mathrm{I}}$  = 1.53).
Figure 11: Worst-case images: Nighttime Driving  $1/3$ . Left: image. Middle: label. Right: prediction.
DeepLabV3+-MobileNetV2: 0_frame_0374_leftImg8bit (IoU<sub>i</sub> = 0.17).
DeepLabV3+-MobileViTV2: 0_frame_1138_leftImg8bit (IoU $_i^{\mathrm{I}}$  = 13.87).
UPerNet-ResNet101: 0_frame_1816_leftImg8bit (IoU_i^I = 11.13).
UPerNet-ConvNeXt: 0_frame_0432_leftImg8bit (IoU_i^I = 41.24).
UPerNet-MiTB4: 0_frame_2548_leftImg8bit (IoU_i^1 = 37.89).
Figure 12: Worst-case images: Nighttime Driving  $2/3$ . Left: image. Middle: label. Right: prediction.
SegFormer-MiTB4: 0_frame_2548_leftImg8bit (IoU_i^I = 32.86).
SegFormer-MiTB2: 0_frame_1687_leftImg8bit (IoU_i^I = 22.12).
DeepLabV3-ResNet101: 0_frame_1171_leftImg8bit (IoU_i^I = 11.02).
PSPNet-ResNet101: 0_frame_1171_leftImg8bit  $(\mathrm{IoU}_i^{\mathrm{I}} = 9.58)$
UNet-ResNet101: 0_frame_0461_leftImg8bit (IoU_i^I = 3.58).
Figure 13: Worst-case images: Nighttime Driving  $3 / 3$ . Left: image. Middle: label. Right: prediction.
G.3 Dark Zurich
Table 16: Results: Dark Zurich. Red: the best for each metric. Green: the worst for each metric.
Method	Backbone	mIoU1	mIoUq	mIoU5	mIoU1
mIoUC	mIoUCq	mIoUC5	mIoUC1
mIoUD	mAcc	Acc
DeepLabV3+	ResNet101	21.36 ± 0.56	16.66 ± 0.33	8.88 ± 0.41	8.17 ± 0.80
18.41 ± 0.69	7.83 ± 0.32	2.07 ± 0.09	1.33 ± 0.47
16.64 ± 1.25	38.15 ± 1.03	49.65 ± 1.22
DeepLabV3+	ResNet50	16.08 ± 0.44	11.66 ± 0.50	6.03 ± 0.88	5.77 ± 0.84
13.44 ± 0.52	5.00 ± 0.21	0.57 ± 0.10	0.00 ± 0.00
12.75 ± 0.79	33.18 ± 1.04	33.52 ± 1.65
DeepLabV3+	ResNet34	12.32 ± 0.13	8.18 ± 0.15	3.33 ± 0.57	2.98 ± 0.64
9.87 ± 0.26	3.54 ± 0.18	0.64 ± 0.41	0.33 ± 0.47
8.42 ± 0.43	26.33 ± 1.04	35.17 ± 2.29
DeepLabV3+	ResNet18	11.28 ± 0.62	7.86 ± 0.39	4.19 ± 0.31	3.90 ± 0.57
9.43 ± 0.50	3.54 ± 0.16	0.92 ± 0.28	0.67 ± 0.47
8.38 ± 0.48	26.04 ± 0.50	33.48 ± 2.66
DeepLabV3+	EfficientNetB0	16.86 ± 0.65	12.48 ± 0.42	6.37 ± 0.83	5.57 ± 1.31
13.42 ± 0.75	5.49 ± 0.23	1.08 ± 0.64	0.33 ± 0.47
11.21 ± 0.79	29.31 ± 1.74	38.19 ± 1.70
DeepLabV3+	MobileNetV2	9.49 ± 0.83	5.98 ± 0.82	1.52 ± 0.65	1.27 ± 0.53
6.95 ± 0.71	2.37 ± 0.24	0.12 ± 0.10	0.00 ± 0.00
6.54 ± 0.93	24.14 ± 1.85	25.17 ± 4.32
DeepLabV3+	MobileViTV2	17.02 ± 1.99	12.96 ± 2.19	6.75 ± 2.42	5.85 ± 2.18
14.62 ± 1.55	6.12 ± 0.92	1.31 ± 0.65	0.67 ± 0.47
11.51 ± 1.55	29.13 ± 2.03	41.76 ± 5.32
UPerNet	ResNet101	17.29 ± 1.14	13.22 ± 1.19	6.87 ± 1.83	6.52 ± 2.13
15.38 ± 0.82	6.10 ± 0.59	1.17 ± 0.32	0.33 ± 0.47
13.39 ± 0.24	35.64 ± 1.18	41.75 ± 2.27
UPerNet	ConvNeXt	26.03 ± 1.26	20.58 ± 1.44	12.90 ± 1.78	12.23 ± 1.50
23.19 ± 0.69	10.05 ± 0.60	1.73 ± 0.09	1.00 ± 0.00
21.98 ± 1.34	44.15 ± 0.73	51.71 ± 1.93
UPerNet	MiTB4	21.76 ± 1.77	16.64 ± 1.66	8.17 ± 1.26	7.27 ± 1.24
18.56 ± 1.22	8.01 ± 0.65	1.74 ± 0.21	1.00 ± 0.00
16.25 ± 1.05	39.52 ± 0.93	47.32 ± 1.67
SegFormer	MiTB4	24.33 ± 1.17	18.68 ± 1.08	8.59 ± 0.62	6.95 ± 0.65
21.01 ± 0.88	9.35 ± 0.66	1.92 ± 0.21	1.00 ± 0.00
18.98 ± 1.25	40.31 ± 1.35	50.83 ± 1.04
SegFormer	MiTB2	23.75 ± 0.49	18.56 ± 0.49	9.73 ± 0.16	8.30 ± 0.80
19.85 ± 0.31	8.44 ± 0.48	1.91 ± 0.41	1.33 ± 0.47
17.90 ± 0.08	36.58 ± 0.59	50.28 ± 0.52
DeepLabV3	ResNet101	19.56 ± 1.60	14.27 ± 1.72	6.55 ± 2.40	5.83 ± 2.21
17.35 ± 1.63	6.68 ± 0.93	1.49 ± 0.74	1.00 ± 0.82
15.38 ± 1.22	35.65 ± 2.53	46.86 ± 3.32
PSPNet	ResNet101	19.80 ± 1.59	15.14 ± 1.57	7.38 ± 2.11	6.62 ± 2.21
17.09 ± 1.32	6.70 ± 0.75	1.21 ± 0.39	0.67 ± 0.47
15.61 ± 0.78	36.83 ± 2.12	45.22 ± 3.65
UNet	ResNet101	16.59 ± 1.06	12.34 ± 0.86	5.57 ± 0.39	5.23 ± 0.28
14.34 ± 1.15	6.02 ± 0.40	2.23 ± 0.12	2.00 ± 0.00
13.21 ± 0.56	34.30 ± 1.80	45.02 ± 1.27
DeepLabV3+-ResNet101  
min: 9.59, max: 35.04  
mean: 21.36, std: 5.93  
skew: 0.38, kurtosis: 0.05
DeepLabV3+-ResNet50 min: 6.33, max: 30.30 mean: 16.08, std: 5.54 skew: 0.38, kurtosis: -0.30
DeepLabV3+-ResNet34  
min: 3.51, max: 27.02  
mean: 12.33, std: 5.24  
skew: 0.68, kurtosis: 0.27
DeepLabV3+-ResNet18 min: 4.65, max: 22.38 mean: 11.28, std: 4.28 skew: 0.90, kurtosis: 0.44
DeepLabV3+-EfficientNetB0 min: 7.17, max: 33.83 mean: 16.86, std: 5.30 skew: 0.68, kurtosis: 0.88
DeepLabV3+-MobileNetV2 min: 1.36, max: 18.21 mean: 9.49, std: 4.21 skew: 0.02, kurtosis: -0.86
DeepLabV3+-MobileViTV2 min: 8.26, max: 28.04 mean: 17.02, std: 4.58 skew: 0.17, kurtosis: -0.36
UPerNet-ResNet101 min: 6.86, max: 29.11 mean: 17.29, std: 5.09 skew: 0.35, kurtosis: -0.23
UPerNet-ConvNeXt min: 13.57, max: 44.89 mean: 26.03, std: 7.25 skew: 0.57, kurtosis: -0.31
UPerNet-MiTB4 min: 7.55, max: 35.92 mean: 21.76, std: 6.27 skew: 0.10, kurtosis: -0.42
SegFormer-MiTB4 min: 7.40, max: 38.94 mean: 24.33, std: 6.59 skew: -0.33, kurtosis: -0.24
SegFormer-MiTB2 min: 9.11, max: 36.02 mean: 23.75, std: 6.28 skew: -0.16, kurtosis: -0.72
DeepLabV3-ResNet101  
min: 6.49, max: 34.28  
mean: 19.56, std: 6.52  
skew: 0.19, kurtosis: -0.20
PSPNet-ResNet101 min: 6.68, max: 31.36 mean: 19.79, std: 5.60 skew: 0.03, kurtosis: -0.46
UNet-ResNet101 min: 5.67, max: 26.99 mean: 16.59, std: 5.19 skew: 0.16, kurtosis: -0.29
Figure 14: Histograms: Dark Zurich.
DeepLabV3+-ResNet101: GOPR0356_frame_000378rgb_anon  $(\mathrm{IoU}_i^{\mathrm{I}} = 7.06)$
DeepLabV3+-ResNet50: GOPR0356_frame_000398_rgb_anon  $(\mathrm{IoU}_i^{\mathrm{I}} = 5.02)$
DeepLabV3+-ResNet34: GOPR0356_frame_000378_rgb_anon  $(\mathrm{IoU}_i^{\mathrm{I}} = 2.45)$
DeepLabV3+-ResNet18: GOPR0356_frame_000398_rgb_anon  $(\mathrm{IoU}_i^{\mathrm{I}} = 3.17)$
DeepLabV3+-EfficientNetB0: GOPR0356_frame_000401_rgb_anon (IoU<sub>i</sub> = 4.38).
Figure 15: Worst-case images: Dark Zurich 1/3. Left: image. Middle: label. Right: prediction.
DeepLabV3+-MobileNetV2: GOPR0356_frame_000473_rgb_anon  $(\mathrm{IoU}_i^i = 0.53)$
DeepLabV3+-MobileViTV2: GOPR0356_frame_000398_rgb_anon  $(\mathrm{IoU}_i^{\mathrm{I}} = 3.05)$
UPerNet-ResNet101: GOPR0356_frame_000378_rgb_anon  $(\mathrm{IoU}_i^{\mathrm{I}} = 4.09)$
UPerNet-ConvNeXt: GOPR0356_frame_000398_rgb_anon  $(\mathrm{IoU}_i^{\mathrm{I}} = 10.12)$
UPerNet-MiTB4: GOPR0356_frame_000333_rgb_anon  $(\mathrm{IoU}_i^{\mathrm{I}} = 5.96)$
Figure 16: Worst-case images: Dark Zurich  $2/3$ . Left: image. Middle: label. Right: prediction.
SegFormer-MiTB4: GOPR0356_frame_000482rgb_anon  $(\mathrm{IoU}_i^{\mathrm{I}} = 6.20)$
SegFormer-MiTB2: GOPR0356_frame_000482rgb_anon  $(\mathrm{IoU}_i^{\mathrm{I}} = 7.29)$
DeepLabV3-ResNet101: GOPR0356_frame_000375_rgb_anon  $(\mathrm{IoU}_i^{\mathrm{I}} = 2.73)$
PSPNet-ResNet101: GOPR0356_frame_000375_rgb_anon  $(\mathrm{IoU}_i^{\mathrm{I}} = 4.76)$
UNet-ResNet101: GOPR0356_frame_000378_rgb_anon  $(\mathrm{IoU}_i^{\mathrm{I}} = 4.90)$
Figure 17: Worst-case images: Dark Zurich  $3 / 3$ . Left: image. Middle: label. Right: prediction.
G.4 Mapillary Vistas
DeepLabV3+-ResNet101  
min: 30.46, max: 91.24  
mean: 62.47, std: 8.25  
skew: -0.05, kurtosis: 0.33
DeepLabV3+-ResNet50 min: 25.47, max: 91.59 mean: 60.61, std: 8.45 skew: -0.03, kurtosis: 0.31
DeepLabV3+-ResNet34 min: 23.90, max: 90.44 mean: 58.35, std: 8.77 skew: -0.01, kurtosis: 0.21
DeepLabV3+-ResNet18 min: 25.02, max: 90.91 mean: 56.55, std: 8.92 skew: 0.02, kurtosis: 0.24
DeepLabV3+-EfficientNetB0 min: 31.16, max: 90.16 mean: 58.28, std: 8.72 skew: 0.05, kurtosis: 0.14
DeepLabV3+-MobileNetV2 min: 24.55, max: 88.36 mean: 55.46, std: 8.92 skew: 0.04, kurtosis: 0.24
DeepLabV3+-MobileViTV2 min: 29.06, max: 91.82 mean: 58.28, std: 8.73 skew: 0.07, kurtosis: 0.21
UPerNet-ResNet101 min: 29.68, max: 92.29 mean: 61.68, std: 8.30 skew: -0.02, kurtosis: 0.35
UPerNet-ConvNeXt min: 33.91, max: 92.64 mean: 65.26, std: 7.79 skew: -0.07, kurtosis: 0.22
UPerNet-MiTB4 min:39.82,max:92.88 mean:64.48,std:7.89 skew:0.00,kurtosis:0.20
SegFormer-MiTB4 min: 38.01, max: 91.99 mean: 64.17, std: 7.90 skew: 0.01, kurtosis: 0.20
SegFormer-MiTB2 min: 32.94, max: 91.26 mean: 62.15, std: 8.12 skew: 0.06, kurtosis: 0.24
DeepLabV3-ResNet101 min: 29.29, max: 91.85 mean: 62.33, std: 8.19 skewness: -0.02, kurtosis: 0.38
PSPNet-ResNet101 min: 27.73, max: 91.76 mean: 61.85, std: 8.17 skew: -0.03, kurtosis: 0.38
UNet-ResNet101 min:21.06,max:89.13 mean:57.29,std:8.88 skew:-0.03,kurtosis:0.27
Figure 18: Histograms: Mapillary Vistas.
DeepLabV3+-ResNet101: bvSGJVPKHwU9ApMiF19h_A (IoU $_i^{\mathrm{I}}$  = 28.85).
DeepLabV3+-ResNet50: bvSGJVPKHwU9ApMiF19h_A (IoU_i^I = 23.41).
DeepLabV3+-ResNet34: bvSGJVPKHwU9ApMiF19h_A (IoU_i^I = 23.42).
DeepLabV3+-ResNet18: bvSGJVPKHwU9ApMiF19h_A (IoU_i^I = 24.09).
DeepLabV3+-EfficientNetB0: bvSGJVPKHwU9ApMiF19h_A (IoU_i^I = 26.10).
Figure 19: Worst-case images: Mapillary Vistas  $1/3$ . Left: image. Middle: label. Right: prediction.
DeepLabV3+-MobileNetV2: 2dUuDESNqgXC2CW0gHWY7A (IoU $_i^{\mathrm{I}} = 24.02$ ).
DeepLabV3+-MobileViTV2: 2dUuDESNqgXC2CW0gHWY7A (IoU $_i^{\mathrm{I}} = 28.26$ ).
UPerNet-ResNet101: bvSGJVPKHwU9ApMiF19h_A (IoU_i^I = 27.10).
UPerNet-ConvNeXt: bvSGJVPKHwU9ApMiF19h_A (IoU $_i^{\mathrm{I}}$  = 33.15).
UPerNet-MiTB4: F1rWWZI_pxNjQ7FNya_QKg (IoU<sub>i</sub> = 37.43).
Figure 20: Worst-case images: Mapillary Vistas 2 / 3. Left: image. Middle: label. Right: prediction.
SegFormer-MiTB4: 1-RJMdtp4X1knyrpLYNPAg (IoU<sub>i</sub> = 36.03).
SegFormer-MiTB2: 7ZYHHsr3uitSnmKAHwQzNg (IoU<sub>i</sub> = 30.82).
DeepLabV3-ResNet101: bvSGJVPKHwU9ApMiF19h_A (IoU_i^I = 25.70).
PSPNet-ResNet101: bvSGJVPKHwU9ApMiF19h_A (IoU_i^I = 26.92).
UNet-ResNet101: bvSGJVPKHwU9ApMiF19h_A (IoU_i^I = 17.85).
Figure 21: Worst-case images: Mapillary Vistas  $3 / 3$ . Left: image. Middle: label. Right: prediction.
G.5 CamVid
Table 18: Results: CamVid. Red: the best for each metric. Green: the worst for each metric.
Method	Backbone	mIoU1	mIoUq	mIoU5	mIoU1
mIoUC	mIoUCq	mIoUC5	mIoUC1
mIoUD	mAcc	Acc
DeepLabV3+	ResNet101	79.97 ± 0.12	74.46 ± 0.20	65.19 ± 0.18	62.46 ± 0.66
79.48 ± 0.12	67.72 ± 0.28	37.01 ± 0.94	25.00 ± 0.82
83.30 ± 0.12	89.84 ± 0.10	95.71 ± 0.05
DeepLabV3+	ResNet50	79.22 ± 0.11	73.58 ± 0.22	63.77 ± 0.61	60.59 ± 0.59
78.69 ± 0.16	66.50 ± 0.27	35.00 ± 0.68	23.33 ± 0.94
82.56 ± 0.10	89.06 ± 0.01	95.50 ± 0.03
DeepLabV3+	ResNet34	79.16 ± 0.24	72.92 ± 0.44	62.06 ± 1.11	58.63 ± 1.84
78.71 ± 0.32	66.18 ± 0.60	33.83 ± 1.15	22.33 ± 1.70
81.98 ± 0.17	88.80 ± 0.03	95.42 ± 0.07
DeepLabV3+	ResNet18	78.12 ± 0.03	71.56 ± 0.05	60.96 ± 0.15	57.11 ± 1.03
77.55 ± 0.03	64.44 ± 0.04	32.25 ± 0.27	18.67 ± 1.25
80.79 ± 0.13	87.89 ± 0.11	95.19 ± 0.01
DeepLabV3+	EfficientNetB0	78.61 ± 0.27	72.30 ± 0.41	62.11 ± 0.91	57.70 ± 1.40
78.15 ± 0.32	65.59 ± 0.40	34.51 ± 0.57	21.00 ± 0.82
80.97 ± 0.38	88.33 ± 0.30	95.17 ± 0.08
DeepLabV3+	MobileNetV2	77.89 ± 0.08	71.35 ± 0.04	61.18 ± 0.45	56.75 ± 0.65
77.39 ± 0.12	64.42 ± 0.21	32.39 ± 0.35	20.33 ± 0.94
80.30 ± 0.16	87.64 ± 0.03	95.08 ± 0.09
DeepLabV3+	MobileViTV2	78.58 ± 0.16	72.03 ± 0.23	61.70 ± 0.34	58.49 ± 0.43
78.07 ± 0.05	65.36 ± 0.04	33.81 ± 0.45	19.00 ± 0.82
80.62 ± 0.18	88.06 ± 0.26	94.91 ± 0.26
UPerNet	ResNet101	79.76 ± 0.01	74.22 ± 0.06	65.47 ± 0.07	63.18 ± 0.34
79.24 ± 0.03	67.47 ± 0.10	37.60 ± 0.63	25.33 ± 0.47
83.00 ± 0.13	89.17 ± 0.06	95.74 ± 0.02
UPerNet	ConvNeXt	80.52 ± 0.13	75.40 ± 0.18	66.30 ± 0.59	63.35 ± 0.52
80.09 ± 0.14	68.92 ± 0.21	38.66 ± 0.35	24.67 ± 0.47
84.01 ± 0.10	90.51 ± 0.12	95.96 ± 0.04
UPerNet	MiTB4	80.53 ± 0.09	75.19 ± 0.20	65.56 ± 0.76	62.82 ± 1.32
80.09 ± 0.13	68.48 ± 0.28	37.77 ± 0.65	24.33 ± 1.25
83.32 ± 0.09	89.69 ± 0.04	95.82 ± 0.03
SegFormer	MiTB4	80.04 ± 0.13	74.61 ± 0.19	64.98 ± 0.44	61.53 ± 0.58
79.52 ± 0.12	67.88 ± 0.19	36.94 ± 0.55	23.00 ± 0.82
82.95 ± 0.23	89.68 ± 0.16	95.71 ± 0.06
SegFormer	MiTB2	79.31 ± 0.05	73.45 ± 0.10	64.22 ± 0.27	62.01 ± 1.04
78.84 ± 0.08	66.73 ± 0.13	35.77 ± 0.40	23.00 ± 0.82
82.27 ± 0.19	89.14 ± 0.15	95.53 ± 0.02
DeepLabV3	ResNet101	78.81 ± 0.04	73.33 ± 0.10	64.12 ± 0.37	61.73 ± 0.55
78.36 ± 0.02	66.63 ± 0.05	35.13 ± 0.39	22.67 ± 2.05
82.43 ± 0.08	89.15 ± 0.05	95.50 ± 0.04
PSPNet	ResNet101	78.62 ± 0.15	73.13 ± 0.27	64.18 ± 0.65	61.57 ± 0.62
78.12 ± 0.14	66.32 ± 0.26	35.74 ± 0.47	23.67 ± 0.47
82.13 ± 0.12	89.14 ± 0.01	95.46 ± 0.05
UNet	ResNet101	79.69 ± 0.29	73.83 ± 0.43	64.32 ± 0.79	61.48 ± 0.98
79.19 ± 0.35	66.99 ± 0.59	35.29 ± 0.64	23.00 ± 1.41
82.84 ± 0.31	89.26 ± 0.26	95.63 ± 0.03
DeepLabV3+-ResNet101  
min: 63.47, max: 92.76  
mean: 79.97, std: 6.78  
skew: -0.25, kurtosis: -0.66
DeepLabV3+-ResNet50  
min: 60.44, max: 92.38  
mean: 79.22, std: 7.02  
skew: -0.16, kurtosis: -0.58
DeepLabV3+-ResNet34 min: 59.57, max: 92.39 mean: 79.15, std: 7.53 skew: -0.32, kurtosis: -0.61
DeepLabV3+-ResNet18 min: 57.14, max: 92.45 mean: 78.12, std: 8.04 skew: -0.21, kurtosis: -0.79
DeepLabV3+-EfficientNetB0 min: 57.27, max: 92.33 mean: 78.62, std: 7.73 skew: -0.23, kurtosis: -0.74
DeepLabV3+-MobileNetV2 min: 56.03, max: 92.20 mean: 77.89, std: 8.07 skew: -0.19, kurtosis: -0.81
DeepLabV3+-MobileViTV2 min:59.59,max:92.39 mean:78.58,std:7.99 skew:-0.26,kurtosis:-0.83
UPerNet-ResNet101 min: 62.88, max: 92.61 mean: 79.77, std: 6.84 skew: -0.18, kurtosis: -0.74
UPerNet-ConvNeXt min: 62.46, max: 92.34 mean: 80.52, std: 6.27 skew: -0.36, kurtosis: -0.42
UPerNet-MiTB4 min:62.45,max:92.92 mean:80.54,std:6.63 skew:-0.30,kurtosis:-0.43
SegFormer-MiTB4 min: 61.12, max: 92.64 mean: 80.04, std: 6.72 skew: -0.31, kurtosis: -0.45
SegFormer-MiTB2 min: 61.87, max: 92.70 mean: 79.31, std: 7.22 skew: -0.27, kurtosis: -0.71
DeepLabV3-ResNet101 min: 61.58, max: 92.22 mean: 78.81, std: 6.78 skew: -0.22, kurtosis: -0.55
PSPNet-ResNet101 min: 61.96, max: 91.91 mean: 78.63, std: 6.82 skew: -0.16, kurtosis: -0.62
UNet-ResNet101 min: 62.01, max: 92.59 mean: 79.69, std: 7.14 skew: -0.24, kurtosis: -0.74
Figure 22: Histograms: CamVid.
DeepLabV3+-ResNet101: 0001TP_007470 (IoU $_i^{\mathrm{I}}$  = 59.90).
DeepLabV3+-ResNet50: 0001TP_007470 (IoU<sub>i</sub> = 58.81).
DeepLabV3+-ResNet34: Seq05VD_f03570 (IoU<sub>i</sub> = 54.26).
DeepLabV3+-ResNet18: Seq05VD_f03570 (IoU_i^I = 56.06).
DeepLabV3+-EfficientNetB0: 0001TP_007560 (IoU $_i^{\mathrm{I}}$  = 53.86).
Figure 23: Worst-case images: CamVid 1 / 3. Left: image. Middle: label. Right: prediction.
DeepLabV3+-MobileNetV2: 0001TP_007470 (IoU $_i^{\mathrm{I}}$  = 52.72).
DeepLabV3+-MobileViTV2: 0001TP_006930 (IoU $_i^{\mathrm{I}}$  = 57.51).
UPerNet-ResNet101: Seq05VD_f00240 (IoU $_i^{\mathrm{I}}$  = 62.33).
UPerNet-ConvNeXt: Seq05VD_f00270 (IoU $_i^{\mathrm{I}}$  = 61.81).
UPerNet-MiTB4: Seq05VD_f00240 (IoU_i^I = 59.61).
Figure 24: Worst-case images: CamVid 2 / 3. Left: image. Middle: label. Right: prediction.
SegFormer-MiTB4: 0001TP_007470 (IoU $_i^{\mathrm{I}}$  = 59.96).
SegFormer-MiTB2: 0001TP_007560 (IoU<sub>i</sub> = 58.83).
DeepLabV3-ResNet101: Seq05VD_f00270 (IoU<sub>i</sub> = 60.78).
PSPNet-ResNet101: Seq05VD_f03570 (IoU_i^I = 60.58).
UNet-ResNet101: 0001TP_007470 (IoU $_i^{\mathrm{I}}$  = 60.12).
Figure 25: Worst-case images: CamVid 3 / 3. Left: image. Middle: label. Right: prediction.
G.6 ADE20K
DeepLabV3+-ResNet101  
min: 11.20, max: 99.80  
mean: 58.79, std: 16.13  
skew: 0.03, kurtosis: -0.19
DeepLabV3+-ResNet50 min: 11.78, max: 99.93 mean: 57.15, std: 16.14 skew: 0.08, kurtosis: -0.15
DeepLabV3+-ResNet34 min: 1.88, max: 99.78 mean: 55.58, std: 16.48 skew: 0.09, kurtosis: -0.19
DeepLabV3+-ResNet18 min: 2.20, max: 99.74 mean: 53.78, std: 16.44 skew: 0.17, kurtosis: -0.04
DeepLabV3+-EfficientNetB0 min: 3.19, max: 99.79 mean: 54.17, std: 16.60 skew: 0.09, kurtosis: -0.12
DeepLabV3+-MobileNetV2 min:0.00,max:99.84 mean:52.85,std:16.57 skew:0.19,kurtosis:-0.08
DeepLabV3+-MobileViTV2 min: 8.10, max: 99.78 mean: 55.05, std: 16.55 skew: 0.16, kurtosis: -0.21
UPerNet-ResNet101 min: 6.08, max: 99.69 mean: 57.85, std: 16.02 skew: 0.07, kurtosis: -0.12
UPerNet-ConvNeXt min: 11.09, max: 99.89 mean: 60.62, std: 15.83 skew: 0.06, kurtosis: -0.29
UPerNet-MiTB4 min: 6.72, max: 99.48 mean: 60.64, std: 16.17 skew: 0.00, kurtosis: -0.22
SegFormer-MiTB4 min: 7.81, max: 99.62 mean: 60.29, std: 16.02 skew: 0.04, kurtosis: -0.16