Table 6: Average and standard deviation of ImageNet pre-trained models distance-based visual alignment and reliability score across 5 seeds. Bold indicates the best performance in each category and underline is the second best. Deep Ensemble does not have standard deviation since it is the output of 5 different seeds. For comparison, please refer to Table 3 for results without pre-training.
Visual Alignment (↓)	Reliability score (↑)
Must-Act	Must-Abstain	Uncertain	Average	RS0	RS450	RS900
Category 1	Category 2	Category 3	Category 4	Category 5	Category 6	Category 7	Category 8	Average
ViT [11]
SP	0.064±0.001	0.107±0.001	0.085±0.001	0.211±0.006	0.760±0.003	0.439±0.004	0.650±0.006	0.262±0.002	0.322±0.002	710	-77590	-155890
ASP	0.033±0.000	0.062±0.001	0.044±0.001	0.999±0.000	0.999±0.000	0.999±0.000	0.999±0.000	0.564±0.000	0.587±0.000	390	-226410	-453210
MD [35]	0.218±0.001	0.341±0.002	0.236±0.001	0.609±0.004	0.764±0.002	0.694±0.004	0.613±0.004	0.402±0.003	0.485±0.001	634	-109616	-219866
KNN [67]	0.399±0.001	0.588±0.001	0.465±0.001	0.450±0.001	0.469±0.001	0.556±0.001	0.300±0.002	0.452±0.000	0.460±0.000	639	-29061	-58761
TAPUDD [13]	0.320±0.017	0.405±0.021	0.315±0.017	0.657±0.021	0.733±0.014	0.753±0.014	0.616±0.029	0.441±0.008	0.530±0.004	587	-132163	-264913
OpenMax [3]	0.042±0.002	0.068±0.000	0.049±0.001	0.728±0.006	0.868±0.006	0.750±0.010	0.820±0.006	0.420±0.002	0.468±0.002	579	-138021	-276621
MC-Dropout [16]	0.034±0.000	0.064±0.001	0.046±0.001	0.909±0.000	0.964±0.000	0.927±0.000	0.947±0.001	0.519±0.000	0.551±0.000	390	-226410	-453210
Deep Ensemble [32]	0.064	0.107	0.085	0.208	0.759	0.437	0.649	0.261	0.321	708	-78042	-156792
Swin Transformer [38]
SP	0.149±0.104	0.179±0.104	0.168±0.100	0.212±0.021	0.711±0.073	0.383±0.060	0.637±0.089	0.319±0.016	0.344±0.010	737	-44263	-89263
ASP	0.083±0.067	0.105±0.068	0.099±0.064	0.999±0.000	0.999±0.000	0.999±0.000	0.999±0.000	0.599±0.028	0.610±0.029	383	-229567	-459517
MD [35]	0.127±0.053	0.183±0.048	0.143±0.051	0.759±0.004	0.854±0.003	0.851±0.002	0.667±0.006	0.485±0.026	0.509±0.022	537	-156963	-314463
KNN [67]	0.293±0.029	0.371±0.024	0.344±0.023	0.280±0.002	0.573±0.001	0.460±0.002	0.386±0.002	0.374±0.013	0.385±0.011	732	-33468	-67668
TAPUDD [13]	0.181±0.041	0.220±0.038	0.189±0.040	0.850±0.006	0.846±0.008	0.926±0.004	0.742±0.017	0.540±0.026	0.562±0.019	421	-211979	-424379
OpenMax [3]	0.092±0.071	0.116±0.072	0.107±0.069	0.762±0.007	0.815±0.010	0.727±0.021	0.800±0.012	0.476±0.029	0.487±0.031	585	-135315	-271215
MC-Dropout [16]	0.086±0.068	0.110±0.069	0.104±0.065	0.910±0.000	0.946±0.007	0.921±0.004	0.932±0.005	0.548±0.027	0.570±0.027	383	-229567	-459517
Deep Ensemble [32]	0.178	0.206	0.195	0.214	0.703	0.383	0.634	0.322	0.354	701	-79849	-160399
DenseNet [27]
SP	0.535±0.375	0.553±0.356	0.561±0.344	0.673±0.190	0.746±0.090	0.735±0.106	0.733±0.118	0.609±0.226	0.643±0.223	361	-135089	-270539
ASP	0.503±0.400	0.517±0.386	0.521±0.379	0.999±0.001	0.999±0.001	0.999±0.001	0.999±0.001	0.777±0.131	0.789±0.162	172	-326528	-653228
MD [35]	0.567±0.316	0.600±0.281	0.578±0.305	0.788±0.123	0.817±0.116	0.821±0.109	0.752±0.099	0.634±0.174	0.695±0.187	209	-305791	-611791
KNN [67]	0.575±0.329	0.604±0.297	0.597±0.302	0.697±0.032	0.723±0.039	0.716±0.038	0.710±0.023	0.606±0.130	0.654±0.146	489	-47661	-95811
TAPUDD [13]	0.655±0.201	0.660±0.204	0.636±0.230	0.853±0.038	0.840±0.064	0.855±0.046	0.791±0.053	0.696±0.093	0.748±0.105	227	-286873	-573973
OpenMax [3]	0.512±0.394	0.529±0.378	0.535±0.367	0.806±0.100	0.847±0.059	0.832±0.059	0.825±0.054	0.674±0.154	0.695±0.194	216	-300384	-600984
MC-Dropout [16]	0.512±0.387	0.543±0.349	0.547±0.343	0.961±0.043	0.963±0.040	0.962±0.041	0.963±0.039	0.755±0.155	0.776±0.175	172	-326528	-653228
Deep Ensemble [32]	0.566	0.575	0.579	0.713	0.794	0.781	0.778	0.622	0.676	583	-132617	-265817
ConvNeXt [39]
SP	0.330±0.393	0.359±0.376	0.338±0.384	0.658±0.197	0.832±0.055	0.686±0.172	0.819±0.064	0.517±0.246	0.567±0.235	369	-237681	-475731
ASP	0.314±0.402	0.335±0.391	0.321±0.395	0.999±0.001	0.999±0.001	0.999±0.001	0.999±0.001	0.685±0.155	0.706±0.168	369	-237681	-475731
MD [35]	0.380±0.348	0.407±0.333	0.402±0.328	0.690±0.095	0.769±0.039	0.639±0.134	0.711±0.024	0.536±0.177	0.567±0.181	630	-97201	-194670
KNN [67]	0.364±0.369	0.398±0.352	0.389±0.348	0.609±0.123	0.715±0.039	0.625±0.082	0.662±0.099	0.462±0.127	0.528±0.107	716	-33934	-68584
TAPUDD [13]	0.628±0.168	0.616±0.176	0.624±0.167	0.808±0.073	0.670±0.050	0.806±0.083	0.710±0.032	0.653±0.104	0.689±0.101	235	-158165	-316565
OpenMax [3]	0.319±0.406	0.345±0.389	0.333±0.393	0.796±0.056	0.807±0.033	0.728±0.121	0.802±0.023	0.537±0.197	0.583±0.194	660	-85290	-171240
MC-Dropout [16]	0.315±0.402	0.337±0.399	0.322±0.394	0.953±0.032	0.971±0.017	0.955±0.030	0.970±0.018	0.658±0.173	0.685±0.182	369	-237681	-475731
Deep Ensemble [32]	0.432	0.448	0.438	0.651	0.827	0.681	0.812	0.532	0.603	593	-134407	-269407
MLP-Mixer [68]
SP	0.198±0.294	0.279±0.269	0.234±0.282	0.550±0.101	0.742±0.005	0.589±0.080	0.650±0.051	0.422±0.160	0.458±0.155	608	-35842	-72292
ASP	0.165±0.295	0.228±0.281	0.196±0.286	0.999±0.000	0.999±0.000	0.999±0.000	0.999±0.000	0.686±0.096	0.659±0.120	289	-268361	-537011
MD [35]	0.303±0.232	0.391±0.210	0.339±0.217	0.726±0.025	0.710±0.030	0.685±0.038	0.706±0.022	0.535±0.118	0.549±0.105	498	-121452	-243402
KNN [67]	0.270±0.249	0.362±0.227	0.320±0.231	0.642±0.020	0.698±0.010	0.648±0.037	0.616±0.014	0.478±0.112	0.549±0.115	639	-29511	-59661
TAPUDD [13]	0.553±0.115	0.572±0.120	0.559±0.114	0.815±0.015	0.688±0.010	0.802±0.018	0.749±0.036	0.649±0.072	0.673±0.046	331	-133319	-266969
OpenMax [3]	0.170±0.298	0.245±0.279	0.203±0.288	0.830±0.007	0.857±0.010	0.838±0.007	0.811±0.011	0.563±0.123	0.565±0.126	318	-249432	-499182
MC-Dropout [16]	0.166±0.295	0.230±0.280	0.197±0.285	0.936±0.017	0.960±0.004	0.939±0.015	0.949±0.010	0.643±0.108	0.628±0.127	289	-268361	-537011
Deep Ensemble [32]	0.310	0.369	0.338	0.558	0.736	0.595	0.652	0.440	0.500	647	-92503	-185653