SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 5: We display additional results of our method combined with various previously established OOD detection methods on both the FS Static (white rows) and FS Static-C datasets(gray rows).
MSP [15]	+ Ours	Entropy [17]	+ Ours	Max logit [15]	+ Ours	Energy [30]	+ Ours	Meta-OOD [4]	+ Ours
AUC ↑	92.36	93.91	93.14	95.18	95.66	95.48	95.90	96.00	97.56	98.19
70.85	92.97	71.23	94.33	74.13	94.80	74.02	95.41	78.34	98.06
AP ↑	19.09	26.57	26.77	39.57	38.64	41.23	41.68	41.84	72.91	83.11
10.52	20.81	14.32	30.78	23.60	31.13	22.36	32.13	52.31	75.75
FPR95 ↓	23.99	20.80	23.31	18.98	18.26	20.89	17.78	17.63	13.57	11.63
100.0	22.58	100.00	20.21	89.94	23.59	89.94	18.63	100.0	11.17
C.2 Segmentation Performance on Seen Classes
In Sec. 4.3 of the main text, we demonstrated the enhancement of OOD detection performance by our method on both the original FS Static dataset and its corrupted version. Here, in Table 6, we further show the improvement of our ATTA method on the segmentation performance of seen classes. We utilize the commonly adopted metrics, mIoU (mean Intersection over Union) and mAcc (mean accuracy), and calculate them exclusively on the inlier pixels.
Notably, our method significantly improves the mIoU metric from  $58.83\%$  to  $85.64\%$  on the corrupted dataset, approaching the performance level of  $87.52\%$  achieved on the original FS Static dataset. This demonstrates the effectiveness of our method not only in OOD detection but also in seen class classification. Besides, we observe that the TBN [36] and Tent [46] methods, although providing satisfactory performance on the corrupted dataset, could harm the segmentation performance on the original dataset. In contrast, the performance gain of our method remains relatively stable across datasets with different levels of domain shift.
Table 6: Segmentation performance evaluation on seen classes in the original FS Static dataset and its corrupted version. Comparison is made with a previous OOD detection method [44] and two test-time adaptation methods [36, 46] using mIoU(%) and mAcc(%) as metrics.
mIoU ↑	mAcc↑
FS Static	FS Static + C	FS Static	FS Static + C
PEBAL [44]	87.52	58.83	97.56	72.32
+ ATTA (Ours)	87.44	85.64	97.54	97.09
+ TBN [36]	84.88	84.02	97.13	96.70
+ Tent [46]	81.66	84.07	96.63	96.72
C.3 Performance on Isolated Domain Shifts
We conduct additional experiments by introducing smog, color shifting, and Gaussian blur individually to the original FS Static dataset. Results, as shown in Table 7, reveal consistent improvements by our method across isolated domain shifts.
Table 7: We modify the original FS Static dataset by introducing fog, color shifting, and Gaussian blur separately, to analyze model performance on isolated domain shifts. We compare our method with the previous OOD detection method, PEBAL.
Fog	Color	Blur
AUC↑	AP↑	FPR95↓	AUC↑	AP↑	FPR95↓	AUC↑	AP↑	FPR95↓
PEBAL [44]	48.37	1.58	91.82	98.58	81.93	6.26	99.46	89.46	2.07
+ ATTA (Ours)	98.92	79.98	3.48	99.15	87.43	2.91	99.55	90.73	1.71
D Additional Results on Dense OOD Detection Benchmarks
D.1 Results of Other TTA Methods on the Benchmarks
We compare our method with two other TTA methods, TBN [36] and Tent [46], on three OOD detection benchmarks: the Road Anomaly dataset [29], the Fishyscapes Lost & Found dataset [1], and the Fishyscapes Static dataset [1]. Our method consistently improves upon the previous state-of-the-art OOD detection method, PEBAL [44], while other TTA methods tend to degrade the performance of the baseline model, especially in terms of the FPR95 metric. This indicates that traditional TTA methods often indiscriminately reduce the uncertainty or OOD scores of novel classes, resulting in poor OOD detection accuracy. In contrast, our method demonstrates superior performance by effectively handling the domain shift and semantic shift jointly.
Table 8: Comparison of our method with other TTA methods on the OOD detection benchmarks: Road Anomaly dataset, Fishyscapes Lost & Found dataset, and Fishyscapes Static dataset.
Methods	Road Anomaly	FS LostAndFound	FS Static
AUC↑	AP↑	FPR95↓	AUC↑	AP↑	FPR95↓	AUC↑	AP↑	FPR95↓
PEBAL [44]	87.63	45.10	44.58	98.96	58.81	4.76	99.61	92.08	1.52
+ ATTA (Ours)	92.11	59.05	33.59	99.05	65.58	4.48	99.66	93.61	1.15
+ TBN	85.56	47.81	59.43	95.40	35.01	29.37	99.05	82.45	4.05
+ Tent	85.35	47.51	60.0	95.39	35.29	29.54	99.02	81.99	4.17
D.2 Results with ResNet 101 backbone
To assess the generalization capability of our method, we conduct experiments using a different segmentation backbone: DeepLabv3+ with ResNet101. Since some previous work, such as [44], does not release their pretrained models on this backbone, we compare our method with two simpler OOD detection baselines, namely Max Logit [15] and Energy Score [30], which do not require additional training. The results in Table 9 demonstrate that our method consistently improves upon these baseline methods across three evaluation metrics.
Table 9: Evaluation of our method with the ResNet101 backbone on the Road Anomaly dataset. Our method consistently improves upon the Max Logit and Energy Score baselines.
AUC ↑	AP ↑	FPR95 ↓
Max Logit [15]	76.59	22.97	66.40
+ ATTA (Ours)	84.39	37.84	49.72
Energy [30]	77.24	24.05	66.20
+ ATTA (Ours)	84.02	38.95	51.27
D.3 Results on the SMIYC benchmark
We evaluate our method on the SegmentMeIfYouCan (SMIYC) [3] benchmark, which particularly includes significant domain shifts such as variations in illumination and weather. We submit our outputs to the benchmark test set and present the results in Table 10. Our experiments on the RoadAnomaly21 and RoadObstacle21 datasets, both part of the SMIYC benchmark, demonstrate significant improvements over the previous SOTA method PEBAL [44]. Notably, in the RoadObstacle21 dataset, PEBAL's performance is hampered by a lack of robustness, whereas our method has increased the AUPRC score from  $5.0\%$  to  $76.5\%$ . This validates our method's adaptability to substantial domain shifts.
Table 10: Results on the SMIYC official test benchmark. The results for our model were obtained by submitting the model outputs to the benchmark organizer, as required. The results for PEBAL were taken from the benchmark's official website.
RoadAnomaly21	AP↑	FPR95↓	sIoU↑	PPV↑	F1↑
PEBAL [44]	49.1	40.8	38.9	27.2	14.5
+ ATTA (Ours)	67.0	31.6	44.6	29.6	20.6
RoadObstacle21	AP↑	FPR95↓	sIoU↑	PPV↑	F1↑
PEBAL [44]	5.0	12.7	29.9	7.6	5.5
+ ATTA (Ours)	76.5	2.8	43.9	37.7	36.6
D.4 Results on the Fishyscapes Online Testing Set
We evaluate our method on the Fishyscapes Online Testing set [1]. As presented in Table 11, our method outperforms the previous state-of-the-art, PEBAL [44].
Table 11: Results on the Fishyscapes online test benchmark. The results for our model were obtained by submitting it to the benchmark organizer. The results for PEBAL were taken from their published paper.
Online FS Lost & Found	AP↑	FPR95↓
PEBAL [44]	44.17	7.58
+ ATTA (Ours)	55.94	4.66
Online FS Static	AP↑	FPR95↓
PEBAL [44]	92.38	1.73
+ ATTA (Ours)	94.68	0.68
E Additional Ablation Results
In this section, we first evaluate our method's inference overhead, then analyze the efficacy of our model components including trainable parameters, optimization techniques, and loss weight.
Analysis of Inference Time We evaluate the average inference time for each image. As shown in Table 12, our method is only 2.25 times slower than direct inference, and faster than another test-time adaptation method, Tent [46], and some ood detection methods with posthoc operations: ODIN [27], Mahalanobis Distance [25] and Synboost [10]. This efficiency is attributed to our design, which
Table 12: Comparison of inference time (seconds per image). We calculate the complete time from input to the final prediction and/or ood score. Experiments are conducted on one NVIDIA Titan Xp device, and results are averaged over all images in the FS Lost & Found validation set, with image size (1024 x 2048).
Methods	Direct Inference	ATTA (Ours)	ATTA (Ours) w/o SBN	Tent	ODIN	SynBoost	Mahalanobis
Time (s)	1.2	2.7	1.5	5.1	9.2	3.0	224.2
updates only once per image and confines learnable parameters to the classifier block. The latter design enables us to perform backward and the subsequent forward pass only on the classifier block, and is the main reason we achieve much faster inference than Tent. Furthermore, in scenarios where data from the same domain are known, we can reduce the computation by performing domain-shift detection only once, maintaining the variable for subsequent images. To illustrate this, we show the inference time without domain-shift detection, which further reduces it to 1.25 times the direct inference speed. Practically, our episodic training model allows for parallel inference across multiple processors. With ongoing hardware advancements, we anticipate a further reduction in the time gap.
Analysis of Memory Overhead For our method, the memory consumption does increase temporarily for storing activation memory during loss computation but aligns with direct inference after backpropagation. Table 13 below illustrates that our memory requirement is higher than direct inference but significantly more efficient than Tent.
Table 13: Comparison of maximum GPU memory consumption during test time (in MB), using DeepLab v3+ with WideResNet38 on an NVIDIA Titan Xp, input image size (1024 x 2048).
Methods	Max Memory (MB)
Direct Inference	1170.2
ATTA (Ours)	3388.6
Tent	12796.7
Analysis of Trainable Parameters In Table 14, we show the results on the Road Anomaly dataset with different trainable parameters. The configurations include: 'All' (all parameters are trainable), 'Body' (only the feature extraction part of the network is trainable while the final classification layers are fixed), 'Head' (only the final classification layers are trainable), and 'BN affine' (only the parameters of the affine transformation of BN layers are learnable, similar to [46]). We observe that all configurations are plausible in our setting, with the performance difference being less than  $1\%$  for most metrics. Notably, by training only the parameters in the head, we enable faster inference for each iteration since we do not need to recalculate the features extracted by the body part.
Table 14: Analysis of the effect of different trainable parameters on the Road Anomaly dataset.
Trainable parameters	AUC ↑	AP ↑	FPR95 ↓
All	92.20	59.60	31.99
Body	92.19	59.58	32.00
BN affine	91.73	58.68	34.30
Head (Ours)	92.11	59.05	33.59
Analysis of Episodic Optimization As stated in the main text, we adapt the model parameters on each image independently in an episodic manner. To evaluate alternative test-time optimization strategies, we conduct experiments on the Road Anomaly dataset with two variants. In the first variant, we continue updating the model parameters with the incoming test data (denoted as 'Continue'). In the second variant, we processed two images together as a batch (batch size = 2) instead of one image at a time. The results are shown in Table 15. We observed that both variants led to a performance decrease compared to our episodic updating method. This can be attributed to the lower correlation between images in the Road Anomaly dataset. The information learned from previous data potentially negatively affected the model performance on subsequent data, and processing different images together as a batch introduced inconsistencies and inaccuracies in domain-shift identification, impacting the final performance. Consequently, our episodic updating method tends to be more robust, enabling better adaptation and OOD detection in scenarios with less correlated or diverse data.
Analysis of the outlier class weight We also analyze the effect of the outlier class weight in our loss (represented by  $\lambda$  in Eq.(10)) by removing this term. The results are shown in Table 16. We note that this weight is designed to address the class imbalance problem between the inlier and outlier classes. In our analysis, we observe that for the Road Anomaly dataset, where the class imbalance is not severe, the performance without the weight is comparable to the performance with the weight.
Table 15: Analysis of optimization manner on the Road Anomaly dataset. For the "Continue" updating, the data order may affect the output. Hence, we examine with two different data orders.
Training manner	Batch Size	AUC ↑	AP ↑	FPR95 ↓
Continue - order 1	1	87.19	57.3	67.88
Continue - order 2	1	89.09	58.14	50.77
Episodic	2	89.57	50.66	39.99
Episodic	1	92.11	59.05	33.59
However, for the Fishyscapes Lost & Found and Fishyscapes Static datasets, where the proportion of outlier objects in the images is significantly lower, there is a notable performance gap. Therefore, our design of outlier class weight enables a more robust performance across different datasets with varying proportions of outlier objects.
Table 16: Analysis of the class weight on three datasets with varying proportions of outlier objects.
Methods	Road Anomaly	FS LostAndFound	FS Static
AUC↑	AP↑	FPR95↓	AUC↑	AP↑	FPR95↓	AUC↑	AP↑	FPR95↓
No Weight	92.05	58.39	33.47	98.78	57.60	6.28	99.57	91.62	1.76
Weight (Ours)	92.11	59.05	33.59	99.05	65.58	4.48	99.66	93.61	1.15
F Additional Qualitative Results
Visualization on the FS Static Dataset Figure 4 shows additional qualitative results on the FS Static Dataset and its corrupted version. Our method effectively mitigates domain shift effects, as evidenced by clearer separation between inliers (blue) and outliers (orange) in the histograms.
Figure 4: Visualization of the original images in the FS Static dataset and their corrupted versions. We present the OOD score outputs by PEBAL [44] (denoted as 'Original') and our method, along with their corresponding histograms. The ground truth OOD objects are highlighted with black lines.
Visualization on the Road Anomaly Dataset As shown in Figure 5, our method effectively reduces the high uncertainty region associated with the inlier classes, and the separation between the inliers (depicted in blue) and outliers (depicted in orange) is more distinct in the histogram.
Figure 5: Visualization results on Road Anomaly. Ground truth OOD objects are marked in black.
Visualization Results on SMIYC validation sets. Figure 6 presents qualitative results on the SMIYC validation set. Our method clearly reduces the high uncertainty region of inlier classes.
Figure 6: Visualization results on the SMIYC validation set, including (a) the Road Anomaly 21 dataset, and (b) the Road Obstacle 21 dataset. Ground truth OOD objects are marked in green.
G Discussion
G.1 Specific Domain Shift in the Road Anomaly Dataset
The specific 'domain shift' in the Road Anomaly dataset, as compared to the 'Cityscapes' dataset, encompasses adverse road conditions, diverse weather and lighting conditions, and various camera perspectives and conditions. Here, we detail each of these aspects:
Adverse Road Conditions The Cityscapes dataset mainly focuses on urban scenes with well-paved roads and uniform coloration, whereas the Road Anomaly dataset extends to include more diverse locations like villages and mountains. These rural pathways exhibit varied textures and colors due to different materials, wear, soil types, and vegetation. Examples in Fig. 3 illustrate this contrast, with the visualization of Fishyscapes Lost & Found / Static serving as a reference for typical Cityscapes road conditions. Besides, we see that previous OOD detection methods often mistake these variations for anomalies, while our algorithm reduces such errors, showing better adaptation to the domain shift in road conditions. We refer to Appendix Fig. 5 for more visualization cases on the Road Anomaly dataset, where other examples with various road conditions can be seen.
Weather and Lighting Variations The Road Anomaly dataset encompasses diverse weather conditions not presented in Cityscapes, including snowy, rainy, foggy weather, and nighttime scenes. These variations affect not only the road but also the surrounding areas and the sky, leading to more false positive errors in existing OOD detection models. Examples of these weather-related differences and their effects on OOD detection can be found in Appendix Fig. 7. Our method strives to account for these variations, enhancing the model's adaptability to changes in weather and lighting.
Various Camera Conditions Besides differences in content, the Cityscapes and Road Anomaly datasets also diverge in the conditions under which the images were captured. In the Road Anomaly dataset, images may be captured from various angles and locations, such as alongside the road, which differs from the typical road-centered perspective in Cityscapes. This variance can disrupt previously learned biases in road surface predictions. Additionally, some Road Anomaly images demonstrate a focus effect where the background is intentionally blurred, an effect not commonly seen in Cityscapes. This can lead to false positive errors by initial OOD detection models, as shown in Figure 3, illustrating the sensitivity of models to different camera conditions.
In summary, the Road Anomaly dataset's construction, with images gathered from various internet sources, reflects a real-world scenario that involves complex domain-level distribution shifts. These shifts present an intriguing problem for existing OOD detection methods. Our research contributes to understanding and addressing these domain shifts within the dataset.
Figure 7: First Row: Examples of images from the Road Anomaly Dataset, showcasing various weather conditions including snowy, rainy, and smoggy scenes, along with differing lighting conditions. Second Row: Corresponding OOD score maps generated by the prior OOD detection method (PEBAL). Third Row: OOD scores after applying our ATTA algorithm for model adaptation. Green arrows indicate regions with prominent domain-shift, highlighting that the previous method in the second row assigned high scores to these regions.
G.2 Architectures without BN
While we primarily target models with BN, it's worth noting that most modern networks utilize certain types of normalization techniques including Instance Normalization (IN) and Layer Normalization (LN). For the case of IN and LN, we note that they inherently offer more stability across domain shifts, thus saving special adjustments in our first cascaded stage. This is because, unlike BN, they normalize individual samples, making them less sensitive to variations between domains. We have also empirically tested our anomaly-aware self-training on the mentioned Segmenter backbone [41] which employs layer normalization. Results shown in Table 17 demonstrate the enhanced performance when incorporating our method.
Table 17: Results on the RoadAnomaly Dataset with Segmenter. We use the pre-trained model weights on the Cityscapes dataset provided by [41].
AUC↑	AP↑	FPR95↓
Energy	95.43	75.60	19.76
+ATTA (Ours)	96.49	84.77	18.56
Max logit	94.81	71.65	20.96
+ATTA (Ours)	96.08	81.88	19.19
Still, we note that, since our goal is to adapt at the test phase without retraining, and that BN is prevalent in most network architectures, it's practical to recognize BN's ubiquity and design accordingly, especially in our task, where many methods [44, 4, 1] are based on the Deep Lab V3+ structure, which utilizes BN.
In summary, our framework is broadly applicable across the vast majority of modern neural network architectures that employ some form of normalization. The test-time adaptation of models without normalization layers remains an open UDA problem, which is out of the scope of this work.