SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Zhitong Gao $^{1}$ , Shipeng Yan $^{1}$ , Xuming He $^{1,2}$
$^{1}$ School of Information Science and Technology, ShanghaiTech University
$^{2}$ Shanghai Engineering Research Center of Intelligent Vision and Imaging {gaozht, yanshp, hexm}  $@$  shanghaitech.edu.cn
Recent advancements in dense out-of-distribution (OOD) detection have primarily focused on scenarios where the training and testing datasets share a similar domain, with the assumption that no domain shift exists between them. However, in real-world situations, domain shift often exits and significantly affects the accuracy of existing out-of-distribution (OOD) detection models. In this work, we propose a dual-level OOD detection framework to handle domain shift and semantic shift jointly. The first level distinguishes whether domain shift exists in the image by leveraging global low-level features, while the second level identifies pixels with semantic shift by utilizing dense high-level feature maps. In this way, we can selectively adapt the model to unseen domains as well as enhance model's capacity in detecting novel classes. We validate the efficacy of our method on several OOD segmentation benchmarks, including those with significant domain shifts and those without, observing consistent performance improvements across various baseline models. Code is available at https://github.com/gaozhitong/ATTA.
Semantic segmentation, a fundamental computer vision task, has witnessed remarkable progress thanks to the expressive representations learned by deep neural networks [33]. Despite the advances, most deep models are trained under a close-world assumption, and hence do not possess knowledge of what they do not know, leading to over-confident and inaccurate predictions for the unknown objects [18]. To address this, the task of dense out-of-distribution (OOD) detection [1, 15], which aims to generate pixel-wise identification of the unknown objects, has attracted much attention as it plays a vital role in a variety of safety-critical applications such as autonomous driving.
Recent efforts in dense OOD detection have primarily focused on the scenarios where training and testing data share a similar domain, assuming no domain shift (or covariant shift) between them [50, 3, 1, 15, 38]. However, domain shift widely exists in real-world situations [39] and can also be observed in common dense OOD detection benchmarks [29]. In view of this, we investigate the performance of existing dense OOD detection methods under the test setting with domain-shift and observe significant performance degradation in comparison with the setting without domain-shift (cf. Figure 1). In particular, the state-of-the-art detection models typically fail to distinguish the distribution shift in domain and the distribution shift in semantics, and thus tend to predict high uncertainty scores for inlier-class pixels.
A promising strategy to tackle such domain shift is to adapt a model during test (known as test-time adaptation (TTA) [46]), which utilizes unlabeled test data to finetune the model without requiring prior information of test domain. However, applying the existing test-time domain adaption (TTA) techniques [46, 7, 31, 47] to the task of general dense OOD detection faces two critical challenges. First, traditional TTA methods often assume the scenarios where all test data are under domain shift
Figure 1: (a) We visualize OOD score maps and the corresponding histograms generated by the SOTA method PEBAL [44] on both an original and a domain-shifted (smog-corrupted) image. (b) We quantify the drop in PEBAL's performance with the added domain shift and compare it to the performance when combined with our method or existing test-time adaptation methods such as TBN [36] and Tent [46]. Please refer to Section 4.3 for additional results.
while our dense OOD detection task addresses a more realistic setting where test data can come from seen or unseen domains without prior knowledge. In such a scenario, TTA techniques like the transductive batch normalization (TBN) [36, 40, 2], which substitutes training batch statistics with those of the test batch, could inadvertently impair OOD detection performance on images from seen domains due to inaccurate normalization parameter estimation (cf. Figure 1(b)). On the other hand, the existence of novel classes in test images further complicates the problem. Unsupervised TTA losses like entropy minimization [46, 12, 32, 45] often indiscriminately reduce the uncertainty or OOD scores of these novel classes, leading to poor OOD detection accuracy (cf. Figure 1(b)). Consequently, how to design an effective test-time adaptation strategy for the general dense OOD detection in wild remains an open problem.
In this work, we aim to address the aforementioned limitations and tackle the problem of dense OOD detection in wild with both domain- and semantic-level distribution shift. To this end, we propose a novel dual-level test-time adaptation framework that simultaneously detects two types of distribution shift and performs online model adaptation in a selective manner. Our core idea is to leverage low-level feature statistics of input image to detect whether domain-level shift exists while utilizing dense semantic representations to identify pixels with semantic-level shift. Based on this dual-level distribution-shift estimation, we design an anomaly-aware self-training procedure to compensate for the potential image-level domain shift and to enhance its novel-class detection capacity based on re-balanced uncertainty minimization of model predictions. Such a selective test-time adaptation strategy allows us to adapt an open-set semantic segmentation model to a new environment with complex distribution shifts.
Specifically, we develop a cascaded modular TTA framework for any pretrained segmentation model with OOD detection head. Our framework consists of two main stages, namely a selective Batch Normalization (BN) stage and an anomaly-aware self-training stage. Given a test image (or batch), we first estimate the probability of domain-shift based on the statistics of the model's BN activations and update the normalization parameters accordingly to incorporate new domain information. Subsequently, our second stage performs an online self-training for the entire segmentation model based on an anomaly-aware entropy loss, which jointly minimizes a re-balanced uncertainty of inlier-class prediction and outlier detection. As the outlier-class labels are unknown, we design a mixture model in the OOD score space to generate the pseudo-labels of pixels for the entropy loss estimation.
We validate the efficacy of our proposed method on several OOD segmentation benchmarks, including those with significant domain shifts and those without, based on FS Static [1], FS Lost&Found [1], RoadAnomaly [29] and SMIYC [3]. The results show that our method consistently improves the performance of dense OOD detection across various baseline models especially on the severe domain shift settings, and achieves new state-of-the-arts performance on the benchmarks.
To summarize, our main contribution is three-folds: (i) We propose the problem of dense OOD detection under domain shift (or covariance shift), revealing the limitations of existing dense OOD detection methods in wild. (ii) We introduce an anomaly-aware test-time adaptation method that jointly tackles domain and semantic shifts. (iii) Our extensive experiments validate our approach, demonstrating significant performance gains on various OOD segmentation benchmarks, especially those with notable domain shifts.
Dense Out-of-distribution Detection The task of Out-of-distribution (OOD) detection aims to identify samples that are not from the same distribution as the training data [18]. While most work focuses on image-level out-of-distribution detection [18, 27, 25, 19, 30, 42], some researchers have begun to study dense OOD detection [1, 15, 3] (or anomaly segmentation), which is a more challenging task due to its requirement for granular, pixel-level detection and the complex spatial relationships in images. Existing work dealing with dense OOD detection often resorts to either designing specialized OOD detection functions [15, 21, 29, 48] or incorporating additional training objectives or data to boost performance in OOD detection [10, 4, 44]. Different from these approaches, our work aims to enhance the model's ability to detect OOD objects at test time by utilizing online test data and design a model-agnostic method applicable to most differentiable OOD functions and training strategies. Furthermore, we target for a more general dense OOD detection problem, where domain-shift potentially exists, which brings new challenges unaddressed in the prior literature.
Test-Time Domain Adaptation The task of test-time domain adaptation (TTA) [46] aims to study the ability of a machine learning model trained on a source domain to generalize to a different, but related, target domain, using online unlabeled test data only. Existing work on TTA mostly tackles the problem from two aspects: adapting standardization statistics in normalization layers and adapting model parameters as self-training. The first line includes utilizing test-time statistics in each batch [36] or moving averages [35, 20], or combining source and target batch statistic [40, 51, 22, 28, 52]. The self-training technique including entropy minimization [12, 32, 45] and self-supervised losses [43, 31]. In our work, we also approach test-time adaptation from these two perspectives. However, our primary distinction lies in addressing a more general open-world scenario, where test data can originate from seen or unseen domains, and novel objects may exist. Consequently, we design our method by explicitly considering these factors and aiming to jointly tackle domain shift and semantic shift challenges.
Novel Class in Unseen Domain While the two primary forms of distribution shift, domain shift and semantic shift, are typically studied independently in literature, there are instances where both are explored in a more complex setting. One such example is Open Set Domain Adaptation [37], which investigates the unsupervised domain adaptation problem in scenarios where the target data may include new classes. These novel classes must be rejected during the training process. Another line of research focuses on zero-shot learning in the presence of domain shift [49]. In this case, novelty rejection must be performed during the testing phase. A domain generalization method is often employed, which requires data from other domains during training. Some work also discuss the impact of covariant-shift in OOD detection [34]. However, our study diverges from these in several key aspects. First, we study the problem of semantic segmentation where the impacts of domain and semantic shifts are more complex and nuanced compared to the general classification problems. Second, we focus on the situation when no additional data or prior knowledge of the test domain can be obtained during the training.
We first introduce our general dense OOD detection problem setting with potential domain and semantic distribution shift. Formally, we denote an instance of training data as  $(x^{s},y^{s})\sim P_{XY}\in$ $\mathcal{X}\times \mathcal{Y}^d$ , where  $x^{s},y^{s}$  denotes the input image and corresponding segmentation label,  $\mathcal{X} = R^{3\times d}$  represents the input space (an image with  $d$  pixels),  $\mathcal{Y} = [1,C]$  is the semantic label space at each pixel, and  $P_{XY}$  is the training data distribution. A test data instance is represented as  $(x,y)\sim Q_{XY}$  where  $Q_{XY}$  is the test data distribution. In the OOD detection problem, we have  $Q_{XY}\neq P_{XY}$  and our goal is to identify all pixels where their labels do not belong to the training label space, i.e.,  $y_{i}\notin \mathcal{V}$ . In contrast to previous works, we here consider a general problem setting, dense OOD detection with potential domain shift, where in addition the input distributions  $P_{X}\neq Q_{X}$  but with overlaps in their support. Such domain and semantic-class changes leads to a complex data distribution shift, which poses new challenges for the conventional dense OOD detection approaches that are unable to distinguish different shift types.
Figure 2: The overview of the two-stage Anomaly-aware Test-Time Adaptation (ATTA) framework. For each test image (or batch), our first stage determines the existence of domain shift and develop a selective Batch-Normalization module to compensate for the input distribution deviation. In the second stage, we devise an anomaly-aware self-training procedure via minimizing a re-balanced uncertainty of model predictions to enhance the OOD detection capacity.
3.2 Model Overview
In this work, we aim to address the general dense OOD detection problem by leveraging unlabeled test data and adapting a segmentation model during test time. Specifically, we begin with a segmentation network that has been trained using data from  $P_{XY}$ . Here we consider a common differentiable OOD network design [5, 30, 15, 1] that consists of two main components: a seen-class classifier  $f_{\theta}: \mathcal{X} \to \mathcal{Y}^d$  and an unseen class detector  $g_{\theta}: \mathcal{X} \to R^d$ . Typically, both network components share the same set of parameters  $\theta$ . For instance,  $g_{\theta}$  can be the negative energy score as in [13, 30, 23] or the maximized logit score as in [15]. During testing, an image (or a batch) sequentially arrives from  $Q_{XY}$ , and our goal is to update the model parameters for each batch and produce anomaly-aware semantic segmentation outputs.
In order to cope with potential domain and semantic shift in test data, we introduce a novel dual-level test-time adaptation framework, which allows us to simultaneously identify two types of distribution shifts and performs an online self-supervised learning in a selective fashion. Specifically, we instantiate our TTA strategy as a two-stage cascaded learning process. Our first stage determines the existence of domain shift in the image by exploiting the consistent changes in image-level feature statistics. Based on the inferred shift probability, we develop a selective Batch-Normalization module to compensate for the input distribution deviation. In the second stage, we devise an anomaly-aware self-training procedure to enhance the model's capability in detecting novel classes. The procedure iteratively refines the outlier estimation based on a mixture model of OOD scores and minimizes a re-balanced uncertainty of pixel-wise model predictions for effective adaptation to the test data. An overview of our framework is demonstrated in Figure 2.
We update the model parameters in an episodic manner, where for each (batch of) image(s) we use a same initial network parameter. This makes it more resilience if there is no apparent connection between two different (batches of) image(s). In the following, we first explain our design of the selective BN module in Sec. 3.3, followed by our anomaly-aware self-training stage in Sec. 3.4.
3.3 Selective Test-Time Batch Normalization
Our first-stage module aims to identify whether an input image is from the seen or unseen domain and perform model adaptation to compensate any potential domain shift. To achieve this, we take inspiration from the transductive Batch-Normalization (TBN) [36], which is a common strategy of exploiting test-time data for model adaptation. The vanilla TBN, however, often suffers from unstable estimation due to the small size of test data, and hence can lead to performance degradation for our general setting where test data may have varying levels of domain shift.
To tackle this, we introduce a selective BN module that first estimates the probability of input image being generated from unseen domain, and then performs a mixture of Conventional batch normalization (CBN) and TBN according to the inferred domain-shift probability. Specifically, we denote  $P(z^{d} = 1|x)$  as the probability of an image  $x$  from an unknown domain, and we estimate the probability by considering the distribution distance between the deep features of test and training data in the Normalization layers of the segmentation network. Formally, let  $\mu_{l}^{tr},\sigma_{l}^{tr}$  be the running mean
and standard deviation at the  $l$ -th BN layer calculated in the end of the model training,  $\mu_l(x), \sigma_l(x)$  be the mean and standard deviation at the  $l$ -th BN layer calculated for each test input  $x$ , we compute the domain-shift probability as follows:
$$
P \left(z ^ {d} = 1 | x\right) = h _ {a, b} \left(\sum_ {l = 1} ^ {L} \left(K L \left(\mathcal {N} \left(\mu_ {l} (x), \sigma_ {l} (x)\right) \mid \mid \mathcal {N} \left(\mu_ {l} ^ {t r}, \sigma_ {l} ^ {t r}\right)\right)\right)\right), \tag {1}
$$
where  $\mathcal{N}$  denotes the normal distribution,  $KL$  denotes the Kullback-Leibler divergence, and the  $h_{a,b}(x) = \mathrm{sigmoid}((x + a) / b)$  is a sigmoid function with linear transform, which normalizes the distance into a probability value. The parameters  $a, b$  are estimated based on the variance of training data statistics such that a data point that is far away from the training statistics has higher probability of image-level domain shift. We then update the BN statistics of the network according to the above probability as follows:
$$
\hat {\mu} _ {l} = P \left(z ^ {d} = 1 | x\right) * \mu_ {l} (x) + P \left(z ^ {d} = 0 | x\right) * \mu_ {l} ^ {t r}, \tag {2}
$$
$$
\hat {\sigma} _ {l} ^ {2} = P \left(z ^ {d} = 1 | x\right) * \sigma_ {l} ^ {2} (x) + P \left(z ^ {d} = 0 | x\right) * \left(\sigma_ {l} ^ {t r}\right) ^ {2}, \tag {3}
$$
where  $l \in [1, L]$  and  $L$  is the max depth of the BN layers. Such an adaptive BN enables us to balance between a stable BN for the data without domain-shift and a test-time BN tailored to the data with domain shift.
3.4 Anomaly-aware Self-training Procedure
After compensating for the potential domain gap, we introduce a second stage of test-time model adaptation, aiming for enhancing the model capacity in OOD detection and closed-set prediction on the test data. To this end, we propose an online self-training procedure for the entire segmentation model based on an anomaly-aware prediction entropy loss. By minimizing a weighted uncertainty of inlier-class prediction and outlier detection, we are able to promote the confidence of model predictions, which leads to a more discriminative pixel-wise classifier on the test data.
Formally, we construct a  $(C + 1)$ -class pixelwise probabilistic classifier based on the two network modules  $f_{\theta}$  and  $g_{\theta}$  and denote its output distribution as  $\hat{Y} \in [0,1]^{(C + 1) \times d}$  with each element  $\hat{Y}_{c,i}$  being the probability of  $P_{\theta}(y_i = c|x)$ ,  $c \in [1,C + 1]$ . For each pixel  $i$ , the first  $C$  channels represent the probabilities of being each closed-set class while the last channel is the probability of being an outlier class. Given the output distribution, we define our learning objective function as,
$$
\mathcal {L} _ {\theta} (x) = - \sum_ {i} \sum_ {c = 1} ^ {C + 1} w _ {c} \hat {Y} _ {c, i} \log (\hat {Y} _ {c, i}), \tag {4}
$$
where  $w_{c}$  is the weight for class  $c$ . We introduce the class weights to cope with the class imbalance problem between the inlier and outlier classes as the latter typically have much lower proportion in images. More specifically, we set  $w_{c} = 1$  for  $1 \leq c \leq C$  and  $w_{c} = \lambda > 1$  for  $c = C + 1$ . In the following, we first describe how we estimate the output probabilities  $\hat{Y}$  based on the seen-class classifier  $f_{\theta}$  and the OOD detector  $g_{\theta}$ , followed by the loss optimization procedure.
Anomaly-aware output representation To derive the output probability  $\hat{Y}$ , we introduce an auxiliary variable  $Z^o\in \{0,1\} ^d$ , where  $Z_{i}^{o} = 1$  indicates the  $i$ -th pixel is an outlier, and  $P_{\theta}(Z_i^o = 1|x)$  denotes the corresponding probability. We also assume that the seen classifier  $f_{\theta}$  outputs a pixelwise probability map  $F\in [0,1]^{C\times d}$  for the known classes  $c\in \mathcal{V}$  and the OOD detector  $g_{\theta}$  generates a pixelwise outlier score map  $G\in \mathbb{R}^d$ . The anomaly-aware output distribution  $\hat{Y}_i$  for the  $i$ -th pixel can be written as,
$$
\begin{array}{l} \hat {Y} _ {c, i} = P _ {\theta} \left(\hat {y} _ {i} = c | x, Z _ {i} ^ {o} = 0\right) P _ {\theta} \left(Z _ {i} ^ {o} = 0 | x\right) + P _ {\theta} \left(\hat {y} _ {i} = c | x, Z _ {i} ^ {o} = 1\right) P _ {\theta} \left(Z _ {i} ^ {o} = 1 | x\right) (5) \\ = F _ {c, i} \left(1 - P _ {\theta} \left(Z _ {i} ^ {o} = 1 | x\right)\right) \llbracket c \in \mathcal {Y} \rrbracket + P _ {\theta} \left(Z _ {i} ^ {o} = 1 | x\right) \llbracket c = C + 1 \rrbracket , (6) \\ \end{array}
$$
where  $\llbracket \cdot \rrbracket$  is the indicator function, and we use the fact that  $P_{\theta}(\hat{y}_i = c|x,Z_i^o = 0) = F_{c,i}$  for  $c\in \mathcal{V}$  and 0 for  $c = C + 1$ , and  $P_{\theta}(\hat{y}_i = c|x,Z_i^o = 1) = 0$  for  $c\in \mathcal{V}$  and 1 for  $c = C + 1$ .
Given the marginal probability in Eqn (6), our problem is reduced to estimating the pixelwise outlier probability  $P_{\theta}(Z_i^o = 1|x)$ . However, as we only have an arbitrary outlier score map  $G$  output by  $g_{\theta}$
it is non-trivial to convert this unnormalized score map into a valid probability distribution. A naive nonlinear transform using the sigmoid function or normalizing the scores with sample mean could lead to incorrect estimation and hurt the performance. To tackle this problem, we develop a data-driven strategy that exploits the empirical distribution of the pixelwise OOD scores to automatically calibrate the OOD scores into the outlier probability  $P_{\theta}(Z_i^o |x)$ .
More specifically, we observe that the empirical distributions of pixelwise OOD scores  $\{G_i\}$  appear to be bimodal and its two peaks usually indicate the modes for inlier and outlier pixels (cf. Figure 1). We therefore fit a two-component Mixture of Gaussian distribution in which the components with lower mean indicates the inliers and the one with higher mean corresponds to the outliers. Given the parameters of the Gaussian components, we now fit a sample-dependent Platt scaling function to estimate the outlier probability as follows,
$$
P _ {\theta} \left(Z _ {i} ^ {o} = 1 | x\right) = \operatorname {s i g m o i d} \left(\left(G _ {i} - a (x)\right) / b (x)\right), \tag {7}
$$
where we set the calibration parameter  $a(x)$  as the value achieving equal probability under two Gaussian distributions, i.e.,  $a(x) = \max \{a : \pi_1 N(a|\mu_1, \sigma_1) = \pi_2 N(a|\mu_2, \sigma_2)\}$  where  $\pi_1, \pi_2, \mu_1, \mu_2, \sigma_1, \sigma_2$  are the parameters of the GMM model, and  $b(x)$  as the standard derivation of the sample OOD scores. We note that while it is possible to analytically compute the outlier probability based on the estimated GMM, we find the above approximation works well in practice and tends to be less sensitive to the noisy estimation.
Entropy minimization After plugging in the estimated  $\hat{Y}$ , we can re-write our learning objective in Eqn (4) as the following form:
$$
\begin{array}{l} \mathcal {L} _ {\theta} (x) = - \sum_ {i} \left(\sum_ {c = 1} ^ {C} \hat {Y} _ {c, i} \log \left(\hat {Y} _ {c, i}\right) + \lambda \cdot \hat {Y} _ {C + 1, i} \log \left(\hat {Y} _ {C + 1, i}\right)\right) (8) \\ = - \sum_ {i} \left(\sum_ {c = 1} ^ {C} F _ {c, i} \left(1 - \bar {G} _ {i}\right) \log \left(F _ {c, i} \left(1 - \bar {G} _ {i}\right)\right) + \lambda \cdot \bar {G} _ {i} \log \bar {G} _ {i}\right), (9) \\ \end{array}
$$