SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Chaoqi Chen $^{1*}$  Luyao Tang $^{2*}$  Yue Huang $^{2}$  Xiaoguang Han $^{3\dagger}$  Yizhou Yu $^{1\dagger}$
<sup>1</sup>The University of Hong Kong <sup>2</sup>Xiamen University
<sup>3</sup>The Chinese University of Hong Kong (Shenzhen)
cqchen1994@gmail.com, lytang@stu.xmu.edu.cn, yhuang2010@xmu.edu.cn
hanxiaoguang@cuhk.edu.cn, yizhouy@acm.org
The generalization capability of machine learning systems degenerates notably when the test distribution drifts from the training distribution. Recently, Domain Generalization (DG) has been gaining momentum in enabling machine learning models to generalize to unseen domains. However, most DG methods assume that training and test data share an identical label space, ignoring the potential unseen categories in many real-world applications. In this paper, we delve into a more general but difficult problem termed Open Test-Time DG (OTDG), where both domain shift and open class may occur on the unseen test data. We propose Compaction and Disambiguation (CODA), a novel two-stage framework for learning compact representations and adapting to open classes in the wild. To meaningfully regularize the model's decision boundary, CODA introduces virtual unknown classes and optimizes a new training objective to insert unknowns into the latent space by compacting the embedding space of source known classes. To adapt target samples to the source model, we then disambiguate the decision boundaries between known and unknown classes with a test-time training objective, mitigating the adaptivity gap and catastrophic forgetting challenges. Experiments reveal that CODA can significantly outperform the previous best method on standard DG datasets and harmonize the classification accuracy between known and unknown classes.
The ability to generalize to unseen environments is considered a key signature of human intelligence [12]. While deep neural networks have achieved great success in many machine learning problems, they are brittle to distribution shifts between training and test domains, which often occur in real-world applications. For example, when deploying object recognition systems in autonomous vehicles, the ever-changing weather conditions (e.g. fog, rain, and snow) may deteriorate the performance and raise concerns about their reliability over time. This motivates a challenging scenario named Domain Generalization (DG) [68, 84], which extrapolates learning machines to related yet previously unseen test domains by identifying the common factors from available source data.
From the perspective of representation learning, the mainstream paradigm for DG includes invariant risk minimization [2, 1, 89], domain alignment [31], feature disentanglement [45, 36, 78], meta-learning [29, 30], and augmentation-based invariant prediction [66, 74, 88]. In spite of the significant progress in DG, the adaptivity gap [14] between source and target domains naturally exists and emerges as an inevitable challenge. Therefore, some prior efforts [23, 24, 72, 11] strive to adapt
*First two authors contributed equally.
the source-trained model through the lens of test-time adaptation [34], which uses unlabeled target samples in an online manner. While these methods alleviate the adaptivity gap to some extent, there is still no guarantee for any specific domain, especially if the domain divergence is large [3]. Moreover, most existing DG methods are designed with the assumption that the label space in the source and target domain are identical, which is too restrictive to be satisfied in practice.
To enable machine learning systems to be resilient to an open world, we aim at a more practical yet under-explored problem named Open Test-time Domain Generalization (OTDG), where both domain shift and open class occur in the unseen target data. The primary challenge of the proposed problem lies in addressing two critical aspects: (1) model generalization with an incomplete training label space, and (2) online model adaptation with asymmetric test label space.
Motivated by this, we propose a novel framework for OTDG, termed Compaction and Disambiguation (CODA). Our key idea is to enforce constraints on the decision boundaries during training using labeled source data and refine them during test time using unlabeled target data. To regularize the model's decision boundary and make the model expandable, CODA introduces a set of virtual unknown classes and optimizes a novel training objective in conjunction with the standard cross-entropy loss. The response to known- and unknown-class logits will be activated for both real and synthesized samples. This process embeds unknowns into the latent space by compacting the embedding space of known source classes, thereby reserving sufficient latent space for target unknown classes. To disambiguate the decision boundaries between known and unknown classes, we propose a novel prototype-based test-time adaptation pipeline. Specifically, the test-time classification should be subjected to three constraints: (1) consistency between the predicted class distributions and the estimated class conditionals, (2) class-wise sample reliability for ensuring the quality of target pseudo labels, and (3) semantic consistency between source-trained and present model predictions.
The main contributions are summarized as follows:
- We propose CODA, a simple and effective DG framework to mitigate domain shift and identify open classes in unseen test environments.
- We introduce a virtual unknown optimization process to make the model expandable for open classes, and a test-time training objective to match the real test data to corresponding known and unknown class patterns.
- We conduct extensive experiments and demonstrate that CODA outperforms previous methods on a series of DG benchmarks.
Problem setup. Let us formally define the OTDG problem. We have access to a source domain  $\mathcal{D}_s = \{(\mathbf{x}_s^i, y_s^i)\}_{i=1}^{n_s}$  of  $n_s$  labeled data points and multiple unseen target domains  $\mathcal{D}_t = \{(\mathbf{x}_t^j)\}_{j=1}^{n_t}$  of  $n_t$  unlabeled data points. Let  $\mathcal{C}_s$  and  $\mathcal{C}_t$  be the source and target class sets, respectively. In OTDG, we have  $\mathcal{C}_s \subset \mathcal{C}_t$  and  $\mathcal{C}_t^u = \mathcal{C}_t \setminus \mathcal{C}_s$  is referred to as unknown classes. Assume that  $\mathcal{X}$  is the input space,  $\mathcal{Z}$  is the latent space, and  $\mathcal{Y}$  is the output space. The predictor  $f = h \circ g$  is comprised of a featurizer  $g: \mathcal{X} \mapsto \mathcal{Z}$  that learns to extract embedding features, and a classifier  $h: \mathcal{Z} \mapsto \mathcal{Y}$  that makes predictions based on the extracted features. The goal of OTDG is to find a predictor  $f: \mathcal{X} \mapsto \mathcal{Y}$  that generalizes well to all unseen target domains. Although the labeling function  $h_t$  is unknown, we assume that we have access to unlabeled instances from  $\mathcal{D}_t$  at test time. In addition, we present the comparison of different problem settings in Table 1.
Unknown-aware training. To solve OTDG, a simple baseline is to train a  $(|\mathcal{C}_s| + 1)$ -way classifier [90, 77, 9], where the additional dimension is introduced to identify the unknown. Formally, we define the standard cross-entropy loss as:
$$
\mathcal {L} _ {\mathrm {C E}} (f (\mathbf {x}), y) = - \log \frac {\exp \left(f _ {k} (\mathbf {x})\right)}{\sum_ {c \in | \mathcal {C} _ {s} | + 1} \exp \left(f _ {c} (\mathbf {x})\right)}, \tag {1}
$$
where  $f(\mathbf{x}) \in \mathbb{R}^{|\mathcal{C}_s| + 1}$  denotes the network's logit and  $f_{k}(\mathbf{x})$  is the  $k$ -th element of  $f(\mathbf{x})$  corresponding to the ground-truth label  $y$ . As shown in Figure 1 (a), however, such optimization fails to activate the network's response to unknown classes. In this work, we first introduce a simple yet very effective baseline [9] for OTDG. Their key idea is to directly activate the unknown's logit by optimizing its
Table 1: Comparison of related machine learning problems. DA refers to Domain Adaptation and OOD stands for Out-of-Distribution. 'One-pass' indicates that target domain data only passes the network once during the whole process including the training and testing phases.
Problem Setting	Training	Test-time	One-pass
Training Data	Training Loss	Testing Loss	Domain shift	Open class	Adaptivity gap
Open-Set DA	xs, ys, xt	L(xs, ys) + L(xs, xt)	-	✓	✓	✓	×
Source-Free DA	xt	L(xt)	-	✓	×	✓	×
OOD Detection	xs, ys	L(xs, ys)	-	×	✓	×	✓
Test-Time Adaptation	xs, ys	L(xs, ys)	L(xt)	✓	×	✓	✓
Test-Time DG	xs, ys	L(xs, ys)	L(xt)	✓	×	✓	✓
Open-Set DG	xs, ys	L(xs, ys)	-	✓	✓	×	✓
OTDG (Ours)	xs, ys	L(xs, ys)	L(xt)	✓	✓	✓	✓
(c)
Figure 1: The decision boundaries learned by different unknown-aware training objectives: (a) Eq. 1, (b) Eq. 2, and (c) Eq. 3 + Eq. 4. These toy examples are generated by scikit-learn toolkit. Yellow, blue, orange, and pink points represent the known-class samples, while black points are unknown-class samples. Different clusters of back points stand for different unknown classes.
likelihood, without affecting the ground-truth classification. For a source sample  $(\mathbf{x}_s, y_s) \in \mathcal{D}_s$ , we minimize the negative log-likelihood w.r.t. unknown logit to increase the unknown probability,
$$
\mathcal {L} _ {\mathrm {U A T}} (f (\mathbf {x}), y) = \mathcal {L} _ {\mathrm {C E}} (f (\mathbf {x}), y) - \log \frac {\exp \left(f _ {u} (\mathbf {x})\right)}{\sum_ {c \in | \mathcal {C} _ {s} | + 1 , c \neq y} \exp \left(f _ {c} (\mathbf {x})\right)}, \tag {2}
$$
where  $f_{u}(\mathbf{x})$  is the unknown's logit. Eq. 2 makes the unknown class probability respond to any input sample, irrespective of its class label. Since the learning process is driven by the cross-entropy loss related to the ground-truth category, Eq. 2 does not hurt the performance of known classes.
We propose CODA (Figure 2), a simple two-stage OTDG framework for discovering unknown classes with the help of known ones. The specific implementation of the two stages is described as follows.
3.1 Training-Time Source Compaction
The source compaction stage preserves sufficient space for the upcoming unknown classes without using real target data during training. Since we have no a priori knowledge or assumptions about the characteristics (e.g., number and attribute) of unknown classes, it is prohibitively difficult to learn meaningful representations beforehand. To begin with, we make a mild assumption that the unknown classes should be far away from all known classes in the embedding space (category shift). In practice, there are two choices for accommodating these unknown classes: (1) low-density region (Figure 1 (b)), and (2) between-class region (Figure 1 (c)). The former is intuitive and could be achieved via Eq. 2, which naturally corresponds to these regions. However, the multi-modal structure of unknown-class data is underspecified, i.e., different unknown classes should not occupy the same region. By contrast, the latter is able to implicitly separate different unknowns and model the relationships between known and unknown classes. To make the model expandable, our key idea
Figure 2: Overview of the proposed CODA, which consists of two novel components: (1) Training-time source compaction to make the model expandable for open classes; (2) Test-time target disambiguation to discriminate the decision boundaries with a test-time training objective.
is to compact the source embedding space by  $(i)$  making real known samples give response to both known and unknown classes, and  $(ii)$  inserting virtual unknown samples between known-class pairs.
Optimization on real known-class samples. In addition to the original source classes, we introduce a set of virtual classes  $\mathcal{C}_v$  within the source embedding space to mimic the existence of unknown classes. To accommodate the virtual unknown classes, we regularize the embedding space by pushing known-class samples closer to the decision boundaries:
$$
\mathcal {L} _ {\text {r e a l}} (f (\mathbf {x}), y) = \mathcal {L} _ {\mathrm {C E}} (f (\mathbf {x}), y) - \log \frac {\exp \left(f _ {\hat {u}} (\mathbf {x})\right)}{\sum_ {c \in \left| \mathcal {C} _ {s} \right| + \left| \mathcal {C} _ {v} \right| , c \neq y} \exp \left(f _ {c} (\mathbf {x})\right)}, \tag {3}
$$
where  $f_{\hat{u}}(\mathbf{x})$  is the unknown's logit, having the largest activation among  $C_v$ .
Optimization on virtual unknown-class samples. Despite the activation of the network's response to the introduced unknown classes, the "winner-takes-all" nature of softmax-based classification still leaves the unknown category unable to compete effectively with known ones (cf. Figure 2). To mimic test environments, we synthesize virtual unknown samples. Technically, we synthesize unknown samples at the feature level for model regularization, without using external data, i.e., dashed points in Fig. 1 (c). We introduce manifold mixup [64] for synthesizing unknowns in the between-class regions, which are less confident for current decision boundaries. For two random samples  $\mathbf{x}_i$  and  $\mathbf{x}_j$  from different classes, we mix their embedding features as:  $\hat{\mathbf{z}} = \mu \cdot g(\mathbf{x}_i) + (1 - \mu)\cdot g(\mathbf{x}_j)$ , where  $\mu$  is the mixing coefficient. The optimization objective for the synthesized unknown  $\hat{\mathbf{z}}$  is defined as:
$$
\mathcal {L} _ {\text {v i r t u a l}} (h (\hat {\mathbf {z}}), \hat {y}) = \mathcal {L} _ {\mathrm {C E}} (h (\hat {\mathbf {z}}), \hat {y}) - \log \frac {\exp \left(h _ {k ^ {\prime}} (\hat {\mathbf {z}})\right)}{\sum_ {c \in | \mathcal {C} _ {s} | + | \mathcal {C} _ {v} | , c \neq \hat {y}} \exp \left(h _ {c} (\hat {\mathbf {z}})\right)}, \tag {4}
$$
where  $\hat{y}$  represents the label of  $\hat{z}$  regarding unknown class, i.e., having the largest activation among  $\mathcal{C}_v$ .  $h_{k'}(\hat{\mathbf{z}})$  is the known's logit, having the largest activation among  $\mathcal{C}_s$ . The first term in Eq. 4 is a standard self-training loss. Similar to Eq. 3, the second term activates the response of  $\hat{\mathbf{z}}$  to its most related known class. In essence, apart from the standard classification loss (the first term in Eq. 3 and Eq. 4), we activate the response of the real known class towards unknowns (Eq. 3) and the response of the virtual unknown class towards the known ones (Eq. 4).
3.2 Test-Time Target Disambiguation
Although we have allocated the embedding space for unknown classes, how to deploy the source-trained model on real test data is yet to be thoroughly studied. In particular, we identify two major challenges: (i) optimality gap between source and target domains, and (ii) catastrophic forgetting in open and dynamic test environments. In OTDG, we define the optimality gap as follows.
Definition 1 (Optimality Gap) Let  $\mathcal{H} \subseteq \{h | h : \mathcal{Z} \mapsto \mathcal{Y}\}$  be the hypothesis class.  $\varepsilon_S(\cdot)$  and  $\varepsilon_T(\cdot)$  denote the expected risk on source and target domains. For any hypothesis  $h$ , we have  $\varepsilon_T(h^t) < \varepsilon_T(h^*)$ , where  $h^* = \arg \min_{h \in \mathcal{H}} \varepsilon_S(h) + \varepsilon_T(h)$  and  $h^t = \arg \min_{h \in \mathcal{H}} \varepsilon_T(h)$ .
Definition 1 suggests that it is not feasible to find a universal optimal classifier that applies to both source and target domains. In that sense, the classifier, initially trained on source data, needs additional refinement to adapt effectively to the target patterns. Technically, we resort to TTA [34] to mitigate the above issues using unlabeled test data in an online manner. Conventional training-based TTA methods [67] usually need batches of data for self-training (e.g. entropy minimization) and/or heuristic self-supervision tasks [10]. On the other hand, training-free methods [23] require expensive tweaking of the threshold and only bring a marginal performance gain. Moreover, these approaches struggle to handle open-class scenarios, making them susceptible to negative transfer (i.e. semantic misalignment). To address these challenges, our proposed target disambiguation stage aligns the unlabeled target samples to their corresponding class patterns through the following process.
We construct a memory bank  $\mathbb{S} = \{\mathbb{S}^1,\dots ,\mathbb{S}^{|\mathcal{C}_s| + |\mathcal{C}_v|}\}$  for memorizing the embedding  $\mathbf{z}$  and logits  $f(\mathbf{x})$  (or  $h(\mathbf{z}))$  of target samples. We compute a set of class prototypes  $\{\mathbf{p}_k\}_{k = 1}^{|C_s| + |C_v|}$  based on logits in  $\mathbb{S}$ . Following [23, 24], the memory bank is initialized with the weights of the linear classifier. Let  $\mathcal{N}(\mathbf{x})$  be the Nearest Neighbor (NN) of  $\mathbf{x}$  in  $\mathbb{S}$ . For each test sample  $\mathbf{x}^3$ , we search its NN in  $\mathbb{S}$  by:  $\mathcal{N}(\mathbf{x}) := \{\mathbf{z} \in \mathbb{S} | \mathrm{sim}(g(\mathbf{x}), \mathbf{z}) \leq \theta_{\mathrm{NN}}\}$ , where  $\mathrm{sim}(\cdot)$  is the cosine similarity and  $\theta_{\mathrm{NN}}$  is a threshold to control the number of NN. The model predictions would be given by the similarity between sample embedding and the class prototype, i.e.,  $p(y|\mathbf{x}) \propto \mathrm{sim}\langle \mathbf{p}_k, \mathbf{z} \rangle$ . Formally, for  $\mathbf{z} \in \mathcal{N}(\mathbf{x})$ , the likelihood of the prototype-based classifier assigning  $\mathbf{z}$  to the  $k$ -th class can be calculated as follows:
$$
p (y = k | \mathbf {z}) = \frac {\exp (- \operatorname {s i m} (h (\mathbf {z}) , \mathbf {p} _ {k}) / \tau)}{\sum_ {c} \exp (- \operatorname {s i m} (h (\mathbf {z}), \mathbf {p} _ {c}) / \tau)}, k = 1, 2, \dots , \left| \mathcal {C} _ {s} \right| + \left| \mathcal {C} _ {v} \right|, \tag {5}
$$
where  $\tau$  is a temperature scaling parameter. We estimate the class conditionals with  $\mathcal{N}(\mathbf{x})$  as:
$$
\hat {\mathbf {p}} _ {k} = \frac {1}{| \mathcal {N} (\mathbf {x}) |} \sum_ {\mathbf {z} \in \mathcal {N} (\mathbf {x})} \mathbb {1} \left(\underset {c} {\arg \max } p (c \mid \mathbf {z}) = k\right), \tag {6}
$$
where  $\mathbb{1}(\cdot)$  is an indicator function. Then, we can update the global class prototype computed from the whole  $\mathbb{S}$  in a moving-average style,
$$
\mathbf {p} _ {k} \leftarrow \mu \mathbf {p} _ {k} + (1 - \mu) \hat {\mathbf {p}} _ {k}, \tag {7}
$$
where  $\mu \in [0,1]$  is a preset scalar and  $\hat{\mathbf{p}}_k$  can be regarded as the local class prototype. Given a batch of test samples  $\mathcal{B}_t$ , we use self-training for model updating ( $g$  and  $h$ ), i.e., minimizing the cross-entropy loss between classifier's prediction  $f(\mathbf{x})$  and the estimated class prior distribution  $\mathbf{p}_k$ :
$$
\mathcal {L} _ {\mathrm {S T}} (\mathbf {x}) = \frac {1}{| \mathcal {B} _ {t} |} \sum_ {x \in \mathcal {B} _ {t}} \mathcal {L} _ {\mathrm {C E}} \left(\mathbf {p} _ {k}, f (\mathbf {x})\right), \tag {8}
$$
Semantic consistency. To resist catastrophic forgetting during the online adaptation process, we enforce semantic consistency between the output of  $f_{0}$  (source-trained model) and  $f_{I}$  (target model) by optimizing the cross-entropy loss between their predictions:
$$
\mathcal {L} _ {\mathrm {S C}} (\mathbf {x}) = - \sigma \left(f _ {0} (\mathbf {x})\right) \log \sigma \left(f _ {I} (\mathbf {x})\right), \tag {9}
$$
where  $I$  represents the number of iterations and  $f_{0}$  is fixed throughout the testing phase.
Reliable sample selection. In the early stage of training, the estimation of target pseudo labels may be unreliable and thus is risky to error accumulation. To improve the quality of pseudo labels and
3We omit the subscript  $t$  for simplicity.
reduce the influence of false class estimations, we introduce an entropy-based weighting strategy to select reliable samples. Specifically, we define the scoring function as follows:
$$
S (\mathbf {x}) = \mathbb {1} (H (\mathbf {x}) <   \theta_ {0}) \exp \left(\theta_ {0} - H (\mathbf {x})\right), \tag {10}
$$
where  $H(\cdot)$  is the Shannon entropy of sample and  $\theta_0$  is a pre-defined threshold. In this way, we can allocate larger weights to target samples with lower uncertainties and smaller weights to those with higher uncertainties, effectively prioritizing more confident predictions.
Test-time training objective. Formally, the overall optimization objective can be formulated as:
$$
\mathcal {L} _ {\mathrm {T T T}} (\mathbf {x}) = S (\mathbf {x}) \mathcal {L} _ {\mathrm {S T}} (\mathbf {x}) + \lambda \mathcal {L} _ {\mathrm {S C}} (\mathbf {x}), \tag {11}
$$
where  $\lambda$  is a trade-off parameter. The proposed  $\mathcal{L}_{\mathrm{TTT}}$  embraces the complementary strengths of parametric (softmax-based) and non-parametric (prototype-based) classifiers.
Benchmarks. We conduct extensive experiments on four standard DG benchmarks to verify the effectiveness of CODA. (1) PACS [28] has 9,991 images and presents remarkable distinctions in image styles. It is comprised of four domains each with seven classes, i.e., Photo, Art Painting, Cartoon, and Sketch. Dog, elephant, giraffe, and guitar are used as  $\mathcal{C}_s$  while the remaining 3 classes are  $\mathcal{C}_t^u$ . (2) Office-Home [63] is gathered from both office and home environments, and its domain shifts originate from variations in viewpoint and image style. It has 15,500 images of 65 classes from four domains, i.e., Artistic, Clipart, Product, and Real World. Arranged in alphabetical order, the initial 15 classes are designated as  $\mathcal{C}_s$ , and the remaining 50 classes are assigned to  $\mathcal{C}_t^u$ . (3) Office-31 [48] encompasses 31 classes harvested from three distinct domains: Amazon, DSLR, and Webcam. The 10 classes shared by Office-31 and Caltech-256 [18] are used as  $\mathcal{C}_s$ . In alphabetical order, the final 11 classes, combined with  $\mathcal{C}_s$ , constitute  $\mathcal{C}_t$ . (4) Digits, a dataset varying in background, style, and color, encompasses four domains of handwritten digits, including MNIST[26], MNIST-M[17], SVHN[42], USPS[22], and SYN [17]. We utilize MNIST as the source domain, while the other datasets serve as target domains. Numbers from 0 to 4 make up  $\mathcal{C}_s$ .
Evaluation Protocols. In line with prior works [5, 90, 77], we utilize the H-score  $(hs)$  [16] as our main evaluation criterion. The  $hs$  score balances the significance between known and unknown classes, emphasizing that both groups should have high and equivalent accuracy. Additionally, the  $hs$  score circumvents the trivial solution where a model would predict all samples as known classes. The classification accuracy for both known  $(acc_k)$  and unknown  $(acc_u)$  classes are also reported.
Implementation Details. For PACS, Office-Home, and Office-31, we employ ResNet-18 [19], pre-trained on ImageNet, as the backbone network. For Digits, we employ the LeNet [25] with the architecture arranged as conv-pool-conv-pool-fc-fc-softmax. The training is performed using SGD with a momentum of 0.9 for 100 epochs, and we set the batch size to 64. Our experiments are built upon Dassl [87] (a PyTorch toolbox developed for DG), covering aspects of data preparation, model training, and model selection. We report the means over 5 runs with different random seeds.
4.2Baselines
In experiments, we empirically compare CODA against five types of baselines: (1) OSDG: CM [90] and One Ring-S [77]. (2) OSDA: OSBP [49] and ROS [5]. (3) OD: MSP [20], LogitNorm [71], and DICE [56]. (4) SFDA: SHOT [35] and AaD [76]. (5) TTA: TTT [58], Tent [67], MEMO [80], TAST [24], and FAU [69]. Since TTA methods are incapable of directly handling unknown-class samples, we adopt the approach from [90], using the entropy of the softmax output as the final prediction. For ERM [60], we follow the same strategy for identifying unknowns.
Our results are summarized in Table 2. For each dataset, CODA outperforms all compared methods by a considerable margin in terms of  $hs$ . For instance, in comparison to the previous best-performing OSDG baseline [77], CODA achieves increases in  $hs$  by  $16.8\%$  for PACS,  $4.0\%$  for Office-Home,