SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Lei Zhang, Ji-Fu Li, Wei Wang*
{zhangl,lijf,wangw}@lamda.nju.edu.cn
Semi-Supervised Domain Generalization (SSDG) aims to learn a model that is generalizable to an unseen target domain with only a few labels, and most existing SSDG methods assume that unlabeled training and testing samples are all known classes. However, a more realistic scenario is that known classes may be mixed with some unknown classes in unlabeled training and testing data. To deal with such a scenario, we propose the Class-Wise Adaptive Exploration and Exploitation (CWAAEE) method. In particular, we explore unlabeled training data by using one-vs-rest classifiers and class-wise adaptive thresholds to detect known and unknown classes, and exploit them by adopting consistency regularization on augmented samples based on Fourier Transformation to improve the unseen domain generalization. The experiments conducted on real-world datasets verify the effectiveness and superiority of our method.
The machine learning community has witnessed the great progress of deep learning models and their applications, e.g., computer vision [1, 2], natural language processing [3, 4]. Such a huge success is mostly based on the i.i.d. assumption, i.e., the training data and testing data are identically and independently distributed. However, when these popular models are evaluated on new testing data, whose distribution is slightly different from the training data, a significant drop in performance will be observed [5, 6].
In order to improve the generalization of the model under distribution shifts, Domain Adaptation (DA) [7, 8, 9] and Domain Generalization (DG) [10, 11, 12] have been widely studied. The goal of DA is to transfer the knowledge learned
Source Domains
known classes
Target Domain
seen unknown classes
unseen unknown classes
Testing Samples
Labeled Samples
Figure 1: Semi-supervised domain generalization with known and unknown classes. Labeled samples on source domains (Art, Cartoon and Photo) are known classes (green box). Known classes (green box) and seen unknown classes (blue box) are mixed in unlabeled samples on source domains, while known classes (green box), seen unknown classes (blue box) and unseen unknown classes (red box) are mixed in testing samples on target domain (Sketch).
from label-rich source domains to the unlabeled or partially labeled target domain. However, the accessibility of the target domain may not always be satisfied in some applications, e.g., autonomous driving and medical diagnosis, since we are not able to anticipate the domain that deployed systems will encounter. To this end, DG is introduced to develop domain-generalizable models on unseen target domain by using multiple different and labeled source domains.
To reduce the expensive cost of the labeling process [13], Semi-Supervised Domain Generalization (SSDG) [14, 15, 16, 17] is proposed recently. In SSDG, each source domain consists of a few labeled samples and a large number of unlabeled samples. The goal is to learn a domain-generalizable model from the partially labeled samples. However, in a more realistic scenario, known classes are probably mixed with some unknown classes in unlabeled training and testing data (Figure 1). In such a scenario, when deployed on an unseen target domain, the learned model is not only required to classify known classes, but also to recognize unknown classes. In this paper, we propose the Class-Wise Adaptive Exploration and Exploitation (CWAEE) method for semi-supervised domain generalization when known and unknown classes are mixed in unlabeled data. The intuition is to first explore unlabeled training data by detecting known and unknown classes, and then exploit them in different ways to improve the generalization of the model. In particular, we use one-vs-rest classifiers and class-wise adaptive thresholds to detect known and unknown classes in unlabeled training data, and adopt consistency regularization on augmented samples based on Fourier Transformation to improve the unseen domain generalization. The experiments on real-world datasets verify that our CWAEE method can achieve better performance than compared methods.
Domain Generalization (DG) aims to help the model generalize to an unseen target domain by utilizing one or several related domains. Most existing DG methods can be classified into three different categories: domain-invariant representation learning, meta-learning and data augmentation. Motivated by the learning theory of domain adaptation [18], domain-invariant representation learning methods attempt to learn a representation space where the discrepancy between different source domains is as small as possible. For example, Muandet et al. [11] proposed a kernel-based optimization algorithm for Support Vector Machine that learns an invariant transformation by minimizing the distribution discrepancy across domains. After the domain-adversarial neural network was proposed for domain adaptation [8], domain-adversarial training is widely used for DG [19, 20]. Following Model-Agnostic Meta-Learning (MAML) [21], Li et al. [22] applied meta-learning strategy to DG through dividing the training data from multiple source domains into meta-train and meta-test sets to simulate domain shifts. The meta-optimization objective was designed to minimize the loss not only on meta-train domains but also on the meta-test domain. Shu et al. [23] conducted meta-learning over augmented domains to learn open-domain generalizable representations for Open Domain Generalization (OpenDG) problem. Compared to domain-invariant representation learning and meta-learning, data augmentation methods have shown better performance on various DG benchmarks recently. Zhou et al. [24] mixed feature statistics extracted by bottom layers of Convolutional Neural Networks (CNNs) between instances of different domains to synthesize instances in novel domains for training. Xu et al. [25] applied MixUp [26] in amplitude spectrums of two images to augment training data while preserving high-level semantics of the original images in phase spectrums. To reduce the labeling cost of DG, Semi-Supervised Domain Generalization (SSDG) [14, 15, 16, 17] receives much attention recently. Sharifi-Noghabi et al. [14] proposed the first SSDG method which combines the pseudo-label method and meta-learning strategy. Zhou et al. [16] adopted stochastic classifier and multi-view consistency to improve the quality of pseudo-labels and generalization of model respectively. However, they assumed that both unlabeled training and testing data have the same label space with labeled training data while we consider a more realistic scenario that the label space of labeled data is a subset of unlabeled training and testing data, i.e., both unlabeled training and testing data are a mixture of known and unknown classes.
Out-of-Distribution (OOD) detection aims to detect test samples whose labels are not contained in the label space of training data [27]. Hendrycks and Gimpel [28] proposed the first effective baseline that uses the maximum softmax probability to identify OOD samples. Post-hoc methods attract much attention because they can be implemented without modifying the training procedure and objective. For example, Liang et al. [29] used temperature scaling and input perturbations to make In-Distribution (ID) and OOD samples more separable. Liu et al. [30] proposed to replace softmax
confidence scores with energy scores to detect OOD samples, and considered samples with higher energy to be OOD samples. Hendrycks et al. [31] maximized the entropy of the model's predictions on enormous unlabeled OOD samples to improve OOD detection. Yang et al. [32] proposed the Unsupervised Dual Grouping (UDG) method that adopts two classification heads to dynamically separate ID and OOD samples in the unlabeled data and optimizes the model with different learning objectives. Since the commonly-used OOD detection benchmarks [28, 29, 31] are based on dataset splitting setup, i.e., one dataset as ID and all the others as OOD, the existing OOD detection methods overfit to low-level dataset statistics instead of learning semantics, which leads to that even the same class from another dataset will be detected as OOD samples [32].
Semi-Supervised Learning (SSL) is an active research area in the last decade, whose goal is to leverage both labeled and unlabeled data to improve the performance of models and reduce the label cost. The most popular methods include consistency regularization [33, 34] and entropy minimization [35, 36]. The basic idea behind consistency regularization is that the model's outputs of similar samples should be similar. Sajjadi et al. [33] proposed the  $\Pi$  model that minimizes consistency loss over two random augmentations of one sample to regularize the model. Tarvainen et al. [34] proposed the Mean Teacher method that averages Student model weights using Exponential Moving Average (EMA) over training steps as Teacher model to provide more stable targets for the consistency loss optimization of the Student model. Entropy minimization methods encourage models to make low entropy predictions on unlabeled data, so that the learned decision boundary passes through a low-density region rather than a high-density area. Lee [36] proposed the Pseudo-label method that sets the maximum confidence prediction of unlabeled samples as pseudo-labels, and then trains the model in a supervised way with labeled and pseudo-labeled data. Sohn et al. [37] proposed the FixMatch method that combines consistency regularization and entropy minimization. It generates highly confident pseudo-labels on weakly-augmented unlabeled samples, and trains the model to predict the pseudo-labels for the strongly-augmented version of the same samples. OpenMatch [38] extended FixMatch with OVANet [39] and open-set soft-consistency regularization to detect outliers in unlabeled data for Open-set Semi-Supervised Learning (OSSL).
We have multiple source domains  $\mathcal{D}_1, \mathcal{D}_2, \ldots, \mathcal{D}_s$  available for training, and each  $\mathcal{D}_{k \in \{1, 2, \ldots, s\}}$  has one labeled set  $\mathcal{D}_k^l = \{(\mathbf{x}_i^{(k)}, y_i^{(k)})\}_{i=1}^{n_k^l}$  and one unlabeled set  $\mathcal{D}_k^u = \{(\mathbf{u}_i^{(k)})\}_{i=1}^{n_k^u}$ , i.e.,  $\mathcal{D}_k = \mathcal{D}_k^l \cup \mathcal{D}_k^u$ . Generally, the size of  $\mathcal{D}_k^u$  may be much larger than that of  $\mathcal{D}_k^l$ , i.e.,  $n_k^u \gg n_k^l$ . The total size of labeled dataset is denoted as  $n^l$  and the total size of unlabeled dataset is denoted as  $n^u$ , i.e.,  $n^l = \sum_{k=1}^{s} n_k^l$ , and  $n^u = \sum_{k=1}^{s} n_k^u$ . The model is evaluated on an unseen target domain  $\mathcal{D}_t = \{(\mathbf{x}_i^{(t)}, y_i^{(t)})\}_{i=1}^{n_t}$ .  $\mathcal{C}^l$  is the label set of labeled data  $\mathcal{D}^l = \mathcal{D}_1^l \cup \mathcal{D}_2^l \cup \dots \cup \mathcal{D}_s^l$ ,  $\mathcal{C}^u$  is the label set of unlabeled data  $\mathcal{D}^u = \mathcal{D}_1^u \cup \mathcal{D}_2^u \cup \dots \cup \mathcal{D}_s^u$ , and  $\mathcal{C}^t$  is the label set of target domain  $\mathcal{D}_t$ . Due to the existence of unknown classes in unlabeled data and target domain, we have  $\mathcal{C}^l \subset \mathcal{C}^u \subset \mathcal{C}^t$ . The classes in  $\mathcal{C}^l$  are called known classes, the classes in  $\mathcal{C}^u \setminus \mathcal{C}^l$  are called seen unknown classes since they are seen in unlabeled data during the training process and the classes in  $\mathcal{C}^t \setminus \mathcal{C}^u$  are called unseen unknown classes since they are unseen during the training process. The goal is to learn the model  $f^*$  that generalizes well on target domain  $\mathcal{D}_t$  for both known and unknown classes, i.e., for each  $(\mathbf{x}_i^{(t)}, y_i^{(t)}) \in \mathcal{D}_t$ , if  $y_i^{(t)} \in \mathcal{C}^l$ , then  $f^*(\mathbf{x}_i^{(t)}) = y_i^{(t)}$ , else  $f^*(\mathbf{x}_i^{(t)}) = unknown$ .
To effectively utilize unlabeled data to improve the generalization and robustness of the model, we should carefully exploit known and unknown classes in them. If known classes are treated as unknown ones, the generalization of the model may decrease significantly; if unknown classes are treated as known ones, the robustness of the model may degrade.
3.1 Detecting Known and Unknown Classes
In order to explore unlabeled data and detect known and unknown classes in them, we rely on the outputs of the deep neural networks and the intuition is that known classes will have higher scores than unknown classes [28]. The deep neural network model  $f_{\Theta} = h_{\omega} \circ g_{\theta}$  contains two parts, a feature extractor  $g_{\theta}$  and a  $|\mathcal{C}^l|$ -way linear classifier  $h_{\omega} = [h_{\omega}^{1}; \ldots; h_{\omega}^{|\mathcal{C}^l|}]$ . The model maps each sample  $\mathbf{x}_i$  into a  $|\mathcal{C}^l|$ -dimensional logits  $\mathbf{z}_i = h_{\omega} \circ g_{\theta}(\mathbf{x}_i)$ , and then feeds  $\mathbf{z}_i$  into a softmax function
Procedure 1 Calculating Thresholds
Input: Score queues  $q^{1:|\mathcal{C}^l|}$ , feature extractor  $g_{\theta}$ , linear classifiers  $h_{\omega}^{1:|\mathcal{C}^l|}$ , validation dataset  $\mathcal{D}^v$ , number of known classes  $|\mathcal{C}^l|$ .
Output: Temperatures  $\tau^{1:|\mathcal{C}^l|}$ , thresholds  $\delta_{knw}^{1:|\mathcal{C}^l|}$  and  $\delta_{unk}^{1:|\mathcal{C}^l|}$ .
1: for  $c = 1$  to  $|\mathcal{C}^l|$  do
2: Calculate temperature  $\tau^c$  to calibrate  $g_{\theta}, h_{\omega}^{c}$  on  $\mathcal{D}^v$  with Eq.1;
3: Fit a two-component beta mixture model  $\alpha_{1:2}^{c},\beta_{1:2}^{c}$  on queue  $q^{c}$  with EM algorithm;
4: Calculate known and unknown classes thresholds  $\delta_{knw}^{c} = \frac{\alpha_{1}^{c}}{\alpha_{1}^{c} + \beta_{1}^{c}},\delta_{unk}^{c} = \frac{\alpha_{2}^{c}}{\alpha_{2}^{c} + \beta_{2}^{c}}.$
5: end for
$S$  to produce the estimated posterior probability  $\hat{\mathbf{p}}_i$  (a.k.a. score) of each class, i.e.,  $\hat{\mathbf{p}}_i = S(f_\Theta(\mathbf{x}_i))$ . The training is to minimize the cross-entropy loss  $\mathcal{L}_{CE}(\Theta)$  between  $\hat{\mathbf{p}}_i$  and one-hot label  $\mathbf{y}_i$  over labeled data, i.e.,  $\Theta^* = \operatorname{argmin}_{\Theta} \mathcal{L}_{CE}(\Theta) = \operatorname{argmin}_{\Theta} -\frac{1}{n^l}\sum_{i=1}^{n^l}\sum_{c=1}^{|\mathcal{C}^l|}\mathbf{y}_i^c\log \hat{\mathbf{p}}_i^c$ , where  $n^l$  is the size of labeled data. The gradient of cross-entropy loss w.r.t. linear classifier  $h_\omega$  corresponding to class  $c$  is  $\frac{\partial\mathcal{L}_{CE}(\Theta)}{\partial h_\omega^c} = \frac{1}{n^l}\sum_{i=1}^{n^l}(\hat{\mathbf{p}}_i^c - \mathbf{y}_i^c)g_\theta(\mathbf{x}_i)$ . Due to the relative normalization of the softmax function, the score of  $\mathbf{x}_i$  on class  $c$ , i.e.,  $\hat{\mathbf{p}}_i^c$ , not only depends on the logit  $\mathbf{z}_i^c$ , but also depends on other logits  $\mathbf{z}_i^j, j \neq c$ . Thus, the update of  $h_\omega^c$  is affected by the predictions of other classifiers  $h_\omega^j, j \neq c$ . When the logit of  $\mathbf{x}_i$  on target class  $y_i$  is large, the scores of  $\mathbf{x}_i$  on non-target class, i.e.,  $\hat{\mathbf{p}}_i^c, c \neq y_i$ , will be small. Since the  $\hat{\mathbf{p}}_i^c - \mathbf{y}_i^c$  is small, the gradient of the non-target classifiers may converge to zero. The parameters of the neural network are updated with the stochastic gradient descent (SGD) algorithm, so, the logits of samples on non-target classifiers induced by  $h_\omega^j, j \neq c$  will not decrease.
In order to update the logits on non-target classifiers induced by  $h_{\omega}^{j}, j \neq c$ , we replace the softmax function  $S$  with the sigmoid function  $\sigma$ . In this way, we obtain an individual score on each way to indicate the probability that one sample belongs to each class, i.e.,  $\hat{\mathbf{p}}_i^c = \sigma(\mathbf{z}_i^c), c \in \mathcal{C}^l$ , and we get  $|\mathcal{C}^l|$  one-vs-rest classifiers. With these one-vs-rest classifiers, we can detect known and unknown classes. A straightforward way is to set a fixed threshold  $\delta = 0.5$  [39, 38]. Let  $j = \operatorname{argmax}_{c} (\hat{\mathbf{p}}_i^c)$ , if  $\hat{\mathbf{p}}_i^j \geq \delta$ , it means that it belongs to class  $j$ ; if  $\hat{\mathbf{p}}_i^j < \delta$ , it means that it is an unknown classes sample. Due to the overconfidence of modern neural networks [40], it is not proper to use such a fixed threshold of 0.5 for all classes. A more reasonable way is to use the class-wise adaptive threshold [41, 42] for each class. To get well-calibrated scores, we first calibrate the classifiers on validation data  $\mathcal{D}^v$  with temperature scaling [40]:
$$
\tau^ {c} = \operatorname {a r g m i n} - \sum_ {\left(\mathbf {x} _ {i}, y _ {i}\right) \in \mathcal {D} ^ {v}} \mathbb {I} \left(y _ {i} = c\right) \log \left(\sigma \left(\mathbf {z} _ {i} ^ {c} / \tau^ {c}\right)\right) + \mathbb {I} \left(y _ {i} \neq c\right) \log \left(1 - \sigma \left(\mathbf {z} _ {i} ^ {c} / \tau^ {c}\right)\right), \tag {1}
$$
where  $\mathbb{I}(\cdot)$  is an indicator function and  $\tau^c$  is the temperature for class  $c\in \mathcal{C}^l$ . Note that the validation dataset  $\mathcal{D}^v$  only contains samples of known classes from source domains. And then, we use a two-component beta mixture model to model the score distributions of known classes and unknown classes in an unsupervised way, since it is a more flexible approximator than the Gaussian mixture model [43]. The adaptive thresholds for known classes  $\delta_{knw}^{j}$  (the large one) and unknown classes  $\delta_{unk}^{j}$  (the small one) can be set as the mean values of two fitted beta distributions. Thus, for an unlabeled sample  $\mathbf{u}_i\in \mathcal{D}^u$ , let  $\tilde{\mathbf{p}}_i^j = \sigma (\mathbf{z}_i^j /\tau^j)$  be the scaled score, if  $\tilde{\mathbf{p}}_i^j >\delta_{knw}^j$ , we consider it belongs to known class  $j$ , i.e.  $\hat{y}_i = j$ ; if  $\tilde{\mathbf{p}}_i^j < \delta_{unk}^j$ , we consider it belongs to unknown classes, i.e.  $\hat{y}_i = \text{unknown}$ ; if  $\delta_{unk}^j\leq \tilde{\mathbf{p}}_i^j\leq \delta_{knw}^j$ , it is difficult to determine its label, hence we set its prediction as null, i.e.  $\hat{y}_i = \text{null}$ . Since the model is updated in a mini-batch way, the scores predicted by the model many steps ago are less useful for modeling the score distributions. Hence, we utilize the most recent scores predicted by the model for the fitting of the beta mixture model. In particular, for each class  $j$  we maintain a queue  $q^{j}$  with fixed length to record the most recent scaled maximum score of unlabeled sample  $\mathbf{u}_i$  if its maximum score belongs to class  $j$ , i.e.,  $\arg\max_c(\tilde{\mathbf{p}}_i^c) = j$ . The overall procedure is summarized in Procedure 1.
3.2 Improving Target Domain Generalization
For labeled training data, we use the following supervised loss,
$$
\mathcal {L} ^ {l} = - \frac {1}{n ^ {l}} \sum_ {i = 1} ^ {n ^ {l}} \sum_ {c = 1} ^ {| \mathcal {C} ^ {l} |} \mathbb {I} (y _ {i} = c) \ln (\hat {\mathbf {p}} _ {i} ^ {c}) + \frac {1}{| \mathcal {C} ^ {l} | - 1} \mathbb {I} (y _ {i} \neq c) \ln (1 - \hat {\mathbf {p}} _ {i} ^ {c}). \tag {2}
$$
For unlabeled training data, we should exploit them carefully to improve the unseen target domain generalization after having detected known and unknown classes. We construct the weakly and strongly augmented version of each sample following FixMatch [37], and calculate the model's scaled confidence score on its weakly augmented version, i.e.,  $\tilde{\mathbf{p}}_{i,weak}^{c} = \sigma\left(f_{\Theta}\left(T_{weak}(\mathbf{u}_{i})\right) / \tau^{c}\right), c \in \mathcal{C}^{l}$ . Then, we assign pseudo-label  $\hat{y}_i$  to it according to Section 3.1, and force the model's prediction on the strongly augmented version  $\hat{\mathbf{p}}_{i,strong}^{c} = \sigma\left(f_{\Theta}\left(T_{strong}(\mathbf{u}_{i})\right)\right), c \in \mathcal{C}^{l}$  to match the pseudo-label. For unlabeled training data predicted as known classes (i.e.,  $\hat{y}_i \in \mathcal{C}^l$ ), the loss is defined as
$$
\mathcal {L} _ {k n w} ^ {u} = - \frac {1}{n _ {k n w} ^ {u}} \sum_ {i = 1} ^ {n _ {k n w} ^ {u}} \sum_ {c = 1} ^ {| \mathcal {C} ^ {l} |} \mathbb {I} (\hat {y} _ {i} = c) \ln \left(\hat {\mathbf {p}} _ {i, s t r o n g} ^ {c}\right) + \frac {1}{| \mathcal {C} ^ {l} | - 1} \mathbb {I} (\hat {y} _ {i} \neq c) \ln (1 - \hat {\mathbf {p}} _ {i, s t r o n g} ^ {c}), \tag {3}
$$
where  $n_{knw}^{u}$  is the number of samples predicted as known classes. For unlabeled training data predicted as unknown classes (i.e.,  $\hat{y}_i = \text{unknown}$ ), the loss is defined as
$$
\mathcal {L} _ {u n k} ^ {u} = - \frac {1}{n _ {u n k} ^ {u}} \sum_ {i = 1} ^ {n _ {u n k} ^ {u}} \sum_ {c = 1} ^ {| \mathcal {C} ^ {l} |} \ln \left(1 - \hat {\mathbf {p}} _ {i, s t r o n g} ^ {c}\right), \tag {4}
$$
where  $n_{unk}^{u}$  is the number of samples predicted as unknown classes.
Besides known and unknown classes, there are some unlabeled samples that are predicted as null. In order to fully exploit these data and improve the generalization of the model on the unseen target domain, we adopt Fourier Transformation to disentangle the semantics and styles of the sample [25, 44], and then MixUp [26] the styles of two randomly sampled samples to augment training data for regularizing the model. Specifically, for each  $\mathbf{x}$ , its Fourier Transformation  $\mathcal{F}(\mathbf{x})$  is formulated as:
$$
\mathcal {F} (\mathbf {x}) (u, v) = \sum_ {h = 0} ^ {H - 1} \sum_ {w = 0} ^ {W - 1} \mathbf {x} (h, w) e ^ {- J 2 \pi \left(\frac {h}{H} u + \frac {w}{W} v\right)}, \tag {5}
$$
where  $H, W$  are the height and width of the sample, and  $J^2 = -1$ . The amplitude component  $\mathcal{A}(\mathbf{x})$  and phase component  $\mathcal{P}(\mathbf{x})$  are then respectively expressed as
$$
\mathcal {A} (\mathbf {x}) (u, v) = \left[ R ^ {2} (\mathbf {x}) (u, v) + I ^ {2} (\mathbf {x}) (u, v) \right] ^ {1 / 2}, \mathcal {P} (\mathbf {x}) (u, v) = \arctan \left[ \frac {I (\mathbf {x}) (u , v)}{R (\mathbf {x}) (u , v)} \right], \tag {6}
$$
where  $R(\mathbf{x})$  and  $I(\mathbf{x})$  represent the real and imaginary part of  $\mathcal{F}(x)$ . It is well-known that, the phase component of the Fourier spectrum preserves high-level semantics of the original signal, while the amplitude component contains low-level statistics. Therefore, we could mix up amplitude components of two samples of the same batch while keeping the phase components to generate the augmented samples  $\tilde{\mathbf{x}}$  [25, 44], i.e.,
$$
\tilde {\mathcal {A}} \left(\mathbf {x} _ {i}\right) = (1 - \lambda) \mathcal {A} \left(\mathbf {x} _ {i}\right) + \lambda \mathcal {A} \left(\mathbf {x} _ {i ^ {\prime}}\right), \tilde {\mathbf {x}} _ {i} = \mathcal {F} ^ {- 1} \left(\tilde {\mathcal {A}} \left(\mathbf {x} _ {i}\right) * e ^ {- J * \mathcal {P} \left(\mathbf {x} _ {i}\right)}\right), \tag {7}
$$
where  $\mathcal{F}^{-1}(x)$  is the inverse Fourier Transformation,  $\lambda \sim U(0,1)$ , and  $U$  is the uniform distribution.
To avoid overfitting on domain-related low-level statistics, we minimize the consistency regularization loss between the original unlabeled samples and the augmented ones to push the model to pay attention to the high-level semantics of the samples, defined as
$$
\mathcal {L} _ {\text {c o n}} ^ {u} = \frac {1}{n ^ {u}} \sum_ {i = 1} ^ {n ^ {u}} \sum_ {c = 1} ^ {| \mathcal {C} ^ {l} |} \left| \hat {\mathbf {p}} _ {i} ^ {c} \left(\mathbf {x} _ {i}\right) - \hat {\mathbf {p}} _ {i} ^ {c} \left(\tilde {\mathbf {x}} _ {i}\right) \right| ^ {2}, \tag {8}
$$
where  $n^u$  is the number of unlabeled data. The overall loss of the training process is formulated as:
$$
\mathcal {L} = \mathcal {L} ^ {l} + \lambda_ {1} \mathcal {L} _ {k n w} ^ {u} + \lambda_ {2} \mathcal {L} _ {u n k} ^ {u} + \lambda_ {3} \mathcal {L} _ {c o n} ^ {u}, \tag {9}
$$