SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The labels in weakly-supervised learning (WSL) could be incomplete, inexact, inaccurate [59] because accurate annotation is often expensive and difficult to obtain. Soft labels are widely used in WSL. Many weakly-supervised learning paradigms can be transformed into learning from soft labels. Due to the lack of the supervisory information, the soft labels could be large-biased but the model can still learn from them. In this section, our theory is adapted to three classic weakly-supervised learning paradigms and can provide theoretical guarantee for the learnability of these problems. These findings reflect that the theory is promising and extensible.
4.1 Incomplete Supervision
In incomplete supervision, there are labeled data and unlabeled data. A common approach is to use the model to label the unlabeled data and then learn with all data iteratively. The soft labels of the unlabeled data are probably biased but make a significant contribution to the training process. We propose an ideal accuracy function and, based on it, we provide a theoretical analysis on incomplete supervision from the perspective of soft labels. Suppose there are  $N$  labeled data and  $M$  unlabeled data sampled from  $\mathcal{X} \times \mathcal{Y}$ . The predictive model  $h$  is an ERM learner on both labeled data and unlabeled data. The label of unlabeled data will be updated iteratively.
Assumption 1. For  $N$  labeled data and  $M$  unlabeled data whose soft labels have unreliability degree  $\eta$  and ambiguity degree  $\gamma$ , the ERM learner  $h$  has a deterministic accuracy function  $\rho(\eta, \gamma)$ , the probability that  $h$  predict correctly. The model architecture, data distribution and optimization are implicitly included in  $\rho(\eta, \gamma)$ .
Assumption 2. Given that smaller values of  $\eta$  or  $\gamma$  result in more supervised information in the soft labels, we assume  $\rho (\eta ,\gamma)$  decreases with  $\eta$  and  $\gamma$
We suppose that the incorrect labels share the equal probability to appear in the top- $k$  set. The distribution of the incorrect labels can be characterized in a more refined manner. For instance, assume that there is a upper bound of  $\frac{p(i|\mathbf{x})}{p(j|\mathbf{x})}, i,j \in \mathcal{V}, i,j \neq y, i \neq j$ . We employ simplified assumptions because they can still capture and reflect this process. Based on the ideal  $\rho(\eta,\gamma)$ , we delineate the progressive performance of  $h$ . Let  $\rho_t$  denote the accuracy of  $h$  at epoch  $t$  and we have
$$
\rho_ {t + 1} \geq \rho \left(1 - \rho_ {t}, \frac {c - k - \rho_ {t}}{c - 1}\right). \tag {3}
$$
In practice, as learner  $h$  learns from labeled data and unlabeled data, the performance of  $h$  will improve and the soft labels of the unlabeled data will be more effective. Consequently,  $h$  and the soft labels may achieve a dynamic equilibrium.
Theorem 3. Based on the ideal accuracy function  $\rho (\eta ,\gamma)$ , with a moderate initial state  $\eta_0,\gamma_0$  satisfying Theorem 2, if final accuracy of  $\rho_{final}$  exists, it can be calculated by the following fixed point equation:
$$
x = \rho (1 - x, \frac {c - k - x}{c - 1}). \tag {4}
$$
where  $k$  accords with the top- $k$  set in  $\eta$  and  $\gamma$ ,  $c$  is the number of class labels. If  $\rho(\eta, \gamma)$  is  $k_L$ -Lipschitz continuous ( $k_L < 1 - \frac{1}{c}$ ), then  $\rho_{\text{final}}$  exists and is unique.
The proof is detailed in A.6. In fact, the deterministic  $\rho(\eta, \gamma)$  is unattainable due to the indeterminacy of the optimization and the potential uncertainty in the data distribution. An intuitive extension is to assume  $\rho(\eta, \gamma)$  is a probability distribution related to the training specifics, which can be
further investigated. Theorem 3 is coarse yet in agreement with the general intuition. The model  $h$  improves as the soft labelsenvolve and finally reach the bottleneck restricted by the model, data and optimization.
In this section, we demonstrate the potential benefits of our theory in several classic weakly-supervised learning paradigms. There remain numerous domains associated with soft labels. It is essential to possess an appropriate theory to analyze the soft labels for the corresponding algorithms. Our theory can be instrumental for comprehending and constructing the soft label based algorithms.
4.2 Partial Label Learning
In Partial Label Learning (PLL), each instace is typically assigned a set of possible labels, i.e., the candidate label set  $s$  [28, 46, 49]. The corresponding candidate label space is denoted by  $S$ , i.e., the non-empty power set of  $\mathcal{V}$ . Traditional PLL assumes the ground-truth label must be in the candidate label set. But recently, Lv et al. [29] considers that ground-truth label could be not in the candidate label set, which is named as Unreliable Partial Label Learning (UPLL).
In UPLL, there are two basic concepts, partial rate  $\nu$  and unreliable rate  $\mu$ . Partial rate  $\nu$  is the ratio of incorrect labels in the candidate label set to total labels. A lower partial rate usually indicates a better performance of the model. Unreliable rate  $\mu$  is the probability of ground-truth label  $y$  not in the candidate label set, which can be formally stated as
$$
\mu = \Pr_ {(\boldsymbol {x}, y, s) \sim \mathcal {X} \times \mathcal {Y} \times \mathcal {S}} (y \notin s).
$$
The discrete candidate label set  $s$  can be transformed into the soft label by
$$
d _ {i} = \left\{ \begin{array}{c c} \frac {1}{| s |} & i \in s \\ 0 & i \notin s \end{array} \right.,
$$
where  $|s|$  is the cardinality of set  $s$ . So PLL also can be viewed as learning from biased soft labels. Then we have the following corollary.
Corollary 1. For UPLL with partial rate  $\nu$  and unreliable rate  $\mu$ , regard the generated soft labels as teacher. Then, we have  $\eta = \mu$  and  $\gamma = \nu$ . With the same conditions in Theorem 2, UPLL is ERM learnable and the sample complexity remains unchanged.
This corollary shows that UPLL is ERM learnable under a moderate condition. We provide new insights from the perspective of soft labels.
4.3 Learning with Additive Noise
Additive noise mechanism [30, 14] is an important methodology for differential privacy. Specifically, Laplace noise or Gaussian noise is added to data for protecting privacy. The privacy budget can be controlled by adjusting the scale of the noise. After normalization, the noisy labels are also biased soft labels in nature. In fact, given the probability density function of noise, we can calculate the corresponding unreliability degree and ambiguity degree in order to measure the effectiveness of the noisy labels. With the biased soft labels, the task is to train a utility model with strong privacy guarantees. Our theory can guarantee the utility of such soft labels.
To calculate unreliability degree and ambiguity degree, we refer to order statistic [8]. Order statistic analyze the  $i$ th-smallest value of random samples from a continuous distribution. We denote the order distribution  $\text{Order}(d, n, i)$  as the  $i$ th-smallest value of  $n$  samples from distribution  $d$ . The software Mathematics [45] provide an efficient API for estimating the order distribution.
Corollary 2. Let  $d$  denote the noise distribution (e.g. Laplace noise and Gaussian noise) and regard the noisy soft labels as the teacher. With the  $k$  in Eq.1 and the total classes  $c$ , for  $k \leq c - 1$ , we can compute the  $\eta$  and  $\gamma$  as
$$
\eta = \operatorname *{Pr}_{\substack{x\sim Order(d,c - 1,n - k + 1)\\ y\sim d}}(1 + y > x),
$$
$$
\gamma = \frac {\eta + k - 1}{c - 1}.
$$
With the same conditions in Theorem 2, the problem learning with additive noise is ERM learnable and the sample complexity remains unchanged.
(c)
Figure 2: Indicators of soft labels generated by SBTs on CIFAR-10 (a), CIFAR-100 (b) and Tiny ImageNet (c). Acc represents the accuracy of the student model trained with the biased soft labels. Unreliability degree  $\eta_{k}(f)$  and ambiguity degree  $\gamma_{k}(f)$  are the two indicators proposed in our study.
As the scale of the noise increases,  $\eta$  and  $\gamma$  will increase, i.e., the effectiveness of the soft labels will decrease. This result agrees with the practical situation.
Our experimentally investigate whether biased soft labels are still effective and whether the proposed indicators can measure the effectiveness of these soft labels, which consist of three parts. Firstly, we design a simple heuristic algorithm to generate Skillful but Bad Teachers (SBTs), which have low accuracy (less than  $35\%$ ) but can teach good students. Secondly, experimental results demonstrate that students learning from SBTs can achieve comparable performance as models trained on the original data. We can explain these phenomenons with unreliability degree and ambiguity degree. Thirdly, we conduct experiments in weakly-supervised learning paradigms, and the results also confirm that unreliability degree and ambiguity degree can reflect the effectiveness of the biased soft labels. Due to space limitations, we present a part of experimental results in the appendix. Additionally, we provide the details of the experiment setup in appendix A.7.
5.1 Skillful but Bad Teachers
In this subsection, we introduced the design of the SBTs. The intuition behind SBTs, inspired by Theorem 2, is to inhibit correct predictions and reduces the unreliability degree and ambiguity degree simultaneously. So we design some heuristic loss functions and have some hyperparameters that qualitatively control the unreliability degree and ambiguity degree. More specifically, the designed loss functions are intended to keep the ground-truth label within the top- $k$  set of the soft labels, without necessarily requiring it to be at the top. Here,  $k$  is an empirical constant, which we have set to 3 or 4 in training SBTs.
Firstly, SBTs will punish those correctly predicted instances as
$$
\mathcal {L} _ {\text {p u n}} (\boldsymbol {x}, y) = - \mathbb {I} _ {j \in \mathcal {Y}} \left(\boldsymbol {d} _ {j}\right) = y) \ell (f (\boldsymbol {x}), y), \tag {5}
$$
(b)
Figure 3: Indicators of learning with Gaussian noise on CIFAR-10 (a) and CIFAR-100 (b).
where  $\ell (\cdot ,\cdot)$  is the cross entropy loss function. In practice, the value of the ground-truth label decreases significantly, which causes the ground-truth label to fall out of the top-  $k$  set, i.e., large  $\eta_{k}$ . So the ground-truth label  $y$  is compensated when  $y\notin \Omega_k(f(\pmb {x}))$ :
$$
\mathcal {L} _ {\operatorname {c o m p}} (\boldsymbol {x}, y) = \mathbb {I} (y \notin \Omega_ {k} (f (\boldsymbol {x}))) \ell (f (\boldsymbol {x}), y). \tag {6}
$$
The compensation term is designed to improve the top-  $k$  accuracy of SBTs, which keeps the statistical effectiveness of the soft labels generated by SBTs. In practice, we found there was a strong correlation among the top-  $k$  labels, which led to the confusion between the ground-truth label and similar labels, i.e., large  $\gamma_{k}$ . To decrease this correlation, we propose an effective method to make the labels in  $\Omega_{k}(f(x))$  as independent as possible. We randomly select  $k - 1$  labels except the ground-truth label  $y$ . Then the selected  $k - 1$  labels are employed as the learning objectives:
$$
\mathcal {L} _ {\mathrm {r n d}} (\boldsymbol {x}, y) = \ell (f (\boldsymbol {x}), s _ {\mathrm {r n d}}). \tag {7}
$$
where  $s_{\mathrm{md}}$  is the set of the  $k - 1$  random labels excluding  $y$ . Consequently, the total objective of SBTs is as follows:
$$
\mathcal {L} (\boldsymbol {x}, y) = \mathcal {L} _ {\mathrm {c e}} (\boldsymbol {x}, y) + \alpha_ {1} \mathcal {L} _ {\text {p u n}} (\boldsymbol {x}, y) + \alpha_ {2} \mathcal {L} _ {\text {c o m p}} (\boldsymbol {x}, y) + \alpha_ {3} \mathcal {L} _ {\text {r n d}} (\boldsymbol {x}, y) \tag {8}
$$
where  $\mathcal{L}_{\mathrm{ce}}(\pmb{x},y)$  is the vanilla cross-entropy loss between the output of the model and the ground-truth label, and  $\alpha_{1},\alpha_{2},\alpha_{3}$  are the trade-off parameters.
5.2 Effectiveness of the Proposed Indicators
In this subsection, our aim is to demonstrate that, despite the soft labels generated by SBTs are large-biased, students still achieve high accuracy. As outlined in Table 2 of the appendix, we adapt four different classic metrics: Chebyshev distance, KL divergence, Manhattan distance, and Euclidean distance. These metrics are employed to measure the discrepancies between the large-biased soft labels generated by SBTs and the ground-truth labels. The soft labels generated by SBTs exhibit substantial differences when compared to those from normally trained teachers. Nevertheless, they remain effective in instructing good students. It should be noted that the accuracy of the students mirrors the effectiveness of the soft labels. However, accuracy is an indirect measure contingent on the student model. The proposed unreliability degree and ambiguity degree can directly measure the effectiveness of these soft labels. The smaller the unreliability degree and ambiguity degree, the more effective the soft labels, and subsequently, the students' accuracy tends to be higher. This phenomenon is illustrated in Figure 2, which agrees with our expectations.
In the experiments, we set  $k = 4$  for unreliability degree  $\eta_k(f)$  and ambiguity degree  $\gamma_k(f)$  ( $k$  in the indicators can be different from  $k$  in training SBTs). Since  $\eta_k(f)$  and  $\gamma_k(f)$  are probabilities over the entire data distribution  $\mathcal{X} \times \mathcal{Y}$  and difficult to compute, we estimate them empirically by the the generated soft labels on the test data. Note that training with the ground-truth labels can achieve accuracy  $95.29\%$  on CIFAR-10,  $78.13\%$  on CIFAR-100 and  $72.53\%$  on Tiny ImageNet. We can find many interesting results in Figure 2:
- When these soft labels are generated by SBTs with accuracy less than  $30\%$ , which means they are quite different from the ground-truth labels, the students can still achieve accuracy much higher than the teachers.
- As  $\eta_k(f)$  and  $\gamma_k(f)$  decrease, Acc (i.e., accuracy of the student) increases. The proposed indicators exhibit an inverse correlation with the students' accuracy, which reflects the effectiveness of the proposed indicators.
- For the more complicated dataset like CIFAR-100 and Tiny ImageNet, accuracy (Acc) is more sensitive to  $\eta_k(f)$  and  $\gamma_k(f)$ . This implies that, for more complex tasks, the quality of the annotation is more crucial.
The hyperparameters in Eq.(8) are pivotal in controlling the indicators of these soft labels, which is elucidated in A.8. Furthermore, we conducted experiments to investigate whether the effectiveness of soft labels is influenced by different model backbones, as detailed in A.9. We tested a range of architectures, including wideresnet 28x2, 28x4, 40x2, and 40x4. The four distinct backbones exhibited consistent patterns, indicating that the proposed indicators are effective across different backbones. In addition, we present the overall distribution of these soft labels in A.10. The experimental results demonstrate that biased soft labels can also teach good students and the proposed indicators can measure the effectiveness of these soft labels, which confirm the validity of our theory.
5.3 Biased Soft Labels in Weakly-Supervised Learning
The aforementioned weakly-supervised learning paradigms have engendered a lot of specialized algorithms. The intention of this paper is not to devise more efficacious algorithms, but to evaluate the effectiveness of the soft labels in the weakly-supervised learning paradigms by unreliability degree and ambiguity degree. The results of partial label learning are shown in Table 1, while the results of learning with additive noise are shown in Figure 3. Due to space limitations, the results of incomplete supervision are shown in A.11. From the three experiments, we can observe that the accuracy of the students decrease when unreliability degree and ambiguity degree increase. All the results reflect that these indicators can measure the effectiveness of the soft labels well, which is consistent with our theory. In addition, since the soft labels in partial label learning and learning with additive noise are generated by a mapping rule rather
Table 1: Students learning from partial labels.
Dataset	γk(f)	ηk(f)	Student
CIFAR-10	0.1	0.1	93.98
0.1	0.3	93.38
0.1	0.5	91.94
0.3	0.1	90.38
0.3	0.3	88.59
0.3	0.5	86.28
0.5	0.1	85.11
0.5	0.3	82.05
0.5	0.5	77.95
CIFAR-100	0.01	0.1	74.19
0.01	0.3	73.16
0.01	0.5	72.29
0.05	0.1	68.08
0.05	0.3	66.28
0.05	0.5	63.69
0.1	0.1	61.18
0.1	0.3	58.6
0.1	0.5	52.41
than a neural network, the results of these experiments are more stable and smooth.
In this paper, we find that even large-biased soft labels can teach a good student and focus on the effectiveness of the biased soft labels. It motivates us to rethink when the biased soft labels (or teachers) are effective. We propose two indicators, unreliability degree and ambiguity degree, to measure the effectiveness of soft labels. Based on the proposed indicators, we provide moderate conditions that guarantee the classifier-consistency and ERM learnability of the biased soft label learning problem. Our theoretical framework can be adapted to elucidate the effectiveness of soft labels in three weakly-supervised learning paradigms, incomplete supervision, partial label learning and learning with additive noise. We design a heuristic method to train Skillful but Bad Teachers (SBTs), which validate that large-biased soft labels can teach good students. Besides, the effectiveness of both soft labels generated by SBTs and soft labels in the weakly-supervised learning paradigms can be measured by the proposed indicators well, which is consistent with our theory.
This work was supported by the National Key Research & Development Plan of China (No. 2018AAA0100104), National Science Foundation of China (62125602, 62076063, 62206048), Natural Science Foundation of Jiangsu Province (BK20220819), and Young Elite Scientists Sponsorship Program of Jiangsu Association for Science and Technology Tj-2022-027.
A.1 Variants of Knowledge Distillation
Self-distillation [55, 25, 50] treats the mixture of the outputs and the ground-truth labels as target soft labels, and the proportion is iteratively adjusted during the training. Ensemble KD [52, 60] employs the ensemble of the soft labels (teachers) to improve generalization. Besides, in mutual learning [56, 51], there is no explicit teacher network and multiple students learn from each other by synthesizing soft labels of other students. Soft labels play an important role in all these variants.
For simplification, we denote the probability of event  $A$  as  $p(A)$ .
$$
\begin{array}{l} \operatorname {E r r} _ {\mathcal {D}} (h | f) = \mathbb {E} _ {(\boldsymbol {x}, y) \sim \mathcal {X} \times \mathcal {Y}} [ \min  _ {i \in \Omega_ {k} (f (\boldsymbol {x}))} \ell (h (x), i) \cdot p (i \in \Omega_ {k} (f (\boldsymbol {x}))) \mid f ] \\ = \mathbb {E} _ {(\boldsymbol {x}, y) \sim \mathcal {X} \times \mathcal {Y}} \left[ \min  _ {i \in \Omega_ {k} (f (\boldsymbol {x}))} \left\{\ell (h (x), i) \cdot p (i \in \Omega_ {k} (f (\boldsymbol {x})) \mid y \in \Omega_ {k} (f (\boldsymbol {x}))) \cdot p (y \in \Omega_ {k} (f (\boldsymbol {x})) \mid f) \right. \right. \\ + \ell (h (x), i) \cdot p (i \in \Omega_ {k} (f (\boldsymbol {x})) \mid y \notin \Omega_ {k} (f (\boldsymbol {x}))) \cdot p (y \notin \Omega_ {k} (f (\boldsymbol {x})) \mid f) \rbrace ]. \\ \end{array}
$$
The coefficient of  $\ell (h(x),y)$  is:
$$
\begin{array}{l} \operatorname {C o f f} [ \ell (h (x), y) ] = p (y \in \Omega_ {k} (f (\boldsymbol {x})) \mid y \in \Omega_ {k} (f (\boldsymbol {x}))) \cdot p (y \in \Omega_ {k} (f (\boldsymbol {x})) \mid f) \\ = 1 - \eta . \\ \end{array}
$$
For  $i\neq y$  ,there is
$$
\begin{array}{l} \operatorname {C o f f} [ \ell (h (x), i) ] = p (i \in \Omega_ {k} (f (\boldsymbol {x})) \mid y \in \Omega_ {k} (f (\boldsymbol {x}))) \cdot p (y \in \Omega_ {k} (f (\boldsymbol {x})) \mid f) \\ + p (i \in \Omega_ {k} (f (\boldsymbol {x})) \mid y \notin \Omega_ {k} (f (\boldsymbol {x}))) \cdot p (y \notin \Omega_ {k} (f (\boldsymbol {x})) \mid f) \\ \leq \gamma (1 - \eta) + \eta . \\ \end{array}
$$
Therefore, when  $1 - \eta >\gamma (1 - \eta) + \eta$  , i.e.,  $\gamma < 1 - \frac{\eta}{1 - \eta}$  , we have  $h^* = \underset {h\in \mathcal{H}}{\arg \min}\operatorname{Err}_{\mathcal{D}}(h|f)$
Let  $\mathbf{z}$ ,  $\mathbf{z}'$  be the training set and the testing set, and each of them is of size  $n$ . Define  $H(\mathbf{z}) = \{h \in \mathcal{H} : \operatorname{Err}_{\mathbf{z}}^{\Delta}(h|f) = 0\}$  as the set of zero-empirical-risk hypotheses. We can bound  $\operatorname*{Pr}(\mathbf{z} \in R_{n,\varepsilon} \mid f \in H_{\eta})$  with  $\operatorname*{Pr}((\mathbf{z}, \mathbf{z}') \in S_{n,\varepsilon} \mid f \in H_{\eta})$  as follows
$$
\begin{array}{l} \Pr \left(\left(\mathbf {z}, \mathbf {z} ^ {\prime}\right) \in S _ {n, \varepsilon} \mid f \in H _ {\eta}\right) \\ = \Pr \left(\left(\mathbf {z}, \mathbf {z} ^ {\prime}\right) \in S _ {n, \varepsilon} \mid \mathbf {z} \in R _ {n, \varepsilon}, f \in H _ {\eta}\right) \\ = \Pr \left(\left\{\exists h \in H _ {\varepsilon} \cap H (\mathbf {z}), \operatorname {E r r} _ {\mathbf {z} ^ {\prime}} (h | f) \geq \frac {\varepsilon}{2} \right\} \mid \mathbf {z} \in R _ {n, \varepsilon}, f \in H _ {\eta}\right) \\ \geq \Pr \left(h \in H _ {\varepsilon} \cap H (\mathbf {z}), \operatorname {E r r} _ {\mathbf {z} ^ {\prime}} (h | f) \geq \frac {\varepsilon}{2} \mid \mathbf {z} \in R _ {n, \varepsilon}, f \in H _ {\eta}\right) \\ \geq 1 - \exp \left(- \cdot \frac {\varepsilon n}{8}\right). \\ \end{array}
$$
When  $n > \frac{8\log 2}{\varepsilon}$ , we have  $\operatorname*{Pr}((\mathbf{z},\mathbf{z}^{\prime})\in S_{n,\varepsilon}\mid f\in H_{\eta})\geq \frac{1}{2}\operatorname*{Pr}(\mathbf{z}\in R_{n,\varepsilon}\mid f\in H_{\eta})$ , which completes the proof.
We follow the methodology of proving the ERM learnability in [26]. Here we need to bound  $\operatorname{Pr}(S_{n,\varepsilon} \mid f \in H_{\eta})$ . The key behind the proof is to refine  $\mathrm{Err}_{\mathbf{z}}^{\Delta}(h|f)$  and  $\mathrm{Err}_{\mathbf{z}'}^{\Delta}(h|f)$ . We use a classic method, i.e., swap, to refine the probability of a single instance. A swap  $\sigma(\mathbf{z}, \mathbf{z}') = (\mathbf{z}^{\sigma}, \mathbf{z}'^{\sigma})$  means exchanging some instances between the training set  $\mathbf{z}$  and testing set  $\mathbf{z}'$  while keeping size  $n$  unchanged. Each  $(x,y) \in \mathbf{z}$  is a training instance while each  $(x,y) \in \mathbf{z}'$  is a testing instance. There are  $2^n$  different swaps in total and we define  $G$  as the set of all swaps. Firstly, we use swap to describe  $\operatorname{Pr}(S_{n,\varepsilon} \mid f \in H_{\eta})$ .
$$
\begin{array}{l} 2 ^ {n} \Pr \left(\left(\mathbf {z}, \mathbf {z} ^ {\prime}\right) \in S _ {n, \varepsilon} \mid f \in H _ {\eta}\right) = \sum_ {\sigma \in G} \mathbb {E} \left[ \Pr \left(\left(\mathbf {z}, \mathbf {z} ^ {\prime}\right) \in S _ {n, \varepsilon} \mid x, y, x ^ {\prime}, y ^ {\prime}, f \in H _ {\eta}\right) \right] \\ = \sum_ {\sigma \in G} \mathbb {E} \left[ \Pr \left(\sigma \left(\mathbf {z}, \mathbf {z} ^ {\prime}\right) \in S _ {n, \varepsilon} \mid x, y, x ^ {\prime}, y ^ {\prime}, f \in H _ {\eta}\right) \right] \\ = \mathbb {E} \left[ \sum_ {\sigma \in G} \Pr \left(\sigma \left(\mathbf {z}, \mathbf {z} ^ {\prime}\right) \in S _ {n, \varepsilon} \mid x, y, x ^ {\prime}, y ^ {\prime}, f \in H _ {\eta}\right) \right]. \\ \end{array}
$$
Here, the expectations are with respect to  $(x,y,x^{\prime},y^{\prime})$ . To further refine  $S_{n,\varepsilon}$ , we define  $S_{n,\varepsilon}^{h}$  for a certain classifier  $h$  as
$$
S _ {n, \varepsilon} ^ {h} = \left\{\left(\mathbf {z}, \mathbf {z} ^ {\prime}\right): \operatorname {E r r} _ {\mathbf {z}} ^ {\Delta} (h | f) = 0, \operatorname {E r r} _ {\mathbf {z} ^ {\prime}} (h | f) \geq \frac {\varepsilon}{2} \right\}.
$$