THE RICH GET RICHER: DISPARATE IMPACT OF SEMI-SUPERVISED LEARNING
Year: 2021

Abstract:
Semi-supervised learning (SSL) has demonstrated its potential to improve the model accuracy for a variety of learning tasks when the high-quality supervised data is severely limited. Although it is often established that the average accuracy for the entire population of data is improved, it is unclear how SSL fares with different sub-populations. Understanding the above question has substantial fairness implications when these different sub-populations are defined by the demographic groups we aim to treat fairly. In this paper, we reveal the disparate impacts of deploying SSL: the sub-population who has a higher baseline accuracy without using SSL (the “rich” sub-population) tends to benefit more from SSL; while the subpopulation who suffers from a low baseline accuracy (the “poor” sub-population) might even observe a performance drop after adding the SSL module. We theoretically and empirically establish the above observation for a broad family of SSL algorithms, which either explicitly or implicitly use an auxiliary “pseudo-label”. Our experiments on a set of image and text classification tasks confirm our claims. We discuss how this disparate impact can be mitigated and hope that our paper will alarm the potential pitfall of using SSL and encourage a multifaceted evaluation of future SSL algorithms.

heading: 1 INTRODUCTION
text: The success of deep neural networks benefits from large-size datasets with high-quality supervisions, while the collection of them may be difficult due to the high cost of data labeling process e.g., medical diagnosis (Agarwal et al., 2016). Practically, a much cheaper and easier solution is to obtain a small labeled dataset and a large unlabeled dataset, and apply semi-supervised learning (SSL). Although the global model performance for the entire population of data is almost always improved by SSL, it is unclear how the improvements fare for different sub-populations. Similar to the goal of multiaccuracy (Kim et al., 2019), the improvement of SSL should not only on the entire population, but also on different sub-populations. Analyzing and understanding the above question will have substantial fairness implications especially when these sub-populations are defined by the demographic groups e.g., race and gender.
Motivating Example Figure 1 shows the change of test accuracy for each class during SSL. Each class denotes one sub-population and representative sub-populations are highlighted. Figure 1 shows two important messages: in SSL, even with some state-of-the-art algorithms, 1) the “rich getting richer” is common and 2) the “poor getting poorer” possibly happens. Specifically, the “rich” subpopulation, such as automobile that has a high baseline accuracy at the beginning of SSL, tends to be consistently benefiting during SSL. But the “poor” sub-population, such as dog that has a low baseline accuracy, will remain a low-level performance as Figure 1(a) or even get worse performance as Figure 1(b). This example shows disparate impact of accuracies for different sub-populations are common in SSL. In extreme cases such as Figure 1(b), we observe the Matthew effect: the rich get richer and the poor get poorer. See more discussions in Appendix B.
In this paper, we aim to understand the disparate impact of SSL from both theoretical and empirical aspects, and propose to evaluate SSL from a different dimension. Specifically, based on classifications tasks, we study the disparate impact of model accuracies with respect to different sub-populations (such as label classes, feature groups and demographic groups) after applying SSL. Different from traditional group fairness (Zafar et al., 2017; Bellamy et al., 2018) defined over one
model, our study focuses on comparing the gap between two models (before and after SSL). To this end, we first theoretically analyze why and how disparate impact are generated. The theoretical results motivate us to further propose a new metric, benefit ratio, which evaluates the normalized improvement of model accuracy using SSL methods for different sub-populations. The benefit ratio helps us uncover the “Matthew effect” of SSL: a high baseline accuracy tends to reach a high benefit ratio that may even be larger than 1 (the rich get richer), and a sufficiently low baseline accuracy may return a negative benefit ratio (the poor get poorer). Following the intuitions of Equal Odds (Hardt et al., 2016), a fair SSL algorithm should be with Equal Benefit Ratio. We then numerically demonstrate using benefit ratios to evaluate SSL and discuss how two possible treatments, i.e., balancing the data and collecting more labeled data, might mitigate the disparate impact. We hope our analyses and discussions could encourage future contributions to promote the fairness of SSL.
Our main contributions and findings are summarized as follows: 1) We propose an analytical framework that unifies a broad family of SSL algorithms, which either explicitly or implicitly use an auxiliary “pseudo-label”. The framework provides a novel perspective for analyzing SSL by connecting consistency regularization to the problem of learning with noisy labels. 2) Based on our framework, we further prove an upper bound for the generalization error of SSL and theoretically show the heterogeneous error of learning with the smaller scale supervised dataset is one primary reason for disparate impacts. This observation reveals a “Matthew effect” of SSL. 3) We contribute a novel metric called benefit ratio to measure the disparate impact in SSL, the effectiveness of which is theoretically guaranteed by our proved generalization bounds. 4) We conduct experiments on both image and text classification tasks to demonstrate the ubiquitous disparate impacts in SSL. 5) We also discuss how the disparate impact could be mitigated.

heading: 1.1 RELATED WORKS
text: Semi-supervised learning SSL is popular in both computer vision and natural language processing communities (Dai & Le, 2015; Howard & Ruder, 2018; Clark et al., 2018; Yang et al., 2019; Sachan et al., 2019; Guo et al., 2020; Wang et al., 2021b; Luo et al., 2021). We briefly review recent advances in SSL. See comprehensive overviews by (Chapelle et al., 2006; Zhu et al., 2003) for traditional SSL methods. A major line of recent works focus on assigning pseudo-labels generated by the supervised model to unlabeled dataset (Lee et al., 2013; Iscen et al., 2019; Berthelot et al., 2019b;a), where the pseudo-labels are often confident or with low-entropy (Sohn et al., 2020; Zhou, 2018; Meng et al., 2018). There are also many works on minimizing entropy of predictions on unsupervised data (Grandvalet et al., 2005) or regularizing the model consistency on the same feature with different data augmentations (Tarvainen & Valpola, 2017; Miyato et al., 2018; Sajjadi et al., 2016; Clark et al., 2018), including mixup (Zhang et al., 2018), adversarial noise (Miyato et al., 2016), and other advanced augmentations (Xie et al., 2019). In addition to network inputs, augmentations can also be applied on some hidden layers (Chen et al., 2020). Besides, some works conduct pre-training on the unlabeled dataset first and then fine-tune the model on the labeled dataset (Peters et al., 2018; Devlin et al., 2018; Gururangan et al., 2019; Chen et al., 2019; Yang et al., 2017), or use ladder networks to combine unsupervised learning with supervised learning (Rasmus et al., 2015).
Disparate impact Over the past few years, the fairness in machine learning has been extensively studied. Even models developed with the best intentions may introduce discriminatory biases. Re-
searchers in various fields have found the unfairness issues, e.g., model compression (Bagdasaryan et al., 2019), differential privacy (Hooker et al., 2019; 2020), recommendation system (Gómez et al., 2021), information retrieval (Gao & Shah, 2021), machine translation (Khan et al., 2021), and learning with noisy labels (Liu, 2021). There also some treatments considering fairness without demographics (Lahoti et al., 2020; Diana et al., 2021; Hashimoto et al., 2018), minimax Pareto fairness (Martinez et al., 2020), multiaccuracy boosting (Kim et al., 2019), and fair classification with label noise (Wang et al., 2021a). Most of these works focus on supervised learning. To our best knowledge, the unfairness issue in SSL methods has not been sufficiently explored.

heading: 2 PRELIMINARIES
text: We summarize the key concepts and notations as follows.

heading: 2.1 SUPERVISED CLASSIFICATION TASKS
text: Consider aK-class classification task given a set ofNL labeled training examples denoted byDL := {(xn, yn)}n∈[NL], where [NL] := {1, 2, · · · , NL} is the set of indices, xn is a high-dimension feature, yn ∈ [K] stands for a particular label class. The corresponding clean data distribution with full supervision is denoted by D. Examples (xn, yn) are drawn according to random variables (X,Y ) ∼ D. The classification task aims to identify a classifier f that maps X to Y accurately. Let 1{·} be the indicator function taking value 1 when the specified condition is satisfied and 0 otherwise. Define the 0-1 loss as 1(f(X), Y ) := 1{f(X) 6= Y }. The optimal f is denoted by the Bayes classifier f∗ = arg minf ED[1(f(X), Y )]. One common choice is training a deep neural network (DNN) by minimizing the empirical risk: f̂ = arg minf 1 N ∑N n=1 `(f(xn), yn). Notation `(·) stands for the cross-entropy (CE) loss `(f(x), y) := − ln(fx[y]), y ∈ [K], where fx[y] denotes the y-th component of f(x). Notations f and f stand for the same model but different outputs. Specifically, vector f(x) denotes the probability of each class that model f predicts given feature x. The predicted label f(x) is the class with maximal probability, i.e., f(x) := arg maxi∈[K] fx[i]. We use notation f if we only refer to a model.

heading: 2.2 SEMI-SUPERVISED CLASSIFICATION TASKS
text: In the semi-supervised learning (SSL) task, there is also an unlabeled (a.k.a. unsupervised) dataset DU := {(xn+NL , ·)}n∈[NU ] drawn from D, while the labels are missing or unobservable. Let N := NL + NU . Denote the corresponding unobservable supervised dataset by D := {(xn, yn)}n∈[N ]. Compared with the supervised learning tasks, it is critical to leverage the unsupervised information in semi-supervised learning. To improve the model generalization ability, many recent SSL methods rely on building consistency regularization terms with unlabeled data to ensure that the model output remains unchanged with randomly augmented inputs (Berthelot et al., 2019b; Xie et al., 2019; Berthelot et al., 2019a; Sohn et al., 2020; Xu et al., 2021). To better formulate the consistency regularization in these popular methods, we introduce soft/pseudo-labels y as follows.
Soft-labels To motivate the definitions, we first consider a special case yn, i.e., the one-hot encoding of yn, where each element writes as yn[i] = 1{i = yn}. More generally, we can extend the one-hot encoding to soft labels by requiring each element y[i] ∈ [0, 1] and ∑ i∈[K] y[i] = 1. The CE loss
with soft y writes as `(f(x),y) := − ∑ i∈[K] y[i] ln(fx[i]). If we interpret y[i] = P(Y = i) as a probability and denote by Dy the corresponding label distribution, the above CE loss with soft labels can be interpreted as the expected loss with respect to a stochastic label Ỹ , i.e.,
`(f(x),y) := ∑ i∈[K] P(Ỹ = i)`(f(x), i) = EỸ∼Dy [ `(f(x), Ỹ ) ] . (1)
Pseudo-labels In consistency regularization, by using model predictions, the unlabeled data will be assigned pseudo-labels either explicitly (Berthelot et al., 2019b) or implicitly (Xie et al., 2019), where the pseudo-labels can be modeled as soft-labels. In the following, we review both the explicit and the implicit approaches and unify them in our analytical framework.
Consistency regularization with explicit pseudo-labels For each unlabeled feature xn, pseudolabels can be explicitly generated based on model predictions (Berthelot et al., 2019b). The pseudolabel is later used to evaluate the model predictions. To avoid trivial solutions where model predictions and pseudo-labels are always identical, independent data augmentations of feature xn are often generated for M rounds. The augmented feature is denoted by x′n,m := Augment(xn),m ∈ [M ]. Then the pseudo-label yn in epoch-t can be determined based on M model predictions as y (t) n = Sharpen ( 1 M ∑M m=1 f̄ (t)(x′n,m) ) , where model f̄ (t) is a copy of the DNN at the beginning of epoch-t but without gradients. The function Sharpen(·) reduces the entropy of a pseudo-label, e.g., setting to one-hot encoding ej , j = arg maxi∈[K] y (t) n (Sohn et al., 2020). In epoch-t, with some consistency regularization loss `CR(·), the empirical risk using pseudo-labels is:
L1(f,DL, DU ) = 1
NL NL∑ n=1 `(f(xn), yn) + 1 NU NL+NU∑ n=NL+1 `CR(f(xn),y (t) n ).
Consistency regularization with implicit pseudo-labels Consistency regularization can also be applied without specifying particular pseudo-labels, where a divergence metric between predictions on the original feature and the augmented feature is minimized to make predictions consistent. For example, the KL-divergence could be applied and the data augmentation could be domain-specific (Xie et al., 2019) or adversarial (Miyato et al., 2018). In epoch-t, the total loss is:
L2(f,DL, DU ) = 1
NL NL∑ n=1 `(f(xn), yn) + λ · 1 NU NL+NU∑ n=NL+1 `CR(f(xn), f̄ (t)(x′n)),
where λ balances the supervised loss and the unsupervised loss, x′n := Augment(xn) stands for oneround data augmentation (m = 1 following the previous notation x′n,m). Without loss of generality, we use λ = 1 in our analytical framework.
Consistency regularization loss `CR(·) In the above two lines of works, there are different choices of `CR(·), such as mean-squared error `CR(f(x),y) := ‖f(x) − y‖22/K (Berthelot et al., 2019b; Laine & Aila, 2016) or CE loss (Miyato et al., 2018; Xie et al., 2019) defined in Eq. (1). For a clean analytical framework, we consider the case when both supervised loss and unsupervised loss are the same, i.e., `CR(f(x),y) = `(f(x),y). Note L2 implies the entropy minimization (Grandvalet et al., 2005) when both loss functions are CE and there is no augmentation, i.e., x′n = xn.

heading: 2.3 ANALYTICAL FRAMEWORK
text: We propose the following analytical framework to unify both the explicit and the implicit approaches. With Eqn. (1), the unsupervised loss term in the above two methods can be respectively written as:
1
NU NL+NU∑ n=NL+1 EỸ∼D y (t) n [ `(f(xn), Ỹ ) ] and 1 NU NL+NU∑ n=NL+1 EỸ∼D f̄(t)(x′n) [ `(f(xn), Ỹ ) ] .
Both unsupervised loss terms inform us that, for each feature xn, we compute the loss with respect to the reference label Ỹ , which is a random variable following distributionD
y (t) n orDf̄(t)(x′n). Compared with the original clean label Y , the unsupervised reference label Ỹ contains label noise, and the noise transition depends on feature xn. Specifically, we have
P(Ỹ = i|X = xn) = y(t)n [i] (Explicit) or P(Ỹ = i|X = xn) = f̄ (t) x′n [i] (Implicit).
Then with model f̄ (t), we can define a new dataset with instance-dependent noisy reference labels: D̃ = {(xn, ỹn)}n∈[N ], where ỹn = yn,∀n ∈ [NL], and ỹn = y (t) n or f̄
(t) x′n , ∀n ∈ {NL+1, · · · , N}. The unified loss is:
L(f, D̃) = 1
N N∑ n=1 `(f(xn), ỹn) = 1 N N∑ n=1 EỸ∼Dỹn [ `(f(xn), Ỹ ) ] . (2)
Therefore, with the pseudo-label as a bridge, we can model the popular SSL solution with consistency regularization as a problem of learning with noisy labels (in the challenging instancedependent label noise setting).

heading: 3 DISPARATE IMPACTS OF SSL
text: To understand the disparate impact of SSL, we study how supervised learning with DL affects the performance of SSL with both DL and DU .
Intuition The rich sub-population with a higher baseline accuracy (from supervised learning with DL) will have higher-quality pseudo-labels for consistency regularization, which helps to further improve the performance. In contrast, the poorer sub-population with a lower baseline accuracy (again from supervised learning with DL) can only have lower-quality pseudo-labels to regularize the consistency of unsupervised features. With a wrong regularization direction, the unsupervised feature will have its augmented copies reach consensus on a wrong label class, which leads to a performance drop. Therefore, when the baseline accuracy is getting worse, there will be more and more unsupervised features that are wrongly regularized, resulting in disparate impacts of model accuracies as shown in Figure 1.
In this section, we first analyze the generalization error for supervised learning with labeled data, then extend the analyses to semi-supervised learning with the help of pseudo-labels. Without loss of generality, we consider minimizing 0-1 loss 1(f(X), Y ) with infinite search space. Our analyses can be generalized to bounded loss `(·) and finite function space F following the generalization bounds that can be introduced using Rademacher complexity (Bartlett & Mendelson, 2002).

heading: 3.1 LEARNING WITH CLEAN DATA
text: Denote the expected error rate of classifier f on distribution D by RD(f) := ED[1(f(X), Y )]. Let f̂D denote the classifier trained by minimizing 0-1 loss with clean dataset D, i.e., f̂D := arg minf R̂D(f), where R̂D(f) := 1 N ∑ n∈[N ] 1(f(xn), yn). Denote by Y
∗|X := arg maxi∈[K] P(Y |X) the Bayes optimal label on clean distribution D. Theorem 1 shows the generalization bound in the clean case. Replacing D and N with DL and NL we have: Theorem 1 (Supervised error). With probability at least 1−δ, the generalization error of supervised learning on clean dataset DL is upper-bounded by RD(f̂DL) ≤ √ 2 log(2/δ) NL + P(Y ∗ 6= Y ).

heading: 3.2 LEARNING WITH SEMI-SUPERVISED DATA
text: We further derive generalization bounds for learning with semi-supervised data. Following our analytical framework in Section 2.3, in each epoch-t, we can transform the semi-supervised data to supervised data with noisy supervisions by assigning pseudo-labels, where the semi-supervised dataset DL ∪DU is converted to the dataset D̃ given the model learned from the previous epoch. Two-iteration scenario In our theoretical analyses, to find a clean-structured performance bound for learning with semi-supervised data and highlight the impact of the model learned from supervised data, we consider a particular two-iteration scenario where the model is first trained to convergence with the small labeled dataset DL and get f̂DL , then trained on the pseudo noisy dataset D̃ labeled by f̂DL . Noting assigning pseudo labels iteratively may improve the performance as suggested by most SSL algorithms (Berthelot et al., 2019b; Xie et al., 2019), our considered twoiteration scenario can be seen as a worst case for an SSL algorithm.
Independence of samples in D̃ Recall x′n denotes the augmented copy of xn. The N instances in D̃ may not be independent since pseudo-label ỹn comes from f̂DL(x ′ n), which depends on xn. Namely, the number of independent instancesN ′ should be in the range of [NL, N ]. Intuitively, with appropriate noise injection or data augmentation (Xie et al., 2019; Miyato et al., 2018) to xn such that x′n could be treated as independent of xn, the number of independent samples in D̃ could be improved toN . We consider the ideal case where allN instances are i.i.d. in the following analyses.
By minimizing the unified loss defined in Eq. (2), we can get classifier f̂D̃ := arg minf R̂D̃(f),
where R̂D̃(f) := 1 N ∑ n∈[N ] (∑ i∈[K] ỹ[i] · 1(f(xn), i) ) . The expected error given classifier f is
denoted by RD̃(f) := ED̃[1(f(X), Ỹ )], where the probability density function of distribution D̃ can be defined as P(X,Ỹ )∼D̃(X = xn, Ỹ = i) = P(X,Y )∼D(X = xn) · ỹn[i].
Decomposition With the above definitions, the generalization error (on the clean distribution) of classifier f̂D̃ could be decomposed as RD(f̂D̃) = (RD(f̂D̃)−RD̃(f̂D̃))︸ ︷︷ ︸
Term-1
+RD̃(f̂D̃)︸ ︷︷ ︸ Term-2 , where Term-1
transforms the evaluation of f̂D̃ from clean distributionD to the pseudo noisy distribution D̃. Term2 is similar to the generalization error in Theorem 1 but the model is trained and evaluated on noisy distribution D̃. Both terms are analyzed as follows.
Upper and Lower Bounds for Term-1 Let η(X) := 12 ∑ i∈[K] |P(Ỹ = i|X) − P(Y = i|X)|, e(X) := P(Y 6= Ỹ |X) be the feature-dependent error rate, Ãf (X) := P(f(X) = Ỹ |X) be the accuracy of prediction f(X) on noisy dataset D̃. Denote their expectations (over X) by η̄ := EX [η(X)], ē := EX [e(X)], Ãf = EX [Ãf (X)]. To highlight that η̄ and ē depends on the noisy dataset D̃ labeled by f̂DL , we denote them as η̄(f̂DL) and ē(f̂DL). Then we have:
Lemma 1 (Bounds for Term-1). (2Ãf̂ D̃ − 1)ē(f̂DL) ≤ RD(f̂D̃)−RD̃(f̂D̃) ≤ η̄(f̂DL).
Note the upper bound builds on η̄(f̂DL) while the lower bound relates to ē(f̂DL). To compare two bounds and show the tightness, we consider the case where Y |X is confident, i.e., each feature X belongs to one particular true class Y with probability 1, which is generally held in classification problems (Liu & Tao, 2015). Lemma 2 shows η(X) = e(X) in this particular case1. Lemma 2 (η vs. e). For any feature X , if Y |X is confident, η(X) is the error rate of the model prediction on X , i.e., ∃i ∈ [K] : P(Y = i|X) = 1⇒ η(X) = P(Ỹ 6= Y |X) = e(X).
Upper bound for Term-2 Denote by Ỹ ∗|X := arg maxi∈[K] P(Ỹ = i|X) the Bayes optimal label on noisy distribution D̃. Following the proof for Theorem 1, we have:
Lemma 3 (Bound for Term-2). W. p. at least 1− δ, RD̃(f̂D̃)(fD̃) ≤ √ 2 log(2/δ) N + P(Ỹ 6= Ỹ ∗).
Wrap-up Lemma 1 shows Term-1 is in the range of [(2Ãf̂ D̃ −1)ē(f̂DL), η̄(f̂DL)]. Lemma 2 informs us η̄(f̂DL) = ē(f̂DL) in classification problems where Y |X,∀X are confident. With a well-trained model f̂D̃ that learns the noisy distribution D̃, we have Ãf̂D̃ = 1− and → 0+, thus Term-1 is in the range of [(1− 2 )η̄(f̂DL), η̄(f̂DL)], indicating our bounds for Term-1 are tight. Besides, noting Lemma 3 is derived following the same techniques as Theorem 1, we know both bounds have similar tightness. Therefore, by adding upper bounds for Term-1 and Term-2, we can upper bound the error of semi-supervised learning in Theorem 2, which has similar tightness to that in Theorem 1. Theorem 2 (Semi-supervised learning error). Suppose the model trained with only DL has generalization error η̄′(f̂DL). With probability at least 1− δ, the generalization error of semi-supervised learning on datasets DL ∪DU is upper-bounded by
RD(f̂DL∪DU ) ≤ η̄(f̂DL)︸ ︷︷ ︸ Disparity due to baseline + P(Ỹ 6= Ỹ ∗)︸ ︷︷ ︸ Sharpness of pseudo labels +
√ 2 log(2/δ) N︸ ︷︷ ︸ Data dependency ,
where η̄(f̂DL) := η̄ ′(f̂DL) ·NU/N is the expected label error in the pseudo noisy dataset D̃.
Takeaways Theorem 2 explains how disparate impacts in SSL are generated.
• Supervised error η̄′(f̂DL): the major source of disparity. The sub-population that generalizes well before SSL tends to have a lower SSL error. Namely, the rich get richer.
• Sharpness of noisy labels P(Ỹ 6= Ỹ ∗): a minor source of disparity depends on how one processes pseudo-labels. This term is negligible if we sharpen the pseudo-label.
• Sample complexity √
2 log(2/δ)/N : disparity depends on the number of instances N and their independence. Recall we assume ideal data augmentations to get N in this term. There will be much less than N i.i.d. instances with poor data augmentations. It would be a major source of disparity if data augmentations are poor. 1We can also prove η(X) = e(X) when Ỹ |X is confident. See more details in the proof.

heading: 4 BENEFIT RATIO: A EVALUATION METRIC
text: Figure 1 demonstrates that SSL leads to disparate impacts of different sub-populations’ accuracies, but it is still not clear how much that SSL benefits a certain sub-population. For example, a subpopulation with a low test accuracy after SSL may be benefited much less from SSL since it is very hard to learn. On the other hand, a sub-population with a high test accuracy after SSL may not get benefits from SSL since it is easy to learn and the baseline accuracy is high enough. To highlight the benefit and evaluate the disparate impacts of SSL, we propose a new metric benefit ratio.
Benefit Ratio We now introduce the benefit ratio, BR(P), which depends on three classifiers: (baseline) supervised learning only with labeled data f̂DL , (ideal) fully-supervised learning f̂D, and SSL f̂DL∪DU , to capture the relative accuracy improvement of sub-population P after SSL. Note BR(P) are defined with three learning results: 1) the baseline accuracy from supervised learning with only a small labeled dataset, denoted by abaseline(P); 2) the accuracy from semi-supervised learning with both labeled dataset and unsupervised dataset, denoted by asemi(P); 3) the accuracy from fully-supervised learning with the ground truth labeled dataset, denoted by aideal(P). Then:
BR(P) = asemi(P)− abaseline(P) aideal(P)− abaseline(P) .
Intuitively, a larger benefit ratio indicates more benefits from SSL. We have BR(P) = 1 when SSL performs as well as the corresponding fully-supervised learning. A negative benefit ratio indicates the poor population is hurt by SSL such that asemi(P) < abaseline(P), i.e., the poor get poorer as shown in Figure 1(b) (sub-population dog). Therefore, compared with the test accuracy, benefit ratio shows more advantages in evaluating the disparate impacts of SSL.
Theoretical Explanation Recall we have error upper bounds for both supervised learning (Theorem 1) and semi-supervised learning (Theorem 2). Both bounds have similar tightness thus we can compare them and get a proxy of benefit ratio as B̂R(P) := sup (RD(f̂DL∪DU |P))−sup (RD(f̂DL|P)) sup(RD(f̂DL∪DU |P))−sup(RD(f̂D|P)) , where sup(·) denotes the upper bound derived in Theorem 1 and Theorem 2, P is a sub-population, and D|P denotes the set of i.i.d. instances in D that affect model generalization on P . By assuming P(Y 6= Y ∗) = P(Ỹ 6= Ỹ ∗) (both distributions have the same sharpness), we have:
Corollary 1. The benefit ratio proxy for P is B̂R(P) = 1 − η̄(f̂DL|P)∆(NP ,NPL ) , where ∆(NP , NPL) =√ 2 log(2/δ) NPL − √ 2 log(2/δ) NP , NP and NPL are the numbers of instances in D|P and DL|P .
Corollary 1 shows the benefit ratio is negatively related to the error rate of baseline models and positively related to the number of i.i.d. instances after in SSL, which is consistent with our takeaways from Section 3. It also informs us that SSL may have a negative effect on sub-population P if η∆(NP ,NPL )
> 1, i.e., the benefit from getting more effective i.i.d. instances is less than the harm from wrong pseudo-labels. This negative effect indicates ”the poor get poorer”.

heading: 5 EXPERIMENTS
text: In this section, we first show the existence of disparate impact on several representative SSL methods and then discuss the possible treatment methods to mitigate this disparate impact.
Settings Three representative SSL methods, i.e., MixMatch (Berthelot et al., 2019b), UDA (Xie et al., 2019), and MixText (Chen et al., 2020), are tested on several image and text classification tasks. For image classification, we experiment on CIFAR-10 and CIFAR-100 datasets (Krizhevsky et al., 2009). We adopt the coarse labels (20 classes) in CIFAR-100 for training and test the performance for each fine label (100 classes). Thus our training on CIFAR-100 is a 20-class classification task and each coarse class contains 5 sub-populations. For text classification, we employ three datasets: Yahoo! Answers (Chang et al., 2008), AG News (Zhang et al., 2015) and Jigsaw Toxicity (Kaggle, 2018). Jigsaw Toxicity dataset contains both classification labels (one text comment is toxic or non-toxic) and a variety of sensitive attributes (e.g., race and gender information) of the identities that are mentioned in the text comment, which are fairness concerns in real-world applications. We defer more experimental details in Appendix C.

heading: 5.1 DISPARATE IMPACT EXISTS IN POPULAR SSL METHODS
text: We show that, even though the size of each sub-population is equal, disparate impacts exist in the model accuracy of different sub-populations, i.e., 1) explicit sub-populations such as classification labels in Figure 2 (and Figure 4 in Appendix C.2), and 2) implicit sub-populations such as finecategories under coarse classification labels in Figure 3a, demographic groups including race in Figure 3b and gender (Figure 5 in Appendix C.3). All the experiments in this subsection adopt both a balanced labeled dataset and a balanced unlabeled dataset. Note we sample a balanced subset from the raw (unbalanced) Jigsaw dataset. Other datasets are originally balanced.
Disparate impact across explicit sub-populations In this part, we show the disparate impact on model accuracy across different classification labels on CIFAR-10, Yahoo! Answers, and AG News datasets. Detailed results on AG News can be found in Appendix C.2. In Figure 2a and 2b, we show the benefit ratios (y-axis) versus baseline accuracies before SSL (x-axis). From left to right, we show results with different sizes of labeled data: 25 per class to 50 on CIFAR-10 and 100 per class to 200 on Yahoo! Answers. Figure 2a and 2b utilized two SSL methods respectively (CIFAR-10: MixMatch & UDA; Yahoo! Answers: MixText & UDA).We statistically observe that the class labels with higher baseline accuracies have higher benefit ratios on both CIFAR-10 and Yahoo! Answers datasets. It means that “richer” classes benefit more from applying SSL methods than the “poorer” ones. We also observe that for some models with low baseline accuracy (left side of the x-axis), applying SSL results in rather low benefit ratios that are close to 0.
Disparate impact across implicit sub-populations We demonstrate the disparate impacts on model accuracy across different sub-populations on CIFAR-100 (fine-labels) and Jigsaw (race and gender) datasets. In Figure 3, we can statistically observe the disparate impact across different subpopulations on both datasets for three baseline SSL methods. Detailed results on Jigsaw with gender are shown in Appendix C.3. We again observe very similar disparate improvements as presented in Figure 2a and 2b - for some classes in CIFAR-100, this ratio can even go negative. Note the disparate impact on the demographic groups in Jigsaw raises fairness concerns in real-world applications.

heading: 5.2 MITIGATING DISPARATE IMPACT
text: Our analyses in Section 3 and Section 4 indicate the disparate impacts may be mitigated by balancing the supervised error η̄(f̂DL|P) and the number of effective i.i.d. instances for different populations. We perform preliminary tests to check the effectiveness of the above findings in this subsection. We hope our analyses and experiments could inspire future contributions to disparity mitigation in SSL.
Balancing and Collecting more labeled data We firstly sample 400 i.i.d. instances from the raw (unbalanced) Jigsaw dataset and get the setting of Unbalanced (400). To balance the supervised error and effective instances for different sub-populations, an intuitive method is to balance the labeled data by reweighting, e.g., if the size of two sub-populations is 1:2, we simply sample instances from two sub-populations with a probability ratio of 2:1 to ensure all sub-population have the same weights in each epoch during training. Balance labeled (400) denotes only 400 labeled instances are reweighted. Balance both (400) means both labeled and unlabeled instances are balanced. Table 1 shows a detailed comparison on the benefit ratios between different race identities of the Jigsaw dataset and their standard deviations, where the first three rows denote the above three settings. We can observe that both the standard deviation and the number of negative benefit ratios become lower with more balanced settings, which demonstrates the effectiveness of the balancing treatment strategy, although there still exists a sub-population that has a negative benefit ratio, indicating an unfair learning result. To further improve fairness, as suggested in our analyses, we “collect” (rather add) another 400 i.i.d. labeled instances (800 in total) from the Jigsaw dataset, and show the result after balancing both labeled and unlabeled data in the last row of Table 3 (Balance both (800)). Both the standard deviation and the number of negative benefit ratios can be further reduced with more labeled data. More experimental results are on CIFAR-100 and Jigsaw (gender) datasets shown in Appendix C.4 (balancing) and Appendix C.5 (more data).

heading: 6 CONCLUSIONS
text: We have theoretically and empirically shown that the disparate impact (the “richer” sub-populations get richer and the poor ones get poorer) exists universally for a broad family of SSL algorithms. We have also proposed and studied a new metric benefit ratio to facilitate the evaluation of SSL. We hope our work could inspire more efforts towards mitigating the disparate impact and encourage a multifaceted evaluation of existing and future SSL algorithms.

heading: 7 ETHICS STATEMENT
text: In recent years, machine learning methods are widely applied in the real world across a broad variety of fields such as face recognition, recommendation system, information retrieval, medical diagnosis, and marketing decision. The trend is likely to continue growing. However, it is noted that these machine learning methods are susceptible to introducing unfair treatments and can lead to systematic discrimination against some demographic groups. Our paper echoes this effort and looks into the disparate model improvement resulted from deploying a semi-supervised learning (SSL) algorithm.
Our work uncovers the unfairness issues in semi-supervised learning methods. More specifically, we theoretically and empirically show the disparate impact exists in the recent SSL methods on several public image and text classification datasets. Our results provide understandings of the potential disparate impacts of an SSL algorithm and help raise awareness from a fairness perspective when deploying an SSL model in real-world applications. Our newly proposed metric benefit ratio contributes to the literature a new measure to evaluate the fairness of SSL. Furthermore, we discuss two possible treatment methods: balancing and collecting more labeled data to mitigate the disparate impact in SSL methods. We have carefully studied and presented their effectiveness in mitigating unfairness in the paper. Our experimental results, such as Table 1, demonstrate the disparate impacts across demographic groups (e.g., gender, race) naturally exist in SSL. We are not aware of the misuse of our proposals, but we are open to discussions.

heading: Appendix
text: The Appendix is organized as follows.
• Section A presents the detailed derivations for generalization bounds.
– Section A.1 proves the upper bound for Term-1. – Section A.2 proves the lower bound for Term-1. – Section A.2 proves the upper bound for Term-2 (also for Theorem 1). – Section A.4 proves Lemma 2. – Section A.5 proves Corollary 1.
• Section C presents more experimental settings and results.

heading: A THEORETICAL RESULTS

heading: A.1 TERM-1 UPPER BOUND
text: RD(f)−RD̃(f)
= ∫ X ( P(f(X) 6= Y |X)− P(f(X) 6= Ỹ |X) ) P(X)
= ∫ X ( P(f(X) = Ỹ |X)− P(f(X) = Y |X) ) P(X)
≤1 2 ∫ X (∣∣∣P(f(X) = Ỹ |X)− P(f(X) = Y |X)∣∣∣+ ∣∣∣P(f(X) 6= Ỹ |X)− P(f(X) 6= Y |X)∣∣∣)P(X) (a) =
∫ X TD(Ỹf (X)||Yf (X))P(X)
(b) ≤ ∫ X TD(Ỹ (X)||Y (X))P(X)
= 1
2 ∫ X ∑ i∈[K] ∣∣∣P(Ỹ = i|X)− P(Y = i|X)∣∣∣P(X) =
∫ X η(X)P(X)
=η
where (a) holds due to the definition of total variation of two distributions, i.e.,
TD(P ||Q) := 1 2
∫ |dP − dQ|.
Inequality (b) holds due to the data processing inequality. η(X) is the accuracy of labels on feature X (in some cases).

heading: A.2 TERM-1 LOWER BOUND
text: Let e(X) := P(Y 6= Ỹ |X) be the feature-dependent error rate, Ãf (X) := P(f(X) = Ỹ |X) be the accuracy of prediction f(X) on noisy dataset D̃. Denote their expectations (over X) by ē := EX [e(X)], Ãf = EX [Ãf (X)]. We have:
RD(f)−RD̃(f)
= ∫ X ( P(f(X) 6= Y |X)− P(f(X) 6= Ỹ |X) ) P(X)
= ∫ X ( P(f(X) = Ỹ |X)− P(f(X) = Y |X) ) P(X)
= ∫ X ( P(f(X) = Ỹ |f(X) = Ỹ , X)− P(f(X) = Y |f(X) = Ỹ , X) ) P(f(X) = Ỹ |X)P(X)
+ ∫ X ( P(f(X) = Ỹ |f(X) 6= Ỹ , X)− P(f(X) = Y |f(X) 6= Ỹ , X) ) P(f(X) 6= Ỹ |X)P(X)
= ∫ X ( 1− P(Ỹ = Y |X) ) P(f(X) = Ỹ |X)P(X)
+ ∫ X ( 0− P(f(X) = Y |f(X) 6= Ỹ , X) ) P(f(X) 6= Ỹ |X)P(X)
= ∫ X e(X)Ãf (X)P(X)
− ∫ X ( P(f(X) = Y |f(X) 6= Ỹ , Y = Ỹ , X)P(Y = Ỹ |f(X) 6= Ỹ , X) ) P(f(X) 6= Ỹ |X)P(X)
− ∫ X ( P(f(X) = Y |f(X) 6= Ỹ , Y 6= Ỹ , X)P(Y 6= Ỹ |f(X) 6= Ỹ , X) ) P(f(X) 6= Ỹ |X)P(X)
≥ ∫ X e(X)Ãf (X)P(X)
− ∫ X ( 0 · P(Y = Ỹ |f(X) 6= Ỹ , X) ) P(f(X) 6= Ỹ |X)P(X)
− ∫ X ( 1 · P(Y 6= Ỹ |f(X) 6= Ỹ , X) ) P(f(X) 6= Ỹ |X)P(X)
= ∫ X e(X)Ãf (X)P(X)− ∫ X P(Y 6= Ỹ |X)P(f(X) 6= Ỹ |X)P(X)
= ∫ X e(X)(2Ãf (X)− 1)P(X)
=(2Ãf − 1)e.

heading: A.3 TERM-2 UPPER BOUND
text: We adopt the same technique to prove Theorem 1 and Lemma 3. The proof follows Liu & Guo (2020). We prove Theorem 1 as follows.
Denote the expected error rate of classifier f on distribution D by
RD(f) := ED[1(f(X), Y )].
Let f̂D denote the classifier trained by minimizing 0-1 loss with clean dataset D, i.e.,
f̂D := arg min f R̂D(f),
where
R̂D(f) := 1
N ∑ n∈[N ] 1(f(xn), yn).
The Bayes optimal classifier is denoted by
f∗D := arg min f RD(f).
The expected error given by f∗D is written as
R∗ := RD(f ∗ D).
Denote by Y ∗|X := arg maxi∈[K] P(Y |X) the Bayes optimal label on clean distribution D. With probability at least 1− δ, we have:
RD(f̂D)−RD(f∗D) =R̂D(f̂D)− R̂D(f∗D) + ( RD(f̂D)− R̂D(f̂D) ) + ( R̂D(f ∗ D)−RD(f∗D) ) (a) ≤0 + 2 max f∈F |R̂D(f)−RD(f)|
(b) ≤ √ 2 log(2/δ)
N ,
where inequality (a) holds since 1) R̂D(f̂D) achieves the minimum empirical risk according to its definition, thus R̂D(f̂D) − R̂D(f∗D) ≤ 0; 2) each of the following two terms will be no greater than the corresponding maximal gap maxf∈F |R̂D(f) − RD(f)|. Inequality (b) holds due to the Hoeffding’s inequality, i.e., with probability at least 1− δ,
max f∈F |R̂D(f)−RD(f)| ≤
√ log(2/δ)
2N .
Noting RD(f∗D) = P(Y 6= Y ∗), we can prove Theorem 1. But substituting D for D̃, we can also prove Lemma 3.

heading: A.4 PROOF FOR LEMMA 2
text: Proof. Note P(Ỹ 6= Y |X) = 1− P(Ỹ = Y |X) = 1− ∑ i∈[K] P(Ỹ = i|X)P(Y = i|X).
and η(X) := 1
2 ∑ i∈[K] |P(Ỹ = i|X)− P(Y = i|X)|.
Assume P(Y = i′|X) = 1. We have P(Ỹ 6= Y |X) = 1− P(Ỹ = i′|X) and
η(X) := 1
2 1− P(Ỹ = i′|X) + ∑ i∈[K],i6=i′ P(Ỹ = i|X)  = 1− P(Ỹ = i′|X).

heading: A.5 PROOF FOR COROLLARY 1
text: B̂R(P) = η +
√ 2 log(2/δ)
NP + P(Ỹ 6= Ỹ ∗)− √ 2 log(2/δ) NPL
− P(Y 6= Y ∗)√ 2 log(2/δ) NP − √ 2 log(2/δ) NPL
= ∆(NP , NPL)− η
∆(NP , NPL) .

heading: B MATTHEW EFFECT
text: We do observe some “poor” classes may benefit from SSL, e.g., class “dog” in Figure 1(a). We are aware of that, for most cases, we do only observe “the rich get richer”. Intuitively, the phenomenon that several poor classes also benefit from SSL is mainly due to two reasons:
• In a resource-constrained scenario, “the rich get richer” is likely to happen with “the poor get poorer”. However, benefiting from the large capacity of deep neural networks and powerful SSL algorithms, the available “resource” after SSL is usually greater than before SSL. Thus the poor may not always get poorer.
• Current popular SSL algorithms, such as MixMatch (Berthelot et al., 2019b) and UDA (Xie et al., 2019), are very powerful. It is reasonable to believe the re-labeling procedures in each epoch in these algorithms could reduce the disparate impact compared to the two-iteration scenario considered in our theoretical analysis. This may also explain why “some poor classes are adequately benefited.” Although there are some exceptions, our Figure 2 and Figure 3 can still show that there are non-negligible disparate impacts including both “the rich get richer” and “the poor get poorer” in current state-of-the-art (SOTA) SSL algorithms. Besides, Figure 3(a) and Table 1 also show that some classes have negative benefit ratios, indicating “the poor get poorer”.

heading: C MORE EXPERIMENTS

heading: C.1 EXPERIMENT SETTINGS
text: Explicit sub-populations: classification labels In this section, we show the statistics of datasets where we conducted the experiments to demonstrate the disparate impact on the explicit subpopulations (classification labels) in Table 2. In addition, the train/valid/test splitting in the image datasets (CIFAR-10 and CIFAR-100) is 45000:5000:10000. As for the splitting in the text datsets (Yahoo! Answers and AG News), we follow the setting in (Chen et al., 2020).
Implicit sub-populations: fine-categories under coarse classification labels In this section, we provide a description about coarse labels in CIFAR-100 dataset when we conducted the experiments to demonstrate the disparate impact on the implicit sub-populations: fine-categories under coarse classification labels. In CIFAR-100, its 100 classes can be categorized into 20 superclasses. Each image can have a “fine” label (the original class) and a “coarse” label (the superclass). The list of superclasses in the CIFAR-100 is as follows: aquatic mammals, fish , flowers orchids, food containers bottles, fruit and vegetables, household electrical devices, household furniture bed, large carnivores, large man-made outdoor things, large natural outdoor scenes, large omnivores and herbivores, medium-sized mammals, non-insect invertebrates, people, reptiles, small mammals, trees, vehicles 1, and vehicles 2.
Implicit sub-populations: demographic groups (race & gender) In this section, we describe the detailed data distribution before and after balancing methods about implicit sub-populations: demographic groups (race & gender on Jigsaw dataset) . The identities in the race sub-population includes ‘asian’, ‘black’, ‘latin’, and ‘white’. In each identity, we consider different classification labels (toxic and non-toxic). Therefore, we have eight identities in the race sub-population. The numbers of samples on these eight race identities in the unbalanced Jigsaw dataset are ‘asian (toxic): 466; black (toxic): 1673; latin (toxic): 282; white (toxic): 3254; asian (non-toxic): 633; black (nontoxic): 1879; latin (non-toxic): 511; white (non-toxic): 2652’. The numbers of samples on these eight race identities in the balanced Jigsaw dataset are ‘asian (toxic): 1419; black (toxic): 1419; latin (toxic): 1419; white (toxic): 1419; asian (non-toxic): 1419; black (non-toxic): 1419; latin (non-toxic): 1419; white (non-toxic): 1419’. Similar to race, we also have eight identities in the gender sub-population by considering four different gender types which are ‘male’, ‘female’, ‘male femal’ and ‘transgender’ and two classification labels (toxic and non-toxic). The numbers of samples on these eight gender identities in the unbalanced Jigsaw dataset are ‘male (toxic): 5532; female
(toxic): 4852; male female (toxic): 1739; transgender (toxic): 405; male (non-toxic): 5167; female (non-toxic): 4985; male female (non-toxic): 2002; transgender (non-toxic): 374’. The numbers of samples on these eight gender identities in the balanced Jigsaw dataset are ‘male (toxic): 3142; female (toxic): 3142; male female (toxic): 3142; transgender (toxic): 3142; male (non-toxic): 3142; female (non-toxic): 3142; male female (non-toxic): 3142; transgender (non-toxic): 3142’. In either race or gender, Jigsaw dataset contains more than four identities, we set the number of identities to 4 due to the relatively small number of other identities. In addition, the ratio of ‘train:valid:test’ on either race or gender sub-population case is ‘8:1:1’.

heading: C.2 MORE EXPERIMENTAL RESULTS FOR DISPARATE IMPACT ACROSS DIFFERENT CLASSES
text: ON AG NEWS
In this work, we also conducted experiments on the AG News text classification dataset. Figure 4 demonstrates the disparate impact on model accuracy across different classification labels on the AG News dataset.

heading: C.3 MORE EXPERIMENTAL RESULTS FOR DISPARATE IMPACT ACROSS DIFFERENT SUB-POPULATIONS ON JIGSAW (GENDER)
text: In this work, we also conducted experiments on the Jigsaw Toxicity text classification dataset with gender sub-populations. Figure 5 shows the disparate impact on model accuracy across different sub-populations (gender) on the Jigsaw Toxicity dataset.

heading: C.4 MORE EXPERIMENTAL RESULTS OF “BALANCING” TREATMENT TO DISPARATE IMPACT
text: In this section, we show more experimental results of “Balancing” treatment to disparate impact described in Section 5.2. In Table 3, we compare the changes of mean and standard deviation on the model accuracy and our proposed benefit ratio in three settings: (a) SSL with unbalanced data→ (b) SSL with balanced labeled datasets only (by reweighting)→ (c) SSL with both balanced labeled & unlabeled datasets. Setting (c) is not practical due to the lack of labels in unlabeled data but it shows the best performance that this simple reweighting can achieve. Note that CIFAR-100 1:2 Accuracy denotes the accuracy of the case CIFAR-100 1:2, where CIFAR-100 1:2 means 40 instances per
class for the first 50 fine classes, 20 instances per class for the next 50 fine classes. Besides, Jigsaw (1.4:4.5:1:7.5) means that the ratios of different sub-populations’ sizes are 1.4:4.5:1:7.5. Table 3 shows reweighting unbalanced dataset helps improve the overall performance (mean) and reduce the disparate impacts (standard deviation).

heading: C.5 MORE EXPERIMENTAL RESULTS OF “COLLECTING MORE LABELED DATA” TREATMENT TO DISPARATE IMPACT
text: In this subsection, we show more experimental results of “Collecting more labeled data” treatment to disparate impact described on Section 5.2. Table 4 demonstrates the changes of mean and standard deviation on model accuracy and benefit ratio with the varying number (grow exponentially from left to right) of labeled data on CIFAR-100 and Jigsaw (race & gender). As shown in Table 4, the mean accuracy tends to be higher and the standard deviation becomes lower when more labeled data is utilized to train the model, which shows the effectiveness of collecting more labeled data.

Review summaries:
The paper claims semi-supervised learning (SSL) does not uniformly improve performance on all the constituent (latent) sub-population of the group.  More alarmingly, they claim that the SSL improves the performance most on the groups that already fare well (before SSL) with deteriorates on groups that are already worse. They argue intuitively and theoretically for the likely cause. With text, image datasets and three SSL algorithms, they provide empirical evidence that can support their main claim. Finally, they briefly discuss strategies that could avoid such disparate impact of SSL. 

This paper presents the disparate impact on semi-supervised learning, where the sub-population that has a higher baseline accuracy tends to benefit more from SSL. The paper targets an important problem to ensure the fairness of machine learning algorithms. The authors also proposed a new metric benefit ratio to facilitate the evaluation of SSL. They also suggested two methods to mitigate the disparate, balancing the labeled data or collecting more labeled data.

This paper analyzes the discriminative performances of semi-supervised learning (SSL) on different subgroups, based on the key message that the already accurate group gets more benefits from the SSL. The paper first proposes a unified loss for analyzing the SSL frameworks. Then, the theoretical analyses show how the SSL produces discriminative performances on the different groups. In addition, a metric called benefit ratio is suggested to measure how much the learning benefits a certain subgroup. Experiments show overall consistent results with the main message from the theory.