The Rich Get Richer: Disparate Impact of Semi-Supervised Learning
Authors: Zhaowei Zhu
Tianyi Luo
Yang Liu
Venue: ICLR 2022 Poster (2022)
Keywords: semi-supervised learning
fairness
disparate impact
Matthew effect
consistency regularization

Abstract: Semi-supervised learning (SSL) has demonstrated its potential to improve the model accuracy for a variety of learning tasks when the high-quality supervised data is severely limited. Although it is often established that the average accuracy for the entire population of data is improved, it is unclear how SSL fares with different sub-populations. Understanding the above question has substantial fairness implications when different sub-populations are defined by the demographic groups that we aim to treat fairly. In this paper, we reveal the disparate impacts of deploying SSL: the sub-population who has a higher baseline accuracy without using SSL (the "rich" one) tends to benefit more from SSL; while the sub-population who suffers from a low baseline accuracy (the "poor" one) might even observe a performance drop after adding the SSL module. We theoretically and empirically establish the above observation for a broad family of SSL algorithms, which either explicitly or implicitly use an auxiliary "pseudo-label". Experiments on a set of image and text classification tasks confirm our claims. We introduce a new metric, Benefit Ratio, and promote the evaluation of the fairness of SSL (Equalized Benefit Ratio). We further discuss how the disparate impact can be mitigated. We hope our paper will alarm the potential pitfall of using SSL and encourage a multifaceted evaluation of future SSL algorithms.  

Published as a conference paper at ICLR 2022
THE RICH GET RICHER: DISPARATE IMPACT OF SEMI-SUPERVISED LEARNING
Zhaowei Zhu', Tianyi Luo",, and Yang Liu Computer Science and Engineering, University of California, Santa Cruz (zwzhu, tlu yangl Bangllul@uusss @ucsc.edu
ABSTRACT
Semi-supervised learning (SSL) has demonstrated its potential improve the model accuracy for a variety of learning tasks when the high-quality supervised data is severely limited. Although it is often established that the average accuracy for the entire population of data is improved, it is unclear how SSL fares with different popp-pppulaiiss... Understanding the above question has substantial fair- ness implications when different uub-populations are defined by the demographic groups that we aim to treat fairly. In this paper, we reveal the disparate impacts deploying SSL: the sub- uub-population who has a higher baseline accuracy without us- ing SSL (the "rich" one) tends to benefit more from SSL; while the sub-population who suffers from a low baseline accuracy (the "poor" one) might even observe a performance drop after adding the SSL. module. We theoretically and empirically establish the above observation for broad family of SSL algorithms, which either explicitly or implicitly use an auxiliary "pseudo-label". Experiments on a set of image and text classification tasks confirm our claims. We introduce a new met- ric, Benefit Ratio, and promote the evaluation of the faimess of SSL (Equalized Benefit Ratio). We further discuss how the disparate impact can be mitigated. We hope our paper will alarm the potential pitfall of using SSL and encourage a multifaceted evaluation of future SSL algorithms.
INTRODUCTION
The success of deep neural networks benefits from large-size datasets with high-quality supervisions, while the collection of them may be difficult due to the high cost of data labeling process (Agarwal al., 2016; Wei et al., (022). A much cheaper and easier solution is obtain a small labeled dataset and a large unlabeled dataset, and apply semi-supervised learning (SSL). Although the global model performance for the entire population of data is almost always improved by SSL., is unclear how the improvements fare for different sub-populations, e.g., multiaccuracy metric (Kim et al.) 2019). Analyzing and understanding the above question will have substantial fairness implications especially when sub-populations are defined by the demographic groups e.g., race and gender.
Motivating Example Figure D shows the change of test accuracy for each class during SSL, Each class denotes one sub- uub-population and representative sub- uub-populations highlighted. Figure deelv- ers two important messages: in SSL, even with some state-of-the -frte-arrt-- algorithms: 1) the observation "rich getting richer" is common, and 2) the observation "poor getting poorer" possibly happens. the "rich" " sub-population such as automobile that has a high baseline accuracy at the beginning of SSL, tends to be consistently benefiting from SSL. But the "poor" aub-population, such as dog that has a low baseline accuracy, will remain low-level performance as Figure [[a) or even get worse performance as Figure bb). This example shows disparate impact of accuracies for different sub sub-populations are common in SSL. In special cases such as Fib) we observe the Matthew effect: the rich get richer and the poor get poorer. See more discussions Appendix B
Specifically,
In this paper, we aim to understand the disparate impact of SSL from both theoretical and em- pirical aspects, and propose to evaluate SSL from a different dimension. Specifically, based on classifications tasks, we study the disparate impact of model accuracies with respect to different sub-populations (such as label classes, feature groups and demographic groups) after applying SSL.
"Equal contributions. Corresponding author: Yang Liu <yynng iutucsc.ddu>>

---

Published as a conference paper at ICLR 2022
NisMatch Balanced aas 25x30
Inbelannced siao 12255
100
100
Sir
Brnd
deer
dog
Fro
20-
Everage
10 15 20
25
30
5
10 15
20
25
30
Epocns (x25)
Epochs (425)
Disparate impacts in the model accuracy of SSL. MixMatch (Berthelot et al. 2019b is applied on CIFAR-10 with (a) 250 clean labels in the balanced case (25 labeled instances per class) and (b) 185 clean labels in the unbalanced case (25 labeled instances in each of the first 5 classes, 12 labeled instances in each of the remaining classes). Other instances are used as unlabeled data.
Figure 1:
Different from traditional group fairness (Zafar et al. 2017 Bellamy et al. 2018: Jiang et al. 2022b) defined over one model, our study focuses on comparing the gap between two models (before and after SSL). To this end, we first theoretically analyze why and how disparate impact are generated. The theoretical results motivate us to further propose a new metric, benefit ratio, which evaluates the normalized improvement of model accuracy using SSL methods for different sub-popu sub-populations. The benefit ratio helps reveal the "Matthew effect" of SSL: a high baseline accuracy tends to reach a high benefit ratio that may even be larger than 1 (the rich get richer), and sufficiently low baseline accuracy may return a negative benefit ratio (the poor get poorer). The above revealed Matthew ef- feet indicates that existing and popular SSL algorithms can be unfair. Aligning with recent literature on fair machine learning (Hardt et al., 2016 Corbett-Davier & Goel [0118] Verma & Rubin 2018), we promote that a fair SSL algorithm should benefit different sub-populations equally, i.e., achiev- ing Equalized Benefit Ratio which we will formally define in Definition [ We then evaluate SSL using benefit ratios and discuss how two possible treatments, i.e., balaneing the data and collecting more labeled data, might mitigate the disparate impact. We hope our analyses and discussions could encourage future contributions to promote the fairness SSL.
Our main contributions and findings are summarized as follows: work that unifies broad family of SSL algorithms, which either explicitly or implicitly use an aux- iliary "pseudo-label". The framework provides a novel perspective for analyzing SSL by connecting consistency regularization to learning with noisy labels. 2) Based on our framework, we further prove an upper bound for the generalization eтror of SSL and theoretically show the heterogeneous error of learning with the smaller scale supervised dataset is one primary reason for disparate im- pacts. This observation reveals "Matthew effect" of SSL. 3) We contribute a novel metric called benefit rutio to measure the disparate impact in SSL, the effectiveness of which is guaranteed by our proved generalization bounds. 4) We conduct experiments on both image and text classification tasks to demonstrate the ubiquitous disparate impacts in SSL. 5) We also discuss how the disparate impact could be mitigated. Code is available at nigthu...b. cwm/UCsC REAL/Dispaasee
1) We propose an analytical frame-
SSL
1.1 RELATED WORKS
Semi-supervised learning SSL is popular in various communities Dai & Le 2015 Howard &
Ruder |0ark
Clark et al. 2018 Yang et al. 2019] [Sachan et al. [2019] Guo al. [2020] Cheng al.
2021
Wang et al.] 2021c Luo et al. 2021] Huang et al. 2021 Bai et al. 2021 Wang et al. 2020 Liu
al. 2020). We briefly review recent advances in SSL. See comprehensive overviews by (Chapelle
ct al. 2006: Zhu et al. 2003) for traditional methods. Recent works focus on assigning pscudo-
labels generated by the supervised model to unlabeled dataset (Lee et al., 2013 Iscen et al., 2019
Berthelot et al. 2019b a). where the pseudo-labels are often confident or with low-entropy (Sohn
ct | 2020: Zhou 2018 Meng et al. 2018). There are also many works on minimizing entropy of predictions on unsupervised data (Grandvalet et al.) 2005) regularizing the model consistency on
the same feature with different data augmentations Tarvainen & Valpola 2017 Miyato et al. 2018
Sajjadi et al. 2016 Zhang al. 2018 Miyato et al.
2016 Xie et al. 2019). In addition to network
inputs, augmentations can also be applied on hidden layers (Chen et al. 2020). Besides, some works
(Peters al. 2018 Devlin al. 2018 Gururangan et al.
2019 (019:Chen al.) 2019 Yang al. 2017)
2

---

Published as a conference paper at ICLR 2022
first conduct pre-training the unlabeled dataset then fine-tune the labeled dataset, use ladder networks to combine unsupervised learning with supervised learning (Rasmus et al. 2015).
Disparate impact Even models developed with the best intentions may introduce discriminatory bi- ases (Raab& & Liu 202T). Researchers in various fields have found the unfairness issues, e.g., vision-
and-language representations (Wang et al., 2022). model compression (Bagdasaryan et al. 2019)
differential privacy (Hooker et al.] 2019: 2020), recommendation system (Gómez et al.] 2021 in-
formation retrieval (Gao & Shah, 2021). image search (Wang et al., 2021b), machine translation
(Khan et al., 202T), message-passiln
(Jiang el al. 2022a). and learning with noisy labels (Liu 2021
Zhu al. [0021a] Liu Wang 2021). There are also some treatments considering fairness without
demographics (Lahoti et al.] 2020; Diana et al. [022]] Hashimoto et al. (2018), minimax Pareto fair- ness (Martinez et al. 2020), multiaccuracy boosting Kim et al.) 2019), 9) and fair classification with
label noise (Wang et al. [2021a). Most of these works focus on supervised learning. To our best
knowledge, the unfairness of SSL. has not been sufficiently explored.
2
PRELIMINARIES
We summarize the key concepts and notations as follows.
2.1 SUPERVISED CLASSIFICATION TASKS
Consider a K-class classification task given set of Nr labeled training examples denoted by Dr = {IIm: yn) )n))neln ne[NN]] where [Nz] := {1, 2,... ..... Nz). Fu an input feature, y/ E [K] is la- bel. The clean data distribution with full supervision denoted by D. Examples (I', yn) are drawn according to random variables (X, Y) ~ DD The classification task aims to identify a classifier f that maps X Y accurately. Let 1.} } be the indicator function taking value when the speci- fied condition is satisfied and O otherwise. Define the 0-1 loss as 1 ((((),, Y) (X)()) & Y}- The optimal f is denoted by the Bayes classifier f = arg min, ED (X()), (())))) Y)]- One com- mon choice is training a deep neural network (DNN) by minimizing the empirical risk: f == arg min, # Ex= ((f(z)) yn). Notation E(-) stands for the cross-entropy (CE) loss {(f(z), == -In(f> (y]), y E [K]. where fzlv] denotes the y-th component of f(x). Notations ʃ and stand for the same model but different outputs. Specifically, vector f (z) denotes the probability of each class that model f predicts given feature x. The predicted label f(x) is the class with maximal probability, i.c., f(z) ;= arg [ [.]. We use notation f if we only refer to model.
2.2 SUPEVISDE CLASSIFICATION TASKS
In the semi-supervised learning (SSL) task, there is also an unlabeled (a.k.a. unsupervised) dataset Du= == {(Inn+ No. )) [eeNN drawn from D, while the labels are missing or unobservable. Let N == Nr + Ny. Denote the corresponding unobservable supervised dataset by D = {( Vn (nee Compared with the supervised learning tasks, it is critical to leverage the unsupervised information in semi-supervised learning. To improve the model generalization ability, many recent SSL meth- ods build consistency regularization with unlabeled data to ensure that the model output remains unchanged with randomly augmented inputs (Berthelot et al.) 2019b; Xie et al. 2019: Berthelot et al. 2019a Sohn et al. 2020 Xu et al., 2021). To proceed, we introduce paftppseudo-lbel y.
Soft-labels Note the one-hot encoding of yn can be written ynu where each element writes as [i] = 1(i = }. More generally, we can extend the one-hot encoding to soft labels by requiring each element y(i] E 1) and Σje[K] yli 1. The CE loss with soft y writes as {(f(x). v) = Σie(K] y In(ff(()) If we interpret y(( = P(Y i) as a probability (Zhu al., 2022a) and denote by Dy the corresponding label distribution, the above CE loss with soft labels can be interpreted as the expected loss with respect to a stochastic label Y, i...
(f(())
Σ =)e((((((>))).)) K
EynD, (((()))
(1)
In consistency regularization, by using model predictions, the unlabeled data will be assigned pseudo-labels either explicitly (Berthelot et al.) (019b) or implicitly (Xie et al. 2019). where the lseudo-labels can be modeled as Jabl-. In the following, we review both the explicit and the implicit approaches and unify them in analytical framework.
Pseudo-labels
3

---

Published as a conference paper at ICLR 2022
Consistency regularization with explicit pseudo-labels For each unlabeled feature In,, pseudo- labels can be explicitly generated based on model predictions (Berthelot et al. 2019b). The pseudo- label is later used to evaluate the model predictions. To avoid trivial solutions where model pre- dictions and pseudo-labels are always identical, independent data augmentations of feature Zn are often generated for M rounds. The augmented feature is denoted by znm Augment(zx)) m [M]] Then the pseudo-label Yn in epoch-t be determined based on M model predictions as (z'm.)) where model f(() is copy of the DNN at the beginning of epoch-t but without gradients. The function Sharpen(-) reduces the entropy of pseudo-label, e.g., setting to one-hot encoding ej:J arg max4e[K] yn (Sohn al. 2020). In epoch-t, with some consistency regularization loss (CR(-). the empirical risk using pseudo-labels is:
yn Sharpen
MT
No.
No Σ
DI, Du)
Σ ((f(rn))u)) No R=N++1
(xn()(
NL x=1
Consistency regularization with implicit pseudo-labels Consistency regularization can also be applied without specifying particular pseudo-labels, where divergence metric between predictions on the original feature and the augmented feature is minimized to make predictions consistent. For example, the KL-divergence could be applied and the data augmentation could be domain-specifer (Xie et al. 2019) or adversarial (Miyato et al. 2018). In epoch-t, the total loss is:
No
Nr No
Dz, Du)
1 No
Σ
((f(En). + R=1
Σ fcR (f (1))),
NU
N==++1
where À balances the supervised loss and the unsupervised loss, s/ Augment(2;; (r,) stands for one- round data augmentation (m = following the previous notation n. Without loss of generality, use À = 1 in our analytical framework.
Consistency regularization loss (cR(.) In the above two lines works, there are different choices of (.)(.). such mean-squared error leR (f (f(x)) .)). = [f(r)
/l3/KK (Berthelot al. 2019b Laine & Aila (016) or CE loss (Miyato et al.) 2018 Xie et al., 2019) defined in Eq. (1). For a clean analytical framework, we consider the case when both supervised loss and unsupervised loss are the same, i.e., tCR (f (xx), y) = (x),, y). Note Le implies the entropy minimization (Grandvalet et al. 2005) when both loss functions are and there is no augmentation, i.e., x'1 Fn.
2.3 ANALYTICAL FRAMEWORK
We propose an analytical framework to unify both the explicit and the implicit approaches. With Eqn. ① the unsupervised loss in the above two methods can respectively written as:
No+No
N
Σ BYND w1 ((((s)))))) and
Σ
EV WYDDDOOG ((((En)))))
No N=+++1
N N=---1
Both unsupervised loss terms inform us that, for each feature In, we compute the loss with respect to the reference label Y, which is a random variable following distribution D p2tt) or D (()())) Com- pared with the original clean label Y, the unsupervised reference label Y contains label noise, and noise transition (Zhu et al. 2021b) depends on feature Fn Specifically, we have
P(Ý x xn)
e [ (Explicit) or P(Ý i xT. f [t] (Implicit).
Then with model (()) we can define new dataset with instance-dependent noisy reference labels: D {Z,, Fn fn))net Nn where Un = yn Vn E [Nz], and ÿn. ! or f(t)) Vn +1, NN.
The unified loss is:
N Σ ((f(r.) N n=1
N Σ Ey-D [(((En)) n=1
L(f. L(F N
(2)
Therefore, with the pseudo-label as a bridge, we can model the popular SSL solution with consis-
tency regularization as a problem of learning with noisy labels Natarajan et al. 2013). But different
from the traditional class deaendee assumption (Liu Tao 2015: Wei Liu 2021 Zhu et al,
20226), the instance-dependent pseudo-labels are more challenging (Liu 2022
4

---

Published as a conference paper at ICLR 2022
3 DISPARATE IMPACTS OF SSL
To understand the disparate impact of SSL, we study how supervised learning with Dr, affects the performance of SSL with both D2 and Dy. In this section, the analyses for one and an arbitrary sub-population, thus we did not explicitly distinguish sub-populations in notation.
Intuition The rich sub- cotb-population with higher baseline accuracy (from supervised learning with D2) will have higher-quality pseudo-labels for consistency regularization, which helps to further improve the performance. In contrast, the poorer sub-population with lower baseline accuracy (again from supervised learning with Dr.) can only have lower-quality pseudo-labels to regularize the consistency of unsupervised features. With wrong regularization direction, the unsupervised feature will have its augmented copies reach consensus on a wrong label class, which leads to a performance drop. Therefore, when the baseline accuracy is getting worse, there will be more and more unsupervised features that are wrongly regularized, resulting in disparate impacts of model accuracies as shown Figure I
In the following, we first analyze the generalization enror for supervised learning with labeled data, then extend the analyses of generality, we consider minimizing 0-1 loss 1(J(X), Y) with infinite search space. Our analyses can generalized to bounded loss E(-) and finite function space F following the generalization bounds that can be introduced using Rademacher complexity (Bartlett & Mendelson 2002).
supi-super learning with the help of pseudo-labels. Without loss
3.1 LEARNING WITH CLEAN DATA
Denote the expected error rate of classifier f on distribution D by RD(f) == ED(I(((X), Y)]- Let fo denote the classifier trained by minimizing 0-1 loss with clean dataset D, i.e., Jn == arg miny [n(J), where Rn(s) == N EnEiN ([((an), vn). Denote by Y* X arg m [P(||X) the Bayes optimal label on clean distribution D. Theorem shows the gen- eralization bound in the clean case. Replacing D and N with DL and NL we have:
Theorem (Supervised crror). With probability at least 6- the generalization error of supervised
learning on clean dataset Dr, is upper-bounded by RD RD (fff)) S v 2log(2/5)
+P *Y).
N2
3.2 LEARNING WITH SEMI-SUPERVISED DATA
We further derive generalization bounds for learning with semi-supervised data. Following analytical framework in Section 2.3 in each epoch-t, we can transform the semi-supervised data to supervised data with noisy supervisions by assigning pseudo-labels, where the semi-supervise dataset Dr Do is converted the dataset Ú given the model learned from the previous epoch.
Two-iteration scenario In our theoretical analyses, to find a clean-structured performance bound for learning with semi-supervised data and highlight the impact of the model learned from super- vised data, we consider particular -ro-iteration scenario where the model is first trained to con- vergence with the small labeled dataset Dr, and get fD,, then trained on the pseudo noisy dataset D labeled by JDz- Noting assigning pseudo labels iteratively may improve the performance as suggested by most SSL algorithms (Berthelot et al.) 2019b] Xie et al. 2019), our considered two- iteration scenario can be seen worst case for an SSL algorithm.
Independence of samples in D Recall x', denotes the augmented copy of xx. The N instances in may not be independent since pseudo-label yn. comes from fD, (x',) which depends on I,-- Namely, the number of independent instances N' should be in the range of [Nz, NV. Intuitively, with appropriate noise injection or data augmentation (Xie et al., 2019 Miyato et al. 2018) to In such that x', could be treated as independent of In. the number of independent samples in D could be improved to N. We consider the ideal case where all N instances are i.i.d. in the following analyses.
By minimizing the unified loss defined in Eq. 2 we can get classifier fB == arg ming lt)(
where Rz(s)= = - Ene(M] [EEEKK] Ul ((((a)) The expected error given classifier f is

---

Published as a conference paper at ICLR 2022
denoted by Rz(f) == Ex(1(f(X), Y), where the probability density function of distribution D can be defined as P (x,,))==(X-= = Fn, Y = ((xx)) = .n) yn [t].
Decomposition With the above definitions, the generalization error (on the clean distribution) of classifier f could be decomposed as RD(f;) = (Ro(f)) - Ro(fr)) + (T(f)) where Term-1
TTrm-1
Tern-2
transforms the evaluation of f from clean distribution D to the pseudo noisy distribution D. Term- 2 is similar to the generalization error in Theorem [ but the model is trained and evaluated on noisy distribution D. Both terms are analyzed as follows.
Upper and Lower Bounds for Term-1 Let xx)) == Eijn IP( i|X) P(Y = i|x)).
e(X) == P(Y # fIX) be the feature-dependent error rate, Ax(X) == P(S()) = F1X) be the accuracy of prediction f(X) on noisy dataset D. Denote their expectations (over X) by :j Ex[m [x)),, ē == [x]e(X)], A, = [Ax[A(X)], To highlight that fj and é depends on the noisy dataset D labeled by fD,, we denote them as r(fD,) and e(fD.). Then we have:
Lemma (Bounds for Term-1). (2 1)e(fo,) RD(fp) Ro(fa) S T(D,))
Note the upper bound builds on n(fD.) while the lower bound relates to e(fp,). To compare two bounds and show the tightness, we consider the where Y|X is confident, i.e., each feature X belongs to one particular true class Y with probability 1, which is generally held in classification problems (Liu & Tao 2015). Lemma 2 shows (xX) = =((X ) in this particular case.
Lemma 2 (1 vs. e). For any feature X, if X is confident, v(X) is the error rate of the model prediction on X, i.e., Di E [K] P(Y x| = (X) IP(Y + xYXX) e(X).
Upper bound for Term-2 Denote by y arg max P(Y i|X) the Bayes optimal label on noisy distribution D. Following the proof for Theorem
we have:
Lemma 3 (Bound for Term-2). W. p. at least 6, R.
2 log(2//0
P( AYY+Y' Y
Wrap-up Lemma shows Term-1 1 is in the range of [(2A -1 -1)((fd,)) A(DD.)) Lemma informs us fl.l. = e(fDL) in classification problems where YJX, VX are confident. With a well-trained and - 04, thus Term-1 is in
model f that learns the noisy distribution D, we have A =
the range of [(1 2e)r((D.)) n(fD.)) indicating our bounds for Term-l are tight. Besides, noting Lemma 3 is derived following the same techniques as Theorem T we know both bounds have similar tightness. Therefore, by adding upper bounds for Term- and Term-2,
upper bound the error of sumi-super learning in Theorem 2 which has similar tightness to that in Theorem []
Theorem 2 (Semi-supervise) learning error). Suppose the model trained with only D2 has gener- alization error vi' (fnz). With probubility least 1 5, the generalization error of semi-supervised learning on datasets DL U Du is apper-bounder by
2log(2/8) N
RD((D.uDu) <
A(fD.)
P
Dispurity due baveline
Sharpmess pscudo labels
Data dependency
where of Nu/N is the expected label error the pseudo noisy dataset D.
Takeaways Theorem 2 explains how disparate impacts in SSL are generated.
Supervised error 7j' (f(p))) ): the major source of disparity. The sub-population that generalizes well before SSL tends to have lower SSL error. Namely, the rich get richer.
Sharpness of noisy labels IP(Y * Y²*): minor source of disparity depends on how one processes pseudo-labels. This term is negligible if we sharpen the pseudo-label.
Sample complexity lo independence. Recall we assume ideal data augmentations to get N in this term. There will be much less than N i..dd instances with poor data augmentations. would be major source of disparity if data augmentations are poor.
disparity depends the number of instances N and their

---

Published as a conference paper at ICLR 2022
4 BENEFI RATIO: AN EVALUATION METRIC
Figure I demonstrates that SSL leads to disparate impacts of different sub-populations' accuracies, but is still not clear how much that SSL benefits a certain sub-population. To quantify the disparate impacts of SSL, we propose new metric called benefit ratio.
Benefit Ratio The benefit ratio BR(P) captures the normalized accuracy improvement of sub- population P after SSL, which depends on three classifiers, i..., fD.: (baseline) supervised learning only with a small labeled data Dr., fn: (ideal) supervised learning if the whole dataset D has ground-truth labels, and DD SSL with both labeled dataset Dz, and unlabeled datasetDr. The test/validation accuracy of the above classifiers are Otweeline (P), Oadeaa()) (P), and (semi respectively. As posterior evaluation of SSL algorithms, the benefit ratio BR(P) is defined as:
BR(P) Osemi (P) Obaseline (P)
(3)
Ojdeal (P) Abaseline (P)
Let pe := P2.. be the set of all the concerned aub-populations. We formally define the Equalized Benefit Ratio as follows.
Definition (Equalized Benefit Ratio). We call an algorithm achieving equalized benefit ratio all the concerned sub-populations have the same benefit ratio: BR(P) = BR(P"), ",PP"","" VP, p E po.
when SSL performs as well as the corresponding fully-supervised learning. A negative benefit ratio indicates the poor population is hurt by SSL such that Qsems (P) < Obaseline (P), i.e., the poor get poorer as shown in Figure [b) (sub-population dog). It has the potential of providing guidance and intuitions for designing fair SSL algorithms on standard datasets with full ground-truth labels. Whether fair SSL algorithm one dataset is still fair on another dataset would be an interesting future work. In real scenarios without full supervisions, we may use some extra knowledge to estimate the highest achievable accuracy of each sub-population and set it as a proxy of the ideal accuracy Gsdeal (P).
Intuitively, a larger benefit ratio indicates more benefits from SSL. We have BR(P)
Theoretical Explanation Recall we have error upper bounds for both supervised learning (Theo- rem and semi-superviee learning (Theorem 2}. Both bounds have similar tightness thus can sup (Rplf (, u -sp (RD((, sup(Rz (f -su (Ro(fog r)) where sup(.) denotes the upper bound derived in Theorem and Theorem 2 P is a sub-population, and D(P denotes the set of i.i.d. instances in D that affect model generalization on P. By assuming P(Y * = P(Ý + Y*) (both distributions have the same sharpness), we have:
compare them and get a proxy of benefit ratio as BR(P)
((fr,1P) Corollary 1. The benefit ratio proxy for P is BR(P) = 1 A(ND.ND,)' where A(Np, No,)) =
2108(2/8) NFL
2208 No and Np, are the effective numbers of instances in D(P and DrlP.
Corollary shows the benefit ratio is negatively related to the error rate of baseline models and pos- itively related to the number of i.i.d. instances after in SSL, which is consistent with our takeaways from Section 3 Note Np and Nr, may be larger than the corresponding pobp-pppultions sizes if P shares information with other sub-population P' during training. e.g., better classification of ''' helps classify P. It also informs us that SSL may have negative effect on sub-population P if N > 1, i.e., the benefit from getting more effective i.i.d. instances less than the harm from wrong pseudo-labed This negative effect indicates "the poor get poorer".
"
EXPERIMENTS
In this section, we first show the existence of disparate impact on several representative SSL methods and then discuss the possible treatment methods to mitigate this disparate impact.
Settings Three representative SSL methods, i.e., MixMatch (Berthelot et al.) (0195), UDA Xic et al. 2019), and MixText (Chen et al., 2020), are tested on several image and text classification tasks. For image classification, we experiment on CIFAR-10 10 and CIFAR-10 100 datasets (Krizhevsky et al. 2009). We adopt the coarse labels (20 classes) in CIFAR-100 for training and test the per- formance for each fine label (100 classes). Thus our training on CIFAR- 100 is a 20-class classi- fication task and each coarse class contains 5 sub-populations. For text classification, we employ
7

---

Published as a conference paper at ICLR 2022
Motic,, labeled sibe
UDA, lobeled 25x10
MiXMatch, siza
50x10
1.0
Brnertt Rocr
L.
Bonertt E50o
0.8
0.6
0.4
0.4
0.4
0.2
60
70
60
82
Acttnncy
Assunneyy
Arssintty
(a) Benefit ratios (g-axis) versus baseline accuracies before SSL (z-axis) on CIFAR-10
MixTest, inbeled rize 100x30
US36 inbeled 100x10
KicTesd ihelled sire
USA, abeted size 200x10
1.4
14
1.2
1.0
Rodi
Bondtte Rodo
0.0
0.0
0.8
0.8
06
0.2
0.2
70
40
68
82
Armunary 80
30
Jcmisry
Arronayy
Accurncy
(b) Benefit ratios (y-axis) versus baseline accuracies before SSL (z-axis) on Yahoo! Answers
Figure 2: Benefit ratios across explicit sub-populations. Dot: Result of each label class. Line: Best linear fit of dots.
three datasets: Yahoo! Answers (Chang et al. 2008), AG News (Zhang et al. 2015) and Jigsaw Toxicity (Kaggle 2018). Jigsaw Toxicity dataset contains both classification labels (one text com- ment is toxic or non-toxic) and a variety of sensitive attributes (e.g., race and gender information) of the identities that are mentioned in the text comment, which are fairness concerns in real-world applications. We defer more experimental details Appendix
5.1 DISPARATE IMPACT EXISTS IN POPULAR SSL METHODS
We show that, even though the size of each -oppuloatttion is equal, disparate impacts exist in the model accuracy of different sub-populations, i.e., 1) explicit sub-populations such as classification labels in Figure 2 (and Figure 5 in Appendix C.2) and 2) implicit rub-populations such as fine- categories under coarse classification labels Figure 3a demographic groups including race in Figure and and gender (Figure 6 in Appendix C.3 All the experiments in this subsection adopt both
a balanced labeled dataset and balanced unlabeled dataset. Note we sample a balanced subset from the raw (unbalanced) Jigsaw dataset. Other datasets originally balanced.
Disparate impact across explicit sub-populations In this part, we show the disparate impact on model accuracy across different classification labels CIFAR-10, Yahoo! Answers, and AG News datasets. Detailed results on AG News can be found in Appendix C.2 In Figure 2a 2b) we show the benefit ratios (y-axis) versus baseline accuracies before SSL (x-axis). From left to right, show results with different sizes of labeled data: 25 per class to 50 on CIFAR-IO and 100 per class 200 on Yahoo! Answers. Figure 2a and 26 utilized two SSL methods respectively (CIFAR- MixMatch & UDA; Yahoo! Answers: Mix Text & UDA). We statistically observe that the class labels with higher baseline accuracies have higher benefit ratios on both CIFAR-10 and Yahoo! Answers datasets. It means that "richer" classes benefit more from applying SSL methods than the "poorer" ones. We also observe that for some models with low baseline accuracy (left side of the xraxis), applying SSL results in rather low benefit ratios that are close to 0.
Disparate impact across implicit sub- sub-populations We demonstrate the disparate impacts on model accuracy across different sub-populations on CIFAR-100 (fine-labels) and Jigsaw (race and gender) datasets. Figure 3 we can statistically observe the disparate impact across different sub- populations on both datasets for three baseline SSL methods. Detailed results Jigsaw with gender are shown in Appendix [C.3] We again observe very similar disparate improvements as presented in Figure 2a and for some classes in CIFAR- 100, this ratio even go negative. Note the disparate impact on the demographic groups in Jigsaw raises fairness concerns in real-world applications.
5.2 MITIGATING DISPARATE IMPACT
Our analyses in Section B and Section indicate the disparate impacts may be mitigated by balancing the supervised error 77( (P) and the number of effective i.i.d. instances for different populations.
8

---

Published as a conference paper at ICLR 2022
MurMatch lubeled sice 3X5x20
UDA, lobuled sicc 5 #20
MisKatch, labeled sice 10X5X20
UDA, dobeled sice 20x5x20
0.6
0.8
0.7
0.5
0.6
Bonett Preete-roo
0.4
0.5
0.2
0.3
0.1
0.0
0.0
2.
40
60
20
30
4.
50
20
60
60
35
40
25
50
Accurscy with 500 dean laba
Accurscy with clean labk
Accuracy with 1000 dean laba
Acouracy with 1000 clean babk
(a) Benefit ratios of fine-labels (3-axxi) versus basclinc accuracies before SSL (2-axis) CIFAR- 100
MixTusi, labr udd sizu 25xx
UDA, labelee view 25w8
MixTust, lnowled siew S6
UOW. loouled size 50xe
0.5
0.5
0.5
0.3
0.4
0.4
0.4
C-a
Banait Robo
Benwhh Dutn
0.3
0.3
0.3
Bonoft
0.2
82
0.2
E
0.3
0.1
0.1
0.0
65
69
70
72
0.0
66
60
70
0.0
70
12
74
0.0
70
72
74
Accuracy witt 200 clean lbbls
Accuracy with 200 clean labels
Accuracy with 400 ciean labels
Accuracy with clean labels
(b) Benefit ratios of different race attributes (y-axis) versus baseline accuracies before SSL on Jigsaw.
Figure 3: Benefit ratios across implicit sub-pon colb-ppulatiios. Experiments on CIFAR- 100 are ran 5 times for stability. Mean (dashed line) and standard deviation (shaded area) are plotted in (a), where points in the shaded area indicate the converged local optima with a non-negligible probability.
Table 1: Comparison on the benefit ratio (%) of all race identities & standard deviation (%) between different settings. tLasian (nt_asian) is the benefit ratio of asian identity with "toxic" labels ("non- toxic" labels). SD denotes standard deviation. Negative benefit ratios are highlighted in red colors.
Digsaw] Settings
1 asian
t Mock
Latm
whate
ne osion
nt Back
nst latan
nf white
SD
Unbulanced (400)
21.71
21.76
-9.57
21.27
-8.33
-13822
29.17
19:28
Balance labeled (400)
28.18
20.00
25.23
33.93
14.33
-11.29
-10.37
3.70
17:29
Balance both (400)
28.37
0.00
11.11
10.63
13.38
11.29
6.90
-7.41
13.29
Balance both (800)
13.04
25.00
7.69
21.00
3.85
10.71
16.67
20.00
7.61
We perform preliminary tests to check the effectiveness of the above findings in this subsection. We hope our analyses and experiments could inspire future contributions to disparity mitigation in SSL.,
Balancing and Collecting more labeled data We firstly sample 400 i.i.d. instances from the raw (unbalanced) Jigsaw dataset and get the setting of Unbalanced (400). To balance the supervised error and effective instances for different sub-populations, an intuitive method is to balance the labeled data by reweighting, e.g., if the size two aub-populations is 1:2, we simply sample instances from two aob-oopulations with a probability ratio of 2:1 to ensure all sub-population have the same weights in each epoch during training. Balance labeled (400) denotes only 400 labeled instances are reweighted. Balance both (400) means both labeled and unlabeled instances shows a detailed comparison on the benefit ratios between different race identities of the Jigsaw dataset and their standard deviations, where the first three rows denote the above three settings. We can observe that both the standard deviation and the number of negative benefit ratios become lower with more balanced settings, which demonstrates the effectiveness of the balancing treatment strategy, although there still exists sub-population that has negative benefit ratio, indicating an unfair learning result. To further improve fairness, as suggested our analyses, we "collect" (rather add) another 400 i.i.d. labeled instances (800 in total) from the Jigsaw dataset, and show the result after balancing both labeled and unlabeled data the last row Table both (800). Both the standard deviation and the number of negative benefit ratios can be further reduced with more labeled data. More experimental results are on CIFAR- 100 and Jigsaw (gender) datasets shown in Appendix C.4 (balancing) and Appendix C.5 (more data).
balanced. Table I
6 CONCLUSIONS
We have theoretically and empirically shown that the disparate impact (the "rich" sub-populations get richer and the "poor" ones get poorer) exists universally for a broad family of SSL algorithms. We have also proposed and studied new metric benefit ratio to facilitate the evaluation of SSL. We hope our work could inspire more efforts towards mitigating the disparate impact and encourage a multifaceted evaluation of existing and future SSL algorithms.

---

Published as a conference paper at ICLR 2022
ACKNOWLEDGMENTS
This work is partially supported by the National Science Foundation (NSF) under grant IIS-2007951, UIS2114395,, and the NSF FAI program in collaboration with Amazon under grant IIS-2040800.
ETHICS STATEMENT
In recent years, machine learning methods are widely applied in the real world across a broad variety of fields such face recognition, recommendation system, information retrieval, medical diagnosis, and marketing decision. The trend is likely to continue growing. However, is noted that these machine learning methods are susceptible introducing unfair treatments and can lead to systematic discrimination against some demographic groups. Our paper echoes this effort and looks into the disparate model improvement resulted from deploying a semi-supervised learning (SSL) algorithm.
Our work uncovers the unfairness issues in semi-supervised learning methods. More specifically, we theoretically and empirically show the disparate impact exists the recent SSL methods on several public image and text classification datasets. Our results provide understandings of the potential disparate impacts of an SSI. algorithm and help raise awareness from a fairness perspective when deploying SSL model in real-world applications. Our newly proposed metric benefit ratio contributes to the literature a new measure to evaluate the fairness SSL. Furthermore, we discuss two possible treatment methods: balancing and collecting more labeled data to mitigate the disparate impact SSL methods. We have carefully studied and presented their effectiveness in mitigating unfairness in the paper. Our experimental results, such as Table demonstrate the disparate impacts across demographic groups (e.g., gender, race) naturally exist in SSL. We are not aware of the misuse of our proposals, but we open discussions.
REFERENCES
Vibhu Agarwal, Tanya Podchiyska, Juan M Banda, Veena Goel, Tiffany Leung, Evan P Minty, Timothy E Sweeney, Elsic Gyang, and Nigam H Shah. Learning statistical models of phenotypes using noisy labeled training data. Journal of the American Medical Informatics Association, 23 (6):1166-117,, 2016.
Eugene Bagdasaryan, Omid Poursaeed, and Vitaly Shmatikov. Differential privacy has disparate im- pact on model accuracy. Advances Neural Information Processing Systems, 32:15479-15488. 2019.
Yingbin Bai, Erkun Yang, Bo Han, Yanhua Yang, Jiatong Li, Yinian Mao, Gang Niu, and Tongliang Liu. Understanding and improving early stopping for learning with noisy labels. Advances in Neural Information Processing Systems, 34, 2021.
Peter L Bartlett and Shahar Mendelson. Rademacher and gaussian complexities: Risk bounds and structural results. Journal of Machine Learning Research, 3(Nov):463 482. 2002.
Rachel KE Bellamy, Kuntal Dey, Michael Hind, Samuel C Hoffman, Stephanie Houde, Kalapriya
Kannan, Pranay Lohia, Jacquelyn Martino, Sameep Mehta, Aleksandra Mojsilovic, et al. fair- ness 360: An extensible toolkit for detecting, understanding, and mitigating unwanted algorithmic bias. arXiv preprint arXiv:1810.01944 2018.
David Berthelot, Nicholas Carlini, Ekin D Cubuk, Alex Kurakin, Kihyuk Sohn, Han Zhang, and Colin Raffel. Remixmatch: Semi-supervised learning with distribution alignment and augmenta- tion anchoring. arXiv preprint arXiv: 1911199911.99.. 2019a.
David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: A holistic approach to semi-supervised learning. Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019b.
Ming-Wei Chang, Lev-Arie Ratinov, Dan Roth, and Vivek Srikumar. Importance of semantic repre- sentation: Dataless classification. In Aaai, volume 2, pp. 830-835, 2008
Olivier Chapelle, Bernhard Scholkopf, and Alexander Zien. semi-supervised learning. MIT Press, 2006.
10

---

Published as a conference paper at ICLR 2022
Jiaao Chen, Zichao Yang, and Diyi Yang. Mixtext: Linguistieallyinnoomee interpolation of hidden space for sumi-super text classification. arXiv preprint vVXi::204.112239, 2020.
Mingda Chen, Qingming Tang, Karen Livescu, and Kevin Gimpel. Variational sequential labelers for semi-supervised learning. arXiv preprini 1rXXv:/906.005533 2019.
Hao Cheng, Zhaowei Zhu, Xingyu Li, Yifei Gong, Xing Sun, and Yang Liu. Learning with instance- dependent label noise: A sample sieve approach. In International Conference on Learning Rep- resentations, 2021.
Kevin Clark, Minh-Thang Luong. Christopher D Mamning, and Quoc V Le. Semi--upervised se- quence modeling with cross-view training. arXiv preprint arXiv:1809.083 2018.
Sam Corvett-Davies and Sharad Goel. The measure and mismeasure of faimess: A critical review of fair machine learning. arXiv preprint arXiv: 1808 00023, 2018.
Andrew M Dai and Quoc V Le. Semi-supervised sequence learning. Advances in neural information processing systems, 28:3079-30000 30890 2015.
Jacob Devlin, Ming- Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint WiXiv:181000480. 2018.
Emily Diana, Wesley Gill, Michael Kearns, Krishnaram Kenthapadi, Aaron Roth, and Saced Sharifi- Malvajerdi. Multiaccurate proxies for downstream fairness. arXiv preprint arXiv:2107.044/2 2021.
Ruoyuan Gao and Chirag Shah. Addressing bias and fairness in search systems. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2643-2646, 2021.
Elizabeth Gómez, Ludovico Boratto, and Maria Salamó. Disparate impact in item recommendation: A case of geographic imbalance. In European Conference on Information Retrieval, pp. 190-206. Springer, 2021.
Yves Grandvalet, Yoshua Bengio, al. Semi-supervised learning by entropy minimization. CAP, 367:281-296, 2005.
Lan-Zhe Guo, Zhen-Yu Yu Zhang, Yuan Jiang, Yu-Feng Li, and Zhi-Hua Zhou. Safe deep semi- supervised learning for unseen-class unlabeled data. In International Conference on Machine Learning, pp. 3897-3906. PMLR, 2020.
Suchin Gururangan, Tam Dang, Dallas Card, and Noah Smith. Variational pretraining for semi- supervised text classification. arXiv preprint arXiv: (eee:9906..2222 2019.
Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. Advances in neural information processing systems, 29:3315-3323, 2016.
Tatsunori Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. Fairness without demographics in repeated loss minimization. In International Conference on Machine Learning, pp. 1929-112999 1938. PMLR, 2018.
Sara Hooker, Aaron Courville, Gregory Clark, Yann Dauphin, and Andrea Frome. What do com- pressed deep neural networks forget? arXiv preprint arXiv: 19/11v99//.00 1.05248, 2019.
Sara Hooker, Nyalleng Moorosi, Gregory Clark, Samy Bengio, and Emily Denton. Characterising bias in compressed models. arXiv preprint arXiv:2010.00558, 2020.
Jeremy Howard and Sebastian Ruder. Universal language model fine-tuning for text classification. 2018
arXiv preprint
Zhuo Huang, Chao Xue, Bo Han, Jian Yang, and Chen Gong. Universal semi-supervised learning. Advances in Neural Information Processing Systems, 34, 2021.

---

Published as a conference paper at ICLR 2022
Ahmet Iscen, Giorgos Tolias, Yannis Avrithis, and Ondrej Chum. Label propagation for deep semi- supervised learning. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 5070-5079 2019.
Zhimeng Jiang, Xiaotian Han, Chao Fan, Zirui Liu, Na Zou, Ali Mostafavi, and Xia Hu. Fmp: Toward fair graph message passing against topology bias, 2022a.
Zhimeng Jiang, Xiaotian Han, Chao Fan, Fan Yang, Ali Mostafavi, and Xia Hu. Generalized de- mographic parity for group fairness. In International Conference Learning Representations, 2022b. URL /ttttp:://pperevviee net/ weet/Eorumidd=iiikI.. jye
Kaggle. Jigsaw Toxicity dataset: Toxic comment classification challenge. https://wwk /www.
kaggle. www.c//cc...... c/ gwwwwunnsnnnnnnnm
-ttpa/oo--ooa----ooooxxo-------- fiasifiiaiion
2018. Accessed: 2021-11-15.
Falaah Arif Khan, Eleni Manis, and Julia Stoyanovich. Translation tutorial: Fairness and friends. In Proceedings of the ACM Conference on Fairness, Accountability, and Transparency, 2021.
Michael P Kim, Amirata Ghorbani, and James Zou. Multiaceuracy: Black-box post-processing for fairness in classification. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pp. 247-254, 2019.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009.
Preethi Lahoti, Alex Beutel, Jilin Chen, Kang Lee, Flavien Prost, Nithum Thain, Xuezhi Wang, and Ed H Chi. Fairness without demographics through adversarially reweighted learning. arXiv preprint arXiv:2006. 131 14, 2020.
Samuli Laine and Timo Aila. Temporal ensembling for remi-supervised learning. arXiv preprint 1XX0:0600.02244 2016.
Dong-Hyun Lee al. Pseudo-label: The simple and efficient sumi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, volume 3, pp. 896, 2013.
Sheng Liu, Jonathan Niles-Weed, Narges Razavian, and Carlos Fernandez- Farly-learning regularization prevents memorization of noisy labels. Advances in neural information processing systems, 33:20331-20342, 2020.
Tongliang Liu and Dacheng Tao. Classification with noisy labels by importance reweighting. IEEE Transactions on pattern analysis and machine intelligence, 38(3):447-411, 2015.
Yang Liu. Understanding instance-level label noise: Disparate impacts and treatments. In Interna- tional Conference on Machine Learning, pp. 6725-6735. PMLR, 2021.
Yang Liu. Identifiability of label noise transition matrix. arXiv e-prints, pp. arXiv-2202, 2022.
Yang Liu and Hongyi Guo. Peer loss functions: Learning from noisy labels without knowing noise rates. In Proceedings of the 37th International Conference on Machine Learning. ICML '20, 2020.
Yang Liu and Jialu Wang. Can less be more? when increasing-te socreasing-tobbllanee balbnannn label noise rates sidered beneficial. Advances in Neural Information Processing Systems, 34, 2021.
Huixiang Luo, Hao Cheng, Yuting Gao, Ke Li, Mengdan Zhang, Fanxu Meng, Xiaowei Guo, Feiyue Huang, and Xing Sun. On the consistency training for open-set semi-supervised learning. arXiv preprint 1rXiv:2101.08/
2021.
Natalia Martinez, Martin Bertran, and Guillermo Sapiro. Minimax pareto fairness: A multi objective perspective. In International Conference Machine Learning, pp. 6755-6764, PMLR, 2020.
Yu Meng, Jiaming Shen, Chao Zhang, and Jiawei Han. Weakly-supervised neural text classifica- tion. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management, pp. 983 992, 2018
12

---

Published as a conference paper at ICLR 2022
Takeru Miyato, Andrew M Dai, and Ian Goodfellow. Adversarial training methods for semi- supervised text classification. arXiv preprint arXiv:1605.07725, 2016.
Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, and Shin Ishii. Virtual adversarial training: regularization method for supervised and semi cemi-supervised learning. IEEE transactions on pattern analysis and machine intelligence, 41(8):1979-199 2018.
Nagarajan Natarajan, Inderjit S Dhillon, Pradcep K Ravikumar, and Ambuj Tewari. Learning with noisy labels. In Advances in neural information processing systems, pp. 1196-1204, 2013.
Matthew Peters, Mark Neumann, Mohit lyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. Deep contextualized word representations, arXiv preprint 2018.
Reilly Raab and Yang Liu. Unintended selection: Persistent qualification rate disparities and inter- ventions. Advances in Neural Information Processing Systems, 34, 2021.
Antti Rasmus, Harri Valpola, Mikko Honkala, Mathias Berglund, and Tapani Raiko. Semi- supervised learning with ladder networks. arXiv preprint arXiv:/507.02667, 2015.
Devendra Singh Sachan, Manzil Zaheer, and Ruslan Salakhutdinov. Revisiting Istm networks for semi-supervised text classification via mixed objective function. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 1940-6948, 2019.
Mehdi Sajjadi, Mehran Javanmardi, and Tolga Tasdizen. Regularization with stochastic transfor- mations and perturbations for deep semi-supervised learning. Advances in neural information processing systems, 29:1163-1171, 2016.
Kihyuk Sohn, David Berthelot, Chun-Liang Li. Zizhao Zhang, Nicholas Carlini, Ekin D Cubuk, Alex Kurakin, Han Zhang, and Colin Raffel. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. arXiv preprint a0Xiv:2001.07685, 2020.
Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight-averaged consis- tency targets improve comi-supervised deep learning results. arXiv preprint 17Xiv:1703.0/780 2017.
Sahil Verma and Julia Rubin. Fairness definitions explained. 2018 ieeelacm international work- shop software fairness (fairware), pp. 1-7. IEEE, 2018.
Jialu Wang, Yang Liu, and Caleb Levy. Fair classification with geup-ddeenneent label noise. In Proceedings of the 2021 ACM Conference on Fairness, Accountabiliry, and Transparency, pp- 526-536, 2021a.
Jialu Wang, Yang Liu, and Xin Eric Wang. Are gender-neutrey queries really gating gender bias in image search. arXiv preprint 2rX1v:2109.05433, 2021b.
miti-
Jialu Wang, Yang Liu, and Xin Eric Wang. Assessing multilingual faimeess in pre-trained multi- modal representations. In the Findings of ACL 2022, Dublin, Ireland, May 2022. Association for Computational Linguistics.
Xiang Wang, Shiwei Zhang, Zhiwu Qing. Yuanjie Shao, Changxin Gao, and Nong Sang. Self- supervised learning for semi-supervised temporal action proposal. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1905-1914, 2021c
Zhuowei Wang, Jing Jiang, Bo Han, Lei Feng. Bo An, Gang Niu, and Guodong Long. Seminll: A framework of noisy-label learning by semi-supervised learning. arXiv preprint arXi::2012.0090 2020.
Jiaheng Wei and Yang Liu. When optimizing $fS-divergence is robust with label noise. In Interna- tional Conference on Learning Representations, 2021. URL
net/
forum? www.d:Weei.crvc.
13

---

Published as a conference paper at ICLR 2022
Jiaheng Wei, Zhaowei Zhu, Hao Cheng. Tongliang Liu, Gang Niu, and Yang Liu. Learning with noisy labels revisited: A study using real-world human annotations. In International Confer- Learning Representations, 2022. URL https:: http://wwwee/oouuueeemem
TRWA6PLJJ000
Qizhe Xie, Zihang Dai, Eduard Hovy. Minh-Thang Luong, and Quoc V Le. Unsupervised data augmentation. arXiv preprint 1904. 12848, 2019.
Yi Xu, Lei Shang, Jinxing Ye, Qi Qian, Yu-Feng Li, Baigui Sun, Hao Li, and Rong Jin. Dash: Semi-supervised learning with dynamic thresholding. In International Conference Machine Learning, pp. 11525-11536. PMLR, 2021.
Diyi Yang. Jiaao Chen, Zichao Yang, Dan Jurafsky, and Eduard Hovy. Let's make your request more persuasive: Modeling persuasive strategies via semi-supervised neural nets crowdfund- ing platforms. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, Volume (Long and Short Papers), pp. 5620-3630, 2019.
Zichao Yang. Zhiting Hu, Ruslan Salakhutdinov, and Taylor Berg-Kirkpatrick, Improved varia- tional autoencoders for text modeling using dilated convolutions. In International conference on machine learning, pp. 3881-3890. PMLR, 2017.
Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P Gummadi. Fair- ness constraints: Mechanisms for fair classification. In Artificial Intelligence and Statistics, pp- 962-970. PMLR, 2017.
Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz. mixup: Beyond em- pirical risk minimization. In International Conference on Learning Representations, 2018. URL
https::/openreview net http://wwweeeetoooneeeeelel Foo
Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text clas- sification. Advances in neural information processing systems, 28:649-657, 2015.
Zui-Hua Zhou. brief introduction weakly supervised learning. National science review, 5(1): 44-53, 2018.
Xiaojin Zhu, Zoubin Ghahramani, and John D Lafferty. Semi-supervine learning using gaussian fields and harmonic functions. In Proceedings of the 20th International conference on Machine learning (ICMI-03), pp. 912-919, 2003.
Zhaowei Zhu, Tongliang Liu, and Yang Liu. A second-order approach to learning with instance- dependent label noise. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10113-10123, 2021a.
Zhaowei Zhu, Yiwen Song, and Yang Liu. Clusterability as an alternative anchor points when learning with noisy labels. In International Conference on Machine Learning, pp. 12912-12923. PMLR, 2021b.
Zhaowei Zhu, Zihao Dong, and Yang Liu. Detecting corrupted labels without training a model to predict, 2022a.
Zhaowei Zhu, Jialu Wang, and Yang Liu. Beyond images: Label noise transition matrix estimation for tasks with lower-quality features. arXiv preprint arXiv:2202.01223, 2022b.
14

---

Published as a conference paper at ICLR 2022
Appendix
The Appendix is organized follows.
Section presents
presents the detailed derivations for generalization bounds.
Section
A.1
proves the upper bound for Term-1.
Section
A.2
proves the lower bound for Term-1.
Section
A.2
proves the upper bound for Term-2 (also for Theorem] I
Section
A.4
proves Lemma 2
Section
A.5
proves Corollary
Section C
presents more experimental settings and results.
A THEORETICAL RESULTS
A.1 TERM-1 UPPER BOUND
RD(f) - Rf(()
/ (P(F(X) # xI P(((X) P P(X) dx
/ (P((()) -100)-110 v(x) P(((X) = v(X) P(X) dx
2 -
1
X1X) +1F(((X)) f1x) P((()) # x1x)) P(X) ax
19
TD( dx
1
PPC = i|x) - P(Y = =(( ax
2
KE[K]
/ (X)P((X) dx
where in equality (a), given model f and feature X, we can treat (X(X) as Berouuli random variable such that
(x) +) P(((X) ÝIx) and (X) -) P() Ý(x).
Then according to the definition of total variation of two distributions, i.e.,
TD(PIIQ) = / dP
dQ
du,
du
du
can summarize the integrand as the total variation between (x) and (X
Inequality (b) holds due to the data processing incquality since the probabilities [P(Y f(X)),P( # f(X)) are generated by [P = i), Vi [K]], and the probabilities [P(Y
(())), + ((X)) are generated by [P((Y = i), Vi E [K]].
(X) is the accuracy of labels on feature X (in the cases specified in Lemma 2b
A.2 TERM-1 LOWER BOUND
Let e(X) = P(Y * YJX) be the feature-dependent error rate, ((X):=((((X)) = FJX) be accuracy of prediction f(X) on noisy dataset D. Note e(X) is independent of AX))). Denote their expectations (over X) by Fx [ex)),) Ä Fx (xx
We have:
15

---

Published as a conference paper at ICLR 2022
RD(s) - Rz(l)
- -/x (IX) V(X)) P(X)
S (PU(X)= = f1x) - ( P(X)
/ P(J(X) = - (15())=))(())=)-----)) P(((X) F(X)((X)
P(J(X)=
w.p.1
X()=Y(x)
+
- + FX -P(((X) XIf(X) (
+8.x) +
P(F(X)+ * (Ix))(()
w.p.
1--PP--IIN = -(x)(()
demoxed by x,(X)
denoted by e(X)
+ x (I-NU P(S()) + PIX)((X)
(X)(X)
- J -, - - -V1(())
P(((X) + FIX)((X)
- -1 (10X)= + PU(CK) # XIX)P(X)
S1
(X)(()
(0 (O.P(Y-
P(Y =
-1 1. 1-P + X10())
P((()) + FIX)P(X)
-PY X) due to independency
+ / (X)(X) / P(Y + F FXX)P(()
denoted by e(X) demoted by (X)
= -/x (())(24((8) 1)P(()
=(2Ā, 1)e.
A.3 TERM-2 UPPER BOUND
We adopt the technique prove Theorem T and Lemma 3 2020). We prove Theorem I as follows.
The proof follows Liu & Guo
Denote the expected error rate of classifier f on distribution by
RD( ()(1)(())))))
16

---

Published as a conference paper at ICLR 2022
Let fo denote the classifier trained by minimizing 0-1 loss with clean dataset D, i.e. fo arg min (f(f),
where
RD( = N Σ 1(f(En):V)).
REIN]
The Bayes optimal classifier is denoted by
f = arg min RD(s)-
The expected error given by fb is written as
R == RD(ft).
Denote by Y*|X= arg [[[]]
X) the Bayes optimal label clean distribution D. With
probability least 1 ö, have:
Ro(fD) RD(fE)
= -Ro(fo) RD(ft) (Rolfo)-Bee Ro(fo)) + (Ro(f6)- -Ro()
(a)
< + 2 max [Rn()) Ro(J)
/EF
(2g()
where inequality (a) holds since 1) RD(fD) achieves the minimum empirical risk according to its definition, thus Rp(fD)- RD(fs) s each of the following two terms will be no greater than the corresponding maximal gap max (fD(f) - Ro(f)). Inequality (b) holds due to the Hoeffding's 's inequality, i.e., with probability at least 1--. 8,
RD(f) RD(f)l log(2/6)
max
JET
2N
Noting (p(ff) = P(Y + Y*), we can prove Theorem
But substituting D for D, we can also
prove Lemma
A.4
PROOF FOR LEMMA 2
Proof: Note
P(Ý + XIX) = = =1- =1 - Σ
P(Ý (X)P(Y x(X).
KE[K]
and
((X) Σ
IP i| - IP(Y i|X)].
KE(K]
Assume P(Y x 1. We have P(Ý # Y 1 - PP x and
1 ((X) = 2
1-P(Ý = x
Σ
P( i|x)
=1 - P-P x'(x).
kE
A.5 PROOF FOR COROLLARY
BR(P) 17 +
(2/8)
P(Y
# 4
22/0(2/18)
TP(Y # Y*) NPL.
No
(2g(2
2208(2
No
NPp.
NP.)
A(Np,Nr.)
17

---

Published as a conference paper at ICLR 2022
B MATTHEW EFFECT
We further illustrate the Matthew effect when the initial labeled dataset is small and unbalanced. In Figure show the change of test accuracy due SSL.. We test two SSL algorithms CIFAR- 10: MixMatch (Berthelot et al. 2019b) (left panel) and UDA (Xie et al. 2019) (right panel), where each label class is treated as sub-population. There are 20 labeled instances in each of the first 5 classes (20 x 5), 10 labeled instances in each of the remaining classes (10 x 5). The remaining instances the first 5 classes and half of the remaining instances the other classes are used as unlabeled data, which ensures the unlabeled dataset has the same ratios among sub-popun sub-populations as the labeled one. Figure I shows two "poor" cob-populations get poorer by using MixMatch and "four" uub-populations get poorer by using UDA, which consolidate the observation of Matthew effect of SSL. Note we also find the observation "the poor getting poorer" does not always happen when there are more labeled instances and more balanced dataset. See more discussions in the next subsection.
B.1
MORE EXAMPLES
[20x5.10x5]
Size [20x5,10X5]
Get poorer
richer
Test accuraryy (%0 after SSL
Test accurary (50 after SSL
80
60
Acc After
Acc. Before Acc After
20-
pooter Get riceer
0-
20
40
60
20
300
40
50
60
Test acturacy (%) before S5L
Test accuracy (55 before SSL
Figure 4: Disparate impacts in the model accuracy of SSL. MixMatch (Berthelot al. 2019b (left panel) and UDA (Xie et al.) 2019) (right panel) are applied on CIFAR-I0 with 150 clean labels. Dashed line: The threshold indicating the test accuracy does not change before and after SSL. Red dots: Sub- Sub-populations that get poorer after SSL. Green dots: Sub-populations that get richer after SSL.
B.2 DISCUSSIONS
We do observe some "poor" classes may benefit from SSL, e.g., class "dog" in Figure Ia), and the dots that have initial test accuracies smaller than 0.3. We are aware of that, for most cases, we do only observe "the rich getting richer". Intuitively, the phenomenon that several poor classes also benefit from SSL is mainly due to two reasons:
scenario, "the rich getting richer" is likely to happen with "the poor get- ting poorer". However, benefiting from large capacity deep neural networks and powerful SSL algorithms, the available "resource" after SSL is usually greater than before SSL. Thus the
In resource-constraiingg
poor may not always get poorer.
Current popular SSL algorithms, such as MixMatch (Berthelot al. 2019b) and UDA (Xie al, 2019), are very powerful. It is reasonable to believe the re-labeling procedures in each epoch in these algorithms could reduce the disparate impact compared to the two-iteration scenario con- sidered in our theoretical analysis. This may also explain why "some poor classes adequately benefited." Although there are some exceptions, our Figure 2 and Figure 3 can still show that there are negligglie disparate impacts including both "the rich get richer" and "the poor get poorer" current state-of-the (SOTA) SSL algorithms. Besides, Figure B[a) and Table show that some classes have negative benefit ratios, indicating "the poor get poorer".
18

---

Published as a conference paper at ICLR 2022
Table 2: Dataset statistics and train/test splitting. The number of train and test samples means the number of data per class.
Datasets
Label Type
Numbee of classes
Number of train samples
Number of test samples
CITAR- 10
Image
10
5000
1000
CIFAR-I0U -00
Image
100
500
100
Yahoo! Answer
Question Answering
140000
5000
AG News
News
30000
1900
C MORE EXPERIMENTS
C.I. EXPERIMENT SETTINGS
classification labels In this section, show the statistics of datasets where we conducted the experiments to demonstrate the disparate impact on the explicit sub- populations (classification labels) in Table 2 In addition, the train/valid/test splitting in the image datasets (CIFAR-10 10 and CIFAR-100) is 45000:5000: 10000. As for the splitting in the text datsets (Yahoo! Answers and AG News), we follow the setting in (Chen et al. 2020).
Explicit sub-populations:
Implicit sub-populations: fine-categories under coarse classification labels In this section, provide a description about coarse labels in CIFAR-100 dataset when we conducted the experiments demonstrate the disparate impact on the implicit sub-populations: fine-categories under coarse classification labels. In CIFAR-100, its 100 classes can be categorized into 20 superclasses. Each image can have a "fine" label (the original class) and "coarse" label (the superclass), The list of superclasses in the CIFAR-100 is as follows: aquatic mammals, fish flowers orchids, food con- tainers bottles, fruit and vegetables, household electrical devices, household furniture bed, large carnivores, large man-made outdoor things, large natural outdoor scenes, large omnivores and her- bivores, medium-sized mammals, non-insect invertebrates, people, reptiles, small mammals, trees, vehicles 1, and vehicles 2.
Implicit sub-populations: demographic groups (race & gender) In this section, we describe the detailed data distribution before and after balancing methods about implicit uub-populations: demographic groups (race & gender on Jigsaw dataset). The identities in the race sub-population includes 'asian', 'black', 'latin', and 'white". In each identity, we consider different classification labels (toxic and non-toxic). Therefore, we have eight identities in the race rub-population. The numbers of samples on these eight race identities in the unbalanced Jigsaw dataset are 'asian (toxic): 466; black (toxic): 1673; latin (toxic): 282; white (toxic): 3254; asian (non-toxic): 633; black (non- toxic): 1879; latin (non-toxic): 511; white (non-toxic): 2652'. The numbers of samples on these eight race identities in the balanced Jigsaw dataset 'asian (toxic): 1419; black (toxic): 1419; latin (toxic): 1419; white (toxic): 1419; asian (non-toxic): 1419; black (non-toxic): 1419; latin (non-toxic): 1419; white (non-toxic): 1419'. Similar to race, also have eight identities in the gender sub-population by considering four different gender types which are 'male', 'female', 'male femal' and 'raansgnderr' and two classification labels (toxic and non-toxic). The numbers of samples on these eight gender identities in the unbalanced Jigsaw dataset are 'male (toxic): 5532; female (toxic): 4852; male female (toxic): 1739; transgender (toxic): 405; male (non-toxic): 5167: female (non-toxicd 4985; male female (non-toxic): 2002; transgender (non-toxic): 374'. The numbers of samples on these eight gender identities in the balanced Jigsaw dataset are 'male (toxic): 3142; female (toxic): 3142; male female (toxic): 3142; transgender (toxic): 3142; male (non-toxic); 3142; female (non-toxic): 3142; male female (non-toxic): 3142; transgender (non-toxic): 3142'. In either race or gender, Jigsaw dataset contains more than four identities, we set the number of identities to 4 due to the relatively small number of other identities. In addition, the ratio of train:valid:tess. on either race or gender sub-population case is 18:1:1'
19

---

Published as a conference paper at ICLR 2022
C.2
MORE EXPERIMENTAL RESULTS FOR DISPARATE IMPACT ACROSS DIFFERENT CLASSES ON AG NEWS
In this work, we also conducted experiments the AG News text classification dataset. Figure demonstrates the disparate impact on model accuracy across different classification labels on the AG News dataset.
MXText. labeled size 100x4
U3A. labeled size 100x4
MixText. labeled size 200
UDA. Jabeled size 200K4
1.4
1.4
Beneñ Balto Balto
Bott
Benere Rato Rato
DeneR Retoo
0.0
0.6
0.4
0.2
75
0.0
0.0
60
70
75
Accuracy
Accuracy
Accuracy
Accuracy
Figure 5: Benefit ratios of different class labels (y-axis) versus baseline accuracies before SSL (x- axis) on AG News
C.3 MORE EXPERIMENTAL RESULTS FOR DISPARATE IMPACT ACROSS DIFFERENT SUB-POPULATIONS ON JIGSAW (GENDER)
In this work, we also conducted experiments on the Jigsaw Toxicity text classification dataset with gender sub-populations. Figure 6 shows the disparate impact on model accuracy across different sub-populations (gender) on the Jigsaw Toxicity dataset.
MixText, labeled size 25x8
UDA, labeleed size 25x8
MixTest, labaled size 50x
UDA, labelee ze 50x8
Denel RRali
Rallo
Nabc
1.0
0.8
Bendr
0.6
0.6
0.6
0.6
0.4
0.4
0.4
0.4
0.2
0.2
0.2
0.2
0.0
65
70
0.0
50
80
62
60
70
0.0
62
64
70
72
0.0
60
70
80
Accuracy 200 clean labels
cleen labels
Accuracy 400
labels
with 400
labels
Figure 6: Benefit ratios of different gender attributes (y-axis) versus baseline accuracies before SSL
(r-axis) on Jigsaw.
C.4 MORE EXPERIMENTAL RESULTS OF "BALANCING" TREATMENT TO DISPARATE IMPACT
In this section, show more experimental results of "Balancing" treatment disparate impact described in Section 5.2 In Table we compare the changes of mean and standard deviation on the model accuracy and our proposed benefit ratio in three settings: (a) SSL with unbalanced data (b) SSL with balanced labeled datasets only (by reweighting) - (c) SSL with both balanced labeled & unlabeled datasets. Setting (c) is not practical due to the lack of labels in unlabeled data but it shows the best performance that this simple reweighting can achieve. Note that CIFAR-100 1:2 Accuracy denotes the accuracy of the case CIFAR-100 1:2, where CIFAR-I00 1:2 means 40 instances per class for the first 50 fine classes, 20 instances per class for the next 50 fine classes. Besides, Jigsaw (1:4:. means that the ratios of different sub- nub-populations sizes are 1.4:4.5:1:7.5. Table 3 shows reweighting unbalanced dataset helps improve the overall performance (mean) and reduce the disparate impacts (standard deviation).
C.5 MORE EXPERIMENTAL RESULTS OF "COLLECTING MORE LABELED DATA" TREATMENT TO DISPARATE IMPACT
In this subsection, we show more experimental results of "Collecting more labeled data" treatment to disparate impact described Section 5.2 Table demonstrates the changes of mean and standard
20

---

Published as a conference paper at ICLR 2022
Table 3: Balance samples with reweighting. We sample unbalanced CIFAR-100 (1:2 means 40 instances per class for the first 50 fine classes, 20 instances per class for the next 50 fine classes). Jigsaw is naturally unbalanced. For race, we consider asian, black, latin, and white identities. For gender, we consider male, female, male female, and transgender identities. The rounded ratios on number of samples between different identities race and gender are listed the table.
Datasets
Mean
Standard Deviation
CIFAR- 100
Accuracy
69.91 69.98
69.99
22.26
21.48
21.46
CIFAR-100 100 1:5 Accuracy
61.90 62.76
63.21
29.50
27.73
26.97
CIFAR- 100 1:2 Benefit Ratio
55.
55.19
56.09
20.85
20.16
+ 19.17
CIFAR-100 100 1:5 Benefit Ratio
44.67
49.47
47.40
37.07
33.82 - 32.188
Jigsaw (race) (1:755 Accuracy
66.71 + 67.25
- 67.88
25.64 + 21.33 - 17.84
Jigsaw (gender) (13.7:12.6:4.8:) Accuracy
66.03 66.74
67.17
12.90
9.75 + 8.68
Jigsaw (race) (1.4:4.5:1:7.55) Benefit Ratio
12.46
13.09
13.67
19.28
15.29
Jigsaw (gender) (13.7:12.6:4.:11) Benefit Ratio
11.90 12.18
12.54
88.52
50.67
34.24
deviation on model accuracy and benefit ratio with the varying number (grow exponentially from left to right) labeled data on CIFAR- R-100 and Jigsaw (race gender). shown in Table f the mean accuracy tends to be higher and the standard deviation becomes lower when more labeled data utilized to train the model, which shows the effectiveness of collecting more labeled data.
Table 4: Collect more data. a + b + c + d: stands for the change of mean or standard deviation with different sizes of labeled dataset. For CIFAR-10K 100, the sizes are 5 x 100, 10 x 100, 20 100, x 100. For Jigsaw (both race and gender), the sizes are 25 x 8, 50 x 8, 100 x 8, 150 x 8.
Datascts
Mean
Standard Deviation
CIFAR- 100 Accuracy
52.87
68.01
74.69
32.38
28.39
23.52
16.76
CIFAR- 100 Benefit Ratio
48.86
54.01
56.40
66.20
28.62
22.11
16.87
Jigsaw (race) Accuracy
64.88
67.88
72.25
73.50
29.55 7 17.84
- 9.93
6.86
Jigsaw (gender) Accuracy
64.50 + 67.17 + 73.50
+ 74,83
20.15 + 8.68 + 6.41
+ 5.99
Jigsaw (race) Benefit Ratio
12.09 + 13.67 - 14.52
- 15.12
18.29
+ 15.29
- 11.69
+ 7.64
Jigsaw (gender) Benefit Ratio
7.80 + 12.54 + 22.46
- 25.81
41.29
+ 34.24
> 27.30
> 13.08
21


Reviews:
[{"review_id": "lSmj2CUfjIe", "reviewer": "Reviewer_FRon", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\nStrengths:\n\n- The paper provides meaningful analyses to investigate the disproportionate performance of semi-supervised learning (SSL) on different subgroups. \n- The theoretical results show that the SSL frameworks possibly provide discriminative benefits to the subpopulations, which is consistent with the intuition. \n- In addition, experiments are performed with three representative SSL methods on several datasets for vision and natural language processing tasks. The experimental results confirm that the discrimination of the SSL frameworks can be a problem in various datasets.\n\nConcerns:\n\n- It would be better to clarify the term “disparate impact.” Disparate impact is used in many model fairness papers as a specific category of unfairness or a group fairness metric [1, 2]. Thus, if the paper used “disparate impact” as a broader term to explain the bias in SSL, it would be nice to explain the difference from the language in the previous papers.\n\n- In Figure 1, several “poor” classes seem to get enough accuracy gain. The intuition for “the rich get richer, and the poor get poorer” is convinced, but not sure why Figure 1 can be the best example for this message. For example, “bird, cat, and deer classes” in the left subfigure and “horse class” in the right subfigure initially have poor accuracy, but the performance adequately increases during SSL. Thus, I am not sure this figure clearly shows the main message. It would be nice if the paper clarifies these phenomena and connect them to the main takeaway.\n\n- It would be helpful if the paper gives more details on Section 3. For example, $\\eta$ is considered the disparity due to baseline and the major source of disparity, but it is not clearly explained. Also, the warm-up paragraph says the range of Term-1 becomes [1-2$\\epsilon$, 1]. However, if the range changes to the corresponding bound, $\\eta$ and $e$ should become 1, which is unclear. Such details might help to strengthen the paper.\n\n- When evaluating the training by the benefit ratio, how can we get the ideal accuracy (i.e., the first term of the denominator)? Since the ideal accuracy is defined as the accuracy from fully-supervised learning with the ground truth labeled dataset, it seems hard to get the value in the real scenarios. Also, is there any reason why the existing group fairness metrics (e.g., demographic parity and equalized odds) cannot be applied in the SSL scenario? These might provide other intuitions on the SSL frameworks.\n\n- Some experimental settings are unclear. For example, why are the experiments assumed that the labeled data are balanced, but the unlabeled data are imbalanced  (Figures 2 & 3)? It would be better to explain why the current setting is a suitable and natural choice for observing fairness in SSL.\n\nReferences\n\n[1] Zafar et al., Fairness Constraints: Mechanisms for Fair Classification, AISTATS 2017.\n\n[2] Bellamy et al., AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias, IBM J. Res. Dev. 2019.\n\n", "summary_of_the_paper": "This paper analyzes the discriminative performances of semi-supervised learning (SSL) on different subgroups, based on the key message that the already accurate group gets more benefits from the SSL. The paper first proposes a unified loss for analyzing the SSL frameworks. Then, the theoretical analyses show how the SSL produces discriminative performances on the different groups. In addition, a metric called benefit ratio is suggested to measure how much the learning benefits a certain subgroup. Experiments show overall consistent results with the main message from the theory. \n", "summary_of_the_review": "This paper provides useful analyses on the fairness issue in the semi-supervised setting. Although I have some comments as described above, the paper is overall well-written.\nThus, I believe that this paper is worth publishing after clarifying several parts.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "fsnzYkzWiEc", "reviewer": "Reviewer_fzF9", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper studies the sources of disparate impact on SSL from both theoretical and experimental demonstrations. Theoretically, the SSL problem was transformed to supervised learning with noisy labels by assigning pseudo labels. Both theoretical and empirical work show that the major source of the disparity comes from the low supervised error.\n\nThe paper has high ethical value. The theoretical proof and experiments are sound and support the conclusion of the paper.  I have a few minor comments for clarification. \n- Could you please clarify why figure3(a) shows a range of the benefit ratio, not the other figures?\n- How is the ideal ground truth label a_ideal(P) calculated in the experiments? How robust is that, and how will it affect the calculation of BR?\n", "summary_of_the_paper": "This paper presents the disparate impact on semi-supervised learning, where the sub-population that has a higher baseline accuracy tends to benefit more from SSL. The paper targets an important problem to ensure the fairness of machine learning algorithms. The authors also proposed a new metric benefit ratio to facilitate the evaluation of SSL. They also suggested two methods to mitigate the disparate, balancing the labeled data or collecting more labeled data.", "summary_of_the_review": "In general, the paper is well written. The task is novel and well supported by theoretical proof and experimental evaluations. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "4cRmwO9-V2C", "reviewer": "Reviewer_x3NJ", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Strength\n- Disparate impact on the constituent sub-population when training through SSL is an important problem. \n- Their theoretical analysis is sound. \n\n## Weakness  \n\n**Matthew effect: Rich getting richer and poor getting poorer**   \nI do not see the poor getting poorer part of their claim. \nIf poor gets poorer we should see the test accuracy of *dog* only decreasing with SSL in Fig 1 (b).\nBesides the 'dog' class, no other \"poor\" sub-population got worser with training in the same figure, for e.g. see *horse, ship, truck* classes. \nSimilarly, in Fig 2, 3, we should have seen negative BR for the poor population if poor gets poorer, but that is not the case in any of the sixteen plots; all the BR values are positve.\nAlthough, I agree that SSL does not benefit all the sub-population equally, and that \"rich\" sub-population gain owing to lower label noise in the induced labels on the unsupervised data.\n \n**What is the fairness concern here?**   \nSSL training benefit some/rich sub-population more than the others.\nAs a result, it improves the average accuracy without hurting the worst group accuracy (since BR>=0 for all groups). \nI would not call out SSL as unfair unless the poor population gets even worser, i.e. the rich groups get richer at the expense of poor groups.\nThe authors should be more formal when discussing the fairness aspect and explain depreciating population disparity in the context of a fairness metric. \n\n**Analysis**   \nIs not $\\bar\\eta'(\\hat f_{D_L})$ simply the risk with clean data, i.e. $R_{D}(\\hat f_{D_L})$? This is so because both measure the expected disagreement of the prediction to the label (in the confident case of Lemma 2).   \nReplacing $\\eta$ with $R_D$ in Thm 2, we have:\n\n$$\nR_D(\\hat f_{D_L\\cup D_U}) \\leq \\frac{N_U}{N}R_D(\\hat f_{D_L}) + \\text{<other terms>} \n$$\n\nWhen $N_U\\rightarrow \\infty, i.e. N_U\\approx N$, we have $R_D(\\hat f_{D_L\\cup D_U}) \\leq R_D(\\hat f_{D_L}) + \\Pr(\\tilde Y\\neq \\tilde Y^*)$, and as a result the upper bound for SSL case is worse than the usual case.\nThe generalization error bound of Thm 2 therefore cannot explain the improved performance on a subset of population. \nPlease clarify if I am missing something. \n\nSince the generalization upper bound of Thm. 2 is used for qualitative judgment on the effect of SSL, its sanity should be established I believe. \n\n**More technical details needed**   \nI found Section 5.2 hard to understand even after consulting the Appendix. \nDescription for Table 1 numbers is unclear. \nWhat does each of the settings mean? What is balanced: group population or label population? Where is the balancing, in labeled or unlabeled?\n\nSomewhere in the intro, the authors mention BR could even be greater than 1, and we also see that in some of the BR plots (Fig 2 a,b), for the richer classes. I do not understand how. \nIf the ideal population size is at least N_L+N_U, any BR value greater than one is only due to the empirical error in accuracy estimation. Please clarify. \n\n\n## Minor\n- In the last line of A.3, Hoeffding's ineq. does not give uniform deviation bound unlike what was stated there. \n- The upper bound of Thm 1 contains constants that go with the $1/\\sqrt{n}$ term, such as the maximum possible value of loss. But the order remains the same. \n- In Tab.3 of appendix B.4, need to explain what 1:2 Benefit ratio, 1:5 accuracy etc. mean.\n", "summary_of_the_paper": "The paper claims semi-supervised learning (SSL) does not uniformly improve performance on all the constituent (latent) sub-population of the group. \nMore alarmingly, they claim that the SSL improves the performance most on the groups that already fare well (before SSL) with deteriorates on groups that are already worse.\nThey argue intuitively and theoretically for the likely cause.\nWith text, image datasets and three SSL algorithms, they provide empirical evidence that can support their main claim.\nFinally, they briefly discuss strategies that could avoid such disparate impact of SSL. ", "summary_of_the_review": "The prime objective of the paper is to demonstrate the Matthew effect of SSL and its fairness consequences.  \nAs described in more detail above, I have concerns regarding both the aspects of their contribution.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "3bnYs_mQpa_", "reviewer": "Reviewer_fRGJ", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper aims to theoretically study the 'Matthew effect' (or disparate impact) of semi-supervised learning (SSL). The authors consider the consistency regularization-based approaches with either explicit or implicit pseudo labels. The authors first show that both types could be unified into one framework that is related to empirical risk minimization, where the pseudo labels are served as the noisy labels. Then the source of the Matthew effect is studied by deriving the generalization bound of the expected risk. A new metric named benefit ratio is further proposed to theoretically understand the Matthew effect of SSL with consideration of the derived generalization bounds. Experimental results shows that (1) popular SSL methods almost all suffer from this effect, and (2) the mitigation strategies guided by analyzing benefit ratio could reduce such Matthew effect.\n\nOverall, I think this paper is studying a very interesting and important problem. Theoretically analyzing the varied performance over different sub-populations may help design a better SSL method so that certain sub-populations are taken care of, which might also improve the overall performance. Please see my concerns regarding the submission as follows.\n\nConcerns\n- Usually in algorithmic fairness, disparate impact is more concerned about the acceptance rate rather than accuracy (though it may not be correct in legal support area). I think it would be better to have a few sentences to explicitly clarify the difference between the 'disparate impact' in this paper and the 'disparate impact' usually used in algorithmic fairness.\n\n- Some key references are missing, which study the varied classification accuracy among different sub-populations. Please see these papers at the end of my comment. And I believe the authors should discuss whether the existing works can be used in SSL setting. If yes, some analysis on their performance are needed; if not, some sentences on discussing why they cannot work are needed. My hunch is that, if we can view SSL as supervised learning with noisy labels, some of these works should work.\n\n- In RHS of Eq. (2), do we really need the average over the expectation? I thought the average should be absorbed into the expectation. I hope the authors could clarify it.\n\n- In proof of Lemma 3, the denominator in the square root has a coefficient 2 while the results in the main body does not have this coefficient, so at least one place is wrong.\n\n- Again, in proof of Lemma 3, is it possible for the authors to provide more details on how the first inequality (<= 0 + 2max|...|) is derived? \n\n- In figure 2, the correlation between baseline accuracy and benefit ratio is not clear for UDA on Yahoo dataset. Some analysis are needed. Also, the authors might want to correct their claim 'We consistently observe that the class labels with higher baseline accuracies have higher benefit ratios on both CIFAR-10 and Yahoo! Answers datasets.' because results of UDA on Yahoo does not reveal such observation.\n\n- Just out of my curiosity (this may be out of the scope of this paper), I wonder whether similar analysis can be generalized to SSL on graphs.\n\nReferences\n* Hashimoto, T., Srivastava, M., Namkoong, H., & Liang, P. (2018, July). Fairness without demographics in repeated loss minimization. In International Conference on Machine Learning (pp. 1929-1938). PMLR.\n* Kim, M. P., Ghorbani, A., & Zou, J. (2019, January). Multiaccuracy: Black-box post-processing for fairness in classification. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society (pp. 247-254).\n* Lahoti, P., Beutel, A., Chen, J., Lee, K., Prost, F., Thain, N., ... & Chi, E. H. (2020). Fairness without demographics through adversarially reweighted learning. arXiv preprint arXiv:2006.13114.\n* Diana, E., Gill, W., Kearns, M., Kenthapadi, K., Roth, A., & Sharifi-Malvajerdi, S. (2021). Multiaccurate Proxies for Downstream Fairness. arXiv preprint arXiv:2107.04423.\n* Martinez, N., Bertran, M., & Sapiro, G. (2020, November). Minimax pareto fairness: A multi objective perspective. In International Conference on Machine Learning (pp. 6755-6764). PMLR.", "summary_of_the_paper": "Please see main review.", "summary_of_the_review": "This paper studies an interesting problem. But some key references are missing, which should be discussed in related work and (potentially) evaluated as baseline methods.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper analyses theoretically the 'Matthew effect' (disparate impact) in the setting 
of the semi-supervised learning and its effect on fairness and performance. 

All reviewers agree that the paper deals with a very interesting topic and important problem. 
The paper discusses and presents a thorough and convincing analysis of the effect.
There were multiple concerns raised mainly around the lack of clarity at parts of the paper.
The authors did a very good job at resolving those and bringing their submission to a good standard. 
In the rebuttal I was glad to see a great dialog evolving among the authors and reviewers. 
I congraultate both sides. 

Happy to recommend acceptance.
Decision: Accept (Poster)