T HE R ICH G ET R ICHER : D ISPARATE I MPACT OF S EMI -S UPERVISED L EARNING Anonymous authors
Paper under double-blind review
A BSTRACT
Semi-supervised learning (SSL) has demonstrated its potential to improve the model accuracy for a variety of learning tasks when the high-quality supervised data is severely limited.
Although it is often established that the average accuracy for the entire population of data is improved, it is unclear how SSL fares with different sub-populations.
Understanding the above question has substantial fairness implications when these different sub-populations are deﬁned by the demographic groups we aim to treat fairly.
In this paper, we reveal the disparate impacts of deploying SSL: the sub-population who has a higher baseline accuracy without using
SSL (the “rich” sub-population) tends to beneﬁt more from SSL; while the subpopulation who suffers from a low baseline accuracy (the “poor” sub-population) might even observe a performance drop after adding the SSL module.
We theoretically and empirically establish the above observation for a broad family of SSL algorithms, which either explicitly or implicitly use an auxiliary “pseudo-label”.
Our experiments on a set of image and text classiﬁcation tasks conﬁrm our claims.
We discuss how this disparate impact can be mitigated and hope that our paper will alarm the potential pitfall of using SSL and encourage a multifaceted evaluation of future SSL algorithms.
I NTRODUCTION
The success of deep neural networks beneﬁts from large-size datasets with high-quality supervisions, while the collection of them may be difﬁcult due to the high cost of data labeling process e.g., medical diagnosis (Agarwal et al., 2016).
Practically, a much cheaper and easier solution is to obtain a small labeled dataset and a large unlabeled dataset, and apply semi-supervised learning (SSL).
Although the global model performance for the entire population of data is almost always improved by SSL, it is unclear how the improvements fare for different sub-populations.
Similar to the goal of multiaccuracy (Kim et al., 2019), the improvement of SSL should not only on the entire population, but also on different sub-populations.
Analyzing and understanding the above question will have substantial fairness implications especially when these sub-populations are deﬁned by the demographic groups e.g., race and gender.
Motivating Example Figure 1 shows the change of test accuracy for each class during SSL.
Each class denotes one sub-population and representative sub-populations are highlighted.
Figure 1 shows two important messages: in SSL, even with some state-of-the-art algorithms, 1) the “rich getting richer” is common and 2) the “poor getting poorer” possibly happens.
Speciﬁcally, the “rich” subpopulation, such as automobile that has a high baseline accuracy at the beginning of SSL, tends to be consistently beneﬁting during SSL.
But the “poor” sub-population, such as dog that has a low baseline accuracy, will remain a low-level performance as Figure 1(a) or even get worse performance as Figure 1(b).
This example shows disparate impact of accuracies for different sub-populations are common in SSL.
In extreme cases such as Figure 1(b), we observe the Matthew effect: the rich get richer and the poor get poorer.
See more discussions in Appendix B.
In this paper, we aim to understand the disparate impact of SSL from both theoretical and empirical aspects, and propose to evaluate SSL from a different dimension.
Speciﬁcally, based on classiﬁcations tasks, we study the disparate impact of model accuracies with respect to different sub-populations (such as label classes, feature groups and demographic groups) after applying SSL.
Different from traditional group fairness (Zafar et al., 2017; Bellamy et al., 2018) deﬁned over one
[Figure]
model, our study focuses on comparing the gap between two models (before and after SSL).
To this end, we ﬁrst theoretically analyze why and how disparate impact are generated.
The theoretical results motivate us to further propose a new metric, beneﬁt ratio , which evaluates the normalized improvement of model accuracy using SSL methods for different sub-populations.
The beneﬁt ratio helps us uncover the “Matthew effect” of SSL: a high baseline accuracy tends to reach a high beneﬁt ratio that may even be larger than 1 (the rich get richer), and a sufﬁciently low baseline accuracy may return a negative beneﬁt ratio (the poor get poorer).
Following the intuitions of Equal Odds (Hardt et al., 2016), a fair SSL algorithm should be with Equal Beneﬁt Ratio.
We then numerically demonstrate using beneﬁt ratios to evaluate SSL and discuss how two possible treatments, i.e., balancing the data and collecting more labeled data, might mitigate the disparate impact.
We hope our analyses and discussions could encourage future contributions to promote the fairness of SSL.
Our main contributions and ﬁndings are summarized as follows: 1) We propose an analytical framework that uniﬁes a broad family of SSL algorithms, which either explicitly or implicitly use an auxiliary “pseudo-label”.
The framework provides a novel perspective for analyzing SSL by connecting consistency regularization to the problem of learning with noisy labels.
Based on our framework, we further prove an upper bound for the generalization error of SSL and theoretically show the heterogeneous error of learning with the smaller scale supervised dataset is one primary reason for disparate impacts.
This observation reveals a “Matthew effect” of SSL. 3)
We contribute a novel metric called beneﬁt ratio to measure the disparate impact in SSL, the effectiveness of which is theoretically guaranteed by our proved generalization bounds.
We conduct experiments on both image and text classiﬁcation tasks to demonstrate the ubiquitous disparate impacts in SSL. 5)
We also discuss how the disparate impact could be mitigated.
1. R ELATED W ORKS
Semi-supervised learning SSL is popular in both computer vision and natural language processing communities (Dai & Le, 2015; Howard & Ruder, 2018;
Clark et al., 2018;
Yang et al., 2019; Sachan et al., 2019; Guo et al., 2020; Wang et al., 2021b;
Luo et al., 2021).
We brieﬂy review recent advances in SSL.
See comprehensive overviews by (Chapelle et al., 2006; Zhu et al., 2003) for traditional SSL methods.
A major line of recent works focus on assigning pseudo-labels generated by the supervised model to unlabeled dataset (Lee et al., 2013; Iscen et al., 2019; Berthelot et al., 2019b;a), where the pseudo-labels are often conﬁdent or with low-entropy (Sohn et al., 2020;
Zhou, 2018; Meng et al., 2018).
There are also many works on minimizing entropy of predictions on unsupervised data (Grandvalet et al., 2005) or regularizing the model consistency on the same feature with different data augmentations (Tarvainen & Valpola, 2017; Miyato et al., 2018; Sajjadi et al., 2016; Clark et al., 2018), including mixup (Zhang et al., 2018), adversarial noise (Miyato et al., 2016), and other advanced augmentations (Xie et al., 2019).
In addition to network inputs, augmentations can also be applied on some hidden layers (Chen et al., 2020).
Besides, some works conduct pre-training on the unlabeled dataset ﬁrst and then ﬁne-tune the model on the labeled dataset (Peters et al., 2018;
Devlin et al., 2018; Gururangan et al., 2019; Chen et al., 2019; Yang et al., 2017), or use ladder networks to combine unsupervised learning with supervised learning (Rasmus et al., 2015).
Disparate impact Over the past few years, the fairness in machine learning has been extensively studied.
Even models developed with the best intentions may introduce discriminatory biases.
Re- searchers in various ﬁelds have found the unfairness issues, e.g., model compression (Bagdasaryan et al., 2019), differential privacy (Hooker et al., 2019; 2020), recommendation system (G´omez et al., 2021), information retrieval (Gao & Shah, 2021), machine translation (Khan et al., 2021), and learning with noisy labels (Liu, 2021).
There also some treatments considering fairness without demographics (Lahoti et al., 2020; Diana et al., 2021; Hashimoto et al., 2018), minimax Pareto fairness (Martinez et al., 2020), multiaccuracy boosting (Kim et al., 2019), and fair classiﬁcation with label noise (Wang et al., 2021a).
Most of these works focus on supervised learning.
To our best knowledge, the unfairness issue in SSL methods has not been sufﬁciently explored.
P RELIMINARIES
We summarize the key concepts and notations as follows.
2. S UPERVISED C LASSIFICATION T ASKS
Consider a K -class classiﬁcation task given a set of N L labeled training examples denoted by D L :
= { ( x n , y n ) } n ∈
[ N L ] , where [ N L ] := { 1 , 2 , · · · , N L } is the set of indices, x n is a high-dimension feature, y n ∈
[ K ] stands for a particular label class.
The corresponding clean data distribution with full supervision is denoted by D .
Examples ( x n , y n ) are drawn according to random variables ( X, Y ) ∼ D .
The classiﬁcation task aims to identify a classiﬁer f that maps X to Y accurately.
Let 1 {·} be the indicator function taking value 1 when the speciﬁed condition is satisﬁed andotherwise.
Deﬁne the 0-1 loss as 1 ( f ( X ) , Y ) :
= 1 { f ( X ) (cid:54) =
Y } .
The optimal f is denoted by the
Bayes classiﬁer f ∗ =
arg min f E D [ 1 ( f ( X ) , Y )] .
One common choice is training a deep neural network (DNN) by minimizing the empirical risk: ˆ f = arg min f 1 N (cid:80) Nn =1 (cid:96) ( f ( x n ) , y n ) .
Notation (cid:96) ( · ) stands for the cross-entropy (CE) loss (cid:96) ( f ( x ) , y ) :
= − ln( f x [ y ]) , y ∈ [ K ] , where f x [ y ] denotes the y -th component of f ( x ) .
Notations f and f stand for the same model but different outputs.
Speciﬁcally, vector f ( x ) denotes the probability of each class that model f predicts given feature x .
The predicted label f ( x ) is the class with maximal probability, i.e., f ( x ) :
= arg max i ∈
[ K ] f x [ i ] .
We use notation f if we only refer to a model.
2. S EMI -S UPERVISED C LASSIFICATION T ASKS
In the semi-supervised learning (SSL) task, there is also an unlabeled (a.k.a. unsupervised) dataset
D U := { ( x n + N L , · ) } n ∈ [ N U ] drawn from D , while the labels are missing or unobservable.
Let N :=
N L + N U .
Denote the corresponding unobservable supervised dataset by D := { ( x n , y n ) } n ∈ [ N ] .
Compared with the supervised learning tasks, it is critical to leverage the unsupervised information in semi-supervised learning.
To improve the model generalization ability, many recent SSL methods rely on building consistency regularization terms with unlabeled data to ensure that the model output remains unchanged with randomly augmented inputs (Berthelot et al., 2019b; Xie et al., 2019;
Berthelot et al., 2019a; Sohn et al., 2020; Xu et al., 2021).
To better formulate the consistency regularization in these popular methods, we introduce soft/pseudo-labels y as follows.
Soft-labels To motivate
the deﬁnitions, we ﬁrst consider a special case y n , i.e., the one-hot encoding of y n , where each element writes as y n [ i ] =
1 { i = y n } .
More generally, we can extend the one-hot encoding to soft labels by requiring each element y
[ i ] ∈ [0 , 1] and (cid:80) i ∈
[ K ] y [ i ] = 1 .
The CE loss with soft y writes as (cid:96) ( f ( x ) , y ) :
= − (cid:80) i ∈ [ K ] y
[ i ] ln( f x
[ i ]) .
If we interpret y [ i ] = P ( Y = i ) as a probability and denote by D y the corresponding label distribution, the above CE loss with soft labels can be interpreted as the expected loss with respect to a stochastic label (cid:101) Y , i.e.,
[Equation]
Pseudo-labels In consistency regularization, by using model predictions, the unlabeled data will be assigned pseudo-labels either explicitly (Berthelot et al., 2019b) or implicitly (Xie et al., 2019), where the pseudo-labels can be modeled as soft-labels.
In the following, we review both the explicit and the implicit approaches and unify them in our analytical framework.
Consistency regularization with explicit pseudo-labels For each unlabeled feature x n , pseudolabels can be explicitly generated based on model predictions (Berthelot et al., 2019b).
The pseudolabel is later used to evaluate the model predictions.
To avoid trivial solutions where model predictions and pseudo-labels are always identical, independent data augmentations of feature x n are often generated for M rounds.
The augmented feature is denoted by x (cid:48) n,m :=
Augment ( x n ) , m ∈
[ M ] .
Then the pseudo-label y n in epoch- t can be determined based on M model predictions as y ( t ) n = Sharpen (cid:16) 1 M (cid:80) Mm
=1 ¯ f ( t ) ( x (cid:48) n,m ) (cid:17) , where model ¯ f ( t ) is a copy of the DNN at the beginning of epoch- t but without gradients.
The function Sharpen ( · ) reduces the entropy of a pseudo-label, e.g., setting to one-hot encoding e j ,
[Equation]
= arg max
[Equation]
[ K ] y ( t ) n (Sohn et al., 2020).
In epoch- t , with some consistency regularization loss (cid:96) CR ( · ) , the empirical risk using pseudo-labels is:
[Equation]
Consistency regularization with implicit pseudo-labels Consistency regularization can also be applied without specifying particular pseudo-labels, where a divergence metric between predictions on the original feature and the augmented feature is minimized to make predictions consistent.
For example, the KL-divergence could be applied and the data augmentation could be domain-speciﬁc (Xie et al., 2019) or adversarial (Miyato et al., 2018).
In epoch- t , the total loss is:
[Equation]
where λ balances the supervised loss and the unsupervised loss, x (cid:48) n :
= Augment ( x n ) stands for oneround data augmentation ( m = 1 following the previous notation x (cid:48) n,m ).
Without loss of generality, we use λ = 1 in our analytical framework.
Consistency regularization loss (cid:96) CR ( · )
In the above two lines of works, there are different choices of (cid:96) CR ( · ) , such as mean-squared error (cid:96) CR ( f ( x ) , y ) :
= (cid:107) f
( x ) − y (cid:107) 22 /K (Berthelot et al., 2019b;
Laine & Aila, 2016) or CE loss (Miyato et al., 2018;
Xie et al., 2019) deﬁned in Eq.
For a clean analytical framework, we consider the case when both supervised loss and unsupervised loss are the same, i.e., (cid:96) CR ( f ( x ) , y )
=
(cid:96) ( f ( x ) , y ) .
Note L 2 implies the entropy minimization (Grandvalet et al., 2005) when both loss functions are CE and there is no augmentation, i.e., x (cid:48) n = x n .
2. A NALYTICAL F RAMEWORK
We propose the following analytical framework to unify both the explicit and the implicit approaches.
With Eqn.
(1), the unsupervised loss term in the above two methods can be respectively written as:
N U N L + N U (cid:88) n = N L +
E
[Equation]
( f ( x n ) , (cid:101) Y )
[Equation]
and
[Equation]
Both unsupervised loss terms inform us that, for each feature x n , we compute the loss with respect to the reference label (cid:101) Y , which is a random variable following distribution D y ( t ) n or D ¯ f ( t ) ( x (cid:48) n ) .
Compared with the original clean label Y , the unsupervised reference label (cid:101)
Y contains label noise, and the noise transition depends on feature x n .
Speciﬁcally, we have
[Equation]
[ i ] ( Explicit ) or P
[Equation]
(cid:101)
[Equation]
t
[Equation]
[ i ] ( Implicit ) .
Then with model ¯ f ( t ) , we can deﬁne a new dataset with instance-dependent noisy reference labels: (cid:101)
D
= { ( x n , ˜ y n ) } n ∈ [ N ] , where ˜ y n = y n , ∀ n ∈
[ N L ] , and ˜ y n = y ( t ) n or ¯ f ( t ) x (cid:48) n , ∀ n ∈ { N L +1 , · · · , N } .
The uniﬁed loss is:
[Equation]
Therefore, with the pseudo-label as a bridge, we can model the popular SSL solution with consistency regularization as a problem of learning with noisy labels (in the challenging instancedependent label noise setting).
D ISPARATE I MPACTS OF SSL
To understand the disparate impact of SSL, we study how supervised learning with D L affects the performance of SSL with both D L and D U .
Intuition
The rich sub-population with a higher baseline accuracy (from supervised learning with
D L ) will have higher-quality pseudo-labels for consistency regularization, which helps to further improve the performance.
In contrast, the poorer sub-population with a lower baseline accuracy (again from supervised learning with D L ) can only have lower-quality pseudo-labels to regularize the consistency of unsupervised features.
With a wrong regularization direction, the unsupervised feature will have its augmented copies reach consensus on a wrong label class, which leads to a performance drop.
Therefore, when the baseline accuracy is getting worse, there will be more and more unsupervised features that are wrongly regularized, resulting in disparate impacts of model accuracies as shown in Figure 1.
In this section, we ﬁrst analyze the generalization error for supervised learning with labeled data, then extend the analyses to semi-supervised learning with the help of pseudo-labels.
Without loss of generality, we consider minimizing 0-1 loss 1 ( f ( X ) , Y ) with inﬁnite search space.
Our analyses can be generalized to bounded loss (cid:96) ( · ) and ﬁnite function space F following the generalization bounds that can be introduced using Rademacher complexity (Bartlett & Mendelson, 2002).
3. L EARNING WITH C LEAN D ATA
Denote the expected error rate of classiﬁer f on distribution D by R D ( f ) :=
E D [ 1 ( f ( X ) , Y )] .
Let ˆ f D denote the classiﬁer trained by minimizing 0-1 loss with clean dataset D , i.e., ˆ f D :=
arg min f ˆ R D
( f ) , where ˆ R D ( f ) :
= 1 N (cid:80) n ∈ [ N ] 1 ( f ( x n ) , y n ) .
Denote by Y ∗ | X := arg max i ∈
[ K ] P ( Y | X ) the Bayes optimal label on clean distribution D .
Theorem 1 shows the generalization bound in the clean case.
Replacing D and N with D L and N L we have:
Theorem 1 (Supervised error) .
With probability at least 1 − δ , the generalization error of supervised learning on clean dataset D L is upper-bounded by R D ( ˆ f D L ) ≤ (cid:113) 2log(2 /δ )
N L + P ( Y ∗ (cid:54) =
Y ) .
3. L EARNING WITH S EMI - SUPERVISED D ATA
We further derive generalization bounds for learning with semi-supervised data.
Following our analytical framework in Section 2.3, in each epoch- t , we can transform the semi-supervised data to supervised data with noisy supervisions by assigning pseudo-labels, where the semi-supervised dataset D L ∪ D U is converted to the dataset (cid:101) D given the model learned from the previous epoch.
Two-iteration scenario
In our theoretical analyses, to ﬁnd a clean-structured performance bound for learning with semi-supervised data and highlight the impact of the model learned from supervised data, we consider a particular two-iteration scenario where the model is ﬁrst trained to convergence with the small labeled dataset D L and get ˆ f D L , then trained on the pseudo noisy dataset (cid:101) D labeled by ˆ f D L .
Noting assigning pseudo labels iteratively may improve the performance as suggested by most SSL algorithms (Berthelot et al., 2019b; Xie et al., 2019), our considered twoiteration scenario can be seen as a worst case for an SSL algorithm.
Independence of samples in (cid:101) D Recall x (cid:48) n denotes the augmented copy of x n .
The N instances in (cid:101) D may not be independent since pseudo-label (cid:101)
y n comes from ˆ f D L ( x (cid:48) n ) , which depends on x n .
Namely, the number of independent instances N (cid:48) should be in the range of [ N L , N ] .
Intuitively, with appropriate noise injection or data augmentation (Xie et al., 2019; Miyato et al., 2018) to x n such that x (cid:48) n could be treated as independent of x n , the number of independent samples in (cid:101) D could be improved to N .
We consider the ideal case where all N instances are i.i.d.
in the following analyses.
By minimizing the uniﬁed loss deﬁned in Eq.
(2), we can get classiﬁer ˆ f (cid:101) D :=
arg min f ˆ R (cid:101) D ( f ) , where ˆ R (cid:101) D ( f ) :
= 1 N (cid:80) n ∈ [ N ] (cid:16)(cid:80)
i ∈
[ K ] ˜ y
[ i ] · 1 ( f ( x n ) , i ) (cid:17) .
The expected error given classiﬁer f is denoted by R (cid:101) D ( f ) :
= E (cid:101) D [ 1 ( f ( X ) , (cid:101) Y )] , where the probability density function of distribution (cid:101) D can be deﬁned as P ( X, (cid:101) Y ) ∼ (cid:101) D
( X = x n , (cid:101)
Y = i )
=
P ( X,Y ) ∼D ( X = x n ) · ˜ y n [ i ] .
Decomposition With the above deﬁnitions, the generalization error (on the clean distribution) of classiﬁer ˆ f (cid:101)
D could be decomposed as R D ( ˆ f (cid:101) D )
=
( R D ( ˆ f (cid:101) D ) − R (cid:101) D ( ˆ f (cid:101) D ))
(cid:124) (cid:123)(cid:122) (cid:125)
Term-
+ R (cid:101) D ( ˆ f (cid:101) D )
(cid:124) (cid:123)(cid:122) (cid:125)
Term- , where Termtransforms the evaluation of ˆ f (cid:101) D from clean distribution D to the pseudo noisy distribution (cid:101) D .
Term2 is similar to the generalization error in Theorem 1 but the model is trained and evaluated on noisy distribution (cid:101) D .
Both terms are analyzed as follows.
Upper and Lower Bounds for Term-1 Let η ( X ) :
= 12 (cid:80) i ∈
[ K ] | P ( (cid:101) Y = i | X ) − P
( Y = i | X ) | , e ( X ) :=
P ( Y (cid:54) = (cid:101) Y | X ) be the feature-dependent error rate, (cid:101)
A f ( X ) :
=
P ( f ( X ) =
(cid:101) Y | X ) be the accuracy of prediction f
( X ) on noisy dataset (cid:101) D .
Denote their expectations (over X ) by ¯ η :=
E X [ η ( X )] , ¯ e := E X
[ e ( X )] , (cid:101)
A f = E X
[ (cid:101)
A f ( X )] .
To highlight that ¯ η and ¯ e depends on the noisy dataset (cid:101) D labeled by ˆ f D L , we denote them as ¯ η ( ˆ f D L ) and ¯ e ( ˆ f D L ) .
Then we have:
Lemma 1 (Bounds for Term-1) .
(2 (cid:101)
A ˆ f (cid:102)
[Equation]
Note the upper bound builds on ¯ η ( ˆ f D L ) while the lower bound relates to ¯ e ( ˆ f D L ) .
To compare two bounds and show the tightness, we consider the case where Y | X is conﬁdent , i.e., each feature X belongs to one particular true class Y with probability 1 , which is generally held in classiﬁcation problems (Liu & Tao, 2015).
Lemma 2 shows η ( X ) =
e ( X ) in this particular case 1 .
Lemma 2 ( η vs. e ) .
For any feature X , if Y | X is conﬁdent, η ( X ) is the error rate of the model prediction on X
, i.e., ∃ i ∈
[ K ] : P ( Y = i | X ) = 1 ⇒ η ( X ) =
P ( (cid:101) Y (cid:54)
=
Y | X )
=
e ( X ) .
Upper bound for Term-2 Denote by (cid:101) Y ∗ | X :=
arg max i ∈
[ K ] P ( (cid:101) Y
= i | X ) the Bayes optimal label on noisy distribution (cid:101) D .
Following the proof for Theorem 1, we have:
Lemma 3 (Bound for Term-2) .
W. p. at least 1 − δ , R (cid:101) D
( ˆ f
[Equation]
[Equation]
f
[Equation]
) ≤
[Equation]
= (cid:101) Y ∗ ) .
Wrap-up Lemma 1 shows Term-1 is in the range of [(2 (cid:101)
A ˆ f (cid:102) D − 1)¯ e
( ˆ f D L ) , ¯ η ( ˆ f D L )] .
Lemma 2 informs us ¯ η
( ˆ f D L ) =
¯ e ( ˆ f D L ) in classiﬁcation problems where Y | X, ∀ X are conﬁdent.
With a well-trained model ˆ f (cid:101) D that learns the noisy distribution (cid:101) D , we have (cid:101)
A ˆ f (cid:102) D = 1 − (cid:15) and (cid:15) → 0 + , thus Term-1 is in the range of [(1 − 2 (cid:15) )
¯ η ( ˆ f D L ) , ¯ η ( ˆ f D L )] , indicating our bounds for Term-1 are tight .
Besides, noting
Lemma 3 is derived following the same techniques as Theorem 1, we know both bounds have similar tightness.
Therefore, by adding upper bounds for Term-1 and Term-2, we can upper bound the error of semi-supervised learning in Theorem 2, which has similar tightness to that in Theorem 1.
Theorem 2 (Semi-supervised learning error) .
Suppose the model trained with only D L has generalization error ¯ η (cid:48)
( ˆ f D L ) .
With probability at least 1 − δ , the generalization error of semi-supervised learning on datasets D L ∪ D U is upper-bounded by
[Equation]
where ¯ η ( ˆ f D L ) :
= ¯ η (cid:48) ( ˆ f D L ) · N U /N is the expected label error in the pseudo noisy dataset (cid:101) D .
Takeaways Theorem 2 explains how disparate impacts in SSL are generated.
• Supervised error ¯ η (cid:48)
( ˆ f D L ) : the major source of disparity.
The sub-population that generalizes well before SSL tends to have a lower SSL error.
Namely, the rich get richer.
• Sharpness of noisy labels P ( (cid:101) Y (cid:54) = (cid:101) Y ∗ ) : a minor source of disparity depends on how one processes pseudo-labels.
This term is negligible if we sharpen the pseudo-label.
• Sample complexity (cid:112) 2 log(2 /δ ) /N
: disparity depends on the number of instances N and their independence.
Recall we assume ideal data augmentations to get N in this term.
There will be much less than N i.i.d.
instances with poor data augmentations.
It would be a major source of disparity if data augmentations are poor.
B ENEFIT R ATIO : A E VALUATION M ETRIC
Figure 1 demonstrates that SSL leads to disparate impacts of different sub-populations’ accuracies, but it is still not clear how much that SSL beneﬁts a certain sub-population.
For example, a subpopulation with a low test accuracy after SSL may be beneﬁted much less from SSL since it is very hard to learn.
On the other hand, a sub-population with a high test accuracy after SSL may not get beneﬁts from SSL since it is easy to learn and the baseline accuracy is high enough.
To highlight the beneﬁt and evaluate the disparate impacts of SSL, we propose a new metric beneﬁt ratio .
Beneﬁt Ratio We now introduce the beneﬁt ratio, BR ( P ) , which depends on three classiﬁers: (baseline) supervised learning only with labeled data ˆ f D L , (ideal) fully-supervised learning ˆ f D , and
SSL ˆ f D L ∪ D U , to capture the relative accuracy improvement of sub-population P after SSL.
Note
BR ( P ) are deﬁned with three learning results: 1) the baseline accuracy from supervised learning with only a small labeled dataset, denoted by a baseline ( P ) ; 2) the accuracy from semi-supervised learning with both labeled dataset and unsupervised dataset, denoted by a semi ( P ) ; 3) the accuracy from fully-supervised learning with the ground truth labeled dataset, denoted by a ideal ( P ) .
Then:
[Equation]
Intuitively, a larger beneﬁt ratio indicates more beneﬁts from SSL.
We have BR ( P ) = 1 when SSL performs as well as the corresponding fully-supervised learning.
A negative beneﬁt ratio indicates the poor population is hurt by SSL such that a semi ( P ) < a baseline ( P ) , i.e., the poor get poorer as shown in Figure 1(b) (sub-population dog ).
Therefore, compared with the test accuracy, beneﬁt ratio shows more advantages in evaluating the disparate impacts of SSL.
Theoretical Explanation Recall we have error upper bounds for both supervised learning (Theorem 1) and semi-supervised learning (Theorem 2).
Both bounds have similar tightness thus we can compare them and get a proxy of beneﬁt ratio as (cid:99) BR ( P ) :
= sup( R D ( ˆ f DL ∪ DU |P ))
− sup( R D ( ˆ f DL |P )) sup
( R D ( ˆ f
DL ∪ DU |P ))
− sup( R D ( ˆ f D |P )) , where sup( · ) denotes the upper bound derived in Theorem 1 and Theorem 2, P is a sub-population, and D |P denotes the set of i.i.d.
instances in D that affect model generalization on P .
By assuming
P ( Y (cid:54) =
Y ∗ ) =
P ( (cid:101) Y (cid:54) = (cid:101) Y ∗ ) (both distributions have the same sharpness), we have:
Corollary 1.
The beneﬁt ratio proxy for P is (cid:99) BR ( P )
= 1 − ¯ η
( ˆ f DL |P ) ∆( N P ,N P L ) , where ∆( N P , N P L ) =
(cid:113) 2log(2 /δ )
N P L − (cid:113) 2log(2 /δ )
N P , N P and N P L are the numbers of instances in D |P and D L |P .
Corollary 1 shows the beneﬁt ratio is negatively related to the error rate of baseline models and positively related to the number of i.i.d.
instances after in SSL, which is consistent with our takeaways from Section 3.
It also informs us that SSL may have a negative effect on sub-population P if η ∆( N P ,N P L ) > 1 , i.e., the beneﬁt from getting more effective i.i.d.
instances is less than the harm from wrong pseudo-labels.
This negative effect indicates ”the poor get poorer”.
E XPERIMENTS
In this section, we ﬁrst show the existence of disparate impact on several representative SSL methods and then discuss the possible treatment methods to mitigate this disparate impact.
Settings Three representative SSL methods, i.e., MixMatch (Berthelot et al., 2019b), UDA (Xie et al., 2019), and MixText (Chen et al., 2020), are tested on several image and text classiﬁcation tasks.
For image classiﬁcation, we experiment on CIFAR-10 and CIFAR-100 datasets (Krizhevsky et al., 2009).
We adopt the coarse labels (20 classes) in CIFAR-100 for training and test the performance for each ﬁne label (100 classes).
Thus our training on CIFAR-100 is a 20 -class classiﬁcation task and each coarse class contains 5 sub-populations.
For text classiﬁcation, we employ three datasets: Yahoo!
Answers (Chang et al., 2008), AG News (Zhang et al., 2015) and Jigsaw
Toxicity (Kaggle, 2018).
Jigsaw Toxicity dataset contains both classiﬁcation labels (one text comment is toxic or non-toxic) and a variety of sensitive attributes (e.g., race and gender information) of the identities that are mentioned in the text comment, which are fairness concerns in real-world applications.
We defer more experimental details in Appendix C.
[Figure]
5. D ISPARATE I MPACT E XISTS IN P OPULAR SSL M ETHODS
We show that, even though the size of each sub-population is equal, disparate impacts exist in the model accuracy of different sub-populations, i.e., 1) explicit sub-populations such as classiﬁcation labels in Figure 2 (and Figure 4 in Appendix C.2), and 2) implicit sub-populations such as ﬁnecategories under coarse classiﬁcation labels in Figure 3a, demographic groups including race in
Figure 3b and gender (Figure 5 in Appendix C.3).
All the experiments in this subsection adopt both a balanced labeled dataset and a balanced unlabeled dataset.
Note we sample a balanced subset from the raw (unbalanced) Jigsaw dataset.
Other datasets are originally balanced.
Disparate impact across explicit sub-populations In this part, we show the disparate impact on model accuracy across different classiﬁcation labels on CIFAR-10, Yahoo!
Answers, and AG News datasets.
Detailed results on AG News can be found in Appendix C.2.
In Figure 2a and 2b, we show the beneﬁt ratios ( y -axis) versus baseline accuracies before SSL ( x -axis).
From left to right, we show results with different sizes of labeled data: 25 per class to 50 on CIFAR-10 and 100 per class to 200 on Yahoo!
Answers.
Figure 2a and 2b utilized two SSL methods respectively (CIFAR-10:
MixMatch & UDA; Yahoo!
Answers: MixText & UDA).We statistically observe that the class labels with higher baseline accuracies have higher beneﬁt ratios on both CIFAR-10 and Yahoo!
Answers datasets.
It means that “richer” classes beneﬁt more from applying SSL methods than the “poorer” ones.
We also observe that for some models with low baseline accuracy (left side of the x -axis), applying SSL results in rather low beneﬁt ratios that are close to 0.
Disparate impact across implicit sub-populations We demonstrate the disparate impacts on model accuracy across different sub-populations on CIFAR-100 (ﬁne-labels) and Jigsaw (race and gender) datasets.
In Figure 3, we can statistically observe the disparate impact across different subpopulations on both datasets for three baseline SSL methods.
Detailed results on Jigsaw with gender are shown in Appendix C.3.
We again observe very similar disparate improvements as presented in
Figure 2a and 2b - for some classes in CIFAR-100, this ratio can even go negative.
Note the disparate impact on the demographic groups in Jigsaw raises fairness concerns in real-world applications.
5. M ITIGATING D ISPARATE I MPACT
Our analyses in Section 3 and Section 4 indicate the disparate impacts may be mitigated by balancing the supervised error ¯ η ( ˆ f D L |P ) and the number of effective i.i.d.
instances for different populations.
We perform preliminary tests to check the effectiveness of the above ﬁndings in this subsection.
We hope our analyses and experiments could inspire future contributions to disparity mitigation in SSL.
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
[Table]
Balancing and Collecting more labeled data We ﬁrstly sample 400 i.i.d.
instances from the raw (unbalanced)
Jigsaw dataset and get the setting of Unbalanced (400).
To balance the supervised error and effective instances for different sub-populations, an intuitive method is to balance the labeled data by reweighting, e.g., if the size of two sub-populations is 1:2, we simply sample instances from two sub-populations with a probability ratio of 2:1 to ensure all sub-population have the same weights in each epoch during training.
Balance labeled (400) denotes only 400 labeled instances are reweighted.
Balance both (400) means both labeled and unlabeled instances are balanced.
Tableshows a detailed comparison on the beneﬁt ratios between different race identities of the Jigsaw dataset and their standard deviations, where the ﬁrst three rows denote the above three settings.
We can observe that both the standard deviation and the number of negative beneﬁt ratios become lower with more balanced settings, which demonstrates the effectiveness of the balancing treatment strategy, although there still exists a sub-population that has a negative beneﬁt ratio, indicating an unfair learning result.
To further improve fairness, as suggested in our analyses, we “collect” (rather add) another 400 i.i.d.
labeled instances ( 800 in total) from the Jigsaw dataset, and show the result after balancing both labeled and unlabeled data in the last row of Table 3 (Balance both (800)).
Both the standard deviation and the number of negative beneﬁt ratios can be further reduced with more labeled data.
More experimental results are on CIFAR-100 and Jigsaw (gender)
datasets shown in
Appendix C.4 (balancing) and Appendix C.5 (more data).
C ONCLUSIONS
We have theoretically and empirically shown that the disparate impact (the “richer” sub-populations get richer and the poor ones get poorer) exists universally for a broad family of SSL algorithms.
We have also proposed and studied a new metric beneﬁt ratio to facilitate the evaluation of SSL.
We hope our work could inspire more efforts towards mitigating the disparate impact and encourage a multifaceted evaluation of existing and future SSL algorithms.
E THICS S TATEMENT
In recent years, machine learning methods are widely applied in the real world across a broad variety of ﬁelds such as face recognition, recommendation system, information retrieval, medical diagnosis, and marketing decision.
The trend is likely to continue growing.
However, it is noted that these machine learning methods are susceptible to introducing unfair treatments and can lead to systematic discrimination against some demographic groups.
Our paper echoes this effort and looks into the disparate model improvement resulted from deploying a semi-supervised learning (SSL) algorithm.
Our work uncovers the unfairness issues in semi-supervised learning methods.
More speciﬁcally, we theoretically and empirically show the disparate impact exists in the recent SSL methods on several public image and text classiﬁcation datasets.
Our results provide understandings of the potential disparate impacts of an SSL algorithm and help raise awareness from a fairness perspective when deploying an SSL model in real-world applications.
Our newly proposed metric beneﬁt ratio contributes to the literature a new measure to evaluate the fairness of SSL.
Furthermore, we discuss two possible treatment methods: balancing and collecting more labeled data to mitigate the disparate impact in SSL methods.
We have carefully studied and presented their effectiveness in mitigating unfairness in the paper.
Our experimental results, such as Table 1, demonstrate the disparate impacts across demographic groups (e.g., gender, race) naturally exist in SSL.
We are not aware of the misuse of our proposals, but we are open to discussions.
Appendix
The Appendix is organized as follows.
• Section A presents the detailed derivations for generalization bounds.
– Section A.1 proves the upper bound for Term-1.
– Section A.2 proves the lower bound for Term-1.
– Section A.2 proves the upper bound for Term-2 (also for Theorem 1).
– Section A.4 proves Lemma 2.
– Section A.5 proves Corollary
1. • Section C presents more experimental settings and results.
A T HEORETICAL R ESULTS A. T ERM -1 U PPER B OUND
[Equation] [Equation]
= η where ( a ) holds due to the deﬁnition of total variation of two distributions, i.e.,
[Equation]
Inequality ( b ) holds due to the data processing inequality.
η ( X ) is the accuracy of labels on feature
X (in some cases).
A. T ERM -1 L OWER B OUND
Let e ( X ) :
= P ( Y (cid:54) = (cid:101) Y | X ) be the feature-dependent error rate, (cid:101)
A f ( X ) :
=
P ( f ( X ) =
(cid:101) Y | X ) be the accuracy of prediction f
( X ) on noisy dataset (cid:101) D .
Denote their expectations (over X ) by ¯ e :=
E X
[ e ( X )] , (cid:101)
A f = E X
[ (cid:101)
A f ( X )] .
We have:
[Equation]
We adopt the same technique to prove Theorem 1 and Lemma 3.
The proof follows Liu & Guo (2020).
We prove Theorem 1 as follows.
Denote the expected error rate of classiﬁer f on distribution D by
[Equation]
Let ˆ f D denote the classiﬁer trained by minimizing 0-1 loss with clean dataset D , i.e.,
[Equation]
where
[Equation]
The Bayes optimal classiﬁer is denoted by
[Equation]
The expected error given by f ∗D is written as
[Equation]
Denote by Y ∗ | X := arg max i ∈ [ K ] P ( Y | X ) the Bayes optimal label on clean distribution D .
With probability at least 1 − δ , we have:
[Equation]
N
[Equation]
where inequality (a) holds since 1)
ˆ R D
( ˆ f D ) achieves the minimum empirical risk according to its deﬁnition, thus ˆ R D ( ˆ f D ) − ˆ R D ( f ∗D ) ≤ 0 ; 2) each of the following two terms will be no greater than the corresponding maximal gap max f ∈F | ˆ R D ( f ) − R D
( f ) | .
Inequality (b) holds due to the
Hoeffding’s inequality, i.e., with probability at least 1 − δ ,
[Equation]
Noting R D ( f ∗D ) =
P ( Y (cid:54) =
Y ∗ )
, we can prove Theorem 1.
But substituting D for (cid:101) D , we can also prove Lemma 3.
A. P ROOF FOR L EMMA
Proof.
Note
[Equation]
i ∈
[Equation]
X ) .
and η ( X ) :
= 1 2 (cid:88) i ∈
[ K ] | P ( (cid:101)
[Equation]
= i | X ) − P ( Y = i | X ) | .
Assume P
( Y = i (cid:48) | X ) = 1 .
We have P ( (cid:101) Y (cid:54) =
Y | X )
= 1 − P ( (cid:101) Y
=
i (cid:48) | X ) and η ( X ) :
= 1
[Equation]
=
i
[Equation]
(cid:88)
[Equation]
A. P ROOF FOR C OROLLARY
[Equation]
B M ATTHEW E FFECT
We do observe some “poor” classes may beneﬁt from SSL, e.g., class “dog” in Figure 1(a).
We are aware of that, for most cases, we do only observe “the rich get richer”.
Intuitively, the phenomenon that several poor classes also beneﬁt from SSL is mainly due to two reasons:
[Table]
• In a resource-constrained scenario, “the rich get richer” is likely to happen with “the poor get poorer”.
However, beneﬁting from the large capacity of deep neural networks and powerful SSL algorithms, the available “resource” after SSL is usually greater than before SSL.
Thus the poor may not always get poorer.
• Current popular SSL algorithms, such as MixMatch (Berthelot et al., 2019b) and UDA (Xie et al., 2019), are very powerful.
It is reasonable to believe the re-labeling procedures in each epoch in these algorithms could reduce the disparate impact compared to the two-iteration scenario considered in our theoretical analysis.
This may also explain why “some poor classes are adequately beneﬁted.”
Although there are some exceptions, our Figure 2 and Figure 3 can still show that there are non-negligible disparate impacts including both “the rich get richer” and “the poor get poorer” in current state-of-the-art (SOTA) SSL algorithms.
Besides, Figure 3(a) and Table 1 also show that some classes have negative beneﬁt ratios, indicating “the poor get poorer”.
C M ORE E XPERIMENTS C. E XPERIMENT S ETTINGS
Explicit sub-populations: classiﬁcation labels In this section
, we show the statistics of datasets where we conducted the experiments to demonstrate the disparate impact on the explicit subpopulations (classiﬁcation labels) in Table 2.
In addition, the train/valid/test splitting in the image datasets (CIFAR-10 and CIFAR-100) is 45000:5000:10000.
As for the splitting in the text datsets (Yahoo!
Answers and AG News), we follow the setting in (Chen et al., 2020).
Implicit sub-populations: ﬁne-categories under coarse classiﬁcation labels In this section, we provide a description about coarse labels in CIFAR-100 dataset when we conducted the experiments to demonstrate the disparate impact on the implicit sub-populations: ﬁne-categories under coarse classiﬁcation labels.
In CIFAR-100, its 100 classes can be categorized into 20 superclasses.
Each image can have a “ﬁne” label (the original class) and a “coarse” label (the superclass).
The list of superclasses in the CIFAR-100 is as follows: aquatic mammals, ﬁsh , ﬂowers orchids, food containers bottles, fruit and vegetables, household electrical devices, household furniture bed, large carnivores, large man-made outdoor things, large natural outdoor scenes, large omnivores and herbivores, medium-sized mammals, non-insect invertebrates, people, reptiles, small mammals, trees, vehicles 1, and vehicles 2.
Implicit sub-populations: demographic groups (race & gender)
In this section, we describe the detailed data distribution before and after balancing methods about implicit sub-populations: demographic groups (race & gender on Jigsaw dataset) .
The identities in the race sub-population includes ‘asian’, ‘black’, ‘latin’, and ‘white’.
In each identity, we consider different classiﬁcation labels (toxic and non-toxic).
Therefore, we have eight identities in the race sub-population.
The numbers of samples on these eight race identities in the unbalanced Jigsaw dataset are ‘asian (toxic): 466; black (toxic): 1673; latin (toxic): 282; white (toxic): 3254; asian (non-toxic): 633; black (nontoxic): 1879; latin (non-toxic): 511; white (non-toxic): 2652’.
The numbers of samples on these eight race identities in the balanced Jigsaw dataset are ‘asian (toxic): 1419; black (toxic): 1419; latin (toxic): 1419; white (toxic): 1419; asian (non-toxic): 1419; black (non-toxic): 1419; latin (non-toxic): 1419; white (non-toxic): 1419’.
Similar to race, we also have eight identities in the gender sub-population by considering four different gender types which are ‘male’, ‘female’, ‘male femal’ and ‘transgender’ and two classiﬁcation labels (toxic and non-toxic).
The numbers of samples on these eight gender identities in the unbalanced Jigsaw dataset are ‘male (toxic): 5532; female
(toxic): 4852; male female (toxic): 1739; transgender (toxic): 405; male (non-toxic): 5167; female (non-toxic): 4985; male female (non-toxic): 2002; transgender (non-toxic): 374’.
The numbers of samples on these eight gender identities in the balanced Jigsaw dataset are ‘male (toxic): 3142; female (toxic): 3142; male female (toxic): 3142; transgender (toxic): 3142; male (non-toxic): 3142; female (non-toxic): 3142; male female (non-toxic): 3142; transgender (non-toxic): 3142’.
In either race or gender, Jigsaw dataset contains more than four identities, we set the number of identities to 4 due to the relatively small number of other identities.
In addition, the ratio of ‘train:valid:test’ on either race or gender sub-population case is ‘8:1:1’.
C. M ORE E XPERIMENTAL R ESULTS FOR D ISPARATE I MPACT ACROSS D IFFERENT C LASSES ON AG N EWS
In this work, we also conducted experiments on the AG News text classiﬁcation dataset.
Figuredemonstrates the disparate impact on model accuracy across different classiﬁcation labels on the AG
News dataset.
[Figure]
M ORE E XPERIMENTAL R ESULTS FOR D ISPARATE I MPACT ACROSS D IFFERENT S UB - POPULATIONS ON J IGSAW ( GENDER )
In this work, we also conducted experiments on the Jigsaw Toxicity text classiﬁcation dataset with gender sub-populations.
Figure 5 shows the disparate impact on model accuracy across different sub-populations (gender) on the Jigsaw Toxicity dataset.
[Figure]
R ESULTS OF “B ALANCING ” T REATMENT TO D ISPARATE I MPACT
In this section, we show more experimental results of “Balancing” treatment to disparate impact described in Section 5.2.
In Table 3, we compare the changes of mean and standard deviation on the model accuracy and our proposed beneﬁt ratio in three settings: (a) SSL with unbalanced data → (b)
SSL with balanced labeled datasets only (by reweighting)
→ (c) SSL with both balanced labeled & unlabeled datasets.
Setting (c) is not practical due to the lack of labels in unlabeled data but it shows the best performance that this simple reweighting can achieve.
Note that CIFAR-100 1:2 Accuracy denotes the accuracy of the case CIFAR-100 1:2 , where CIFAR-100 1:2 means 40 instances per class for the ﬁrst 50 ﬁne classes, 20 instances per class for the next 50 ﬁne classes.
Besides, Jigsaw (1.4:4.5:1:7.5) means that the ratios of different sub-populations’ sizes are 1.4:4.5:1:7.5.
Tableshows reweighting unbalanced dataset helps improve the overall performance (mean) and reduce the disparate impacts (standard deviation).
Jigsaw is naturally unbalanced.
For race, we consider asian, black, latin identities.
For gender, we consider male, female, male female, and transgender identities.
The rounded ratios on the number of samples between different identities in race and gender are listed in the table.
[Table]
C. M ORE E XPERIMENTAL R ESULTS OF “C OLLECTING M ORE L ABELED D ATA ” T REATMENT TO D ISPARATE I MPACT
In this subsection
[Table]
we show more experimental results of “Collecting more labeled data” treatment to disparate impact described on Section 5.2.
Table 4 demonstrates the changes of mean and standard deviation on model accuracy and beneﬁt ratio with the varying number (grow exponentially from left to right) of labeled data on CIFAR-100 and Jigsaw (race & gender).
As shown in Table 4, the mean accuracy tends to be higher and the standard deviation becomes lower when more labeled data is utilized to train the model
[Table]
which shows the effectiveness of collecting more labeled data
[Table] [Table]