On Non-Random Missing Labels in Semi-Supervised Learning

1 INTRODUCTION . Semi-supervised learning ( SSL ) aims to alleviate the strong demand for large-scale labeled data by leveraging unlabeled data ( Zhu , 2008 ; Yang et al. , 2021 ) . Prevailing SSL methods first train a model using the labeled data , then uses the model to impute the missing labels with the predicted pseudolabels for the unlabeled data ( Van Buuren , 2018 ) , and finally combine the true- and pseudo-labels to further improve the model ( Sohn et al. , 2020 ; Berthelot et al. , 2019a ) . Ideally , the “ improve ” is theoretically guaranteed if the missing label imputation is perfect ( Grandvalet & Bengio , 2005 ) ; otherwise , imperfect imputation causes the well-known confirmation bias ( Arazo et al. , 2019 ; Sohn et al. , 2020 ) . In particular , the bias is even more severe in practice because the underlying assumption that the labeled and unlabeled data are drawn from the same distribution does not hold . We term this scenario as label Missing Not At Random ( MNAR ) ( Hernán & Robins , 2020 ) , as compared to the naı̈ve assumption called label Missing Completely At Random ( MCAR ) . MNAR is inevitable in real-world SSL due to the limited label annotation budget ( Rosset et al. , 2005 ) —uniform label annotations that keep MCAR are expensive . For example , we usually deploy low-cost data collecting methods like crawling social media images from the Web ( Mahajan et al. , 2018 ) . However , the high-quality labels appear to be severely imbalanced over classes due to the imbalanced human preferences for the “ class ” ( Misra et al. , 2016 ; Colléony et al. , 2017 ) . For example , people are more willing to tag a “ Chihuahua ” rather than a “ leatherback ” ( a huge black turtle ) since the former is more lovely and easier to recognize . Thus , the imputation model trained by such biased supervision makes the model favoring “ Chihuahua ” rather than “ leatherback ” . The “ Chihuahua ” -favored pseudo-labels , in turn , negatively reinforce the model confidence in the false “ Chihuahua ” belief and thus exacerbate the confirmation bias . We design an experiment on CIFAR-10 to further illustrate how existing state-of-the-art SSL methods , e.g. , FixMatch ( Sohn et al. , 2020 ) , fail in the MNAR scenario . As shown in Figure 1 ( a ) , the overall training data is uniformly distributed over classes , but the labeled data is long-tailed distributed , which simulates the imbalanced class popularity . Trained on such MNAR training data , FixMatch even magnifies the bias towards the popular classes and ignores the rare classes ( Fig- ure 1 ( b ) ) . We point out that the devil is in the imputation . As shown in Figure 1 ( c ) , the confidence scores of FixMatch for popular classes are much higher than those for rare classes ( i.e. , averagely > 0.95 vs. < 0.05 ) . Since FixMatch adopts a fixed threshold for the imputation samples selection , i.e. , only the samples with confidence larger than 0.95 are imputed , FixMatch tends to impute the labels of more samples from the popular classes , which still keeps the updated labels with the newly added pseudo-labels long-tailed . As a result , FixMatch is easily trapped in MNAR . In fact , MNAR has attracted broad attention in statistics and Inverse Propensity Weighting ( IPW ) is one of the commonly used tools to tackle this challenge ( Seaman & White , 2013 ; Jones et al. , 2006 ) . IPW introduces a weight for each training sample via its propensity score , which reflects how likely the label is observed ( e.g. , its popularity ) . In this way , IPW makes up a pseudo-balanced dataset by duplicating each labeled data inversely proportional to its propensity—less popular samples should draw the same attention as the popular ones—a more balanced imputation . To combine the IPW true labels and imputed missing labels , a Doubly Robust ( DR ) estimator is used to guarantee a robust SSL model if either IPW or imputation is unbiased ( Seaman & Vansteelandt , 2018 ; Vansteelandt et al. , 2010 ) . However , the above framework overlooks the role of “ class ” in causing the MNAR problem of SSL ( Section 3 ) , and we therefore propose a unified Class-Aware Doubly Robust ( CADR ) framework to address this challenge ( Section 4 ) . CADR removes the bias from both the supervised model training and the unlabeled data imputation ends . For the former end , we propose Class-Aware Propensity ( CAP ) that exploits the unlabeled data to train an improved classifier using the biased labeled data ( Section 4.1 ) . We use an efficient moving averaging strategy to estimate the propensity . For the latter end , we design Class-Aware Imputation ( CAI ) that dynamically adjusts the pseudolabel assignment threshold with class-specific confidence scores to mitigate the bias in imputation ( Section 4.2 ) . Intuitively , CAI lowers the threshold requirement of rare class samples to balance with frequent class samples . Experiments on several image classification benchmarks demonstrate that our CADR framework achieves a consistent performance boost and can effectively tackle the MNAR problem . Besides , as MCAR is a special case of MNAR , our method maintains the competitive performance in the conventional SSL setting . The key contributions of this work are summarized as follows : • We propose a realistic and challenging label Missing Not At Random ( MNAR ) problem for semisupervised learning , which is not extensively studied in previous work . We systematically analyze the bias caused by non-random missing labels . • We proposes a unified doubly robust framework called Class-Aware Doubly Robust ( CADR ) estimator to remove the bias from both the supervised model training end—by using Class-Aware Prospesity ( CAP ) , and the unlabeled data imputation end—by using Class-Aware Imputation ( CAI ) . • Our proposed CADR achieves competitive performances in both MNAR and conventional label Missing Completely At Random ( MCAR ) . 2 RELATED WORKS . Missing Not At Random . Missing data problems are ubiquitous across the analysis in social , behavioral , and medical sciences ( Enders , 2010 ; Heckman , 1977 ) . When data is not missing at random , estimations based on the observed data only results in bias ( Heckman , 1977 ) . The solutions are three-fold : 1 ) Inverse probability weighting ( IPW ) assigns weights to each observed datum based on the propensity score , i.e. , the probability of being observed . A missing mechanism-based model is used to estimate the sample propensity , such as a logistic regression model ( Rosset et al. , 2005 ; Wang et al. , 1997 ) or a robit regression model ( Kang & Schafer , 2007 ; Liu , 2005 ) . 2 ) Imputation methods aim to fill in the missing values to produce a complete dataset ( Kenward & Carpenter , 2007 ; Little & Rubin , 2014 ) . An imputation model is regressed to predict the incomplete data from the observed data in a deterministic ( Kovar & Whitridge. , 1995 ) or stochastic ( Lo et al. , 2019 ) way . 3 ) As the propensity estimation or the regressed imputation are easy to be biased , doubly robust estimators propose to integrate IPW and imputation with double robustness ( Seaman & Vansteelandt , 2018 ; Vansteelandt et al. , 2010 ; Jonsson Funk et al. , 2011 ) : the capability to maintain unbiased when either the propensity or imputation model is biased . In this work , as the image labels are missing not at random , we design the class-aware propensity and imputation considering the causal role of label classes in missing , avoiding the uncertain model specification and generating unbiased estimation . These two class-aware modules can be naturally integrated into a doubly robust estimator . Semi-Supervised Learning ( SSL ) . It aims to exploit unlabeled data to improve the model learned on labeled data . Prevailing SSL methods ( Sohn et al. , 2020 ; Berthelot et al. , 2019b ; a ) share a similar strategy : training a model with the labeled data and generating pseudo-labels for unlabeled data based on the model predictions . Pseudo-labeling methods ( Lee , 2013 ; Xie et al. , 2020 ; Rizve et al. , 2021 ) predict pseudo-labels for unlabeled data and add them to the training data for re-training . Consistency-regularization methods ( Sajjadi et al. , 2016 ; Laine & Aila , 2016 ; Berthelot et al. , 2019b ) apply a random perturbation to an unlabeled image and then use the prediction as the pseudo-label of the same image under a different augmentation . Recent state-of-the-art SSL methods ( Sohn et al. , 2020 ; Berthelot et al. , 2019a ) combine the two existing techniques and predict improved pseudolabels . However , when labels are missing at random , these methods can be inefficient as the model learned on labeled data is biased and significantly harms the quality of pseudo labels . Though some works also notice the model bias ( Wei et al. , 2021 ; Kim et al. , 2020 ) , they neglect the causal relationship between the bias and the missing label mechanism in SSL , which is systematically discussed in our work . 3 MISSING LABELS IN SEMI-SUPERVISED LEARNING . In semi-supervised learning ( SSL ) , the training dataset D is divided into two disjoint sets : a labeled dataset DL and an unlabeled data DU . We denote DL as { ( x ( i ) , y ( i ) ) } NLi=1 ( usually NL≪N ) where x ( i ) ∈ Rd is a sample feature and y ( i ) ∈ { 0 , 1 } C is its one-hot label over C classes , and DU as { ( x ( i ) ) } Ni=NL+1 where the remaining N−NL labels are missing in DU . We further review SSL as a label missing problem , and define a label missing indicator set M with m ( i ) ∈ { 0 , 1 } , where m ( i ) =1 denotes the label is missing ( i.e. , unlabeled ) and m ( i ) =0 denotes the label is not missing ( i.e. , labeled ) . In this way , we can rewrite the dataset as D= ( X , Y , M ) = { ( x ( i ) , y ( i ) , m ( i ) ) } Ni=1 . Traditional SSL methods assume that M is independent with Y , i.e. , the labels are Missing Completely At Random ( MCAR ) . Under this assumption , prevailing SSL algorithms can be summarized as the following multi-task optimization of the supervised learning and the unlabeled imputation : θ̂ = argmin θ ∑ ( x , y ) ∈DL Ls ( x , y ; θ ) + ∑ x∈DU Lu ( x ; θ ) , ( 1 ) where Ls and Lu are the loss function designed separately for supervised learning on labeled data ( M =0 ) and regression imputation on unlabeled data ( M =1 ) . In general , SSL methods first train a model with parameters θ̂ using Ls on DL , then impute the missing labels ( Van Buuren , 2018 ) by predicting the pseudo-labels ŷ = f ( x ; θ̂ ) , and finally combine the true- and pseudo-labels to further improve the model for another SSL loop , e.g. , another optimization iteration of Eq . ( 1 ) . The implementation of Ls and Lu are open . For example , Ls is normally the standard cross-entropy loss , and Lu can be implemented as squared L2 loss ( Berthelot et al. , 2019b ) or cross-entropy loss ( Berthelot et al. , 2019a ; Sohn et al. , 2020 ) . As you may expect , the key to stable SSL methods is the unbiasedness of the imputation model . When M is independent with Y , we have P ( Y |X=x , M=0 ) = P ( Y |X=x ) , and thus E [ ŷ ] = E [ y|θ̂ ] = ∑ ( x , y ) ∈DL y · P ( y|x ) = ∑ ( x , y ) ∈D y · P ( y|x , M = 0 ) = ∑ ( x , y ) ∈D y · P ( y|x ) = E [ y ] , ( 2 ) which indicates that the model is an ideally unbiased estimator , and the imputed labels are unbiased . However , such MCAR assumption is too strong and impractical . For example , annotators may tend to tag a specific group of samples due to their preferences or experiences . This motivates us to ask the following realistic but overlooked question : what if M is dependent with Y , i.e. , the labels are missing not at random ( MNAR ) ? In this case , we have P ( Y |X = x , M = 0 ) ̸= P ( Y |X = x ) , and thus E [ ŷ ] = ∑ ( x , y ) ∈D y · P ( y|x , M = 0 ) = ∑ ( x , y ) ∈D y · P ( y|x ) · P ( M = 0|x , y ) P ( M = 0|x ) ̸= ∑ ( x , y ) ∈D y · P ( y|x ) = E [ y ] . ( 3 ) We can see it is no longer unbiased , further leading to biased imputed labels . As a result , the bias will be increasingly enhanced with the joint optimization loop in Eq . ( 1 ) moving forward . This analysis indicates that previous SSL methods are not applicable for the MNAR problem as they overlook the role of “ class ” Y in causing the non-randomness of M . In the following , we will introduce our Class-Aware Doubly Robust framework that targets at an unbiased estimator .

Summary: The paper addresses the semi-supervised learning problem under the setting of imbalanced label data by extending FixMatch with a class-aware method to reweight the loss and an adaptive thresholding method to select examples with pseudo labels. The two methods are termed Class-Aware Propensity (CAP) and Class-Aware Imputation (CAI), respectively. The paper also proposes the Class-Aware Doubly Robust (CADR) estimator by combining the two methods in the double robustness framework. Experiments are performed on Cifar-10/100, STL-10, and mini-ImageNet.