Positive-Unlabeled Learning with Uncertainty-aware Pseudo-label Selection

1 INTRODUCTION . Many real-world applications involve positive and unlabeled ( PU ) datasets in which only some of the data is labeled positive while the majority is unlabeled and contains both positives and negatives . PU learning aims to learn a binary classifier in this challenging setting without any labeled negative examples . Learning from PU data can reduce deployment costs in many deep learning applications that otherwise require annotations from experts such as medical image diagnosis ( Armenian and Lilienfeld , 1974 ) and protein function prediction ( Gligorijević et al. , 2021 ) , and it can even enable applications in settings where the measurement technology itself can not detect negative examples ( Purcell et al. , 2019 ) . Some recent approaches such as unbiased PU ( Du Plessis et al. , 2014 , uPU ) and non-negative PU ( Kiryo et al. , 2017 , nnPU ) formulate this problem as cost-sensitive learning . Others approach PU learning as a two-step procedure first identifying and labeling some reliable negative examples , and then re-training the model based on this newly constructed labeled dataset ( Bekker and Davis , 2020 ) . These approaches show similarities with pseudo-labeling in semi-supervised classification settings ( Lee , 2013 ) . Such pseudo-labeling techniques are however especially vulnerable to incorrectly assigned labels of the selected examples as these errors will propagate and magnify in the retrained model , resulting in a negative feedback loop . Worse yet , since the true labels are unavailable in PU learning , this situation is hard to detect by any metrics computed on the training set . This erroneous selection of unreliable pseudo-labels occurs when wrong model predictions are associated with excessive model confidence . Such poor model calibration is accompanied by a distortion of the signal for the pseudo label selection ( Van Engelen and Hoos , 2020 ) . In recent literature on pseudo-labeling , this problem is recognized and successfully addressed by explicitly estimating the prediction uncertainty ( Abdar et al. , 2021 ; Rizve et al. , 2021 ; Arazo et al. , 2020 ) . While this is the case for semi-supervised classification , there does not yet exist a method that explores the use of uncertainty quantification for pseudo-labeling in a PU learning context . Contributions : Motivated by this , we propose a novel , uncertainty-aware pseudo-labeling framework for PU learning that uses established uncertainty quantification techniques to identify reliable examples to pseudo-label ( Fig . 1 ) . In particular , our contributions are : ( 1 ) We introduce PUUPL ( Positive-Unlabeled , Uncertainty-Aware Pseudo-Labeling ) , a simple uncertainty-aware pseudolabeling framework for PU learning . ( 2 ) PUUPL can use any loss function for PU learning , improving model performance while being robust to the specific data biases that the respective loss considers . ( 3 ) We evaluate our methods on a wide range of benchmarks and PU datasets , achieving state-of-the-art performance in PU learning . ( 4 ) Our extensive ablation studies provide new insights into uncertainty-aware pseudo-labeling for PU learning . Further , they show that our method is robust to the choices of hyperparameters , with 1 % or less variability in test accuracy among different choices as well as distribution shifts between labeled positives in the train and test datasets . To the best of our knowledge , PUUPL is the first framework for PU learning which leverages uncertainty information during pseudo-labeling . 2 RELATED WORK . PU Learning PUL was introduced as a variant of binary classification ( Liu et al. , 2003 ) and is related to one-class learning ( Ruff et al. , 2018 ; Li et al. , 2010 ) , multi-positive learning ( Xu et al. , 2017 ) , multi-task learning ( Kaji et al. , 2018 ) , and semi-supervised learning ( Chapelle et al. , 2009 ) . There exist three main research branches for PUL : two-step techniques , class prior incorporation , and biased PUL ( Bekker and Davis , 2020 ) . In this work , we combine Pseudo-Labeling which has similarities to two-step techniques , with biased PUL , also coined as reweighting methods , and refer to Bekker and Davis ( 2020 ) for a comprehensive overview of the field . In this context , Du Plessis et al . ( 2014 ) introduced the unbiased risk estimator uPU . Kiryo et al . ( 2017 ) showed this loss function is prone to overfitting in deep learning contexts as it lacks a lower bound and proposed the non-negative risk estimator nnPU as a remedy . Follow-up work on loss functions for PUL has focused on robustness w.r.t biases in the sampling process such as PUSB ( Kato et al. , 2019 ) , PUbN ( Hsieh et al. , 2019 ) or PULNS ( Luo et al. , 2021 ) . Uncertainty-aware Pseudo-Labeling Pseudo-labeling follows the rationale that the model leverages its own predictions on unlabeled data as pseudo training targets to enable iterative semisupervised model training . The first such approach for deep learning was introduced by Lee ( 2013 ) , simply selecting the class with the highest predicted probability as a pseudo label . One weakness of pseudo-labeling is that erroneously selected pseudo-labels can amplify for training , potentially leading to model degradation . This is grounded in poor model calibration distorting the signal for the pseudo label selection ( Van Engelen and Hoos , 2020 ) . Iscen et al . ( 2019 ) try to mitigate this issue using confidence and class weights . Shi et al . ( 2018 ) use confident scores based on the geometric neighborhood of the unlabeled samples while Arazo et al . ( 2020 ) effectively tackle this confirmation bias using Mixup ( Zhang et al. , 2017 ) , Label Noise ( Tanaka et al. , 2018 ) , and Entropy Regularization ( Grandvalet et al. , 2005 ) . Rizve et al . ( 2021 ) introduced a pseudo-labeling framework using a weighting scheme for class balancing and MC dropout ( Gal and Ghahramani , 2016 ) for calibration , while Beluch et al . ( 2018 ) found deep ensembles ( Lakshminarayanan et al. , 2017a ) to yield the best model calibration in an active learning context , especially in low-label regimes . The commonality of these works is the explicit consideration of model uncertainty to improve pseudo-label selection , which motivates us to apply this in the context of PU learning . Pseudo-Labeling for PU Learning Two-step approaches in PU learning first identify negative samples from the unlabeled dataset , and then train a binary classification model on the original dataset augmented with the newly identified negatives ( Bekker and Davis , 2020 ) . These approaches share similarities with pseudo-labeling but lack an iterative feedback loop after the completion of the second step . A first attempt to combine pseudo-labeling with PU learning was made with Self-PU ( Chen et al. , 2020b ) , where self-paced learning , a confidence-weighting scheme based on the model predictions and a teacher-student distillation approach are combined . Via this complex training scheme , Self-PU was shown to marginally outperform recent baselines . With PUUPL , we propose an alternative PL strategy for PU learning that performs better in a simpler and more principled way using implicitly well-calibrated models to improve the pseudo-label selection . Uncertainty-aware pseudo-labeling for PU learning To the best of our knowledge , we are the first to introduce an uncertainty-aware pseudo-labeling paradigm to PU learning . Although our method shares the same motivation as that from Rizve et al . ( 2021 ) for semi-supervised classification , we differ in several important aspects : ( 1 ) we specifically target PU data with a PU loss , ( 2 ) we quantify uncertainty with an ensemble instead of Monte Carlo dropout , ( 3 ) we use epistemic uncertainty instead of the predicted class probabilities for the selection , ( 4 ) we do not use temperature scaling , and ( 5 ) we use soft labels . 3 METHOD . We propose PUUPL ( Positive Unlabeled , Uncertainty-aware Pseudo-Labeling ) , an iterative pseudolabeling procedure to progressively select and label the most confident examples from unlabeled data . The pseudo-code for PUUPL is shown as Algorithm 1 . Our method separates the training set Xtr into the sets P , U , and L which contain the initial positives , the currently unlabeled , and the pseudo-labeled samples respectively . The set L is initially empty . At each pseudo-labeling iteration , we first train our model using all samples in P , U , and L until some convergence condition is met ( Section 3.2 ) . Then , samples in U are predicted and ranked w.r.t their predictive uncertainty ( Section 3.3 ) and samples with the most confident score are assigned the predicted label and moved into the set L ( Section 3.4 ) . Similarly , samples in L are also predicted and the most uncertain samples are moved back to the unlabeled set U ( Section 3.5 ) . Next , the model is re-initialized to the same initial weights and a new pseudo-labeling iteration starts . In the following , we first describe the notation used in this paper and then explain in detail the training procedure of PUUPL . 3.1 NOTATION . Consider input samples X with label y and superscripts ·tr , ·va and ·te for training , validation , and test data respectively . The initial training labels ytr are set to one for all samples in P and zero for all others in U . We group the indices of original positives , unlabeled , and pseudo-labeled samples Algorithm 1 Pseudocode for the PUUPL Training Procedure Input • Train , validation and test data Xtr , ytr , Xva , yva , Xte , yte • Number K of networks in the ensemble ( suggested K = 2 ) • Maximum number T of pseudo-labels to assign at each round ( suggested T = 1000 ) • Maximum uncertainty threshold tl to assign pseudo-labels ( suggested tl = 0.05 ) • Minimum uncertainty threshold tu to remove pseudo-labels ( suggested tu = 0.35 ) Output Model parameters θ∗ 1 : P ← indices of positive samples in Xtr 2 : U ← indices of unlabeled samples in Xtr 3 : L← ∅ . Indices of pseudo-labeled samples 4 : θ0 ← Random weight initialization 5 : while not converged do 6 : Initialize model weights to θ0 . Training 7 : Train an ensemble of K networks on Xtr , ytr using the loss in Eq . 1 8 : Validate on Xva , yva and update weights θ∗ if accuracy improved 9 : f̂ ← ensemble predictions for Xtr . Uncertainty 10 : Compute epistemic uncertainty ûe with f̂ via Eq . 6 11 : Lnew ← Balanced set of examples to pseudo-label via Eq . 7 using ûeU , T and tl . Pseudo-labeling 12 : U new ← Examples to pseudo-unlabel via Eq . 10 using ûeL and tu 13 : L← L ∪ Lnewb \ U new . Update indices 14 : U ← U \ Lnewb ∪ U new 15 : yLnew ← p̂Lnew . Update pseudo-labels 16 : yUnew ← 0 17 : end while 18 : Restore the weights θ∗ that scored highest on the validation set 19 : Compute accuracy on the held-out test set Xte , yte 20 : return θ∗ in Xtr into the sets P , U , and L respectively . Our proposed model is an ensemble of K deep neural networks whose random initial weights are collectively denoted as θ0 . The predictions of the k-th network for sample i are indicated with p̂ik = σ ( f̂ik ) , with σ ( · ) the logistic function and f̂ik the predicted logits . The logits and predictions for a sample averaged across the networks in the ensemble are denoted by f̂i and p̂i respectively . We subscript data and predictions with i to index individual samples , and use an index set in the subscript to index all samples in the set ( e.g. , XtrU = { Xtri |i ∈ U } denotes the features of all unlabeled samples ) . We denote the total , epistemic and aleatoric uncertainty of sample i as ûti , û e i , and û a i , respectively .

Summary: This paper studies the PU learning problem. It proposes a two-step approach that can estimate the pseudo-label uncertainty so that more reliable pseudo-labels can be assigned, which improves the predictive performance. The proposed estimation method is different from previous methods.