Positive-Unlabeled Learning with Uncertainty-aware Pseudo-label Selection

1 INTRODUCTION . Many real-world applications involve positive and unlabeled ( PU ) datasets in which only some of the data is labeled positive while the majority is unlabeled and contains both positives and negatives . PU learning aims to learn a binary classifier in this challenging setting without any labeled negative examples . Learning from PU data can reduce deployment costs in many deep learning applications that otherwise require annotations from experts such as medical image diagnosis ( Armenian and Lilienfeld , 1974 ) and protein function prediction ( Gligorijević et al. , 2021 ) , and it can even enable applications in settings where the measurement technology itself can not detect negative examples ( Purcell et al. , 2019 ) . Some recent approaches such as unbiased PU ( Du Plessis et al. , 2014 , uPU ) and non-negative PU ( Kiryo et al. , 2017 , nnPU ) formulate this problem as cost-sensitive learning . Others approach PU learning as a two-step procedure first identifying and labeling some reliable negative examples , and then re-training the model based on this newly constructed labeled dataset ( Bekker and Davis , 2020 ) . These approaches show similarities with pseudo-labeling in semi-supervised classification settings ( Lee , 2013 ) . Such pseudo-labeling techniques are however especially vulnerable to incorrectly assigned labels of the selected examples as these errors will propagate and magnify in the retrained model , resulting in a negative feedback loop . Worse yet , since the true labels are unavailable in PU learning , this situation is hard to detect by any metrics computed on the training set . This erroneous selection of unreliable pseudo-labels occurs when wrong model predictions are associated with excessive model confidence . Such poor model calibration is accompanied by a distortion of the signal for the pseudo label selection ( Van Engelen and Hoos , 2020 ) . In recent literature on pseudo-labeling , this problem is recognized and successfully addressed by explicitly estimating the prediction uncertainty ( Abdar et al. , 2021 ; Rizve et al. , 2021 ; Arazo et al. , 2020 ) . While this is the case for semi-supervised classification , there does not yet exist a method that explores the use of uncertainty quantification for pseudo-labeling in a PU learning context . Contributions : Motivated by this , we propose a novel , uncertainty-aware pseudo-labeling framework for PU learning that uses established uncertainty quantification techniques to identify reliable examples to pseudo-label ( Fig . 1 ) . In particular , our contributions are : ( 1 ) We introduce PUUPL ( Positive-Unlabeled , Uncertainty-Aware Pseudo-Labeling ) , a simple uncertainty-aware pseudolabeling framework for PU learning . ( 2 ) PUUPL can use any loss function for PU learning , improving model performance while being robust to the specific data biases that the respective loss considers . ( 3 ) We evaluate our methods on a wide range of benchmarks and PU datasets , achieving state-of-the-art performance in PU learning . ( 4 ) Our extensive ablation studies provide new insights into uncertainty-aware pseudo-labeling for PU learning . Further , they show that our method is robust to the choices of hyperparameters , with 1 % or less variability in test accuracy among different choices as well as distribution shifts between labeled positives in the train and test datasets . To the best of our knowledge , PUUPL is the first framework for PU learning which leverages uncertainty information during pseudo-labeling . 2 RELATED WORK . PU Learning PUL was introduced as a variant of binary classification ( Liu et al. , 2003 ) and is related to one-class learning ( Ruff et al. , 2018 ; Li et al. , 2010 ) , multi-positive learning ( Xu et al. , 2017 ) , multi-task learning ( Kaji et al. , 2018 ) , and semi-supervised learning ( Chapelle et al. , 2009 ) . There exist three main research branches for PUL : two-step techniques , class prior incorporation , and biased PUL ( Bekker and Davis , 2020 ) . In this work , we combine Pseudo-Labeling which has similarities to two-step techniques , with biased PUL , also coined as reweighting methods , and refer to Bekker and Davis ( 2020 ) for a comprehensive overview of the field . In this context , Du Plessis et al . ( 2014 ) introduced the unbiased risk estimator uPU . Kiryo et al . ( 2017 ) showed this loss function is prone to overfitting in deep learning contexts as it lacks a lower bound and proposed the non-negative risk estimator nnPU as a remedy . Follow-up work on loss functions for PUL has focused on robustness w.r.t biases in the sampling process such as PUSB ( Kato et al. , 2019 ) , PUbN ( Hsieh et al. , 2019 ) or PULNS ( Luo et al. , 2021 ) . Uncertainty-aware Pseudo-Labeling Pseudo-labeling follows the rationale that the model leverages its own predictions on unlabeled data as pseudo training targets to enable iterative semisupervised model training . The first such approach for deep learning was introduced by Lee ( 2013 ) , simply selecting the class with the highest predicted probability as a pseudo label . One weakness of pseudo-labeling is that erroneously selected pseudo-labels can amplify for training , potentially leading to model degradation . This is grounded in poor model calibration distorting the signal for the pseudo label selection ( Van Engelen and Hoos , 2020 ) . Iscen et al . ( 2019 ) try to mitigate this issue using confidence and class weights . Shi et al . ( 2018 ) use confident scores based on the geometric neighborhood of the unlabeled samples while Arazo et al . ( 2020 ) effectively tackle this confirmation bias using Mixup ( Zhang et al. , 2017 ) , Label Noise ( Tanaka et al. , 2018 ) , and Entropy Regularization ( Grandvalet et al. , 2005 ) . Rizve et al . ( 2021 ) introduced a pseudo-labeling framework using a weighting scheme for class balancing and MC dropout ( Gal and Ghahramani , 2016 ) for calibration , while Beluch et al . ( 2018 ) found deep ensembles ( Lakshminarayanan et al. , 2017a ) to yield the best model calibration in an active learning context , especially in low-label regimes . The commonality of these works is the explicit consideration of model uncertainty to improve pseudo-label selection , which motivates us to apply this in the context of PU learning . Pseudo-Labeling for PU Learning Two-step approaches in PU learning first identify negative samples from the unlabeled dataset , and then train a binary classification model on the original dataset augmented with the newly identified negatives ( Bekker and Davis , 2020 ) . These approaches share similarities with pseudo-labeling but lack an iterative feedback loop after the completion of the second step . A first attempt to combine pseudo-labeling with PU learning was made with Self-PU ( Chen et al. , 2020b ) , where self-paced learning , a confidence-weighting scheme based on the model predictions and a teacher-student distillation approach are combined . Via this complex training scheme , Self-PU was shown to marginally outperform recent baselines . With PUUPL , we propose an alternative PL strategy for PU learning that performs better in a simpler and more principled way using implicitly well-calibrated models to improve the pseudo-label selection . Uncertainty-aware pseudo-labeling for PU learning To the best of our knowledge , we are the first to introduce an uncertainty-aware pseudo-labeling paradigm to PU learning . Although our method shares the same motivation as that from Rizve et al . ( 2021 ) for semi-supervised classification , we differ in several important aspects : ( 1 ) we specifically target PU data with a PU loss , ( 2 ) we quantify uncertainty with an ensemble instead of Monte Carlo dropout , ( 3 ) we use epistemic uncertainty instead of the predicted class probabilities for the selection , ( 4 ) we do not use temperature scaling , and ( 5 ) we use soft labels . 3 METHOD . We propose PUUPL ( Positive Unlabeled , Uncertainty-aware Pseudo-Labeling ) , an iterative pseudolabeling procedure to progressively select and label the most confident examples from unlabeled data . The pseudo-code for PUUPL is shown as Algorithm 1 . Our method separates the training set Xtr into the sets P , U , and L which contain the initial positives , the currently unlabeled , and the pseudo-labeled samples respectively . The set L is initially empty . At each pseudo-labeling iteration , we first train our model using all samples in P , U , and L until some convergence condition is met ( Section 3.2 ) . Then , samples in U are predicted and ranked w.r.t their predictive uncertainty ( Section 3.3 ) and samples with the most confident score are assigned the predicted label and moved into the set L ( Section 3.4 ) . Similarly , samples in L are also predicted and the most uncertain samples are moved back to the unlabeled set U ( Section 3.5 ) . Next , the model is re-initialized to the same initial weights and a new pseudo-labeling iteration starts . In the following , we first describe the notation used in this paper and then explain in detail the training procedure of PUUPL . 3.1 NOTATION . Consider input samples X with label y and superscripts ·tr , ·va and ·te for training , validation , and test data respectively . The initial training labels ytr are set to one for all samples in P and zero for all others in U . We group the indices of original positives , unlabeled , and pseudo-labeled samples Algorithm 1 Pseudocode for the PUUPL Training Procedure Input • Train , validation and test data Xtr , ytr , Xva , yva , Xte , yte • Number K of networks in the ensemble ( suggested K = 2 ) • Maximum number T of pseudo-labels to assign at each round ( suggested T = 1000 ) • Maximum uncertainty threshold tl to assign pseudo-labels ( suggested tl = 0.05 ) • Minimum uncertainty threshold tu to remove pseudo-labels ( suggested tu = 0.35 ) Output Model parameters θ∗ 1 : P ← indices of positive samples in Xtr 2 : U ← indices of unlabeled samples in Xtr 3 : L← ∅ . Indices of pseudo-labeled samples 4 : θ0 ← Random weight initialization 5 : while not converged do 6 : Initialize model weights to θ0 . Training 7 : Train an ensemble of K networks on Xtr , ytr using the loss in Eq . 1 8 : Validate on Xva , yva and update weights θ∗ if accuracy improved 9 : f̂ ← ensemble predictions for Xtr . Uncertainty 10 : Compute epistemic uncertainty ûe with f̂ via Eq . 6 11 : Lnew ← Balanced set of examples to pseudo-label via Eq . 7 using ûeU , T and tl . Pseudo-labeling 12 : U new ← Examples to pseudo-unlabel via Eq . 10 using ûeL and tu 13 : L← L ∪ Lnewb \ U new . Update indices 14 : U ← U \ Lnewb ∪ U new 15 : yLnew ← p̂Lnew . Update pseudo-labels 16 : yUnew ← 0 17 : end while 18 : Restore the weights θ∗ that scored highest on the validation set 19 : Compute accuracy on the held-out test set Xte , yte 20 : return θ∗ in Xtr into the sets P , U , and L respectively . Our proposed model is an ensemble of K deep neural networks whose random initial weights are collectively denoted as θ0 . The predictions of the k-th network for sample i are indicated with p̂ik = σ ( f̂ik ) , with σ ( · ) the logistic function and f̂ik the predicted logits . The logits and predictions for a sample averaged across the networks in the ensemble are denoted by f̂i and p̂i respectively . We subscript data and predictions with i to index individual samples , and use an index set in the subscript to index all samples in the set ( e.g. , XtrU = { Xtri |i ∈ U } denotes the features of all unlabeled samples ) . We denote the total , epistemic and aleatoric uncertainty of sample i as ûti , û e i , and û a i , respectively .

Summary: This paper proposes PUUPL, an uncertainty-aware pseudo-label selection method for positive-unlabeled (PU) learning. To improve the performance of pseudo-labeling, the authors suggest using the epistemic uncertainty (the difference between the entropy of the mean prediction and the mean of entropies of each prediction). In the experiments, PUUPL outperformed the existing state-of-the-art PU learning methods.