FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling

1 Introduction . Semi-supervised learning ( SSL ) has attracted increasing attention in recent years due to its superiority in leveraging a large amount of unlabeled data . This is particularly advantageous when the labeled data are limited in quantity or laborious to obtain . Consistency regularization [ 1–3 ] and pseudo labeling [ 4–8 ] are two powerful techniques for utilizing unlabeled data and have been widely used in modern SSL algorithms [ 9–13 ] . The recently proposed FixMatch [ 14 ] achieves competitive results ∗Equal contribution . †Corresponding author . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . by combining these techniques with weak and strong data augmentations and using cross-entropy loss as the consistency regularization criterion . However , a drawback of FixMatch and other popular SSL algorithms such as Pseudo-Labeling [ 4 ] and Unsupervised Data Augmentation ( UDA ) [ 11 ] is that they rely on a fixed threshold to compute the unsupervised loss , using only unlabeled data whose prediction confidence is above the threshold . While this strategy can make sure that only high-quality unlabeled data contribute to the model training , it ignores a considerable amount of other unlabeled data , especially at the early stage of the training process , where only a few unlabeled data have their prediction confidence above the threshold . Moreover , modern SSL algorithms handle all classes equally without considering their different learning difficulties . To address these issues , we propose Curriculum Pseudo Labeling ( CPL ) , a curriculum learning [ 15 ] strategy to take into account the learning status of each class for semi-supervised learning . CPL substitutes the pre-defined thresholds with flexible thresholds that are dynamically adjusted for each class according to the current learning status . Notably , this process does not introduce any additional parameter ( hyper-parameter or trainable parameter ) or extra computation ( forward or back propagation ) . We apply this curriculum learning strategy directly to FixMatch and call the improved algorithm FlexMatch . While the training speed remains as efficient as that of FixMatch , FlexMatch converges significantly faster and achieves state-of-the-art performances on most SSL image classification benchmarks . The benefit of introducing CPL is particularly remarkable when the labels are scarce or when the task is challenging . For instance , on the STL-10 dataset , FlexMatch achieves relative performance improvement over FixMatch by 18.96 % , 16.11 % , and 7.68 % when the label amount is 400 , 2500 , and 10000 respectively . Moreover , CPL further shows its superiority by boosting the convergence speed – with CPL , FlexMatch takes less than 1/5 training time of FixMatch to reach its final accuracy . Adapting CPL to other modern SSL algorithms also leads to improvements in accuracy and convergence speed . To sum up , this paper makes the following three contributions : • We propose Curriculum Pseudo Labeling ( CPL ) , a curriculum learning approach of dynamically leveraging unlabeled data for SSL . It is almost cost-free and can be easily integrated to other SSL methods . • CPL significantly boosts the accuracy and convergence performance of several popular SSL algorithms on common benchmarks . Specifically , FlexMatch , the integration of FixMatch and CPL , achieves state-of-the-art results . • We open-source TorchSSL , a unified PyTorch-based semi-supervised learning codebase for the fair study of SSL algorithms . TorchSSL includes implementations of popular SSL algorithms and their corresponding training strategies , and is easy to use or customize . 2 Background . Consistency regularization follows the continuity assumption of SSL [ 1 , 2 ] . The most basic consistency loss in SSL , such as in Π Model [ 9 ] , Mean Teacher [ 10 ] and MixMatch [ 12 ] , is the ℓ-2 loss : µB∑ b=1 ||pm ( y|ω ( ub ) ) − pm ( y|ω ( ub ) ) ||22 , ( 1 ) where B is the batch size of labeled data , µ is the ratio of unlabeled data to labeled data , ω is a stochastic data augmentation function ( thus the two terms in Eq . ( 1 ) are different ) , ub denotes a piece of unlabeled data , and pm represents the output probability of the model . With the introduction of pseudo labeling techniques [ 5 , 7 ] , the consistency regularization is converted to an entropy minimization process [ 16 ] , which is more suitable for the classification task . The improved consistency loss with pseudo labeling can be represented as : 1 µB µB∑ b=1 1 ( max ( pm ( y|ω ( ub ) ) ) > τ ) H ( p̂m ( y|ω ( ub ) ) , pm ( y|ω ( ub ) ) ) , ( 2 ) where H is cross-entropy , τ is the pre-defined threshold and p̂m ( y|ω ( ub ) ) is the pseudo label that can either be a ‘ hard ’ one-hot label [ 4 , 14 ] or a sharpened ‘ soft ’ one [ 11 ] . The intention of using a threshold is to mask out noisy unlabeled data that have low prediction confidence . FixMatch utilizes such consistency regularization with strong augmentation to achieve competitive performance . For unlabeled data , FixMatch first uses weak augmentation to generate artificial labels . These labels are then used as the target of strongly-augmented data . The unsupervised loss term in FixMatch thereby has the form : 1 µB µB∑ b=1 1 ( max ( pm ( y|ω ( ub ) ) ) > τ ) H ( p̂m ( y|ω ( ub ) ) , pm ( y|Ω ( ub ) ) ) , ( 3 ) where Ω is a strong augmentation function instead of weak augmentation ω . Of the aforementioned works , the pre-defined threshold ( τ ) is constant . We believe this can be improved because the data of some classes may be inherently more difficult to learn than others . Curriculum learning [ 15 ] is a learning strategy where learning samples are gradually introduced according to the model ’ s learning process . In such a way , the model is always optimally challenged . This technique is widely employed in deep learning research [ 17–21 ] . 3 FlexMatch . 3.1 Curriculum Pseudo Labeling . While current SSL algorithms render pseudo labels of only high-confidence unlabeled data cut off by a pre-defined threshold , CPL renders the pseudo labels to different classes and at different time steps . Such a process is realized by adjusting the thresholds according to the model ’ s learning status of each class . However , it is non-trivial to dynamically determine the thresholds according to the learning status . The most ideal approach would be calculating evaluation accuracies for each class and use them to scale the threshold , as : Tt ( c ) = at ( c ) · τ , ( 4 ) where Tt ( c ) is the flexible threshold for class c at time step t and at ( c ) is the corresponding evaluation accuracy . In this way , lower accuracy that indicates a less satisfactory learning status of the class will lead to a lower threshold that encourages more samples of this class to be learned . Since we can not use the evaluation set in the model learning process , one may have to separate an extra validation set from the training set for such accuracy evaluations . However , this practice show two fatal problems : First , such a labeled validation set separated from the training set is expensive under SSL scenario as the labeled data are already scarce . Second , to dynamically adjust the thresholds in the training process , accuracy evaluations must be done continually at each time step t , which will considerably slow down the training speed . In this work , we propose Curriculum Pseudo Labeling ( CPL ) for semi-supervised learning . Our CPL uses an alternative way to estimate the learning status , which does not introduce additional inference processes , nor needs an extra validation set . As believed in [ 14 ] , a high threshold that filters out noisy pseudo labels and leaves only high-quality ones can considerably reduce the confirmation bias [ 22 ] . Therefore , our key assumption is that when the threshold is high , the learning effect of a class can be reflected by the number of samples whose predictions fall into this class and above the threshold . Namely , the class with fewer samples having their prediction confidence reach the threshold is considered to have a greater learning difficulty or a worse learning status , formulated as : σt ( c ) = N∑ n=1 1 ( max ( pm , t ( y|un ) ) > τ ) · 1 ( argmax ( pm , t ( y|un ) = c ) . ( 5 ) where σt ( c ) reflects the learning effect of class c at time step t. pm , t ( y|un ) is the model ’ s prediction for unlabeled data un at time step t , and N is the total number of unlabeled data . When the unlabeled dataset is balanced ( i.e. , the number of unlabeled data belonging to different classes are equal or close ) , larger σt ( c ) indicates a better estimated learning effect . By applying the following normalization to σt ( c ) to make its range between 0 to 1 , it can then be used to scale the fixed threshold τ : βt ( c ) = σt ( c ) max c σt , ( 6 ) Tt ( c ) = βt ( c ) · τ . ( 7 ) One characteristic of such a normalization approach is that the best-learned class has its βt ( c ) equal to 1 , causing its flexible threshold equal to τ . This is desirable . For classes that are hard to learn , the thresholds are lowered down , encouraging more training samples in these classes to be learned . This also improves the data utilization ratio . As learning proceeds , the threshold of a well-learned class is raised higher to selectively pick up higher-quality samples . Eventually , when all classes have reached reliable accuracies , the thresholds will all approach τ . Note that the thresholds do not always grow , it may also decrease if the unlabeled data is classified into a different class in later iterations . This new threshold is used for calculating the unsupervised loss in FlexMatch , which can be formulated as : Lu , t = 1 µB µB∑ b=1 1 ( max ( qb ) > Tt ( argmax ( qb ) ) ) H ( q̂b , pm ( y|Ω ( ub ) ) ) , ( 8 ) where qb = pm ( y|ω ( ub ) ) . The flexible thresholds are updated at each iteration . Finally , we can formulate the loss in FlexMatch as the weighted combination ( by λ ) of supervised and unsupervised loss : Lt = Ls + λLu , t , ( 9 ) where Ls is the supervised loss on labeled data : Ls = 1 B B∑ b=1 H ( yb , pm ( y|ω ( xb ) ) ) . ( 10 ) Note that the cost of introducing CPL is almost free . Practically , every time the prediction confidence of an unlabeled data un is above the fixed threshold τ , the data , and its predicted class are marked and will be used for calculating βt ( c ) at the next time step . Such marking actions are bonus actions each time the consistency loss is computed . Therefore , FlexMatch does not introduce additional forward propagation processes for evaluating the model ’ s learning status , nor new parameters .

Summary: The paper proposes to incorporate ideas from curriculum learning into semi-supervised learning. Semi-supervised learning consists on training models with only a subset of labeled samples, and a typically larger set of unlabeled samples. The paper proposes to improve on ideas from pseudo-labeling whereas fixed thresholds are defined to decide the pseudo-labels for unlabeled samples as the training process goes. In this approach, the idea is to modulate the thresholds used during the training process as a function of time (iterations) and the individual classes being classified. The motivation is that some classes might be harder to learn. In order to deal with initial conditions, the paper proposes to use a warming up strategy whereby an additional class (undecided) is considered for samples that have not reached a confidence to be assigned to any class. The proposed method in combination with other methods based on consistency regularization such as FixMatch lead to improved accuracies.