FlexMatch: Boosting Semi-Supervised Learning with Curriculum Pseudo Labeling

1 Introduction . Semi-supervised learning ( SSL ) has attracted increasing attention in recent years due to its superiority in leveraging a large amount of unlabeled data . This is particularly advantageous when the labeled data are limited in quantity or laborious to obtain . Consistency regularization [ 1–3 ] and pseudo labeling [ 4–8 ] are two powerful techniques for utilizing unlabeled data and have been widely used in modern SSL algorithms [ 9–13 ] . The recently proposed FixMatch [ 14 ] achieves competitive results ∗Equal contribution . †Corresponding author . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . by combining these techniques with weak and strong data augmentations and using cross-entropy loss as the consistency regularization criterion . However , a drawback of FixMatch and other popular SSL algorithms such as Pseudo-Labeling [ 4 ] and Unsupervised Data Augmentation ( UDA ) [ 11 ] is that they rely on a fixed threshold to compute the unsupervised loss , using only unlabeled data whose prediction confidence is above the threshold . While this strategy can make sure that only high-quality unlabeled data contribute to the model training , it ignores a considerable amount of other unlabeled data , especially at the early stage of the training process , where only a few unlabeled data have their prediction confidence above the threshold . Moreover , modern SSL algorithms handle all classes equally without considering their different learning difficulties . To address these issues , we propose Curriculum Pseudo Labeling ( CPL ) , a curriculum learning [ 15 ] strategy to take into account the learning status of each class for semi-supervised learning . CPL substitutes the pre-defined thresholds with flexible thresholds that are dynamically adjusted for each class according to the current learning status . Notably , this process does not introduce any additional parameter ( hyper-parameter or trainable parameter ) or extra computation ( forward or back propagation ) . We apply this curriculum learning strategy directly to FixMatch and call the improved algorithm FlexMatch . While the training speed remains as efficient as that of FixMatch , FlexMatch converges significantly faster and achieves state-of-the-art performances on most SSL image classification benchmarks . The benefit of introducing CPL is particularly remarkable when the labels are scarce or when the task is challenging . For instance , on the STL-10 dataset , FlexMatch achieves relative performance improvement over FixMatch by 18.96 % , 16.11 % , and 7.68 % when the label amount is 400 , 2500 , and 10000 respectively . Moreover , CPL further shows its superiority by boosting the convergence speed – with CPL , FlexMatch takes less than 1/5 training time of FixMatch to reach its final accuracy . Adapting CPL to other modern SSL algorithms also leads to improvements in accuracy and convergence speed . To sum up , this paper makes the following three contributions : • We propose Curriculum Pseudo Labeling ( CPL ) , a curriculum learning approach of dynamically leveraging unlabeled data for SSL . It is almost cost-free and can be easily integrated to other SSL methods . • CPL significantly boosts the accuracy and convergence performance of several popular SSL algorithms on common benchmarks . Specifically , FlexMatch , the integration of FixMatch and CPL , achieves state-of-the-art results . • We open-source TorchSSL , a unified PyTorch-based semi-supervised learning codebase for the fair study of SSL algorithms . TorchSSL includes implementations of popular SSL algorithms and their corresponding training strategies , and is easy to use or customize . 2 Background . Consistency regularization follows the continuity assumption of SSL [ 1 , 2 ] . The most basic consistency loss in SSL , such as in Π Model [ 9 ] , Mean Teacher [ 10 ] and MixMatch [ 12 ] , is the ℓ-2 loss : µB∑ b=1 ||pm ( y|ω ( ub ) ) − pm ( y|ω ( ub ) ) ||22 , ( 1 ) where B is the batch size of labeled data , µ is the ratio of unlabeled data to labeled data , ω is a stochastic data augmentation function ( thus the two terms in Eq . ( 1 ) are different ) , ub denotes a piece of unlabeled data , and pm represents the output probability of the model . With the introduction of pseudo labeling techniques [ 5 , 7 ] , the consistency regularization is converted to an entropy minimization process [ 16 ] , which is more suitable for the classification task . The improved consistency loss with pseudo labeling can be represented as : 1 µB µB∑ b=1 1 ( max ( pm ( y|ω ( ub ) ) ) > τ ) H ( p̂m ( y|ω ( ub ) ) , pm ( y|ω ( ub ) ) ) , ( 2 ) where H is cross-entropy , τ is the pre-defined threshold and p̂m ( y|ω ( ub ) ) is the pseudo label that can either be a ‘ hard ’ one-hot label [ 4 , 14 ] or a sharpened ‘ soft ’ one [ 11 ] . The intention of using a threshold is to mask out noisy unlabeled data that have low prediction confidence . FixMatch utilizes such consistency regularization with strong augmentation to achieve competitive performance . For unlabeled data , FixMatch first uses weak augmentation to generate artificial labels . These labels are then used as the target of strongly-augmented data . The unsupervised loss term in FixMatch thereby has the form : 1 µB µB∑ b=1 1 ( max ( pm ( y|ω ( ub ) ) ) > τ ) H ( p̂m ( y|ω ( ub ) ) , pm ( y|Ω ( ub ) ) ) , ( 3 ) where Ω is a strong augmentation function instead of weak augmentation ω . Of the aforementioned works , the pre-defined threshold ( τ ) is constant . We believe this can be improved because the data of some classes may be inherently more difficult to learn than others . Curriculum learning [ 15 ] is a learning strategy where learning samples are gradually introduced according to the model ’ s learning process . In such a way , the model is always optimally challenged . This technique is widely employed in deep learning research [ 17–21 ] . 3 FlexMatch . 3.1 Curriculum Pseudo Labeling . While current SSL algorithms render pseudo labels of only high-confidence unlabeled data cut off by a pre-defined threshold , CPL renders the pseudo labels to different classes and at different time steps . Such a process is realized by adjusting the thresholds according to the model ’ s learning status of each class . However , it is non-trivial to dynamically determine the thresholds according to the learning status . The most ideal approach would be calculating evaluation accuracies for each class and use them to scale the threshold , as : Tt ( c ) = at ( c ) · τ , ( 4 ) where Tt ( c ) is the flexible threshold for class c at time step t and at ( c ) is the corresponding evaluation accuracy . In this way , lower accuracy that indicates a less satisfactory learning status of the class will lead to a lower threshold that encourages more samples of this class to be learned . Since we can not use the evaluation set in the model learning process , one may have to separate an extra validation set from the training set for such accuracy evaluations . However , this practice show two fatal problems : First , such a labeled validation set separated from the training set is expensive under SSL scenario as the labeled data are already scarce . Second , to dynamically adjust the thresholds in the training process , accuracy evaluations must be done continually at each time step t , which will considerably slow down the training speed . In this work , we propose Curriculum Pseudo Labeling ( CPL ) for semi-supervised learning . Our CPL uses an alternative way to estimate the learning status , which does not introduce additional inference processes , nor needs an extra validation set . As believed in [ 14 ] , a high threshold that filters out noisy pseudo labels and leaves only high-quality ones can considerably reduce the confirmation bias [ 22 ] . Therefore , our key assumption is that when the threshold is high , the learning effect of a class can be reflected by the number of samples whose predictions fall into this class and above the threshold . Namely , the class with fewer samples having their prediction confidence reach the threshold is considered to have a greater learning difficulty or a worse learning status , formulated as : σt ( c ) = N∑ n=1 1 ( max ( pm , t ( y|un ) ) > τ ) · 1 ( argmax ( pm , t ( y|un ) = c ) . ( 5 ) where σt ( c ) reflects the learning effect of class c at time step t. pm , t ( y|un ) is the model ’ s prediction for unlabeled data un at time step t , and N is the total number of unlabeled data . When the unlabeled dataset is balanced ( i.e. , the number of unlabeled data belonging to different classes are equal or close ) , larger σt ( c ) indicates a better estimated learning effect . By applying the following normalization to σt ( c ) to make its range between 0 to 1 , it can then be used to scale the fixed threshold τ : βt ( c ) = σt ( c ) max c σt , ( 6 ) Tt ( c ) = βt ( c ) · τ . ( 7 ) One characteristic of such a normalization approach is that the best-learned class has its βt ( c ) equal to 1 , causing its flexible threshold equal to τ . This is desirable . For classes that are hard to learn , the thresholds are lowered down , encouraging more training samples in these classes to be learned . This also improves the data utilization ratio . As learning proceeds , the threshold of a well-learned class is raised higher to selectively pick up higher-quality samples . Eventually , when all classes have reached reliable accuracies , the thresholds will all approach τ . Note that the thresholds do not always grow , it may also decrease if the unlabeled data is classified into a different class in later iterations . This new threshold is used for calculating the unsupervised loss in FlexMatch , which can be formulated as : Lu , t = 1 µB µB∑ b=1 1 ( max ( qb ) > Tt ( argmax ( qb ) ) ) H ( q̂b , pm ( y|Ω ( ub ) ) ) , ( 8 ) where qb = pm ( y|ω ( ub ) ) . The flexible thresholds are updated at each iteration . Finally , we can formulate the loss in FlexMatch as the weighted combination ( by λ ) of supervised and unsupervised loss : Lt = Ls + λLu , t , ( 9 ) where Ls is the supervised loss on labeled data : Ls = 1 B B∑ b=1 H ( yb , pm ( y|ω ( xb ) ) ) . ( 10 ) Note that the cost of introducing CPL is almost free . Practically , every time the prediction confidence of an unlabeled data un is above the fixed threshold τ , the data , and its predicted class are marked and will be used for calculating βt ( c ) at the next time step . Such marking actions are bonus actions each time the consistency loss is computed . Therefore , FlexMatch does not introduce additional forward propagation processes for evaluating the model ’ s learning status , nor new parameters .

Summary: This work extends the popular FixMatch framework with dynamic, class-specific thresholds. Specifically, a multiplier is designed to scale the fixed threshold for each class, which is computed based on the proportion of pseudo-labeled instances with confidence values passing the threshold—the motivation is to lower the threshold for classes that have fewer pseudo-labeled instances. The effectiveness of the proposed approach, named FlexMatch, is demonstrated on CIFAR, STL-10 and ImageNet where clear improvements over FixMatch are observed.