Credal Self-Supervised Learning

1 Introduction . Recent progress and practical success in machine learning , especially in deep learning , is largely due to an increased availability of data . However , even if data collection is cheap in many domains , labeling the data so as to make it amenable to supervised learning algorithms might be costly and often comes with a significant effort . As a consequence , many data sets are only partly labeled , i.e. , only a few instances are labeled while the majority is not . This is the key motivation for semi-supervised learning ( SSL ) methods [ 7 ] , which seek to exploit both labeled and unlabeled data simultaneously . As one simple yet effective methodology , so-called self-training [ 26 ] , often also referred to as pseudolabeling , has proven effective in leveraging unlabeled data to improve over training solely on labeled data . The key idea is to let the learner itself generate “ pseudo-supervision ” for unlabeled instances based on its own current hypothesis . In the case of probabilistic classifiers , such pseudo-targets are usually provided in the form of ( perhaps degenerate ) probability distributions . Obviously , since pseudo-labels are mere guesses and might be wrong , this comes with the danger of biasing the learning process , a problem commonly known as confirmation bias [ 46 ] . Therefore , self-training is nowadays typically combined with additional regularization means , such as consistency regularization [ 2 , 40 ] , or additional uncertainty-awareness [ 38 ] . Labeling an instance x with a probability distribution on the target space Y is certainly better than committing to a precise target value , for example a single class label in classification , as the latter would suggest a level of conviction that is not warranted . Still , one may argue that even a probability distribution represents an excessive level of informedness . In fact , it actually suggests that the learner 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . precisely knows the ground-truth conditional probability p ( y |x ) . In our approach , we therefore allow the learner to label instances in the form of credal sets , that is , sets of ( candidate ) probability distributions [ 27 ] . Thanks to this increased expressiveness , the learner is able to represent uncertainty and a lack of knowledge about the true label ( distribution ) in a more flexible and more faithful manner . For example , by assigning the biggest credal set consisting of all probability distributions , it is able to represent complete ignorance — a state of knowledge that is arguably less well represented by a uniform probability distribution , which could also be interpreted as full certainty about this distribution being the ground truth . Needless to say , this ability is crucial to avoid a confirmation bias and account for the heteroscedastic nature of uncertainty , which varies both spatially ( i.e. , among different regions in the instance space ) and temporally : typically , the learner is less confident in early stages of the training process and becomes more confident toward the end . Existing methods are well aware of such problems but handle them in a manner that is arguably ad-hoc . The simple yet effective SSL framework FixMatch [ 41 ] , for instance , applies a thresholding technique to filter out presumably unreliable pseudo-labels , which often results in unnecessarily delayed optimization , as many instances are considered only lately in the training . Other approaches , such as MixUp [ 53 ] , also apply mixing strategies to learn in a more cautious manner from pseudolabels [ 1 , 3 , 4 ] . Moreover , as many of these approaches , including FixMatch , rely on the principle of entropy minimization [ 19 ] to separate classes well , the self-supervision is generated in the form of rather peaked or even degenerate distributions to learn from , which amplifies the problems of confirmation bias and over-confidence [ 33 ] . An important implication of credal pseudo-labeling is the need for extending the underlying learning algorithm , which must be able to learn from weak supervision of that kind . To this end , we leverage the principle of generalized risk minimization , which has recently been proposed in the realm of so-called superset learning [ 22 ] . This approach supports the idea of data disambiguation : The learner is free to ( implicitly ) choose any distribution inside a credal set that appears to be most plausible in light of the other data ( and its own learning bias ) . Thus , an implicit trade-off between cautious learning and entropy minimization can be realized : Whenever it seems reasonable to produce an extreme distribution , the learner is free but not urged to do so . Effectively , this not only reduces the risk of a potential confirmation bias due to misleading or over-confident pseudo-labels , it also allows for incorporating all unlabeled instances in the learning process from the beginning without any confidence thresholding , leading to a fast and effective semi-supervised learning method . To prove the effectiveness of this novel type of pseudo-labeling , we proceed from FixMatch as an effective state-of-the-art SSL framework and replace conventional probabilistic pseudo-labeling by a credal target set modeling . In an exhaustive empirical evaluation , we study the effects of this change compared to both hard and soft probabilistic target modeling , as well as measuring the resulting network calibration of induced models to reflect biases . Our experiments not only show competitive to superior generalization performance , but also better calibrated models while cutting the time to train the models drastically . 2 Related Work . In semi-supervised learning , the goal is to leverage the potential of unlabeled in addition to labeled data to improve learning and generalization . As it constitutes a broad research field with a plethora of approaches , we will focus here on classification methods as these are most closely related to our approach . We refer to [ 7 ] and [ 48 ] for more comprehensive overviews . As one of the earliest ideas to incorporate unlabeled data in conventional supervised learning , selftraining has shown remarkably effective in various domains , including natural language processing [ 12 ] and computer vision [ 11 , 17 , 39 ] . The technique is quite versatile and can be applied for different learning methods , ranging from support vector machines [ 30 ] to decision trees [ 45 ] and neural networks [ 35 ] . It can be considered as the basic training pattern in distillation , such as self-distillation from a model to be trained itself [ 23 ] or within student-teacher settings [ 36 , 51 ] . Recently , this technique also lifted completely unsupervised learning in computer vision to a new level [ 6 , 20 ] . As a common companion of self-training for classification , especially in computer vision , consistency regularization is employed to ensure similar model predictions when facing multiple perturbed versions of the same input [ 2 , 37 , 40 ] , resulting in noise-robustness as similarly achieved by other ( stochastic ) ensembling methods such as Dropout [ 42 ] . Strong augmentation techniques used in this regard , e.g. , CTAugment [ 3 ] or RandAugment [ 9 ] , allow one to learn from instances outside of the ( hitherto labeled ) data distribution and , thus , lead to more accurate models [ 10 ] . For semi-supervised learning in image classification , the combination of consistency regularization with pseudo-labeling is widely adopted and has proven to be a simple yet effective strategy [ 1 , 4 , 25 , 37 , 41 , 50 , 55 ] . As pseudo-labeling comprises the risk of biasing the model by wrong predictions , especially when confidence is low , uncertainty awareness has been explicitly considered in the generic self-supervision framework UPS to construct more reliable targets [ 38 ] . Within their approach , the model uncertainty is estimated by common Bayesian sampling techniques , such as MC-Dropout [ 15 ] or DropBlock [ 16 ] , which is then used to sort out unlabeled instances for which the model provides uncertain predictions . Related to this , the selection of pseudo-labels based on the model certainty has also been used in specific domains , such as text classification [ 32 ] or semantic segmentation [ 54 ] . 2.1 FixMatch . As already mentioned , FixMatch [ 41 ] combines recent advances in consistency regularization and pseudo-labeling into a simple yet effective state-of-the-art SSL approach . It will serve as a basis for our new SSL method , as it provides a generic framework for fair comparisons between conventional and our credal pseudo-labeling . In each training iteration , FixMatch considers a batch of B labeled instances Bl = { ( xi , pi ) } Bi=1 ⊂ X × P ( Y ) and µB unlabeled instances Bu = { xi } µBi=1 ⊂ X , where X denotes the input feature space , Y the set of possible classes , P ( Y ) the set of probability distributions over Y , and µ ≥ 1 the multiplicity of unlabeled over labeled instances in each batch . Here , the probabilistic targets in Bl are given as degenerate “ one-hot ” distributions . When aiming to induce probabilistic classifiers of the form p̂ : X −→ P ( Y ) , FixMatch distinguishes between two forms of augmentation : While As : X −→ X describes “ strong ” augmentations that perturbs the image in a drastic manner by combining multiple operations , simple flip-and-shift transformations are captured by “ weak ” augmentations Aw : X −→ X . As an iteration-wise loss to determine gradients , a combination of the labeled loss Ll and unlabeled loss Lu is calculated . For the former , the labeled input instances from Bl are weakly augmented and used in a conventional cross-entropy loss H : P ( Y ) 2 −→ R. For the latter , the model prediction q : = p̂ ( Aw ( x ) ) on a weakly-augmented version of each unlabeled instance x is used to construct a ( hard ) pseudo-label q̃ ∈ P ( Y ) when meeting a predefined confidence threshold τ . While q̃ is in FixMatch a degenerate probability distribution by default , one could also inject soft probabilities , which , however , turned out to be less effective ( cf . [ 41 ] ) . The pseudo-label is then compared to a strongly-augmented version of the same input image . Hence , the unlabeled loss Lu is given by Lu : = 1 µB ∑ x∈Bu Imax q≥τH ( q̃ , p̂ ( As ( x ) ) ) . 3 Credal Self-Supervised Learning . In this section , we introduce our credal self-supervised learning ( CSSL ) framework for the case of classification , assuming the target to be categorical with values in Y = { y1 , . . . , yK } . Before presenting more technical details , we start with a motivation for the credal ( set-valued ) modeling of target values and a sketch of the basic idea of our approach . 3.1 Motivation and Basic Idea . In supervised learning , we generally assume the dependency between instances x ∈ X and associated observations ( outcomes ) to be of stochastic nature . More specifically , one typically assumes a “ ground-truth ” in the form of a conditional probability distribution p∗ ( · |x ) ∈ P ( Y ) . Thus , for every y ∈ Y , p∗ ( y |x ) is the probability to observe y as a value for the target variable in the context x . Ideally , the distribution p∗ ( · |x ) would be provided as training information to the learner , along with every training instance x . In practice , however , supervision comes in the form of concrete values of the target , i.e. , a realization y ∈ Y of the random variable Y ∼ p∗ ( · |x ) , and the corresponding degenerate distribution py assigning probability mass 1 to y ( i.e. , py ( y |x ) = 1 and py ( y′ |x ) = 0 for y′ 6= y ) is taken as a surrogate for p∗ . Obviously , turning the true distribution p∗ = p∗ ( · |x ) , subsequently also called a “ soft label ” , into a more extreme distribution py ( i.e. , a single value y ) , referred to as “ hard label ” , may cause undesirable effects . In fact , it suggests a level of determinism that is not warranted and tempts the learner to over-confident predictions that are poorly calibrated , especially when training with losses such as cross-entropy [ 33 ] . So-called label smoothing [ 44 ] seeks to avoid such effects by replacing hard labels py with ( hypothetical ) soft labels p̂ that are “ close ” to py . Coming back to semi-supervised learning , training of the learner and self-supervision can be characterized for existing methods such as FixMatch as follows : • The learner is trained on hard labels , which are given for the labeled instances xl and constructed by the learner itself for unlabeled instances xu . • Construction of ( pseudo- ) labels is done in two steps : First , the true soft label p∗ = p∗ ( · |xu ) is predicted by a distribution p̂ = p̂ ( · |xu ) , and the latter is turned into a hard label p̂y afterward , provided p̂ suggests a sufficient level of certainty ( support for y ) . This approach can be challenged for several reasons . First , training probabilistic predictors on hard labels comes with the disadvantages mentioned above and tends to bias the learner . Second , pseudo-labels p̂y constructed by the learner tend to be poor approximations of the ground truth p∗ . In fact , there will always be a discrepancy between p∗ and its prediction p̂ , and this discrepancy is further increased by replacing p̂ with p̂y . Third , leaving some of the instances — those for which the prediction is not reliable enough — completely unlabeled causes a loss of information . Roughly speaking , while a part of the training data is overly precise1 , another part remains unnecessarily imprecise and hence unused . This may slow down the training process and cause other undesirable problems such as “ path-dependency ” ( training is influenced by the order of the unlabeled instances ) . To avoid these disadvantages , we propose to use soft instead of hard labels as training information for the learner . More specifically , to account for possible uncertainty about a true soft label p∗ , we model information about the target in the form of a set Q ⊆ P ( Y ) of distributions , in the literature on imprecise probability also called a credal set [ 27 ] . Such a “ credal label ” is supposed to cover the ground truth , i.e. , p∗ ∈ Q , very much like a confidence interval in statistics is supposed to cover ( with high probability ) some ground-truth parameter to be estimated . This approach is appealing , as it enables the learner to model its belief about the ground-truth p∗ in a cautious and faithful manner : Widening a credal label Q may weaken the training information but maintains or even increases its validity . Moreover , credal labeling elegantly allows for covering the original training information as special cases : A hard label y provided for a labeled instance xl corresponds to a singleton set Qy = { py } , and the lack of any information in the case of an unlabeled instance xu is properly captured by taking Q = P ( Y ) . Starting with this information , the idea is to modify it in two directions : • Imprecisiation : To avoid possible disadvantages of hard labels Qy , these can be made less precise through label relaxation [ 29 ] , which is an extension of the aforementioned label smoothing . Technically , it means that Qy is replaced by a credal set Q containing , in addition to py itself , distributions p close to py . • Precisiation : The non-informative and maximally imprecise labels Q = P ( Y ) for unlabeled instances xu are successively ( iteration by iteration ) “ shrunken ” and made more precise . This is done by replacing a set Q with a smaller subset Q′ ⊂ Q , provided the exclusion of certain candidate distributions p ∈ Q is sufficiently supported by the learner .

Summary: For the task of semi-supervised learning, this work proposes the use of credal sets to model uncertainty in pseudo-labels. By proposing simple scheme for generating and learning from these credal sets, the authors show how the FixMatch baseline can be extended and improved in settings where the number of labeled examples is small. Furthermore, they show that this approach can improve the efficiency of this semi-supervised learning approach and reduce calibration errors.