Enhancing semi-supervised learning via self-interested coalitional learning

1 INTRODUCTION . Despite the huge success of supervised learning with deep neural networks , they require large amounts of labeled data for model training to achieve high performance ( Meng et al. , 2019 ; Prabhu & Varma , 2014 ) . Collecting labeled data can be very difficult and costly in practice ( Meng et al. , 2020 ) . The challenge can be further exacerbated as in many cases , human or machine labeled data may contain errors or noises ( Natarajan et al. , 2017 ; Schnabel et al. , 2016 ) . Semi-supervised learning ( SSL ) ( Chapelle et al. , 2006 ) that leverages both labeled and unlabeled data has become an increasingly promising yet still challenging area , and is widely applicable to real-world problems . To utilize the hidden information in the unlabeled data , a general and popular semi-supervised learning paradigm is through self-training ( Yarowsky , 1995 ; Lee et al. , 2013 ) . Self-training uses a previously learned model to predict labels for the unlabeled data ( pseudo-labeling ) which are then used for subsequent model training ( Yarowsky , 1995 ; Lee et al. , 2013 ; Laine & Aila , 2017 ; Tarvainen & Valpola , 2017b ; Iscen et al. , 2019b ) . Despite some empirical successes , self-training methods still suffer from two core challenges , which are over-reliance on labeled data and error accumulation . Most existing semi-supervised learning methods ( Yarowsky , 1995 ; Lee et al. , 2013 ; Laine & Aila , 2017 ; Sohn et al. , 2020 ; Xie et al. , 2020 ; Kipf & Welling , 2017 ) assume the labeled and unlabeled data follow the same or similar data distribution , thus the data-label mapping mechanisms learned from the labeled data are somewhat transferable to unlabeled data . Essentially , the algorithm is still reinforcing the information in the labeled data rather than mining additional information from the unlabeled data . Second , the pseudo-labels of unlabeled data can be incorrectly predicted . Using such biased information for training in subsequent epochs could increase confidence in erroneous predictions , and eventually leading to a vicious circle of error accumulation ( Cai et al. , 2013 ; Arazo et al. , 2020 ) . The situation can be even worse when the labeled data contain errors or noises , as the model learned from labeled data will make more mistakes , resulting in more severe error accumulation . Alleviating the issue of over-reliance on labeled data requires extracting additional sources of information from the unlabeled data . One recent approach is to introduce input consistency regularization ( Sajjadi et al. , 2016 ) . Consistency regularization enforces the stability of the predictions with respect to the transformations of the unlabeled data ( e.g . data augmentation on images such as rotation , flip and shift , etc. ) . Although such data perturbation or augmentation are well-defined for images , they are not directly transferable to general settings , hence the success of consistency regularization methods are mostly restricted to image classification tasks ( Sajjadi et al. , 2016 ; Liu et al. , 2019 ; Xie et al. , 2020 ; Sohn et al. , 2020 ) . We provide a new insight by noting that , under the self-training paradigm , every semi-supervised learning task hides another auxiliary task of discriminating whether the data label is real or a pseudo-label predicted by a machine oracle . Jointly solving the main task together with the auxiliary task allows sufficient utilization of the hidden information in unlabeled data . To break the vicious circle of error accumulation , one needs to not always trust the machine-labeled data . A natural idea is to consider the reliability or confidence of data labels and treat them differently during training . A connected approach in supervised learning literature is to use the notion of “ soft label ” with confidence probability or weights , which has been verified in a number of noisy label learning problems ( Natarajan et al. , 2017 ; Schnabel et al. , 2016 ; Shen et al. , 2020 ; Algan & Ulusoy , 2021 ) . This treatment can also help a semi-supervised learning algorithm gain the ability to robustly learn from noisy labels . Combining previous insights , we develop a novel semi-supervised learning framework , called Selfinterested Coalitional Learning ( SCL ) . SCL contains three ingredients . First , it simultaneously solves a companion task by learning a discriminator as a critic to predict label observability ( whether it is a true label or a pseudo-label ) , which is used to mine additional information in the unlabeled data while facilitating main task learning . Second , the output of the discriminator is in fact a measure of label reliability , thus can be interpreted as the confidence probabilities of labels ( Qin et al. , 2021 ) . This converts the original problem into a soft label learning problem . Finally and most importantly , SCL introduces a cooperative-yet-competitive learning scheme to boost the performance of both the main task and the companion task . Specifically , SCL forges cooperation between both tasks by providing extra information to each other ( the main task provides prediction loss , companion task provides label confidence measures ) , leading to improved performance of both tasks . Moreover , we consider the main task model to be self-interested which tries to challenge the discriminator by providing as little information as possible . This design forms a semi-cooperative “ game ” with a partially adversarial main task model , which can be shown theoretically equivalent to a loss reweighting mechanism on noisy soft labels . Under this design , the main task and the companion task in SCL are neither fully cooperative as in multi-objective optimization ( Deb , 2014 ) nor fully adversarial to each other as in adversarial learning ( Goodfellow et al. , 2014 ; Matyasko & Chau , 2018 ) , but balance the benefit of cooperation and competition . In summary , the contribution of this work include : • We develop a novel and very general self-interested coalitional learning framework for SSL that fully utilizes the hidden information in the unlabeled data while addressing the error accumulation issue in the self-training paradigm of SSL . • SCL can be easily incorporated into a wide range of semi-supervised algorithms ( e.g . image classification , label propagation and data imputation ) with limited changes on the original method . We show with comprehensive empirical experiments that SCL consistently achieves superior performance and robustness compared with the original and self-training version of different semisupervised learning algorithms . • SCL naturally provides the data confidence measures from the discriminator as a byproduct of the learning process , which offers additional interpretability of the semi-supervised learning task . This can be particularly useful for many practical applications , while also providing extra benefits for scenarios with noisy data labels . 2 PRELIMINARIES . 2.1 PROBLEM DEFINITIONS AND FORMULATIONS . We formulate the semi-supervised learning problem as learning a model f ( · ) with input data x ∈ X to predict the label y ∈ Y . In SSL , only a partial set of data labels YL with ground truth are given in the training set , the rest are unlabeled YU ( Y = YL ⋃ YU ) . For convenience of later discussion , we denote L as the set of indices of data samples with labels and U as the set of indices for unlabeled data . In many real-world SSL problems , the size of labeled data |L| is often limited . In extreme cases , YL may potentially contain errors or noises . We further define a mask vector M to denote the observability of labels over each data sample . We set Mi ∈ M equals to 1 if i ∈ L and Mj ∈ M equals to 0 if j ∈ U . In our proposed SCL framework , we consider following common settings of semi-supervised classification ( inductive classification , label propagation ) and regression ( data imputation ) tasks : Inductive classification : a typical SSL task setting is to construct a classifier to predict labels y ∈ Y for any object in the input space x ∈ X . Common examples include semi-supervised image classification tasks ( Sajjadi et al. , 2016 ; Sohn et al. , 2020 ; Xie et al. , 2020 ) , where only a subset of data samples are given the known labels YL and the rest data labels YU are unknown . Label propagation : another class of SSL tasks under transductive setup is to use all input samples X and observed labels YL to train a classifier to predict on unseen labels YU . Examples include classifying nodes in a graph given only small subset of node labels YL ( Kipf & Welling , 2017 ) . Data imputation : one of the special case of SSL problem is that the missing state of input and output data are strongly correlated ( Richardson et al. , 2020 ; Qin et al. , 2021 ) , such as the data imputation tasks . In data imputation , the inputs X are partially filled and the labels Y are equivalent to a reconstructed version of X obtained through regression . 2.2 AUXILIARY TASK IN SEMI-SUPERVISED LEARNING PROBLEMS . We begin our discussion by first noting that , if we feed the unknown labels YU with the model f predicted labels ỸU = Ŷ ( k ) U ( Ŷ ( k ) U = f ( X ) represents the labels from the k-th round of pseudolabeling ) or other machine-generated labels , there actually hides an auxiliary task of discriminating whether the data label is real or a pseudo-label . Denote all the data labels under pseudo-labeling as Ỹ = YL ⋃ ỸU . We can train a discriminator d ( · ) to tell the confidence measure p ∈ P of whether x → ỹ ( x ∈ X , ỹ ∈ Ỹ ) is a valid mapping . This is always learnable since the ground truth of P is exactly the observability mask M . With a slight abuse of notation , we formulate the main SSL task A and the auxiliary companion task B as follows : A : f ( X ) = Ŷ , LA = lossA ( YL , ŶL ) B : d ( X , Ỹ ) = P , LB = lossB ( P , M ) ( 1 ) where lossA is the original loss function of the main task and lossB can be any binary classification loss between P and M , such as the binary cross entropy ( BCE ) loss . Jointly solving the above two tasks allows exploiting the underlying relationship between input data X and data label Y from another angle , which can potentially provide more information to facilitate main task training . There are some other works that embody the similar idea of jointly learning two tasks . For example , the multi-task style multi-objective optimization methods ( Ruder , 2017 ; Deb , 2014 ) exploit the shared information and underlying commonalities between two tasks and solve the problem by minimizing an augmented loss . The generative adversarial learning ( Goodfellow et al. , 2014 ) makes a generator and a discriminator learn against each other , thus improves the performance of both tasks . However , naı̈vely applying these two approaches has some drawbacks . For example , multiobjective optimization can not handle potential contradictions of the two tasks in certain settings , where jointly minimizing the augmented loss may impede both tasks from achieving the best performance ( Qin et al. , 2021 ) . Whereas under the adversarial learning framework , lossA is implicitly optimized through the discriminator , which could result in potential loss of information . Moreover , solving the minimax optimization problem under a GAN-style model is much harder compared with directly minimize both loss functions explicitly in ( 1 ) in a supervised learning fashion .

Summary: This paper proposes a new semi-supervised learning method. Motivated by the error accumulation problem of typical self-training paradigms, the authors propose to explicitly model the confidence of pseudo labels as an auxiliary task. They come up with a self-interested coalitional learning (SCL) strategy to solve both tasks jointly. Under the new framework, the main task is transformed into a cost-sensitive learning problem. Experiments demonstrate that pseudo labels are substantially more accurate with the new method and better performance of the main tasks at different label missing rates.