Enhancing semi-supervised learning via self-interested coalitional learning
Authors: Huiling Qin
Xianyuan Zhan
Yuanxun li
Haoran Xu
yu zheng
Venue: ICLR 2022 Submitted (2022)
Keywords: self-interested coalitional learning
semi-supervised learning
soft labeling
loss reweighting

Abstract: Semi-supervised learning holds great promise for many real-world applications, due to its ability to leverage both unlabeled and expensive labeled data. However, most semi-supervised learning algorithms still heavily rely on the limited labeled data to infer and utilize the hidden information from unlabeled data. We note that any semi-supervised learning task under the self-training paradigm also hides an auxiliary task of discriminating label observability. Jointly solving these two tasks allows full utilization of information from both labeled and unlabeled data, thus alleviating the problem of over-reliance on labeled data. This naturally leads to a new learning framework, which we call Self-interested Coalitional Learning (SCL). The key idea of SCL is to construct a semi-cooperative ``game”, which forges cooperation between a main self-interested semi-supervised learning task and a companion task that infers label observability to facilitate main task training. We show with theoretical deduction its connection to loss reweighting on noisy labels. Through comprehensive evaluation on both classification and regression tasks, we show that SCL can consistently enhance the performance of semi-supervised learning algorithms.

Under review as a conference paper at ICLR 2022
ENHANCING SEMI-SUPERVISED LEARNING VIA SELF- INTERESTED COALITIONAL LEARNING
Anonymous authors
Paper under double-blind review
ABSTRACT
Semi-supervised learning holds great promise for many real-world applications, due its ability to leverage both unlabeled and expensive labeled data. However, most semi-superviser learning algorithms still heavily rely on the limited labeled data to infer and utilize the hidden information from unlabeled data. We note that any semi-supervised learning task under the self-training paradigm also hides an auxiliary task of discriminating label observability. Jointly solving these two tasks allows full utilization of information from both labeled and unlabeled data, thus alleviating the problem of over-reliance on labeled data. This naturally leads learning framework, which we call Self- self-interested Coalitional Learning (SCL). The key idea of SCL to construct a cooperateee "game", which forges cooperation between main self-interested semi-supervised learning task and a companion task that infers label observability to facilitate main task train- ing. We show with theoretical deduction its connection loss reweighting on noisy labels. Through comprehensive evaluation on both classification and re- gression tasks, we show that SCL can consistently enhance the performance of semi-supervised learning algorithms.
INTRODUCTION
Despite the huge success of supervised learning with deep neural networks, they require large amounts of labeled data for model training achieve high performance (Meng al.) [019:] Prabhu & Varma 2014). Collecting labeled data can be very difficult and costly in practice (Meng et al., 2020). The challenge can be further exacerbated as in many cases, human or machine labeled data may contain errors or noises (Natarajan et al. 2017 Schnabel et al.) (016). Semi-supervised learn- ing (SSL) (Chapelle et al., 2006) that leverages both labeled and unlabeled data has become an increasingly promising yet still challenging area, and is widely applicable to real-world problems.
To utilize the hidden information the unlabeled data, a general and popular semi-supervised learn- ing paradigm is through self self-training (Yarowsky 1995 Lee et al. 2013). Self-training uses previ- ously learned model to predict labels for the unlabeled data (pseudo-labeling) which are then used for subsequent model training (Yarowsky) 1995; Lee al. 2013 Laine & Aila 2017: Tarvainen & Valpola. 2017b; Iscen et al.) 2019b). Despite some empirical successes, self-training methods still suffer from two core challenges, which are over-reliance on labeled data and error accumulation.
Most existing semi-supervised learning methods (Yarowsky) 1995; Lee et al.) 2013 Laine & Aila 2017 Sohn al. 2020, Xie et al. 2020 Kipf Welling (011)) assume the labeled and unlabeled
data follow the same or similar data distribution, thus the data-label mapping mechanisms learned from the labeled data are somewhat transferable unlabeled data. Essentially, the algorithm still reinforcing the information in the labeled data rather than mining additional information from the unlabeled data. Second, the pseudo-labels of unlabeled data can be incorrectly predicted. Using such biased information for training in subsequent epochs could increase confidence in erroneous predic- tions, and eventually leading to a vicious circle of error accumulation (Cai et al., 2013] Arazo et al.) 2020). The situation be even worse when the labeled data contain errors noises, the model learned from labeled data will make more mistakes, resulting in more severe error accumulation.
Alleviating the issue of over-reliance labeled data requires extracting additional sources of infor- mation from the unlabeled data. One recent approach is to introduce input consistency regularization (Sajaadi et al.] 2016). Consistency regularization enforces the stability of the predictions with respect

---

Under review as a conference paper at ICLR 2022
to the transformations of the unlabeled data (e.g. data augmentation on images such rotation, flip and shift, etc.). Although such data perturbation or augmentation are well-defined for images, they not directly transferable to general settings, hence the success of consistency regularization meth- ods are mostly restricted to image classification tasks (Sajjadi et al. 2016 Liu et al. 2019) Xie et al.) 2020 [Sohn et al. 2020). We provide new insight by noting that, under the self-training paradigm, every semi semi-supervised learning task hides another auxiliary task of discriminating whether the data pseudo-label predicted by a machine oracle. Jointly solving the main task together with the auxiliary task allows sufficient utilization of the hidden information in unlabeled data.
label is real
To break the vicious circle of error accumulation, one needs to not always trust the machine- labeled data. A natural idea is to consider the reliability or confidence of data labels and treat them differ- ently during training. A connected approach in supervised learning literature to use the notion of "soft label" with confidence probability or weights, which has been verified in number of noisy label learning problems (Natarajan et al. 2017] Schnabel al.) 2016; [hhen et al., 2020 Algan & Ulusoy 2021). This treatment can also help semi-supervised learning algorithm gain the ability to robustly learn from noisy labels.
Combining previous insights, we develop novel semi-supervised learning framework, called Self- interested Coalitional Learning (SCL). SCL contains three ingredients. First, it simultaneously solves a companion task by learning a discriminator is a true label or a pseudo-label), which is used to mine additional information in the unlaheled data while facilitating main task learning. Second, the output of the discriminator is in fact measure of label reliability, thus can be interpreted as the confidence probabilities of labels (Qin et al. 2021), This converts the original problem into a soft label learning problem. Finally and most importantly, SCL introduces a eiperaaive-yet-compeiiiiv learning scheme to boost the performance of both the main task and the companion task. Specifically, SCL forges cooperation between both tasks by providing extra information to each other (the main task provides prediction loss, companion task provides label confidence measures), leading to improved performance of both tasks. Moreover, we consider the main task model to be self-interested which tries to challenge the discriminator by providing as little information as possible. This design forms a semi-cooperative "game" with a par- tially ad adversarial main task model, which can be shown thcoretically equivalent a loss reweighting mechanism on noisy soft labels. Under this design, the main task and the companion task in SCL. neither fully cooperative as in multi-objective optimization (Deb) 2014) nor fully adversarial to each other as in adversarial learning (Goodfellow et al.] 2014, Matyasko & Chau 2018), but balance the benefit of cooperation and competition. In summary, the contribution of this work include:
critic to predict label observability (whether
We develop novel and very general self-interested coalitional learning framework for SSL that fully utilizes the hidden information in the unlabeled data while addressing the error accumulation issue in the self- self-training paradigm of SSL.
SCL can be easily incorporated into a wide range of semi- supervipe algorithms (e.g. image clas- sification, label propagation and data imputation) with limited changes the original method. We show with comprehensive empirical experiments that SCL. consistently achieves superior per- formance and robustness compared with the original and self-training version of different semi- supervised learning algorithms.
SCL naturally provides the data confidence measures from the discriminator as a byproduct of the learning process, which offers additional interpretability of the semi- cami-supervised learning task. This can be particularly useful for many practical applications, while also providing extra benefits for scenarios with noisy data labels.
2 PRELIMINARIES
2.1 PROBLEM DEFINITIONS AND FORMULATIONS
We formulate the semi- sumniisseeeieed learning problem as learning a model f(-) with input data a X to predict the label y E Y. In SSL, only a partial set of data labels Yr. with ground truth are given in the training set, the rest are unlabeled Yu (Y = Yz UY)). For convenience of later discussion, we denote L as the set of indices of data samples with labels and U as the set of indices for unlabeled data. In many roal-world SSL problems, the size of labeled data |L] is often limited. In extreme cases, Yr, may potentially contain emrrr or noises. We further define mask vector M to denote the
2

---

Under review as a conference paper at ICLR 2022
observability of labels over each data sample. We set M, E M equals to if E L and Mj M equals to 0 j E U. In our proposed SCL framework, we consider following common settings of semi-supervised classification (inductive classification, label propagation) and regression (data imputation) tasks:
Inductive classification: a typical SSL task setting is to construct classifier to predict labels y E Y for any object in the input space I E X. Common examples include semi-supervised image classification tasks (Sajjadi al. 2016; [Shn et al., 2020; Xie et al. 2020), where only a subset of data samples are given the known labels Yr. and the rest data labels Yo are unknown.
Label propagation: another class of SSL tasks under transductive setup is X and observed labels Yt to train classifier to predict on unseen labels Yv. Examples include classifying nodes in graph given only small subset of node labels Yz (Kipf & Welling 2017).
all input samples
Data imputation: one of the special case of SSL problem is that the missing state of input and output data are strongly correlated (Richardson et al.] 2020 Qin et al. 2021), ) such as the data imputation tasks. In data imputation, the inputs X are partially filled and the labels Y are equivalent to reconstructed version of obtained through regression.
2.2 AUXILIARY TASK IN SEMI-SUPERVISHR LEARNING PROBLEMS
We begin our discussion by first noting that, if we feed the unknown labels Yo with the model f predicted labels Yu U (+(()) fXX represents the labels from the k-th round of pseudo- labeling) or other machine-genneatte labels, there actually hides an auxiliary task of discriminating whether the data label is real pseudo-label. Denote all the data labels under pseudo-labeling as ý Y. We can train discriminator d(.) tell the confidence measure p E P of whether + T (a E X, Y) is a valid mapping. This always learnable since the ground truth of P is exactly the observability mask M. With slight abuse of notation, we formulate the main SSL task A and the auxiliary companion task B as follows:
A: A(f(X==
LA = loss (Yi, Vr.) LB = loss P, M)
(1)
B: Bxd(X,))
where Ioss is the original loss function of the main task and loss can be any binary classification loss between P and M, such as the binary cross entropy (BCE) loss. Jointly solving the above two tasks allows exploiting the underlying relationship between input data X and data label Y from another angle, which can potentially provide more information facilitate main task training.
There are some other works that embody the similar idea of jointly learning two tasks. For ex- ample, the multi-task style multi-objectio optimization methods (Ruder) 2017 Deb 2014) exploit the shared information and underlying commonalities between two tasks and solve the problem by minimizing an augmented loss. The generative adversarial learning (Goodfellow et al. 2014) makes generator and a discriminator learn against each other, thus improves the performance of both tasks. However, naively applying these two approaches has some drawbacks. For example, multi- objective optimization cannot handle potential contradictions of the two tasks in certain settings, where jointly minimizing the augmented loss may impede both tasks from achieving the best per- formance (Qin al. 2021). Whereas under the adversarial learning framework, loss is implicitly optimized through the discriminator, which could result in potential loss of information. Moreover, solving the minimax optimization problem under GAN-style model is much harder compared with directly minimize both loss functions explicitly in in a supervised learning fashion.
SELF-INTERESTED COALITIONAL LEARNING
In this section, we introduce a new learning strategy to jointly solving the two tasks in Problem I for semi-supervised learning, which is called self-interested coalitional learning (SCL). As its name suggests, SCL forges cooperation between a self-interested main task and an unselfish companion task for boosted performance. We show with theoretical derivation that under the pseudo-labeling labeling paradigm, this special learning scheme will lead to a soft label learning formulation for the original SSL task. We also provide an intuitive interpretation of the SCL scheme.
3

---

Under review as a conference paper at ICLR 2022
Information from the main task
Pseudo-labeling periodically
f
loss A
YL
X
y
Reweighting
+
loss
lossw
YU
d
loss
M
P
Figure Illustration of self self-interested coalitional learning
3.1 ALTERNATIVE FORMULATION UNDER SCL
We consider an alternative formulation Problem T by modifying the loss of the main task La and the inputs of the companion task to establish cooperation and information sharing:
A: f(X)=
LA = loss (Y, Y) + a lossw (o,:wYYYK (Y.Y,P)
(2)
B d(d(X, ,((, P, LB = loss M).
where s 0) is some form of information provided by the main task. We can simply model the information g()) as the element-wise loss term of the main task, i.e. g(.. 5) = loss (7. i). such as the cross entropy loss for classification tasks and mean square error (MSE) loss for regression tasks. Moreover, lossw( (Y, Y, P) is corrective additional loss measure information P provided by the discriminator d, and a is its weight parameter. Due to the existence of the corrective information from lossw, we now train the main task with all data labels Y rather than only with observed labels Yz. thus allows sufficient utilization of information from the unlabeled data. We illustrate the strategy of SCL in Figure
impacted by the label confidence
The design of SCL has several interesting properties. First, since additional information are provided to both tasks, they can be both learned better. Second, a less well-learned model / will produce large element-wiss error loss A (ū. 9) for a predicted label ÿ. which in turn provides more information in g(-) for the discriminator d to distinguish whether a label real or a pseudo-label. As f is learned better during the cooperative learning process, when 9 longer provides useful information for discriminating the label observability, then we may have reason to believe ff has achieved satisfactory inference performance. Hence minimizing the information in g(-) coincides with improving the learning of f. This suggests that if we alter the learning direction of f using lossw to encourage providing as little information g(-) as possible, then we might obtain a better learned f. This in fact forms cooperative-yet--o -etcoo-ptttiiiiiiiee "game" between both tasks, with the main task been self-interested and partially competitive. Finally, also allows the discriminator detect errors or noises in observed data labels. As the erroneous (x., y) data pairs may possess different mapping patterns against normal data, thus are more likely to be identified as pseudo-labels. Such negative impact can be further corrected using lossw with label confidence measure P.
3.2 DERIVATION OF THE ADDITIONAL LOSS TERM lossw
Until this point, the only mystery about SCL is the exact form of lossw that embodies the self- interested behavior for the main task model f. We resort to calculus of variations to provide theo- retical derivation of lossw-
Under the alternative formulation (2). both the discriminator and its loss La impacted by the information g(-) provided by f. thus they are now functionals of f. We wish to inspect how the variation of model / impacts Ln. Denote 0y, Oa the model parameters of model and d. Here, we fix the the diseriminator and its parameters 0g to exclude its own influence, then d(0,, x, f) can be replaced by d(x, f). Similarly, with fixed, we can define the functional ((f) for LB and further climinate the effect of changes in on d by integrating over x, mathematically
0LB Dd
J( LB
dx
(3)
4

---

Under review as a conference paper at ICLR 2022
where F(x, f²) = OCB/ / is assumed to be continuously differentiable function with respect to z, f and f'. Although the discriminator d aims to minimize its loss LB. as mentioned previously, little information in
want the main task model to challenge the discriminator by providing as possible. This essentially requires changing the behavior of f such that the functional J()) attains its maxima. The extrema (maxima or minima) of functional a function f where the functional derivative of J() is equal to zero. By Taylor expansion,
JJ)) be obtained by finding
J(f) = J(fa)) + Jo + d(ff)
(4)
where 8f is the variation of Since o(ō.) is higher order infinitesimal term 0, + Of. and assuming that f( is small and ignorable, we can obtain the finite difference approximation of the functional J(f) by plugging it into E3.
( AJ(F) (()) - -
8f
(5)
d0s
If the uniform continuity of F and its derivative is satisfied, we can simplify the derivative of J(J) with respect to , as follows, while swapping the order of da with 8f:
AJ = (r, d'dd 55 = (x ofdr
(6)
00,
00g
As f can be any function an arbitrary function space F. Ensuring the functional derivative of ((f) equal to zero (AJ(f) 0) requires (F f0) O, which is
OF f.))
OF 0g 0f
OF
losss = =0
a0
0g
05
f
0g
The last step derivation in above equation is due to our design of letting g(-) be the element-wise loss function of f (g(ū, 1) = a 9)).
As example, suppose we use BCE loss for Lg. and write pt the output value of d(X, y, 9(Y, .)) with index i, the 7 be written
OF OF 00,
Vo, = Σ -
Vo, loss a (Ti, 11) + Ps
Σ Va, loss (ūj 0s)
(8)
0g
Pj
REL
JEU
Note that the result of above equation can be equivalently perceived the gradient of new term L Y, P) (OF) /00 -0L,/08g) with following form:
- LÝ 8) P) Σ SL
loss A (Di.
loss ( -1-
Σ Σ JEU 1-
(9)
Consequently, minimizing - with respect 0, will drive the functional derivative of J(J) to zero (-OL,/00) = 0F/a0, = = AJ(() = 0), which also lead to the maxima of J(f) (update with gradient direction -OF/00, We now uncover the exact form of lossw, which can be set as lossw (Ý, Y, P) CY, Y, P) to realize the self-interested behavior of f. The resulting lossw can be considered an additional reweighting loss that complements the original loss term Loss based on the estimated confidence measures P from the discriminator d.
3.3 INTERPRETATION OF SCL
Adding lossw the original loss term LoSSA, we can obtain the complete loss for the main task as:
L
Σ 1 () loss (3:, Di) T.T.
)
Σ JEU
loss x
(ū,
(10)
This essentially transforms the original main task into a cost-sensitive learning problem (Zarrzzy et al. 2003) by imposing following soft-labeling weights on the errors of each data label as:
+ a/pr. a/ p.).
i L
Soft-laheling weights
(11)
j jEU
5

---

Under review as a conference paper at ICLR 2022
We notice that the above soft-labeling weights induce very different behaviors the loss A 3) of labeled and pseudo-labelee samples. As p; represents the judgment of the discrimina- tor d on whether y; is labeled not. For labeled samples, the loss are boosted by additional weight of a/pps, which means if the discriminator d judges an labeled samples E Yy. to be unlabeled or unreliable (confidence p: - 0), the loss of this sample will get significantly boosted. This forces the main task model f to pay more attention to infer the problematic labeled samples, As unlabeled samples yj Yu are periodically filled with pseudo-labels generated by (initially filled with ran- dom values), SCL does not fully trust these labels and may even choose to enlarge the gap with the pseudo-label when p 1 a (a S 1). Specifically, the more likely an label yj considered reliable by the discriminator (p) - 1), the stronger it forces the learning process to ignore further improvement on inferring V/--
Note that uniform continuity needs to be satisfied in Eq. however, the derivative of BCE loss will result 1/p; and 1/(1 - p,) which can potentially take infinite value, violating the continuity assumption. We thus clip their values as min(1/p;, HI and min(1/(1 p:), Hf) in our practical algorithm, where I is a predetermined threshold (H = 10 in our implementation). Algorithm presents the practical algorithmic framework of SCL.
Algorithm Self-interested coalitional learning framework
Require: Training samples {x, ) = {Xx, U Xv, Yr. UYv). observability mask M, weight parameter & I: Randomly initialized 0,, 0a. Yu. and Yu + Yu
2: while training do
3: Inference the all data labels Ý using the main task model
4: Calculating the element-wise loss of the main task loss A Y) as g (Y, Y)
5: Predict the lahel confidence measures P using the discriminator d and g Y)
6: Update gc+1 by minimizing La in Eq.
Update d with information from main task
7: Update 0f by minimizing LA in Eq. 10
// Update f with information from companion task
8: update the pseudo-label Yu= < Yv
9:
end while
4 EXPERIMENTS
The proposed SCL framework can be easily incorporated in variety of SSL tasks. We choose three representative SSL tasks and apply SCL on their official implementations to evaluate the perfor- mance of SSL, including: image classification (He et al., 2016) for inductive classification, label propagation (Kipf & Welling 2017) for transductive classification and data imputation (Gondara & Wang 2018) for regression.
Image classification: We use RestNet18 for image classification on the fashion-mnis dataset (Xiao al. 2017). with 6,000/10,000 training/testing set partition. Note that the training set only contains 10% of the original fashion-mnist dataset which aims to substantially increase the task difficulty.
Label propagation: We trained two-layer GCN (Kipf & Welling, 2017) integrated with SCL. and evaluate the prediction accuracy on Cora dataset(Sen et al., (2008). The test set consists of 1000 labeled examples extracted from knowledge graph with 2,708 nodes and 5,429 edges. The labeling rate represents the number of labeled nodes used for training divided by the total number of nodes in the training set.
Data imputation: We use deep denoising autoencoder (DAE) (Gondara & Wang 2018) for data imputation task under regression setting. We use the UCI online news popularity dataset (Asuncion & Newman 2007) for evaluation. The dataset is split the training/testing set with the ratio of 0.8/0.2.
For all tasks, we remove certain proportion (referred as missing rate) of data labels in the training set and treat them as unlabeled data in the semi-supervised setting. At initial step of SCL, as the model is not yet trained to provide pseudo-labels, we fill the initial pseudo-labels for unlabeled data with random labels. For the regression task (data imputation), we compute the mean l of the labeled data and fill the missing entries with adding random noise; for the classification tasks, we fill the missing label Yo by randomly selecting data label. We present the average results over 10 random seeds for all the experiments. The implementation details and extra comparative experiments between SCL. and several advanced SSL algorithms are presented in the Appendix.

---

Under review as a conference paper at ICLR 2022
Image classification Accuracy (%)
Label propagation Accuracy (%)
Data imputation
Missing rate (%)
MSE
ResNet18
ResNet18+SCL.
GCN
GCN+SCL
DAE
DAE+SCL
00
83.85 + 0.12
86.23 + 0.13
10
83.57 = 0.12
84.69 0.08
82.18 + 1.59
84.33 0.56
0.8055
0.7899
20
83.06 + 0.16
85.15 + 0.13
82.18 + 1.69
84.00 + 1.81
0,9029
0.8716
30
82.89 + = 0.21
85.38 + 0.16
81,80 - 1.75
83.33 + 0.68
0,9335
0.8867
40
8 0.13
84.98 + 0.21
82.28 + 2.69
83.00 0.95
0.9761
0.9229
50
41.94
84.61 + 0.21
81.93 + 2.56
81.33
0.9726
0.9412
60
81.23 + 0.16
83.87 0.20
80.03 + 6.37
82.33 2.10
0.9787
0.9727
70
79.70 0.13
82.90 + 0.21
77.84 + 7.52
83.67 3.73
1.1593
1.1582
80
78.84 + 0.35
81.31 + 0.42
74.33 +: 7.00
80.33 + 1.00
1.1429
1.1425
90
76.02 1 0.41
77.96 1 0.47
69.66 + 7.66
74.26 + 0.53
Table 1: Comparison of the original models and the SCL-integrated version for different tasks (with
mean + std). The std results in the imputation task are omitted because they are less than le-04.
4.1 PERFORMANCE EVALUATION OF SCL
We compared our method with the original model in three representative SSL tasks to demonstrate the effectiveness and robustness of SCL. under diverse problem setting. Results summarized in Table I We report the classification accuracies and imputation error measured by MSE. SCL consistently achieves superior performance over all tasks, which shows the universality of SCL under diverse SSL problem setting. We also use 10 different levels of missing rates to verify the robustness of SCL, By randomly dropping some labels, we increase the missing rate of the training set from 0% (fully supervised) to 90%. As label propagation and data imputation tasks are not well- defined under fully supervised setting, only the image classification task under 0% missing rates is shown in the table. It is observed in Table T that most of the original models experience substantial performance deterioration with the increase of missing rate. By contrast, SCL remains high level of robustness with limited performance drop in most tasks.
Supprisingly, for the inductive image classification task, we observe that in the case of 0% missing rate (fully supervised), SCL can achieve even higher accuracy compared with the fully supervised base model (accuracy: 86.2 + 0.13% (SCL) vs 83.85 = 0.13% (base model)). Moreover, SCL achieves similar level of accuracy the fully supervised base model even with 60% missing rate. These results suggests that SCL can also be used in supervised learning problems to boost model performance. SCL performs cost-sensitive learning for different data samples maximally utilize the hidden information from both labeled and unlabeled data, which lead to more stable and accurate inference under different missing rates. For the transductive classification task like label propaga- tion, all the input data X and the labeled data Y, are used for inference on the unobserved labels Yu. The amount of information from the labeled data plays a vital role for overall prediction aceu- racy. Hence the missing rate becomes large, we observe substantial performance drop for the original GCN model. SCL, on the other hand, maintains high accuracy in spite of high missing rate by continuously soft iseudo-labeling the data to ensure the reliable pseudo- labels are properly used to facilitate the main task training, thus exhibits high robustness under missing information.
4.2 COMPARISON OF SCL AND SELF-NRAININ VIA PSEUDO-LABELING
To further validate the effectiveness of SCL under semi-supervised settings, we compare the per- formance of SCL with the widely used pseudo-labeling approach in self-training (Yarowsky 1995]
Lee al. 2013 Laine & Aila 2017 Iscen al. 2019b). We compare their performances in three different aspects, I.C., convergence speed, accuracy under different missing rates and the degree of error accumulation under pseudo-labelimg
Figure 2 shows us the convergence of image classifica- tion, label propagation, and data imputation for missing rates of 40%, 20%, and 30%, respectively. In Figure 2 we can see that the learning speed of the main task model with the corrective reweight- ing loss term lossw in SCL is significantly faster than the original model. Meanwhile, since the hard pseudo-labels in self trff-training are used as true label for model learning rather than carefully reweighted as soft labels in SCL, certain degree of oscillation is observed during the learning pro- cess, which impacts the learning speed as well as the final performance of the model. This demon- strates the benefit of using soft-labeling weights to correct unreliable label information in SCI.

---

Under review as a conference paper at ICLR 2022
100
1.08
1.01
DAE
80-
WSB Mame
1.00
Rocureyy (Nc
Accuraay (15)
0.90
80
60
0.96
40
0:94
0.02
78
Ro wwfeeriingg
CCN+ scltuningg
GON 90
0.90
OCN
0.88
100 125 150 Eporh
175
200
0
30
80
90
120
150
0.88
60
120
150
180
Epoch
Epech
(a) Image classification
(b) Label propagation
(c) Data imputation
Figure 2: The convergence of test accuracy under different methods and tasks
80
1.27
Racleil
Seleaiing
Sapriin SC1
BAE
1.19
85
(Nomalad))
1.11
86
80
75
0.96
70-
0.70
30
40
80
80
70
80
30
40
50
60
00
20
40
50
70
80
Masinng sate (%)
Missing rade (56)
Missing rate (56)
(a) Image classification
(b) Label propagation
(c) Data imputation
Figure 3: Performance under different missing rate
Aasunaey of panuditiablle (%)
100
Accuracy of psoudlolaaae labele (%6)
100
80-
95-
85
SO. GCN tofino
20
GEN SCL GCN seftioliing
80
0
2
8 12 Rounds labeling
0
3
5
10 10
Rounds paeudolabeling
(a) Accuracy of pseudo-labels under 20% missing (b) Accuracy of pseudo-labels under 70% missing rate
Figure 4: Error accumulation of different methods in the label propagation task
Figure Segureessooe shows the accuracy of the original, self-trained and SCL versions of models for classifica- tion tasks with different missing rates. We observe that SCL maintains high accuracy under different missing rates and consistently outperforms the original and self-training versions. the label prop- agation task, although self- training increases the training samples by periodically labelig high confidence unlabeled data, the accuracy of the model could decreases due to the introduction of wrong labeling information, which results in subsequent error accumulation and performance drop.
To further investigate the impact of error accumulation in pseudo-labels during model training, we record the accuracy of pseudo-labele after each round of pseudo-labeling in self-training. The test accuracy of these samples in the SCL version of the model are also recorded for comparison. To ensure the stability of self-training, we add pseudo-labels every 10 training epochs. Different from self-training that only label high confidence unlabeled samples, SCL introduces pseudo-labels for all unlabeled data Yv but with soft-labeling weights. Figure shows the error accumulation trends for self-trainne training and SCL in the label propagation task. When the missing rate is low (Figure 4a)), self-training maintains high accuracy at the beginning, but model performance will fluctuate and decrease with the involvement of more potentially problematic pseudo-labels On the other hand, when the missing rate high and available information is scarce (Figure (()),, gradually involv- ing pseudo-labeled data is beneficial for self-training, resulting in slowly increasing test accuracy, although the model still struggles to achieve high accuracy. Despite the occurrence of error accu- mulation in self- self-training, lseudo-labels remains high and stable across different rounds of pseudo-labeling, which clearly suggests the ability of SCL in avoiding accumulation in pseudo-labels.
barely observe such phenomenon in SCL. The test accuracy of the
8

---

Under review as a conference paper at ICLR 2022
Label Falas lotel
Epoch: 30
Label Paex label
Epoch: 70
Labrl
Epoch: 140
Labe
Epoch: 235
50
30
30
Fata abv
30
Faka ihbs
Probabiiyy
20
20
20
20
8.0
0.2
0.4
0.6
0.8
1.0
9.0
02
0.4
0.6
0.8
1.0
0.2
0.4
0.6
0.8
1.0
8.0
0.2
0.4
0.6
0.8
1,0
Output value of the discriminator
Output value of the discriminator
Ouliptt value of the discriminator
Outpul value of the discriminnitor
Figure 5: The probability distribution given by the discriminator during the training process
4.3 INTERPRETABILITY
DISCRIMINATOR OUTPUTS
To evaluate the judgement of the discriminator d, we analyze its output value distribution the labeled and intentionally introduced fake/noisy labeled data in different training stages of image classification task. The four plots in Figure 5 clearly clearly show the transition of improved discrimina- tor, which gradually gains better capability to discriminate true labels (p + 1) and fake/noisy labels (p- + 0). The interpretability of the discriminator is a byproduct of SCL, which can be particularly useful for many real-world SSL tasks or scenarios involving error or noise in labeled data.
5 RELATED WORK
Self-training via pseudo-labeling. Pseudo-labeling (Tiguero et al.
2013 Yarowsky
1995 Lee
jet al. 2013) is one of the most widely used self-training method semi- semi-supervised learning, which uses supervised model that is iteratively trained on both labeled and pseudo-labeled data in previ-
ous iterations of the algorithm. The basic assumption in most models (Yarowsky 1995 Lee et al.
2013
Laine & Aila 2017 Tarvainen & Valpola. 2017b Iscen et al. 2019bb that an unlabeled instance can be labeled for training if the underlying model has high confidence in its prediction. When the model makes inaccurate predictions, it is likely to cause error accumulation that nega- tively impacts training (Cai et al.] 2013 Arazo et al. 2020). To alleviate this problem, many studies NNatarajan al. 2017 Schnabel et al. 2016 Shen al. 2020 Rizve et al. 202T ?) guide the
model learning based on various label confidence measures and avoid the model from being overly
confident on incorrect pseudo-labels. The above works only focus on leveraging both labeled and unlabeled data, but not fully applicable to scenarios with noisy labeled data. SCL considers the contribution of both labeled and unlabeled data through soft labeling, which better addresses afore- mentioned drawbacks.
scenar- ios (Natarajan et al., 2017 (Schnabel et al. 2016). Erroneous noisy labels can severely degrade the generalization performance of deep neural networks. Learning from noisy labels (Song et al.) 2020 is becoming increasinly important task in modern deep learning applications. Early studies CCssaBBianchi et al. 2011) require unbiased gradient estimates of the loss to provide learning guar- antees. Some recent studies (Natarajan et al. 2017]; Schnabel et al., 2016 Shen al.) 2020 Algan & Ulusoy 2021) provide unbiased estimators for evaluating and training under noise data. How- ever, these methods still require that the observed data have relatively well-behaved (noise-free) underlying distribution, thus the noisy data are weighted according to the same distribution. Satis- fying such requirements is difficult in practice and the observed distribution may be biased across the data. In contrast, SCL bypasses the dependence on the original data distribution and learn label confidence measures adaptive using an additional discriminator to facilitate model training.
Learning from noisy labels. The lack of high-quality labels is common in many
CONCLUSION
We introduce a novel self-interested coalitional learning framework, which can easily incorpo- rated into wide range of semi-supervised learning algorithms for enhanced performance. SCL im- proves semi-supervised model learning by constructing a semi-cooperative "game" that establishes cooperation between the main self-interested semi-supervised learning task and companion task of inferring the observability of labels. Meanwhile, it also offers additional confidence measures for the original and inferred data labels, which can be particularly useful for the practical applications with noisy labels. Through comprehensive evaluations, we show that SCL can consistently improve the performance and robustness of semi-supervised classification and regression algorithms.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Görkem Algan and Iikay Ulusoy. Meta soft label generation for noisy labels. In 2020 25th Interna- fional Conference on Pattern Recognition (ICPR), pp. 7142-7148. IEEE, 2021.
Eric Arazo, Diego Ortego, Paul Albert, Noel E O'Connor, and Kevin McGuinness. Pseudo- and confirmation bias in deep semi-supervised learning. In 2020 International Joint Conference on Neural Networks (IJCNN), pp. 1-8. IEEE, 2020.
Arthur Asuncion and David Newman. Uci machine learning repository, 2007.
David Berthelot, Nicholas Carlini, Ian J. Goodfellow, Nicolas Papernot, Avital Oliver, and Colin Raffel. Mixmatch: A holistic approach to semi-supervised learning. In Advances Neural In- formation Processing Systems 32: Anmual Conference on Neural Information Processing Systems 2019, NewrIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp- 5050-5060, 2019.
Xiao Cai, Feiping Nie, Weidong Cai, and Heng Huang. Heterogeneous image features integra- tion via multi-modal seme-supe learning model. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1737-1744, 2013.
Nicolò Cesa-Bianchi Shai Shalev-Shwartz, and Ohad Shamir. Online learning of noisy data. IEEE Trans. Inf. Theory, 57(12):700-7733, 2011.
Olivier Chapelle, Bernhard Schölkopf, and Alexander Zien (eds.). Semi-Supervvsed Learning. The MIT Press, 2006.
Kalyanmoy Deb. Multi-objective optimization. In Search methodologies, pp. 403-449. Springer, 2014.
Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp. 249-256. JMLR Workshop and Conference Proceedings, 2010.
Lovedeep Gondara and Ke Wang. MIDA: multiple imputation using denoising autoencoders. In Advances in Knowledge Discovery and Data Mining 22nd Pacific-Asia Conference, PAKDD 2018, Melbourne, VIC, Australia, June 3-6, 2018, Proceedings, Part III. pp. 260-277 272, 2018.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27:2672-2680, 2014.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In 2015 IEEE International Conference on Computer Vision, ICCV 2015, Santiago, Chile, December 7-13, 2015, pp. 1026-1034, 2015.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and patlern recognition, pp- 770-778, 2016.
Ahmet Iscen, Giorgos Tolias, Yannis Avrithis, and Ondrej Chum. Label propagation for deep semi- supervised learning. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pp. 5070-5079, 2019a.
Ahmet Iscen, Giorgos Tolias, Yannis Avrithis, and Ondrej Chum. Label propagation for deep semi- supervised learning. In Proceedings the IEEF/CV Conference on Computer Vision and Pattern Recognition, pp. 5070-5079, 2019b.
Zhanghan Ke, Daoye Wang, Qiong Yan, Jimmy S. J. Ren, and Rynson W. H. Lau. Dual student: Breaking the limits of the teacher in semi-supervised learning. In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 November 2, 2019, pp. 6727-6735, 2019.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In 3rd Inter- national Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
10

---

Under review as a conference paper at ICLR 2022
Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional net- works. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings, 2017.
Samuli Laine and Timo Aila. Temporal ensembling for semi-supervised learning. 5th Interna- tional Conference Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings, 2017.
Dong-Hyun Lee et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. Workshop on challenges representation learning, ICML, volume 3, pp. 896, 2013.
Lu Liu, Yiting Li, and Robby Tan. Decoupled certainty-driven consistency loss for semi- supervised learning. arXiv preprint NKI::19911006577 2019.
Alexander Matyasko and Lap-Pui Chau. Improved network robustness with adversary critic. In Advances in Neural Information Processing Systems 31: Anmual Conference on Neural Infor- mation Processing Systems 2018, NeurlPS 2018, December 3-8, 2018, Montréal, Canada, pp- 10601-10610, 2018
Yu Meng. Jiaming Shen, Chao Zhang, and Jiawei Han. Weakly-supervice hierarchical text classi- fication. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pp. 6826- 6833, 2019.
Yu Meng, Yunyi Zhang. Jiaxin Huang, Chenyan Xiong, Heng Ji, Chao Zhang, and Jiawei Han. Text classification using label names only: language model self-training approach. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pp. 9006-9011 2020.
Nagarajan Natarajan, Inderjit S. Dhillon, Pradeep Ravikumar, and Ambuj Tewari. Cost-sensitive learning with noisy labels. Mach. Learn. Res., 18:155:1-155:33, 2017.
Avital Oliver, Augustus Odena, Colin Raffel, Ekin Dogus Cubuk, and Ian J. Goodfellow. Realis- tic evaluation of deep semi-supervised learning algorithms. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurrPS 2018, December 3-8, 2018, Montréal, Canada, pp. 9239-32500 2018
Yashoteja Prabhu and Manik Varma. Fastxml: a fast, accurate and stable tree-classifier for extreme multi-label learning. The 20th ACM SIGKDD International Conference on Knowledge Dis- covery and Data Mining, KDD '14, New York, NY, USA August 24 - 27, 2014, pp. 263-272, 2014.
Huiling Qin, Xianyuan Zhan, Yuanxun Li, Xiaodu Yang, and Yu Zheng. Network-wide traffic states imputation using self-interested coalitional learning. In KDD 21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, Singapore, August 14-18, 2021, pp. 1370-1378, 2021.
Trevor W. Richardson, Wencheng Wu, Lei Lin, Beilei Xu, and Edgar A. Bernal. Mcflow: Monte carlo flow models for data imputation. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pp. 14193-14202, 2020.
Mamshad Nayeem Rizve, Kevin Duarte, Yogesh Singh Rawat, and Mubarak Shah. In defense of pseudo-labeling: An uncertainty-aware pseudo-label selection framework for semi-supervised learning. In 9th International Conference Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021, 2021.
Sebastian Ruder. An overview of multi-task learning in deep neural networks. arXiv preprint arXiv: 2017.
Mehdi Sajjadi, Mehran Javanmardi, and Tolga Tasdizen. Regularization with stochastic transfor- mations and perturbations for deep semi-supervised learning. Advances in neural information processing systems, 16:1163-11 2016.
11

---

Under review as a conference paper at ICLR 2022
Tobias Schnabel, Adith Swaminathan, Ashudeep Singh, Navin Chandak, and Thorsten Joachims. Recommendations as treatments: Debiasing learning and evaluation. In Proceedings of the 33nd International Conference Machine Learning. ICML 2016, New York City, NY, USA, June 19- 24, 2016, pp. 1670-1679, 2016.
Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collective classification network data, Al magazine, 29(3):93-9)) 2008.
Zheyan Shen, Peng Cui, Tong Zhang, and Kun Kunag. Stable learning via sample reweighting. In Proceedings of the AAAI Conference Artificial Intelligence, volume 34, pp. 5692-5699, 2020.
Weiwei Shi, Yihong Gong, Chris Ding, Zhiheng Ma, Xiaoyu Tao, and Nanning Zheng. Transductive semi-supervised deep learning using min-max features. In Computer Vision ECCV 2018 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part V, pp. 311- 327, 2018.
Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin Raffel, Ekin Do- gus Cubuk, Alexey Kurakin, and Chun-Liang Li. Fixmatch: Simplifying semi-supervised learn- ing with consistency and confidence. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020.
Hwanjun Song, Minseok Kim, Dongmin Park, and Jae-Gil Lee. Learning from noisy labels with deep neural networks: A survey. CoRR, abs/2007.00 190000000999,,, 2020.
Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight-averaged con- sistency targets improve semi-supervised deep learning results. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Workshop Track Proceedings, 20
Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight-averaged consis- tency targets improve supi-super deep learning results. In Proceedings of the 31st Interna- tional Conference on Neural Information Processing Systems, pp. 1195-1204, 2017b.
Isaac Triguero, Salvador García, and Francisco Herrera. Self-labeled techniques for semi-supervised learning: taxonomy, software and empirical study. Knowledge and Information systems, 42(2): 245-284, 2015.
Vikas Verma, Alex Lamb, Juho Kannala, Yoshua Bengio, and David Lopez-Paz. Interpolation con- sistency training for semi-supervised learning. In Proceedings of the Twenty- International Joint Conference Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019, pp- 3635-3641, 2019.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmark- ing machine learning algorithms. CORR, abs/1708.00774 2017.
Qizhe Xie, Minh-Thamg Luong, Eduard H. Hovy. and Quoc V. Le. Self-training with noisy student improves imagenet classification. In 2020 JEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA. USA, June 13-19, 2020. pp. 10684-10695, 2020.
David Yarowsky. Unsupervised word sense disambiguation rivaling supervised methods. In 33rd annual meeting of the association for computational linguistics, pp. 189-196, 1995.
Bianca Zadrozny, John Langford, and Naoki Abe. Cost-sensitive learning by cost-proportionate example weighting. In Third IEEE international conference on data mining, pp. 435 442, IEEE, 2003.
12

---

Under review as a conference paper at ICLR 2022
A EXPERIMENT SETTINGS
Image classification. The base learner (f) is a ResNet18 (He et al. 2016) where the input channels of the first convolutional layer are replaced with 1. We use the Kaiming Normal- ization (He et al. 2015) to initialize the parameters of f. All input samples of training set and test set are transformed with mean and standard deviation for normalization. The discriminator consists of two feature extractors for X and hatY, respectively, and a fusion layer whose inputs are the extracted X and hatY features and the main task information g- The feature extractor of X is also a ResNet18 with the number of input channels of the first convolutional layer as 1, and outputs features with dimension 1. The feature extractor for hatY is a network with three-layer FC, where the first two layers are each followed by ReLU layer. We fuse the extracted features of X and hatY by the Hadamard product. And concatenate the loss 9 to the fused embeddings of X, hatY and fuse them through a one- layer FC. We feed the output into a sigmoid layer to output the probabilities. We use the Adam optimizer with learning rate of 0.001 to train the base learner ʃ and discriminator d simultaneously for a total of 300 epochs. We set a = 0.6 in all experiments for this task.
Label propagation. We train a two-layer GCN as described in (Kipf & Welling 2017) and evaluate prediction accuracy on test set of 1,000 labeled examples. We initialize weights using the initialization described in (Glorot & Bengio 2010) and accordingly (row- )normalize input feature vectors. For discriminator d, we use five-layer and two-layer FC encode the input X and Y, and we fuse the hidden embedding of X and Y by hadamard product. We connect g to the fusion embedding of X, Y and fuse it through one-layer FC and then fed to sigmoid layer to give the final probability. All FC layers is followed by a ReLU operations except the output layer. We train all models for a maximum of 150 epochs (training iterations) using Adam (Kingma & Ba 2015) with a learning rate of 0.01. We set a in all experiments for this task.
Data imputation. The denoise autoencoder we used in the data imputation task as de- scribed in (Gondara & Wang 2018) consists of a dropout layer and an encoder composed of four-layer FC and a decoder composed of four-layer FC, where the dropout layer has p = 0.1. For discriminator d, we use three two-layer FC to encode the input X, Y and g, and we fuse the hidden embeddings by two one-layer FC and then fed to a sigmoid layer to give the final probability. All FC layers in the discriminator are followed by ReL.U opera- tions, while all FC layers in the DAE are followed by tanh operations, with the exception of the output layer. We train all models for a maximum of 100 epochs using Adam (Kingmal & Bal 2015) with a learning rate of 10 and early stopping with window size of 10, i.c. we stop training if the validation loss does not decrease for 10 consecutive epochs. We set 1 in all experiments for this task.
B ADDITIONAL. EXPERIMENT RESULTS
B. ADDITIONAL ABLATION STUDY
In this section we present the ablation study of the weight parameter a under the image classifi- cation task to analyze the impact of a in SCL. We selected three different levels of missing rates in our experiments - low (30%), medium (50%) and high (80%). F shows the classification accuracy obtained by SCL using different a values under different missing rates. We observe that larger a will make the main task favors more on minimizing the reweighting loss rather than simply minimizing the difference between model predicted labels and the given labels Y. In our exeriment results in Figure 6, it actually suggests that with the taking value from the recommended range [0.1, 0.9], SCL is not very sensitive to a. The maximum difference of accuracy between the worst and best choice of a is only about 2% under different missing rates. The insensitivity to a is another advantage of SCL, as we do not need to perform complex parameter tuning in order to learn a robust and high performance model, which is very implementation friendly.
13

---

Under review as a conference paper at ICLR 2022
Missing mi
Missing rate
82%
50%
30%
82%
50%
53%
86
87
85
83
Accuracy (%)
83
Apcuracy
79-
82-
75-
80
71
79
78
0.1
0.3
0.5
0.7
0.9
0.1
0.3
0.5
0.7
0.9
of
(a) Image classification
(b) Label propagation
Figure 6: Test accuracy under different weight parameter
B.2 COMPARISON WITH ADVANCED SSL METHODS
We conduct supplementary experiments on the semi-supervised image classification task on CIFAR- 10 dataset with 4000 labeled data samples. We compare SCL with several advanced semi-supervised learning methods, including the state-of-the-arr method: uncertainty-ave -aware pseudo-label selection (UPS) Rizve al. 2021). We the CNN-13 architecture that is commonly used in benchmarking SSL algorithms Ollverr et al., 2018) and set a 0.9 for SCL in this task. For fair comparison, we compare against the methods which report results using the same CNN-13 backbone architec- ture: DeepLP (Iscen al. 2019a), TSSDL Shi et al. 2018). MT (Tarvainen & Valpola 201 7ap.
MT + Dee ICT (Verma al. 2019), DualStudent Ke et al. 2019), and MixMatch (Berthelot
al. 2019 We keep the experiment setting the same in the UPS paper (Rizve et al. (222 for
comparison. The technical details of above basclines algorithms please refer to their papers.
We present in Table the mean and standard deviation of each method across three differrntt random splits on training and testing data. SCL achieves comparable results to the state-of-the-as method UPS (92.2% accuracy compared to 93.6% in UPS), despite that SCL is much more fightweeighte and simpler as compared to UPS. Under the same hardware setting (8-core 4.7GHz CPU and Nvidia RTX3070 GPU), it takes 19 hours train UPS to reach 92% accuracy and at least 60 hours fully converge; while SCL takes only 7 hours to reach final convergence, a 60% reduction in training time reach the same level of accuracy. UPS employs the uncertainty measure select high- quality pscudo-labels and also uses negative sample learning to improve the model performance. The uncertainty-awart pseudo-label selection scheme used in UPS very costly, which lead high training cost. By contrast, in SCL, all unlabeled data are pseudo-labelled and used for training, but with soft-labeling weights to unevenly update the loss different samples. does not involve any pseudo-label selection process used in most existing pseudo-labeling methods. Moreover, SCL is a very general SSL framework which broadly applicable to remi-supervised inductive classification, transductive classification and regression tasks, whereas most of the previous baseline algorithms are only applicable to semi-supervised inductive classification tasks. This shows the great potential and computational efficiency of using SCI. solving real-world large-scale applications.
Table 2: Error rate (%) the CIFAR-10 test set. Methods with * are pseudo-labeling based.
Methods
CIFAR-10 (4000 labels)
DeepLP*
12.69 + 0.29
TSSDL*
10.90 + 0.23
MT
11.41 0.25
MT+DeepLp
10.61 0.28
ICT
7.29 1 0.02
DualStudent
8.89 + 0.09
MixMatch
6.48
UPS*
6.39 0.03
SCL
7.85 + 0.03
14


Reviews:
[{"review_id": "ZyAxzCIhYGN", "reviewer": "Reviewer_HSTc", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The idea of introducing the auxiliary task that discriminates whether an instance is labeled is quite interesting. In effect, such a strategy is first introduced in active learning [1]. In the VAAL method [1], a similar discriminator is introduced to identify where an example is labeled or not, which is then used to indicate the uncertainty of an example for active selection. Therefore, the proposed method has a close connection to a recent work in SSL [2] that also employs the uncertainty measure to select high-quality pseudo-labels.\n\nI have the following concerns.\n\n1. The derivation in section 3.2 is confusing. For example, in Eq. (3), the second equality is incorrect and the term $\\frac{dd}{dx}$ should be added. Also, it would be better to change the notation of $d$ (discriminator) to another one, since derivative dx also uses the notation d. Besides, I actually did not understand why $\\mathcal{L}_B$ depends on $f$, since $f$ and $\\mathcal{L}_B$ are from two different branches without sharing network blocks (Figure 1).\n2. The experimental section is not convincing and this is my main concern. The datasets and the baselines are too simple. State-of-the-art SSL methods should be employed to support the claims. In particular, the uncertainty-based SSL method [2] should be compared. As I have discussed above, the proposed method can implicitly be equal to existing techniques in SSL. Is the proposed method complementary to existing methods? Or it is contradictory to some techniques? These questions require an in-depth empirical analysis.\n\nOverall, this work is below the bar of an ICLR paper regarding its poor experiments.\n\n\n[1] Sinha S, Ebrahimi S, Darrell T. Variational adversarial active learning[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 5972-5981.\n\n[2] Rizve M N, Duarte K, Rawat Y S, et al. In defense of pseudo-labeling: An uncertainty-aware pseudo-label selection framework for semi-supervised learning[J]. arXiv preprint arXiv:2101.06329, 2021.", "summary_of_the_paper": "This paper proposes a new semi-supervised learning framework by introducing an auxiliary task that distinguishes whether the pseudo-labels are truly labeled or not. Then, this information is used to add a reweighting loss to the main objective. Experiments on several simple benchmark datasets show that the proposed method outperforms some naive baselines.", "summary_of_the_review": "Interesting idea, poor experiments and confusing derivation.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "KCCRqM1Qe6m", "reviewer": "Reviewer_3FG6", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths\n- The proposed framework seems to be novel.\n- It works well in experiments, on a wide range of tasks (classification, label propagation, and data imputation).\n- It seems to be potentially beneficial for many domains, since it does not have domain-restrictions, while many previous SSL methods rely on certain image domain techniques such as consistency regularization (and data augmentation).\n\nWeaknesses\n- Since the proposed method is only compared with the original pseudo-label method, comparing with other extensions of pseudo-labelling methods that are mentioned in Section 5 will make the contributions more clear.\n- In addition to the papers mentioned in Section 5, there are a few papers that try to address the error accumulation in semi-supervised learning methods that is observed in pseudo-labelling. For example: \"In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learning\" from ICLR 2021 and \"Repetitive Reprediction Deep Decipher for Semi-Supervised Learning\" from AAAI2020.\n\nQuestions\n- I am not sure if I understood the experiments correctly. As the missing rate goes higher, do we have more unlabeled samples (as explained in the last paragraph of page 6), or do we have more noisy-labelled samples (as explained in 1st paragraph of Section 4.1)?\n- Can we show the 3rd task (data imputation) in Figures 2 to 4?\n- One of the benefits of the method seems to be that it can be incorporated into a wide range of SSL algorithms. I think the paper demonstrated that it can be used to enhance pseudo-labelling method, but what kind of other SSL algorithms can SCL incorporate?\n\nMinor questions and comments\n- SSL is a very hot topic and there has recently been many advances. Since the experiments do not compare with many of the recent works, it would be better to emphasize why they were not compared. (For example, Section 1 has a discussion on how recent SSL methods utilize consistency regularization, which relies on heavy data augmentation techniques that is only available in certain domains.)\n- What kind of value for parameter alpha is used in the image classification? (For the other two tasks, I think the appendix explains that alpha is 1).\n- If we are given a labeled dataset L and unlabeled dataset U, it seems we can automatically construct vector M (which is explained in end of page 2). If this is correct, then why do we need M as an input in Algorithm 1 in page 6?\n- What is P introduced in the beginning of Section 2.2? It seems like it is a set from the $p \\in P$ notation but since it compares with M in the loss function, it also looks like a vector.\n- typo \"perforamnce\" in page 6\n- Should $m_i, m_j$ in the beginning of page 3 be $M_i, M_j$?\n- Is $Y$ a label space ($y \\in Y$), or is it the full set of labels in the training dataset ($Y = Y_L \\cup Y_U$)?\n- Ideally it would be better to perform several trials and report mean/standard error in Table 1.\n\n=========== after rebuttal\n\nThank you for answering my questions. The additional experiments are helpful to have a better understanding about the proposed method. It looks like the advatangeous points of the proposed method is now about the low computational costs, according to the new experiments including UPS, rather than better performance. Although this still may be beneficial for the research community, it seems to be slightly less significant and also may affect the storyline. I would like to also recommend to put the new experiments with UPS in the main paper instead of the appendix.", "summary_of_the_paper": "The paper proposes a novel framework for semi-supervised learning, that solves two issues of previous methods: 1) over-reliance on labeled data and 2) error accumulation. It shows that jointly solving the main task together with another task (that discriminates whether the data label is real or not) leads to better performance.", "summary_of_the_review": "The proposed method seems to have some nice benefits, but I feel there are a few weaknesses that should be addressed. I also have a few questions and it would be helpful if the authors can take a look at the previous section (main review).", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "GiGu7IJ53f0", "reviewer": "Reviewer_Qifr", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This is an interesting paper on a topic with important practical applications: semi-supervised learning. The contribution appears to be original, and it is likely to influence future work in the field. The authors are explicitly calling out and addressing the two main weaknesses of traditional self-learning approaches: error accumulation and over-reliance on the labeled data. \n\nThe paper would greatly benefit from an additional section that would provide an intuitive, illustrative example of how and why the proposed approach outperforms self-training. Ideally, it should compare and contrast the convergence of (1) self training, (2) the auxiliary task, and (3) SCL. \n\nThe paper would also benefit by tightening the narrative around the ALPHA parameter, which, in the main paper, is only discussed in the theoretical framework. Appendix A provides no value of ALPHA for the first dataset, and it proposed (without any justification) a value of 1 for the other two domains. Appendix B is extremely brief and not very helpful. The authors make no recommendation on how to tune alpha, and the argument that even the worst alpha (in the 0.1 - 0.9 range) is better than the original model is fairly weak, given the wide variations of the accuracy due to changes the value of alpha.  \n\nOTHER COMMENTS:\n- for Table 1, please add three more rows: 0%, 90%, and 99%.  The former is critical to understanding the upper-bound performance, while the later two will bring SCL into a more realistic semi-supervised regime, where unlabeled data is one or two orders of magnitude more abundant than the labeled data  \n- please add to Figure 6 the horizontal lines with the accuracy of the original model for each of the three missing rates \n- it is still unclear why did you choose to use only 10%of the data for image classification (page 6); is scalability to large datasets a concern?\n- please spell-check the paper - eg, \"perforamnce\" on page 4\n\n- page 2:  please replace \"more sufficient\"\n- page 3: \"jointly solving above two tasks\"   -->  \"jointly solving THE above two tasks\"\n- page 3: \"there are some other works embody\" --> \"there are some other works THAT embody\"\n- page 4: \"are impacted the influence\" -->  \"are impacted BY the influence\"\n- page 7: please replace \"well learn\"", "summary_of_the_paper": "The paper introduces Self-interested Coalitional Learning (SCL), which is a novel approach to semi-supervised learning. SCL combines the traditional self-training approach to semi-supervised learning with an auxiliary task that infers label observability. The empirical results show that, in a variety of scenarios, SCL outperforms both self-training and the original model.   ", "summary_of_the_review": "Overall, this paper uses a novel idea to improve the state of the art for semi-supervised training.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "7JcX58NEYVU", "reviewer": "Reviewer_dWui", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n- Overall the paper is well-structured and easy to follow.\n- The new method achieves its original goals and improves SSL effectiveness by jointly solving the main and the auxiliary tasks.\n- The authors introduce a new SCL strategy to solve the problems, which can be applied to a broader class of learning problems.\n \nCons:\n- Lack of experiments\n    - The proposed method is only compared with the self-learning method (with the same base learner). While this demonstrates how the model is improved with SCL, it is also necessary to compare with state-of-art SSL methods.\n    - It's also valuable to include the supervised method with fully-labeled dataset as a reference in all experiments.\n    - For data imputation, a more common case is that missing state is correlated with input/output instead of simply random missing. It also checks the method robustness against labeled/unlabeled distribution shift.\n    - Compared with original self-learning method, the new method has an extra discriminator model, which are based on the same base learners as for the main tasks. It's meaningful and more fair to compare with supervised models of higher capacity.\n- The paper doesn't cover how SCL can work together with consistency regularization, which is commonly used together with self-learning.\n\n\nBesides, I have a few questions:\n- Although Table 1 doesn't have a row for Missing rate = 0% (full dataset), it seems SCL methods have better accuracy than model trained with full dataset for the first two tasks. Is this because the SCL has double model capacity due to the extra discriminator?\n- Why is the test accuracy of pseudo-labels 100% for SCL method in Figure 4? Are they calculated differently?", "summary_of_the_paper": "This paper proposes a new semi-supervised learning method. Motivated by the error accumulation problem of typical self-training paradigms, the authors propose to explicitly model the confidence of pseudo labels as an auxiliary task. They come up with a self-interested coalitional learning (SCL) strategy to solve both tasks jointly. Under the new framework, the main task is transformed into a cost-sensitive learning problem. Experiments demonstrate that pseudo labels are substantially more accurate with the new method and better performance of the main tasks at different label missing rates.", "summary_of_the_review": "This is an interesting paper from technique perspective. But it definitely needs more empirical studies to demonstrate practical value. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a new method for the important problem of semi-supervised learning. This method relies on an auxiliary task, label observability prediction, to weight the examples according to the confidence in their pseudo-labels, so as to avoid the propagation of errors encountered in self-training. Limited experiments show that the proposed method can compete with other methods in terms of performance or training time. On the positive side, all evaluators agree on the potential value of the proposed approach, which is generic in nature. On the negative side, the experimental evaluation, although strengthened during the discussion, is not yet strong enough to have really convinced of the real merits of the method. In particular, comparisons with the state of the art still need to be improved. In addition, the paper would benefit from some rewriting, in particular of the mathematics (e.g. the d notation for task B should be avoided as suggested by one reviewer, there is a misplaced partial derivative in equation 6). The authors could also simplify their derivation by using the envelope theorem. I therefore recommend rejection, with an encouragement to strengthen the experimental part, and to improve the derivation of the proposed method.
Decision: Reject