Semi-supervised learning by selective training with pseudo labels via confidence estimation
Authors: Masato Ishii
Venue:  (2021)
Keywords: semi-supervised learning
confidence estimation

Abstract: We propose a novel semi-supervised learning (SSL) method that adopts selective training with pseudo labels. In our method, we generate hard pseudo-labels and also estimate their confidence, which represents how likely each pseudo-label is to be correct. Then, we explicitly select which pseudo-labeled data should be used to update the model. Specifically, assuming that loss on incorrectly pseudo-labeled data sensitively increase against data augmentation, we select the data corresponding to relatively small loss after applying data augmentation. The confidence is used not only for screening candidates of pseudo-labeled data to be selected but also for automatically deciding how many pseudo-labeled data should be selected within a mini-batch. Since accurate estimation of the confidence is crucial in our method, we also propose a new data augmentation method, called MixConf, that enables us to obtain confidence-calibrated models even when the number of training data is small. Experimental results with several benchmark datasets validate the advantage of our SSL method as well as MixConf.

Under review as a conference paper at ICLR 2021
SEMI-SUPERVISED LEARNING BY SELECTIVE TRAINING WITH PSEUDO LABELS VIA CONFIDENCE ESTIMATION
Anonymous authors Paper under double-biee blind review
ABSTRACT
We propose a novel semi-supervised learning (SSL) method that adopts selective training with pseudo labels. In our method, we generate hard pseudo-labels and also estimate their confidence, which represents how likely each pseudo-label to be correct. Then, explicitly select which pseudo-labeled data should be used to update the model. Specifically, assuming that loss on incorrectly pscudo-labeles data sensitively increase against data augmentation, ing to relatively small loss after applying data augmentation. The confidence is used not only for screening candidates of pseudo-labeled data to be selected but also for automatically deciding how many pseudo-labeles data should be selected within a mini-batch. Since accurate estimation of the confidence is crucial in our method, we also propose a new data augmentation method, called MixConf, that enables us to obtain comfidence-calibrrated models even when the number of train- ing data is small. Experimental results with several benchmark datasets validate the advantage of our SSL method as well as MixConf.
select the data correspond-
INTRODUCTION
Semi-supervised learning (SSL) is powerful technique to deliver full potential of complex mod- els, such as deep neural networks, by utilizing unlabeled data as well as labeled data to train the model. It is especially useful in some practical situations where obtaining labeled data is costly due to, for example, necessity of expert knowledge. Since deep neural networks are known to be "'dtaa-ungryy" models, SSL for deep neural networks has been intensely studied and has achieved surprisingly good performance in recent works (Van Engelen & Hoos 2020). In this paper, we focus on SSL for a classification task, which is most commonly tackled in the literature.
Many recent SSL methods adopt a common approach in which two processes are iteratively con- ducted: generating pseudo labels of unlabeled data by using currently training model and updating the model by using both labeled and pscudo-labeled data. In the pioneering work (Lee) 2013),
pseudo labels hard ones, which are represented by one-hot vectors, but recent methods (Tar-
vainen & Valpola 2017 Miyato et al., 2018;) Berthelot et al. 2019 2020) Verma et al.) 2019 Wang
al. 2019: Zhang & Qi (020)) often utilize soft pseudo-labes labels, which may contain several non- zero elements within each label vector. One simple reason to adopt soft pseudo-labels is to alleviate confirmation bias caused by training with incorrectly pseudo-labeled data, and this attempt seems to successfully contribute to the excellent performance of those methods. However, since soft pseudo- labels only provide weak supervisions, those methods often show slow convergence in the training [Lokhande et al. 2020). For example, MixMatch (Berthelot et al., 2019), which is one of the state- of-the-art SSL. methods, requires nearly 1, 1,000,000 iterations for training with CIFAR-10 dataset. On the other hand, in this paper, we aim to utilize hard lacudo-labels to design easy-to-try SSL method in terms of computational efficiency. Obviously, the largest problem to be tackled this approach is how to alleviate the negative impact caused by training with the incorrect pseudo-labels.
In this work, we propose a novel SSL method that adopts selective training with pseudo labels. To avoid to train model with incorrect aseudo-labels. we explicitly select which pseudo-labeles data should be used to update the model. Specifically, assuming that loss on incorrectly pseudo-labelee data sensitively increase against data augmentation, we select the data corresponding to relatively

---

Under review as a conference paper at ICLR 2021
Pseudo-tabeling
Selective training (step 1)
Unlabeled data
Pscudo labels and confidence
g
go
Confidence thresholding
Automatic setting of n! and n
Pscudo-labe od data
Mixed data
nt small-loss samples
x
9o
13
Sort by loss
e
Loss minimization
MixConf
x
yd
g+
g4
Sort by loss
ge
n small-loss samplees
Labeled data
Mixod data
Selective training (step 2)
Figure I: An overview of the proposed semi-supervised learning method.
small loss after applying data augmentation. To effectively conduct this selective training, we esti- mate confidence of pseudo labels and utilize it not only for screening candidates of pseudo-labeled data to be selected but also for automatically deciding how many pseudo-labeled data should be selected within mini-batch. For accurate estimation of the confidence, augmentation method, called MixConf, that enables us to obtain confidence-calibrale models even when the number of training data is small. Experimental results with several benchmark datasets validate the advantage of our SSL method as well as MixConf.
also propose
data
2 PROPOSED METHOD
Figure shows an overview of our method. Given mini-batch from labeled data and that from unlabeled data, we first generate pseudo labels of the unlabeled data based on predictions of the current model. Let E IR" E (1, 2..CC,, and f Rm - IRC denote input data, labels, and the classifier to be trained, respectively. Given the input unlabeled data U the pseudo label j is generated by simply taking arg max of the classifier's output f(x²). Then, we conduct selective training using both the labeled data and the pseudo-labeled data. In this training, to alleviate negative effect caused by training with incorrect pecudo-labels, explicitly select which data should be used to update the model. Below, we describe details of this selective training.
2.1 SELECTIVE TRAINING WITH PSEUDO LABELS BASED ON CONFIDENCE
As described previously. the pseudo labels are generated based on the predictions of the current model, and we assume that the confidence of those predictions can be also computed in addition to the pseudo labels. When we use a popular architecture of deep neural networks, can be obtained by simply taking max of the classifier's output (Hendrycks & Gimpel 2016) as:
max jE(1,......C) [xx (xx
C₁
where &; is the confidence of the classifier's prediction on the i-th unlabeled data x; and [() the j-th element of (x)) When the model is sufficiently confidence-calibratee the confidence e is expected to match the accuracy of the corresponding prediction f(x!) (Guo et al. 2017). which
means also matches the probability that the pseudo label o!! is correct.
To avoid training with incorrect pseudo-labels,
explicitly select the data to be used train the model based on the confidence. This data selection comprises two steps: thresholding the confidence and selecting relatively small loss caleulated with augmented pseudo-labeled data. The first step is
2

---

Under review as a conference paper at ICLR 2021
quite simple; we pick up the pseudo-labeles data that have higher confidence than certain threshold Othr and discard the remaining. In the second step, MixConf, which will be introduced later but is actually variant of Mixup (Zhang al., 2018), is applied to both the labeled and unlabeled data to augment them. As conducted in (Berthelot et al.) 2019), we shuffle all data and mix them with the original labeled and pseudo-labeled data, which results in { (())))))
P Bi, and {(0), +))) 1
respectively, where p E IRC is vector-style representation of the label that is adopted to represent a mixed label. Then, we calculate the standard cross entropy loss for each mixed data. Finally, we select the mixed data that result in relatively small loss among the all augmented data, and only the corresponding small-loss is minimized to train the model.
Why does the small-loss selection work? Our important assumption is that the loss calculated with incorrect labels tends to sensitively increase when the data is augmented. This assumption would be supported by effectiveness of the well-known technique, called test-time augmentation (Simonyan & Zisserman 2013). in which incorrect predictions are suppressed by taking an average of the model's outputs over several augmentations. Since we conduct the confidence thresholding, the loss corresponding to the pscudo-labeled data is guaranteed to be smaller than a certain loss level defined by the threshold Cthr- However, when we apply data augmentation, that MixConf, to the pseudo-labeles data, the loss related to incorrectly pseudo-labeled data becomes relatively large, if the above assumption valid. It means that selecting relatively small loss after applying MixConf leads to excluding incorrect pseudo-labels, and we can safely train the model by using only the selected data.
Han et al. (2118) and Lokhande et al. (2020) have presented similar idea, called small-loss trick (Han et al. 2018 or speed as a supervisor (Lokhande et al. 2020). to avoid training with incorrect labels. However, their assumption is different from ours; it is that loss of incorrectly labeled data decreases much slower than that of correctly labeled data during training. Due to this assumption, their methods require joint training of two distinct models (Han al.] 2018) nested loop for train- ing (Lokhande et al., 2020) to confirm which data show relatively slow convergence during training, which leads to substantially large computational cost. On the other hand, since our method focuses on change of loss values against data augmentation, not that during training, we can efficiently conduct the selective training by just utilizing data augmentation each iteration.
Since the confidence of the pseudo label represents the probability that the pseudo label is correct, we can estimate how many data we should select based on the confidence by calculating an expected number of the mixed data generated from two correctly labeled data. Specifically, when the averaged confidence within the unlabeled data is equal Caves the number of the data be selected can be determined as follows:
Br. Cu
BL:
(2)
Br. + Bu
BL Cave
nu min
BL.
CaveBt)
(3)
B1 Bu
where nt. is for the data generated by mixing the labeled data and shuffled data, and ny is for those generated by mixing the unlabeled data and shuffled data. Here, to avoid too much contribution from the pseudo-labeled data, we restrict ny to be smaller than B₁, Within this restriction, we can observe that, if we aim to perfectly balance nı, and nu. BU should be set to Bt./Cave- However, Cave cannot be estimated before training and can fluctuate during training. Therefore, for stable training, set Bu = BL/ Cehe instead and fix it during training.
Finally, the total loss L to be minimized in our method is formulated as the following equation:
Σ 1( L = BL J Pl()) + Au
BL Σ ( P
(4)
where / is the standard cross entropy loss, 5 and t represent the sample index sorted by loss in an ascending order within each mini-batch, and Au is syperr-arammeter that balances the two terms.
To improve the accuracy of pseudo labels as well as their confidence, we can average the model's outputs over K augmentations to estimate pseudo labels as conducted in (Berthelot et al., 2019). In that case, we conduct MixConf for all augmented pseudo-labeled data, which results in k mim- batches each of which contains By mixed data. Therefore, we need modify the second term in the
3

---

Under review as a conference paper at ICLR 2021
MixConf
Randomly sample An from the distribution shown in Eq. (13)
Caleulate the corresponding du
Kernel function: X'
according to Eq. (14)
x aa
Xg
P
Loss
1 -
Ap
X1
x (1 - 2a)
dog
Randomly sample d from the given beta distribution
distribution: Reta(a, a)
Sg dg
Mixup
Figure 2: An overview of MixConf and its comparison with Mixup. MixConf basically follows the scheme of Mixup, but it carcfully sets the interpolation ratios (Aa, A&) so that training with the interpolated data leads better calibrated models. Unlike Mixup, A is not necessarily equal to Xa-
right-hand side of Eq. to take the average of losses over all the mini-batches. In our experiments, we used K = 4 except for an ablation study.
2.2 MIXCONF TO OBTAIN BETTER CALIBRATED MODELS
In the previous section, we assumed that the model is sufficiently confidence-calibree neural networks are often over-confident problem gets bigger case training with a small-scale dataset as we will show in our experiments. Consequently, should occur in SSL setting, because there only small amount of labeled training data in the early stage the training. If the confidence over-estimated, incorrect pseudo- labels more likely to be selected calculate the loss due to loose confidence-threshoee over-estimated (nL, nu). which should significantly degrade the performance of the trained model. To tackle this problem, well-calibrated models even when the number of training data is small. MixConf basically follows the scheme of Mixup, which is known contribute model's calibration (Thulasidasan et al. 2019), but is more carefully designed for confidence calibration.
but deep their predictions in general (Guo et al. 2017). This
and
propose novel data augmentation method, called MixConf, to obtain
Figure shows an overview of MixConf. In similar way with Mixup, MixConf randomly picks up two samples {(Zo,Pu) (I1,Pı)} from the given training dataset and generates a new training sample (Ø. () by linearly interpolating these samples as the following equations:
f Aa + A,)=1.
(5)
ñ = Appo + As)p1,
(6)
where Aa E [0, 1] and A E [0, 1] denote interpolation ratios for data and labels, respectively. Note that Àr is not restricted to be equal to As in MixConf, while X As in Mixup. Since Mixup is not originally designed to obtain confidence-calibrate models, we have to tackle the following two questions obtain better calibrated models by such Mixup-like data augmentation method:
How should we set the ratio for the data interpolation?
(In case of Mixup, X is randomly sampled from the beta distribution)
How should we determine the labels of the interpolated data? (In case of Mixup. A
be equal to Xa))
We first tackle the second question to clarify what kind of property the generated samples should have. Then, derive how set X and A that the generated samples have this property.

---

Under review as a conference paper at ICLR 2021
2.2.1 How TO DETERMINE THE LABELS OF THE INTERPOLATED DATA
Let us consider the second question shown previously. When the model predicts ğ for the input x, the expected accuracy of this prediction should equal the corresponding class posterior probability p()]z). It means that, if the model is perfectly calibrated, its providing confidence should match the class posterior probability. On the other hand, from the perspective of maximizing the prediction accuracy, the error rate obtained by the ideally trained model should match the Bayes error rate, which is achieved when the model successfully predicts the class that corresponds to the maximum class-posterior probability. Considering both perspectives, we argue that max, [=) of the ideally trained model should represent the class posterior probability to have the above- mentioned proper- ties. Therefore, to jointly achieve high predictive accuracy and confidence calibration, we adopt the class posterior probability the supervision of the confidence the generated data. Specifically, we aim to generate a new sample (ă, p) so that it satisfies p = p(y(ū).
Although it is quite difficult to accurately estimate the class posterior probability for any input data in general, we need to estimate it only for the linearly interpolated data in our method. Here, we es- timate it via simple kemel-deensty estimation based on the original sample pair P(ro, (x1, First, we rewrite p(y|z) by using Bayes's 's theorem as the following equation:
ply = ق|از= k,p(oly=
p(t)
where Jj denotes ply j) and j E {1, .... estimate both p(ž) and p(āly) by using a kernel function as
..., C}. Then, intead of directly estimating p(ylž), we
p(t)= = E pplily = vi). p( = { iE(0,1)
k(ū mo) if yo, k( x1)
yf y1. otherwise.
(8)
0
Since we only use the two samples, (zo. Po) and (I1.P1), for this estimation, Ty is set to 1/2 if j E {yo.31 y1 } and 0 otherwise. By substituting Eqs. ( into Eq. 77. we obtain the following equation:
k(ž-zo) Eie(o.1) k(B
if y = yo:
p(yla)=
k(2 ((2-21)
if y = V/. otherwise.
Eie k(0-0,)
To make represent this class posterior probability, we need to set the interpolation ratio A in Eq. 6 as the following equation:
k(ū - xo)
A = ply = yul Eie(o.)) k(i - x;:)
(10)
Once we generate the interpolated data, we can determine the labels of the interpolated data by using Eq. (0).. Obviously, As is not necessarily equal to Aa which is different from Mixup.
2.2.2 HOW TO SET THE RATIO FOR THE DATA INTERPOLATION
Since we have already formulated p(o), we have to carefully set the ratio for the data interpolation so that the resulting interpolated samples follow this distribution. Specifically, we cannot simply use the beta distribution to sample X₂, which is used in Mixup, and need to specify an appropriate distribution p(>a) to guarantee the interpolated data follow p(8) shown in Eq. (8)
By using Eq. 5 and Eq. 8
we can formulate p( x(aa) as the following equation:
dø
p(Aa)= = p(
I20 Ø1
Σ
2 k(z 2z)) 22).
(11)
dAa
1E
Since the kernel function lo is defined in the x-space, is hard to directly sample X from the distribution shown in the right-hand side of Eq. (TT). To re-formulate this distribution make easy to sample, we define another kernel function in the -space as
k' (((xa)= | x1 k<Aa 21)).
(12)
5

---

Under review as a conference paper at ICLR 2021
By using this new kernel function, rewrite p(Aa)) in Eq- follows:
11 and also Ab in Eq. 10 as
p()a) E
1 2x
( -1
(13)
(E{0,1}
Wd -1
A6 Eie k' -1-
(14)
This formulation enables us to easily sample Xa and to determine its corresponding A5. Note that we need to truncate p(>a) when sampling X to guarantee that the sampled Aa is in the range of [0, 1]. The kernel function k' should be set manually before training. It corresponds to hyper- pypeme-arameter setting of the beta distributon in case of Mixup.
3 EXPERIMENTS
3.1
CONFIDENCE ESTIMATION
We first conducted experiments to confirm how much MixConf contributes to improve the confi- dence calibration especially when the number of training data is small. We used object recognition datasets (CIFAR-10 10 and CIFAR- 100 (Krizhevsky) 2009)) and a fashion-product recognition dataset (Fashion MNIST (iiao al. 2017)) as in the previous study (Thulasidasan et al.) (019). The num- ber of training data is 50,000 for the CIFAR-10 / CIFAR-10M and 60,000 for the Fashion MNIST. To make small-scale training dataset, we randomly chose subset of the original training dataset while keeping class priors unchanged from the original. Using this dataset, we trained ResNet- 18 (He al., 2016) by the Adam optimizer (Kingma & Ba 2015). After training, we measured the confidence calibration of the trained models at test data by the Expected Calibration Error (ECE)
(Naeini et al. 2015):
Bml |acc( (mm) conf( N
RCE Σ
(15)
where Bn is set of samples whose confidence fall into m-th bin, acc acc(B,n) B,n) represents averaged accuracy over the samples in Bn, calculated by 1 Eie 1 = y:). and conf repre- sents an averaged confidence in Bm, calculated by |Bml | 1 LieB é We split the original test data into two datasets, namely 500 samples for validation and the others for testing. Note that the valida- tion was conducted by evaluating the prediction accuracy of the trained model on the validation data, not by the ECE. For each setting, we conducted the experiments five times with random generation of the training dataset and initialization of the model, and its averaged performance will be reported.
For our MixConf, we tried the Gaussian kernel and the triangular kernel, and we call the former one MixConf-G and the latter one MixConf-T. The width of the kernll, which is pppee--rrreeter of MixConf, is tuned via the validation. For comparison, we also trained the models without Mixup as baseline method and those with Mixup. Mixup has its hyper-paramette a to determine p(>a) Beta(a, a.,, which is also tuned in the experiment.
Figure 3 shows the ECE of the trained models. The horizontal axis represents the proportion of the original training data used for training, and the vertical axis represents the ECE of the model trained with the corresponding training dataset. In all methods, the ECE increases when the number of training data gets small, which indicates that the over-confidence problem of DNNs gets bigger in case of the small-scale training dataset. As reported (Thulasidasan et al., 2019). Mixup sub- stantially reduces the ECE compared with the bascline method, but its ECE still increases to some extent when the training dataset becomes small-scale. MixConf-G succeeds in suppressing such increase and achieves lower ECE in all cases. The performance of MixConf-T is not so good as that of MixConf-G especially in case of CIFAR-IO/100. Since the actual width of the kernel function in the data space gets small according to the increase of the training data due to smaller |zo x1l (see Eq. 12) ), the difference between MixConf and Mixup becomes small, which results in similar performance of these methods when the number of the training data is large. Through the almost all settings, MixConf-G with a = 0.4 performs best. Therefore, we used in our SSL method in the experiments shown in the next section.

---

Under review as a conference paper at ICLR 2021
0.4
Baseiie Nisup
HikCont-G
0.6
Micup
MixConf-G
Bassing
0.125
Buselin
Micup
MicContiG
MicConi-1
MixConf-1
0.200
MicConf-1
0.075
0.650
0.1
0.325
0.0
10P
10-7 The propertion of orighnal training dets used for traninng
10-3
The propertion original training used for nraiingg
The proportion ongg training used For traring
(a) CIFAR-10
(b) CIFAR-100
(c) Fashion MNIST
Figure 3: The Expected Calibration Error (ECE) of the trained models.
3.2 SEMI-SUPERVISED LEARNING
To validate the advantage of our SSL method, we conducted experiments with popular benchmark datasets: CIFAR-10 10 and SVHN dataset (Netzer et al. 2011]. We randomly selected 1,000 or 4,000 samples from the original training data and used them as labeled training data while using the re- maining data as unlabeled ones. Following the standard setup (Oliver et al., 2018), we used the WideResNet-28 model. We trained this model by using our method with the Adam optimizer and evaluated models using exponential moving average of their parameters as in (Berthelot et al.) 2019). The number of iterations for training is set to 400,000. The hyper-paramettrs (Au, (Au,Cthr) in method are set to (2, 0.8) for CIFAR- 10 and (3, 0.6) for SVHN dataset, unless otherwise noted. We report the averaged error rate as well as the standard deviation over five runs with random se- lection of the labeled data and random initialization of the model. We compared the performance of our method with those of several recent SSL methods, specifically, Virtual Adversarial Training (VAT) (Miyato et al.] 2018). , Interpolation Consistency Training (ICT) (Verma al. 2011), Mix- Match [Berthelot et al. 2019), Pscudo-labeling (Lee) 2013). and Hermite-SaaS (Lokhande et al.) 2020). Note that the former three methods utilize soft pseudo-labels, while the latter two use hard ones. We did not include ReMixMatch (Berthelot et al. (2220) in this comparison, because it adopts an optimization of data- augmentation policy that heavily utilizes domain knowledge about the clas- sification task, which is not used in the other methods including ours.
Table shows the test error rates achieved by the SSL methods for each setting. For CIFAR-10 10, our method has achieved the lowest error rates in both settings. Moreover, our method has shown relatively fast convergence; for example, in case of 1,000 labels, the number of iterations to reach 7.75% in our method was around 160,000, while that in MixMatch is about 1,000,000 as reported in (Berthelot et al.) 2019). Lokhande et al. (020)) have reported much faster convergence, but our method outperforms their method in terms of the test error rates by significant margin. For SVHN dataset, our method has shown competitive performance compared with that of the other state-of- the-art methods.
We also conducted an ablation study and investigated performance sensitivity to the hyper- parameters using CIFAR-10 with 1,000 labels. The results are shown in Table [ When we set K = 1 or use Mixup instead of MixConf, the error rate substantially increases, which indicates that it is important to accurately estimate the confidence of the pseduo labels in our method. On the other hand, the role of the small-loss selection is relatively small, but it shows distinct improvement. Decreasing the value of Au leads to degraded performance, because it tends to induce overfitting to small-scale labeled data. However, if we set too large value to Au. the training often diverges due to overly relying on pseudo labels. Therefore, we have to carefully set Au; as large as possible within a range in which the model is stably trained. The confidence threshold Cthe is also important; the test error rate varies according to the value oshr as shown in Fig. 4 Considering to accept pseudo- labeled data as much as possible, smaller Cthr is preferred, but too small Othr substantially degrade the performance due to increasing a risk of selecting incorrectly pseudo-labeles data to calculate the loss. We empirically found that, when we gradually decrease the value of othe, the training loss of the trained model drastically decreases at a little smaller Ghr than the optimal value as shown by a red line in Fig. 4 This behavior should provide a hint for appropriately setting Cthr-
7

---

Under review as a conference paper at ICLR 2021
Table 1: Experimental results CIFAR-10 and SVHN dataset.
CIFAR-I0
SVHN
Method
1,000 labels
4,000 labels
1,000 labels
4.000 labels
VAT
18.68+0.40
11.05=0031
5.98+0.21
4.20=10115
Miyato et al.
2018
Soft pseudo-
ICT
7.66=0.17
3.53+0..07
Verma et al.
2019
labels
MixMatch
7.75++032
6.24
3.27++0.1
2.89+00066
Berthelot et al. 2019)
Pscudo-labeling
31.53
17.41=0.37
10.19+0.41
5.71=0.07
(Lee 2013)
Hard pseudo- labels
Hermite-Saat
20.77
10.65
3.57:00.04
Lokhande al. 2020)
Our method
7.13:10.08
5.81:10.12
3.63:L0.12
3.23:10.06
1.5
14
Table 2: Ablation study. We used CIFAR-10
Test ettor rate
Training loos
dataset with 1,000 labels. Ablation
0.20
1.3
Error rate
Training loss
Proposed method
0.15
1.2
7.13
(K 4, Au = 2, Cahr = 0.80)
With K =1
8.33
0.10
With Au =1
7.89
With Mixup instead of MixConf
7.73
0.70
0.75
0.80
0.85
0.90
Without the small-loss selection
7.39
Figure 4: Test error rates and training loss of the trained model in our method.
4 CONCLUSION
In this paper, we presented novel SSL method that adopts selective training with pseudo labels. In our method, we explicitly select the pseudo-labeled data that correspond to relatively small loss after the data augmentation applied, and only the selected data are used to train the model, which leads to effectively preventing the model from training with incorrectly pseudo-labeke the confidence of the pseudo labels when generating them and use determine the number of the samples to be selected as well as to discard inaccurate pseudo labels by thresholding. We also proposed MixConf, which is the data augmentation method that enables us train more confidence- calibrated models even in case of small-scale training data. Experimental results have shown that our SSL method performs on par or better than the state-of-the-art methods thanks to the selective training and MixConf.
data. We estimate
REFERENCES
David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: A holistic approach semi-supervised learning. Advances in Neural Information Processing Systems, pp. 5050-5060, 2019.
David Berthelot, Nicholas Carlini, Ekin D Cubuk, Alex Kurakin, Kihyuk Sohn, Han Zhang, and Colin Raffel. Remixmatch: Semi-supervised learning with distribution matching and augmenta- tion anchoring. In International Conference Learning Representations, 2020.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In Proceedings of the International Conference on Machine Learning, pp. 1321-1330, 2017.
8

---

Under review as a conference paper at ICLR 2021
Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. In Advances in neural information processing systems, pp.
2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In Proceedings of the International Conference on Learning Rep- resentations, 2016.
Diederik P. Kingma and Jimmy Ba. Adam: method for stochastic optimization. In Proceedings of the International Conference on Learning Representations, 2015.
Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical Report, Depart- ment of Computer Science, University of Toronto, 2009.
Mong-Hyun Lee. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, volume 3, pp. 2, 2013.
Vishnu Suresh Lokhande, Songwong Tasneeyapant, Abhay Venkatesh, Sathya N Ravi, and Vikas Singh. Generating accurate lseudo-labels in semi-supervised learning and avoiding overconfident predictions via hermite polynomial activations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1435-11443, 2020.
Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, and Shin Ishii. Virtual adversarial training: regularization method for supervised and semi-supervised learning. IEEE transactions on pattern analysis and machine intelligence, 41(8):1979 1993, 2018.
Mahdi Pakdaman Naeini, Gregory Cooper, and Milos Hauskrecht. Obtaining well calibrated proba- bilities using bayesian binning. In Proceedings of the AAAI Conference on Artificial Intelligence, 2015.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop Deep Learning and Unsupervised Feature Learning, 2011.
Avital Oliver, Augustus Odena, Colin A Raffel, Ekin Dogus Cubuk, and Ian Goodfellow. Realis- tic evaluation of deep semi-supervised learning algorithms. In Advances in neural information processing systems, pp. 3235-3246, 2018
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In Proceedings of the International Conference Learning Representations, 2015.
Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight-averaged consis- tency targets improve semi- semi-supervised deep learning results. In Advances in neural information processing systems, pp. 1195-1204, 2017.
Sunil Thulasidasan, Gopinath Chennupati, Jeff A Bilmes, Tanmoy Bhattacharya, and Sarah Micha- lak. On mixup training: Improved calibration and predictive uncertainty for deep neural networks. In Advances Neural Information Processing Systems, pp. 13888 13899, 2019.
Jesper Van Engelen and Holger H Hoos. A survey on semi-supervised learning. Machine Learn- ing, 109(2):333-440, 2020.
Vikas Verma, Alex Lamb, Juho Kannala, Yoshua Bengio, and David Lopez-Paz. Interpolation con- sistency training for semi-supervised learning. Proceedings of the 28th International Joint Conference on Artificial Intelligence, pp. 3635-3641. AAAI Press, 2019.
Qin Wang, Wen Li, and Luc Van Gool. Semi Semi-supervised learning by augmented distribution align- ment. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1466-1475, 2019.

---

Under review as a conference paper at ICLR 2021
Han Xiao, Kashif Rasul, and Roland Vollgraf. Mushion-mnisse a novel image dataset for benchmark- ing machine learning algorithms, 2017.
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In Proceedings of the International Conference on Learning Representations, 2018.
Liheng Zhang and Guo-Jun Qi. Wep: Worst-case perturbations for semi-supervised deep learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 3912-3921, 2020.
10


Reviews:
[{"review_id": "y4aD850Lba_", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a new method to perform semi-supervised learning by self-annotating data at each iteration, simply choosing to do so by selecting a subset of instances to which are assigned a pseudo-label, those chosen instances being the one for which a loss is minimal once data have been mixed through a MixConf procedure. \n\nWhile familiar with semi-supervised learning and label propagation, I must say I had some difficulties following part of the paper, partly because I was not familiar with the initial mixing methods and their justifications. This means that I would like some more justifications of why this is a sound method to select items to label. The results of the experiments are also a bit disappointing, as out of two data sets, the provided method seems to give better results on only one data set, albeit one of its advantage is to require less iterations (by a magnitude of 10) than other soft-label based methods to reach similar performances (albeit the practical advantage of such a speed up heavily depends on the time taken by an iteration). \n\nHere are some clarifications that could be helpful to better understand the paper:\n\n* In equations (2) and (3), I did not find the information (in the experimental setting) of how were fixed B_U and B_L, that in turns allow to select n_L and n_U data. A relation between B_U and B_L is given, but that still requires to settle one of the number for n_L (and n_U) to be well defined. \n\n* In equation (4), how is it that the loss is between an instance and a probability distribution over classes? Or is \\tilde{x}^L_{s[i]} the class? \n\n* Section 2: what theoretical argument have we that the proposed approach will provide well-calibrated outputs? The method actually produces a bunch of bi-modal distributions whose posterior density and input density are estimated by non-parametric kernels over two data. Also, it may well be that the average \\lambda_a x_0 + (1-\\lambda_a) x_1 is a non-observable point (in this case, image), hence that it has actually probability zero? The experiments seem to show that it works reasonably well on the three selected data sets (as did MixMatch), but do we have formal arguments indicating that this approach would provide well-calibrated outputs in general? \n\n* Figure 3 and calibration curves: beyond the baseline, all three methods seem to be on par regarding calibration problems, so it is unclear whether MixConf actually brings any water to the mill of semi-supervised learning, data augmentation or calibration? Also, how can we explain that more data actually degrade the calibration results, as one would expect calibration to improve with data? Would it be possible to show calibration curves? \n\n* Semi-supervised learning: basically, the proposed approach consists in two different bricks, the calibration/mixing approach, and the selection/labelling approach. In the comparison with MixMatch, both aspects are changed. What happens when we apply the same selection/labelling approach to MixMatch? It would probably give us the same speed up, but with a different mixing techniques. I guess what I would like to see is an ablation study between the present paper and the MixMatch approach, as right now the real added value of MixConf is unclear to me (both in Figure 3 and in Table 1).  \n\n* There are also some typos remaining (in a small amount), so a final read would probably be helpful. e.g.: \"not that during training\".", "extra_scores": {}, "extra_text": {}}, {"review_id": "v4VOBKMGsO", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper uses selective training with pseudo labels. Specifically, the method selects the pseudo-labeled data associated  with small loss after performing  the data augmentation, and then uses  the selected data for training the model. Here, the model computes the confidence of the pseudo labels and then puts a threshold to determine  the number of the selected samples and ignore inaccurate pseudo labels. Moreover, MixConf, a variation of mixup, for data augmentation is proposed  to train a more confidence calibrated model. Finally, experimental results on the standard datasets  show the effectiveness of the proposed model compared to SOA SSL methods.\n\n\nThe paper is well-written, and experiments are sufficient to some extent. However, the technical novelty of the work is marginal. This is because,  treating unlabeled data differently for SSL has been recently investigated in the following work [1], and this work can be readily modified to perform in a selective training fashion (e.g., select sample or not during the training). Moreover, the work is very similar to the SSL models that use MixUp for data augmentation such as Mixmatch,  ReMixMatch and  [2]\n\n[1]- https://arxiv.org/abs/2007.01293\n\n[2]- Arazo E, Ortego D, Albert P, O’Connor NE, McGuinness K. Pseudo-labeling and confirmation bias in deep semi-supervised learning. In2020 International Joint Conference on Neural Networks (IJCNN) 2020 Jul 19 (pp. 1-8). IEEE.\n\n-Finally, extracting confident examples with small losses, or  selecting noisy/out of distribution samples based on loss or output of the softmax is explored previously in the literature for  sample selection and training deep models with noisy labels, examples of this are provided as follows:\n\n[3] Han B, Yao Q, Yu X, Niu G, Xu M, Hu W, Tsang I, Sugiyama M. Co-teaching: Robust training of deep neural networks with extremely noisy labels. InAdvances in neural information processing systems 2018 (pp. 8527-8537).\n\n[4] Xingrui Yu, Bo Han, Jiangchao Yao, Gang Niu, Ivor W Tsang, and Masashi Sugiyama. How\ndoes disagreement benefit co-teaching? In ICML, 2019.\n\n[5] Xiaobo Wang, Shuo Wang, Jun Wang, Hailin Shi, and Tao Mei. Co-mining: Deep face recognition with noisy labels. In ICCV, pages 9358–9367, 2019.\n\n[6] Chen Y, Zhu X, Li W, Gong S. Semi-Supervised Learning under Class Distribution Mismatch. InAAAI 2020 (pp. 3569-3576).\n\n\n-I think Tuning c_{thr} needs more  attention and it highly depends on many factors such as the datasets, number of initially labeled data for training, and etc. Considering these, it may affect the performance of the model differently. So I encourage the authors to elaborate more on tuning this parameter. \n\n-The authors may add the performance of the model in a fully supervised fashion, it  is good to know the lower and upper bound of the network's  performance. Since  the results of the proposed model are marginally better than the Mixmatch, it is a bit difficult to compare these methods, specially when  the  training set-up is different.  In such a case, the gap between the lower bond and the reported results might be better for a fair comparison.\n\n-Could you please provide some discussion on the similarity and differences between your methods and the works discussed above?\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "rT-ICD3xv-F", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a pseudo-labeling-based method for semi-supervised leanring. The proposed method consists of (1) pseudo-labeled data selection based on prediction confidence for efficient training and (2) a data augmentation method named mixconf, which is a modification of mixup. \n\nOverall, the paper is well written, but i can't find much novelty of the proposed method. For the part (1), data selection for semi-supervised learning is a quite well-known idea. For the part (2),  It's just a very simple and obvious modification of the existing data augmentation method (which may lead to marginal performance improvement. it's effectiveness should be additionally demonstrated using other benchmark datasets).\n\nAlso I think the authors should perform more experimental investigations to support their idea. For example,\n1) The main assumption the authors made is \"the loss calculated with incorrect labels tends to sensitivey increase when the data is augmented\". I think they can give examples about that on the experimented datasets to show that their assumption is more likely. \n2) it seems the proposed method involves so many hyperparameters and it seems its outcome is sensitive to the settings of the hyperparameters, while how they tuned the hyperparameters is unclear, hidden at the back. Table 2 and Figure 4 (results on CIFAR-10 dataset) show that the performance of the proposed method is considerably affected by the hyperparameters. One thing they must do is to carry out ablation study on the SVHN dataset and if possible, other new datasets.\n3) For semi-supervised learning, there is no reason to prefer pseudo-labeling approach to others (e.g. semi-supervised VAE, ladder nets). Because the other methods have been very effective in semi-supervised learning, they can be additionally compared with the proposed one.\n4) One benefit of the proposed method the authors mentionedis its computational efficiency (by selecting highly confident pseudo-labeled data). This could be wrong if it involves more training iterations due to the other components of the proposed method (such as training configurations and data augmentation stuffs, etc.). To clearly demonstrate if the proposed method is actually computationally efficient, the authors should provide the comparison of the training time each method required. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "Cua4yVjsqUw", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, a novel semi-supervised learning (SSL) method that adopts selective training with pseudo labels is proposed based on confidence estimation. The confidence is used not only for screening candidates of pseudo-labeled data to be selected but also for automatically deciding how many pseudo-labeled data should be selected within a mini-batch.\n[1] In the data selection, it selects the data corresponding to relatively small loss after applying data augmentation. What is the advantage of the proposed method compared with the conditional GAN. \n[2] How to balance the number of samples that are generated for training in each class.\n[3] In the proposed method, it needs to select the  samples with small loss. It may be time-consuming, since the loss of each samples should be estimated. Meanwhile, the Gaussian kernel is adopted in the proposed method. As we know, the Gaussian kernel is very cost on both memory and time with the number of samples increasing. Hence, how about the overhead of the proposed method? The overhead of the proposed method analysis may be very important to further inspect the proposed method.\n[4] Whether the proposed method is still effective if the number of the labeled training data is very limited. ", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes an approach that generates pseudo-labels along with confidence to help semi-supervised learning. Then, selected pseudo-labels are used to update the model. Moreover, the authors include a variation of mixup for data augmentation to train a more calibrated model. Experimental results justify the validity of the proposed approach.

Several reviewers believe that the paper is somewhat well-written. The main concern is on the novelty of the work. In particular, many works have discussed selected treatment of unlabeled data, data augmentation for semi-supervised learning, and label confidence estimation. Those works deserve more discussions/comparisons. The paper can also be improved with deeper experimental studies that better justify the main assumption and merits of the proposed approach. 

Decision: Reject